TaylorGPT
A 1.4M-parameter transformer language model built from scratch in PyTorch, with hand-written multi-head attention, trained on Taylor Swift lyrics.
TaylorGPT is a character-level transformer decoder model with 1.4 million parameters, built from scratch to generate Taylor Swift lyrics. The project features custom implementations for masking, multi-head attention, and transformer blocks, as well as model profiling, logging, and multi-GPU training.