TaylorGPT
Designed a 1.4 million parameter character-level transformer decoder model to generate Taylor Swift lyrics from scratch using PyTorch, including custom implementations for masking, multi-head attention, and transformer blocks along with model profiling, logging, and multi-GPU training.
TaylorGPT is a character-level transformer decoder model with 1.4 million parameters, built from scratch to generate Taylor Swift lyrics. The project features custom implementations for masking, multi-head attention, and transformer blocks, as well as model profiling, logging, and multi-GPU training.