Learning project · PyTorch
GPT from Transformer Fundamentals
A decoder-only language model built from tokenization through masked attention, training, and text generation.
The story
Following Andrej Karpathy’s “Let’s Build GPT” series, I built a GPT-style decoder-only model in PyTorch. The work covered tokenization, embeddings and positional encoding, masked multi-head self-attention, stacked transformer blocks, a training loop, and generation. It strengthened my first-principles understanding of the models behind production agents.
Project summary · Conceptual illustration