JustinAngel's free workshop teaches you to build GPT-2 from scratch
No math or ML prerequisites needed to build your own transformer.
JustinAngel, a Reddit user, has published a comprehensive YouTube workshop titled "Build Your Own LLM" that guides viewers through recreating a GPT-2-style transformer without any prior math or machine learning prerequisites. The workshop was originally delivered in-person in San Francisco and is now freely available online. It breaks down every component of modern LLM development using slides, hands-on Excel examples, and Python code in PyTorch.
The curriculum spans from perceptrons and activation functions (ReLU, GELU, SwiGLU) to advanced topics like multi-head attention variants (MHA, GQA, MQA, MLA), tokenization methods (BPE, SentencePiece), backpropagation, and pre-training data pipelines. It also covers instruction tuning (Alpaca format) and reinforcement learning (SimPO). The only requirement is basic comfort with learning through code and spreadsheets. Users can watch the video or download slides and exercises for self-paced learning.
- No math/ML prerequisites required; only code and Excel familiarity needed.
- Covers over 30 topics including attention mechanisms, tokenizers, training loops, and instruction tuning.
- Available as YouTube video or self-paced slides with exercises and code examples.
Why It Matters
Democratizes LLM understanding for engineers, enabling hands-on learning of transformer internals without a PhD.