Three-module course · 23 short lessons

A short course for modern large language models.

Build the mathematical and implementation intuition to follow a modern decoder-only language model from token embeddings to next-token logits: derive the essential position-encoding math, keep a tensor-shape ledger through normalization, causal multi-head attention, residual paths, and SwiGLU, and read, run, and explain the course's PyTorch TransformerModel implementation.

About 5.5 hours totalNo calculus requiredPaper and pencil recommended

Module 1 · Mathematical foundations for attention and position

Study one lesson at a time. Write every exercise answer before opening its solution; the effort of retrieval is part of the course.

#	Lesson and tangible win	Time
01	Vectors and pairs Read adjacent RoPE dimensions as a point in a 2D plane.	12 min
02	Radians and the unit circle Turn an angle into the coordinates \((\cos\theta,\sin\theta)\).	12 min
03	Complex numbers as vectors Move between \((x,y)\) and \(x+iy\) without mystique.	10 min
04	Complex multiplication rotates See why multiplying by \(e^{i\theta}\) changes phase but not length.	14 min
05	Matrices as motions Read and apply the 2x2 rotation matrix.	14 min
06	Dot products and transposes Prove that rotation preserves length and transpose undoes it.	15 min
07	Attention is comparison Locate the query-key dot product that RoPE changes.	10 min
08	Position as frequency Explain paired dimensions and fast-to-slow rotations.	14 min
09	Derive relative position Derive \(R_m^{\mathsf T}R_n=R_{n-m}\) and teach it back.	18 min

Module 2 · LLM preliminaries: text, training, and generation

Connect the foundation math to the complete language-model system before opening a Transformer block. Follow raw text through tokenization and embeddings, construct next-token targets, read one training update, generate with sampling controls, and finish with one end-to-end model map.

#	Lesson and tangible win	Time
10	Tokenization and embeddings Trace text through subword IDs into a learned \(B\times T\times D\) tensor.	16 min
11	The next-token objective Shift one sequence into inputs and targets, then calculate cross-entropy.	16 min
12	Training updates model weights Narrate forward, loss, backward, optimizer step, batch, and epoch.	18 min
13	Autoregressive generation and sampling Compare greedy decoding, temperature, top-k, and top-p.	18 min
14	The complete decoder-only LLM map Connect embeddings, blocks, logits, training stages, and generation.	20 min

Module 3 · One Transformer decoder block

Begin when you can trace a prompt from text to next-token logits and explain \(R_m^{\mathsf T}R_n=R_{n-m}\). Keep a shape ledger beside every calculation. Each lesson now ends with a Practice studio: recall the theory, read the code contract, run an engineering challenge, and compare with a standard answer. Use the inline Transformer block walkthrough in Lesson 23 to review how the stages connect, then use its engineering bridge to extend the lesson checkpoints toward MiniMind's GQA, KV cache, Flash Attention fallback, MoE, and generation wrapper.

#	Lesson and tangible win	Time
15	Hidden states and tensor shapes Read \(B\times T\times d_{\text{model}}\) and follow one token row.	12 min
16	Normalize each token Calculate LayerNorm and distinguish RMSNorm and Pre-LN.	14 min
17	Project Q, K, V and split heads Track the projection and reshape dimensions numerically.	15 min
18	Mask the future before softmax Construct the causal score row and replace forbidden scores by \(-\infty\).	14 min
19	Softmax and weighted values Turn scores into probabilities and blend value vectors.	15 min
20	Multi-head attention splits features Reject the “one token per head” misconception.	12 min
21	Residual paths carry state Read \(X+F(\operatorname{Norm}(X))\) as a learned update.	12 min
22	SwiGLU feed-forward sublayer Calculate gate, value branch, product, and projection back.	15 min
23	Walk one complete block Narrate every operation, then stage GQA, KV cache, attention runtime paths, MoE, and the model wrapper.	25 min

Course toolkit

Study, translate, and review.

Start with Lesson 1. Complete its short diagnostic, then use the same loop for every lesson: predict before revealing, work the exercises on paper, correct your answer, and explain the idea aloud.

Keep one translation page with three columns: vector form, complex form, and matrix form. They often describe the same motion in different notation.