Printable reference

Glossary

The vocabulary used consistently across all 23 lessons.

Term	Working meaning in this course	RoPE connection
Scalar	One number, such as an angle or scale.	Position \(m\) and frequency \(\theta_i\) are scalars.
Vector	An ordered list of components; geometrically, an arrow.	Each query and key is a vector.
Dimension pair	Two adjacent vector components treated as coordinates in one plane.	\((q_{2i},q_{2i+1})\) receives one rotation frequency.
Magnitude	The length \(\lVert\mathbf v\rVert=\sqrt{x^2+y^2}\).	A rotation preserves it.
Radian	An angle measured as arc length divided by radius.	\(m\theta_i\) is a radian angle.
Phase	Angular location around a cycle.	RoPE makes position act as phase.
Frequency	Angle advanced per token in this course.	Each pair uses its own \(\theta_i\).
Complex number	\(x+iy\), another notation for the point \((x,y)\).	A pair can be rotated by multiplication with \(e^{im\theta}\).
Conjugate	\(x-iy\), reflection across the real axis.	For unit phases, conjugation reverses rotation.
Matrix	A rectangular array representing a linear transformation.	\(R_m\) rotates every dimension pair.
Transpose	Swap rows and columns: \(A^{\mathsf T}\).	For a rotation, \(R_m^{\mathsf T}=R_{-m}=R_m^{-1}\).
Dot product	\(\mathbf q^{\mathsf T}\mathbf k=\sum_i q_i k_i\), a signed alignment score.	Attention compares each query and key with it.
Orthogonal matrix	A matrix whose transpose is its inverse.	Rotation matrices are orthogonal and length-preserving.
Relative position	The displacement \(n-m\) between two token indices.	\(R_m^{\mathsf T}R_n=R_{n-m}\) puts it inside the attention score.
Hidden state	The current feature vector representing one token at one layer.	Its normalized form is projected into Q, K, and V.
Tensor axes	The named directions of an array: batch \(B\), token \(T\), feature \(d_{\text{model}}\), head \(h\), and head feature \(d_h\).	RoPE pairs coordinates only inside the \(d_h\) axis.
Projection	A learned matrix multiplication that maps features into another feature space.	\(W_Q,W_K,W_V\) produce the vectors used by RoPE and attention.
LayerNorm	Centers and scales one token using statistics across its feature coordinates.	Often appears before attention in a pre-norm block.
RMSNorm	Scales one token by its root mean square without subtracting its mean.	A common modern normalization used with RoPE models.
Pre-norm	Normalize before a sublayer: \(X+F(\operatorname{Norm}(X))\).	Describes the placement used in this course's block.
Causal mask	Adds \(-\infty\) to score entries where key position \(j>i\).	Prevents a query from attending to future positions.
Softmax	Turns one score row into nonnegative weights that sum to one.	Converts RoPE-modified query-key scores into attention weights.
Attention head	One learned feature subspace that attends over the full token sequence.	Each head applies RoPE within its own paired feature coordinates.
Residual path	An identity route that adds a sublayer's learned update to its input.	Keeps the block input and update in the same model-width shape.
SiLU	The smooth activation \(z\sigma(z)\).	Forms the nonlinear gate in SwiGLU.
SwiGLU	A feed-forward layer with gate and up projections multiplied elementwise, then projected down.	Transforms each token after its attention residual.

Definitions follow the course's cited sources in RESOURCES.md. Also keep the RoPE sheet and block sheet nearby.