Algebraic view
\(\displaystyle \mathbf v=\begin{bmatrix}3\\2\end{bmatrix}\)
Component 0 is 3. Component 1 is 2.
Lesson 1 of 9 · Linear algebra foundation
A vector is a list. A pair is a place.
Your first win: look at two adjacent components of a long vector and see a point in its own 2D plane.
Mission link: RoPE rotates dimensions in pairs. Before “rotation” can mean anything, each pair must become a geometric object.
One object, two readings
The ordered pair \((3,2)\) is both a list of two numbers and an arrow from the origin to the point three units right and two units up. Order matters: \((3,2)\) and \((2,3)\) are different arrows.
Geometric view
\(\displaystyle (0,0)\longrightarrow(3,2)\)
Horizontal change is 3. Vertical change is 2.
A long vector contains many little planes
Suppose a query has eight components:
\[\mathbf q=\begin{bmatrix}q_0&q_1&q_2&q_3&q_4&q_5&q_6&q_7\end{bmatrix}\]
RoPE groups adjacent components. Each group can be treated independently as coordinates in a 2D plane:
| Pair index i | Components | 2D point |
|---|---|---|
| 0 | \(q_0,q_1\) | \((q_0,q_1)\) |
| 1 | \(q_2,q_3\) | \((q_2,q_3)\) |
| 2 | \(q_4,q_5\) | \((q_4,q_5)\) |
| 3 | \(q_6,q_7\) | \((q_6,q_7)\) |
The essential compression: an even-dimensional vector is a stack of 2D planes. RoPE turns each plane without mixing it with the others.
Length is content strength
The magnitude of \(\mathbf v=(x,y)\) is its distance from the origin:
\[\lVert\mathbf v\rVert=\sqrt{x^2+y^2}\]
For \((3,4)\), the magnitude is 5. A rotation will change the direction but preserve this magnitude. That preservation matters: position should alter how a query or key is oriented for comparison, not arbitrarily amplify it.
Retrieval check
How many adjacent pairs are in an 8-dimensional query?
Entry diagnostic
Where does the vector \(\begin{bmatrix}-2\\5\end{bmatrix}\) end?
Practice before moving on
- Split a 12-dimensional vector into adjacent pairs. How many planes result?
- For \(\mathbf v=\begin{bmatrix}-3\\4\end{bmatrix}\), name the horizontal coordinate, vertical coordinate, and quadrant.
- Calculate \(\lVert(5,12)\rVert\).
- Write the third adjacent pair of \(\mathbf q=[q_0,q_1,\ldots,q_9]\).
- Explain in one sentence why swapping \((q_2,q_3)\) to \((q_3,q_2)\) changes the geometry.
Check solutions
- Six planes.
- Horizontal \(-3\), vertical \(4\), quadrant II.
- \(\sqrt{5^2+12^2}=13\).
- \((q_4,q_5)\), when counting the first pair as pair 1.
- The components are ordered axes, so swapping them reflects the point across the line \(y=x\).
Primary source: MIT OCW: Linear Transformations and Their Matrices. Read the session overview now; return to the full lecture after Lesson 5.
Ask the teaching agent about any step that felt like a rule rather than a picture. Report the entry diagnostic result before changing the course pace.