Lesson 6 of 9 · Linear algebra foundation

Transpose turns columns into comparisons.

Your win: use the dot product as alignment, and prove that the transpose of a rotation undoes the rotation.

15 minutesNeeds: matrix multiplicationOutcome: \(R_\theta^{\mathsf T}=R_{-\theta}\)

Mission link: the RoPE identity appears when the rotated query is transposed inside the query-key dot product.

A dot product measures signed alignment

\[ \mathbf q^{\mathsf T}\mathbf k = \begin{bmatrix}q_1&q_2\end{bmatrix} \begin{bmatrix}k_1\\k_2\end{bmatrix} =q_1k_1+q_2k_2. \]

Geometry	Dot product	Reading
Same direction	positive and large	strong alignment
Perpendicular	zero	no alignment
Opposite direction	negative and large in magnitude	strong opposition

Transpose reverses product order

\[(A\mathbf x)^{\mathsf T}=\mathbf x^{\mathsf T}A^{\mathsf T}\]

This is not cosmetic. It is the exact move that brings \(R_m^{\mathsf T}\) next to \(R_n\) in the final RoPE derivation.

Transpose a rotation

\[ R_\theta^{\mathsf T} = \begin{bmatrix} \cos\theta&\sin\theta\\ -\sin\theta&\cos\theta \end{bmatrix} = \begin{bmatrix} \cos(-\theta)&-\sin(-\theta)\\ \sin(-\theta)&\cos(-\theta) \end{bmatrix} =R_{-\theta}. \]

The equality uses the even/odd identities \(\cos(-\theta)=\cos\theta\) and \(\sin(-\theta)=-\sin\theta\). Transpose changes a counterclockwise turn into the corresponding clockwise turn.

Numbers substituted: transpose a quarter turn

\[ R_{\pi/2} =\begin{bmatrix}0&-1\\1&0\end{bmatrix}, \qquad R_{\pi/2}^{\mathsf T} =\begin{bmatrix}0&1\\-1&0\end{bmatrix} =R_{-\pi/2}. \]

Apply both in sequence to \(\mathbf v=(3,2)\):

\[ \begin{bmatrix}3\\2\end{bmatrix} \xrightarrow{R_{\pi/2}} \begin{bmatrix}-2\\3\end{bmatrix} \xrightarrow{R_{\pi/2}^{\mathsf T}} \begin{bmatrix}3\\2\end{bmatrix}. \]

Numbers substituted: one dot product

\[ \begin{bmatrix}2&-1\end{bmatrix} \begin{bmatrix}3\\4\end{bmatrix} =2(3)+(-1)(4)=2. \]

The comparison is positive but weaker than the product of two parallel vectors with the same lengths.

Why length is preserved

\[ \lVert R_\theta\mathbf v\rVert^2 =(R_\theta\mathbf v)^{\mathsf T}(R_\theta\mathbf v) =\mathbf v^{\mathsf T}\underbrace{R_\theta^{\mathsf T}R_\theta}_{I}\mathbf v =\lVert\mathbf v\rVert^2. \]

A matrix with \(R^{\mathsf T}R=I\) is orthogonal. Its transpose is its inverse: \(R^{\mathsf T}=R^{-1}\).

Retrieval check

What motion does \(R_\theta^{\mathsf T}\) perform?

Practice before moving on

Compute \((2,-1)^{\mathsf T}(3,4)\).
Transpose \(\begin{bmatrix}1&2\\3&4\end{bmatrix}\).
Multiply \(R_{\pi/2}^{\mathsf T}R_{\pi/2}\) numerically.
Apply \(R_{\pi/2}^{\mathsf T}\) to \((-5,2)\).
Show that \((R_{\pi/2}\mathbf v)^{\mathsf T}(R_{\pi/2}\mathbf v)=\mathbf v^{\mathsf T}\mathbf v\) for \(\mathbf v=(1,3)\).
Explain why \((A\mathbf x)^{\mathsf T}\) becomes \(\mathbf x^{\mathsf T}A^{\mathsf T}\), not \(A^{\mathsf T}\mathbf x^{\mathsf T}\).

Check solutions

\(2\).
\(\begin{bmatrix}1&3\\2&4\end{bmatrix}\).
\(\begin{bmatrix}1&0\\0&1\end{bmatrix}=I\).
\((2,5)\), a clockwise quarter turn.
\((1,3)\mapsto(-3,1)\); both squared lengths are \(10\).
Transpose reverses factor order so the shapes remain valid: column-to-row first, then matrix.

Keep this chain: transpose = reverse rotation = negative angle = inverse. You will use all four descriptions in Lesson 9.

Primary source: MIT OCW: Transpose, Inverse, and Determinant. Focus on inner products and inverse/transpose notation.

I can explain why \(R_\theta^{\mathsf T}=R_{-\theta}=R_\theta^{-1}\).

Ask the teaching agent to inspect your proof of \(R_\theta^{\mathsf T}R_\theta=I\) if any cancellation is unclear.