A Short Course for Modern Large Language ModelsGlossary

Lesson 11 of 23 · LLM preliminaries

Every position predicts what comes next.

Your win: construct autoregressive inputs and targets, read vocabulary logits, and calculate one next-token cross-entropy loss.

16 minutesNeeds: token IDs and softmaxOutcome: build training pairs
Module bridge: the decoder learns by repeatedly answering one question: given the visible prefix, which vocabulary token should come next?

One sequence supplies many training examples

For tokens \([t_0,t_1,t_2,t_3,t_4,t_5]\), remove the last token to form the input and remove the first token to form the targets:

\[x=[t_0,t_1,t_2,t_3,t_4],\qquad y=[t_1,t_2,t_3,t_4,t_5].\]
InputThecatsatonthe
Targetcatsatonthemat

At input position 2, the visible prefix ends in “sat” and the target is “on.” A causal mask prevents that position from reading the target token before predicting it.

Logits score every vocabulary item

For batch size \(B\), sequence length \(T\), and vocabulary size \(V\), the language-model head returns

\[L\in\mathbb R^{B\times T\times V}.\]

The slice \(L[b,t,:]\) contains \(V\) unnormalized scores for one next-token decision. Softmax converts that score vector to probabilities.

Numbers substituted: one prediction over three tokens

Let the logits be \((2,1,0)\), with target index \(1\). Then

\[ p=\operatorname{softmax}(2,1,0) =\frac{(e^2,e^1,e^0)}{e^2+e^1+e^0} \approx(0.665,0.245,0.090). \] \[ \mathcal L=-\log p_{\text{target}}=-\log(0.245)\approx1.407. \]

Increasing the target logit raises its probability and lowers the loss.

Do not confuse positions: the model returns one vocabulary distribution at every input position during training. Generation normally reads only the distribution at the final visible position.

Code checkpoint · shift, flatten, compare

Show next-token labels and cross-entropy
import torch
import torch.nn.functional as F

tokens = torch.tensor([[4, 8, 15, 16, 23, 42]])
inputs = tokens[:, :-1]   # [B=1, T=5] = [4, 8, 15, 16, 23]
targets = tokens[:, 1:]   # [B=1, T=5] = [8, 15, 16, 23, 42]

logits = model(inputs)    # [B=1, T=5, V]
loss = F.cross_entropy(
    logits.reshape(-1, logits.size(-1)),
    targets.reshape(-1),
)

Trace it: Why do inputs and targets have the same \(B\times T\) shape even though they contain different tokens?

Retrieval check

For tokens \([5,9,2,7]\), which targets train next-token prediction?

Practice before moving on

  1. Form inputs and targets from \([10,20,30,40,50]\).
  2. If \(B=4,T=128,V=32{,}000\), state the logits shape.
  3. What does \(L[2,7,:]\) represent?
  4. Compute \(\operatorname{softmax}(0,0)\) and the loss when either class is correct.
  5. Why must a causal model hide future target tokens during training?
  6. Explain why one length-100 sequence provides 99 aligned next-token targets.
Check solutions
  1. Inputs \([10,20,30,40]\); targets \([20,30,40,50]\).
  2. \(4\times128\times32{,}000\).
  3. All vocabulary scores for the token following position 7 in batch item 2.
  4. Probabilities \((0.5,0.5)\); loss \(-\log0.5\approx0.693\).
  5. Otherwise it could copy the answer instead of learning from the prefix.
  6. Every token after the first becomes the target for the prefix ending immediately before it.

Source: Bengio et al., A Neural Probabilistic Language Model; PyTorch CrossEntropyLoss.