Backpropagation — how one wrong prediction changes billions of weights

Suppose the final hidden state is:

\[ h=[2,1] \]

and the vocabulary contains only Paris, London, and Rome.

Let:

\[ w_{Paris}=[0.5,0.5] \]

\[ w_{London}=[1.0,-0.5] \]

\[ w_{Rome}=[0,0.5] \]

Logits are dot products:

\[ z_i=h\cdot w_i \]

So:

\[ z_{Paris}=1.5,\quad z_{London}=1.5,\quad z_{Rome}=0.5 \]

Softmax gives roughly:

\[ P(Paris)=0.42,\quad P(London)=0.42,\quad P(Rome)=0.16 \]

The correct token is Paris.

Cross-entropy gradient¶

For softmax + cross entropy:

\[ \frac{\partial L}{\partial z_i}=p_i-y_i \]

with one-hot target:

\[ y=[1,0,0] \]

Thus:

\[ \frac{\partial L}{\partial z} = [-0.58,0.42,0.16] \]

Interpretation: - Paris has negative gradient, so increasing its logit lowers loss. - London and Rome have positive gradients, so increasing theirs raises loss.

Change a weight¶

Since:

\[ z_{Paris}=h\cdot w_{Paris} \]

we have:

\[ \frac{\partial L}{\partial w_{Paris}} = -0.58[2,1]=[-1.16,-0.58] \]

With learning rate \(0.1\):

\[ w_{new} = [0.5,0.5]-0.1[-1.16,-0.58] = [0.616,0.558] \]

The new Paris logit becomes:

\[ [2,1]\cdot[0.616,0.558]=1.79 \]

up from 1.5.

But hidden representations change too¶

The gradient with respect to hidden state is:

\[ \frac{\partial L}{\partial h} = \sum_i\frac{\partial L}{\partial z_i}w_i \]

That signal propagates into previous Transformer layers.

If an earlier operation was:

\[ h=xW \]

then:

\[ \frac{\partial L}{\partial W} = x^T\frac{\partial L}{\partial h} \]

and:

\[ \frac{\partial L}{\partial x} = \frac{\partial L}{\partial h}W^T \]

So the model can change both its output mapping and the internal representation machinery.

Connect to Q/K/V¶

Since:

\[ Q=XW_Q,\quad K=XW_K,\quad V=XW_V \]

gradients can eventually reach:

\[ \frac{\partial L}{\partial W_Q}, \quad \frac{\partial L}{\partial W_K}, \quad \frac{\partial L}{\partial W_V} \]

Thus training can change what queries look for, what keys advertise, and what values carry.

No human labels an attention head “pronoun resolver”; useful behavior emerges because it reduces next-token loss.

Backprop through attention is what teaches retrieval behavior¶

Recall:

\[ Attention(Q,K,V) = softmax(QK^T/\sqrt d)V \]

If the model should have paid more attention to an earlier token, the loss gradient can alter the query representation, the relevant key representation, the value content, or all three.

Through the chain rule, gradients reach the learned projection matrices:

\[ W_Q,\;W_K,\;W_V \]

and the layers that produced their inputs.

That means a high-level behavior such as “when a pronoun appears, retrieve the likely antecedent” can emerge without an explicit pronoun-resolution label. The only requirement is that better retrieval improves prediction loss often enough.

Why gradients are tiny but meaningful¶

A giant model may receive an extremely small change to any one weight in one update. But the same useful statistical regularity appears repeatedly across billions or trillions of tokens.

Training is therefore cumulative:

one example → tiny pressure
millions of related examples → stable feature

This also explains why individual weight values are rarely interpretable. A behavior is generally the result of coordinated changes distributed across many parameters.

Gradient clipping¶

Large training systems sometimes encounter unusually large gradient norms. Gradient clipping limits the magnitude of an update to reduce catastrophic jumps.

Again, optimization is not merely “follow the gradient”; it is engineering a stable trajectory through an enormous parameter space.

Backprop is repeated application of the chain rule¶

If:

\[ a=f(x),\qquad b=g(a),\qquad L=h(b) \]

then:

\[ \frac{\partial L}{\partial x} = \frac{\partial L}{\partial b} \frac{\partial b}{\partial a} \frac{\partial a}{\partial x}. \]

A Transformer is just an enormous composition of functions, so autograd repeatedly applies this rule backward through the graph.

The forward pass asks:

Given these weights, what output does the model produce?

Backward asks:

If the final loss moved a tiny amount, which internal quantities and parameters were responsible?

This is why saved activations matter: many local derivatives depend on the exact values seen in the forward pass.

One sequence trains every layer simultaneously¶

A single next-token mistake does not only adjust the vocabulary projection. Its gradient can flow through every layer all the way to embeddings.

So the training objective is global: attention patterns, MLP features, normalization scales, and output mappings are all jointly shaped by the same prediction errors.

Why loss gradients can be dense even when the error looks local¶

The target is one token, but the hidden state producing that token depends on many earlier tokens and many layers. A single prediction error can therefore create nonzero gradients across a huge fraction of the network.

That does not mean every parameter receives an equally useful update. Most individual effects are tiny. But across a large batch, related gradient signals accumulate into a coherent direction.

Backpropagation is therefore best viewed as a mechanism for distributing credit and blame numerically across a computation graph, not as identifying one guilty neuron.