The residual stream — the Transformer's shared scratchpad

Yesterday we looked at mechanistic interpretability: trying to understand the internal features and circuits that make a Transformer produce an answer.

Today we'll zoom in on the piece of the architecture that makes those circuits much easier to reason about:

The residual stream is a shared communication channel running through the entire Transformer.

Attention heads and MLPs don't usually pass private messages directly to one another. Instead, they repeatedly read from and write to this common vector.

A useful first approximation is:

\[ \boxed{\text{residual stream} \approx \text{the model's shared scratchpad}} \]

Understanding this one idea makes the architecture feel much less like a mysterious stack of layers.

Start with the usual picture¶

We often draw a Transformer like this:

tokens
  ↓
embedding
  ↓
attention
  ↓
MLP
  ↓
attention
  ↓
MLP
  ↓
...
  ↓
logits

That picture suggests each component replaces the representation produced by the previous one.

But modern Transformers are built around residual connections.

A better picture is:

              attention
             ↗         ↘
residual ───────────────────── residual
stream                         stream
             ↘         ↗
                 MLP

Each component computes some update and adds it to the existing state.

Schematically:

\[ x_{l+1}=x_l+f_l(x_l) \]

where:

\(x_l\) is the current residual-stream state,
\(f_l\) is an attention or MLP computation.

The old information is not discarded.

The component contributes a delta.

One token has one residual vector at each layer¶

Suppose:

\[ d_{model}=4096. \]

For a sequence of \(T\) tokens:

\[ X_l\in\mathbb R^{T\times4096}. \]

Each token position therefore has a 4096-dimensional residual vector.

For the prompt:

The capital of France is

we can imagine:

The      → x₁
capital  → x₂
of       → x₃
France   → x₄
is       → x₅

Every layer modifies these vectors.

But because the modifications are residual additions, the computation looks like:

\[ X_0 \rightarrow X_0+\Delta X_1 \rightarrow X_0+\Delta X_1+\Delta X_2 \rightarrow\cdots \]

So after many components:

\[ X_L = X_0+ \sum_{l=1}^{L}\Delta X_l. \]

This is an extremely useful mental model.

The final representation is the original token representation plus a long sequence of learned edits.

Attention reads the scratchpad and writes back into it¶

Recall attention:

\[ Q=XW_Q,\qquad K=XW_K,\qquad V=XW_V. \]

More precisely, an attention block reads the current residual stream and projects it into Q, K and V spaces.

After attention computes its result, another projection maps that result back into the model dimension:

\[ \Delta X_{attn} = \operatorname{Attention}(Q,K,V)W_O. \]

Then:

\[ X'=X+\Delta X_{attn}. \]

So attention does not replace \(X\).

It says something like:

I found useful information elsewhere in the sequence. Let me write that information into the shared state.

This gives attention heads a useful interpretation:

\[ \boxed{\text{attention moves information between token positions}} \]

through the residual stream.

A toy example: copying “France”¶

Consider:

The capital of France is

Suppose the residual vector at the final token is initially contains information roughly corresponding to:

current position
grammatical context
expects a noun phrase next

An attention head at is might strongly attend to France.

Its output could write a direction into the residual stream corresponding roughly to:

relevant country = France

So before the head:

\[ x_{is}=a \]

and the head computes:

\[ \Delta x_{attn}=b. \]

After the residual addition:

\[ x'_{is}=a+b. \]

Now later components can read \(x'_{is}\) and see both:

expects completion
+
France is the relevant entity

The head didn't have to call the later MLP directly.

It simply wrote information onto the shared scratchpad.

MLPs use the same channel¶

Now imagine the next MLP receives:

\[ x'_{is}. \]

An MLP is roughly:

\[ MLP(x)=W_2\sigma(W_1x). \]

It can detect combinations of features in the residual stream.

Perhaps the current vector contains directions corresponding to:

country = France
relation = capital
factual completion

The MLP may transform that combination into another useful direction:

likely answer = Paris

Its output is again added:

\[ x''_{is} = x'_{is} + MLP(x'_{is}). \]

So the computation could look conceptually like:

residual stream:
"we need a factual completion"
        ↓
attention writes:
"the relevant country is France"
        ↓
MLP reads both and writes:
"capital-of-France points toward Paris"
        ↓
later layers refine
        ↓
unembedding produces high Paris logit

This is much closer to how interpretability researchers often reason about Transformer circuits.

The output layer also reads the residual stream¶

At the end, the model converts the final residual vector into vocabulary logits.

Ignoring normalization details for a moment:

\[ z=W_Ux_L \]

where:

\[ W_U\in\mathbb R^{V\times d_{model}} \]

is often called the unembedding matrix.

For token \(v\):

\[ z_v=w_v^Tx_L. \]

So if some component writes a vector into the residual stream that aligns with:

\[ w_{Paris}, \]

the Paris logit increases.

This gives us a surprisingly concrete way to think about model computation:

Components write vectors into the residual stream; those vectors eventually push vocabulary logits up or down.

A tiny numerical example¶

Let's shrink the model dimension to:

\[ d_{model}=3. \]

Suppose the current residual state is:

\[ x= \begin{bmatrix} 1\\ 0\\ 0 \end{bmatrix}. \]

An attention head writes:

\[ \Delta x_{attn} = \begin{bmatrix} 0\\ 2\\ 0 \end{bmatrix}. \]

Now:

\[ x'= \begin{bmatrix} 1\\ 2\\ 0 \end{bmatrix}. \]

Then an MLP writes:

\[ \Delta x_{MLP} = \begin{bmatrix} 0\\ 0\\ 3 \end{bmatrix}. \]

Final state:

\[ x''= \begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix}. \]

Suppose the unembedding direction for Paris is:

\[ w_{Paris} = \begin{bmatrix} 0\\ 0.5\\ 1 \end{bmatrix}. \]

Then the Paris logit is:

\[ z_{Paris} = w_{Paris}^Tx'' \]

\[ = 0(1)+0.5(2)+1(3) \]

\[ =4. \]

Before the attention and MLP updates, it was:

\[ w_{Paris}^Tx=0. \]

So two components collaboratively moved the model from:

Paris logit ≈ low

to:

Paris logit ≈ high

by writing different pieces of information into the same vector space.

This explains why circuits can span layers¶

Suppose head 7 in layer 10 writes a feature:

current subject = France

into the residual stream.

An MLP in layer 12 can read it.

An attention head in layer 17 can combine it with information from another token.

An MLP in layer 21 can transform the result again.

Nothing requires those components to be adjacent.

They communicate because the residual stream persists through the network.

Conceptually:

Layer 10 head
     │ writes A
     ▼
RESIDUAL STREAM
     │
Layer 12 MLP reads A, writes B
     ▼
RESIDUAL STREAM
     │
Layer 17 head reads B, writes C
     ▼
RESIDUAL STREAM
     │
Layer 21 MLP reads C

This is why the residual stream is so important for mechanistic interpretability.

It is the bus connecting the model's computational components.

Residual connections also help optimization¶

Residual streams were not invented primarily for interpretability.

They make deep networks much easier to train.

Without a residual connection:

\[ x_{l+1}=f_l(x_l). \]

Every layer must completely transform the representation passed to the next layer.

With a residual connection:

\[ x_{l+1}=x_l+f_l(x_l). \]

A layer can initially learn:

\[ f_l(x)\approx0 \]

and leave the representation mostly unchanged.

It only needs to learn useful corrections.

Gradients also have a direct path backward through the identity connection.

If:

\[ x_{l+1}=x_l+f_l(x_l), \]

then:

\[ \frac{\partial x_{l+1}}{\partial x_l} = I+ \frac{\partial f_l}{\partial x_l}. \]

That identity term:

\[ I \]

provides a clean route for gradient flow through very deep networks.

This is one reason residual architectures made extremely deep neural networks practical.

Pre-norm Transformers add one wrinkle¶

Many modern Transformers use a structure roughly like:

\[ x' = x+ Attention(Norm(x)) \]

followed by:

\[ x'' = x'+MLP(Norm(x')). \]

The normalization affects what the component reads, while the result is added back into the persistent residual stream.

So a more accurate picture is:

residual stream
      │
      ├─────── identity path ───────────────┐
      │                                     │
      ↓                                     │
     norm                                   │
      ↓                                     │
  attention                                 │
      ↓                                     │
    output                                  │
      └──────────── add ────────────────────┘
                    ↓
             residual stream

Then the MLP performs a similar read-transform-add operation.

The exact normalization scheme varies by architecture, but the read/write intuition remains useful.

Why “scratchpad” is useful but imperfect¶

Calling the residual stream a scratchpad can make it sound like there are explicit slots:

dimension 17 = country
dimension 892 = sentiment
dimension 2041 = current owner

Yesterday's lesson warned us that this is usually too simple.

Because of superposition, features are often represented as directions distributed across many dimensions.

So the scratchpad is not like a spreadsheet with named columns.

It's more like a high-dimensional analog workspace.

A component can write:

\[ v_{France} \]

and another can detect that direction through a learned projection.

Many such feature directions may coexist in the same 4096-dimensional vector.

This leads to a powerful interpretability trick¶

Suppose we want to know when the model begins to “know” that the answer is Paris.

At every layer \(l\), we have a residual vector:

\[ x_l. \]

We can project that intermediate vector through the model's unembedding matrix:

\[ z^{(l)}=W_Ux_l. \]

Then inspect the Paris logit as depth increases.

Perhaps we see:

Layer  2: Paris rank 18,000
Layer  8: Paris rank  2,400
Layer 14: Paris rank    190
Layer 20: Paris rank     12
Layer 26: Paris rank      2
Layer 32: Paris rank      1

This kind of technique is often called a logit lens.

It asks:

If we interpreted the residual stream right now as though it were the final representation, what answer is it leaning toward?

There are caveats—intermediate representations are not necessarily calibrated for the final unembedding, and normalization matters—but it gives us a window into how predictions develop across layers.

We can go even finer: logit attribution¶

Because residual updates are additive:

\[ x_L = x_0+ \Delta x_1+ \Delta x_2+\cdots+\Delta x_n, \]

and logits are approximately linear in the final residual state:

\[ z_v=w_v^Tx_L, \]

we can write:

\[ z_v = w_v^Tx_0 + \sum_i w_v^T\Delta x_i. \]

That means we can ask:

How much did this particular attention head's output push the Paris logit?

If one head writes:

\[ \Delta x_h, \]

its direct contribution is roughly:

\[ \Delta z_{Paris} = w_{Paris}^T\Delta x_h. \]

Maybe:

Layer 18, Head 7      +1.4 Paris logit
Layer 20, MLP         +2.1
Layer 24, Head 3      -0.3
Layer 27, MLP         +1.7

This doesn't completely explain the computation, because components also change what later components will do.

But it gives us a powerful accounting system.

We can literally measure which components are writing in directions that favor particular outputs.

Connect this back to yesterday's sparse features¶

Yesterday we said an SAE might identify a feature corresponding to:

country-capital relationship

Where does that feature live?

Often, what we're analyzing is precisely an activation in the residual stream or another internal activation derived from it.

So the interpretability stack starts to fit together:

RESIDUAL STREAM
      ↓
messy high-dimensional representation
      ↓
sparse autoencoder
      ↓
candidate interpretable features
      ↓
trace which components
read/write those features
      ↓
identify circuits

The residual stream gives us the shared medium.

SAEs try to give us a vocabulary for describing what's represented in that medium.

Circuit analysis tries to explain how components manipulate those representations.

The deeper mental model: a shared whiteboard¶

Imagine 64 specialists working on one problem.

Instead of talking directly to one another, they share a giant whiteboard.

Specialist 1 writes:

Relevant entity: France

Specialist 7 notices that and writes:

Question asks for capital

Specialist 19 combines both:

Candidate: Paris

Specialist 31 notices another clue and strengthens it.

Specialist 44 writes:

Paris confidence ↑

Nobody owns the whole solution.

Each specialist:

\[ \boxed{\text{reads the board}} \]

performs a specialized transformation, then:

\[ \boxed{\text{writes a small update back}} \]

The board persists.

That's the residual stream.

And it gives us a much more concrete picture of what a Transformer layer is doing:

\[ \boxed{ x \rightarrow x+\text{attention update} \rightarrow x+\text{attention update}+\text{MLP update} \rightarrow\cdots } \]

The Transformer is not repeatedly replacing its thoughts.

It is progressively editing a shared high-dimensional state.

That perspective will be useful for the next step: looking at what MLPs actually do when they read the residual stream—why the feed-forward network is much more than a generic nonlinear layer, and why it is often described as a kind of learned key-value memory.