Test-time compute — making a model smarter without changing its weights

A model can improve in two ways: 1. change the model, 2. let the same model do more work on this question.

The second is test-time compute.

Longer reasoning trajectories¶

For the bat-and-ball problem, a quick model might answer “10 cents.”

A more deliberate trajectory writes:

The extra tokens create extra Transformer evaluations and scratch state.

Multiple samples¶

Generate several candidate solutions and aggregate them.

If one independent attempt succeeds with probability 0.7, majority vote over three succeeds with:

under the independence assumption.

Real errors are correlated, so gains may be smaller.

Generate then verify¶

For code: - sample several candidate programs, - run tests, - select the candidate that passes.

This turns inference into search.

Branching¶

Instead of committing to one reasoning path, explore multiple promising branches and prune weak ones.

The LLM becomes a learned proposal policy inside a search algorithm.

Elastic compute¶

Easy questions may use a tiny budget.

Hard questions may use: - many samples, - tools, - search, - revision, - verification.

That creates a new scaling dimension:

in addition to model size \(N\) and training data \(D\).

Limits¶

More compute helps only if: - a correct proposal is reachable, - sampling explores diverse possibilities, - verification can identify quality.

Otherwise extra computation merely repeats the same mistake.

One model call versus a computation graph of calls¶

Ordinary generation appears as:

prompt → one trajectory → answer

Test-time scaling can instead create:

prompt
 ├→ attempt A ─┐
 ├→ attempt B ─┤
 ├→ attempt C ─┼→ verifier/search → answer
 └→ attempt D ─┘

The “AI system” is now larger than the model.

Self-consistency needs diversity¶

If every sample uses the same flawed shortcut, voting merely amplifies the mistake.

Temperature, prompt variation, branching strategy, and model stochasticity can increase diversity, but useful diversity is not arbitrary randomness.

The objective is to explore meaningfully different plausible approaches.

Verification changes the economics of sampling¶

If generating candidates is cheap and verification is reliable, you can intentionally overproduce.

For a candidate success probability \(p\), \(N\) independent attempts have:

This rapidly increases when \(p\) is nonzero.

But without a verifier, finding the successful candidate among many outputs becomes the hard part.

Test-time compute as a third scaling axis¶

We previously had: - model size \(N\), - data \(D\).

Now add:

A deployed system can dynamically allocate \(I\) according to problem difficulty and value.

That makes intelligence partly a resource-allocation problem at inference time.

Compute allocation can itself be learned¶

A sophisticated system need not use the same reasoning budget for every prompt.

It can first estimate difficulty or uncertainty.

Easy problem:

one short trajectory

Hard problem:

multiple branches
tool calls
verification
longer search

This resembles adaptive algorithms that spend work only where necessary.

Stopping is part of reasoning¶

More inference compute is not automatically better.

A model that continues generating after it already has a correct solution can: - waste latency, - introduce new mistakes, - consume expensive tokens.

So test-time scaling also requires a stopping policy:

Do I know enough to answer now, or is another step/sample/tool call worth its cost?

The ideal system optimizes not raw accuracy alone but something like:

That turns reasoning into an economic decision as well as an algorithmic one.

Verification can dominate the value of extra samples¶

Suppose one-shot accuracy is 40%.

Generating ten candidates is potentially powerful—but if your selector chooses the correct candidate only slightly better than chance, much of the gain disappears.

Thus inference scaling has a bottleneck analogous to training data quality:

This is why reasoning systems invest in: - outcome checkers, - process reward models, - tool execution, - self-consistency, - external judges.

The next lesson focuses precisely on that verifier layer.