AI Engineer Roadmap All lessons Interactive Quiz GitHub

Reasoning Models and Test-Time Compute

Advanced · 10 min read · Updated · By Saurabh Pathak

In short: A reasoning model spends extra tokens thinking before it answers, and test-time compute is the idea that you can buy better answers with more thinking at inference time instead of a bigger model.

Play the interactive lesson: Test-time computeRead on GitHub
On this page
  1. Explain it like I'm new
  2. Watch it
  3. How it works
  4. Code example
  5. Common mistakes
  6. Recap
  7. Interview questions
  8. Go deeper

Level: Advanced · Reading time: 10 min · Phase: 10 · Frontier AI Engineering

Before you start: Temperature and Prompt Engineering.

Explain it like I'm new

Imagine a maths exam. One student blurts the first answer that comes to mind. Another uses scrap paper, tries an approach, checks it, and only then writes the final answer. Both students have the same brain. The second just spends more time. A reasoning model is trained to use scrap paper: it writes a long hidden or visible chain of thought, then the answer.

The second idea is that you can also buy accuracy by asking the same question many times and taking the most common answer (self-consistency, or majority voting). Both tricks trade money and delay for quality. That trade is called test-time compute.

Watch it

Test-time compute: Spend more tokens at answer time to buy accuracy.
Test-time compute: Spend more tokens at answer time to buy accuracy.

It plays on a loop. Read the caption in the box, one step at a time. Want to go at your own pace? Step through it with Previous and Next.

How it works

1. Three ways to spend more compute at answer time

  1. Think longer. The model writes more intermediate steps before the answer. Cost grows with the number of thinking tokens, and those tokens are normally billed as output.
  2. Sample many, then vote. Run n independent attempts and pick the most common final answer (Wang et al., 2022).
  3. Sample many, then verify. Generate n candidates and let a checker (unit tests, a calculator, a trained verifier) choose. This works best when checking is easier than solving.

2. How reasoning models are trained, in one paragraph

Models are trained with reinforcement learning on problems that have checkable answers (maths, code). The reward is whether the final answer is right. Over training, the model learns to write longer, more careful chains: re-reading the question, trying a different route, catching its own mistake. You do not hand-write the reasoning; the reward shapes it. See How LLMs Are Trained for where this sits in the pipeline.

3. Why voting works, with real numbers

Suppose one attempt is right with probability p = 0.6 and a wrong attempt always lands on the same single wrong answer. Voting over n samples is right when more than half the samples are right. From the demo (exact binomial arithmetic):

Samples nP(majority correct)
10.6000
30.6480
50.6826
110.7535
210.8256

Hand check for n = 3: 3 × 0.6² × 0.4 + 0.6³ = 0.432 + 0.216 = 0.648.

It gets better when wrong answers disagree with each other. With p = 0.4 and the wrong answers spread over five different values, a seeded simulation of 20,000 trials gives 0.397 at n = 1, 0.701 at n = 5, 0.830 at n = 11 and 0.930 at n = 21. A model that is right only 40 percent of the time becomes right 93 percent of the time, because the right answer is the only one that repeats.

4. When voting hurts

If the model makes the same mistake more often than it finds the right answer, voting makes things worse. With p = 0.45 and a single common wrong answer, the demo gives 0.450 at n = 1, 0.407 at n = 5 and 0.321 at n = 21. Voting amplifies whichever answer is most likely, right or wrong. Independent errors are the assumption that makes voting work, and real model errors are often correlated.

5. The bill

Sampling 21 attempts of 800 tokens each, against one 100-token answer, costs 21 × 800 / 100 = 168 times as many tokens. Latency grows too unless the samples run in parallel. Thinking tokens also fill the context window.

6. Choosing how much to think

Worked example

From real output of fa_reasoning.py.

  1. One attempt is right 60 percent of the time. Three attempts with a vote: 0.648.
  2. Twenty-one attempts: 0.8256. That is 22 points better, and it costs 21 times as many calls.
  3. Change the setup so the model repeats one wrong answer 55 percent of the time. Now 21 votes give 0.321, worse than one attempt.
  4. Lesson: measure on your own data before paying for voting.

Code example

from math import comb

def majority_correct_binary(p, n):
    # n odd; each sample right with probability p
    return sum(comb(n, k) * p**k * (1 - p) ** (n - k)
               for k in range(n // 2 + 1, n + 1))

print(round(majority_correct_binary(0.6, 3), 4))    # 0.648
print(round(majority_correct_binary(0.6, 21), 4))   # 0.8256

Run python 12-frontier-ai/code/fa_reasoning.py; tests: pytest 12-frontier-ai/code/test_fa_reasoning.py.

In real projects you call a model with a thinking or reasoning setting and a budget. You do not write the loop above, but you do need to know that the cost multiplier is real and that voting needs independent errors.

Common mistakes

Recap

If you remember only three things:

Interview questions

1. Why can majority voting turn a 40 percent model into a 90 percent one, and when does it fail? <details><summary>Answer</summary> When wrong answers are spread over many values, the right answer is the only one that repeats, so it wins the vote as n grows (0.40 becomes about 0.93 at n = 21 in the demo with five wrong values). It fails when mistakes are correlated: if the model gives one specific wrong answer more often than the right one, voting amplifies the mistake. </details>

2. How would you decide whether a task deserves extended thinking? <details><summary>Answer</summary> Run a held-out set at several thinking budgets and plot accuracy against cost and latency. Tasks with many dependent steps or a cheap checker usually gain; lookups, formatting and classification usually do not. Choose the smallest budget where the curve flattens. </details>

3. What is the difference between voting and verifying? <details><summary>Answer</summary> Voting picks the most common answer and needs no checker. Verifying generates candidates and uses a checker (tests, a calculator, a trained verifier) to pick one that passes. Verifying beats voting when checking is much cheaper than solving. </details>

Go deeper

Prev: Phase 9 · Multimodal AI · Next: Context Engineering and Prompt Caching · Back to phase: Phase 10