Thinking (reasoning) models
A third training stage that lets the model stop and reason — worth the wait on hard problems.
After pre- and post-training comes a third stage: reinforcement learning, where the model practices on piles of math and code problems and discovers its own thinking strategies — trying ideas, backtracking, revisiting assumptions. The result is a "thinking model": give it a hard problem and it emits a long internal monologue for seconds or minutes before answering. Slower, but markedly more accurate where it counts.
Reinforcement learning is where the model gets to practice, like doing every exercise in the back of the textbook. In that practice it finds thinking strategies no human labeler could hand-script.
Big wins on math, code, and genuinely hard reasoning. Little benefit on simple asks (travel ideas, casual writing) — you'd just wait longer for the same thing. Karpathy's rule: try the fast model first; if you suspect the answer could be better, switch to a thinking model and let it cook.
His concrete case: a gradient-check bug. Plain GPT-4o listed plausible-but-wrong things to double-check and didn't solve it. Switched to a thinking model, it churned for ~a minute and found the real bug — mismatched parameter packing.
“There exists a class of models that we call thinking models… most effective for difficult problems in math and code. In those kinds of cases, they can push up the accuracy of your performance.”
Andrej Karpathy·23:05
Naming is a mess. OpenAI's thinking models start with “o” (o1, o3-mini, o3-mini-high, o1-pro). Grok has a Think toggle. Perplexity hosts DeepSeek-R1 and shows its raw thoughts. Look for a “think / reason” control — it's the same idea everywhere.
The load-bearing points
- Thinking models come from a third stage: reinforcement learning.
- They emit a long internal monologue — slower, but more accurate on hard problems.
- Use them for math / code / hard reasoning; skip them for easy asks.
- Default fast; escalate to thinking when the answer needs to be right.
A/B the same bug
Take a tricky logic or code problem. Ask a normal model, then a thinking model with the exact same prompt. Compare correctness and time.
Show the point
This is his gradient-check experiment. The gap is widest on problems that need multi-step reasoning.
Don't over-think
Ask a thinking model something trivial and time the wait. Feel why you wouldn't use it for everything.
Show the point
Latency is the cost. Reserve reasoning models for when accuracy on a hard task is worth the minute.
From reinforcement learning: practicing on many math/code problems, the model discovers effective reasoning moves (try, backtrack, re-check) on its own — strategies too open-ended for humans to hard-code.