There is no single correct way to make a language model handle numbers. There are several competing approaches, and each one is genuinely better than the others along some axis and worse along another. The teams that struggle are the ones looking for the universally best method. The teams that succeed pick the right method for the specific task in front of them, and they can articulate why.
The three approaches that dominate practice are reasoning in natural language (chain-of-thought), reasoning by writing and running code (program-of-thought or code execution), and reasoning by sampling many attempts and reconciling them (self-consistency and verification). They are not mutually exclusive — the strongest production systems combine them — but understanding each in isolation is what lets you combine them deliberately instead of by accident.
This piece lays out the competing approaches, the axes along which they differ, and a decision rule you can apply without a benchmark in front of you. The aim is a way of thinking, not a verdict.
The Competing Approaches
Chain-of-thought reasoning
The model works through the problem in natural language, narrating each step before committing to an answer. This is cheap, requires no external tools, and works anywhere the model runs.
- Strength: it excels at problems where the hard part is figuring out what to compute — multi-step word problems, setting up the right equation, deciding which figures matter.
- Weakness: the actual arithmetic still happens in the token stream, so the final calculation can be wrong even when the reasoning is right.
Program-of-thought and code execution
The model writes code that performs the calculation, and a deterministic engine runs it. The model's job shifts from computing to translating the problem into an executable form.
- Strength: the arithmetic is exact and auditable; you can read the code and see precisely what was computed.
- Weakness: it requires a sandbox, adds latency and operational overhead, and can fail when the model writes subtly wrong code.
Self-consistency and verification
Rather than trusting a single pass, you sample several independent attempts and take the majority answer, or you run a separate check against the result.
- Strength: it catches one-off errors and surfaces instability — when the answers disagree, you know to look closer.
- Weakness: it multiplies cost and latency, and a confidently wrong model can agree with itself.
The Axes That Actually Matter
Once you see the approaches, the decision reduces to weighing a handful of axes against the demands of your task.
Accuracy ceiling versus cost floor
Code execution has the highest accuracy ceiling for the arithmetic itself, but it costs more to build and run. Chain-of-thought is nearly free but caps out lower on exactness. The question is whether your task's accuracy requirement justifies the cost, or whether "usually right" is genuinely good enough.
Latency tolerance
Self-consistency and multi-pass verification are accuracy multipliers bought with time. An interactive assistant that must respond in a second cannot afford five samples; a nightly batch report can. Match the method's latency profile to the patience of the consumer.
Auditability requirements
If a human or regulator will ever ask "how was this number derived," code execution wins decisively because the derivation is the code. Chain-of-thought offers a narrative that reads convincingly but is not a guarantee — the words can describe one calculation while the model performed another.
Failure cost
When a wrong number is merely annoying, lean cheap. When a wrong number causes financial loss, harms a person, or breaks a contract, layer verification on top of computation regardless of the added expense. The cost of the method should be proportional to the cost of being wrong.
Operational and security surface
A quieter axis is how much surface each approach adds to maintain and secure. Chain-of-thought adds none beyond the model itself. A code interpreter adds a sandbox you must lock down, monitor, and keep patched. Multi-pass verification adds orchestration logic that can itself harbor bugs. Two approaches with similar accuracy can differ sharply in how much they cost to operate over a year, and that ongoing burden often matters more than the one-time build. Weigh the long-run maintenance and security cost, not just the headline accuracy.
A Decision Rule You Can Apply Today
You can route most tasks with a short cascade of questions.
- Is the hard part figuring out what to compute, or computing it? If setting up the problem is the challenge, start with chain-of-thought; if the setup is obvious and exactness is the challenge, go straight to code execution.
- Will anyone need to audit the number? If yes, code execution is nearly mandatory, because narrative reasoning is not an audit trail.
- What does a wrong answer cost? Low cost: a single cheap pass is fine. High cost: add verification on top of whatever computes the number.
- How much latency can the consumer absorb? Tight budgets rule out multi-sample methods; generous budgets let you buy accuracy with extra passes.
Run a task through those four questions and the appropriate combination usually falls out. A high-stakes, auditable financial figure with a generous latency budget lands on code execution plus verification. A casual estimate in a chat interface lands on plain chain-of-thought. For the tooling that implements each branch, see Which Tools Actually Make Models Do Math Reliably.
Revisit the decision as conditions change
The decision is not permanent. A task that started as a low-stakes estimate can become a client-facing figure as the product matures, at which point the right approach shifts toward execution and verification. Latency budgets loosen when a feature moves from interactive to batch, and the cost of running code falls over time, which steadily tilts borderline cases toward exactness. Treat the routing as something you re-run when the task's stakes, audience, or constraints move, not as a one-time choice you make and forget.
Why Combining Approaches Beats Picking One
The framing as a choice is a useful simplification, but the best systems refuse the binary. They use chain-of-thought to understand the problem, generate code to compute the answer exactly, and run a verifier to confirm the result against domain constraints. Each approach covers the others' weak spot: reasoning handles the setup, code handles the arithmetic, verification handles the trust.
The cost of combining is real — more latency, more tokens, more moving parts — so you escalate only as the task warrants. Start with the cheapest approach that could plausibly work, measure where it fails, and add the next layer precisely where the failures cluster. This keeps you from paying for verification you do not need while ensuring you have it where you do. When you formalize this into a repeatable method, the structure behind it is worth naming, which we explore in Going Past Basic Math Prompts Into Expert Territory.
Frequently Asked Questions
Is code execution always better than chain-of-thought?
No. Code execution wins on arithmetic exactness and auditability, but it adds latency, operational cost, and a sandbox to secure. For problems where the challenge is reasoning rather than calculation, or where stakes are low, chain-of-thought is the more efficient choice.
When is self-consistency worth the extra cost?
When errors are intermittent rather than systematic, sampling several passes and reconciling them catches the one-off mistakes. It is most valuable for borderline-difficulty tasks where the model is right most of the time but not reliably. It does not help when the model is confidently and consistently wrong.
How do I decide without running a benchmark?
Use the four-question cascade: what is the hard part, who needs to audit, what does a wrong answer cost, and how much latency can you absorb. Those four answers route most tasks correctly even before you measure anything.
Can I start cheap and escalate later?
Yes, and you should. Begin with the least expensive approach that might work, instrument where it fails, and add the next layer where failures concentrate. This avoids paying for accuracy you do not need.
Does combining approaches just multiply the cost?
It adds cost, but you control how much by escalating selectively. You do not run verification on every request — you run it on the requests where the stakes or instability justify it. Targeted escalation keeps the combined system affordable.
What is the most common mistake teams make here?
Searching for a single universally best method instead of matching the method to the task. The approaches have genuine trade-offs; the skill is selection, not finding a winner.
Key Takeaways
- There is no universally best numerical method; chain-of-thought, code execution, and verification each win on different axes.
- Chain-of-thought excels at setting up problems, code execution at exact and auditable arithmetic, verification at catching intermittent errors.
- Decide using four questions: what is the hard part, who audits, what a wrong answer costs, and how much latency you can absorb.
- Auditability requirements and high failure costs push you toward code execution plus verification regardless of expense.
- The strongest systems combine all three, escalating layers only where measured failures justify the added cost.
- Start with the cheapest viable approach and add layers precisely where failures cluster.