Inference-time compute changed the deal. For a decade a model's cost per query was roughly fixed, and capability lived in the weights. Reasoning models broke that: they spend a variable number of thinking tokens before answering, capability now scales with spend at inference, and the meter runs on both.
The research on test-time scaling is genuinely impressive. On hard mathematics, competitive code and multi-hop analysis, accuracy climbs with thinking budget in a way that looks like a new scaling law, and it is the most interesting capability development since instruction tuning.
The commercial translation has quietly become: the model now decides how much your query costs. And on many workloads it decides badly.
The gains are real and narrow
Test-time compute pays on problems with three properties, and the third is the one that gets dropped.
The problem has to be hard — the model does not get it right immediately, so there is headroom for additional computation to occupy. It has to be decomposable, so that additional computation has somewhere to go. And critically it has to be verifiable: wrong paths must be catchable and correctable mid-reasoning, because that is the mechanism by which extra steps improve the answer rather than merely extending it.
Verifiability is the load-bearing property and the one most often assumed. Where a model can check its own intermediate work — a calculation that either balances or does not, code that either passes a test or does not — extra steps are error-correcting. Where it cannot, extra steps are error-accumulating, and each additional inference is another opportunity to drift. This single distinction explains most of the variance in whether reasoning helps.
Which is a specific and fairly small problem class. On extraction, classification, retrieval-grounded lookup, routing, formatting and short-text summarisation — most of the high-volume work enterprises actually run — the measured gain from long reasoning is small to nil, while the token bill multiplies five- to fiftyfold.
Paying reasoning rates for lookup work is the single most common silent waste in 2026 inference budgets. That is the reasoning tax, and it is silent precisely because nothing breaks: the answers are fine, the system works, and the only symptom is an invoice nobody has decomposed.
Overthinking is measured, not metaphorical
The second finding is the one that should change configurations, because it says the failure is not merely economic.
Studies of test-time scaling report an inverted-U: accuracy rises with reasoning length, peaks, and then declines as chains become excessively long. One reported measurement makes the shape concrete — accuracy increasing from 82.2 percent to 87.3 percent as average thinking tokens rise from 385 to 1,100, then falling to 70.3 percent as tokens are pushed to 15,980.
Read the ends of that curve against each other. The model finished substantially worse than it started, having spent roughly forty times as many thinking tokens to get there. Both halves of that sentence are bad, and the first half is the one people miss because they are watching the invoice.
The associated finding is equally useful operationally: incorrect answers frequently correspond to longer reasoning chains than correct ones. Length is therefore an observable signal — not a reliable classifier of correctness, but a cheap indicator worth instrumenting, because a distribution of trace lengths that is drifting upward is telling you something before the accuracy metrics do.
The mechanisms reported are unsurprising once stated. Models second-guess correct first answers. They elaborate into distractor paths. They accumulate error across steps that had nothing to correct them. And on tasks that were flat, they hallucinate structure — producing text that superficially resembles reasoning without corresponding problem-solving.
If that shape sounds familiar, it should: like context capacity, reasoning capacity is a resource whose blind maximisation the research keeps punishing. More is a budget, not a strategy.
Budgets are controllable, and almost nobody controls them
The third finding is the encouraging one, and it is why this is a management problem rather than a research problem.
The APIs expose reasoning-effort controls. Routing lets you send hard cases to thinkers and volume to workhorses. And a small classifier in front — is this query in the pays-for-thinking class? — recovers most of the tax at trivial cost, because the classification is easy: the two classes look different.
The tooling exists. The discipline mostly does not. The median deployment picks one model, one setting, and lets the meter decide — which is a defensible choice at pilot scale and becomes an expensive one at volume without anyone making a second decision.
The reason to build the classifier early is not the money, though the money is real. It is that reasoning budget is a per-action-class decision, and decisions of that shape are much cheaper to make before there are twenty action classes in production. The same argument applies to context budgets, and for the same reason: both are governance decisions wearing an infrastructure costume.
In agentic systems the two bills multiply
Here is the consequence that pilots systematically hide, and it is the one I would take to a CFO.
An agent that reasons at every step of a twenty-step workflow is paying the reasoning tax times the re-read multiplier from the context literature. The cost is a product, not a sum — reasoning multiplier by step count by context size — and a pilot, which is one call at low volume, exhibits neither multiplier.
There is a compounding effect on top that is easy to miss. Thinking traces are themselves context: long, model-generated, and increasingly carried across steps in agentic systems. So step one's reasoning output becomes part of step two's input cost, and of step three's, and so on. A system that reasons verbosely early is inflating every subsequent step, and the growth is superlinear in a way nobody modelled.
The practical control is the same one that works for context: set reasoning budgets per action class, and treat a step that carries prior traces forward as a design decision requiring justification rather than a default. Most agentic frameworks carry traces forward by default because it sometimes improves coherence; almost none surface what it costs.
The evidence question: what a trace is worth
The governance consequence sharpens the same way. When a reasoning agent acts, is the trace part of the attestation record?
The honest current answer is that traces are neither reliable explanations nor safely ignorable, and both halves matter.
They are not explanations because the faithfulness literature is bruising: a model's stated reasoning demonstrably diverges from the computation that actually produced its output. A trace is a plausible narrative generated alongside an answer, not a causal account of how the answer arose. That distinction is invisible to a non-specialist reader — which is exactly why filing traces as justification is dangerous. A reviewer handed a coherent chain of reasoning will treat it as an explanation, and it will be doing work it cannot support.
They are not ignorable because they are genuinely useful telemetry: for debugging a bad answer, for detecting overthinking through length distributions, for spotting behavioural drift after a model update. Discarding them throws away the cheapest observability signal the system produces.
So: treat traces as diagnostic telemetry, never as justification. The record that survives review remains the one captured outside the model — what it knew, from where, under which grant, past which check. The trace sits alongside that as context, and is never filed under why this was allowed.
One retention note follows from the cost argument. Traces are large, and retaining all of them at production volume is expensive. The defensible policy is to retain traces for actions above a materiality threshold and sample the rest — which is a decision someone should make deliberately rather than discover when storage costs arrive.
The transfer rules
Four, in the order I would apply them.
Route by problem class, not by model brand. Build the ten-line classifier before the ten-thousand-dollar month. The classification is easy and the saving is immediate.
Set reasoning budgets per action class, the way the context discipline sets context budgets. They are the same control at different layers, and a programme that has one and not the other has done half the work.
Evaluate reasoning models on your distribution at production settings. A leaderboard score achieved at maximum thinking budget describes a cost structure, not just a capability — and the gap between benchmark configuration and production configuration is where the surprise lives.
And when accuracy on a workload does not move with thinking budget, believe the flat line. You have found lookup work wearing a hard problem's costume, and the workhorse tier is waiting. This is the cheapest experiment in the whole stack: run your evaluation at two effort settings and compare. If the line is flat, you have just found your saving.
How to find your turn, in an afternoon
The claim that you have to locate your own turning point is only useful if locating it is cheap, so here is the protocol.
Take one representative workload and a held-out evaluation set you already trust — a hundred cases is usually enough to see the shape, and if you do not have a hundred trusted cases that is the more urgent finding. Run the same set at three or four reasoning-effort settings from minimum to maximum, holding everything else constant.
Record two numbers per setting: accuracy on your metric, and mean output tokens. Plot them against each other. Three shapes are possible and each has a different action.
A flat line means the workload does not pay for thinking, and you should route it to the workhorse tier today. A monotone rise means you are on the left side of the curve and can afford to spend more, though it is worth extending the range to check the rise does not stop just past where you looked. And a rise-then-fall means you have found your turn, and the setting to deploy is at or slightly below the peak — not at the peak itself, because the peak is estimated from a sample and you want margin on the side that is merely more expensive rather than the side that is both more expensive and less accurate.
That last asymmetry is the practical heart of it. The two failure directions are not symmetric: undershooting the turn costs you a little accuracy at low price, while overshooting costs accuracy and money together. Bias low.
The whole exercise is an afternoon and it is repeatable per action class. What it produces is not a permanent answer — model updates move the curve — but a number you chose rather than one the vendor's default chose for you, and a method you can re-run when the model changes.
Where the tax actually lands, by deployment shape
The three findings hit different architectures with very different force, which is worth knowing before deciding how much of this to act on.
A single-call assistant answering user questions is the most forgiving shape. The volume is bounded by human attention, the cost is linear, and overthinking shows up as latency the user notices — which is a feedback signal, however crude. Routing helps here and the classifier pays for itself, but nothing is compounding.
A high-volume batch pipeline — classification, extraction, enrichment over a queue — is where the reasoning tax is largest and most invisible, because there is no human waiting and no latency signal at all. This is the shape where a default reasoning setting silently multiplies a bill by an order of magnitude and the only symptom is the invoice. It is also the shape where the flat-line test almost always comes back flat, which makes it the highest-return place to run the protocol above.
A multi-step agent is where the accuracy finding bites hardest rather than the cost one. Overthinking at step three produces a wrong intermediate that steps four through twenty build on, so the error does not stay local — it propagates and compounds, and the final output is wrong for a reason that is now several steps away from where it originated. That is the shape in which trace-length instrumentation earns its keep, because a step whose trace has grown unusually long is a plausible place to look when the end of the chain is wrong.
The strongest objection: this optimises for today's cost curve
The serious counter-argument is about timing, and it is the one I find hardest to dismiss.
It runs: inference prices are collapsing, and the reasoning premium will collapse with them. Building routing infrastructure, classifiers and per-action-class budgets is engineering effort spent optimising a cost curve that will flatten on its own. In eighteen months the price difference between the thinking tier and the workhorse tier will be small enough that the whole discipline is unnecessary, and the teams that ignored it will have shipped more features. Optimise for capability now and let the market solve the cost.
The price observation is correct and well evidenced. Where the argument breaks is that it addresses only one of the three findings — and the weakest one.
Overthinking is an accuracy problem, not a cost problem. If accuracy falls from 87 percent to 70 percent at high thinking budgets, that damage does not become acceptable when the tokens get cheaper. Cheap tokens make it worse, in fact, because they remove the economic signal that would otherwise have prompted someone to investigate. The reasoning tax is at least self-announcing through the invoice; overthinking is silent, and the falling price curve makes it more silent.
And the multiplication argument is structural rather than price-dependent. In a twenty-step workflow carrying traces forward, the growth pattern holds at any unit price; a lower price changes when it hurts, not whether. So the honest scoping is: if your concern is purely the bill, the objection has real force and you might reasonably wait. If your systems act on their outputs, the accuracy finding binds now and does not care what tokens cost.
Three markets, three different bindings
The findings are universal. Which one binds first is not, and it changes what to do this quarter.
In North America the binding constraint is usually accuracy rather than cost, because budgets absorb the tax and the workloads that matter are decision-support rather than high-volume batch. The finding to act on is overthinking: run the effort sweep on the workloads that feed consequential decisions, and be prepared for the answer that your most important workload is configured past its turn.
In India the cost finding binds hardest and binds immediately. Price pressure is severe, volumes are high, and the batch-pipeline shape dominates — which is precisely the configuration where the tax is largest and least visible. Here the classifier is not an optimisation, it is a margin decision, and the flat-line test is the highest-return afternoon available to most teams.
In the Gulf, new-build agentic deployments mean the multiplication finding is the one to internalise before it is expensive. Systems being architected now will carry traces across steps by default unless someone decides otherwise, and that decision is nearly free at design time and structurally awkward afterwards. The question to put into the design review is simply whether prior reasoning is carried forward, and what it costs when it is.
Where this argument is weakest
Two admissions.
First, the inverted-U figures cited are one reported measurement on one task family. The qualitative shape is reported consistently across studies, but the specific turning point — around 1,100 thinking tokens there — is a property of that setup and emphatically not a number to configure against. The transferable claim is that a turn exists and that you have to find yours; the transferable method is running your evaluation at several effort settings.
Second, the five- to fiftyfold range for the token multiplier is wide because it genuinely varies by model, setting and task, and I have not narrowed it because narrowing it honestly would require your workload. Treat it as an order-of-magnitude warning rather than a planning figure.
The summary
The capability is real. The tax is real. And overthinking means the two are not simply traded against each other — past the turn you are paying more for less, which is the only genuinely indefensible position available.
The difference between programmes that get the capability without the tax is, as ever, measurement.