The Cheapest Model Is the One That Finishes: Cost Per Solved Task

Every pricing page quotes dollars per million tokens. Nobody buys tokens. You buy finished work: a merged pull request, a triaged ticket, a migration that runs.

So the number worth knowing is the cost of a task that actually got done, which is the price of one attempt divided by how often the attempt succeeds. A model at a third of the price that succeeds a third as often has saved you nothing.

Here is what that arithmetic says right now, and the answer is more favourable to open models than the sticker prices already suggest.

The two inputs

Start with what an attempt costs. These are list prices per million tokens:

Then how often each one finishes. Terminal-Bench 2.1 is the benchmark with the widest coverage across these four, so it is the one that supports a comparison at all:

Look at those two charts together and the argument is already visible. The success rates sit within four points of each other. The prices differ by nearly two orders of magnitude.

The arithmetic

Take a mid-sized agentic coding task: 250,000 input tokens accumulated across turns of repository context, and 30,000 output tokens of reasoning and diffs. Cost of one attempt is (0.25 × input price) + (0.03 × output price).

Then divide by the success rate, which is the same as saying: if a model finishes 84% of the time, you pay for about 1.19 attempts per finished task.

GLM-5.3-Flash lands within 0.3 points of Claude Fable 5 on this benchmark and gets there for about one seventy-sixth of the cost per finished task.

At a thousand tasks a month, that is $62 against $4,728.

The strongest objection, worked through

Prompt caching. Agentic coding re-sends the same repository context on every turn, cached input is dramatically cheaper than fresh input, and the table above ignores that entirely. Anthropic has also been pushing cache reads down hard, the Fable 5.1 release cut them by 75%.

So give the expensive model the most generous version of its own case: 90% of that 250K input served from cache at $0.25 per million, the remaining 10% at full price.

Caching more than halves it, and the gap is still 34×. The reason is structural: output tokens are where agentic coding spends, caching does nothing for them, and $50 per million output is $50 per million output.

Run the sensitivity the other way, an output-heavy profile of 100K in and 80K out, and the ratio widens rather than narrows, because output is the expensive half.

What this number leaves out

Publishing a ratio like 76× obliges you to say where it breaks.

Retries are treated as independent, which flatters the cheaper model. Dividing by the success rate assumes a second attempt is a fresh roll of the dice. Failures correlate: a task a model cannot do, it usually cannot do on the third try either. For the slice of work that a weaker model simply never completes, the honest cost is not "more attempts", it is that the task needs a stronger model. That is an argument for routing, which is the conclusion below.

Scores are vendor-reported and harnesses differ. GLM-5.3's figure comes from Zhipu's model card; the Qwen and Fable numbers come from Alibaba's published comparison table. Different scaffolds, different prompts, no independent referee. Treat four points of separation as noise, which is exactly why the price column carries the argument.

Terminal-Bench 2.1 is one benchmark, and the family is confusing. Terminal-Bench 2.1, 3.0 and 4.0 are different tests, and their scores are not interchangeable. We have written separately about why those version numbers matter.

Engineer time is the real budget. A failed run costs a person's attention as well as tokens. That argues for spending more on the attempts a human is waiting on, and less on the ones running unattended overnight.

The move this suggests

The table has an obvious reading and a better one.

The obvious reading is "use the cheap model". The better one is that cost per solved task varies by 76× while quality varies by 4%, so the win is in matching the model to the task rather than standardising on one.

That means routing. Send routine work, the bulk of it, to GLM-5.3-Flash at six cents. Send the genuinely hard cases to a larger model. Put a verifier between the model and the merge button so that "finished" means tests passed rather than the model saying it was done.

Three practical consequences:

• Measure your own rate, not the benchmark's. Your success rate on your codebase is the only number in that table you can actually change. Log attempts and completions per model and the arithmetic becomes yours.
• Route by difficulty, and let the cheap model try first. A first pass at six cents that succeeds four times out of five makes the expensive fifth attempt easy to afford.
• Spend the savings on verification. The money not spent on tokens buys a test suite, a judge pass, or a second opinion. All of those raise the success rate, which is the denominator of the whole calculation.

Keeping that choice open is the reason the model layer is a slot rather than a vendor: frontier endpoint, open weights in your region, or fully on-premises, swapped by configuration.

For how we assemble that into a working setup, see running open models at frontier level. For the current field, see the best open source coding models.

Method

Cost of one attempt is (input_tokens ÷ 1M × input_price) + (output_tokens ÷ 1M × output_price), using a 250K input / 30K output profile. Expected cost per solved task is that figure divided by the benchmark success rate, which models retries as independent. Prices and scores are the list prices and published Terminal-Bench 2.1 results cited in the charts above, all as of 30 August 2026. The arithmetic is deliberately simple so you can redo it with your own token profile and your own measured success rate.