Costing LLM usage is not impossible, but it is hard

1Introduction

The inspiration behind this article came from the growing number of news reports of companies exceeding their token budgets, and a wider realisation that capturing the ROI of AI is harder than it sounds. There are two halves to any ROI calculation, and I will focus on what it costs rather than what it is worth, because cost is the less familiar side: tokens, and the way agents consume them, are a newer kind of cost driver. Valuing the return (through revenue growth, cost reduction or cost avoidance) runs on well-trodden methods by comparison.

The prevailing assumption about the cost side is that because models are probabilistic, cost is fundamentally unpredictable. That is the assumption this article sets out to test.

Testing it relies largely on data from a paper published in April 2026, titled “How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks” (Bai et al.). The paper also sets out, in brief, why forecasting this is hard:

1. LLMs are fundamentally probabilistic systems, which means there is always going to be some variance in cost and outcome.
2. Agentic use of LLMs amplifies this variance, because we introduce outside, non-human inputs (e.g. results from tool calls, messages from other agents).
3. Humans are unreliable estimators of how many tokens a task will take.
4. Equally, LLMs also struggle to estimate their own token consumption.
5. Differences between models introduce another layer of complexity.

Despite these challenges, the paper gives us a starting point for forecasting token consumption so businesses can adopt AI in a financially responsible way.

2Same task, different bill

When ChatGPT first arrived, most of us ran the same experiment of giving two sessions the same prompt and seeing whether we would get the same answer. The two answers would always differ, even if ever so slightly, and so would the number of tokens each one consumed.

Agentic tasks amplify this variability, because we provide the model with the latitude to choose its own path to solving a problem with the tools we have given it.

The chart below shows a single real coding task, run four times.¹

Despite holding the problem, the model, and the prompt constant, the most expensive run still costs roughly 50% more than the cheapest. If you assume this variability holds at scale, it is easy to conclude that agentic work is simply too unpredictable to cost.

3Not all tasks are equal

Looking at one agentic coding task in isolation does not tell us much, because the tasks themselves vary so greatly from one to the next. Moving from a single task to a portfolio of them is the first step towards forecasting the cost of agentic work. The chart below shows all 500 coding tasks in the SWE-bench Verified benchmark, each run four times with Claude Sonnet 4.5 and then averaged.

The spread across those 500 tasks is real: they average $1.86 but range from $0.79 at the cheapest to $5.23 at the dearest, with a standard deviation of $0.66, approximately 36% of the mean.

The spread also provides us with an initial shape we can begin to work with: costs are floored at the low end, clustered in the middle, and skewed out to the right. The following section illustrates the usefulness of this with a worked example.

4An illustrative methodology

Moving from theory into practice: imagine a company that has built an agentic coding tool to help develop and maintain its software. It is now seeking approval to roll the tool out, and needs a credible estimate of what it will cost to run. The steps below outline a methodology for arriving at that estimate.

Step 1: Sample

As section 2 showed, the cost of a single task tells you very little, partly because the same task run four times gives you four different bills, but mostly because tasks differ so much from one another (section 3). So you begin by measuring cost across a representative sample of the real workload.

For this illustration, the 500 SWE-bench Verified tasks, run four times each in the paper, serve as our company’s sample.³ Building a sample like this is not free: across all 2,000 runs it cost roughly $3,715.

³The paper ran each of the 500 tasks four times and based its per-task figures on the average of those four runs. A company doing this for itself could run a lighter pilot (each task once) and accept noisier per-task estimates. For this illustration we keep to the paper’s dataset and its four-runs-per-task approach throughout.

Step 2: Extrapolate

The sample gives us its headline figures: across 500 tasks, a mean of $1.86 per task, a median of $1.73, and a standard deviation of $0.66 (~36% of the mean). The next step is to apply them to the proposed usage, which we will assume is 5,000 tasks a month.

The obvious move is to scale the mean, $1.86 × 5,000, to arrive at roughly $9,300 a month. But a single number hides the variability the sample revealed, so finance’s next natural step is to apply that ~36% spread (giving a low case of ~$5,950, a base case of ~$9,300, and a high case of ~$12,650) and budget off the high case to be safe. At first glance that looks prudent, but it is far too conservative, as it assumes the spread scales with volume just as the total does.

The average cost per task never changes (it is $1.86 whether you run one task or 5,000), but the total grows in direct proportion to volume: 5,000 tasks at $1.86 each comes to $9,300.

The spread grows too, but far more slowly. Because tasks are roughly independent, an unusually expensive one is typically offset by others coming in cheap; the deviations fall in random directions and partly cancel rather than stacking up. So while the total grows in step with the number of tasks, its spread grows only with the square root of that number.

This does not make the bill steadier in absolute terms; it actually swings wider. A single task varies by about $0.66 around its $1.86 average; a 5,000-task month varies by roughly $47 around its $9,300 total. What shrinks is the swing as a share of the bill: on one task that swing is 36% ($0.66 of $1.86); on the month it is just 0.5% ($47 of $9,300).

One task is barely predictable: one standard deviation in its cost is 36% of the average, and the spread is skewed; most runs land near or below it, while a few run several times higher. Pooled into a 5,000-task month, that same randomness moves the total by only 0.5%, about $47 on a $9,300 bill. That is the shift from unbudgetable to budgetable, and it comes purely from volume.

On the real data, the collapse looks like this.

Step 3: Budget

Zooming in on the needle chart above and translating it into dollar terms shows us the histogram of the 20,000 simulated monthly runs.

The simulation shifts the conversation from accepting a 36% spread to one about risk appetite. Budgeting at the mean of $9,286 means coming in over budget every other month. A percentile-based approach adds more certainty:

• p95 ($9,364): 19 months in 20 come in under.
• p99 ($9,395): 99 in 100 come in under.

Both sit well below the +36% buffer of $12,650 (which, as the chart shows, did not arise once in 20,000 monthly runs) and, unlike that buffer, each comes with a stated overrun probability.

It is also of note that the residual risk is minor: the single worst month in 20,000 simulated draws cost $9,489, just 2.2% above the mean and 1% over the p99 cap. That also makes choosing a conservative budget cheap: roughly $125 above p95 or $94 above p99 to cover every simulated month.

5Real-world limitations and considerations

While the purpose of this article and the illustrative methodology was to stay as practical as possible and grounded in real-world data, it still comes with some real-world limitations and considerations.

Your sample may not be representative, or may not stay that way

The foundation of the above approach relies on your sample being representative of your intended tasks. This has three key parts:

1. The mix of tasks.
2. The model itself.
3. The environment the agent works in.

Any changes in the above three parts mean that you will need to re-run your sample and recalibrate your forecasting. A model change is the obvious case, as you instigate it deliberately (i.e. you choose to begin using a different model). A drifting task mix or a slowly changing environment requires ongoing monitoring.

The strategic solution to this is to use a dynamic sample, refreshed continuously from a rolling pool of recent real tasks.

A separate caveat sits underneath the headline figures. The 0.5% monthly swing assumes those task-level deviations stay independent, which makes it a floor on the uncertainty rather than the whole of it. A shock that lands on every task at once, such as a pricing or routing change, or a prompt regression, does not average away with volume the way idiosyncratic noise does, because the square root only works against deviations that fall in random directions. The residual risk is not forecast by the method itself; it is held off by the controls around it: the rolling re-pilot above, ongoing monitoring, the per-task cap below, and a scenario buffer kept for a regime change rather than for everyday variance.

A single task can still run away

Because an agent can loop, a single task can in principle spend without limit. Rather than trying to build that into a forecast, you can set a hard per-task cap on tokens. The paper (Bai et al.) lends this some support: accuracy peaks at an intermediate cost and then degrades at the highest cost levels, with the most expensive runs dominated by redundant, repeated file operations rather than real progress. So a cap is more likely to trim waste than to cost you a solution, but it is not free, and the level should be calibrated against its measured effect on solve rate.

Cost is only one side of the equation

As the opening section noted, cost is only one side of the equation; the financial case for any AI solution relies on the value it creates as much as on what it costs to run. The usual disciplines of value creation and capture still apply, for example:

• It is easy to treat AI as a hammer and every problem as a nail, but not every problem is one.
• Even where AI fits, implementation only gets you halfway; benefits realisation matters just as much.

The useful part is that the same sample run that prices the work also measures how often it succeeds, so both sides can be sized from the one exercise.

6Conclusion

The instability of AI spend is real and fully conceded in section 2, but that per-run instability does not make the spend category as a whole unbudgetable. The method illustrated here leans on the same law of large numbers that insurers have used for decades.

The per-run variance is itself measurable, and to a degree you can influence it. The paper tested several models,⁶ and each showed a different spread; the type of task you hand the system is a second lever. You are choosing a cost profile, not accepting a fixed unknown.

None of this is offered as a standard. It is one indicative method, built to be explanatory and practical rather than definitive. In a discipline still taking shape, more sophisticated (and easier) methods are likely to emerge.

There are also exciting advances happening in this field that will only improve the accuracy and accessibility of understanding token spend:

1. Consumers can use open-source projects like agentsview⁷ to better understand their spend.
2. Researchers are making agents more aware of their budgets, so they spend more deliberately against a fixed cap (e.g. the BATS paper).⁸
3. Initiatives like the Tokenomics Foundation⁹ (which the Linux Foundation has announced its intent to launch) aim to mature and standardise the way enterprises manage token spend.

⁶Eight frontier models in all: Claude Sonnet-3.7, Sonnet-4 and Sonnet-4.5; GPT-5 and GPT-5.2; Gemini-3-Pro (Preview); Kimi-K2; and Qwen3-Coder 480B.

⁷https://github.com/kenn-io/agentsview

⁸Tengxiao Liu et al., “Budget-Aware Tool-Use Enables Effective Agent Scaling” (UC Santa Barbara and Google, arXiv:2511.17006, November 2025).

⁹https://www.linuxfoundation.org/press/linux-foundation-announces-the-intent-to-launch-the-tokenomics-foundation-to-establish-open-standards-for-ai-cost-management