The hidden economics of agentic AI
Why the cost of enterprise AI depends on how work is executed

A service team pilots an AI agent on 500 cases. The agent reads each email, looks up the customer, searches the knowledge base, drafts a reply and updates the record. The pilot works, and the month's model bill comes to a few hundred dollars.
Then the agent goes live on the whole queue: 100,000 cases a month. Built as a single general-purpose agent on a frontier model, the worked example later in this article puts that at about $78,600 a month in model charges alone.

The volume was always going to grow. What surprises teams is the cost of each case. In the worked example, a routine request takes six model calls, and the queue averages seven once complex cases, rework and retries are counted. Every call carries the same instructions, and many reason at length before they answer. None of that shows on a price list.
Model prices set the rate. How the work is executed decides how often you pay it, how much context you send each time, and whether a step needed a model at all.
What one case costs in model calls
A simple question to a chat assistant may take one model call. An agent works in a loop. In the worked example, a routine case takes six turns: read the case and request the customer record; read the record and request a knowledge search; read the results and request the entitlement and SLA details; read those; draft the reply; then update the record and notify the team. In a design where the agent decides each action, each turn is a separate model call.
Each call sends the agent's instructions and tool definitions again, plus what it has seen so far. In the worked example, a general-purpose agent with 25 tools carries 9,000 tokens of instructions and tool definitions. Across the 6 calls of a routine case, that is 54,000 tokens of repeated prefix before caching discounts. The case history grows with every turn, so later calls are larger than early ones.

Anthropic reported the same pattern in its own research system (June 2025): its agents used about 4 times the tokens of chat interactions, and its multi-agent system about 15 times. The figures come from one system, but the direction is general. More turns and more handoffs mean more calls and more context.
We see it in customer conversations too. Teams that process email point out that each reply carries the whole thread, so the same message can be read many times as a case develops. And a single unusually long email can cost many times the average case.
Three billing details then multiply the effect.
- Output costs more than input. On every model in the table below, output tokens cost 5 times input tokens.
- Reasoning is billed as output. OpenAI's reasoning guide says reasoning tokens are not visible through the API but are billed as output. Anthropic's adaptive thinking documentation says you are billed for the full thinking process, not the summary you see. Both providers let you set how much effort a model spends reasoning, so it is a design choice.
- Caching discounts repeated context, at a price. Context that repeats across calls can be cached and read back for 2.5% to 10% of the normal input rate, depending on the model. Writing to the cache costs 1.25 times the input rate on the default short cache at both providers (at OpenAI, for GPT-5.6 and later models). It pays only when the same context is read again before the cache expires.
The spread between models is wide.

Example models: small, GPT-6 Luna and Claude Haiku 5.5; mid-tier, GPT-6.1 Sol and Claude Sonnet 5.5; frontier, GPT-6 Astra and Claude Fable 5.1. USD per 1 million tokens, standard processing. Sources: OpenAI API pricing and Claude API pricing, checked 17 November 2026. Cached-input rates do not scale the same way: at OpenAI, frontier cached input costs 10 times the mid-tier rate, so designs that lean on caching scale by more than the headline 5 times. Full rates and other tiers are in the technical appendix.
A 100-fold range makes model choice look like the main decision. It is an important one, but it is not the only one, and it is not the first.
The same service process, built different ways
Take a Salesforce service operation handling 100,000 cases a month, a quarter of them complex. We modelled the model charges for the same process built in different ways, at OpenAI's standard rates.
Architecture A is one general-purpose agent. It has 25 tools, retrieves the full record with its activity history, and takes 6 turns per case (8 on complex cases). It caches its instructions and earlier turns. 4% of cases fail a rule-based check and are processed again in full.
Architecture B is an orchestrated process. A small model classifies the case and extracts its fields. A function retrieves the 12 record fields the next step needs. A rule decides the path. A mid-tier model drafts routine replies. A frontier model investigates complex cases, with 5 tools across 3 calls. A small model checks every draft, routine and complex, against explicit policy rules. Functions update the record and notify the team. 8% of routine drafts fail the check and are escalated to the complex path.
Both include 3% technical retries.

Illustrative estimate of model inference charges only, not customer data. The single agent is Architecture A and the orchestrated process is Architecture B. Model cost per case, in the order shown: $0.79, $0.14, $0.13 and $0.03. Token counts are assumptions. Calls per case include rework and retries. Per-turn token budgets, cache-hit rates, formulas and sensitivity are in the technical appendix.
Four things stand out.
In this example, design does most of the work. Replacing agent turns with functions and trimming each step's context cuts about $58,000 a month, before any model changes. Moving routine work to smaller models cuts another $10,100. The checks and escalations that the cheaper path needs add $2,400 back. The split depends on the order the changes are applied in, so treat it as an illustration of scale, not a precise attribution.
Frontier spend concentrates where the hard cases are. In Architecture B, the frontier path takes 94% of the model spend while handling 31% of cases. Whether that path needs a frontier model at all is the next question. If a mid-tier model handles the investigations to the required standard, the bill falls to about $3,100.
The orchestrated design is not always cheaper. Rebuilt on a mid-tier model, the single agent costs $14,400, about 10% more than Architecture B with a frontier model on its complex path. That comparison changes the design and the strongest model at the same time. It only makes sense where the single agent's broad context needs no frontier model but B's investigations still do. In that case, if more than about 13% of routine drafts were escalated, the single agent would be cheaper. If a mid-tier model is good enough everywhere, B on a mid-tier model ($3,100) stays cheaper at any escalation rate. The orchestrated design wins clearly when some work needs a frontier model and most does not.
Caching matters in both. Without it, Architecture A would cost about $199,800 a month, because every turn would pay full price for the instructions and history it resends.
Several limits apply to all of this. Architecture A is a deliberately broad design. A well-engineered single agent can also bound its context, limit its tools and call conventional functions.
Architecture B assumes a small model can check drafts written by stronger models. That is realistic for explicit rules, such as required fields or commitments a reply must not make. Whether it catches subtler errors in a stronger model's draft has to be tested. The check itself is cheap whichever model does it: a mid-tier checker adds about $400 a month, a frontier checker about $2,400. What matters is how accurately it escalates.
The model also assumes both designs reach the same quality after rework and escalation, and it counts model charges only. It leaves out Ortoo's fee, engineering, evaluation and human review. Where two configurations sit within about $1,500 a month of each other, as the mid-tier single agent and Architecture B do, those costs and the quality of the result decide, not the model bill. How software vendors price AI on top of model charges is covered in AI pricing should be simple to understand and granular to control.
What moved the number
Fewer model calls
Many steps in a business process do not need a language model. Retrieving a Salesforce record, applying a routing rule, checking that required fields are filled in and updating a status are done faster by conventional software, with the same result for the same inputs and no model inference.
We covered when a step should use AI and when it should use a rule in AI where it helps. Rules when required. The economic point is simple. Every model call that a rule or a function can replace is a call you do not pay for, wait for or have to check. A function still has an execution cost. It just has no inference cost.
Less context per call
Every token a model reads is paid for on every call that carries it, at the cached rate or the full one. The usual sources of excess are easy to spot.
- Broad retrieval. A tool that returns a full record with its activity history can send 8,000 tokens where 12 fields and 600 tokens would do.
- Tool sprawl. An agent with access to 25 tools carries all 25 definitions into every call.
- Accumulated history. Agent loops resend the conversation on each turn. Between separate steps or models, where the cache does not carry over, passing extracted fields or a short structured summary costs less than the transcript. Within a loop on the same model, the opposite can hold: keeping the conversation append-only preserves cache hits, while writing a summary costs output tokens and changes the prefix the cache depends on.
- Repeated threads. When a new reply arrives as a new step, reading only the new part of an email thread, plus the fields already extracted, costs less than reading the whole thread again.
Context has a quality cost too. A step that lacks the information it needs makes worse decisions. The aim is the information each step needs to produce a reliable result, not everything the system happens to know.
The right model for each step
A service process contains work of very different difficulty. Classifying a request, extracting an account number and summarising a short email are within a small model's reach. Investigating an unusual fault or judging a policy exception is where a frontier model earns its rate.
The teams we work with tend to arrive at this on their own. They start a step on the cheapest model that might do the job, move up only when it fails, and keep stronger models for steps with many instructions or many conditions. One team tested the newest frontier model on a complex review step, liked the results, and decided the cost per case was not justified. Choices are also bounded by governance: many enterprises approve one or two providers, and model selection happens inside that list.
Assigning a model to each step at design time is different from letting a router choose at runtime, and the evidence on runtime routing depends on how it is done. In one LangChain experiment on a coding agent, a router chose a model tier once at the start of each thread. Across 973 threads it cut median cost per thread by 64% and mean cost by 42%, with no measurable change in merged pull requests. In a benchmark of NVIDIA's Switchyard router, every task started on a cheaper model and a judge model reviewed each turn, escalating the task to a frontier model when it went off track. Cost per run fell by 74%, but accuracy fell from 86% to 80%, the cost per run varied by 67% across test rounds, and the judge took 21% of the routed spend.
Architecture B is not immune to the same effect. Its classifier and its policy check are runtime decisions, and the frontier path takes most of its spend, so a higher escalation rate raises the bill: from about $12,900 at 8% to $16,400 at 20%. The difference is that each escalation rule is designed and tested in advance, and its rate can be measured and reviewed. Providers also retire models, so the choice for each step needs a periodic review.
Aiven, the Finnish data infrastructure company, applies the same principle to its employees' own AI use. Its VP of strategy, Ville Lehto, told The Register that open-weight models do most of the work and frontier models are kept for planning the most critical tasks. Aiven reported a roughly 80% fall in spend among its heavy users. That is employee tooling rather than process automation, but the principle carries over.
Where orchestration fits
Anthropic's guide to building effective agents separates workflows, where models and tools follow predefined code paths, from agents, which direct their own process and tool use. It notes that agentic systems often trade latency and cost for better task performance, and recommends the simplest solution that works. The worked example is that trade, priced.
A model gateway helps decide which model answers a request, and it can enforce budgets on that traffic. Orchestration works one level up. It decides which steps of a business process need a model request at all, what each step receives, and what happens when a step fails. The two are complementary, but only the second can remove a call.
This is how Ortoo Orchestrator is designed. A business process, such as a service case from intake to resolution, is built from specialised agents and functions inside Salesforce. Each AI step can use its own model, from providers such as OpenAI, Anthropic or Google, through the customer's own API keys. Functions handle the steps that need no model. Because the keys are the customer's, the model bill sits in their own provider account, where they can see it.
Cost per successful outcome
The unit that matters is not the token or the call. It is the completed piece of work, at the quality the business needs.

The numerator includes failed attempts. The denominator counts only work that was completed to standard. Cost per attempt flatters a design; cost per success counts the retries, rework and checking that turned attempts into finished work. Researchers evaluating language models have proposed the same idea as cost-of-pass: the expected monetary cost of generating a correct solution.
The model bill is often not the largest part. For a banking customer service agent, McKinsey puts token costs at 20 to 25% of variable run costs, and human oversight at 70 to 75%. In agentic software engineering, research cited by McKinsey found that about 60% of an agentic task's cost was tied to refining answers. Where people review AI output, the design of exceptions and review thresholds can matter more than the token rate. A configuration that halves the model bill and doubles escalations to people has saved nothing.
Customers frame it the same way. The teams we talk to justify the AI line by comparing it with the cost of people doing the same work, and review it monthly. What frustrates them is when they cannot see it at that level. Usage reported per day, or held by a central IT team, does not tell an operations lead what one case costs or what one test cost.
So measurement comes before optimisation. McKinsey's guidance is to instrument cost, latency and outcomes at the level of the workflow and the agent step, and to review model choices every quarter. This practice is increasingly called AI FinOps. It needs two kinds of measure.
| Business measures | Diagnostic measures |
|---|---|
| Cost per successfully handled case | Model calls per case |
| Resolution rate and quality | Reasoning tokens per step |
| Human intervention rate | Cache-hit rate |
| Cost variability, including the most expensive 5% of cases | Retry and escalation rates |
| SLA achievement | Context consumed per step |
Leaders review the left column. Engineering teams use the right one to explain it.
Execution control is economic control
If every model provider cut its prices by 90% tomorrow, most of this would still apply. Teams would still need to decide where AI is used, which model each step needs, how much context it reads, and what happens when it is wrong. They would still need to know what a completed case costs.
The economics of agentic AI are not settled when a company negotiates model rates. They are shaped earlier, when it decides how work will be handled: where interpretation is needed, where a rule is enough, and who deals with the exceptions.
Spend expensive tokens only where cheaper intelligence, or no AI at all, cannot achieve the same reliable outcome.
See how Ortoo Orchestrator works
Sources
- OpenAI, OpenAI API pricing, developer documentation, accessed 17 November 2026
- Anthropic, Claude API pricing, developer documentation, accessed 17 November 2026
- OpenAI, Reasoning models guide, developer documentation, accessed 12 November 2026
- Anthropic, Adaptive thinking, developer documentation, accessed 12 November 2026
- OpenAI, Prompt caching guide, developer documentation, accessed 17 November 2026
- Anthropic, How we built our multi-agent research system, engineering blog, June 2025
- Erik Schluntz and Barry Zhang, Anthropic, Building effective agents, 19 December 2024
- Mehmet Hamza Erol, Batu El, Mirac Suzgun, Mert Yuksekgonul and James Zou, Cost-of-pass: an economic framework for evaluating language models, arXiv:2504.13359, April 2025
- McKinsey & Company, Where AI agents pay off: a practical guide to the economics of agentic workflows, 24 August 2026
- McKinsey Quarterly, Is that AI agent worth it? Agentic economics and the modern operating model, 13 July 2026
- McKinsey & Company, Cost versus value: managing agentic AI system performance, 8 July 2026
- LangChain, How to build a model router in the harness, 1 October 2026
- LangChain, Switchyard agent routing benchmark, 11 August 2026
- Lindsay Clark, The Register, Euro open source specialist proposes LLM gateway to ease token bills, 26 October 2026
Related insights

Introducing Corrections in Ortoo Orchestrator
When an AI agent puts a support request in the wrong category, your team knows what needs fixing. Corrections lets you explain the right outcome, send it for approval and update the guidance the agent uses for future requests.

When to use an AI agent in Salesforce (and when to use a rule)
"Can this be an agent?" comes up in every Salesforce meeting now. How to decide when an agent earns its place, plan for the cases it can't finish, and get the handoff right.

Governed AI in production: more capability needs more control as the OpenAI Hugging Face Hack reminds us
AI agent governance is about what an AI agent can and is allowed to do in production. Capability and ability get confused constantly, and a security incident in July 2026 showed why the difference matters – also in your Salesforce org.
READY TO SEE IT IN ACTION
Map your workflows with our team.
30 minutes, no prep needed. We will map one workflow you handle today and identify where orchestration would change the outcome.