What does it cost to run an AI system in production? Seven budget lines

What it costs to run an AI system in production: the token math with official prices dated November 2026, and the seven budget lines a pilot never carried.

By Published 6 min read

What it costs to run an AI system in production: the model bill is one line of seven, next to retrieval and data, evaluation, human review, security review, monitoring and records, and ownership.

Running an AI system in production costs more than the model bill. In the example below, the model costs about 10,200 dollars for 30 days, and checking just 5 percent of the answers by hand costs about 40,000 dollars over the same period, at an assumed 40 dollars an hour. The model is the line everyone prices. It is one of seven.

The cost of running an AI system in production is the model bill plus six lines a pilot never carried: retrieval and data, evaluation, human review, security review, monitoring and records, and ownership.

This article is for the person who has to approve the budget for a pilot that now has to run every day. It shows the token arithmetic with the provider’s official prices, names the other six lines, and gives you a sheet to fill in before the budget is signed. OrNsoft is an advanced AI engineering firm, founded in Florida in 2009, that brings client AI systems from concept or pilot to production.

Key takeaways

  • The model bill is one of seven lines. Tokens are easy to price. The other six are mostly people and process.
  • Volume is the first multiplier. The same price per request gives a bill about 100 times larger at 100 times the volume.
  • Caching and batching move the model line only. They do not touch review, evaluation or ownership.
  • Every line needs an owner and a unit. A line with no owner has no budget in practice.

What does the model itself cost?

Language models are priced per million tokens. A token is a small piece of text, often part of a word, and it is what the provider counts and bills. Input and output tokens are priced separately. As of November 12, 2026, Anthropic’s official pricing page lists Claude Sonnet 5.5 at 2 dollars per million input tokens and 10 dollars per million output tokens. We use it as the worked example because its pricing is published in full. Other providers price differently, and prices change, so redo the sums with yours.

The assumptions below are ours, chosen to be plain, not measured on a client. Change them to match your system.

Pilot Production
Requests per day 200 20,000
Input tokens per request (instructions plus retrieved context) 6,000 6,000
Output tokens per request 500 500
Cost per request $0.017 $0.017
Cost per day $3.40 $340
Cost for 30 days $102 $10,200

The cost per request is the same: 6,000 input tokens at 2 dollars per million is 1.2 cents, and 500 output tokens at 10 dollars per million is half a cent. Only the volume changed. A pilot budget of about 100 dollars a month says nothing about the production bill.

Model bill for 30 days in the worked example: a pilot at 200 requests a day costs 102 dollars, production at 20,000 requests a day costs 10,200 dollars, and with prompt caching on a stable prefix it costs 5,640 dollars.

How much can the model line come down?

Two levers are in the provider’s own price list, and your builder will know them. Both change the model line only.

For your builder: caching and batching

Prompt caching lets the provider reuse a prompt prefix that does not change from one request to the next. For Sonnet 5.5, a cache read is priced at 0.05 times the base input price, and writing the cache costs 1.25 times the base input price for the five-minute option. The Batch API gives a 50 percent discount on both input and output tokens for work that does not need an immediate answer.

In our example, suppose 4,000 of the 6,000 input tokens are a stable prefix, such as instructions and reference text, and every request reads it from the cache.

  • Cost per request: 4,000 cached tokens at 0.10 dollars per million, plus 2,000 uncached input tokens at 2 dollars per million, plus the 500 output tokens: $0.0094.
  • For 20,000 requests a day over 30 days: about $5,640, 45 percent less than $10,200.
  • This ignores cache writes, which cost 1.25 times the base input price whenever the cache is filled or has expired. Your real figure sits between the two.
  • If the same work can wait, the Batch API halves the uncached bill: $5,100 instead of $10,200.

For a buyer, the point is simple: in our example the model line falls by up to about half, and none of that saving reaches the other six lines.

What are the six lines a pilot never carried?

The pilot ran with a handful of experts who forgave its mistakes. Production runs with everyone, and each of these lines appears the day it does.

  1. Retrieval and data. The search index or database that feeds the model, the refresh of its content, and the connections to the systems it reads. Unit: cost per source system per month, plus the work to keep the content current. Ask your platform for volume tiers, because they differ by vendor.
  2. Evaluation. The test set you score before every change, and the people who write and maintain it. Re-running 500 test cases through the example system costs about $8.50 in tokens, so the tokens are cheap. Unit: hours a week to keep the set current, and one full run per prompt or model change. See how to evaluate AI agents.
  3. Human review. The answers a person checks before they act. This is the line most budgets skip. At a 5 percent review rate, the example sends 1,000 of its 20,000 daily answers to a person. At 2 minutes each, that is 2,000 minutes, about 33 hours a day. Unit: review rate, minutes per case, and who does it. At an assumed 40 dollars an hour (replace it with your loaded cost), that is about $1,333 a day, or about $40,000 over 30 days, about four times the model bill. Gates like these are covered in approval gates for AI actions.
  4. Security review. A reviewer outside the build team reads the access and data risks, and the ways a user could trick the system into ignoring its instructions, and reads them again when the system changes. Unit: reviews per year, hours per review.
  5. Monitoring and records. Alerts for health and cost, plus a record of each request detailed enough to investigate an error. Storage grows with volume. Unit: cost per million requests stored, and the person on call. See LLM observability vs monitoring.
  6. Ownership and change. A named owner, a runbook, and a plan for the day the model changes. The pricing page itself lists models marked as retired, so a budget that assumes one model forever is already wrong. Unit: owner hours a month, and one model migration a year as a planning figure.

Which production word does each line belong to?

Production asks four things of a system: Integrated, Secure, Governed and Accountable. The business owner and the technical owner sign a production definition with acceptance criteria, described in how to define production for an AI system. Each budget line serves one of those four words, which gives you a quick test: if a word has no line, nobody has costed it.

The seven budget lines mapped to the four production requirements: integrated has model usage and retrieval and data, secure has security review, governed has evaluation, human review and monitoring and records, accountable has ownership and change.

Read it from the other side. A pilot usually carries only the model line and part of the data line. The other five are where a production budget grows, and they are the lines a go-live review asks about.

What should you do before the budget is approved?

  1. Write four numbers. Requests per day, tokens in and out per request, the share of answers a person reviews, and minutes per review.
  2. Price each line with its unit. Use the six lines above as the sheet. A line without a unit is not priced. In our example the two lines we could size come to about $50,000 over 30 days, $10,200 of model and $40,000 of review, before retrieval, evaluation, security, monitoring and ownership are added from your own quotes.
  3. Name an owner per line. One person, not a team.
  4. Run the model line twice. Once with your real prompt, once with caching and batching where they apply.
  5. Add a change line. One model migration a year, and the evaluation run that goes with it.
  6. Redo the sheet when volume doubles or the model changes. Prices and volumes move, so the sheet is dated, and so is the budget.

If you are costing a pilot that has to run every day and want a second pair of eyes on the sheet, bring us the problem, not the specification.

Sources

  • Anthropic, Pricing, model prices, prompt caching multipliers and Batch API discount, read November 12, 2026.
  • Worked example: assumptions and arithmetic by OrNsoft, not a client measurement.

Questions

How do you estimate the cost of running an LLM application?
Multiply requests per day by the tokens in and out of each request, price them at your model's input and output rates, then add the lines that are not tokens: retrieval, evaluation, human review, security review, monitoring and ownership. Write the assumptions down, because every figure moves when one of them moves.
Why is a pilot cheaper to run than the production system?
Volume. A pilot answers a few hundred requests a day for a few people. Production answers every request the operation sends, so the same price per request becomes a bill about a hundred times larger in our example. The review, evaluation and monitoring work arrives at the same time.
What is the biggest cost of an AI system in production?
It depends on the system, and the sheet in this article is how you find out. In our worked example, reviewing 5 percent of the answers by hand takes about 33 hours every day, about 1,333 dollars a day at an assumed 40 dollars an hour, while the model bill is about 340 dollars a day. Check your own review rate before you assume the model is the large line.
How can you lower LLM costs without lowering quality?
Two levers are in the provider's own price list: cached prompt prefixes are read at a fraction of the normal input price, and batch processing is discounted by half for work that can wait. Both change the model line only. Test quality on your evaluation set before and after.
Do these prices apply to every provider?
No. The structure is common, input and output priced separately with caching and batch options, but the rates differ by provider and change often. Use the arithmetic here with your provider's current price list.

Bring us the problem.

Book a call