AI cost per conversation: tokens, caching and routing | Persiscal

There's a question every founder building an AI product should ask before writing a single line of code: how much will each conversation with a user cost me?

Not "AI is cheap", not generic infrastructure costs. Real numbers: opening your API usage dashboard and knowing exactly how many cents each interaction costs.

Because that's the difference between an AI product that scales and one that eats your margin.

The meter is called a token

Every time your application answers, a meter runs and counts tokens. A token is a piece of text: sometimes a word, sometimes half of one. As a rule of thumb, one token is about 4 characters of English; in Spanish and Portuguese the same message usually takes more tokens.

Everything is billed in tokens: instructions, history, documents, tools, and even what the model "thinks" before answering.

The meter has three rates:

Input: everything the model reads.
Output: everything it writes, including its reasoning. This is the expensive rate: about 5 times the input price.
Cache: repeated content the provider reuses. Reading it costs 5% to 10% of the normal price.

The price list, in context

Anthropic list prices as of November 2026, per million tokens:

Claude Haiku 5.5: USD 0.10 input / USD 0.50 output
Claude Sonnet 5.5: USD 2 / USD 10
Claude Opus 5.5: USD 4 / USD 20

There's a 40x gap between the smallest and the largest model. Other providers have similar ladders. But these numbers say nothing on their own: you have to translate them into cost per conversation.

Anatomy of a conversation

Take a customer support agent, like the ones we build at Persiscal for real businesses:

System prompt: 1,500 tokens (instructions, business context, rules)
5 conversation turns
User message: ~100 tokens per turn
Agent reply: ~300 tokens per turn

The most common mistake is counting the input once. But models remember nothing: the system prompt and the whole history are resent on every turn. Turn 5 pays for the previous 4.

Real input: 1,600 + 2,000 + 2,400 + 2,800 + 3,200 = 12,000 tokens
Output: 5 × 300 = 1,500 tokens

On Sonnet 5.5: (12,000 × USD 2 + 1,500 × USD 10) / 1,000,000 = USD 0.039 per conversation.

Four cents. And that's a simple conversation. At the same prices, the real range looks like this:

Simple query (2-3 turns): ~2,000 input, ~600 output → ~USD 0.01
Average conversation (5-7 turns): ~6,000 / ~2,000 → ~USD 0.03
Long session (10+ turns): ~15,000 / ~5,000 → ~USD 0.08
Session with tools: ~30,000 / ~10,000 → ~USD 0.16
Session with documents: 50,000+ / 15,000+ → USD 0.25 or more

From one cent to a quarter per conversation. Two customers on the same plan can cost you twenty times differently. That spread decides how you need to price your product.

Why the cost spikes

Long conversations: the history is resent on every turn, so cost grows faster than the number of messages.
Agents: calling a tool, reading the result and thinking again means several calls per user request.
Documents: a 50-page PDF can cost more than a hundred short questions.
Retries: if an answer goes wrong, the meter runs twice.

The most profitable lever: prompt caching

If you pay for the full system prompt and history on every turn, you're burning money.

With prompt caching, the provider stores the repeated part and charges a fraction for it: on Sonnet 5.5, a cache read costs USD 0.10 per million versus the normal USD 2, 95% less. The first write costs 25% extra and pays for itself from the second read.

Let's recalculate the 5-turn conversation, caching the system prompt and the history:

Input: from USD 0.024 to ~USD 0.009 (−63%)
Total: from USD 0.039 to ~USD 0.024 (−39%)

Two details that make the difference:

The cache lasts 5 minutes by default. If the user takes longer to reply, it's gone. There's a 1-hour option, with a more expensive write.
Order matters. Stable content goes first (instructions, tools, reference documents) and variable content last. A single changing value at the start, like the current date and time, silently invalidates the whole cache.

Note that, with caching, output becomes more than half the cost. That's why the next lever is writing less.

Not everything needs the biggest model

One of the most common mistakes is using the most powerful model for everything. It's like taking a Ferrari to the grocery store.

In a support agent, most queries (FAQs, status checks, basic navigation) are handled perfectly by a small model. The strategy is routing by complexity:

Tier 1, Haiku 5.5: FAQs, structured answers, classification. The sample conversation costs ~USD 0.002.
Tier 2, Sonnet 5.5: complex conversations, reasoning, tool use. ~USD 0.02 to 0.08.
Tier 3, Opus 5.5: deep analysis and tasks that demand top quality. ~USD 0.08 to 0.50.

If 80% of conversations go to tier 1 and 20% to tier 2, average cost drops by about 75%. Subtract the cost of the classifier that decides where each query goes, and check that perceived quality holds.

The costs that don't show up on your API bill

Knowledge base search (RAG): embeddings, a vector database and the extra tokens of retrieved context.
Tools: every call adds tokens both ways. An agent that queries an API can double its consumption.
Retries and errors: rate limits, timeouts, malformed responses.
Moderation: filtering content before or after the model also costs.
Observability: logs, traces and analytics have their own price.

In our experience, these costs add 20% to 40% on top of the pure API cost.

Is your agent viable? Do this math

Average cost per conversation × conversations per user per month = monthly cost per user. Is it lower than what the user is willing to pay?

A concrete example:

Average cost per conversation: USD 0.03, with caching and routing
Conversations per user per month: 30
Monthly cost per user: USD 0.90

If your cheapest plan is USD 29 a month, AI cost is 3% of revenue. A healthy SaaS.

But if your agent is intensive (long sessions, heavy tool use, large models) and reaches USD 0.25 per conversation with 100 conversations a month, that's USD 25 per user. That changes the business model.

What isn't measured can't be managed

Your provider's bill tells you how much you spent, not on what. To manage tokens you need to measure:

Cost per conversation and per user, not just the monthly total.
Cost per feature: which part of the product consumes the most.
Cost per completed task, not per call. A cheap model that needs three attempts isn't cheap.
Cache ratio: how many input tokens are reused.
Alerts and limits before the budget is exceeded, not after.

The trend is on your side

In less than a year, prices dropped sharply:

Haiku: from USD 1 / 5 (Haiku 4.5, March 2026) to USD 0.10 / 0.50 (Haiku 5.5): 10x cheaper.
Sonnet: from USD 3 / 15 to USD 2 / 10.
Opus: from USD 5 / 25 to USD 4 / 20.

And each generation is more capable than the last. If your numbers are tight but positive today, they'll improve on their own. It's a bet with the wind at your back, as long as your architecture lets you switch models without rewriting the product.

The 8 levers we use at Persiscal

Well-designed prompt caching: −40% to −60% depending on the conversation. It's free, it just needs to be implemented right.
Model routing: up to −75% if most queries are simple.
Short, structured outputs: we ask for exactly what's needed, no filler.
Minimal context: we don't send the whole document when three paragraphs will do. We summarize long histories and clear stale tool results.
Batch processing: −50% for anything that can wait a few hours, like reports or bulk classification.
Tailored reasoning: more effort on complex work, less on simple work.
Limits and alerts per agent, per customer and per day.
Cost per completed task as a product metric, on par with speed and accuracy.

Not theory: this is how we run our products

In Persiscal Agents, every agent shows its cost per conversation in real time, with limits and alerts before it goes over budget.
In Readquire, AI cost is measured per customer and per automation from the very first read.
In ReconcileAI and GEOScan, customers can bring their own API keys and control spend directly.

So, what does it really cost?

The honest answer: between one and twenty-five cents per conversation, depending on complexity, model choice and how well you optimize. A fraction of what an interaction handled by a person costs.

AI isn't expensive. What's expensive is not knowing what it costs.

Building an agent and need to run the numbers? Let's talk: we'll help you measure the real cost of your application and bring it down without losing quality. You can also reach us at @persiscal or @fjbotti.