Why an AI agent costs more with every turn

An AI agent that runs for 60 turns does not cost twice as much as one that runs for 30. It can cost three times as much. Each turn sends the whole conversation back to the model, so the bill grows with the length of the conversation times the number of calls. Caching cuts that cost sharply. It does not remove the growth.

Two widely read posts made this point well. Philip Zeyliger's Expensively Quadratic on the exe.dev blog charted it from real agent traffic. Sam Rose's prompt caching explainer on the ngrok blog showed what a cache stores. This guide puts their findings next to Anthropic's published prices and documentation, and turns them into steps an operator can take this week.

Why the bill grows faster than the work

A model keeps no memory between calls. An agent that reads a file, runs a search, then writes a reply makes three calls. Each call sends the instructions, the tool definitions and every earlier turn again. Turn 30 carries the text of turns 1 to 29.

Zeyliger sampled 250 conversations from exe.dev's own gateway. In one example conversation that cost about $12.93, cache reads made up 87% of the total by the end. Cache reads passed half of that conversation's cost at 27,500 tokens (exe.dev).

It isn't really the number of tokens squared; it's the number of tokens times the number of calls.

What a cache stores

A prompt cache does not store answers. Rose explains that the provider keeps the model's internal working state for the start of a prompt, so it does not recompute that part on the next call (ngrok). The answer is still generated fresh each time.

The cache works on the beginning of the prompt only. Anthropic's documentation says a cache hit needs the prompt to be identical, character for character, up to the point you marked. The order is fixed: tool definitions first, then the system prompt, then the messages (Anthropic). One changed character early in the prompt means everything after it is billed at the full rate.

The numbers on one agent

Here is Emeri's arithmetic for a sample agent on Claude Sonnet 5.5, using Anthropic's published rates (Anthropic). Uncached input costs $2 per million tokens. Writing to the five-minute cache costs $2.50. Reading from the cache costs $0.10. Output costs $10. The sample agent starts with 20,000 tokens of instructions and tool definitions. Each turn adds 2,000 tokens of tool results and replies, and writes a 500-token response.

• 30 turns, no cache: 1.47 million input tokens. About $3.09 in total.
• 30 turns, cached: about $0.48. Cache reads $0.14, cache writes $0.20, output $0.15.
• 60 turns, no cache: 4.74 million input tokens. About $9.78 in total, 3.2 times the 30-turn bill for twice the turns.
• 60 turns, cached: about $1.11. Cache reads are now the largest item, at $0.46.

These figures assume every turn hits the cache. Real agents miss some turns. Anthropic's own measurements found that caching cut agent-loop cost by a factor of 2.7 to 5.3 (Anthropic). Emeri did not run this agent; the figures above are arithmetic from published prices.

Step 1: turn on caching and keep the start of the prompt fixed

On the Claude API, one top-level request field turns on automatic caching: cache_control, with its type set to ephemeral. The API then places the cache marker on the last cacheable block and moves it forward as the conversation grows (Anthropic). Prompts shorter than 512 tokens on the current Claude models are not cached, and the API returns no error when that happens.

Then keep the start of the prompt identical between calls. The usual cause of misses is a value that changes on every request, placed near the top. Anthropic's documentation lists the changes that reset the cache (Anthropic):

• Any change to a tool definition resets everything after it.
• Any change to the system prompt resets the system prompt and the messages. A timestamp or request ID in the system prompt does this on every call.
• Changing the thinking setting or the effort level resets the cached messages.
• Switching between fast and standard speed resets the system prompt and the messages.

Step 2: check the cache is working

Every response reports three input counts: cache_read_input_tokens, cache_creation_input_tokens and input_tokens. Their sum is the total input (Anthropic). If both cache counts are zero, the prompt was not cached.

Step 3: keep conversations short

Caching makes each re-read cheaper. Fewer re-reads are cheaper still. Zeyliger's suggestions follow from the arithmetic (exe.dev):

• Start a new conversation when a task is done. Restating the context can cost less than carrying a long history.
• Send bulky side work, such as a broad search, to a separate sub-agent. Only its short result enters the main conversation.
• Read a large file in one call, not in five pieces. Five calls means five re-reads of everything before them.

Step 4: batch the work nobody is waiting for

The Message Batches API charges half price on every token, input and output. The discount stacks with caching (Anthropic). A batch holds up to 100,000 requests or 256 MB. Most batches finish within an hour, and a batch expires if it does not finish within 24 hours (Anthropic). Overnight reports, document backfills and evaluation runs fit this. Anything a customer is waiting on does not.

Step 5: set effort on purpose

The effort setting (output_config.effort) controls how much a Claude model thinks and writes. The levels are low, medium, high, xhigh and max. Claude Opus 5.5 and Claude Haiku 5.5 default to medium. Most other models default to high (Anthropic). Setting it explicitly avoids a surprise when you change models.

Lower effort is a trade, not a free saving. On Anthropic's research benchmarks, low effort gave up 1 to 3 points of accuracy for a third to a half off the cost per task (Anthropic). Test it on a sample of your own tasks before you change the default. Pick one level at the start of a conversation and keep it: changing it partway resets the cache.

What to do this week

1. Find the agent or workflow with the largest bill.
2. Turn on caching, and move any timestamps or IDs out of the system prompt.
3. Log the three input counts and check the cache-read share after a day.
4. Move one overnight job to the Batches API.
5. Test one lower effort level on 20 real tasks before changing anything else.

Caching lowers the price of each call. Choosing which model takes each call is the next step: see Send easy work to a cheap model, and escalate on a check.

Emeri's certification grading runs on Claude models.