How prompt caching works, and how to get cache hits in AI apps and agents

In short

Prompt caching lets an AI provider reuse the work it already did on the beginning of your prompt, so repeated requests that share that beginning cost less and start faster. It only works on a prefix: everything up to the first byte that differs. To benefit, put stable content first, mark where it ends, keep it identical between calls, and check the usage fields in the response to confirm you are getting hits.

This guide uses Anthropic's documented behaviour as the worked example, because it is explicit about prices and rules. Other providers cache differently, so check their docs before assuming the same numbers.

Why it matters now

Agent-style applications resend a growing conversation on every model call. A session that makes 16 calls sends the system prompt, tool definitions and earlier turns 16 times. In a 19 October 2026 post, Cloudflare reported that this resending dominated the bill for its internal AI code reviewer: about 43% of its spend was cache writes and 34% cache reads, with output tokens only about 23%. After it fixed how its prompts were cached and made other changes, it said the average review cost fell 68%, from $2.43 to $0.77. We covered the details in Cloudflare cuts its AI code review cost 68%. The lesson below is the general one behind it.

Step 1: Understand what a cache prefix is

A language model reads your prompt token by token. Processing a long prompt takes compute, and providers charge for it. If a later request starts with exactly the same tokens, the provider can skip that work and reuse its stored result for the shared part.

Anthropic's documentation says the cache references the whole prompt in a fixed order: tools, then system, then messages, up to and including the block you mark with cache_control. A change at one level invalidates that level and every level after it. That ordering is the reason the rules below exist.

Two consequences follow:

• Only the beginning can be reused. A difference in the middle means everything after it is processed again.
• "Identical" means identical. A timestamp, a working-directory path or a reordered JSON key at the top of the prompt breaks reuse for everything below it.

Step 2: Know what it costs

The documentation gives these multipliers relative to the base input price:

• A write to the default five-minute cache costs 1.25 times the base input price.
• A write to the one-hour cache costs 2 times.
• A cache read costs 0.1 times, with lower read multipliers listed for some newer models (0.05 times for Claude Opus 5.5 and Sonnet 5.5, 0.025 times for Fable 5.1 and Mythos 5.1, per the page when we read it).

The arithmetic: writing costs a little more than normal input, reading costs a small fraction. At a 0.1 read price, a cached prefix that is read nine times after one write costs 1.25 + 9 × 0.1 = 2.15 units, against 10 units uncached, so about 78% less for that part of the prompt. If it is written and never read, you paid 25% extra. Caching helps only when the same prefix really recurs.

The default lifetime is five minutes, refreshed at no extra cost each time the entry is read. The lifetime counts from the start of the request that wrote or read it. For longer gaps you can request a one-hour lifetime with "ttl": "1h", at the higher write price.

There is also a minimum size. Prompts shorter than the model's minimum are not cached and no error is returned. The documentation's examples range from 512 to 4,096 tokens depending on the model, so check the table for the model you use.

Step 3: Mark the breakpoint

Cache writes happen at breakpoints. You add a cache_control field of type ephemeral to a content block, and the prefix up to and including that block becomes cacheable. You can define up to four explicit breakpoints, and on each request the system looks back through up to 20 positions per breakpoint to find an earlier entry.

There are two ways to place them:

• Automatic caching: one top-level cache_control field; the system puts the breakpoint on the last cacheable block and moves it forward as the conversation grows. The docs recommend it as the starting point for multi-turn chat. It uses one of the four breakpoint slots.
• Explicit breakpoints: cache_control on individual blocks, useful when sections change at different rates, such as tool definitions that rarely change and context that updates daily. The two can be combined.

The most important rule in the best-practice list is where to put the marker: on the last block that stays identical across requests. For a prompt with a static prefix and a varying suffix (timestamps, per-request context, the incoming message), the breakpoint goes at the end of the prefix, not on the varying block.

Step 4: Order your prompt for reuse

Put everything stable first and everything that varies last:

1. Tool definitions, in a fixed order, with fixed descriptions.
2. The system prompt and standing instructions.
3. Large reference material that many requests share.
4. Per-session or per-request details: dates, working directories, user input.

Cloudflare's post is a real example of getting this wrong. Its reviewer's system prompt put the agent instructions first, then an environment section with the working directory and date, and placed the only breakpoint after both. The shared instructions, between 11,000 and 21,000 tokens identical in every repository, were never stored on their own, so a review in one repository could not reuse what another had written. The first request of a session found a cached prefix only 14% to 26% of the time. Cloudflare split the block at a marker so the shared half got its own breakpoint, removed repository-specific content from in front of it, and says that took the cost per review from $2.47 to $2.02.

Step 5: Keep the things that silently invalidate the cache constant

Anthropic's documentation lists settings that invalidate parts of the cache when they change between calls:

• Tool definitions (names, descriptions, parameters) invalidate the tools, system and messages caches.
• tool_choice and adding or removing images anywhere in the prompt invalidate the tools and system caches.
• Changes to the web search toggle, the citations toggle or the speed setting invalidate the tools cache.
• Changes to thinking parameters or to the effort setting always invalidate the message blocks; the effect on tools and system depends on the model.

The documentation also warns to keep JSON key ordering stable in tool_use blocks, because some languages randomise key order and break the cache.

The practical advice: pick one configuration per workflow and keep it. If you want to compare effort levels, be aware each variant maintains its own cache.

Step 6: Handle parallel requests

The docs state that for concurrent requests, "a cache entry only becomes available after the first response begins." If you launch three calls at once with the same prefix, none of them can read what another is still writing; all three pay the write price.

Cloudflare ran into this when three reviewers started at the same time with the same first request. Its fix was to make the second and third wait, up to 20 seconds, for the first response to begin, so one writes the cache and two read it. It reports this saves about nine cents per reviewer and slows them slightly. Whether that trade is right depends on your latency budget; the principle is to send one request first when you need parallel hits.

Step 7: Measure, don't assume

The response reports usage in three fields:

• cache_creation_input_tokens: tokens written to the cache.
• cache_read_input_tokens: tokens read from the cache.
• input_tokens: tokens after the last breakpoint that were not cached.

Total input is the sum of the three. If both cache fields are zero, the prompt was not cached, most likely because it was below the minimum length.

Log these for every call, and compute the read share of the first request in each session, not just the average across all calls. Averages hide the problem: Cloudflare's overall hit rate was high (85.7% in April and 97% by September), yet the first request of each session was missing the shared prefix most of the time. Its improvement came from recording the first call's token usage and a hash of the first system block exactly as sent, which showed the prefix was changing.

A checklist

1. Is the stable content at the start, in a fixed order?
2. Is there a breakpoint at the end of the stable part, not on the varying part?
3. Is the prompt above the model's minimum cacheable length?
4. Are tools, tool_choice, images and thinking or effort settings constant across the calls you expect to share a cache?
5. Do calls arrive within the five-minute lifetime, or have you requested the one-hour option?
6. For parallel calls, does one go first?
7. Are you logging the cache fields and checking the first call of each session?

Common mistakes

• Putting the date or a session identifier at the top. It looks harmless but invalidates everything below it.
• Marking the last block, which changes every turn. The breakpoint belongs at the end of the stable prefix.
• Assuming a good average means good hits. Check the first request of each session.
• Caching prompts that are never reused. A write costs more than normal input, so a prefix read zero times is a loss.
• Changing tool lists per request. A different tool set is a different prefix.

Limits of this guide

Prices and minimums change, and the documentation lists them per model, so treat the multipliers above as a snapshot of the page as we read it on 20 October 2026. The Cloudflare figures are the company's own measurements from one workload. Other providers, including their automatic caching and their pricing, work differently; read their documentation for details.

Sources

• Anthropic documentation: Prompt caching
• Cloudflare: Cutting our inference bill by 68% by being obsessive about optimization (19 October 2026)
• audiotale.net: Cloudflare cuts its AI code review cost 68%