Prompt caching is one of the most-marketed cost-saving features in the LLM market. Anthropic's published claim — up to 90% cost reduction on cached prompts — is real. The problem: in agentic production workloads, the actual savings are often half of what you think you're getting, and the gap between expected and realized savings is invisible without instrumentation.
This post is for finance and engineering leaders who've signed up for prompt-caching savings and want to understand the trap before it shows up in next quarter's bill.
Anthropic's prompt cache lets you mark a prefix of your prompt as cacheable. On the first call, you pay a write cost (about 1.25× the standard input price) to populate the cache. On every subsequent call within the cache TTL, you pay a read cost (about 0.10× the standard input price) for the cached portion.
For long, repeated context — system prompts, retrieved documents, tool schemas — the math is dramatic. A 50,000-token system prompt that costs $0.15 on a normal call costs about $0.015 on a cached call. Repeat that 100 times in an hour, and you've saved $13.50 per cached prompt × 99 reads = roughly $1,335 from $1,500 down to $165.
That's the marketing pitch. It is correct, in the simple case.
In production agentic workloads — multi-turn conversations, long-running sessions, RAG-heavy prompts, anything where the cached prefix is almost identical between calls — the realized savings collapse.
We analyzed a production session (~175,000 token context) and observed an alternating pattern:
Call 1: cache_write → expensive
Call 2: cache_hit → cheap
Call 3: cache_write → expensive
Call 4: cache_hit → cheap
Call 5: cache_write → expensive
...About half the calls were cache writes. The expected pattern was one write followed by 99 hits. The actual pattern was a write every other call. The realized savings were roughly half of the published expectation.
Three structural causes, all rooted in how the cache interacts with how you're actually building prompts:
Anthropic's prompt cache TTL is 5 minutes. Every five minutes of inactivity expires the cache. In an agentic workflow with a slow tool call, a long retrieval, or any human-in-the-loop pause, the next call becomes a cache write — paying the write premium and losing the read discount.
Computeleap documented this most starkly for Claude Code: the cache TTL was silently reduced from 1 hour to 5 minutes on March 6, 2026. Cache waste — overpayment from re-writes — went from 1.1% in February 2026 to 15-53% in subsequent months, depending on session pattern. Organizations that budgeted on January-February economics were instantly underwater without knowing why.
This is the most common architectural error and the hardest to detect. Many AI gateways and middleware tools include "prompt compression" — automatically pruning long prompts to reduce token count.
Compression and prompt caching are typically adversarial without explicit coordination. A compression layer that prunes the prefix of a prompt invalidates the cache key. Every "compressed" call becomes a cache write, eliminating the savings the cache was supposed to provide.
The right behavior — what we built into Trimio — is cache-aware compression: the compression layer knows which prefix is the cached anchor and refuses to prune below it. The cache stays warm; the compression still trims dynamic content. The savings stack.
The wrong behavior, common in naive implementations, is to compress first and let the cache layer figure it out. The cache never warms. The advertised savings never arrive.
The third trap is more subtle. In one of the production sessions we analyzed, the original token counts the application was reporting were 33× off from the actual billed usage (897 vs ~29,900 tokens for the same request). On heavily-cached requests, the baseline cost calculation could overstate the baseline by 10× — meaning the savings looked great in dashboards even when the actual bill was unchanged.
If your AI cost dashboards rely on application-reported token counts rather than provider-reported billed token counts, your savings story may be largely illusory.
A finance leader doesn't need to understand the implementation. They need to be able to ask three questions of whatever caching/compression stack their organization is running:
Prompt caching is one of the highest-leverage cost levers in the AI stack — when it works. In production agentic workloads, it routinely doesn't work as advertised because the architecture between the application and the provider is fighting the cache rather than coordinating with it.
If your organization signed up for prompt-caching savings and the bill hasn't moved as expected, the answer is almost never "the cache isn't working." It's "the cache is fighting something else in your stack."
The good news: it's solvable. The bad news: most teams don't know they need to solve it until they look hard at the read/write ratio.
Trimio is the LLM API gateway with cache-aware compression — we coordinate with provider-side prompt caching rather than fighting it. The result: teams routinely see the full advertised cache savings, not half of them. See how it works.