Prompt caching explained (and when it saves money)

Caching sounds like a switch that makes every request cheaper. It's pickier than that. The repeated part of your prompt has to be big, stay exactly the same, sit right at the start, and get reused often enough to count. When those line up, though, the savings can be large.

What caching actually does

When you send a prompt, the provider can store the front part of it. The next time you send something that starts with the exact same text, it reuses that stored work and charges you far less for it - often a tenth to a quarter of the normal price - instead of processing it from scratch again. There's one catch worth knowing: storing it the first time can cost a touch more than a normal request, so caching only comes out ahead once you've reused it enough to cover that. That's the point where it starts paying off.

It pays off in the same situation every time: a big chunk of text reused a lot. Agents and coding tools re-send the same instructions or context call after call, so they're the obvious winners. To get lots of reuse, put the unchanging text first and keep it identical down to the last character - even a changing date at the top is enough to miss. Put the user's actual question after it. The caching savings calculator shows your cost with and without it, and the point where it starts to pay.

Order matters

Caching doesn't hunt through your whole prompt for similar sentences - it matches the unchanging text at the start. So put the long-lived stuff first: instructions, tool definitions, examples, reference material. Put the question and anything that changes at the end.

A timestamp, a request ID or a shuffled list near the top makes two otherwise identical prompts look different to the cache. Tiny details like that are behind most disappointing results.

Where caching fits naturally

A coding tool sends the same project guidance and tools every time. A support bot carries the same policy instructions. An agent uses the same set of tools across many steps. All of them have a big chunk that repeats - perfect.

A one-off document summary is the opposite. If the document is unique and you only use the prompt once, there's nothing to reuse. (Saving the finished answer for next time is a separate trick that helps when the exact same document comes back - but caching itself can't reuse what was never repeated.)

Keep an eye on the reuse rate

Don't guess a reuse rate once and forget it. Track how much of your text is cacheable, how much actually got reused, and what each request really costs - broken down by feature. A healthy overall number can hide one feature that keeps changing its text and never caches.

Watch speed too. Reusing stored text often makes replies faster, but it varies by provider. Measure both cost and speed from your real usage, not a spreadsheet.

The provider's rules matter

Providers differ on the smallest chunk worth caching, what it costs to store and to read, whether caching is automatic or you turn it on, how long it lasts, and which models support it at all. These rules change, so check the current docs before you treat an estimate as a real budget. In the calculator, try a cautious case as well as a best case - if the whole thing only works at near-perfect reuse, the plan is fragile.

A simple rollout

Pick one busy feature with a big repeated chunk.
Make that chunk identical every time and push the changing bits to the end.
Turn caching on and record how often it's reused.
Compare cost and speed against the old version over a full week of traffic.
Roll it out wider only once the measured saving is worth the extra fuss.

Caching works best when it follows the natural shape of your product. Don't twist every prompt into an awkward template just to chase an impressive reuse chart.