The Cheapest Optimization You're Probably Skipping
If your app resends the same system prompt, tool definitions, retrieved documents, or few-shot examples on every call, you're paying full price for tokens the provider has already processed. Prompt caching fixes that — often cutting repeated-input cost by 90–98% with little or no code change. For agents, chatbots, and RAG, it's frequently the second-biggest saving after model choice.
How It Works
Providers cache the stable prefix of your prompt. On the next call, if the beginning is byte-for-byte identical, they serve it from cache at a steep discount instead of re-processing it. The rule that makes or breaks it: put the fixed part first, keep it identical, and put the variable part last.
Cache Discounts by Provider
| Provider | Cache-hit input | Effect |
|---|---|---|
| DeepSeek V4 | ~$0.0028 vs $0.14 | ~98% off |
| Kimi K2.6 | $0.16 vs $0.95 | ~6x (83–87%) |
| Claude | ~10% of input (Opus cached ~$0.50) | ~90% off |
| Gemini | ~10% of input | ~90% off |
| GLM-5.1 | ~$0.26 vs ~$1.40 | large |
DeepSeek's automatic caching is the most aggressive; Claude, Gemini, and Kimi all offer ~85–90%+ on cache hits.
What to Put in the Cached Prefix
- System instructions and role setup
- Tool / function definitions
- Few-shot examples
- Reference documents or a repo map you reuse
- Any long, stable context
Keep the user's variable input (the actual question) at the end, after the stable block.
Common Mistakes
- Reordering the prefix. Any change to the beginning breaks the cache — even a timestamp or a shuffled tool list.
- Putting variable data early. A dynamic date or user ID near the top invalidates everything after it.
- Tiny prefixes. Caching helps most when the stable block is large relative to the variable part.
- Ignoring TTL. Caches expire; very infrequent calls may miss. Batch related calls close together.
Worked Example
An agent with a 20K-token system prompt + repo map, called 15 times per task, on DeepSeek V4 Flash:
- Without caching: 15 × 20K × $0.14/1M ≈ $0.042 per task on the prefix alone.
- With ~98% cache: ≈ $0.001 per task.
Across thousands of tasks a month, that's the difference between a real bill and a rounding error. Combine it with the routing and batching in how to reduce AI API costs for compounding savings, and see the agent math in how much it costs to run an AI agent.
Doing It Across Models
Every major model supports caching, but the mechanics differ. Through one gateway (LinkModel) you keep one request shape across providers, so the "stable prefix first" discipline applies uniformly as you route between models.
Bottom Line
Structure prompts as [stable prefix] + [variable input], keep the prefix identical, and you'll cut repeated-input cost 85–98% on every major provider. It's the highest-ROI change most LLM apps can make.
Start free with a $1 credit and measure your cache hit rate.