Choosing Between Caching and Batching by Bottleneck
Engineers often treat prompt caching and batch APIs as interchangeable, but they optimize different halves of the token equation. Prompt caching targets repeated input prefixes (system rules, tool definitions, few-shot examples, static reference manuals) while keeping latency real-time. Batch APIs halve both input AND output billing for workloads where no human sits waiting on the response.
| Model | Standard Input / 1M | Cache Read / 1M | Async Batch SLA |
|---|---|---|---|
| Claude Fable 5.1 | $10.00 | $0.25 | Yes (50% off) |
| GPT-6 Astra | $10.00 | $1.00 | Yes (50% off) |
| GPT-6.1 Sol | $2.00 | $0.20 | Yes (50% off) |
| Claude Sonnet 5.5 | $2.00 | $0.20 | Yes (50% off) |
| Gemini 3.8 Flash | $0.75 | $0.075 | Yes (50% off) |
| DeepSeek Flash | $0.15 | $0.015 | Not offered |
Catalog cache-read rates vs Batch API support across major tiers
Common Cache-Breaking Mistakes to Avoid
Reordering tool schemas dynamically, inserting per-request timestamps at line 1 of the system prompt, or switching reasoning effort mid-conversation resets prefix hashes and triggers fresh cache-write charges. Keeping static context pinned first unlocks predictable steady-state savings.