Prompt Cache Hit Rate Is Not Your Cost Saving

A higher prompt-cache hit rate does not establish a lower bill—or a cheaper successful task. First normalize the provider’s usage fields, then price each token category once, then compare workloads with the same acceptance criteria. Output, retries, tools, and cache writes can change the result.

DigitalOcean’s announcement, updated 23 October 2026, advertises model-dependent cached-input discounts of up to 90%. That is a discount on eligible input, not a promise about your entire application bill. Its performance figures describe its own testing, not a benchmark we reproduced. Announcement

This guide is for engineers deciding whether a cache-related change actually improved an AI feature. It is not a provider ranking. The documentation below is a 29 October 2026 snapshot; rates and fields depend on the model, API, host, and selected cache lifetime.

Normalize usage before calculating a percentage

Use four conceptual buckets: ordinary input, cache writes, cache reads, and output. Do not infer these buckets from similarly named JSON fields across APIs.

API scope	Accounting distinction to preserve
OpenAI Responses, models supporting prompt_cache_options	input_tokens includes reads and writes. Subtract input_tokens_details.cached_tokens and input_tokens_details.cache_write_tokens to obtain ordinary input. Lifetime-specific write counts partition the write total; they are not extra tokens. Guide
Direct Claude Messages API	input_tokens excludes cache creation and reads. Add cache_creation_input_tokens and cache_read_input_tokens to obtain total input. Documentation
DigitalOcean-hosted models	The documented example includes cached tokens in prompt_tokens. cache_read_input_tokens and prompt_tokens_details.cached_tokens report the same cached subset; do not add them. Its Anthropic-router examples use a different breakdown. Documentation

Build an adapter for the exact endpoint and provider route you use. Preserve the original aggregate counters alongside your normalized fields, without retaining prompt contents or secrets merely for cost analysis. If a field is absent, investigate the schema instead of silently treating missing telemetry as a measured zero.

For a defined period, calculate token cache-hit rate = total cache-read input tokens / total input tokens. Sum the numerator and denominator before dividing. Averaging request percentages gives a tiny request the same weight as a large one. Separately label any request-level measure, such as the fraction of calls with at least one cached token.

Figure 1. Original conceptual review path. A token-reuse metric is an input to the decision, not the decision itself.

Cache writes are not universally free

OpenAI’s current guide distinguishes GPT-5.6-and-later behavior from earlier models: the documented default 30-minute write rate is 1.25 times ordinary input; the optional 12-hour rate in the Responses API is 2.5 times. These are total rates, not extra surcharges. Read rates vary by model. Guide

Claude documents 1.25-times writes for five-minute caching and 2-times writes for one-hour caching, with model-specific read-rate exceptions. Do not assume a quoted standard read multiplier applies to every current model. Documentation

DigitalOcean says cache writes are free for its hosted models, but documents premium-priced writes for supported Anthropic models through its router. The host name alone does not determine the billing rule. Documentation

Our accounting rule: multiply each non-overlapping bucket by its applicable total rate. Separate write lifetimes when their rates differ. Add output and other billed charges afterward. A provider’s unit-price table and invoice remain authoritative for the chosen service; this worksheet is not an invoice substitute.

Worked example: 80% reuse, 34.75% total saving

The following is a fictional workload, not a price quote or measured provider result. Assume one million input tokens in a fixed set of accepted tasks, unchanged output, no other charges, and these invented rates:

Bucket	Tokens	Price per million tokens	Cost
Ordinary input	100,000	$2.00	$0.20
Cache writes	100,000	$2.50	$0.25
Cache reads	800,000	$0.20	$0.16
Output	200,000	$10.00	$2.00
Total	—	—	$2.61

The token cache-hit rate is 80%. Processing the same input entirely at the assumed ordinary rate would cost $2.00; input with caching costs $0.61, a 69.5% input saving. Including the unchanged $2.00 output cost, the comparison is $4.00 versus $2.61: 34.75% lower, not 80% or 90%.

Now suppose a changed workflow generates another 200,000 output tokens while achieving exactly the same accepted results. At the assumed output rate, it costs $4.61, which is 15.25% more than the original $4.00 baseline—even with the same 80% input reuse. This is arithmetic, not evidence that caching causes longer answers. The point is to measure the whole change rather than attribute every cost movement to the cache.

Decide using accepted tasks, not just requests

Before a rollout, write down what counts as a successful task: for example, a response that passes a fixed factuality rubric and requires no correction. Then use a comparison worksheet:

• Comparable workload: task mix, input lengths, model version, service tier, and acceptance rubric. Record any changes rather than attributing their effects to caching.
• Complete cost: all attempts, including retries and failures, plus cache writes, output, tools, and other billed components. Divide by accepted tasks; with zero accepted tasks, the ratio is undefined, not zero.
• Separate timing measures: time to first token and end-to-end task completion. A faster first token is not proof that the whole task finished faster.
• Reuse conditions: warm and cold starts, pauses between requests, prompt revisions, and realistic concurrency. A repeated warm prompt is not representative of every production session.

OpenAI’s September announcement introduces cache diagnostics and a dashboard for investigating missed reuse. These help explain why input was not reused; they do not independently establish a better task outcome. Announcement

If your change preserves quality and reduces complete cost per accepted task across a representative sample, that supports adopting it for that workload. If reuse rises but total cost also rises, inspect the token mix and extra attempts before celebrating or abandoning caching. If the sample is too small or the task mix changed, keep the result provisional.

Finally, cheaper execution does not establish that an agent should receive broader permissions. For a repository-writing workflow, review safe outputs and write authority separately from its token economics.