A 90% cache hit ratio does not remove 90% of all traffic.

First decide what the cache can answer

A cache hit ratio needs a denominator. Does it cover all requests, all reads, or only a small group of cacheable reads? Those definitions produce different capacity estimates. For this example, assume a read-through cache that can answer eligible reads. Every write still reaches the database. Each read miss produces one database read. We ignore invalidation traffic, retries and background refreshes for the first calculation.

Suppose an API receives 1,000 requests per second. Eight hundred are reads and two hundred are writes. If 90% of reads hit the cache, 720 reads finish there. Eighty reads miss and continue downstream. All 200 writes also continue. The backend receives 280 requests per second.

Use a formula with named assumptions

read_rate = total_rate × read_fraction
write_rate = total_rate × (1 - read_fraction)
read_misses = read_rate × (1 - read_hit_ratio)
backend_rate = read_misses + write_rate

Example: 1,000 × 0.8 × 0.1 + 1,000 × 0.2 = 280

The resulting total reduction is 72%. A 90% read hit ratio is therefore compatible with a 72% reduction in total downstream traffic. Neither number is misleading when its definition is stated clearly. Problems arise when the denominator disappears from the conversation.

Try the same hit ratio with a write-heavy API: 20% reads and 80% writes. The backend receives 20 read misses and 800 writes, or 820 requests per second. Buying a larger cache will not fix an overloaded writer if the write path is the real bottleneck.

Throughput is only part of the design

A hit can be fast while the remaining misses are painfully slow. Measure hit latency, miss latency and database latency separately. A blended mean can hide a poor tail experience. The formula also assumes one miss causes one backend operation; applications that fan out to several tables or services need a richer model.

Check correctness as carefully as speed. Can a user tolerate stale data? Which events invalidate a key? Is tenant identity part of the key? A cache that serves another tenant's response is a security failure regardless of its hit ratio. Do not cache authenticated responses globally without a deliberately reviewed key and policy.

Test the uncomfortable conditions

Test a cold cache after deployment, the loss of a cache node, a traffic burst and many concurrent requests for one expired popular key. Those cases can produce much more backend work than the normal average. Request coalescing, bounded refresh work and capacity headroom may help, but choose them based on the actual workload rather than adding every caching pattern at once.

In a teaching simulator, keep the incoming rate fixed, change only the read hit ratio, and inspect database traffic. Then change the read fraction. The two experiments explain why hit ratio alone cannot predict application capacity. A model is useful when it exposes its assumptions and gives you a measurement to collect next.