Caching Introduction: Why, Where & What to Cache
What caching is, the layers of caching in a web system, hit ratio, TTLs, and how to decide what is worth caching.
What is caching?
A cache stores a copy of data somewhere faster to reach than the original source, so repeated requests don't redo expensive work. Reading from memory takes microseconds; a database query might take milliseconds, and a call to another service even longer.
Caching is one of the most powerful tools for making backends fast and cheap to scale — and one of the easiest ways to introduce stale-data bugs if you're careless.
Where caches live in a web system
Browser cache → CDN (edge) → Reverse proxy → Application cache (Redis/in-memory) → Database cache → Disk
| Layer | Example | Caches |
|---|---|---|
| Browser | HTTP cache | Static files, API responses with cache headers |
| CDN | Cloudflare, CloudFront, Fastly | Images, CSS/JS, public pages, close to users |
| Reverse proxy | Nginx, Varnish | Whole HTTP responses |
| Application | Redis, Memcached, in-process dict | Query results, computed values, sessions |
| Database | Buffer pool, query cache | Hot rows and index pages |
The closer to the user a cache sits, the more work it saves — but the harder it is to invalidate.
Key terms
- Hit — the data was in the cache. Miss — it wasn't, so we went to the source.
- Hit ratio — hits ÷ total lookups. 95% means only 1 in 20 requests reaches the database.
- TTL (time to live) — how long an entry stays valid before it expires.
- Eviction — removing entries when the cache is full (LRU, LFU, FIFO).
- Stale data — a cached value that no longer matches the source.
Why caching helps so much
Suppose a product page query takes 50 ms and a Redis lookup takes 1 ms. With a 90% hit ratio:
average latency = 0.9 × 1 ms + 0.1 × 50 ms = 5.9 ms (≈ 8× faster)
database load = 10% of requests (10× fewer queries)
The second number often matters more: the cache protects your database from traffic spikes.
What is worth caching?
Good candidates:
- Data that is read much more often than written (product catalogues, course lists, user profiles).
- Expensive computations (reports, recommendations, rendered pages).
- Slow external API responses.
- Data where slight staleness is acceptable (view counts, trending lists).
Poor candidates:
- Data that must be exactly current (bank balances during a transfer, stock counts at checkout).
- Data that is rarely read twice.
- Highly personalised data with low reuse (unless cached per user deliberately).
Choosing a TTL
| Data | Typical TTL |
|---|---|
| Static assets with versioned file names | 1 year (never changes) |
| Public catalogue pages | Minutes to hours |
| User profile | Minutes, plus invalidation on update |
| Trending / counters | Seconds to minutes |
| Prices, stock levels | Seconds — or invalidate on every change |
A TTL is your safety net: even if you forget to invalidate, data eventually refreshes.
Local vs distributed caches
| In-process cache | Distributed cache (Redis) | |
|---|---|---|
| Speed | Fastest (no network) | ~sub-millisecond network hop |
| Shared between servers | No — each server has its own copy | Yes |
| Survives restarts | No | Optional (persistence) |
| Best for | Small, rarely changing data | Shared data across many app servers |
Many systems use both: a tiny in-memory cache in front of Redis in front of the database.
Common mistakes
- Caching without a TTL, so mistakes stay wrong forever.
- Caching errors or empty results by accident — or never caching "not found" and getting hammered by repeated misses.
- Caching per-user data under a shared key, leaking one user's data to another.
- Adding a cache before measuring — optimise the slow query or add an index first.
Key takeaways
- Caches trade freshness for speed and reduced load.
- Measure hit ratio; even 80–90% can cut database load dramatically.
- Cache read-heavy, expensive, staleness-tolerant data, always with a TTL.
- Caches exist at every layer, from the browser to the database.