Caching Strategies
Storing a fast copy of expensive-to-compute or expensive-to-fetch data — write policies, eviction, and the invalidation problem.
Why cache
A cache stores a copy of data that's expensive to compute or fetch (a database query, an API call) in a fast-access layer (in-memory, e.g. Redis) so repeat requests skip the expensive path.
Write policies
- Write-through — write to cache and the underlying store at the same time; slower writes, but the cache is never stale.
- Write-back — write to cache immediately, flush to the underlying store later (asynchronously); faster writes, but risks data loss if the cache fails before flushing.
- Cache-aside (lazy loading) — the application checks the cache first, and on a miss, reads from the store and populates the cache; the most common pattern for read-heavy workloads.
Eviction policies
When the cache is full, something has to go: LRU (evict the least-recently-used entry) is the most common default, since recently-used data is likelier to be used again soon.
The hard part: invalidation
"There are only two hard things in computer science: cache invalidation and naming things." Deciding when a cached value becomes stale and must be refreshed or evicted — especially under concurrent writes — is where most caching bugs live, not in the caching mechanism itself.
Prerequisite
Assumes the vocabulary from Scalability Fundamentals (a cache is one specific technique for handling more load).