Cache Invalidation: Stale Data, TTLs, and Herds

Why this matters

Your product catalog service caches item prices in Redis with a 5-minute TTL. A merchant updates the price of their best-selling item from $29.99 to$19.99 — a flash sale. For the next 4 minutes and 47 seconds, some users see the old price, some see the new one. A customer buys at $29.99, sees their friend bought at$19.99 two seconds earlier, and opens a support ticket.

This is cache invalidation: the problem of ensuring that cached data reflects the current state of the source of truth. Phil Karlton famously called it one of the two hardest problems in computer science (the other being naming things). He wasn’t joking.

The difficulty isn’t in the concept — it’s in the combinatorial explosion of edge cases. What happens when a cache key expires and 10,000 concurrent requests all try to rebuild it simultaneously? What happens when a write-behind cache crashes before flushing dirty entries to the database?

What happens when two application instances write to the same key with a race condition between cache and DB updates? Every caching strategy is a tradeoff between consistency, latency, and availability.

Practical note Cache bugs are usually timing bugs wearing a performance costume. Before choosing a pattern, decide what stale data would cost you.

The mental model

There are four primary caching strategies, each with fundamentally different invalidation semantics:

Cache-Aside (Lazy Loading): The application manages the cache explicitly. On read: check cache → if miss, read from DB → populate cache. On write: update DB → invalidate (delete) cache entry. The next read triggers a fresh cache fill. This is the most common pattern and gives the application full control, but it means the first reader after every write eats a cache miss.

Write-Through: Every write goes to both cache and database synchronously. The cache is always consistent with the DB at the cost of higher write latency — every write now has two round trips. Read performance is excellent because the cache is always warm after a write.

Write-Behind (Write-Back): Writes go to the cache first, and dirty entries are flushed to the database asynchronously on a configurable interval. This gives the fastest write latency (just a cache write), but if the cache crashes before flushing, you lose data. The interactive demo models this with a configurable flush interval (1–30 seconds) — longer intervals batch more writes but risk more data loss.

Read-Through: The cache layer itself handles DB fetches on miss — the application only talks to the cache. This simplifies application code but couples your cache layer to your data access logic. Combined with write-through, it provides a fully transparent caching layer.

The interactive demo uses a Zipf distribution over a configurable key space to model realistic access patterns — a few hot keys get the majority of requests, just like production workloads. This skew is what makes cache stampedes so devastating.

Decisions that matter

TTL (1–300 seconds): Time-to-live for cache entries. At 5 seconds, you’ll see the cache hit ratio drop to 60–70% as entries constantly expire and need refilling. At 300 seconds, hit ratio approaches 95%+ but stale reads can persist for 5 minutes after a DB update. Production sweet spot depends on your consistency requirements — session data might tolerate 30-minute TTL, while inventory counts need 5-second TTL.

Cache Size (10–500 entries): Maximum entries before LRU eviction kicks in. When the working set exceeds cache capacity, every new entry evicts an old one. If your Zipf skew is low (many equally popular items), a small cache causes constant churn. Redis tracks this via evicted_keys in INFO stats.

Write-Behind Flush Interval (1–30 seconds): How often dirty cache entries are flushed to the database. At 1 second, you get near-real-time consistency but almost the same DB write load as write-through.

At 30 seconds, you batch heavily — reducing DB writes by 10–50× through deduplication (latest version per key wins) — but a crash costs you up to 30 seconds of unflushed writes.

Read/Write Ratio (10–99% reads): Shifts between read-heavy and write-heavy workloads. Read-heavy workloads (90% reads) benefit enormously from caching — hit ratios stay high. Write-heavy workloads (30% reads) constantly invalidate or update cache entries, reducing the effectiveness of any caching strategy.

Key Skew / Zipf Exponent: Controls how concentrated access is on hot keys. At high skew, 5% of keys receive 80% of requests — caching is extremely effective. At low skew, access is spread evenly across all keys — cache hit ratio drops because the working set exceeds cache capacity.

Where it breaks: Stale Data from Race Conditions

The failure mode: Two application servers process requests concurrently. Server A reads user.balance = $100 from cache. Server B updates user.balance = $150 in DB and invalidates the cache.

Server A’s request finishes processing and writes the stale $100 value back to the cache (a lagging cache-aside populate). The cache now shows $100 while the DB has$150. This stale entry persists until TTL expires.

What you will notice: Users report seeing outdated data that “flickers” — correct after a hard refresh, wrong again on the next page load. redis-cli GET user:balance returns a different value than SELECT balance FROM users.

The staleness appears intermittently and correlates with high concurrent write rates on the same keys. In the interactive demo, you can observe stale reads by watching the version comparison: cache entry version trails behind the current DB version.

Warning Cache-aside with concurrent writers is inherently prone to stale data. If two servers read and write to the same key, the slower server can overwrite the cache with outdated values. Use versioned keys or write-through to prevent this.

What I would do: Use versioned cache keys — include a version number or timestamp in the key itself. When writing to DB, increment the version and invalidate the old key:

# Versioned cache invalidation
def update_balance(user_id, new_balance):
    # 1. Update DB and get new version
    version = db.execute(
        "UPDATE users SET balance = %s, version = version + 1 "
        "WHERE id = %s RETURNING version", 
        (new_balance, user_id)
    )
    
    # 2. Write to cache with version in key
    cache_key = f"user:{user_id}:balance:v{version}"
    redis.setex(cache_key, 300, new_balance)
    
    # 3. Update version pointer
    redis.set(f"user:{user_id}:balance:latest", version)

Alternatively, use write-through strategy where every write atomically updates both cache and DB — eliminating the window for race conditions.

Where it breaks: Thundering Herd on Cache Invalidation

The failure mode: Your most popular product page is cached with a 60-second TTL. When the TTL expires, 3,000 concurrent requests all experience a cache miss simultaneously. All 3,000 requests hit the database to rebuild the same cache entry.

The database connection pool (200 connections) is exhausted instantly. The query that normally takes 5ms now takes 8 seconds under contention. Downstream services waiting for product data start timing out, cascading the failure.

What you will notice: Redis INFO stats shows keyspace_misses spiking periodically (every TTL interval). PostgreSQL pg_stat_activity shows hundreds of identical queries running simultaneously.

Database CPU spikes to 100% every 60 seconds like clockwork. Application error rates show periodic sawtooth patterns aligned with TTL expiry.

What I would do: Implement request coalescing (also called single-flight) — only the first request on a cache miss actually queries the database. All other concurrent requests wait for that single DB query to complete:

import asyncio
from functools import lru_cache

# Single-flight cache rebuilding
_locks = {}

async def get_product(product_id):
    cache_key = f"product:{product_id}"
    
    # Try cache first
    cached = await redis.get(cache_key)
    if cached:
        return json.loads(cached)
    
    # Acquire per-key lock to prevent thundering herd
    if cache_key not in _locks:
        _locks[cache_key] = asyncio.Lock()
    
    async with _locks[cache_key]:
        # Double-check after acquiring lock
        cached = await redis.get(cache_key)
        if cached:
            return json.loads(cached)
        
        # Only one request hits DB
        product = await db.fetch_product(product_id)
        await redis.setex(cache_key, 60, json.dumps(product))
        return product

Hint The interactive demo’s “Cache Stampede” preset helps reproduce this scenario. Set a short TTL combined with high read rate on hot keys to trigger the thundering herd effect and observe the difference with request coalescing enabled.

Where it breaks: Write-Behind Data Loss on Cache Crash

The failure mode: Your write-behind cache has a 15-second flush interval. During those 15 seconds, 200 write operations update cache entries that haven’t been flushed to the database yet. Redis crashes (OOM kill, hardware failure, or network partition).

All 200 dirty entries are lost. The database still has the old values. When Redis restarts with an empty cache, the next reads populate from DB with stale data — and no one knows those 200 writes ever happened.

What you will notice: After a Redis restart, users report data “reverting” to old values. Business metrics show a gap — orders, balance changes, or profile updates from the last N seconds are missing.

Redis INFO persistence shows rdb_last_bgsave_status:err or no AOF file. The application logs show no errors because from the app’s perspective, the writes succeeded (they went to cache successfully).

What I would do: Never use pure write-behind for data you can’t afford to lose. Use write-through with async invalidation as a hybrid — write to DB first (synchronous), then update cache (synchronous), then publish an invalidation event for other cache nodes (async):

def update_user_profile(user_id, data):
    # 1. Write to DB FIRST (source of truth)
    db.execute(
        "UPDATE users SET name=%s, email=%s WHERE id=%s",
        (data['name'], data['email'], user_id)
    )
    
    # 2. Update local cache (write-through)
    cache_key = f"user:{user_id}"
    redis.setex(cache_key, 300, json.dumps(data))
    
    # 3. Publish invalidation event for other nodes (async)
    redis.publish("cache:invalidate", cache_key)

If you must use write-behind for performance, enable Redis AOF persistence with appendfsync everysec — you’ll lose at most 1 second of writes on crash instead of the full flush interval.

# redis.conf - minimize write-behind data loss
appendonly yes
appendfsync everysec
no-appendfsync-on-rewrite no

The tradeoff

Strategy	Consistency	Read Latency	Write Latency	Data Loss Risk
Cache-Aside	Eventual (TTL window)	Cache hit: ~1ms, miss: ~50ms	Low (DB only)	None (DB is source of truth)
Write-Through	Strong (immediate)	Always ~1ms (always warm)	High (cache + DB sync)	None
Write-Behind	Eventual (flush interval)	Always ~1ms	Very low (~0.5ms, cache only)	High (unflushed dirty entries)
Read-Through	Eventual (TTL window)	Cache hit: ~1ms, miss: ~50ms	Depends on write strategy	None
Short TTL (5s)	Near-real-time	More frequent misses (~70% hit)	N/A	None
Long TTL (300s)	Up to 5 minutes stale	High hit ratio (~95%)	N/A	None

Hint Most production systems use cache-aside because it’s simple, safe, and flexible. Start with a 30–60 second TTL and add request coalescing for hot keys. Only move to write-through or write-behind when you have specific latency or consistency requirements.

A practical checklist

Step 1: Measure your cache hit ratio. In Redis, run INFO stats and calculate keyspace_hits / (keyspace_hits + keyspace_misses). Below 80% for a read-heavy workload indicates your cache size is too small, TTL is too short, or your key space is too large. Target 90%+ for production read-heavy services.

Step 2: Identify hot keys. Use redis-cli --hotkeys (requires LFU maxmemory policy) or analyze access logs. If 5% of keys handle 80% of traffic, those keys are your stampede risk. Implement request coalescing and consider pre-warming them with longer TTLs.

Step 3: Monitor staleness. Track the time between a DB write and the corresponding cache update. For cache-aside, this is the TTL duration. For write-through, it should be near-zero. Set up Prometheus metrics comparing cache_version vs db_version per key to quantify consistency lag.

Step 4: Stress-test cache failure. Use the interactive demo’s “Cache Crash” trigger to simulate total cache loss and observe cold-start recovery behavior. In production, use Redis Sentinel or Cluster for high availability. Test failover regularly.

Step 5: Choose strategy based on workload. Read-heavy with tolerance for staleness → cache-aside with 30–60s TTL. Read-heavy with strict consistency → write-through. Write-heavy with performance requirements → write-behind with AOF persistence. Most production systems use cache-aside because it’s simple, safe, and flexible.

Try it yourself

Cache invalidation bugs are notoriously hard to reproduce in development because they depend on timing, concurrency, and specific access patterns. The interactive demo lets you trigger stampedes on demand, crash the cache mid-flush, and watch stale reads accumulate — all with real-time metrics showing hit ratio, staleness, and DB load.

Try the interactive Cache Invalidation demo to compare write-through, write-behind, and cache-aside strategies side-by-side, trigger thundering herds, and see exactly why this problem earned its reputation.