Day 6 — Cache Strategies Deep Dive

Goal: Go beyond the "which write strategy" from Day 5 into how real FAANG-scale caches actually work. By the end you should be able to: - Design cache invalidation at fleet scale using CDC (Change Data Capture) patterns - Reason about multi-region caches and cache consistency across geographies - Choose refresh patterns (refresh-ahead, background refresh, stale-while-revalidate) - Design tiered caches (L1 in-process + L2 distributed) for the best latency + cost trade-off - Read the classic cache case studies (Facebook Memcache, Netflix EVCache) and know what they got right - Instrument caches for observability (hit rate, latency, eviction, hot keys) - Avoid the top cache anti-patterns

Estimated time: 50–60 min read + reflection.

1. Recap and go beyond

Day 5 gave you the fundamentals — cache placement (6-layer cake), 4 write strategies (cache-aside / read-through / write-through / write-back), eviction (LRU/LFU/TTL), stampede prevention (single-flight), and hot keys.

Today we zoom into the harder parts of cache operations at scale: 1. Fleet-wide invalidation without race conditions 2. Multi-region cache consistency 3. Refresh patterns to preempt cache misses 4. Tiered caches for the best latency-per-dollar 5. Real systems: Memcache / EVCache 6. Observability — what to measure 7. Anti-patterns — what to never do

2. Fleet-wide invalidation — the CDC pattern

The problem. You have a Postgres primary, 5 replicas, 100 app servers, a distributed cache with 20 Redis nodes, an internal CDN, and a browser cache. When someone updates user:42's profile, how does every one of those layers know to invalidate the stale value?

Naive answer: "the app that wrote the DB also deletes the cache key." Fine for one process. Doesn't scale when: - Writes come from many services (batch jobs, admin tools, third-party webhooks). - You want to invalidate multiple derived caches (user:42 profile, user:42:friends, feed:for:42). - You need cross-region invalidation.

Change Data Capture (CDC) — the FAANG-scale pattern

Rather than each writer sending invalidations, the DB itself emits a stream of every change. Downstream consumers subscribe and invalidate.

    Postgres primary                         Kafka                                  Consumers
    ────────────────────                     ─────                                  ─────────
    ┌──────────────┐    WAL / logical repl   ┌───────┐    fan-out to consumers    ┌───────────────┐
    │  writes...   │─────────────────────────▶│ topic │─────────────────────────▶│ cache-inval    │
    │              │      (via Debezium /     │       │                            │ consumer       │
    │              │       Debezium-style)    └───────┘                            └───────────────┘
    └──────────────┘                              │                                ┌───────────────┐
                                                  ├───────────────────────────────▶│ search-index   │
                                                  │                                │ consumer       │
                                                  └───────────────────────────────▶│ analytics ETL  │
                                                                                   └───────────────┘

Debezium (or AWS DMS, GCP Datastream) reads the DB's WAL (write-ahead log) or binlog.
Emits every INSERT/UPDATE/DELETE as an event to Kafka (or another log).
Downstream consumers subscribe and react:
Cache invalidator deletes the affected keys from Redis.
Search-index consumer updates Elasticsearch.
Analytics consumer writes to a warehouse.
Feed builder recomputes derived data.

Why it works so well: - Atomicity: every DB write is guaranteed to appear in the stream. No race. - Decoupling: writers don't need to know about caches, search, or feeds. - Replay: if a consumer bugs out, you can rewind Kafka and reprocess. - Multi-target: the same event triggers many downstream updates.

Trade-offs: - Adds latency to invalidation (~10–100 ms typical). - Adds an entire streaming stack you have to operate. - Non-trivial to get exactly-once semantics right (Day 21 will cover).

3. Multi-region caches

Once your users span continents, you have two choices for how caches behave across regions:

3.1 Cache-per-region (independent)

Each region has its own Redis / Memcached cluster. No cross-region cache traffic. Simplest.

Pros: minimal latency (all cache hits are in-region, <1 ms); no cross-region bandwidth; blast radius contained (one region's cache outage doesn't affect others).
Cons: cache is warmed independently per region; less efficient (you cache the same thing 5×); cross-region invalidation still requires coordination.

When to use: most read-heavy apps. This is the default.

3.2 Geo-replicated cache

The cache itself replicates across regions (Redis Cluster with cross-region replication, or Aerospike / DynamoDB Global Tables + DAX).

Pros: consistent view; a write in us-east eventually shows up in eu-west.
Cons: replication lag (~100 ms cross-region best case); more complex; cross-region bandwidth cost.

When to use: when a stale cache in another region would be user-visible and problematic (e.g. financial pricing, inventory that must be strictly consistent globally).

3.3 The classic pattern: per-region cache + CDC-based invalidation

       us-east                                              eu-west
    ┌─────────────────┐                                 ┌─────────────────┐
    │ Postgres primary│                                 │ Read replica   │
    └────────┬────────┘                                 └────────┬────────┘
             │ WAL                                                │
             ▼                                                    ▼
    ┌─────────────────┐          global Kafka topic     ┌─────────────────┐
    │ CDC (Debezium)  │═══════════════════════════════▶│ CDC consumer    │
    └────────┬────────┘                                 └────────┬────────┘
             ▼                                                    ▼
      ┌───────────┐                                          ┌───────────┐
      │ us-east   │                                          │ eu-west   │
      │ Redis     │  (in-region only)                        │ Redis     │
      └───────────┘                                          └───────────┘

Writes go to the primary; CDC propagates events globally; each region's cache invalidator listens and evicts the affected keys locally. Reads are always in-region and fast.

4. Refresh patterns — preempt the miss

The problem: even with a hot cache, keys expire → the next read is a slow miss → user sees latency spike. Can we refresh before the key expires?

4.1 Refresh-ahead

When a cache read notices the key is close to expiry (say, within 10% of its TTL), it serves the value AND fires an async refresh in the background.

Great for hot keys with predictable access.
Only refreshes keys that are actually read (no wasted work on cold keys).
Standard in Caffeine (Java), some Redis client libraries.

4.2 Background refresh (scheduled)

A separate worker periodically refreshes hot keys unconditionally (before their TTL expires).

Good for a small set of always-hot keys (e.g. feature flags, top-level catalog).
Wasteful for large key spaces.

4.3 Stale-while-revalidate (HTTP / CDN)

The Cache-Control: stale-while-revalidate=30 header (RFC 5861) tells caches: "if the entry is within 30 s past its TTL, serve it stale AND fetch a fresh copy in the background."

Great for HTTP / CDN caches. Absorbs bursts of traffic on freshly-expired keys.
Users always get a fast response; the freshness bound is soft.

4.4 Comparison

Pattern	Trigger	Serves stale?	Best for
Refresh-ahead	Read close to TTL	No (has fresh)	Cache-level, hot reads
Background refresh	Scheduler	No	Small fixed hot set
Stale-while-revalidate	Read after TTL	Yes	CDN, HTTP, tolerable staleness

5. Tiered caches — the L1 + L2 pattern

Adding an in-process cache in front of a distributed cache is one of the highest-leverage optimizations in a real system.

                                Read path:
   ┌───────────────┐            1. Check L1 (in-process). Hit? Return in ~500 ns.
   │  App process  │            2. Miss? Check L2 (Redis). Hit? Return in ~1 ms.
   │               │            3. Miss? Query DB. Return in ~50 ms.
   │  ┌─────────┐  │
   │  │ L1 cache│  │  ← in-process (Caffeine, Node lru-cache, Python functools.lru_cache)
   │  │ ~500 ns │  │
   │  └─────────┘  │
   └───────┬───────┘
           │
           ▼
   ┌───────────────┐
   │ L2: Redis     │  ← distributed cache, shared across app fleet
   │ ~1 ms         │
   └───────┬───────┘
           │
           ▼
   ┌───────────────┐
   │ Postgres      │
   │ ~50 ms        │
   └───────────────┘

Why the L1 tier matters

L1 hit is ~2000× faster than L2. For very hot keys, this dwarfs the L2 speedup.
L1 has no network hop. Cheaper on the network layer; the cache node isn't a bottleneck.
L1 is per-process — no shared load, no contention.

The tricky part: L1 invalidation

Each app process has its own L1 → invalidating across N processes is coordination-heavy. Options: - Short TTL on L1 (say, 10 s). Accept staleness for cheapness. Most-common in practice. - Pub/sub invalidation — a Redis channel broadcasts invalidations; each app subscribes and drops the L1 key. Extra complexity. - Version keys — store user:42:v in L2; L1 checks the version on every access (cheap; still 1 network call, but avoids DB).

Rule of thumb: L1 tier makes sense when the top-1% of keys account for >50% of traffic (heavy skew).

6. Real case studies

6.1 Facebook Memcache (2013 paper — go read it, seriously)

Facebook runs the world's largest deployment of Memcached. Key insights:

Cache-aside with delete-on-write (not write-through).
Leases to prevent stampedes: when a cache miss happens, the cache returns a special "lease token." Only the holder of the lease is allowed to write back — everyone else must wait or re-fetch.
Separate cache pools for different workloads (news feed, chat, ads) so a bad workload doesn't evict good data.
Warm-up region for new clusters: a new cluster pulls values from an existing warm cluster instead of directly from the DB.
Regional cache with a shared canonical region (the "master" region) that CDC-invalidates to secondaries.
Insight: most user requests are read-heavy (~99%), so cache is life-or-death.

6.2 Netflix EVCache

Netflix built EVCache on top of Memcached with:

Multi-region replication built in (the cache itself, not just CDC).
Client-side sharding (no cluster proxy).
Bloom filters to avoid futile cache lookups for keys that definitely don't exist.
Regional failover — if a client's local region is down, transparently fall back to another region.
Insight: cache tier is treated as a first-class service, not just a "hopefully warm" optimization.

6.3 Discord's cache invalidation

Discord uses CDC from PostgreSQL → Redis invalidation for user-visible caches like guild membership. Their post-mortems describe: - Debezium reading Postgres WAL. - Kafka topic per table. - Consumer service that deletes the right Redis keys. - Median invalidation lag: ~15 ms globally.

7. Cache observability — what to measure

If you can't see it, you can't tune it. Every serious cache deployment measures:

Metric	What it tells you	Alert if
Hit rate	Effectiveness	Drops > X% (something's cold or invalidation storm)
Miss rate	Backing store load	Spikes (thundering herd, key eviction wave)
Latency (P50 / P99)	Client experience	P99 > network baseline (cache node saturated)
Eviction rate	Cache under memory pressure	Sustained high evictions (need more memory or smaller values)
Key count / memory used	Capacity headroom	Approaching maxmemory
Connection count	Client pool health	Spikes (connection storm, retry storm)
Hot key detection	Skew	Any key > 1% of total ops
Replication lag (multi-region)	Consistency risk	> freshness SLA

Redis has INFO, MONITOR (careful, expensive), CLIENT LIST, and SLOWLOG for these. Prometheus exporters make this easy.

8. Cache anti-patterns — what to never do

1. Caching everything by default. If read/write ratio is close to 1:1, or keys are almost never re-read, the cache is pure overhead. Measure first.

2. Storing writes in the cache before the DB confirms. Unless you're deliberately doing write-back with durability, this is a data-loss bug waiting to happen.

3. Using in-process cache for session state in a horizontally-scaled fleet. Different app servers = different session state. Users see phantom logins. Use a distributed cache.

4. Long TTLs on data with strong-consistency needs. 5-minute stale price on a checkout page is a lawsuit waiting to happen. Match TTL to consistency SLA.

5. Cache keys with unbounded cardinality. Caching per-user-per-hour-per-region timeseries → billions of keys → cache is useless (nothing gets re-hit).

6. Ignoring the cold-cache problem. A cache restart / cluster resize with no warm-up plan → traffic spike hits the DB → cascading failure. (Day 5 Q9 scenario.)

7. Reading through the cache in a hot loop without single-flight. Every request creates its own stampede. Coalesce.

8. Storing gigantic values in Redis. Redis is single-threaded per shard; a single 10 MB GET blocks the shard for tens of ms. Break big values into chunks or use a different store (S3 for blobs, cache the pointer).

9. Interview angle (what FAANG asks)

Common flavors:

"How would you invalidate the cache at your scale?" → CDC + Kafka + consumers. Show you know that "the writer deletes the key" doesn't scale to many services.
"You have data centers in 3 regions — how do caches work?" → Per-region cache + CDC-based cross-region invalidation is the standard answer. Only geo-replicate if consistency SLA requires it.
"Would you use in-process cache?" → Yes, if there's heavy skew. Combined with short TTL or pub/sub invalidation. Show you know the trade-off.
"Read Facebook's Memcache paper, what's clever about it?" → Leases for stampede prevention; warm-up regions; separate pools per workload.
"How do you know your cache is working?" → Hit rate, eviction rate, latency, hot-key detection. Alert when hit rate drops.

Trap they set: they'll say "we have consistent user experience globally." A weak candidate says "cache everything." A strong candidate asks: "What's the freshness SLA per data type? Auth tokens must be strong; product catalog can be 5 min stale; user profile is ~30 s tolerable. That drives whether we per-region-cache or geo-replicate."

10. Quick reference card

Fleet-wide invalidation → CDC (Debezium + Kafka). Not per-writer deletes.
Multi-region → per-region cache + CDC invalidation. Geo-replicate only if you must.
Refresh patterns: refresh-ahead (cache tier), background refresh (fixed hot set), stale-while-revalidate (CDN/HTTP).
Tiered cache (L1 + L2) wins when top-1% of keys = >50% of traffic.
L1 invalidation is the hard part — usually short TTL or pub/sub.
Case studies: Memcache (leases, warm regions, pool separation), EVCache (multi-region, Bloom filters).
Observe: hit rate, latency, eviction rate, hot keys, replication lag.
Anti-patterns: caching everything, unbounded cardinality, gigantic values, no cold-start plan.

Ready for the Day 6 quiz? Open it in the sidebar. Includes: - One spaced-repetition callback on bandwidth vs latency (High severity — from Day 4 Q5) - One fresh tail-amp application question (to maintain your newly-locked skill)

Answer in chat when done.