Day 7 — Content Delivery Networks (CDN) Deep Dive

Time budget: ~60 min · Level: Foundational depth. This is your first "user-facing infrastructure at planet scale" topic.

Why CDNs exist — the physics argument

Recall Day 2: light in fiber travels ~200,000 km/s (2/3 of vacuum c due to refractive index). One round-trip London → Sydney is ~17,000 km one-way = 170 ms just for photons. Add TCP handshakes, TLS, and the origin's own latency, and a "single request" from Sydney to a London-hosted origin easily costs 500-800ms — before it does any work.

CDNs exist because the fastest byte is one that never has to cross an ocean. Push a copy to a server 10ms from Sydney, and now the same request is 20-50ms.

The single equation to remember:

Effective user latency ≈ (RTT_to_edge) + (edge_processing) + (P_miss × RTT_to_origin + origin_latency)

Where P_miss is the cache miss probability. A well-tuned CDN pushes P_miss below 5% — meaning 95% of user requests never touch your origin at all. That's the magic.

The CDN topology (know this map)

                            ┌──────────────┐
                            │    ORIGIN    │  (your servers, S3, etc.)
                            │  (1 region)  │
                            └──────┬───────┘
                                   │
                          ┌────────┴────────┐
                          │  ORIGIN SHIELD  │  (optional mid-tier)
                          │   (1-2 regions) │  ← protects origin from thundering herd
                          └────────┬────────┘
                                   │
              ┌────────────────────┼────────────────────┐
              │                    │                    │
        ┌─────┴────┐         ┌─────┴────┐         ┌────┴─────┐
        │ REGIONAL │         │ REGIONAL │         │ REGIONAL │  Mid-tier PoPs
        │  CACHE   │         │  CACHE   │         │  CACHE   │  (~20-50 globally)
        └─────┬────┘         └─────┬────┘         └────┬─────┘
              │                    │                    │
         ┌────┼────┐          ┌────┼────┐          ┌───┼────┐
         │ EDGE PoP│          │ EDGE PoP│          │ EDGE PoP│  Edge tier
         │  x 100s │          │  x 100s │          │  x 100s │  (200-400 globally)
         └────┬────┘          └────┬────┘          └────┬────┘
              │                    │                    │
           👤 users              👤 users             👤 users

• PoP (Point of Presence): a data center in a specific city containing racks of caching servers.
• Edge tier: the outermost layer — the server the user's TCP connection actually terminates on.
• Mid-tier / regional cache: aggregates requests from many edge PoPs. Reduces load and miss rate at the shield layer.
• Origin shield: a single (or small set of) chosen mid-tier that ALL misses funnel through before hitting origin. This is your origin overload protection.

Why tiers? Because a request that misses at edge but hits at mid-tier still doesn't touch origin. This dramatically reduces origin bandwidth. Real-world tiered offloads are often 98-99.5% vs 90-95% single-tier.

How a user's request actually flows

Trace this end-to-end for https://cdn.example.com/hero.jpg:

1. User's browser does DNS on cdn.example.com.
2. DNS returns the IP of the CDN's nearest PoP — usually via one of: - Anycast — the CDN advertises the same IP from every PoP over BGP. Internet routing itself picks the nearest one. (Cloudflare, Fastly.) - DNS-based geo/latency routing — the DNS resolver returns a different IP based on the resolver's location. (Akamai, CloudFront classically.)
3. Browser opens TCP + TLS to the edge PoP (one RTT × 1-2 for QUIC, 2-3 for TCP+TLS 1.3).
4. Edge PoP checks its local cache using the cache key (see below).
5. HIT → returns bytes. Total user latency: 20-80ms typical.
6. MISS → request goes to regional cache → still miss → origin shield → still miss → origin. Bytes flow back and are populated at every tier on the way down.
7. Now the next nearby user gets a HIT.

The critical insight: the FIRST user to request a hot new asset "pays for" it. Everyone else free-rides.

Cache keys — the #1 source of CDN bugs

A cache key is the string the CDN uses to decide "have I seen this exact request before?"

Default cache key ≈ hostname + path. But that's often not enough. The truth is:

cache_key = normalize(scheme + hostname + path + selected_query_params + selected_headers)

Query strings

Do these serve the same content? /product?id=42 vs /product?id=42&utm_source=twitter

• If your app returns the same page, strip utm_* from the cache key, or you'll cache the same page 50 times.
• If different query params return different content, they must be part of the key.
• Anti-pattern: using query strings as cache-busters (?v=1723456789 on every request) — this destroys your hit rate.

The Vary header

Vary: Accept-Encoding tells the CDN "cache a separate copy per compression scheme." Common values: - Vary: Accept-Encoding — separates gzip vs br vs identity. Almost always safe. - Vary: User-Agent — DANGER. There are millions of unique user-agents. Your hit rate goes to zero. - Vary: Cookie — same danger.

Rule of thumb: every value in Vary multiplies your cache size and hurts hit rate. Use it sparingly.

Personalization + CDN — the classic clash

If your homepage shows "Hello, Alice", you cannot cache the same HTML for everyone. Options: 1. Cache the shell, hydrate on client: cache /home.html (public), fetch /api/me (uncached) to fill personalized bits. 2. Edge Side Includes (ESI): the CDN assembles the page from cached static fragments + uncached personalized fragments. 3. Vary by user tier (not user ID): if there are only 3 tiers, Vary: X-User-Tier still gives 3× cache but is workable.

HTTP caching headers — the standard vocabulary

You cannot separate CDN work from HTTP cache-control semantics.

Common recipe for static assets (immutable versioned):

Cache-Control: public, max-age=31536000, immutable

Meaning: cache forever, don't even revalidate. Requires hero-abc123.js style content-hashed filenames.

Common recipe for HTML pages:

Cache-Control: public, s-maxage=60, max-age=0, stale-while-revalidate=600, stale-if-error=86400

Meaning: CDN caches 60s, browser doesn't cache. Serve stale up to 10 min while refreshing async. Serve stale up to 1 day if origin fails.

Push vs pull CDNs

Which do FAANG use? Both. Netflix uses push for video assets (they know exactly what to cache). Twitter/Instagram use pull for user-generated content (they don't know which post goes viral).

Cache invalidation — the hard part

Two schools:

1. TTL-based (default)

Set a short TTL (e.g., 60s). Content self-expires. Simple, no infrastructure needed. Tolerates staleness.

2. Explicit purge

When you publish an update, you call the CDN's purge API. Two flavors: - URL-based purge: PURGE /article/123. Simple but doesn't scale — you must know every URL. - Tag-based purge (surrogate keys): attach tags at cache time (Surrogate-Key: article-123, user-42, homepage), then purge all URLs tagged article-123 in one call. Fastly and Cloudflare Enterprise support this.

Purge propagation time is a metric to know: Cloudflare's global purge is ~150-500ms globally; Fastly's is ~150ms. Legacy Akamai used to take minutes; modern ~5s.

The versioned-URL trick (best of both worlds)

Instead of purging, publish article-v2.html alongside article-v1.html and change the reference. Old cache entries expire naturally. No purge needed. This is how static asset pipelines work (Webpack's content-hash filenames).

Origin shield & thundering herd protection

Scenario: A hot article goes viral. 300 edge PoPs simultaneously get a cache miss. Without protection, 300 concurrent requests hit your origin.

Origin shield: all 300 misses funnel through 1 mid-tier PoP → that PoP dedupes them (request coalescing — you saw this on Day 5 Q9!) → 1 request goes to origin. The other 299 wait for the same response.

This is Day 5/6 caching material applied at CDN scale.

TLS termination at the edge

The CDN terminates the user's TLS connection (holds your certificate for cdn.example.com) and then re-encrypts to the origin over an internal connection.

Why: - The user's TLS handshake completes in ~1 RTT (to a nearby PoP), not ~1 RTT to origin. - The CDN can keep persistent TLS connections to origin, amortizing handshake cost across millions of user requests. - Modern CDNs also do HTTP/2 → HTTP/1.1 downgrade for older origins, and HTTP/3 (QUIC) at the edge even if origin only speaks HTTP/1.1.

Security consideration: the CDN sees plaintext of your traffic. If you can't accept that, you need either: - A private/self-hosted CDN edge (Netflix Open Connect), or - End-to-end encryption above TLS (rare for HTTP; standard for chat/E2EE apps).

Signed URLs and access control

For paywalled or private content:

https://cdn.example.com/premium/movie.mp4?Expires=1730000000&Signature=abc123

• Origin signs the URL with a shared secret. CDN validates the signature and expiry before serving. Common at CloudFront (CloudFront-Signed-*), Cloudflare (Signed URLs), Fastly, and S3 pre-signed URLs.
• Also enables hotlink protection (only your domain can embed the asset) and geo-restriction (Signature embeds allowed country codes).

Edge compute (Workers, Lambda@Edge, Compute@Edge)

Modern CDNs run your code at the edge, not just cache bytes. Examples:

• A/B test cookie assignment at the edge (no origin roundtrip).
• Request routing — pick which origin based on user's country/tier.
• Auth token validation — reject invalid requests without hitting your API.
• Personalization — assemble a page from cached fragments + a per-user database lookup, all at the edge.
• Bot mitigation.

Runtimes: Cloudflare Workers (V8 isolates, ~5ms cold start), Lambda@Edge (regular Lambda, ~100ms cold start), Fastly Compute@Edge (WebAssembly, ~1ms).

When to use it: whenever you'd otherwise send a whole request to origin just to make a small routing/auth decision. Massive latency win.

Video streaming — a case study in CDN specialization

A 2-hour movie is not "one file." It's split into thousands of small chunks (2-10 seconds each) at multiple bitrates (240p, 480p, 720p, 1080p, 4K). The client dynamically switches quality based on bandwidth. This is HLS (Apple) or DASH (open standard) — collectively called ABR (adaptive bitrate).

Why chunks? Because: - Each chunk is small (~1-5 MB) → cacheable at CDN edge. - Client can switch bitrate mid-video without re-buffering (next chunk comes from the new bitrate URL). - Rebuffers become "one missing chunk," not "video stalled."

CDNs are essential here because: - Origin bandwidth for millions of concurrent streams would be impossible. - Latency to the first chunk (Time To First Frame) dominates user perception. - Netflix goes further: Open Connect Appliances — Netflix-owned caches physically installed in ISP data centers, closer to users than any commercial CDN.

Observability — metrics that matter

A CDN report card: if your hit rate is 60% and your origin bandwidth is 40% of user bandwidth, something is wrong (cache key issues, low TTLs, Vary overuse, Cache-Control: private on content that should be public).

Vendor landscape (know the names)

Anti-patterns (interview trap answers)

1. Vary: User-Agent on public content — destroys hit rate.
2. Random cache-busting query strings on every request — CDN caches every unique URL once, then never again.
3. Setting no-cache when you meant must-revalidate — no-cache still stores; just forces revalidation. no-store is what forbids storage.
4. Long TTLs on frequently-changing content — you'll serve stale for hours and users will complain.
5. Short TTLs on rarely-changing content — you're paying origin bandwidth for nothing.
6. Setting Cache-Control: private on public assets — CDN won't cache, everyone hits origin.
7. Not using origin shield at massive scale — origin gets hammered by 300 PoPs on every hot key.
8. Trying to cache authenticated API responses without Vary: Authorization or per-user keying → cross-user data leaks. Security bug.

Interview angles

"How would you serve 100M users a global product catalog with < 100ms latency worldwide?"

Answer skeleton: 1. Origin = single region. Catalog data changes rarely (writes per hour). 2. CDN in front with s-maxage=3600, stale-while-revalidate=600, stale-if-error=86400. 3. Cache key = path only; strip UTM query params. 4. On catalog update: publish new version with tag catalog-v42; purge by tag OR just wait TTL. 5. Origin shield to protect origin from thundering herd on cold cache after purge. 6. Personalization (recommendations, price): keep out of cached response — fetch via uncached /api/user/prefs and merge client-side. 7. Metric: hit rate target ≥ 98%; if lower, investigate Vary, cache keys, or TTL config.

"CDN vs Reverse Proxy vs Application Cache — when do you use each?"

• CDN: geographic distribution matters, most traffic is cacheable, users are worldwide.
• Reverse proxy (nginx, Varnish) in your DC: full control, local caching, TLS termination, but only helps your DC users unless multi-region.
• Application cache (Redis/Memcached): caching computed results from your app (query results, session state, rate-limit counters). Not user-content optimization.

Most designs use ALL THREE, at decreasing scale: CDN (planet) → reverse proxy (region) → Redis (per-DC).

Spaced-repetition callbacks in today's quiz

Two concepts from earlier days that keep appearing:

• Push mechanism bi-directional check — reappears in Q6 (Day 3 Q6.1 was your weak spot). Ask yourself: does this need client→server too, or just server→client?
• JWT vs session cookies — reappears in Q4 T/F (Days 1 & 4 flagged as "improving"). Precise trade-off matters.

Both are here on purpose. Don't skim them.