Track 2: data-layerID: caching-layer-placement

Mode: structured•Difficulty: intermediate•⏱️ 20 mins

Module · Data Layer · 20 min

Caching Strategies & Placement — Speed of Light at the Data Layer

Even with Sharding and Replication, disk I/O takes 5ms to 50ms. Placing an in-memory caching tier like Redis in front of your database serves 95%+ of queries in <1ms directly from RAM.

01 / Concept Cards (Theory)

In-Memory Caching Architecture & Strategies

Master the memory hierarchy, read/write caching patterns, eviction algorithms, and the 3 classic cache failure disasters.

1. First Principles: The Latency Numbers Every Engineer Must Know

Why does caching make systems blindingly fast? Because reading from RAM is orders of magnitude faster than reading from Disk or Network.

⚡ L1 / L2 CPU Cache~0.5 – 1 nsOn-chip hardware cache

🚀 Main Memory (RAM)~100 ns (0.0001 ms)Redis / Memcached speed

💾 NVMe SSD (Flash)~100 µs (0.1 ms)1,000× slower than RAM

🌐 DB Query over Network~5 – 50 ms50,000× slower than RAM

💡 The 80/20 Pareto Principle: In almost all web applications (Instagram, Amazon, Twitter), 80% of read requests query just 20% of your data (popular tweets, active products, user profiles). By caching that 20% in fast RAM, 80%+ of database queries vanish.

2. The 4 Core Read & Write Caching Patterns

How should your application coordinate writes and reads between the cache and the primary database?

1. Cache-Aside (Lazy Loading)App coordinates Cache and DB directly• Read: App checks Cache. If found (Hit), return it. If not found (Miss), fetch from DB, write into Cache, and return.
• Write: App writes to DB, then deletes/invalidates the cache key.

✅ Pros: Resilient — if cache dies, app still reads from DB. Only caches requested data.
⚠️ Cons: Three network round trips on cache miss (Cache check → DB read → Cache write).

2. Write-ThroughApp writes to Cache; Cache writes to DB• The application treats Cache as the main store.
• When data is written, the Cache synchronously writes to the DB before confirming success.

✅ Pros: Zero stale data; cache is always in sync with DB.
❌ Cons: Higher write latency because every write must complete both in Cache and DB.

3. Write-Back (Write-Behind)Write to Cache immediately; async batch to DB• App writes only to Cache and receives instant success (1ms).
• Cache periodically batches and flushes writes to the DB in the background.

✅ Pros: Blazing fast write performance. Ideal for write-heavy apps (e.g. tracking video view counts, real-time analytics).
🚨 Danger: If the cache server crashes before flushing to DB, data is permanently lost.

4. Write-AroundWrites bypass Cache directly to DB• Data is written straight to the database without touching the cache.
• Only when that data is read later does it enter the cache.

✅ Pros: Prevents cache flooding for data that is written once and rarely read (e.g. archival logs, legal contracts).

3. Visualizing Cache-Aside: Hit vs. Miss Request Flows

Watch the live query routing below: Cache Hits resolve instantly from RAM in <1ms, while Cache Misses query the database and backfill the cache.

⚡ Cache-Aside Request Journey — Live Simulation

4. Cache Eviction Policies & Invalidation: Update vs. Delete

Memory is finite. When Redis runs out of RAM, it must evict older keys to make room for new ones.

🗑️ LRU (Least Recently Used)Evicts the key that hasn't been accessed for the longest time. Implemented via Doubly-Linked List + Hash Map in $O(1)$ time. Most widely used default.

🔢 LFU (Least Frequently Used)Tracks access count counters and evicts keys with the lowest hit frequency. Great for distinguishing evergreen popular data from transient spikes.

⏱️ TTL (Time To Live)Keys automatically expire after a fixed duration (e.g. TTL = 3600s). Guarantees eventually consistent data even if invalidation logic fails.

🤔 When Data Changes: Should You UPDATE Cache or DELETE Cache?

Always DELETE the cache key (cache.del(key)), never UPDATE it!
If two concurrent requests (Request A and Request B) update the database at the same time, network delays can cause Request A's cache update to arrive after Request B's cache update, leaving permanently corrupt/stale data in Redis. Deleting the key forces the next read to fetch the latest truth from the database cleanly.

5. The 3 Classic Cache Disasters (Interview Gold)

In high-scale production, three specific failure modes can bring down entire data infrastructures. Here is how they happen and how senior engineers prevent them:

Disaster 1Cache Penetration (Querying Non-Existent Keys)

The Attack: A malicious bot or bug requests millions of non-existent IDs (e.g. GET /user/-9999). Because the key does not exist in Redis, every request misses cache and hits the database directly, overloading the DB.

🛠️ The Fix: Use a Bloom Filter in front of Redis to immediately reject keys that don't exist in $O(1)$ time, or cache a NULL value with a short 60-second TTL.

Disaster 2Cache Breakdown / Thundering Herd (Hot Key Expiry)

The Scenario: A viral breaking news article receiving 100,000 requests/sec has its cache key expire at 12:00:00. At that exact millisecond, all 100,000 requests miss cache simultaneously and rush to query the database to rebuild the cache. The database immediately dies.

🛠️ The Fix: Use a Distributed Mutex Lock (SETNX) so only one worker queries the DB to rebuild cache while all other requests wait or receive stale data.

Disaster 3Cache Avalanche (Simultaneous Mass Expiry)

The Scenario: At midnight, an e-commerce platform caches 500,000 products with a fixed TTL = 2 hours (7200s). At exactly 2:00:00 AM, all 500,000 keys expire simultaneously. An avalanche of requests crashes the database.

🛠️ The Fix: Add Random TTL Jitter: TTL = 7200 + random(0, 600). This scatters key expiration smoothly over a 10-minute window.

6. Caching Patterns Comparison Matrix

Quick reference guide for System Design Interviews:

Pattern	Cache-Aside	Write-Through	Write-Back	Write-Around
Read Performance	Fast (on Hit)	Fast (always Hit)	Fast (always Hit)	Fast (after 1st read)
Write Latency	Low (Writes to DB)	High (Writes to both)	Ultra Low (<1ms RAM)	Normal (DB only)
Data Consistency	Eventual (via TTL/Del)	Strong (always in sync)	Weak (lagged flush)	Eventual
Data Loss Risk	None (DB is primary)	None	High (if Cache crashes)	None
Best Used For	General Web, User Profiles	Financial, Core Metadata	Analytics, View Counters	Logs, Write-Once Data

7. Interactive Caching Scenario Quiz

Test your caching engineering intuition:

📰 A breaking news alert triggers 50,000 requests/second for the article. The article's cache key expires, and the database CPU instantly spikes to 100%. What disaster occurred and how do you fix it?

✅ Cache Breakdown is correct! A single popular hot key expired, causing all 50k concurrent requests to miss cache simultaneously and crush the DB. A distributed mutex lock ensures only 1 request queries the DB to rebuild the cache while others wait.

🤖 An attacker writes a script generating random UUIDs and sends 100,000 requests/sec. Every request misses Redis and forces a disk scan on MySQL. What is the most efficient defense?

✅ Bloom Filter is correct! This is a Cache Penetration attack. A Bloom Filter uses a tiny bit array to verify whether an ID could possibly exist in the DB before allowing any database lookup.

🎮 You are designing the live view counter for a YouTube live stream with 2 million concurrent viewers. Every second, 50,000 view increments occur. Which caching pattern should you use?

✅ Write-Back is correct! Writing 50,000 times/sec directly to a SQL database would melt the disk. Using Redis `INCR` in memory and flushing aggregated counts to the database every 10 seconds is the industry standard.

🛒 An e-commerce platform caches 1,000,000 product pages at midnight with `TTL = 3600s`. At 1:00 AM, the database experiences a massive latency spike. How should this be prevented?

✅ Random TTL Jitter is correct! This is a Cache Avalanche caused by simultaneous key expiration. Adding random jitter spreads key expirations evenly over several minutes so the DB receives a gentle trickle instead of a tidal wave.

02 / Practice Exercise

Build Your Cache-Aside Architecture

Design an in-memory caching topology: Route user traffic through the App Server to an in-memory Redis Cache for instant reads, with fallback and persistence to the Primary Database.

Interactive CanvasMode: Freeform Drag & Connect

Task: Cache-Aside Layer Placement

Grid Overlay

🎯

Scenario Prompt: Wire up a high-performance Cache-Aside architecture: Connect the Client to the App Server. Connect the App Server to the Redis Cache for in-memory reads, and connect the App Server to the Database for cache-miss fallbacks and writes.

Available Components (Click to place on canvas):

🖱️Canvas is emptyClick components above to place them onto the canvas.
Drag components to arrange them freely, and click two nodes to connect them.

💡 Drag to move • Click Node A ➔ Node B to connect • Click ✕ to delete0 components placed