SystemDesign.website
Track 3 · Module 3.4 · DataStores & Storage Engines

Database Indexes & B-Tree Internals

How database storage engines locate a single row out of 100 million in under 2 milliseconds. Master disk page block I/O, B+ Tree fan-out mechanics, Clustered vs Secondary indexes, Covering Indexes, and the severe write-penalty trade-off.

⏱️ Reading Time: ~25 min
🎯 Case Study: 10M User E-Commerce DB (users & orders)
🎬 Format: Textbook Analogy + Step-by-Step SVGs + Live B+ Tree Lab

💥 1. The Real-World Nightmare: The Full Table Scan

📖 The Simple Analogy: Finding a Word in an 800-Page Textbook

Imagine you have an 800-page computer science textbook without an index in the back. You need to find the definition of "Consistent Hashing". What must you do?

You have no choice but to start at Page 1 and scan every single line of text on every single page until you reach Page 742. This is an O(N) Full Table Scan.

💽 How Hard Drives and SSDs Actually Store Database Records:Databases do NOT read rows one-by-one from disk. Disks read and write in fixed-size blocks called Pages (MySQL InnoDB uses 16 KB per page, PostgreSQL uses 8 KB).
📊 Let's Do the Disk Page Math on a 10,000,000 Row Table:
Table Size:10,000,000 Users~200 bytes per row
Rows Per 16 KB Page:~80 Rows / Page16,384 bytes / 200 bytes
Total Pages on Disk:125,000 Disk Pages= 2.0 Gigabytes of Data
When a user runs:SELECT * FROM users WHERE email = 'alice@gmail.com';Without an index on email, the storage engine must load all 125,000 pages (2 GB) from disk into RAM sequentially. Even at 150 MB/s disk speed, this single query takes 13.3 SECONDS, saturates 100% of Disk I/O, and causes the entire database server to freeze.

📑 2. What is an Index, Really?

An Index is an auxiliary, sorted data structure kept alongside the table. It holds only two items:

1. The Search Key

The value you are searching for (e.g. alice@gmail.com), kept in strict sorted alphabetical order.

2. Direct Record Pointer

The exact physical location of the full row on disk (e.g. Page #412, Slot #14 or Primary Key id = 8941).

❓ Why can't we just sort the main data table itself?

A table can only be physically stored in ONE sorted order on disk (e.g. by id). But in production, you need fast lookups by email, username, phone_number, and created_at. Each auxiliary index gives you a fast-path lookup for that specific column.

🌳 3. Why Binary Search Trees Fail on Disk (The Birth of B-Trees)

In university algorithms courses, we learn that balanced Binary Search Trees (Red-Black Trees, AVL Trees) give fast O(log₂ N) lookups. Why don't production databases like MySQL, Postgres, Oracle, or SQLite use Binary Search Trees?

❌ Binary Search Tree on Disk
• Branching Factor (Fan-out): Exactly 2 children per node.
• Tree Height for 10M Rows: log₂(10,000,000) ≈ 24 levels.
• Disk I/O Seeks: You must fetch 24 separate disk pages sequentially from disk to find one record!
• Disk Page Waste: A 16 KB disk page holds only 1 tiny node (32 bytes), wasting 99.8% of read bandwidth.
Latency: ~24 Disk Reads = 100ms – 240ms 🐌
✅ B-Tree / B+ Tree (High Fan-out)
• Branching Factor (Fan-out): 500 to 1,000 children per node page.
• Tree Height for 10M Rows: log₅₀₀(10,000,000) ≈ 3 levels.
• Disk I/O Seeks: Exactly 3 page reads to reach ANY record in the database!
• RAM Caching: Root & Level-1 pages stay in RAM buffer pool $\implies$ only 1 physical disk read!
Latency: 1–3 Page Reads = 0.5ms – 2ms ⚡
1 Visual Comparison: Tree Height & Disk Seeks

Every level you descend in a disk tree requires a separate disk page fetch. Increasing the fan-out from 2 to 500 flattens the tree into a broad, shallow 3-level structure.

BINARY TREE (Height = 24)24 sequential disk seeks per query↓ 21 More Levels Deep ↓24 Page Reads = 120ms LatencyB+ TREE (Height = 3, Fan-out = 500)Holds 125,000,000 keys in 3 page readsRoot Page [16 KB]Page 2Page 3Page 501250,000 Leaf Pages (Full Rows & Pointers)3 Page Reads = 1.2ms Latency ⚡

⚡ 4. B-Tree vs B+ Tree: The 3 Rules of Modern Databases

Almost every relational and key-value database in production (MySQL InnoDB, Postgres, SQLite, Oracle) uses a B+ Tree rather than a standard B-Tree. Here are the 3 structural rules that make B+ Trees supreme:

1Internal Nodes Store ONLY Keys & Page Pointers

In a standard B-Tree, internal nodes store actual table rows. In a B+ Tree, internal nodes store strictly search keys and child page pointers (e.g. [Key: 50 | Pointer: Page 14]). Because keys are tiny (8 bytes), a single 16 KB page holds 1,000+ keys, maximizing fan-out and keeping the tree razor-flat.

2ALL Data Records Reside Exclusively in Leaf Nodes

Every query traverses to the bottom leaf level. Every point lookup takes the exact same predictable number of I/O steps (O(log N)).

3The Superpower: Horizontal Doubly-Linked Leaf Chain

All leaf pages are linked horizontally left-to-right via pointers (prev <--> next). This makes Range Scans lightning fast!

To run SELECT * FROM orders WHERE amount BETWEEN 100 AND 500: the engine does 1 single tree lookup to find leaf node 100, then simply walks right along the linked list pages without ever backtracking up the tree!
2 Step-by-Step B+ Tree Page Layout & Horizontal Scan
ROOT PAGE [16 KB][ < 30 | 30 | 70 | >= 70 ]INTERNAL PAGE A[ 10 | 20 | 30 ]INTERNAL PAGE B[ 40 | 50 | 60 ]INTERNAL PAGE C[ 70 | 80 | 90 ]LEAF PAGE #1K: 10, 15, 20[Row Data / Pointers]LEAF PAGE #2K: 30, 40, 50[Row Data / Pointers]LEAF PAGE #3K: 70, 80, 90[Row Data / Pointers]↔ Fast Horizontal Range Scan ↔Walk leaves directly without traversing up parent nodes

🔍 5. Clustered vs Secondary Index & The Double Lookup

Understanding the difference between Clustered and Secondary (Non-Clustered) indexes is the #1 question asked in senior system design and database interviews.

1. Clustered Index (Primary Key)

In MySQL InnoDB, the table IS the clustered index. The leaf pages do not store pointers — they store the actual, complete row columns (id, name, email, password, created_at).

Rule: Only 1 Clustered Index allowed per table.
2. Secondary Index (e.g. on `email`)

An auxiliary B+ Tree created on a non-primary column. The leaf pages store the indexed value plus the Primary Key value (e.g. email -> id: 8941).

Rule: You can have multiple secondary indexes.
🔄 The Two-Step "Double Lookup" (Bookmark Lookup)

When you run SELECT * FROM users WHERE email = 'alice@gmail.com':

Step 1:Database traverses the Secondary Index B+ Tree on email $\implies$ finds leaf containing id = 8941 (3 page reads).
Step 2:Database takes id = 8941 and traverses the Clustered Index B+ Tree $\implies$ retrieves the full table row (3 page reads).
🚀 Senior Engineer Trick: Covering Indexes

If you write: SELECT id, email FROM users WHERE email = 'alice@gmail.com', the database finds both email and id directly inside the secondary index leaf node. It skips Step 2 completely! This is called an Index-Only Scan / Covering Index (0 clustered disk seeks).

⚖️ 6. The Hidden Cost: The Write Penalty & Page Splits

Why not just index every column in the table? Because indexes speed up READS, but drastically slow down WRITES.

1. Every INSERT/UPDATE/DELETE Pays

When you insert 1 row into a table with 6 secondary indexes, the database must write to the table AND perform 6 separate B+ Tree rebalancing writes on disk!

2. Expensive Page Splits

When a 16 KB leaf page fills up, inserting a new key forces the storage engine to split the page into two 8 KB pages, allocate new disk space, and update parent tree pointers up to the root.

Number of Indexes on TableSELECT Latency (Point Query)INSERT / Batch Write ThroughputRecommendation
0 Indexes (Heap Scan)13,000 ms (Catastrophic)65,000 writes/secTerrible for queries
1 – 3 Indexes (Targeted)1.2 ms (Optimal)48,000 writes/secProduction Sweet Spot
8+ Indexes (Over-Indexed)1.2 ms6,500 writes/sec (10x drop!)Write bottleneck & RAM bloat

💻 7. Low-Level B+ Tree Node Search Implementation

How does a database storage engine search inside a 16 KB page? It uses binary search over the sorted page keys in O(log K) time.

Select Language:
// Python 3.10+ B+ Tree Node Traversal & Binary Search
import bisect

class BPlusTreeNode:
    def __init__(self, is_leaf: bool = False):
        self.is_leaf = is_leaf
        self.keys = []        # Sorted keys in 16KB page
        self.children = []    # Child page pointers (if internal)
        self.records = []     # Row data / RecordIDs (if leaf)
        self.next = None      # Horizontal linked list pointer

    def find_child(self, key):
        # O(log K) binary search to find child page index
        idx = bisect.bisect_right(self.keys, key)
        return self.children[idx]

    def search(self, key):
        curr = self
        # Descend from Root to Leaf
        while not curr.is_leaf:
            curr = curr.find_child(key)
        
        # Search within leaf page
        idx = bisect.bisect_left(curr.keys, key)
        if idx < len(curr.keys) and curr.keys[idx] == key:
            return curr.records[idx]
        return None

🧪 8. Interactive Lab: Live B+ Tree & Query Execution Plan

⚡ Real-Time B+ Tree Visualizer

Interactive B+ Tree Search & EXPLAIN Simulator

Insert keys, trigger page splits, and compare Query Execution Plans
1. Insert Key into Tree:
2. Simulate Query Mode:
3. Reset Tree:
B+ Tree Structure (Max 3 keys per page)Height: 2 Levels
Leaf Chain: ↔ LinkedTotal Keys: 6
Simulated SQL Query:
SELECT * FROM users WHERE id = 30;
Database EXPLAIN Plan:
Access Type:Index Seek (ref)
Pages Fetched:2 Pages (32 KB)
Est. Latency:< 1.5 ms
Clustered Lookup:Direct Leaf
⚡ Index Seek: The storage engine traversed 1 Root Page and 1 Leaf Page to locate record 30. Zero full table scanning required.

🎯 9. Test Your Understanding

Scenario 1: The Bulk CSV Import Meltdown

A backend engineer adds 12 secondary indexes to an orders table to speed up all possible reporting queries. Later that night, a nightly batch job importing 5,000,000 orders takes 14 hours instead of 15 minutes. What is the root cause?

Scenario 2: The Ignored Index Mystery

You create an index on column is_active (BOOLEAN) in a 20,000,000 row table. When you run EXPLAIN SELECT * FROM users WHERE is_active = true, the optimizer chooses a Full Table Scan (type: ALL) instead of using your index. Why?

○

Completed this module?

Mark as finished to update your curriculum progress tracker.