SystemDesign.website
Track 5 Β· Module 5.3 Β· Communication Patterns & Asynchronous Systems

Unique ID Generator (Twitter Snowflake)

How to mint 4 million sortable IDs per second per node with zero coordination. Master the 64-bit Snowflake layout, clock-drift guards, sequence exhaustion, node ID assignment, and UUIDv7.

⏱️ Reading Time: ~26 min
🎯 Core Architecture: 41-bit Time + 10-bit Node + 12-bit Sequence
🎬 Format: Hospital Analogy + Live 64-Bit Visualizer

🎫 1. The Real-World Intuition: 50 Hospitals, No Phone Lines

πŸ₯ The Analogy: Numbering Every Baby Born in a Country

You run a national health system with 50 hospitals. Every newborn needs a certificate number that is unique across the whole country β€” and the hospitals cannot call each other. There is no central registry to ask. Babies arrive at 3 a.m. during a network outage and still need a number, instantly.

❌ Every Hospital Counts From 1

Each hospital keeps its own notepad: 1, 2, 3, 4… It works perfectly in isolation. Then you merge the national database and discover that certificate #1001 belongs to a baby in Mumbai and a baby in Delhi.

Result: Collisions everywhere. Your "unique" primary key is a duplicate 50 times over, and you cannot tell which record is real.
βœ… Date + Hospital Code + Counter

Give every hospital a permanent 3-digit code, then stamp each certificate as20260824-042-0007β€” the date, the hospital, and a small counter for babies born the same day.

Result: Guaranteed unique with zero coordination β€” forever. Bonus: sorting the numbers sorts the babies by birth date.

Notice why the second scheme needs no phone call. Two hospitals can never collide because their hospital code differs. One hospital can never collide with itself because either the date differs, or β€” if two babies arrive the same day β€” the counter differs. Every possible collision is closed off by construction.

πŸ’‘ The Core Rule of Distributed IDs:A globally unique ID needs exactly three ingredients β€” WHEN (a timestamp), WHO (a node identifier), and WHICH (a counter to break ties inside the same instant). Get all three into one number and you never have to ask another machine for permission again. Twitter Snowflake is precisely this idea, packed into 64 bits.

πŸ’₯ 2. Why the Obvious Answers Break

Before reaching for Snowflake, understand the four schemes it replaces β€” and the specific way each one dies.

1Single-Database AUTO_INCREMENT

The database hands out 1, 2, 3… under a lock. Flawless on one node, and genuinely the right answer until you shard.The failure: the counter lives inside one database. That database is now a single point of failure for every write in your system, and every ID request pays a network round trip. Shard it and both shards start at 1.

2Multi-Master with Offsets

Set auto_increment_increment = 2 and give server A offset 1 (odd IDs) and server B offset 2 (even IDs). Clever, and it works.The failure: the step size is baked into the topology. Adding a third server means re-planning every offset across the fleet, and IDs from different servers interleave unpredictably so they are no longer time-ordered.

3The Flickr Ticket Server

Two dedicated MySQL boxes do nothing but REPLACE INTO Tickets64 then return LAST_INSERT_ID(), one serving odd numbers and one even.The failure: it is still a network hop on the critical path of every insert. You have traded a database bottleneck for a smaller, faster bottleneck β€” and you still cannot generate an ID while offline or partitioned.

4UUID v4 (128 Bits of Randomness)

122 random bits, generated locally, no coordination. Collision risk is negligible β€” you would need to mint billions of UUIDs before a collision became as likely as a meteor strike.The failure is not collisions β€” it is index locality. Random IDs land at random positions in a B+Tree. Instead of appending to the rightmost leaf page (which stays hot in the buffer pool), every insert dirties a different page: page splits, fragmentation, and a buffer pool that thrashes because the working set is now the entire index instead of its right edge.

πŸ’‘ The Distinction That Matters:Engineers reject UUIDv4 for the wrong reason. It is not a collision problem β€” the math is overwhelmingly safe. It is a write-locality problem. Time-prefixed IDs append to the end of the index; random IDs scatter across it. That single property is why Snowflake, ULID, and UUIDv7 all put the clock in the high bits.
SchemeSizeTime-Sortable?Coordination Needed?Index Locality
AUTO_INCREMENT8 bytesβœ… Perfectly❌ Yes β€” every insertβœ… Excellent
Ticket Server8 bytesβœ… Perfectly❌ Yes β€” network hopβœ… Excellent
UUID v416 bytes (36 as text)❌ Neverβœ… None❌ Terrible
Snowflake8 bytesβœ… Roughly⚠️ Node ID only, onceβœ… Excellent
UUID v716 bytesβœ… Roughlyβœ… Noneβœ… Very good

🧬 3. Anatomy of a 64-Bit Snowflake ID

Snowflake takes the hospital scheme and compresses it into a single 64-bit signed integer β€” small enough to be a primary key, an int64, and a Kafka partition key all at once.

The 64-bit layout (most significant bit on the left):
0
1b
TIMESTAMP
41 bits Β· milliseconds since custom epoch
NODE
10 bits
SEQUENCE
12 bits
1 sign41 timestamp10 node12 sequence= 64 bits
41 bits β†’ 69.7 years

241 = 2,199,023,255,552 ms.
Γ· 86,400,000 ms/day = 25,451.7 days
Γ· 365.25 = 69.7 years of headroom.

10 bits β†’ 1,024 nodes

210 = 1,024 distinct node IDs (0–1023). Twitter originally split this as 5 bits datacenter + 5 bits worker.

12 bits β†’ 4,096 per ms

212 = 4,096 IDs per millisecond per node
= 4,096,000 IDs/second from a single node.

The custom epoch is what makes 41 bits enough. Counting milliseconds from 1970 would already have burned ~56 years of the budget. Twitter instead counted from 1288834974657 ms β€” 2010-11-04T01:42:54.657Z β€” buying the full 69.7 years from that date, which exhausts in July 2080.

πŸ”¬ A worked example β€” packing and unpacking a real ID
// PACK β€” an ID minted on node 42, 8th ID of that millisecond
wall clock   = 1787572800000        // 2026-08-24T12:00:00.000Z
EPOCH        = 1288834974657        // 2010-11-04T01:42:54.657Z
delta        = 1787572800000 - 1288834974657 = 498737825343 ms

id = (delta << 22) | (node << 12) | sequence
   = (498737825343 << 22) | (42 << 12) | 7
   = 2091858055787618311

// UNPACK β€” pure shifts and masks, no database lookup needed
timestamp = (id >> 22) + EPOCH   = 1787572800000  // 2026-08-24T12:00:00.000Z
node      = (id >> 12) & 1023    = 42
sequence  =  id        & 4095    = 7

The shift amounts fall straight out of the layout: the sequence occupies the low 12 bits, so the node shifts left by 12; the node occupies 10 bits above that, so the timestamp shifts left by 10 + 12 = 22. The masks are just (1 << 12) - 1 = 4095 and (1 << 10) - 1 = 1023.

πŸ’‘ Why the Top Bit Is Always 0:64 bits would allow 42 timestamp bits, but Snowflake deliberately wastes bit 63. A Java long, a Go int64, and a Postgres BIGINT are all signed β€” setting the high bit produces a negative ID. Leaving it zero guarantees every Snowflake ID is a positive integer that sorts correctly in every database on earth.
🚨 The Bug That Bites Every Team Once: JavaScript Silently Corrupts Snowflake IDs

A Snowflake ID is 19 digits. JavaScript numbers are IEEE-754 doubles, exact only up toNumber.MAX_SAFE_INTEGER = 9007199254740991 β€” 16 digits. A Snowflake ID is roughly 232Γ— larger than the largest integer JavaScript can represent exactly. So JSON.parse does not error β€” it rounds, and quietly hands your front-end the wrong ID.

// The server sends this JSON:
{ "id": 2091858055787618311 }

// The browser parses it:
JSON.parse(body).id  // => 2091858055787618300   // ← last 2 digits destroyed!

// The fix: serialize IDs as STRINGS on the wire.
{ "id": "2091858055787618311" }   // βœ… survives intact

Symptoms in the wild: two different records appear to share an ID, clicking a row opens the wrong record, and a permalink 404s β€” while the API returns perfectly correct data. This is why the Twitter API has always returned both id and id_str.

βš–οΈ "Roughly Sortable", Not Strictly Monotonic

Snowflake IDs are k-sorted, not globally ordered. Within one node, IDs strictly increase. Across nodes, two IDs minted in the same millisecond order by node ID, not by the true instant β€” and clocks drift by a few milliseconds anyway. So Snowflake gives you excellent index locality and a reliable coarse chronological sort, but never treat "id_a < id_b" as proof that A happened before B. If you need true ordering, you need logical clocks or consensus, not timestamps.

⏰ 4. The Three Ways Snowflake Fails in Production

The bit layout is the easy part β€” you can draw it in an interview in thirty seconds. What separates someone who has operated an ID service from someone who has only read about one is knowing these three failure modes.

1Clock Drift β€” The Duplicate-ID Bomb

Snowflake's uniqueness rests entirely on the assumption that the wall clock never moves backward. But NTP does exactly that: when it finds the local clock ahead of the true time, it can step the clock back. The generator then re-enters a millisecond it has already used, restarts the sequence at 0, and re-emits IDs it already handed out. You get duplicate primary keys β€” the one failure a unique ID generator must never have.

Cloud VMs are especially exposed: live migration, VM pause/resume, and snapshot restore can all warp the guest clock in a single jump.

MitigationHow It WorksThe Trade-Off
Refuse & throwTrack lastTimestamp; if now < last, raise an exception. Twitter's original behaviour.Chooses correctness over availability β€” the service returns errors until the clock recovers.
Spin until caught upBlock in a loop until the wall clock passes lastTimestamp.Fine for a 5 ms jump; a 10-minute jump hangs every caller. Always cap the wait.
Monotonic clock sourceTake the wall clock once at startup, then advance using a monotonic timer that NTP cannot rewind.Immune to steps, but slowly diverges from real time β€” decoded timestamps become approximate.
Coarser resolutionSonyflake-style: use 10 ms ticks instead of 1 ms, spending the saved bits on lifetime and nodes.Absorbs small drift and extends lifespan, at the cost of per-node throughput.
Clock-free modeAlibaba Seata-style: treat the ID as a pure monotonic counter, only consulting the clock at startup.Cannot drift at all, but IDs no longer decode to a trustworthy wall-clock time.

Operational note: configure time sync to slew (adjust gradually) rather than step (jump) β€” chronyd with makestep restricted, or ntpd -x. Never run a cron job that calls ntpdate on a host that mints IDs.

2Sequence Exhaustion β€” The Latency Cliff

12 bits give you 4,096 IDs per millisecond. Ask for the 4,097th within the same millisecond and the sequence wraps back to 0 β€” which would duplicate the IDs you just issued. The correct behaviour is to busy-wait until the clock ticks over to the next millisecond, then reset the sequence:

if (ts == lastTimestamp) {
    sequence = (sequence + 1) & 4095;      // SEQ_MASK
    if (sequence == 0)                   // all 4096 slots used this ms
        ts = tilNextMillis(lastTimestamp); // ← spin here
}

long tilNextMillis(long last) {
    long ts = System.currentTimeMillis();
    while (ts <= last) ts = System.currentTimeMillis();
    return ts;
}

This is correct but it creates a latency cliff: p50 stays at nanoseconds while p99.9 jumps to a full millisecond of spinning. At 4M IDs/sec you are riding that cliff constantly. The fix is horizontal β€” add nodes, since each one gets its own 4,096/ms budget.

3Node ID Assignment β€” The Hardest Part

Everything above assumes each generator has a unique node ID. If two pods both believe they are node 3, they emit identical IDs β€” silently, forever, with no error anywhere. This is the problem interviewers actually probe, because it is the one with no clean answer.

ApproachProsConsVerdict
Static config fileZero dependencies, trivial to reason about.One copy-pasted config and you have silent duplicates. Cannot autoscale.❌ Only for fixed, hand-managed fleets.
ZooKeeper ephemeral sequential znodeStrong guarantee; the node ID is released automatically when the process dies.Operating a ZooKeeper ensemble; ID churn on every restart.βœ… The classic, battle-tested answer.
etcd / Consul leaseSame guarantee with a modern API and TTL-based expiry.Still an external dependency on the startup path.βœ… Preferred if you already run etcd.
Kubernetes StatefulSet ordinalFree β€” the pod is already named gen-0, gen-1; no extra system to run.Ordinals are reused on scale-down then scale-up; capped at 1,024 replicas.⚠️ Elegant, but read the trap below.
Derive from private IP / MACNo coordination at all.10 bits cannot hold an IP β€” you truncate, and truncation collides.❌ A collision waiting to happen.

The StatefulSet trap: scale 8 β†’ 3 β†’ 8 and the new gen-5 inherits the node ID the old gen-5 was using seconds ago. If the old pod's clock was even slightly ahead, the new pod re-issues IDs from a millisecond the old pod already consumed. A node ID must not be reusable until longer than your maximum clock-skew window β€” which is exactly what a lease TTL gives you and a bare ordinal does not.

πŸ’‘ The 41-Bit Cliff:Every Snowflake deployment has an expiry date set by its epoch. Twitter's runs out in July 2080; if you pick your own epoch today, write the exhaustion date in the runbook. When the timestamp overflows, IDs wrap and collide with IDs from 69 years earlier β€” and you cannot widen a 64-bit column's meaning without rewriting every foreign key that references it.
🚨 What to Alarm On
Clock-step events β€” any backward jump on an ID-generating host. Page immediately.
Sequence-exhaustion rate β€” how often tilNextMillis spins. Rising = you need more nodes.
Node-ID lease failures β€” a generator that starts without a confirmed unique node ID must refuse to serve.
Duplicate-ID canary β€” a background job asserting uniqueness. Your only proof the other three worked.

πŸ†š 5. The Modern Alternatives: UUIDv7, ULID, KSUID

Here is the honest 2026 answer that many system design write-ups skip: most teams should not build a Snowflake service. UUIDv7 β€” standardised in RFC 9562, which obsoleted the old RFC 4122 in May 2024 β€” gives you time-ordered, index-friendly IDs with no node IDs, no ZooKeeper, and no clock-drift runbook. Reach for Snowflake when you specifically need 64 bits (half the storage, cheap in every index and foreign key) or more than 4,096 IDs per millisecond per process.

⭐ UUID v7 β€” the new default (RFC 9562)

128 bits: a 48-bit big-endian Unix millisecond timestamp, then the 4-bit version (0111), 12 bits of randomness, the 2-bit variant (10), and 62 more random bits. Time in the high bits means it sorts and indexes almost as well as Snowflake, while the 74 random bits remove the need for any node coordination.

48 bits of milliseconds from 1970 lasts until the year 10899 β€” no epoch cliff. Native uuidv7() landed in PostgreSQL 18, and every major language has a library. UUIDv6 is a legacy-compatible reordering of v1; UUIDv8 is a free-form slot for custom layouts.

ULID

128 bits as 26 characters of Crockford Base32: 48-bit millisecond timestamp + 80 random bits. Its trick is that the text form sorts lexicographically β€” perfect for object-store keys and log filenames. A monotonic variant increments the random part within the same millisecond.

KSUID

20 bytes as 27 Base62 characters: a 32-bit second-resolution timestamp (custom 2014 epoch) + 128 random bits. Segment's design. Second resolution means it will not order events inside the same second β€” fine for records, wrong for a high-rate feed cursor.

MongoDB ObjectId

12 bytes: 4-byte second timestamp + 5 random bytes per process + a 3-byte counter. The 5 random bytes replaced the old machine-ID + PID scheme, which leaked host details and collided in containers. Same three ingredients as Snowflake, in 96 bits.

Instagram (in-database)

64 bits: 41-bit ms timestamp + 13-bit logical shard ID + 10-bit sequence, minted by a PL/pgSQL function inside Postgres. Embedding the shard ID means any ID tells you which shard to query β€” no lookup table. Proof you may not need a separate ID service at all.

🧭 Which one should you actually pick?
You need…ChooseBecause
A sharded OLTP primary key, minimum storageSnowflake8 bytes instead of 16 β€” it multiplies across every secondary index and foreign key.
A primary key with the least operational burdenUUID v7Time-sortable and coordination-free. No node registry, no drift runbook, no epoch cliff.
An ID generated offline on a client or mobile deviceUUID v7 / ULIDEnough entropy to be safe without ever contacting a server to claim a node ID.
A sortable cursor for an infinite feedSnowflake / ULIDMillisecond resolution orders events inside a second; KSUID's 1-second granularity cannot.
A key in an object store or a filenameULIDThe 26-char text form sorts lexicographically, so listings come back in time order.
A public-facing ID in a URLOpaque random IDNever expose a sortable internal key β€” see the security warning below.
πŸ” Every Time-Ordered ID Leaks Business Intelligence

A Snowflake ID is reversible. Anyone holding one can extract the exact millisecond the record was created β€” no key required, just a right-shift. Worse, IDs from two points in time reveal your growth rate: sample /orders/<id> on Monday and Friday, subtract, and a competitor knows how many orders you processed that week. This is the German tank problem, and sequential IDs make it trivial.

Sequential IDs also invite enumeration attacks: /invoice/1002 implies /invoice/1001 exists, turning one weak authorization check into a full data breach.The standard fix: keep the sortable Snowflake as your internal primary key, and expose a separate opaque public ID β€” a random UUIDv4 or a signed slug β€” in URLs and API responses. Internal locality and external opacity are different jobs; use two columns.

πŸ’» 6. Production Snowflake Generator (Multi-Language)

A complete, thread-safe generator in 5 languages. Every version implements the same four essentials: named bit constants, a lock, the sequence-rollover busy-wait, and the clock-drift refusal.

Select Language:
# Python 3.10+ β€” thread-safe Twitter Snowflake generator
import time, threading

class SnowflakeGenerator:
    EPOCH_MS       = 1288834974657   # 2010-11-04T01:42:54.657Z (Twitter's epoch)
    SEQUENCE_BITS  = 12
    NODE_BITS      = 10
    NODE_SHIFT     = SEQUENCE_BITS               # 12
    TS_SHIFT       = NODE_BITS + SEQUENCE_BITS   # 22
    SEQ_MASK       = (1 << SEQUENCE_BITS) - 1    # 4095
    MAX_NODE       = (1 << NODE_BITS) - 1        # 1023

    def __init__(self, node_id: int):
        if not 0 <= node_id <= self.MAX_NODE:
            raise ValueError(f"node_id must be 0..{self.MAX_NODE}")
        self.node_id  = node_id
        self._seq     = 0
        self._last_ts = -1
        self._lock    = threading.Lock()   # the GIL alone is NOT enough

    def _now(self) -> int:
        return int(time.time() * 1000)

    def next_id(self) -> int:
        with self._lock:
            ts = self._now()
            if ts < self._last_ts:                    # CLOCK DRIFT GUARD
                raise RuntimeError(f"clock moved back {self._last_ts - ts}ms")
            if ts == self._last_ts:
                self._seq = (self._seq + 1) & self.SEQ_MASK
                if self._seq == 0:                   # all 4096 slots used this ms
                    ts = self._til_next_ms(self._last_ts)
            else:
                self._seq = 0
            self._last_ts = ts
            return ((ts - self.EPOCH_MS) << self.TS_SHIFT) \
                 | (self.node_id << self.NODE_SHIFT)  \
                 | self._seq

    def _til_next_ms(self, last: int) -> int:
        ts = self._now()
        while ts <= last:
            ts = self._now()
        return ts
πŸ”‘ The 3 Things Everyone Gets Wrong
1. The shift arithmetic. TS_SHIFT is 22, not 41 β€” a field shifts left by the number of bits below it (10 + 12), never by its own width.
2. The busy-wait is mandatory. When the sequence wraps to 0 you must spin to the next millisecond. Skip it and you silently re-issue the 4,096 IDs you just handed out.
3. JavaScript needs BigInt and a string. Regular JS numbers cannot hold a 19-digit ID. Use BigInt internally and return .toString() at every boundary.

πŸ§ͺ 7. Interactive Lab: Live 64-Bit Snowflake Visualizer

⚑ Real-Time Bit-Level ID Generator

Generate, Decode & Break a Snowflake ID

Mint real IDs and watch all 64 bits light up!
64-Bit Layout (bit 63 β†’ bit 0)1 Sign41 Timestamp10 Node12 Sequence
Decoded FieldsTotal generated: 0
Generated ID (decimal)
β€”
Timestamp (41b)
β€”
β€”
Node ID (10b)
β€”
range 0–1023
Sequence (12b)
β€”
range 0–4095
42
πŸš€ Generator Ready on node 42. Click ⚑ Generate ID to mint your first Snowflake!
Recent IDs (newest first)watch only the low bits change
No IDs generated yet.

πŸ’‘ What to notice: consecutive IDs differ only in their last few digits β€” the timestamp and node bits are identical, so every insert lands right next to the previous one in the B+Tree. That is the index locality Snowflake buys you.

🎯 8. Test Your Understanding

Scenario 1: The Midnight Duplicate Key Storm

At 02:00 your ID service starts throwing duplicate-primary-key errors for 700 milliseconds, then recovers on its own. The only correlated event in the logs is an NTP synchronization. What happened, and what is the fix?

Scenario 2: The Vanishing Last Digits

Your React app lists orders. Two clearly different orders display the same ID, and clicking one opens the wrong record. You curl the same endpoint and the JSON is completely correct. Where is the bug?

Scenario 3: The Kubernetes Scale-Down

You derive each generator's node ID from its StatefulSet ordinal (gen-0 β†’ node 0, and so on). An autoscaler scales 8 β†’ 3 β†’ 8 pods within a minute, and duplicate IDs appear. Why?

β—‹

Completed this module?

Mark as finished to update your curriculum progress tracker.