Twitter Snowflake ID Generator
Design a distributed unique ID generator like Twitter Snowflake: requirements, algorithms, and trade-offs.
What is a distributed unique ID generator?
A distributed unique ID generator gives each new order, message, upload, or event an identifier that no other generator can issue. It matters because a database sequence is simple but can become a shared dependency, while random strings often cost more storage and do not sort naturally.
In plain language, a Snowflake-style generator makes an ID from three coordinates: when it was created, which generator created it, and which request it was within that millisecond. Its invariant is precise: no two live generators may emit the same timestamp, generator identity, and sequence tuple.
This case study designs a positive 64-bit, roughly time-sortable ID for a multi-region product. You will turn requirements into a bit budget, make the request path independent, then test the clock and identity failures that can still break uniqueness.
Define what an ID promises
An ID is not a database row and it is not automatically a security token. Start by deciding what callers may rely on, because that determines whether a local, sortable integer is the right tool.
Functional requirements
- Issue one unique numeric ID for every accepted create request.
- Keep IDs roughly ordered by creation time so common index and range-scan patterns remain efficient.
- Generate locally with no database or consensus call on the normal request path.
- Decode an ID during investigation to recover its timestamp, generator identity, and sequence position.
Non-functional constraints
- Scale: 1,000 active generators, each able to absorb a burst of 50,000 IDs per second.
- Latency: in-process generation normally completes well below 1 ms.
- Lifetime: retain a format that lasts at least 30 years from a chosen custom epoch.
- Availability: a bad generator may fail closed; it must never preserve availability by risking a duplicate.
"Roughly ordered" means IDs from healthy clocks usually increase with time. It does not mean a globally serial order across regions, nor does it establish the business order of two concurrent writes.