Design Distributed Cache
Design a file storage system like Dropbox or Google Drive: sync, sharing, and versioning.
What is a distributed cache?
A distributed cache keeps frequently requested key-value data in memory across multiple machines. It reduces latency and protects a slower source of truth when one machine cannot hold the working set or serve the request rate.
Distribution creates the real design problem. Every key needs one known owner for the current cluster version, useful replicas must survive failures, and a cache miss must not become an unlimited request to the backing store. The core invariant is:
For one ring version, each key has deterministic primary ownership, and every miss, retry, migration, and failover path has a strict work bound.
This walkthrough designs a 2 TB, read-heavy cache at 1.5 million peak operations per second. Review Caching Strategies for cache-aside and write policies, Consistent Hashing for the partitioning algorithm, and CAP Theorem for partition trade-offs.
Ownership
Route deterministically
Clients or proxies hash a key against one versioned ring and reach its primary shard without broadcasting to the cluster.
Protection
Bound the miss path
Request coalescing, origin admission limits, TTL jitter, and stale-value policies keep cache loss from becoming a backing store outage.
Operations
Recover deliberately
Replica promotion, fenced ownership, gradual range movement, and warm-up controls make membership changes observable and reversible.
Begin with the cache contract
A distributed cache is not automatically authoritative storage. First establish which values may be stale or lost, which operations users observe, and what the application does when the cache is unavailable.
Functional
Core operations
GET(key)returns a value, metadata, or a miss.SET(key, value, ttl, version)stores a bounded value with expiration.DELETE(key, version)invalidates a value without reviving an older write.MGET(keys)groups independent lookups while preserving per-key results.
Non-functional
Service promises
- Sustain 1.5M operations/s peak with an 80/20 read-write mix.
- Keep read P95 below 2 ms and write P95 below 4 ms inside a region.
- Target 99.99% read availability and 99.9% write availability.
- Recover one failed node without an unbounded backing store surge.
Consistency
Correctness boundaries
- Derived objects may be eventually consistent for up to 100 ms.
- Session namespaces require read-your-writes within the serving region.
- Versions and tombstones prevent delayed replicas from reviving old values.
- Locks, balances, and durable counters remain outside the cache contract.
Out of scope
First-version boundary
- No arbitrary server-side queries or joins.
- No cross-key transaction guarantee.
- No promise that cache data survives a total regional loss.
- No values above 1 MB; large objects belong in object storage or a CDN.
Questions that change the architecture
- Is the cache disposable? If a value is the only copy, the design is a database and needs a different durability review.
- What is the staleness budget by namespace? Product catalogs may accept seconds; authentication and session state may not.
- What is the access distribution? Averages hide hot keys, scans, and synchronized expiration waves.
- What is the failure unit? Replicas on the same host, rack, or zone do not protect against that shared failure.
- What happens on a miss? Define origin deadlines, retry budgets, stale serving, admission limits, and user-visible degradation before choosing software.
State one namespace contract in the interview: for example, product summaries use cache-aside with 60-second TTLs, versioned invalidation, asynchronous replicas, and a 150K reads/s global origin fallback budget.