What is System Design?
Learn the fundamentals of system design: core concepts, goals, and why it matters for building scalable applications.
What is system design?
System design is the process of turning a product need into a technical plan that keeps working under real constraints. It names the parts of a system, the data each part owns, the path a request follows, and the behavior users can expect when traffic rises or a dependency fails.
In plain language, a product requirement says what a person needs to do. System design explains how the software can do it reliably, quickly, safely, and at a cost the team can operate.
The first invariant is simple: protect the user action and the facts that must remain true. Technology is a means to that end, not the starting point. A good design makes its assumptions, limits, and trade-offs visible.
Before this lesson, it helps to know that a request is one attempt by a client to use a service, and that a database is a system that stores authoritative records. This lesson introduces the decisions that connect those building blocks.
The forces a design must balance
Every design spends complexity to protect something important. Name the priority before selecting an implementation, because no architecture maximizes every quality at once.
Growth
Scalability
Keep useful performance as requests, users, and stored data increase. Scale can mean less work per request, more workers, or data split by a stable key.
Failure
Reliability
Keep the promised behavior available when a process, machine, network link, or dependency fails. Isolation, redundancy, and rehearsed recovery make this credible.
Speed
Performance
Meet response-time and throughput goals for the real workload. Tail latency matters because a small group of slow requests can shape the user experience.
Trust
Security
Protect identities, data, and capacity with validation, least privilege, encryption, audit trails, and abuse controls placed before expensive work.
Maintainability and cost apply across all four. A design that reaches a latency target but cannot be observed, restored, or afforded is incomplete.
Start with constraints, then choose the shape
A diagram is an answer, not the first question. First identify the dominant constraint: a small product may need fast iteration; a read-heavy product may need to protect the database; a critical write path may need a clear authority for the record.
Choose a scenario in the lab, then select a component in the map. The active route and result show why the architecture changed. The point is not to collect components; it is to add the smallest mechanism that protects the stated constraint.
Begin with a modular monolith when one deployable unit can meet the workload. Add a load balancer when one worker is the limit, a cache when repeated reads are the limit, and replicas or partitions only when their consistency and operational costs are justified.
Turn vague scale into a small estimate
Scale is not only the number of registered users. A useful first estimate states the peak request rate, read-to-write ratio, response-size range, latency target, and how much traffic can be lost before users notice.
For example, a redirect service receiving 60,000 requests per second at peak with a 92% cache hit rate sends about 4,800 requests per second to its authoritative store: 60,000 x (1 - 0.92). The store must be tested for a colder cache, not only this warm average.
60K/s
Peak requests
The demand entering the system
92%
Cache hit rate
A warm-cache assumption, not a guarantee
4.8K/s
Store fallbacks
The derived miss-path load
8K/s
Protection threshold
The tested fallback envelope
Use the control below to model a cache regression or a growing working set. It preserves the 60,000 requests per second total, recomputes both branches, and turns into a warning when the authoritative store crosses its tested fallback limit.
The lesson is not "maximize cache hits." A longer cache lifetime can return data that is too stale for inventory, price, or permissions. Choose the freshness rule first, then provision the miss path for the lowest acceptable hit rate.
Trace one critical request before expanding the system
The critical path is the sequence that must complete before the user can receive a correct answer. For a URL shortener, redirection is usually critical; analytics collection is valuable but can happen after the redirect.
A simple read path with deliberate boundaries
A request should cross only the components required to satisfy the user action. Non-critical work belongs off the response path when correctness allows it.
1. Ask
Client request
The client asks for a short link and expects a redirect within its latency budget.
2. Decide
Stateless service
The service validates the key, applies rate limits, and owns the response contract.
3. Read
Cache or record store
The cache serves reusable mappings; a miss reads the authoritative record.
4. Observe
Async analytics
Click events are queued after the redirect so analytics trouble does not block the user.
When tracing a path, ask three questions:
- Which component owns the authoritative fact?
- What deadline and fallback apply if the next dependency is slow?
- Which work can move out of the request without changing the promised user result?
Design for failure as part of normal behavior
Failures are ordinary operating conditions. A timeout does not prove a write failed, a replica is not a backup, and retrying an unsafe request can create a duplicate side effect. Define what the user sees and which component owns recovery before an incident does it for you.
1 Contain
Set a deadline
Give dependency calls a bounded time budget so one slow component cannot consume the entire request.
2 Degrade
Choose a safe fallback
Return a cached, limited, queued, or explicit unavailable result only when it preserves the product's invariant.
3 Restore
Recover with evidence
Use health checks, failover, idempotency keys, backups, and tested restore procedures for the failure you actually observed.
Useful failure rules include:
- Timeouts and retries: retry only bounded, idempotent work. Add jitter so many clients do not retry together.
- Replica lag: route read-after-write confirmation to the primary or expose a freshness wait; do not pretend an asynchronous replica is immediately current.
- Data loss and corruption: maintain backups separately from high availability, then practice a restore to prove the recovery objective.
- Dependency outages: isolate optional work with queues or fallbacks so it cannot block the critical path.
Make trade-offs and operations explicit
System design is a set of reversible and irreversible commitments. Write down what each choice buys, what it costs, and which signal would make the team revisit it.
Simplicity versus distribution
A single deployable unit is easier to change and debug. More services can isolate scale or ownership, but add network failures and coordination work.
Freshness versus load
Caches reduce store work, but invalidation and bounded staleness become product decisions. The miss path still needs capacity.
Availability versus consistency
Some facts, such as a balance or inventory reservation, need strict coordination. Other views can lag if the user-facing rule says so.
Speed versus protection
Rate limits and validation add work, but rejecting abusive or invalid input early preserves capacity and protects authoritative data.
Redundancy versus recovery
Replicas improve availability. Backups and restore drills protect against deletion and corruption. Plan for both when the data matters.
Features versus operability
Logs, metrics, traces, alerts, runbooks, and capacity tests let operators prove a design is healthy and recover it when it is not.
Before shipping, name the operating signals: request rate, error rate, p95 or p99 latency, saturation, queue depth, cache hit rate, replica lag, and recovery success. Alerts should point to an owner and a usable response, not merely report that a graph changed.