Rules of Thumb
Essential rules of thumb for system design: capacity planning, performance estimation, and back-of-envelope calculations.
What are system-design rules of thumb?
System-design rules of thumb are approximate numbers and relationships used to build a first model before representative measurements exist. They help turn a vague product statement such as “millions of users” into explicit estimates for requests per second, bandwidth, storage, fleet size, and latency headroom.
They matter because architecture decisions need quantities before every detail is known. A useful heuristic makes its unit, workload, evidence, and uncertainty visible. It is not a production guarantee. The core rule is: estimate to find the sensitive assumptions, then replace those assumptions with measurements before committing capacity or an SLO.
Anchor estimates in unit facts
Some planning anchors are definitions or conversions. They are safer to reuse than a remembered claim about how many requests a particular server or database can handle.
86,400
Seconds per day
Divide daily operations by this value to estimate average operations per second.
8
Bits per byte
Convert bytes per second to bits per second before comparing traffic with a network link.
125 MB/s
1 Gbit/s ceiling
Theoretical decimal conversion before protocol overhead, contention, and a safe utilization target.
10x
Order of magnitude
A useful scale boundary, not permission to round away a capacity risk.
Keep units attached to every term
8.64 million operations/day / 86,400 seconds/day = 100 operations/secondon average.100 operations/second x 5 peak factor = 500 peak operations/secondfor the modeled busy window.500 responses/second x 4 KB/response x 8 bits/byte = 16 Mb/sbefore protocol overhead.200 writes/second x 1 KB/write x 86,400 seconds/day = 17.28 GB/daybefore replicas, indexes, compression, and backups.
Writing the units makes dimensional mistakes visible. It also prevents an average daily rate from being presented as a peak capacity requirement.
Build one traceable workload chain
A back-of-the-envelope estimate should form a chain in which every output can be traced to a product input or an explicit engineering assumption.
1 People
Describe active demand
Start with the relevant active-user cohort, not the largest registered-user number available.
2 Behavior
Count operations
Multiply active users by actions per user, then separate reads, writes, and payload shapes.
3 Peak
Model the busy window
Apply a peak factor or use a known peak interval. Keep the average beside it as a sanity check.
4 Capacity
Map demand to resources
Use measured per-node throughput, a utilization target, and explicit failure reserve to size the first fleet.
Lab 1: couple users, behavior, traffic, and capacity
Change the active cohort, actions, read share, peak shape, payload, and capacity evidence. The lab keeps the arithmetic visible while request rate, bandwidth, write volume, and fleet size move together.
Preserve the calculation as reviewable evidence
The exact language is secondary to the contract: name every assumption, preserve units, round resources up, and return the intermediate values reviewers need to challenge.
Separate a planning estimate from a capacity promise
“One server handles 1,000 requests per second” is not a universal rule. The result changes with request mix, payload, CPU work, cache state, dependencies, concurrency, latency target, runtime limits, and failure conditions. Treat a remembered throughput number as a placeholder until a representative benchmark replaces it.
Starting point
Heuristic
A remembered or analogous number gives the first order of magnitude. Record its source and use a wide range.
Causal estimate
Model
Workload inputs and unit conversions expose which assumption dominates the result and what must be measured next.
Decision evidence
Benchmark
A representative test measures the target request mix, percentile SLO, failure reserve, and resource saturation together.
Express uncertainty as a range
- Use low, expected, and high cases when an input is not measured.
- Change one assumption at a time to find the terms that move the answer most.
- Add headroom for bursts, forecast error, background work, and a stated failure scenario.
- Round resource counts up only after applying the utilization target and reserve policy.
- Tighten the range with telemetry or a benchmark; do not hide uncertainty with extra decimal places.
Challenge the estimate before it becomes architecture
A sanity check asks whether the answer is dimensionally valid, plausible at the right order of magnitude, consistent with an independent bound, and safe enough for the consequence of being wrong. The evidence required for an inexpensive experiment is lower than the evidence required for durability, correctness, or an external SLO.
Lab 2: review confidence, contradictions, and downside
Choose a planning claim, the strongest evidence available, and the consequence of underestimating. The review desk widens or narrows the range, checks independent bounds, and recommends whether to explore, stage, measure, or reject the point estimate.
A precise answer can still be wrong. If the units fail, an independent bound contradicts the claim, or the benchmark omits the launch workload, do not repair the estimate by adding an arbitrary safety factor. Fix the model or collect the missing evidence.
Use a repeatable estimation review
Before calculating
- Define the user-visible operation and the time window.
- Name average and peak separately.
- Separate reads, writes, payload classes, and background work.
- State retention, replication, compression, and failure assumptions.
Before committing capacity
- Verify units and recompute the answer with an independent method.
- Compare low, expected, and high cases.
- Benchmark the representative request mix at the target latency percentile.
- Check the fleet after a node, zone, or dependency loss named by the reliability goal.
- Record which telemetry will replace each uncertain assumption after launch.
When a remembered number is still useful
Use it to reject impossible orders of magnitude, identify the likely bottleneck, or choose what to benchmark first. Do not use it as a vendor comparison, a universal database limit, or a final procurement number. For deeper unit modeling, continue to Data Sizes; for end-to-end response budgets, continue to Latency Numbers.
Sources and further reading
- Google SRE: Handling Overload explains why capacity limits, admission control, and graceful overload behavior must be measured together.
- Google SRE: Service Level Objectives connects user-facing targets to measurable service behavior.
- AWS Well-Architected: Performance Efficiency emphasizes evidence, monitoring, and revisiting architecture as requirements change.
- Latency Numbers Every Programmer Should Know presents representative approximate numbers with historical context.