Rate Limiter Workbench
How do you protect a backend without making healthy callers share an abusive caller's fate?
A rate limiter enforces a time-bounded admission contract for a specific identity. Its algorithm controls when budget is spent, its identity key controls fairness, and its failure policy decides whether limiter availability or backend safety wins.
Step 1
Model
Choose the algorithm, request budget, time window, burst or queue capacity, identity key, traffic mix, retry behavior, regions, counter lag, backend capacity, and fail-open or fail-closed behavior.
Step 2
Observe
Trace tokens or windows, admitted and rejected requests, queueing, caller fairness, retry amplification, distributed overshoot, response headers, backend pressure, and the user-visible consequence.
Step 3
Challenge
Compare a launch burst, abusive tenant, stale distributed counter, retry storm, counter-store outage, and controlled recovery. Keep the backend inside its tested capacity without throttling healthy tenants unnecessarily.
Traffic protection workbench
Make the admission contract visible
Tune the limiter, shape caller behavior, and inject distributed failures to trace exactly who is admitted, delayed, or rejected.
Challenge the healthy policy
Each preset changes traffic or shared-state health. Policy choices stay under your control, so the same outage can be tested fail-open and fail-closed.
Define the admission policy
Algorithm, rate, burst state, and identity decide the budget each caller can consume.
Algorithm
Budget assigned to each identity.
6.0 req/s sustained per identity.
Tokens initially available above sustained refill.
Identity key
The boundary that receives one independent budget.
Token bucket: tokens available
650 units remain across 20 tenant budgets. Absorbs short bursts while preserving a sustained rate.
Shape callers and distributed state
Demand, identity skew, retries, regions, and synchronization change both fairness and backend pressure.
One tenant is modeled as the noisy caller.
70% of rejected requests retry per configured attempt.
Counter-store failure
Used when the Store outage challenge is active.
Retry amplification
+0
extra attempts over 10s
Distributed overshoot
0
requests admitted on stale evidence
Request path | Healthy baseline
Trace admission, rejection, and backend load
Values cover a deterministic 10-second observation. Queue depth is accepted work waiting to drain; it is not backend throughput.
Callers
1,050 attempts
1,050 original + 0 retries
Token bucket
1,050 admitted
20 tenant budgets; 0 queued
Protected backend
105.0 req/s
70% of tested capacity
Admission result
No requests rejected
Every caller remains within the configured admission contract.
Admitted
1,050
100.0% of attempts
Rejected
0
No client throttling
Queue
0
No admitted request is waiting
Backend load
70%
105.0 of 150 req/s
See demand collide with the budget
Each bar is one second. Violet is admitted work; rose is rejected demand. The burst preset raises seconds four and five.
Inspect noisy-neighbor isolation
Compare completion rates directly; a single global budget cannot protect normal tenants from an abusive caller.
Normal tenants
987 of 987 attempts
100.0%
Noisy tenant
63 of 63 attempts
100.0%
All callers are within contract
No caller group needs to consume another group's budget in this observation.
Return a usable client contract
Clients need an explicit limit, remaining budget, reset time, and retry delay. Add jitter and cap retries.
- HTTP status
- 200
- RateLimit-Limit
- 60;w=10
- RateLimit-Remaining
- 8
- RateLimit-Reset
- 10
- Retry-After
- not sent
Retry after the advertised delay with exponential backoff and jitter. Immediate retries convert intentional load shedding into a retry storm.
Read the operational consequence
The limiter is correct only when its availability policy, identity boundary, and backend budget match the product contract.
Protection contract holds
The limiter preserves backend headroom and callers complete without throttling.
Shared-state verdict
Synchronized
2 regions | 50ms propagation
Recovery guidance
Ramp, do not release
Refill budgets gradually, expire retry backlogs, and verify counters converge before restoring full admission.