Concurrency Patterns

Separate task coordination from simultaneous execution

A web server may concurrently manage thousands of connections while only a bounded worker pool performs CPU work in parallel. Naming both limits prevents “one task per request” from becoming an accidental resource policy.

Trace a lost update one operation at a time

The expression counter = counter + 1 is a read, a calculation, and a write. Two workers can both read 0, both calculate 1, and both write 1. No instruction is corrupted, but one logical increment disappears.

Choose an attempted interleaving and a synchronization strategy. The lab shows the exact operation order, final counter value, and any waiting or retry work.

The unsafe schedule is deterministic evidence of the bug, not a claim that a scheduler will choose that order every time. A race detector can report conflicting accesses that actually execute, but passing one run cannot prove that every possible interleaving is safe.

Choose synchronization from the invariant

The following Java example protects one counter with one lock. The lock surrounds the whole read-modify-write operation, and finally guarantees release.

Pick a concurrency model with an ownership rule

• Threads with shared memory can run CPU work in parallel and share objects directly. Every shared mutable object still needs a documented synchronization contract.
• Event loops and async tasks multiplex waiting work with few threads. Blocking a loop delays unrelated tasks, so CPU work and blocking libraries need an explicit offload boundary.
• Actors and single-owner loops isolate mutable state behind messages. Mailbox ordering, bounded capacity, cancellation, and restart behavior become part of the protocol.
• Channels and pipelines make handoff visible between stages. Channel capacity, closure ownership, fan-out, and fan-in determine whether the pipeline drains or stalls.

Prefer the model that makes ownership and cancellation easiest to explain. Syntax such as async, a goroutine, or a thread constructor does not choose a safe lifecycle for you.

Turn worker capacity into a backpressure decision

A queue absorbs a temporary mismatch between arrivals and service. It cannot make a sustained mismatch disappear. For identical workers:

service capacity = worker count × 1000 / service time in milliseconds

The lab runs eight one-second windows. In every window, workers take up to their exact service capacity, then the selected policy places, blocks, or rejects the remainder. Change the arrival rate, worker count, service time, queue capacity, and overload policy; inspect every second rather than trusting an average.

A bounded queue keeps memory bounded. It does not make the system healthy by itself: blocking transfers waiting to the caller, while rejection requires a retry, fallback, or explicit error contract.

Implement the overload contract at admission

Java's BlockingQueue API exposes distinct full-queue behaviors: insertion can throw, return a special value, wait, or wait with a timeout. The caller must deliberately choose which behavior matches the product contract.

This example configures two workers and a queue of two tasks. The first two tasks hold the workers, the next two occupy the queue, and the fifth submission is rejected.

Before production, replace demo constants with measured service-time distributions and test bursts, slow dependencies, retries, cancellation, and shutdown. Average service time alone does not predict tail latency.

Prevent deadlock and other failures of progress

Correct values are not enough; the system must continue making progress.

• Deadlock: tasks form a wait cycle. Prevent it with one global lock order, or remove simultaneous ownership of multiple locks.
• Livelock: tasks keep changing state but repeatedly undo one another's progress. Bound retries and add randomized or coordinated backoff where appropriate.
• Starvation: one task waits indefinitely while others repeatedly win access. Inspect scheduler, queue, lock-fairness, and priority behavior instead of assuming equal service.
• Leak: a task outlives its request, owner, or cancellation scope. Propagate cancellation and wait for child work during shutdown.

For a two-lock operation, write the order as an invariant such as “always acquire account ID 17 before account ID 42.” A timeout detects excessive waiting; it does not make a partially completed operation atomic.

Test schedules, limits, and lifecycle boundaries

Use several forms of evidence because each finds a different class of failure:

1. run deterministic tests that force known harmful interleavings;
2. run the language or runtime race detector on realistic exercised paths;
3. stress cancellation, timeout, and shutdown while work is queued and running;
4. record queue depth, admission outcomes, active workers, lock wait time, task age, retries, and completion latency;
5. inject dependency slowdown and confirm backpressure reaches the intended boundary;
6. capture deadlock diagnostics and thread or task dumps before restarting a failed process.

Treat “no race was observed” and “no request failed” as observations, not proofs. The design proof is the ownership, happens-before, capacity, and lifecycle contract.

Verify semantics against primary documentation

The examples and claims in this lesson follow current runtime documentation:

• the Go memory model defines data races and synchronization through channels, locks, and atomic operations;
• Go's data race detector documents what -race observes and its exercised-path limitation;
• Java BlockingQueue distinguishes throwing, special-value, blocking, and timed queue operations;
• Java ThreadPoolExecutor defines bounded work queues and rejected-execution handlers;
• Java ThreadMXBean exposes deadlock detection for monitors and ownable synchronizers.

Pin the language and runtime version in operational documentation. Memory-model, diagnostic, scheduler, and executor details are runtime contracts, not folklore.