Event-Driven Architecture | System Designer

What is event-driven architecture?

Event-driven architecture (EDA) organizes a system around events: durable descriptions of facts that have already happened. A producer publishes a fact such as OrderCreated; one or more consumers react later without the producer calling each consumer directly.

In plain language, the order service says, "order 42 was accepted," then continues. Inventory, payment, notifications, and analytics can each respond on their own schedule. This removes those reactions from the checkout call path, but it also introduces eventual consistency, duplicate delivery, ordering boundaries, and asynchronous recovery.

Core invariant

An accepted business fact has one stable event identity and a durable path from its source of truth to every required consumer. Delivery may repeat, so each consumer must make the same event converge on one business effect.

Review communication patterns if commands, queues, and publish-subscribe are new. Scalability basics provides the throughput and storage vocabulary used in the capacity lab.

Read an event as a contract, not a callback

A command asks an owner to decide: CreateOrder. It can be accepted or rejected. An event reports the result after the owner commits it: OrderCreated. Consumers must not reinterpret that event as a request that might never have happened.

The CloudEvents specification offers a vendor-neutral envelope. Its required context includes id, source, specversion, and type; application data remains separate from delivery metadata.

Deduplication

Identity

The producer assigns one id for one distinct fact. A retry republishes the same identity, allowing consumers to recognize duplicate delivery.

Domain contract

Meaning

The event type names a fact in past tense. A versioned name or schema gives independently deployed consumers a compatibility boundary.

Entity scope

Subject and order

An aggregate key such as order-42 identifies the entity whose events require local ordering. Global order is rarely necessary or affordable.

Traceability

Causation context

Correlation and causation identifiers connect an asynchronous chain to the user action or prior event that started it.

Trace the smallest reliable publish path

The producer owns the business decision. The broker owns transport and retention. Each consumer owns its local effect. A transactional outbox connects the first two without requiring one distributed transaction across the database and broker.

1
Synchronous boundary
Decide the command
Validate CreateOrder, enforce the order invariant, and choose one authoritative order ID while the caller waits.
2
One transaction
Commit state and intent
Write the order and its OrderCreated outbox record together. Either both commit or neither does.
3
Retryable handoff
Publish from the outbox
A relay sends committed outbox records to the broker. A crash can cause another send, so the event keeps the same ID.
4
Independent consumers
Apply local effects
Each consumer records the event ID with its own state change, then acknowledges delivery. One slow consumer does not enter checkout latency.

Size fan-out and retention as different costs

Published event rate, payload size, subscriber groups, retention, and partitions answer different capacity questions. Adding a subscriber usually does not create another retained copy of the event in a log, but it does create another read and processing path. Increasing retention consumes storage without increasing current event rate.

Change the inputs below. First create an overloaded partition plan, then recover headroom without changing the event rate. Next add subscriber groups and observe which totals change.

Fan-out capacity lab

Separate ingest, delivery, retention, and partition pressure

Change the workload and observe four different costs. A broker stores each retained event once per replica, while every independent subscriber group creates another logical delivery path.

Loading event capacity model...

Choose the event shape before choosing a product

EDA is an architectural style, while pub/sub, retained streams, event sourcing, and CQRS solve different problems inside or alongside it.

Who receives it?

Publish-subscribe

Each subscription receives a copy of a published event. Use it when independent capabilities react to the same fact and do not require one shared worker queue.

Can it replay?

Retained event stream

Events remain in an ordered log for a retention window. Consumer groups track positions, enabling catch-up and replay within the ordering scope of a partition.

What is authoritative?

Event sourcing

The event sequence is the source of truth for an aggregate, and current state is rebuilt by replay. This is a persistence model, not a requirement for every event-driven system.

Which model serves reads?

CQRS

Commands update a write model while queries use a separate read model. Events often build projections, but CQRS can exist without event sourcing and adds consistency lag to manage.

Inject failures where state crosses boundaries

Asynchronous systems fail between durable steps. Switch among the scenarios, trace the active path, and select components to inspect ownership. Notice that an outbox closes the database-to-broker gap, while idempotency closes the broker-to-effect gap; neither pattern replaces the other.

Make delivery, ordering, and evolution explicit

Treat delivery and business effect separately

A broker can redeliver after a timeout, lost acknowledgement, consumer restart, or replay.
A consumer should record the event ID and its business mutation in one local transaction.
Acknowledge only after that transaction commits. Bound transient retries, then preserve persistent failures for controlled recovery.
Read "exactly once" claims at the full boundary. A broker transaction does not automatically make an external payment API or database mutation happen exactly once.

Order only where the domain requires it

Partition by the entity whose transitions must stay ordered, such as orderId.
Include an aggregate version when a consumer must reject stale or skipped transitions.
Avoid global order unless the domain truly requires one serial stream; it limits parallelism and enlarges the failure domain.
Design replay as an operation with a start position, rate limit, destination, and proof that repeated effects remain safe.

Evolve schemas for independently deployed consumers

Prefer additive changes: add optional fields and preserve the meaning of existing fields.
Introduce a new event type or explicit version for an incompatible semantic change.
Test old consumers against new producer events and new consumers against retained historical events.
Keep secrets and unnecessary personal data out of broadly visible events; retention and replay extend exposure.

Implement the two local transactions

This dependency-free Python example uses SQLite to make both correctness boundaries visible. create_order commits domain state with an outbox record. apply_payment commits a processed-event marker with the payment effect. The relay and broker may deliver twice, but the charge remains singular.

transactional-outbox.py

Operate lag and recovery as product behavior

Monitor the path from the business commit to each required effect, not only broker health.

Producer boundary

Publication age

Track the oldest unpublished outbox record and publish failures. A healthy broker does not reveal an event stranded before it.

Freshness

Consumer lag

Measure both position lag and age of the oldest unprocessed event. A small count can still violate a time-sensitive business deadline.

Recovery ownership

Failure inventory

Alert on retry exhaustion and recovery-queue age. Preserve event ID, schema, error, attempts, and correlation context.

Business correctness

Effect reconciliation

Compare accepted orders with required downstream effects. Transport metrics alone cannot prove that inventory, payment, or notification is correct.

Create a replay runbook before an incident: define who authorizes it, which event range is eligible, how throughput is capped, how duplicate effects are prevented, and how completion is reconciled.

Know when not to use event-driven architecture

EDA is useful when several capabilities react independently, work can finish after the initiating request, bursts need buffering, or retained facts must be replayed. It is a poor default when a direct request-response call already meets the need, the caller requires an immediate strongly consistent answer across every participant, or the team cannot yet operate lag, retries, schema compatibility, and replay safely.

Use synchronous and asynchronous paths together. Keep the smallest user-facing decision synchronous; publish committed facts for work that can proceed independently.

Review the production contract

Before shipping one event flow, write down:

The fact owner, event type, stable identity, subject key, and schema compatibility policy.
The atomic path from business commit to publish intent, including relay retry behavior.
Every subscriber group, its local idempotency boundary, acknowledgement point, and ordering scope.
The event-rate, payload, fan-out, retention, partition, and operating-headroom assumptions.
The retry budget, recovery destination, replay authorization, and reconciliation proof.
Publication age, consumer age, failure inventory, correlation fields, and business-effect alerts.

Use these official references for the underlying contracts:

CloudEvents 1.0.2 specification for the portable event envelope.
Apache Kafka delivery-semantics design for producer, consumer, and transaction boundaries.
AWS transactional outbox guidance for the dual-write failure and outbox pattern.
Microsoft event-driven architecture guidance for architecture variants, suitability, and operational challenges.

No quiz questions available

Could not load questions file