Cloud Native Technologies
Master cloud-native technologies: CNCF landscape, patterns, best practices, and architecture.
What is cloud-native architecture?
Cloud-native architecture builds and operates small, replaceable workloads that can be deployed automatically, observed continuously, and recovered without treating any one machine as permanent. Containers and Kubernetes are common tools for this model, but installing them does not make an application cloud-native.
In plain language, the application describes its desired state, the platform keeps working toward that state, and teams use measured feedback to release and recover in small steps. The goal is not the largest possible collection of services. The goal is to change software frequently while keeping failures bounded and repairable.
The core invariant is: a workload instance is disposable, but its contract, state, and operating evidence must survive replacement. Store durable data outside an individual container, declare configuration, automate reconciliation, and make every release observable and reversible.
Treat cloud-native as an operating model
Cloud-native systems combine four responsibilities. Missing one usually turns the architecture into fragile distributed software with more infrastructure to manage.
Immutable workload
Package
Build a versioned image once and promote the same artifact through environments. Keep runtime configuration and credentials outside the image.
Desired state
Declare
Describe replicas, resources, routing, policy, and rollout intent in version control. Controllers reconcile observed state instead of relying on one-off commands.
Measured behavior
Observe
Correlate metrics, logs, and traces with service, version, region, and request identity. Use service objectives to decide whether a change may continue.
Bounded failure
Recover
Spread replicas, bound retries and queues, protect dependencies, and practice rollback and restore before an incident forces the first attempt.
What cloud-native does not require
- A monolith can use immutable images, declarative deployment, autoscaling, and safe rollout before it is split into services.
- A microservice is not automatically resilient; synchronous dependency chains can multiply latency and availability risk.
- A managed cloud service may reduce operational work without being portable across providers. Portability is a business constraint, not a universal quality metric.
- A CNCF project is a tool with a lifecycle and support model, not a substitute for a system requirement or an operating owner.
Follow the control loop from code to recovery
Cloud-native delivery is a feedback system. Each boundary produces evidence for the next decision, and production remains authoritative after the deployment controller has acted.
Cloud-native delivery control loop
Automation moves an immutable artifact forward; production telemetry decides whether the desired state should expand, pause, or roll back.
Intent
Source and policy
Review code, tests, dependency policy, deployment declarations, and the identity of the change before building an artifact.
Package
Artifact and provenance
Sign a content-addressed image, attach an SBOM and scan result, and promote by digest rather than rebuilding for each environment.
Apply
Reconciliation and rollout
The platform creates replicas, waits for readiness, shifts bounded traffic, and keeps the previous revision available for rollback.
Decide
Runtime evidence
Service-level indicators, saturation, business guardrails, and incident signals close the loop by allowing promotion or forcing containment.
The controller can repair a missing pod because the desired state is declared. It cannot decide whether a payment was charged twice or whether a schema migration is backward compatible. Application-level correctness remains with the service and its data owner.
Estimate capacity before choosing platform knobs
Start from workload and service objectives, then translate them into replicas and failure headroom. A pod count copied from another service says nothing about the current request cost, startup behavior, or dependency limits.
25,000 rps
Peak demand
Measured entry traffic, not an average
1,200 rps
Tested pod limit
At the target p95 and error rate
30%
Failure headroom
Capacity retained during replacement
28 pods
Starting fleet
ceil(25,000 x 1.30 / 1,200)
Keep the estimate honest
- Use the busiest credible interval and the slowest important request class, not one blended daily average.
- Measure sustainable per-pod throughput at the latency and error objective. A load test that ignores dependencies produces imaginary capacity.
- Add headroom for one failure domain, rollout overlap, autoscaler delay, and traffic forecast error.
- Verify that databases, queues, connection pools, and third-party APIs can absorb the resulting concurrency. Scaling stateless callers can overload a fixed dependency.
- Recalculate cost and capacity from telemetry after launch; requests and limits are hypotheses until production validates them.
Availability compounds across mandatory synchronous dependencies. If four required services each deliver 99.9% availability independently, the path is roughly 99.6% available before client, network, and regional failures are counted. Remove unnecessary synchronous hops instead of expecting orchestration to erase the multiplication.
Bound release exposure instead of debating strategy names
Rolling, canary, and blue-green deployments move risk differently. Change the release rate, failure rate, first-wave traffic, and recovery time to see how many requests can be exposed before the control loop contains a bad version.
Loading the release envelope
The lesson is loading its rollout strategies and risk thresholds.
Loading rollout strategies...
Read the result as a risk budget
- Release frequency changes the number of opportunities for failure. It should rise only when tests, review, observability, and rollback keep change failure and recovery time low.
- First-wave exposure limits blast radius only when the canary population is representative and promotion waits for enough evidence.
- Detection plus rollback time determines how long a known-bad version can continue affecting requests. A rollback button without a tested decision signal is not a recovery plan.
- Overlap capacity must fit the cluster during rollout. Blue-green buys a fast traffic switch by temporarily running two environments; a rolling update uses less spare capacity but mixes versions for longer.
Encode the workload contract in the deployment
A deployment declaration should give the scheduler and controllers enough information to place, start, stop, and replace the workload without guessing.
Minimum workload contract
- Identity: Pin the image by digest and label the service, owner, and version so runtime evidence can be attributed to one artifact.
- Resources: Set requests from measured steady use and limits from tested safety boundaries. CPU throttling and memory termination have different failure behavior.
- Health: Use startup checks for slow initialization, readiness for traffic eligibility, and liveness only when restart can repair a stuck process.
- Placement: Spread replicas across hosts and zones while preserving enough schedulable capacity for replacement and maintenance.
- Shutdown: Stop accepting traffic, drain bounded work, and finish or hand off state before the grace period expires.
- Disruption: Protect minimum healthy capacity during voluntary maintenance, but do not make the policy so strict that a node can never be drained.
GitOps can apply this declaration repeatedly and detect drift, but it does not prove that the change is safe. Admission policy checks structure and provenance; rollout evidence checks behavior; an application migration still needs its own compatibility and recovery plan.
Inject failures and contain the blast radius
Choose a failure, then tune replica count, zone spread, timeout, and request handling. The lab distinguishes surviving compute capacity from user-visible availability: spare pods do not help when all of them wait forever on the same dependency.
Loading the failure model
The lesson is loading its failure scenarios and capacity envelope.
Loading failure scenarios...
Match the control to the failure
- A pod crash should be absorbed by ready peers while the controller replaces the failed instance. Readiness must remove the pod before clients continue routing to it.
- A zone outage requires replicas and dependencies in another failure domain. Three replicas on one node or in one zone are one failure unit.
- A slow dependency needs a shorter caller deadline, bounded concurrency, and often a circuit breaker. More caller replicas can amplify the overload.
- A traffic burst needs admission control, autoscaling lead time, and a bounded queue. An unlimited queue converts overload into unbounded latency and memory use.
Design state and communication explicitly
Stateless compute is easy to replace because another instance can handle the next request. Durable state, ordering, and external side effects need stronger contracts.
Immediate result
Synchronous request
Use for a bounded query or command whose caller needs an immediate outcome. Set one end-to-end deadline and pass the remaining budget downstream.
Decoupled work
Durable message
Use when work can survive caller disconnect and be processed later. Expect duplicate delivery; make effects idempotent and expose queue age, not only queue depth.
System of record
Owned data
Give one service authority over a data boundary. Publish facts or APIs rather than letting unrelated services share write access to its tables.
Retry only where the outcome is understood
- Give the original operation a stable idempotency key.
- Retry only transient failures and stop within the caller's deadline.
- Add jittered backoff and a retry budget so every layer does not multiply attempts.
- Treat a timed-out write as unknown until the owner reconciles the original key.
- Send exhausted asynchronous work to a visible recovery path with an owner and replay procedure.
Make telemetry support a decision
Metrics, logs, and traces are useful only when they identify a user-visible symptom, the responsible version, and the action an operator should take.
User experience
Service-level signals
Measure request success, latency distributions, freshness, durability, and correctness for important journeys. Alert on fast and slow error-budget burn rather than every small utilization change.
Saturation
Resource signals
Track CPU throttling, memory working set and termination, queue age, connection use, disk pressure, and scheduler delay to explain why the service objective is at risk.
Diagnosis
Correlated events
Carry request, trace, tenant, service, version, region, and rollout identity through structured events. Keep credentials and sensitive payloads out of unrestricted logs.
Response
Actionable ownership
Every alert needs severity, owner, runbook, dashboard, and a safe first action. A page that cannot change a decision is usually noise.
Observe the control plane separately from the workload. A healthy application can continue serving during a short control-plane interruption, while deployment, autoscaling, or reconciliation operations may be delayed.
Secure the software and runtime supply chain
Cloud-native security spans build identity, artifact integrity, workload identity, network policy, and runtime authority. A private image registry alone does not prove what code was built or what it may do.
Build and admission
- Pin dependencies and builders, scan source and images, generate an SBOM, and sign the immutable image digest with build provenance.
- Admit only trusted registries, signatures, approved base images, non-root workloads, bounded privileges, and declared resource policy.
- Separate build, deployment, and runtime identities. A workload should not inherit the credentials that can change its own production declaration.
Runtime
- Issue short-lived workload identity and authorize the exact service and operation, not merely a network location.
- Deny unnecessary egress and service-to-service paths; validate policy behavior in a staging topology before enforcing it broadly.
- Mount secrets only where needed, rotate them without rebuilding an image, and prevent their values from entering logs, traces, environment dumps, or crash reports.
- Keep node and cluster administration separate from application deployment rights and audit every privileged operation.
Choose complexity only when it pays for itself
Default boundary
Modular monolith
Start here when one team can deploy together and one scaling profile is acceptable. Keep modules and ownership clear so later extraction is possible without distributed transactions from day one.
Selective extraction
Independent services
Extract when a domain needs independent ownership, scaling, isolation, release cadence, or technology. Include on-call, data, compatibility, and platform cost in the decision.
Operational leverage
Managed platform
Prefer managed databases, queues, and control planes when their limits meet the system contract. Portability work is justified only by a credible migration or continuity requirement.
Watch for a distributed monolith: services that must deploy together, share one schema, call each other synchronously for every request, and fail as one unit. It has the coordination cost of services without independent change or containment.
Operate one production change through evidence
1 Plan
Define the contract
Name the user journey, service objective, peak load, state owner, failure domains, dependencies, and rollback boundary before choosing platform features.
2 Build
Prove the artifact
Create one immutable image, attach tests, provenance, policy, and scan evidence, then promote that digest without rebuilding it.
3 Verify
Exercise failure
Load-test the dependency envelope and inject pod, zone, network, and slow-dependency failures. Measure recovery instead of inferring it from replica count.
4 Deploy
Release in steps
Limit first exposure, wait for representative service and business signals, and record who or what may promote, pause, or roll back.
5 Operate
Learn and recalculate
Compare the estimate with real capacity, error-budget use, cost, and incident evidence. Update defaults, runbooks, and the next release gate.
Production review
- Can a new instance start from declared configuration without copying local state?
- Can the service lose one planned failure domain and still meet its user contract?
- Do readiness, deadlines, retry budgets, and shutdown behavior agree across callers and dependencies?
- Can operators identify the deployed digest, affected traffic, and rollback target from one alert?
- Has restore been tested from durable backups, not only replica failover?
- Does each platform component have an owner, upgrade path, capacity signal, and exit plan?