Docker
Learn Docker image construction, build cache behavior, runtime boundaries, storage, networking, secrets, and container replacement.
What is Docker?
Docker is a platform for building, distributing, and running applications as containers. An image is an immutable package of application files, dependencies, and runtime metadata. A container is a runnable instance of that image plus process, network, storage, resource, and security configuration.
Docker matters because one identified image can move through development, CI, and production without rebuilding the application differently in each environment. That does not make every run identical: hosts, secrets, mounted data, ports, limits, and kernel behavior remain deployment inputs.
Core invariant
Treat the image as a versioned delivery artifact and the container as a replaceable process. Anything that must survive replacement belongs outside the container writable layer, and every runtime capability should be declared deliberately.
Dockerfile
Describe the build
Order instructions so stable inputs can stay cached, and use stages to keep build-only tools out of the runtime artifact.
Image
Package immutable files
An image is content-addressed, read-only, and composed of filesystem layers plus configuration such as its default command and user.
Container
Run one isolated process
Docker creates a writable layer and configures namespaces, control groups, mounts, and networking around the image's process.
Registry
Store and distribute
A registry stores image manifests and layers. Deploy by immutable digest when the exact artifact matters; a mutable tag is only a reference.
Docker's official platform overview defines the client, daemon, registry, images, and containers that make up this lifecycle.
Follow one artifact from source to a running process
The same image should cross environments. Build-time inputs determine its content; run-time inputs determine how one instance behaves.
Docker delivery path
Build once, identify the result, then create replaceable containers from that exact artifact.
Inputs
Source and Dockerfile
The build context, .dockerignore, Dockerfile, base-image references, dependency locks, and build-time secrets define what the builder can observe.
Build
BuildKit
Execute instructions, reuse valid cache records, and produce an image manifest plus content-addressed layers. A changed step invalidates dependent work.
Distribute
Registry
Push the manifest and missing blobs. Sign, scan, promote, or attest the immutable digest that CI actually tested.
Run
Docker Engine
Pull the image if needed, create a writable layer, attach declared mounts and networks, apply resource and security controls, then start the configured process.
Containers are not small virtual machines
Shared host kernel
Container boundary
Containers isolate processes and selected kernel resources. Multiple Linux containers on one host use that host's Linux kernel, so kernel compatibility and host hardening remain part of the security model.
Guest kernel
Virtual-machine boundary
A virtual machine presents virtual hardware to a guest operating system with its own kernel. Teams commonly run containers inside VMs to combine deployment density with a stronger infrastructure boundary.
Choose the boundary from the threat model and operating model. Image size or startup speed alone does not establish isolation strength.
Make rebuild work proportional to the change
Docker evaluates a build as ordered instructions. When an input changes, Docker must rerun that instruction and dependent instructions after it. Put expensive, stable work before frequently changing source, and keep the build context narrow.
all required steps
Cold build
Establishes the complete artifact without reusable cache records
invalidated path
Warm rebuild
A source edit should not reinstall unchanged dependencies
runtime files only
Final image
Builder tools and caches do not belong in the serving artifact
image digest
Promotion key
Tags can move; a digest identifies immutable manifest content
Trace cache invalidation through an image build
Loading an illustrative Docker build model.
Loading build model
Preparing the layer trace.
The lab uses illustrative timings and sizes to make invalidation visible. Measure your own build with BuildKit output and image inspection before setting a CI or rollout budget. Docker's official build-cache guide explains why a changed layer affects downstream layers, while the cache optimization guide covers layer ordering, smaller contexts, cache mounts, and external caches.
Read the Dockerfile as two dependency graphs
- The cache graph asks which instructions and files can change a build result.
- Copy dependency manifests before source when dependencies change less often.
- Keep generated output, local dependencies, credentials, and irrelevant files out of the context with
.dockerignore. - Use cache mounts for package downloads that remain useful across rebuilds.
- The artifact graph asks which files reach the final stage.
- Copy only runtime dependencies and selected build output.
- Start the final stage from a base that contains the libraries the process needs.
- Verify runtime behavior before choosing an extremely minimal or
scratchbase.
Docker's multi-stage build documentation defines how each FROM starts a stage and how selected artifacts cross stage boundaries.
Keep credentials out of image history
A build may need a package-registry token, SSH agent, or cloud credential. Passing that value through ARG, ENV, COPY, or a generated configuration file can persist it in image metadata, a layer, the cache, or build logs.
Use BuildKit secret or SSH mounts instead. They expose the credential to the specific build instruction that needs it without making the value part of the resulting layer.
The boundary still needs controls
- Give the builder only the credential scope and lifetime required for that build.
- Keep secret values out of command output and error traces.
- Pin and verify downloaded artifacts; a successful authenticated download is not proof of content integrity.
- Assume remote caches and CI logs are shared infrastructure unless their access and retention are explicitly constrained.
See Docker's official build secrets documentation for the current CLI, Bake, secret-mount, SSH-mount, and remote-context contracts.
Declare state, reachability, resources, and privilege
The image does not decide whether data persists, who can connect, or how much of the host one process may consume. Those are run-time policies. Test them by replacing the container, not only by restarting the process inside it.
Predict what survives and what becomes reachable
Loading an illustrative runtime policy model.
Loading runtime model
Preparing the container replacement test.
Place each write by lifecycle
- Container writable layer: temporary process-local changes that may disappear with the container. Do not put authoritative database files here.
- Named volume: Docker-managed persistent data whose lifecycle is independent of a container.
- Define ownership, backup, restore, migration, encryption, and deletion policy.
- A volume survives replacement; it does not automatically provide a backup.
- Bind mount: a daemon-host path exposed directly to the container.
- Useful when host and container tools must share source or generated files.
- Couples the workload to host paths, permissions, and platform behavior.
- tmpfs mount: memory-backed temporary data that should not reach persistent disk.
- Size it explicitly and plan for loss on stop, restart, or host reboot.
Docker's storage overview distinguishes the container writable layer, volumes, bind mounts, and tmpfs lifecycle.
Expose only the required callers
- A user-defined bridge gives containers on one Docker host name-based project connectivity without publishing every service to the host.
- Port publishing maps an explicit host address and port to a container port.
- Bind development services to
127.0.0.1when only the host should reach them. - Treat a port published on all interfaces as external exposure subject to host and network policy.
- Bind development services to
- Host networking removes the separate network namespace. Use it only when its reduced isolation and shared port space are part of the design.
- The
nonedriver provides no external network interface for work that should be network-isolated.
The official network-driver summary documents bridge, host, overlay, macvlan, ipvlan, and none semantics.
Bound host impact
Containers have no CPU or memory constraints by default. Measure representative peaks, then set limits and observe throttling, OOM kills, restart behavior, and host headroom. Docker's resource-constraint guide details memory, swap, CPU quota, shares, and host OOM behavior.
Operate the process as disposable, not invisible
A container is a deployment unit, not an observability or recovery strategy. The application still needs a startup contract, a termination contract, bounded retries, and evidence that connects an incident to the exact image and runtime policy.
1 CI
Build and prove
Create the image from reviewed inputs. Run tests, inspect dependencies, scan the final artifact, and retain provenance for the immutable digest.
2 Deploy
Configure and start
Inject environment-specific configuration and secrets at runtime. Declare user, mounts, network, limits, health behavior, and restart policy before the process starts.
3 Operate
Observe and drain
Collect standard output and error, application metrics, container resource signals, and events. On shutdown, stop admission, finish bounded work, and exit before the grace period expires.
4 Failure
Replace and recover
Recreate from the same image and policy, attach declared persistent state, and verify service plus data recovery. Roll back by digest when the artifact is the fault.
Evidence to retain
- Artifact identity: image digest, source revision, base-image digests, build provenance, dependency inventory, and scan results.
- Runtime identity: container ID, image digest, host, effective user, mounts, published ports, limits, capabilities, health, and restart count.
- Service behavior: request latency, errors, saturation, queue depth, dependency failures, termination duration, and application-defined readiness.
- Recovery behavior: replacement time, volume attachment, restore result, data validation, traffic re-entry, and rollback decision.
Failure drills worth automating
- Kill the main process and verify restart behavior does not duplicate external side effects.
- Replace the complete container and prove all required data returns from declared storage or an authoritative service.
- Exceed a representative memory limit and verify the OOM signal, alert, and recovery path are visible.
- Remove network access to a dependency and verify timeouts, retries, and admission are bounded.
- Revoke a runtime secret and verify rotation does not require baking a new credential into the image.
Review the production contract
Before promoting an image, answer these questions with evidence:
- Which digest was tested, and can deployment select that exact digest?
- Which files are build-only, and how was their absence from the final image verified?
- What changes invalidate the slowest build steps?
- Which writes survive container replacement, and how are they backed up or rebuilt?
- Which callers can reach each port from the project network, host, and external network?
- What CPU, memory, process, filesystem, user, and capability boundaries are effective?
- What happens after
SIGTERM, an OOM kill, host loss, dependency timeout, or partial rollout? - Can operators connect logs, metrics, events, and recovery results to one immutable image and one runtime configuration?
A green container status proves only that the container is running according to its configured health behavior. It does not prove the application is correct, durable, authorized, recoverable, or safe under replacement.