TensorFlow

Read tensors as contracts, not containers

A tensor has a dtype, a shape, and a value. The rank is the number of shape dimensions. A shape such as [64, 32] commonly means 64 records with 32 features each, but TensorFlow cannot infer that business meaning. The producer and consumer must agree on it.

Values and mutable state are different

• tf.Tensor represents a value produced or consumed by operations.
• tf.Variable owns mutable state such as learned weights, counters, or running statistics.
• A layer or tf.Module tracks variables assigned to it, which lets checkpoints and exports discover state.
• A tensor with the expected shape but the wrong feature order is structurally valid and semantically wrong.

Validate rank, dimensions, dtype, finite values, ranges, and feature semantics at the boundary where bad data can still be rejected. The TensorFlow basics guide introduces tensors, variables, gradients, modules, training loops, and saving from the same execution model.

Use eager execution to understand; use graphs deliberately

TensorFlow 2 executes operations eagerly by default. Each operation runs immediately, so Python control flow, stack traces, values, and breakpoints are easier to inspect.

tf.function traces TensorFlow-compatible code into one or more graphs. A graph can reduce Python dispatch, enable runtime optimization, run across devices, and become part of an exported program. It is not a universal speed switch: a function with a few expensive operations may see little improvement, while repeated small operations may benefit more.

Tracing is a cache of concrete programs

The first compatible call traces Python and records TensorFlow operations. Later calls reuse the concrete graph when their inputs match. New Python values, dtypes, ranks, or incompatible shapes can create more traces.

• Give hot functions tensors rather than changing Python objects.
• Use an input_signature when the accepted rank, dtype, and dimensions are known.
• Bucket or pad variable-length examples when unbounded shapes create excessive traces.
• Use tf.print for graph-time output; ordinary Python side effects usually happen during tracing, not on every graph execution.
• Treat tf.py_function as an explicit escape hatch. It is not portable in SavedModel and does not work well with distributed or accelerator execution.

Test the same step eagerly and through tf.function, including variable updates and numerical tolerances. The official tf.function guide documents tracing, polymorphism, Python side effects, and the portability limits of tf.py_function.

Balance the input producer with the device consumer

A training device cannot compute on a batch that has not arrived. The input path usually reads records, parses or decodes them, transforms features, groups examples into batches, and stages the next batch while the current step runs.

Use the lab to compare eager and graph paths, stable and variable shape contracts, parallel mapping, cache placement, prefetch, and a Python callback. Its times are deterministic teaching assumptions, not TensorFlow benchmarks.

The steady-state rule is simple: with effective overlap, the slower of input preparation and device compute limits the step. Without overlap, the times add. A deeper prefetch buffer can absorb jitter, but it cannot make a persistently slow producer faster.

Build a tf.data pipeline with explicit semantics

tf.data.Dataset represents a sequence of elements and lazy transformations. Its value is not the method chain itself; it is the reproducible contract for ordering, cardinality, batching, parallelism, and failure behavior.

1. 1

   Source

   Read bounded records

   Define file patterns, schemas, compression, shard ownership, and behavior for missing or corrupt data. Keep source and preprocessing versions with the run.

2. 2

   CPU

   Transform and validate

   Parse, decode, normalize, and reject invalid examples. Prefer TensorFlow operations so mapping can be parallelized and serialized.

3. 3

   Semantics

   Shuffle and batch

   Choose shuffle scope, repeat rules, deterministic order, bucket boundaries, batch size, and drop_remainder from the training contract.

4. 4

   Overlap

   Stage the consumer

   Use parallel reads or maps where measured, then prefetch to overlap producer work with the device step.

Cache only what has a valid identity

• Cache before random augmentation when each epoch should see fresh augmentation.
• Cache after expensive deterministic transforms only when the larger decoded footprint fits.
• Include source, schema, vocabulary, preprocessing, and split versions in a persistent cache key.
• Do not cache a partial first pass by accident; confirm cardinality before declaring the cache complete.

Make distributed input behavior testable

• State whether each worker reads disjoint shards or repeats the full dataset.
• State whether batch size is global or per replica.
• Keep evaluation order deterministic and prevent train/evaluation overlap by entity or time.
• Count skipped records by reason. Silently ignoring parse errors changes the data distribution.

The official tf.data performance guide recommends profiling and then applying prefetch, parallel interleave, parallel map, cache, and vectorized transforms where they address measured work.

Make layer and model contracts serializable

A layer transforms inputs and may own variables or non-trainable state. A model composes layers and gives the computation a training, evaluation, and export boundary.

Use Sequential for one linear stack, the Functional API for directed acyclic graphs with multiple inputs, outputs, or shared layers, and subclassing when the forward pass needs custom control. More flexibility creates a larger serialization and test surface.

The official custom layer and model guide covers tracked state, call, training behavior, losses, metrics, and serialization hooks.

Treat training as a resumable state machine

Training moves model variables and optimizer state through steps. A production run also owns dataset identity, code and environment versions, random seeds, distribution topology, evaluation results, and checkpoints.

Choose the narrowest loop that preserves control

• Use Keras Model.fit when its compile, callback, checkpoint, validation, and distribution behavior matches the required lifecycle.
• Use a custom loop when the algorithm needs non-standard updates, multiple optimizers, gradient accumulation, custom control flow, or precise step-level coordination.
• Keep a single source of truth for loss reduction and metric aggregation. A local replica loss is not automatically the global reported loss.

Distribution changes the system contract

tf.distribute.Strategy can place synchronous replicas on one host, multiple workers, or supported accelerators. The strategy scope owns variable creation; the input path supplies per-replica batches; updates are reduced according to the strategy.

• Record global batch, per-replica batch, replica count, gradient accumulation, and learning-rate policy together.
• Confirm every worker sees the intended data shard and the same step count where synchronization requires it.
• Let one designated owner publish checkpoints and final artifacts. Other workers should not race on the same path.
• Test worker loss, restart, checkpoint restore, and rendezvous time before scaling the expensive run.

The current distributed training guide lists supported strategies and their API constraints. Treat experimental labels and estimator guidance on that page as moving contracts and recheck them against the runtime version you pin.

Evaluate the model and the system separately

Model.evaluate can compute configured losses and metrics over a dataset. That is one layer of evidence, not a release decision.

Model evidence

• Compare against a named baseline on a frozen, versioned evaluation set.
• Report task metrics by critical slice, not only as one aggregate.
• Check calibration and threshold-dependent costs when scores trigger actions.
• Test invariances, invalid inputs, missing features, and known difficult cases.
• Bound numerical differences between eager, graph, restored, and exported execution.

System evidence

• Measure cold load, warmup, batch-size response, throughput, and p50, p95, and p99 latency.
• Test memory growth, long runs, cancellation, malformed requests, concurrency, and overload.
• Verify preprocessing and postprocessing parity between training and serving.
• Record the exact artifact, runtime image, hardware, dataset, and load-generator settings.

The Keras training and evaluation guide documents built-in fit, evaluate, prediction, metrics, callbacks, weighting, and dataset inputs. Product-specific release thresholds still belong to the owning team.

Export a callable SavedModel contract

Saving and serving solve different lifecycle needs. The current TensorFlow serialization guide recommends the .keras format for a reloadable Keras training artifact and model.export() for a lightweight SavedModel inference artifact. The lower-level tf.saved_model.save API remains useful when a tf.Module needs explicit signatures.

A serving signature should make these finite

• Signature name and model version.
• Input names, dtypes, ranks, fixed and dynamic dimensions, defaults, and invalid-input behavior.
• Output names, dtypes, shapes, ranges, and score interpretation.
• Preprocessing and vocabulary versions that are inside or outside the artifact.
• Numerical tolerance and golden examples for clean reload.

See the official Keras serialization and export guide and SavedModel guide for the current APIs and signature model.

Serve versions without confusing availability and correctness

TensorFlow Serving loads versioned servables and exposes prediction APIs. A model that is loaded is available; it is not necessarily correct, compatible, fast enough, or safe for the current client.

Put a stable boundary in front of the model server

• Authenticate callers and enforce request size, concurrency, and deadlines before inference.
• Validate feature names, dtypes, dimensions, and allowed ranges before expensive work.
• Route by explicit stable and canary version labels or an equivalent deployment policy.
• Retry only known transient failures and only while the request deadline allows it.
• Keep preprocessing, model, and postprocessing versions visible in logs and responses where appropriate.

Understand version policy before rollout

TensorFlow Serving can load specific versions, assign labels, poll model configuration, batch requests, and expose Prometheus-format metrics. Availability-preserving loading may require enough memory for old and new versions at the same time; resource-preserving replacement can create an availability gap. Choose deliberately.

The official Serving configuration guide documents model version policy, labels, monitoring, and batching. The Serving architecture guide explains sources, managers, servable versions, and availability-versus-resource loading policies.

Observe causes across data, execution, and serving

TensorBoard can visualize scalar summaries and training behavior. TensorFlow Profiler can collect host and accelerator traces and provides views such as the Input Pipeline Analyzer, TensorFlow Stats, Trace Viewer, kernel statistics, and memory tools where supported.

Training telemetry

• Run ID, code and environment version, data manifest, random seeds, strategy, replica count, global batch, and checkpoint lineage.
• Step time split into input, host, device, collective communication, and checkpoint work.
• Loss, learning rate, gradient norms, metric slices, invalid examples, retrace count, memory, and device utilization.

Serving telemetry

• Artifact and signature version, request class, response status, and validation failure reason.
• Queue time, model time, end-to-end latency, batch size, throughput, rejection, timeout, and cancellation.
• Loaded and unavailable versions, load latency, worker restarts, memory pressure, and rollback events.
• Delayed quality signals by model version and critical product slice, with privacy-aware logging.

Do not optimize from utilization alone. High accelerator utilization can coexist with failed requests, poor tail latency, or low useful throughput. Capture a baseline, profile a representative window, change one boundary, and verify correctness and service objectives again. The official TensorFlow Profiler guide describes supported capture modes and analysis tools.

Migrate artifacts and runtimes as data changes

A TensorFlow or Keras upgrade can change serialization, supported operations, default behavior, performance, device kernels, or integration code without changing your model source. Treat it like a production data migration.

1. 1

   Discover

   Inventory the old contract

   Record runtime and dependency versions, custom objects, checkpoints, SavedModel signatures, preprocessing, serving flags, metrics, and client assumptions.

2. 2

   Convert

   Build a reproducible candidate

   Pin the new environment, migrate one artifact path, and preserve an immutable copy of the original training and serving assets.

3. 3

   Prove

   Compare every execution path

   Run eager, graph, restored, and exported parity; replay evaluation slices; profile training and serving; and inspect signature diffs.

4. 4

   Release

   Canary with independent rollback

   Load the candidate beside the reference, route bounded traffic, compare version-specific signals, and remove the old version only after the rollback window closes.

For TensorFlow 1 migrations, replace graph collections, sessions, and estimator-specific behavior with explicit TensorFlow 2 objects and functions in bounded stages. The official migration overview and SavedModel migration guide document current compatibility and export paths. Avoid making tf.compat.v1 the permanent architecture without an exit plan.

Make release evidence change the decision

Use the lab to select a custom layer change, runtime upgrade, or input contract change. Remove evidence, widen traffic, share the failure domain, and inject a serving failure. The initial-canary threshold and recovery penalties are an illustrative local policy, not TensorFlow defaults.

Minimum production release gates

• The artifact reloads in the exact serving image without training-only source code or undeclared custom objects.
• Signature names, input and output contracts, invalid-input behavior, and golden-output tolerances pass.
• Eager, graph, restored, and exported paths agree within the declared numerical tolerance.
• Aggregate and critical-slice evaluation meet their quality, calibration, and safety floors.
• Representative load meets cold-load, warmup, latency, throughput, error, memory, and stability targets.
• Candidate dashboards, version labels, alerts, and incident ownership work before traffic arrives.
• Migration rehearsals cover checkpoints, preprocessing, custom layers, operations, clients, and metrics.
• An isolated known-good version can recover traffic inside the rollback budget.

Promote only when every required gate is attached to the immutable candidate. A green training loss, successful export, or loaded server is one fact, not a production verdict.

Choose TensorFlow from system constraints

TensorFlow is a reasonable fit when the team needs its tensor runtime, Keras training APIs, distribution strategies, SavedModel contract, Serving integration, or target-platform ecosystem and is prepared to operate those pieces. It is a weak fit when a smaller numerical library, a hosted prediction API, or another framework already satisfies the training and deployment constraints with less integration work.

Before choosing it, answer:

• Which training and serving targets must share an artifact or operator set?
• Which model, layer, preprocessing, and signature contracts require custom code?
• Which distribution strategy has been tested for this workload and failure model?
• Who owns input data correctness, artifact compatibility, serving policy, and delayed quality?
• Can the team reproduce, profile, canary, observe, and roll back the exact runtime it will ship?

Framework adoption does not remove these responsibilities. It gives the team APIs with which to make them explicit.

Primary sources and moving contracts

• TensorFlow basics connects tensors, variables, gradients, modules, training, and saving.
• Better performance with tf.function documents tracing, concrete functions, retracing, side effects, and Python boundaries.
• Better performance with tf.data documents prefetch, parallel reads and maps, cache, vectorization, and memory trade-offs.
• Custom layers and models documents tracked state, training behavior, losses, and serialization.
• Distributed training documents the current strategy set and API support.
• Keras serialization and export distinguishes reloadable Keras artifacts from inference export.
• SavedModel documents tracked objects, functions, signatures, loading, and inspection.
• TensorFlow Serving configuration documents version policies, labels, batching, and Prometheus monitoring.
• TensorFlow Profiler documents current capture APIs and analysis views.
• TensorFlow migration documents supported TensorFlow 1 to TensorFlow 2 migration paths.

These pages describe moving APIs and support levels. Recheck them against the exact TensorFlow, Keras, TensorBoard, and TensorFlow Serving versions pinned by the release.