LLM Models & Families

Follow the model through its full lifecycle

An LLM is not created by one training job and then dropped into an API. Its lifecycle connects data decisions, optimization, adaptation, serving, and evaluation. A failure at any stage can appear later as a wrong answer, unsafe behavior, excessive latency, or an unmanageable rollback.

1. 1

   Data contract

   Define and curate data

   Select lawful, relevant sources; remove harmful or low-value duplication; define mixtures, filters, provenance, and holdout boundaries.

2. 2

   General capability

   Pretrain a base model

   Optimize token prediction over large corpora while tracking loss, data quality, compute use, contamination, and checkpoint health.

3. 3

   Product fit

   Adapt behavior

   Use prompts, retrieval, supervised examples, or preference data to satisfy a task without making change more persistent than necessary.

4. 4

   Runtime

   Serve token generation

   Schedule prefill and decode work, manage per-request attention state, enforce quotas, call tools, and validate outputs.

5. 5

   Release loop

   Evaluate and operate

   Gate versions on task, safety, latency, cost, and failure evidence; monitor drift and keep a tested rollback path.

A model family is a set of compatible bundles

A family may include base and instruction-tuned checkpoints, multiple sizes, quantized variants, multimodal versions, adapters, tokenizers, and runtime-specific packages. Compare the exact bundle you can deploy. Two checkpoints with a shared family name can have different context contracts, tool behavior, memory needs, licenses, and failure profiles.

Pretraining learns a distribution, not a fact database

Autoregressive pretraining presents a sequence of tokens and asks the model to predict the next token. The training system compares predicted probabilities with the observed token, computes a loss, and updates weights through backpropagation. Repeating this over diverse sequences creates reusable representations and generation behavior.

Data quality is an architecture decision

• Define source ownership, permitted use, retention, provenance, and removal procedures before training.
• Measure duplication, language and domain balance, toxicity, sensitive data, and evaluation-set contamination.
• Keep preprocessing, tokenizer, mixture weights, filters, and dataset snapshots versioned with the checkpoint.
• Preserve held-out slices that represent real product tasks and difficult failure conditions.
• Treat synthetic data as generated evidence with its own lineage, filtering, and feedback-loop risks.

Scale parameters, tokens, and compute together

Scaling-law research shows that loss changes predictably with model size, data, and compute within studied regimes. The practical lesson is not one permanent ratio. It is that an oversized model trained on too few useful tokens can waste a fixed compute budget, while more low-quality or duplicated data can also fail to buy useful capability.

For a dense Transformer, a common planning approximation makes training work proportional to parameters x training tokens. The real budget must also include optimizer state, activation memory, communication, failed work, evaluation, checkpoints, and data processing.

Adapt only the layer that needs to change

Many teams reach for fine-tuning before they have named the missing capability. First decide whether the requirement is temporary behavior, fresh knowledge, a stable domain pattern, or a broad ranked preference. Each belongs in a different state boundary.

Use the workbench to move the same requirement across adaptation layers. Watch how state location changes update speed, rollback, latency, and evaluation burden.

Make adaptation evidence match its persistence

The more persistent the change, the stronger its data and release contract must be.

Prompt and retrieval checks

• Version the complete prompt template, examples, tool schema, and truncation policy.
• Test instruction conflicts, prompt injection, missing evidence, low-recall retrieval, stale indexes, and unauthorized sources.
• Require citations to support generated claims rather than merely appearing in the response.
• Define empty-result and degraded-dependency behavior before relying on external evidence.

Learned-adaptation checks

• Compare against prompt and retrieval baselines at matched evaluation coverage and serving cost.
• Split training, validation, and test data by meaningful entities or time boundaries to reduce leakage.
• Track base checkpoint, tokenizer, adapter, training code, data snapshot, hyperparameters, and evaluator versions together.
• Measure general-capability regressions, memorization, safety shifts, calibration, and important subgroup slices.
• Roll back the complete model bundle; an adapter may not remain compatible with a different base or runtime.

Inference has two different compute phases

At request time, the serving system tokenizes the input, constructs the context, and runs the model. Decoder-style generation then has two operationally different phases:

• Prefill processes the supplied prompt in parallel and creates attention state for its tokens. Long prompts directly increase this work.
• Decode emits output tokens one step at a time. Each step reads prior attention state, so token latency and memory pressure matter across active sequences.
• KV cache stores per-request attention keys and values. It grows with context, active sequences, architecture, and precision even when model weights already fit.
• Scheduling trades queue delay against batching efficiency. A larger batch can improve throughput while hurting an individual request's first-token latency.
• Prefix reuse can skip repeated prefill work only when the cached prefix identity includes every input that changes its result.

Build a production request path around the model

The model server should not own authentication, evidence authorization, business side effects, or final policy by itself. Keep those responsibilities visible in the system architecture.

Bound failure before adding fallback

• Set independent deadlines for context assembly, model queueing, generation, tools, validation, and streaming.
• Cancel abandoned work and account every retry against the original quota and deadline.
• Retry only idempotent operations or reconcile an unknown result before trying again.
• Use a fallback model only after validating its context, tool, safety, and output-schema compatibility.
• Protect downstream tools with least privilege, argument validation, rate limits, and explicit user confirmation for consequential actions.
• Degrade visibly when retrieval, safety, or tool dependencies are unavailable; do not silently remove a mandatory control.

Evaluate the task and the complete system

A public benchmark can provide one signal, but production selection must use representative prompts, adversarial cases, operational load, and human review where automated scoring is weak. Keep the candidate, baseline, evaluator, dataset, prompt template, and sampling configuration versioned.

Release with thresholds, not impressions

1. Freeze the candidate bundle and evaluation manifest.
2. Compare it with the current production baseline on the same cases and operating envelope.
3. Require thresholds for critical failures, not only a better average score.
4. Shadow or canary the complete path with bounded exposure and version-separated telemetry.
5. Promote only when online task, safety, latency, and cost signals agree with release assumptions.
6. Keep the prior bundle runnable until rollback and replay drills pass.

Select a model family from constraints

Do not begin with a vendor leaderboard. Begin with the workload and eliminate bundles that violate hard constraints.

• Task contract: required languages, modalities, tool use, structured output, context behavior, and minimum quality by critical slice.
• Serving contract: first-token and per-token latency, concurrency, throughput, memory, availability, and geographic placement.
• Data contract: retention, training use, residency, encryption, access, deletion, auditability, and model-license obligations.
• Adaptation contract: prompting, retrieval, adapter support, preference tuning, tokenizer stability, and version compatibility.
• Operational contract: observability, quotas, deprecation policy, reproducible bundles, rollback, incident support, and fallback behavior.
• Economic contract: complete task cost at representative input/output lengths, retries, tools, review, and reserved capacity.

Choose the smallest bundle that clears the complete contract with enough headroom. A larger model can be justified by measured task value, but parameter count by itself is not a quality, safety, or reliability guarantee.

Primary papers and maintained guidance

• Attention Is All You Need introduces the Transformer architecture based on attention mechanisms.
• Scaling Laws for Neural Language Models studies empirical relationships among model size, data, compute, and language-model loss.
• Training Compute-Optimal Large Language Models studies balancing parameter count and training tokens under a compute budget.
• Training language models to follow instructions with human feedback describes supervised instruction tuning followed by preference-based reinforcement learning and reports its measured limitations.
• LoRA: Low-Rank Adaptation of Large Language Models presents low-rank learned updates while freezing base-model weights.
• Efficient Memory Management for Large Language Model Serving with PagedAttention analyzes KV-cache memory management and continuous serving pressure.
• Holistic Evaluation of Language Models motivates multi-metric, scenario-based evaluation rather than one aggregate capability score.
• NIST AI 600-1 provides maintained generative-AI risk-management guidance across the system lifecycle.