Translation Systems

Define translation as a typed contract

A request needs more than a text string. It should name the source and target locales, domain, terminology policy, protected spans, risk tier, and deadline. The response should carry the model and tokenizer versions, quality evidence, and a release decision.

Keep these fields explicit

• Language direction: sourceLocale and targetLocale, not a single ambiguous language field.
• Domain: support, legal, medical, commerce, or another governed vocabulary and risk context.
• Protected spans: variables, tags, URLs, product identifiers, numbers, and text that must be copied exactly.
• Terminology set: versioned source-to-target terms with precedence and ownership.
• Risk tier: determines release thresholds, fallback behavior, and human-review obligations.
• Trace identity: request, segment, model, tokenizer, glossary, policy, and evaluation versions.

Tokenization is part of the quality contract

Models do not read words directly. A tokenizer maps source text to subword IDs and later maps generated target IDs back to text. SentencePiece demonstrated language-independent subword tokenization learned from raw sentences, which is useful when whitespace does not provide a universal word boundary.

Audit representation by language and content shape

• source tokens per character and per sentence;
• unknown, byte-fallback, and replacement-character rates;
• segmentation around names, numbers, URLs, and markup;
• round-trip preservation of protected spans;
• truncation rate by language, script, and document type;
• tokenizer-version drift between training, evaluation, and serving.

Use architecture to preserve source evidence

The original Transformer was evaluated on machine translation with an encoder-decoder design based on attention. That structure still provides a useful mental model even when a production fleet mixes specialized translation models, multilingual models, and prompted general-purpose models.

1. 1

   Source side

   Encode source context

   Build contextual source representations so a token can be interpreted using the surrounding sentence or bounded document context.

2. 2

   Target side

   Attend while decoding

   Generate each target token using prior target tokens and cross-attention to the encoded source.

3. 3

   Policy

   Constrain invariants

   Apply target-language, terminology, placeholder, length, and prohibited-output constraints where the serving stack supports them.

4. 4

   Release

   Verify completed text

   Run deterministic checks and learned quality estimation before any irreversible downstream action.

Choose a model route, not one universal winner

• Specialized pair model: useful when one language direction and domain justify dedicated quality, vocabulary, and capacity.
• Dense multilingual model: shares parameters across many directions and simplifies the fleet, but needs per-pair capacity and quality evidence.
• Sparse multilingual model: activates a subset of parameters per token; routing balance and expert availability become operational concerns.
• General-purpose language model: can use instructions and context flexibly, but needs translation-specific faithfulness, terminology, latency, and cost gates.
• Human workflow: remains the correct route when impact is high and machine evidence cannot support release.

The No Language Left Behind work is a useful primary reference for multilingual data mining, conditional computation, broad language-pair evaluation, and toxicity measurement. Its reported results describe that research system, not a universal production baseline.

Decoding turns probabilities into a product decision

Autoregressive translation generates one target token at a time. Greedy decoding keeps one locally best continuation. Beam search keeps several partial hypotheses and can recover a better complete sequence, but wider search spends more compute and can still prefer a fluent candidate that violates terminology or source facts. Beam-search research explicitly treats candidate width as a quality-versus-speed decision.

Separate model preference from release constraints

1. Generate or retrieve a bounded candidate set under a deadline.
2. Reject candidates that break placeholders, numbers, language, or required terms.
3. Rank the admissible candidates with model and quality evidence.
4. Return no machine translation when the remaining evidence is insufficient.

The workbench below uses illustrative candidate evidence so you can see why a higher model score is not equivalent to a releasable translation.

Budget target-token throughput and tail latency

Source encoding can process tokens in parallel, while a standard autoregressive decoder repeatedly generates target tokens. Capacity therefore depends on target length, beam width, batch composition, model route, and queue delay, not just requests per second.

Worked serving envelope

Assume an illustrative peak of 1,500 requests per second, 95 generated target tokens per request, and 1,100 effective target tokens per second per serving replica for the approved mix:

1,500 requests/s x 95 target tokens/request = 142,500 target tokens/s

142,500 / 1,100 = 129.5, so the fleet needs at least 130 equivalent replicas before headroom. Adding 30% recovery and skew headroom gives ceil(130 x 1.30) = 169 replicas.

These are design assumptions, not benchmark claims. Measure the actual deployed model on representative language directions, lengths, hardware, batch policies, and decoding settings.

Control pressure before queues hide it

• bucket batches by compatible model route and similar sequence length;
• cap queue age and propagate the remaining request deadline;
• admit by target-token budget rather than request count alone;
• reduce beam width or use an approved smaller route only under an explicit degraded-mode policy;
• reject or defer work that cannot finish safely before its deadline;
• test loss of a model pool, tokenizer service, glossary store, and complete serving region.

Evaluate meaning, form, and harm separately

BLEU measures n-gram overlap against references and remains useful for controlled regression tracking. Learned metrics such as COMET can align more closely with human judgments by using source and reference representations. Neither proves that every number, entity, negation, or required term is correct.

Slice every release claim

• language direction, not language family alone;
• domain, content type, and source quality;
• sentence length, document context, and formatting density;
• named entities, numbers, negation, ambiguity, and terminology;
• gender, dialect, script, and other relevant user-impact slices;
• model, tokenizer, glossary, decoding, and policy version.

The MQM-based human evaluation study shows why expert context and explicit error categories matter when systems are already strong. Use automatic metrics for coverage and speed, then use targeted human evidence to understand consequential errors.

Treat safety as source faithfulness plus user impact

Translation can add toxicity, erase identity information, amplify stereotypes, expose private text, or turn a benign source into harmful target content. A large multilingual study found that added-toxicity behavior varied across its evaluated languages and recommended language-specific mitigation rather than assuming one global safety rate. See Toxicity in Multilingual Machine Translation at Scale for the study design and measured scope.

Put controls at independent boundaries

• Before inference: authenticate, authorize, classify content, minimize retained text, and reject unsupported language or risk routes.
• During preparation: preserve immutable spans, bound document context, and keep tenant terminology isolated.
• After generation: detect target language, prohibited additions, unsupported claims, critical omissions, toxicity changes, and invariant failures.
• Before release: require human review for high-impact content or weak evidence, and retain the decision lineage.
• After release: support correction, incident investigation, deletion obligations, model rollback, and affected-segment replay.

Operate a versioned translation fleet

1. 1

   Detect

   Observe by slice

   Track volume, queue age, p95 and p99 latency, abstention, fallback, invariant failures, quality estimates, and human corrections by language direction and domain.

2. 2

   Diagnose

   Attribute the change

   Compare model, tokenizer, glossary, decoding, normalization, traffic mix, and dependency versions before blaming one aggregate score.

3. 3

   Respond

   Contain the route

   Pause the affected direction, restore a compatible bundle, tighten release gates, or move eligible work to a proven fallback.

4. 4

   Verify

   Prove recovery

   Replay critical suites and canaries, compare against the last approved bundle, and restore traffic gradually with slice-level evidence.

Version and roll back together

• model weights and architecture;
• tokenizer, language tags, and normalization rules;
• glossary, protected-span parser, and locale formatter;
• decoding parameters and candidate-ranking policy;
• quality estimators, thresholds, and safety classifiers;
• evaluation sets, human-review rubric, and release manifest.

Primary references

• Attention Is All You Need introduces and evaluates the Transformer encoder-decoder architecture on translation tasks.
• SentencePiece describes language-independent subword tokenization learned directly from raw text.
• Beam Search Strategies for Neural Machine Translation studies candidate-width and decoding-speed trade-offs.
• BLEU and COMET document two different automatic-evaluation families.
• Experts, Errors, and Context presents contextual expert evaluation based on explicit MQM error analysis.
• No Language Left Behind documents a multilingual translation research system, its evaluation breadth, and toxicity benchmark.
• Toxicity in Multilingual Machine Translation at Scale measures added toxicity across a broad multilingual evaluation set.

Research results are attributed to their published systems and datasets. Capacity numbers and thresholds in this lesson are explicitly illustrative design inputs.