Prompt Engineering
Learn prompt engineering techniques: few-shot learning, chain-of-thought prompting, and optimization strategies.
What is prompt engineering?
Prompt engineering is the practice of turning a product task into a clear, bounded input contract for a language model. That contract tells the model what job to perform, which evidence it may use, which rules it must follow, and what output a caller can validate.
In plain language, a model cannot reliably recover requirements that the application never supplied. Prompt engineering makes those requirements visible and testable instead of relying on clever wording or luck.
The core invariant
A prompt version is production-ready only when its behavior is measured on representative inputs and every required safety, correctness, and format gate passes. A fluent example or a higher average score cannot replace that evidence.
Review LLM Models and Families first if model context, tokens, or decoding are unfamiliar.
Task
Requested outcome
Name the decision, transformation, or artifact the model must produce.
Evidence
Permitted context
Supply current facts and mark untrusted content as data, not instructions.
Rules
Behavior boundary
State constraints, exclusions, fallback behavior, and authority limits.
Contract
Validated output
Define the schema or structure that downstream software will accept.
Build prompts as layered contracts
A useful prompt separates concerns. Each layer answers a different question, can be versioned independently, and has a distinct failure mode.
What to do
Instruction layer
Use a direct verb, a named artifact, and a definition of success. Classify the ticket is clearer when the allowed classes and escalation rule are explicit.
What to use
Context layer
Include only relevant, authorized evidence. Add provenance and freshness where facts can change, and distinguish quoted data from trusted instructions.
What must remain true
Constraint layer
Bound scope, tone, length, policy, tool authority, and abstention. Constraints should describe observable behavior, not vague goals such as "be good."
What callers receive
Output layer
Specify fields, allowed values, required evidence, and error behavior. Validate the response after generation; prose that resembles JSON is not a contract.
Instruction priority and trust are not the same as text position. Application-owned instructions define the task; user input, retrieved pages, emails, and documents remain untrusted data even when they contain imperative language.
Trace the prompt lifecycle
Prompting is a software delivery loop. Start from a task contract, build the request deterministically, evaluate complete responses, and preserve the evidence behind each release.
1 Contract
Frame the task
Name the user outcome, allowed evidence, non-goals, risk level, and the behavior expected when information is missing.
2 Build
Assemble the request
Combine versioned instructions with authorized context, bounded examples, and a machine-checkable output contract.
3 Run
Generate and validate
Call a pinned model configuration, then check schema, citations, policy, business rules, and tool permissions outside the model.
4 Prove
Replay and release
Compare candidates on frozen and adversarial slices, inspect the weakest required slice, canary the winner, and retain rollback evidence.
Loading prompt scenarios
Preparing the contract layers and consequence model.
Choose a technique for a specific uncertainty
Prompt techniques are tools, not maturity levels. Add one only when it addresses an observed failure and its benefit survives evaluation.
Known task
Direct instruction
Use a zero-shot instruction for familiar, bounded work when the task, labels, evidence, and output contract are already unambiguous.
Ambiguous boundary
Few-shot examples
Show representative input-output pairs when labels, edge cases, style, or formatting are hard to define in prose. Include difficult and negative cases, not only ideal ones.
Multi-stage work
Decomposition
Split a complex task into observable sub-results or tool calls. Validate each boundary instead of requesting private reasoning as proof that the final answer is correct.
Software consumer
Structured output
Use a native schema or constrained decoder when software consumes the result. Retry only bounded, repairable failures and keep the original request identity.
Treat role and tone as secondary controls
- A role can set audience, vocabulary, or perspective, but it does not grant expertise, facts, or tool authority.
- Positive instructions usually test better than long lists of prohibitions because they define the desired behavior directly.
- Delimiters improve readability, but they do not make untrusted text safe or stop prompt injection by themselves.
- Asking for a concise rationale can aid review, but generated reasoning is still model output and needs independent verification.
Budget context, output, and retries together
Every prompt competes for a finite context window. Reserve output capacity first, then allocate the remaining input budget to instructions, current user data, evidence, history, tool contracts, and examples.
available evidence tokens = context limit - fixed instructions - user input - examples - tool schemas - output reserve
For a request with a 16,000 token limit, 1,200 fixed instruction tokens, 2,300 user and history tokens, 1,500 example tokens, 1,000 tool-schema tokens, and a 2,000 token output reserve, only 8,000 tokens remain for evidence.
8k
Evidence allowance
Useful evidence must fit after every fixed and reserved segment.
2k
Output reserve
Input admission cannot consume the tokens required for the promised response.
Tail
Latency pressure
Longer inputs increase prefill work and can increase queue pressure and time to first token.
Accepted
Cost denominator
Retries, validation, and review belong in cost per accepted task.
Estimate complete task cost
Suppose one call uses 1,200 input tokens at $0.50 per million and 250 output tokens at $1.50 per million. Raw model cost is $0.000975 per call. At two million calls, that is $1,950 before retries and review. With an 8% retry rate and 90% accepted-task rate, modeled cost per accepted task rises to about $0.00117, or $2,340 for two million attempted tasks.
Put validation outside the model
A prompt template belongs inside a request path with explicit owners for context, model execution, validation, and authority. The model proposes an output; application controls decide whether that output is usable.
A governed prompt request
Untrusted content never becomes application authority merely because it appears inside the context window.
Identity and budget
Request gateway
Authenticate the caller, authorize the task, apply rate and token limits, attach a request ID, and reject impossible deadlines.
Versioned assembly
Prompt builder
Load a pinned template, select authorized context, escape or delimit data, reserve output capacity, and record every component version.
Generation
Model runtime
Execute a pinned model and decoding configuration. Propagate cancellation and never infer tool permission from generated text.
Evidence and schema
Validation gate
Parse the response, verify required fields and citations, apply policy and business rules, and send repairable failures through bounded retries.
Authority boundary
User or tool
Present the answer with uncertainty or execute only explicitly permitted actions. Consequential side effects require approval and idempotency.
Design for adversarial and ordinary failures
Most prompt failures are not fixed by adding more prose. Identify the failing boundary, then add a control that can be evaluated independently.
Trust failure
Prompt injection
Retrieved or user-supplied text asks the model to ignore policy, reveal secrets, or call tools. Keep authority outside the prompt, minimize data access, validate tool arguments, and test hostile content.
Evidence failure
Missing or stale context
The model fills a gap from learned patterns or obsolete material. Require provenance and freshness, and abstain or retrieve again when evidence is insufficient.
Contract failure
Format drift
The output is incomplete, malformed, or semantically invalid. Prefer constrained structured output, validate it, and cap repair attempts under one request identity.
Release failure
Evaluation overfit
Examples or instructions improve a familiar test set while degrading noisy, multilingual, long-context, or adversarial traffic. Preserve holdouts and gate required slices separately.
Fail in bounded ways
- Ask a clarifying question when the task is ambiguous and clarification is allowed.
- Abstain when required evidence is absent, conflicting, stale, or unauthorized.
- Return a typed validation error when the output contract fails after the retry budget.
- Disable or narrow tool use when identity, policy, or approval services are unavailable.
- Roll back the complete prompt-model-validator bundle when a monitored slice regresses.
Evaluate prompts as product changes
An evaluation set should represent the traffic and risks the prompt is expected to handle. Keep test inputs, expected behavior, scorers, model configuration, and prompt version traceable as one evidence bundle.
Build an evaluation portfolio
- Happy paths: common inputs with clear expected behavior.
- Boundary cases: missing fields, conflicting evidence, long inputs, uncommon labels, and valid abstentions.
- Adversarial cases: injection attempts, data exfiltration requests, unsafe actions, and malicious tool arguments.
- Format cases: empty arrays, escaped text, maximum lengths, and schema-invalid model output.
- Governed slices: language, region, customer tier, risk class, input source, and accessibility needs relevant to the product.
Use deterministic validators for schema and business invariants. Use human review for nuanced quality with a rubric and agreement checks. If an LLM grades another model, calibrate it against human labels, version the judge prompt and model, and monitor position, verbosity, and self-preference bias.
Loading release evidence
Preparing candidate scores, risk slices, and release policies.
Release and operate a complete bundle
A production prompt is coupled to a model, decoding configuration, context schema, tool contract, and validators. A text-only diff cannot explain the behavior of that bundle.
1 Qualify
Replay offline
Run frozen holdouts and adversarial suites. Compare quality, abstention, schema success, latency, token use, and the weakest required slice.
2 Observe
Shadow real traffic
Execute the candidate without changing user-visible behavior. Verify context assembly, validation, cost, and policy telemetry.
3 Limit
Canary by cohort
Expose a stable, reversible cohort. Preserve the control group and stop automatically when a hard gate or telemetry requirement fails.
4 Decide
Expand or roll back
Increase exposure only with healthy slice evidence. Keep the previous bundle deployable and record why the release decision was made.
Observe causes, not raw text by default
- Record prompt, model, context-schema, tool, validator, and experiment versions.
- Track accepted-task rate, abstention, schema failures, bounded retries, validation reasons, and cost per accepted task.
- Break quality, latency, and safety metrics down by governed slice instead of relying only on fleet averages.
- Minimize retained prompt and response text; apply purpose, access, encryption, and deletion controls to sampled traces.
- Alert on missing evaluation or policy telemetry because an unobservable release cannot be governed.
Keep these prompt engineering rules
- Start from the task contract, not from a catalog of prompt tricks.
- Separate trusted instructions from untrusted user, retrieved, and tool data.
- Use examples to clarify boundaries; keep holdouts separate so examples do not become the test.
- Reserve tokens for output and validation before filling the context window.
- Treat structured output as a schema plus external validation, not a formatting suggestion.
- Measure the weakest required slice; averages can hide a release-blocking failure.
- Version prompts with the model, context schema, tools, validators, and evaluation evidence.
- Keep secrets, permissions, and consequential authority outside model-generated text.