LangChain

Start with the smallest execution model

Not every model-powered feature needs an agent. More autonomy adds more runtime choices, state, failure paths, and evaluation work.

An agent is useful only when runtime choice is part of the requirement. If the path is known, keeping it deterministic usually makes latency, cost, testing, and recovery easier to reason about.

See what the agent harness actually does

LangChain describes an agent as a model plus a harness. The harness owns the loop around the model: messages enter state, the model proposes a tool call or final answer, application code executes an allowed tool, and the result returns to the model.

A valid tool call is not automatically an authorized tool call. Schema validation proves shape; application policy proves whether this caller may perform this action now.

Compose an execution pattern before writing code

Choose a workload and compare a direct call, deterministic workflow, and agent loop. Then change the output and state contracts to see when a design is incomplete or unnecessarily wide.

The goal is not to maximize framework features. It is to make each model call, retrieval, state transition, and side effect explicit enough to test and operate.

Keep model, message, tool, and state boundaries distinct

The main primitives solve different problems. Mixing their responsibilities is a common source of accidental complexity.

Use runtime context for immutable invocation facts such as user ID, role, locale, or tenant. Use state for mutable thread facts. Use a long-term store only for information that should survive across threads.

Make machine-consumed output typed

Free-form text is appropriate when the user is the consumer. When application code will branch, store, or call another service, ask the model for a schema and validate the result before use.

A typed boundary gives you

• A finite set of fields and allowed values.
• A validation failure instead of silent string parsing.
• A stable object for tests, metrics, and downstream routing.
• A place to enforce length, range, and optionality constraints.

A typed boundary does not give you

• Proof that a classification or extracted fact is correct.
• Authorization to read or change a business resource.
• Idempotency for a downstream write.
• Protection from an unsafe or irrelevant input.

The example uses init_chat_model and with_structured_output for one bounded transformation. It deliberately does not introduce an agent or persistent state.

Add retrieval as a bounded context step

Retrieval does not automatically require an agent. First choose who controls when retrieval happens.

Preserve the context contract

• Treat retrieved text as untrusted data, not as new system instructions.
• Attach source identity, version, access scope, and chunk location to every passage.
• Filter by tenant and permissions before content reaches the model.
• Measure retrieval relevance separately from answer quality.
• Return citations that map to the exact evidence used.

The official retrieval guide distinguishes deterministic two-step RAG from agentic retrieval and hybrid designs.

Give state a lifetime and an owner

LangChain agents keep short-term memory in their graph state. A checkpointer saves that state by thread so later invocations can resume it. Persistence is useful, but it creates a data contract that needs retention, isolation, and migration rules.

1. 1

   Scope

   Identify the thread

   Bind state to an authenticated thread identifier and tenant. Do not use a model-provided value as proof of identity.

2. 2

   Read

   Load bounded state

   Restore recent messages and explicit fields. Trim or summarize stale history before it degrades relevance, latency, and cost.

3. 3

   Execute

   Run one transition

   Call the model and tools through the same policy boundary. Persist checkpoints at meaningful interruption and recovery points.

4. 4

   Govern

   Retain deliberately

   Expire, delete, encrypt, and migrate state according to product and compliance requirements. Keep long-term user memory in a separately governed store.

Long conversation history is not free memory. Even when it fits the model context window, stale messages can reduce quality while increasing latency and spend. The short-term memory guide documents checkpointers, trimming, deletion, and summarization patterns.

Put policy around every tool loop

An agent loop can amplify a small mistake because the model may call tools repeatedly or combine untrusted context with an allowed action. Inject an incident and compare control profiles before choosing a retry policy.

The important boundaries are outside the prompt

• Filter tools from authenticated runtime context and current permissions.
• Validate arguments and business invariants again inside the tool.
• Require human review for consequential writes.
• Give writes caller-scoped idempotency keys and reconcile ambiguous outcomes.
• Limit model and tool calls so a non-converging loop terminates explicitly.
• Redact sensitive values before exporting traces or feedback.

Middleware runs inside the compiled agent graph, so controls such as human approval, tool-call limits, retries, redaction, and summarization can travel with the agent when it is composed into a larger workflow.

Build a narrow guarded agent

The example below exposes one read tool and one write tool. It caps tool calls, keeps thread state in a checkpointer, and interrupts before the refund tool executes.

For local learning, InMemorySaver makes the interruption contract easy to see. In production, use a durable checkpointer and persist the interrupt before returning an approval request to the user.

Operate traces as evidence, not decoration

Final answer quality is only one part of an agent system. A production trace should reveal which model ran, what context it received, which tools were exposed, each tool argument and result boundary, retries, latency, token usage, errors, and the final disposition.

1. 1

   Contract

   Define success

   Create examples for correct final answers, structured fields, retrieval evidence, tool selection, tool arguments, refusal, and side-effect policy.

2. 2

   Offline

   Evaluate before release

   Run representative datasets against each prompt, model, tool, or middleware change. Compare quality, trajectory, latency, and cost.

3. 3

   Online

   Observe real behavior

   Trace production runs with privacy controls, sample intentionally, and alert on policy failures, loops, latency, errors, and quality signals.

4. 4

   Regression

   Close the loop

   Turn important failing traces into reviewed offline cases, prove the fix, then confirm the production signal recovers.

LangSmith can provide tracing and evaluation for LangChain applications, but it is optional infrastructure rather than proof of quality. The evidence comes from well-defined evaluators, reviewed datasets, and operational decisions tied to trace data.

Review the production contract

Execution and state

• Choose direct call, deterministic workflow, or agent from the workload rather than fashion.
• Bound model calls, tool calls, context size, wall-clock time, and spend per request.
• Persist only state that must survive and define tenant isolation, retention, deletion, and schema migration.

Tools and authority

• Expose the smallest tool set for the authenticated user and current stage.
• Validate schema, permissions, ownership, business rules, and approval before every side effect.
• Reconcile ambiguous writes before retrying and make repeatable operations idempotent where possible.

Quality and recovery

• Version prompts, schemas, tools, middleware configuration, and evaluation datasets together.
• Trace the complete trajectory while redacting sensitive content.
• Test provider errors, tool timeouts, malformed results, non-converging loops, interrupted approvals, and checkpoint recovery.
• Define an explicit incomplete or escalated outcome instead of forcing every run to produce a confident answer.

Sources and version notes

LangChain APIs and recommended architecture continue to evolve. This lesson follows the current official Python documentation while authoring and avoids older imports such as LLMChain, ConversationBufferMemory, and RetrievalQA that appeared in the previous page.

• LangChain overview and create_agent
• Models and the standard model interface
• Agents, tools, state, and structured output
• Tool runtime context
• Middleware and the agent loop
• Prebuilt middleware controls
• Short-term memory and production checkpointers
• LangSmith evaluation concepts