AI Agent Architectures
Build AI agent architectures: autonomous systems, multi-agent coordination, and intelligent agent design patterns.
What is an AI agent?
An AI agent is a software system that uses a model to decide what to do next, calls bounded tools to affect or inspect an environment, observes the result, and continues until it reaches a stopping condition.
A chatbot normally turns one request into one response. An agent adds an execution loop: it can gather evidence, choose a tool, update working state, retry a safe operation, ask for approval, or stop when the next action would exceed its authority.
Agents matter when a task cannot be completed by one model response. They also create a new systems problem because generated text now influences real APIs, records, messages, and money.
Core invariant: the model may propose an action, but deterministic controls must decide whether that action is authorized, affordable, valid, and safe to execute.
Review Function Calling first if tool schemas and structured model outputs are unfamiliar.
An agent is a controlled feedback loop
The model is only one component. A useful agent repeatedly connects a goal to evidence and an authorized action while preserving a trace of every decision.
From goal to verified outcome
Each loop iteration must spend a bounded budget and finish with an observation or an explicit stop.
Contract
Frame the goal
Define success, scope, deadlines, budgets, and actions that require approval.
Reason
Choose the next step
Use current evidence to select a plan step, a tool, a question, or a stop decision.
Control
Authorize and execute
Validate schema, identity, permission, policy, idempotency, and resource limits outside the model.
Feedback
Observe the result
Capture typed output, errors, provenance, latency, and side effects instead of trusting a prose summary.
Outcome
Verify or stop
Check the success condition, request approval, recover safely, or stop at the iteration and cost limit.
The loop needs explicit state
- Goal state: what must be true before the task is complete.
- Execution state: the current plan step, prior observations, remaining budget, and pending approvals.
- Environment state: authoritative data returned by tools, not assumptions retained in model text.
- Control state: identity, permissions, risk class, retry policy, and stop reason.
Choose the smallest architecture that can finish the job
Agent architecture is the shape of the decision loop. More autonomy is useful only when the task needs it and the system can observe and constrain the added behavior.
One decision
Direct router
Classify the request and call one known tool. This is fast and inspectable for narrow tasks with a stable route.
Several dependent steps
Plan and execute
Create a bounded plan, execute one step at a time, and re-plan only when an observation invalidates the next step.
Specialized paths
Supervisor and workers
A supervisor delegates typed subtasks to specialized workers. Parallelism helps only when dependencies and merge rules are explicit.
Durable state
Long-running workflow
Persist checkpoints around queues, approvals, and external events. A durable workflow engine owns time and retries; the model does not.
Complexity must buy a measurable capability
Use a direct router when one bounded action is enough. Add planning for dependent steps, delegation for separable specialist work, and durable orchestration for tasks that must survive process restarts or wait for people. The lab makes those trade-offs visible.
Tools are capability boundaries, not prompt suggestions
An agent tool is a typed interface to a capability such as search, calculation, record lookup, messaging, or mutation. Its description helps the model choose it; its runtime contract determines whether the call is allowed.
A production tool contract should define
- Input and output schemas: reject missing, unexpected, oversized, and incorrectly typed values before execution.
- Caller identity and scope: bind each call to a user, tenant, task, and least-privilege credential.
- Effect class: label the operation as read-only, reversible write, irreversible write, or privileged administration.
- Execution limits: enforce deadline, rate, cost, payload, network, and concurrency budgets.
- Retry semantics: name whether a retry is safe, requires an idempotency key, or must never happen automatically.
- Evidence and audit: return stable result codes, provenance, side-effect IDs, and timing for later verification.
Natural-language instructions inside a web page, document, tool result, or memory record are untrusted data. They do not gain authority merely because the model can read them.
The example below keeps model choice separate from deterministic authorization, execution, observation, and stop conditions.
Memory is scoped state with a lifecycle
Agent memory is any information carried from one decision to a later decision. Treating every transcript, tool result, and retrieved record as one permanent prompt makes relevance, privacy, and correctness worse.
Current run
Working state
Keep the goal, current plan, typed observations, pending approvals, and remaining budget. Discard it when the run ends unless the product needs a durable trace.
Resume safely
Durable task state
Persist checkpoints, side-effect IDs, owner, deadline, and status so a process restart does not repeat completed work.
Evidence, not authority
Retrieved knowledge
Search approved sources with provenance, access checks, freshness, and relevance. Retrieved instructions remain untrusted unless policy grants them authority.
Keep memory honest
- Store typed facts and source references instead of only free-form summaries.
- Separate user preferences from task evidence and from security policy.
- Apply tenant boundaries, retention limits, deletion rules, and access checks before retrieval.
- Detect stale or contradictory records and surface uncertainty to the planner.
- Never let remembered text expand tool permissions or override the current task contract.
Put deterministic control around probabilistic decisions
The control plane should assume that a model can choose the wrong tool, repeat a completed action, follow hostile retrieved text, exceed a budget, or claim success before verification.
1 Schema and evidence
Validate the proposal
Parse a typed action, reject unknown tools and arguments, and preserve the evidence used to propose it.
2 Identity and policy
Authorize the effect
Compare required scope with the task's delegated authority. Request approval before high-impact or exceptional actions.
3 Budget and idempotency
Execute once
Use deadlines, isolation, rate limits, and idempotency keys so timeout and retry behavior cannot create hidden duplicate effects.
4 Observation and stop
Verify and record
Read back the authoritative state, append an audit event, and stop when success, denial, budget exhaustion, or a terminal failure is proven.
The next lab injects different failures. Change permission, approval, idempotency, and recovery policy to see whether the system blocks, duplicates, or safely recovers an action.
Design failure behavior before adding autonomy
An agent run should end in a typed state such as completed, needs_approval, blocked, retryable, failed, or expired. A vague loop that keeps asking the model what to do can turn one transient fault into repeated cost or repeated side effects.
Common failures and the required response
- Tool timeout: determine whether the operation completed before retrying; use an idempotency key or read-after-write verification for mutations.
- Invalid tool output: reject the result at the schema boundary, record the raw response securely, and choose a bounded fallback.
- Prompt injection in evidence: keep the content as data, deny requested privilege changes, and continue only with trusted task instructions.
- Planner loop: stop at repeated-state, iteration, token, time, or cost limits and expose the unresolved blocker.
- Partial completion: checkpoint confirmed side effects and resume from authoritative state instead of replaying the entire plan.
- Approval timeout: expire the proposal and revalidate context before executing a later approval.
Evaluate the whole agent system
Offline answer quality is not enough because the product risk lives in trajectories, tools, and side effects.
Task and decision quality
- Measure end-state task success, not whether the trace merely looks plausible.
- Score plan efficiency, unnecessary tool calls, unsupported assumptions, and correct stopping.
- Evaluate normal, ambiguous, adversarial, stale-memory, and unavailable-tool scenarios.
Safety and control
- Track denied calls, excessive permission requests, approval rate, policy overrides, and untrusted-instruction resistance.
- Test duplicate delivery, timeout-after-success, process restart, partial failure, and idempotency behavior.
- Verify that every consequential action can be attributed to a task, identity, policy decision, and side-effect ID.
Reliability and cost
- Monitor tool latency and error rate, loop depth, tokens, cost, retries, queue age, and completion time by task class.
- Set budgets before execution and fail visibly when they are exhausted.
- Compare the agent with a simpler fixed workflow; autonomy should earn its additional latency, cost, and operational risk.
Production readiness checklist
- Start with read-only tools and the narrowest task contract.
- Keep credentials, authorization, validation, retries, and audit outside model-controlled text.
- Require human approval for high-impact, irreversible, exceptional, or policy-sensitive actions.
- Persist durable checkpoints around side effects and external waits.
- Version prompts, models, tools, schemas, policies, and evaluation sets together.
- Provide a kill switch, per-run budgets, deterministic stop reasons, and an incident replay path.
- Release by task slice with canaries and compare against a non-agent baseline.