Agentic AI Security

Threat-model authority, not just text

A threat model names the assets, trust boundaries, attacker influence, possible side effects, and controls for a concrete agent workflow. Current OWASP agentic guidance emphasizes that agents amplify familiar application risks because natural-language content, delegated identity, memory, and tools interact across a trajectory.

Inventory the attack surface

• Principals and identities
  • Who initiated the task, whose credential reaches each tool, and where delegation changes the actor?
  • Can the agent retain or reuse a token after the user's authorization changes?
• Context and memory
  • Which sources are user-authored, retrieved, generated, remembered, or supplied by another agent?
  • Which stores cross tenants, sessions, purposes, or retention windows?
• Tools and side effects
  • Which calls read, write, send, spend, delete, deploy, execute code, or change permissions?
  • Which effects are reversible, idempotent, rate-limited, or independently reconcilable?
• Control and recovery paths
  • Where can policy stop a proposal, where can an operator stop a running trajectory, and what evidence survives containment?
  • What clean state and verification are required before authority is restored?

Draw trust boundaries before choosing guardrails

A trust boundary is a point where data, identity, authority, or execution scope changes. Crossing one requires an explicit contract; the model's confidence is not a substitute for that contract.

1. 1

   Identity

   Bind the principal

   Capture the authenticated subject, tenant, purpose, permissions, and expiry in an intent envelope the model cannot expand.

2. 2

   Provenance

   Label context

   Carry source, tenant, freshness, and trust metadata with retrieved content and memory instead of flattening everything into one prompt.

3. 3

   Policy

   Constrain the proposal

   Require typed tool arguments, compare them with the original intent, and recheck subject, resource, permission, limits, and approval.

4. 4

   Runtime

   Contain execution

   Issue short-lived credentials, isolate risky tools, cap steps and spend, record effects, and expose a reliable stop mechanism.

Use the lab to test a common design mistake: controls that validate syntax but never bind an action back to the user's goal and current authority.

Enforce intent and authority outside the model

An action gateway is deterministic application code between a model proposal and a tool executor. It rejects malformed input, but it also answers the more important questions: who is acting, for what purpose, on which resource, with what current permission and approval?

The example keeps four decisions separate:

• Schema: Are the tool name and argument types valid?
• Intent: Does the proposal stay inside the user's approved task and resource envelope?
• Authorization: Does the current principal still have the required permission in the target tenant?
• Approval: Has a human confirmed the exact high-consequence action and parameters that will execute?

Give the agent the least agency it needs

Least agency means minimizing unnecessary autonomy as well as privilege. Reduce the number of available tools, the authority behind each tool, the duration of credentials, the number of autonomous steps, and the reach of each side effect.

Match controls to consequence

• Read-only, low-sensitivity actions: user-scoped identity, resource allowlists, rate limits, and trajectory logging.
• Private reads or reversible writes: fresh authorization, tenant checks, bounded arguments, idempotency, and visible review.
• External communication, money, deletion, deployment, or permission changes: exact-parameter confirmation, short-lived credentials, independent policy checks, and reconciliation.
• Code execution or critical infrastructure: isolated per-run runtime, restricted network and filesystem access, no root authority, artifact verification, hard budgets, and a human release gate.

Contain first, then investigate and recover

An agentic security incident is a credible boundary violation, unsafe side effect, or uncontrolled trajectory. Response must stop active authority and persistent influence before the team has perfect root-cause certainty.

1. 1

   Contain

   Stop propagation

   Pause the affected run, principal, capability, or tenant; revoke delegated credentials and block new side effects at the gateway.

2. 2

   Evidence

   Preserve the trajectory

   Snapshot minimized prompts, context provenance, model and policy versions, tool proposals, approvals, credentials used, and observed effects.

3. 3

   Eradicate

   Remove persistent influence

   Quarantine poisoned memory and artifacts, rotate exposed secrets, patch the failed boundary, and search related trajectories.

4. 4

   Recover

   Reconcile before resume

   Verify external state, repair or reverse effects, replay regression cases, and restore authority through a low-risk canary from clean context.

The response drill separates containment scope from the recovery gate. A restart is not recovery when credentials, memory, or ambiguous side effects remain unsafe.

Make the response sequence executable

A containment playbook turns policy into ordered runtime operations. The order matters: preserve enough evidence to reconstruct the incident, stop active authority, quarantine persistent context, reconcile real-world effects, and only then test recovery.

Preserve evidence without creating a second leak

• Record stable IDs for the user, agent, session, model, prompt, policy, tool, credential, approval, memory item, and external effect.
• Store minimized or redacted content separately from immutable event metadata and restrict who can inspect either form.
• Capture denied proposals as well as successful calls; repeated near misses can expose a weak boundary before harm occurs.
• Track containment actions with actor, timestamp, scope, reason, and result so responders can distinguish attack behavior from response behavior.
• Reconcile against the external system of record because an agent timeout does not prove that a write failed.

Evaluate trajectories and operational controls

A security evaluation tests whether the complete agent system prevents, detects, contains, and recovers from representative attacks. A safe final message does not erase an unsafe intermediate tool call.

Define release and runtime evidence

• Version the model, system instructions, tool schemas, policy rules, retrieval sources, memory behavior, and runtime image under test.
• Include benign tasks so stricter controls do not silently make the product unusable or train users to approve everything.
• Test multi-step attacks that begin with ordinary actions and escalate through memory, delegation, retries, or tool chaining.
• Measure failures by side-effect severity and reach, not only attack-detection accuracy.
• Set owners and automatic actions for denied critical calls, approval mismatches, anomalous write rates, context-integrity failures, and runaway budgets.
• Convert incidents and near misses into protected regression trajectories before restoring broad authority.

Review the production design

A production security review should prove that authority and recovery remain outside the model even when the model, context, or a connected system is compromised.

• Can every consequential action be traced to an authenticated principal, current permission, declared purpose, exact resource, and approval?
• Are untrusted documents, tool outputs, memories, and peer-agent messages prevented from becoming authority?
• Does each tool have a narrow schema, risk class, resource boundary, rate limit, timeout, retry policy, and idempotency strategy?
• Are code execution, network access, filesystem access, package installation, and secret access isolated and denied by default?
• Can operators pause one run, one principal, one capability, or one tenant without taking down unrelated work?
• Can the team revoke credentials, quarantine context, preserve minimized evidence, and reconcile ambiguous side effects?
• Is recovery gated on a known-good control, clean context, regression evidence, and a bounded canary?
• Are users shown the actual action and consequence before approval rather than a persuasive model-generated rationale?

Use current security guidance as a maintained baseline

These primary sources define useful threat categories and operating practices, but the application still needs a threat model and evidence for its own identities, tools, data, and consequences.

• OWASP Top 10 for Agentic Applications 2026 covers goal hijack, tool misuse, identity abuse, context poisoning, unexpected execution, cascading failures, and related agentic risks.
• OWASP Securing Agentic Applications Guide 1.0 translates the Agentic Security Initiative threat model into technical guidance for builders and defenders.
• NIST AI 600-1, the Generative AI Profile organizes generative-AI risk work across the lifecycle and recommends mapping, measuring, managing, and governing risks in context.
• NIST SP 800-61 Revision 3 integrates preparation, detection, response, and recovery into cybersecurity risk management.
• OpenAI's practical guide to building agents recommends layered guardrails, risk-rated tool safeguards, failure thresholds, and human intervention for high-risk actions.