AI Agent Security: The Threat Model and the Controls That Contain It

AI agent security is the practice of containing what a tool-using AI agent can do when its reasoning is wrong or manipulated, by bounding its identity, its tool permissions, its data reach and its egress paths. It assumes the agent will sometimes be wrong and limits what that costs.

Key takeaways

• Agent security is not model security. The model is a component; the security boundary is the set of actions the agent’s credentials permit.
• Design for containment rather than prevention. No filter reliably stops a probabilistic system from being talked into something, so the question is what it can reach when it is.
• Tool permissions, not agent permissions. An agent scoped to the union of every workflow it supports is over-permissioned in every one of them.
• Egress is the control most often missing. An agent that can read sensitive data and call an external API has an exfiltration path whether or not anyone designed one.
• The hardest detection problem is an action that is authorized, logged correctly and wrong. Volume and trajectory anomalies catch these; access logs do not.

A conventional application has a fixed set of code paths. A reviewer can read them and say what the system will do. An agent chooses its own path at runtime, from a tool set, on the basis of text it has read. That text can come from a document, a ticket, an email, an ERP field or another agent, and not all of it is trusted.

This is why agent security reviews that focus on the model, its provider and its training data tend to miss the real exposure. The exposure is the credential the agent holds and the set of calls it can make with it.

What is AI agent security, and why is it not model security?

AI agent security governs what the agent can do; model security governs what the model can say. Only the first one writes to your systems.

Model-level concerns are real and they are not this. Whether a provider trains on your prompts, whether the model refuses harmful requests, whether outputs are biased: those are procurement and governance questions. Agent security starts at the point where the system has credentials. From there, the controls that matter are the ones from privileged access management and zero trust networking, applied to a non-deterministic actor.

NIST SP 800-207 sets out the zero trust principle that applies most directly here: access is granted per request, per resource, on the basis of the requesting subject’s identity and the policy for that resource. An agent is a subject. It should be enrolled, scoped and evaluated like one.

What is the attack surface of an enterprise AI agent?

The attack surface is every place text enters the agent and every place the agent can act, and in most deployments the second list is shorter than the team assumes until someone draws it.

Six entry points recur across enterprise deployments:

1. The task instruction. What the user or the scheduler asks for. Usually trusted, not always, and almost never validated.
2. Retrieved content. Documents, wiki pages, tickets and records pulled into context. The retrieval index is the single largest ingress path in most agents, and its contents were written by people who did not know an agent would read them.
3. Tool responses. What a called system returns. An API that echoes a user-supplied field returns attacker-controlled text into the agent’s reasoning.
4. Other agents. In a multi-agent system, one agent’s output is another’s input, and trust is usually implicit.
5. The tool catalog. Which tools are registered and what their descriptions say. A tool description is text the model reads and acts on, which makes the catalog a supply chain. The MCP security considerations for enterprise deployments cover this path in more detail.
6. Credentials and configuration. Static keys in config, over-broad service accounts, and secrets the agent can read as data rather than hold as a capability.

Content arriving through paths 2, 3 and 4 is the mechanism behind indirect prompt injection, which is covered in full in our post on prompt injection in enterprise AI systems. This post takes the ingress problem as given and deals with what the agent can do once something gets through.

Which controls actually reduce blast radius?

Four controls reduce blast radius; the rest reduce likelihood, which is worth less because likelihood cannot be driven to zero.

A distinct identity per agent, per environment. The agent acts as itself, not as the user who triggered it and not as a shared service account. Without this, attribution during an incident is impossible and least privilege is unachievable, because the account’s permissions are the union of everything anyone needed. This is the design problem set out in agent identity and permissions.

Per-task tool allow-lists. The agent gets the tools the current task requires, not every tool it has ever needed. An agent handling a refund inquiry does not need the tool that issues refunds until the workflow reaches the step where it does.

Schema validation on every proposed action. The orchestrator checks the structure and the bounds of the action before executing it: amount within range, record in scope, target inside the allowed set. This is the control that converts “the model decided to do something strange” into a rejected call rather than a posted transaction.

An enforced gate on irreversible actions. Enforced in the orchestration layer. A gate implemented as an instruction in the prompt is a request, and a request can be overridden by other text in the context window.

Against those, the controls that mostly reduce likelihood: input filtering, instruction hardening, keyword blocklists, and asking the model to ignore embedded instructions. They are worth having and they are not a boundary.

How should an agent’s tools be scoped?

Scope tools to the workflow step, with read and write split, and treat the scope as part of the tool definition rather than as a property of the agent.

The practical test is to write down, for each tool, the smallest set of objects and operations that the step needs, then check four bounds:

Splitting read from write is the cheapest improvement available in most deployments, because it is usually a configuration change rather than a redesign, and it removes the largest class of accidental writes.

Where does data leave, and how is that bounded?

Data leaves through three paths, and all three have to be enumerated, because an agent that can read sensitive data and make an outbound call has an exfiltration path by construction.

The paths are: the model provider, if context is sent to a hosted model; outbound tools, any call that sends content to a system outside the trust boundary; and the response itself, if the agent’s output reaches a recipient who should not see what was in context.

Bounding them is three decisions. First, filter at retrieval rather than at output: do not place data in context that the requester is not entitled to see, because once it is in context, preventing its restatement is a filtering problem rather than an access control one. Second, treat every outbound tool as an egress point with its own allow-list of destinations. Third, make the recipient of the output part of the authorization decision, not an afterthought.

How do you detect an agent that is acting wrongly but legitimately?

Detect on behavior rather than on authorization, because the hard case produces a clean audit trail: the action was permitted, the credential was correct, the log entry is accurate, and the action was wrong.

Three signals carry most of the value. Volume anomalies, where an agent that normally performs a handful of writes an hour performs hundreds. Trajectory anomalies, where the sequence of tool calls differs from the established pattern for that workflow. And rejection rate, where human approvers start declining what the agent proposes, which is a correctness signal before it is a security signal.

All three require a baseline, which means the instrumentation has to exist before the incident. What to capture and what should raise an alert is covered in agent observability.

What does an agent security review cover?

A review covers the agent’s permissions, its ingress paths, its egress paths and its stop mechanism, and it is performed against the running configuration rather than the design document.

The questions worth asking in order: whose identity does this agent act as, and is that identity in the directory; what can that identity do, listed as operations rather than as role names; which of those operations are irreversible, and what enforces the gate on each; what text reaches the model that the organization did not author; which calls can leave the trust boundary, and to where; what would a reviewer see if this agent performed a thousand wrong actions in an hour, and how long would it take to notice; and who can stop it right now, without a change request.

An answer of “the team would notice” to the sixth question is the finding.

The decision

Treat the agent as a privileged non-human identity and secure it with the controls you already apply to those, then add the two that are specific to agents: per-task tool allow-lists and schema validation on proposed actions. If one change is made this quarter, split read credentials from write credentials. It is usually configuration, it is reversible, and it removes the largest class of unintended writes.

GrowExx reviews agent deployments against the running configuration rather than the design document. Our AI code audit and validation practice covers the implementation; our AI agent development team builds the containment into the orchestration layer.

FAQs