Securing Agentic AI Systems

Everything on the rest of this site's AI governance pages — the lifecycle, the identity tiers, the cloud-specific control mappings — still applies once an AI system starts acting instead of just answering. What changes is where the risk actually concentrates. A single model has one blast radius. A chain of agents — one that reads, one that plans, one that acts — has a blast radius at every hop, plus a new failure mode that doesn't exist for a single model at all: one agent's output becoming another agent's trusted input, with nobody in the loop to catch the handoff going wrong.

This isn't a hypothetical concern from an AI-safety lab — it's the shared position of six national cyber agencies (Australia's ACSC, the US's CISA and NSA, Canada's Cyber Centre, and the UK and New Zealand's NCSCs) writing jointly on how agentic systems should be designed, built, deployed, and operated.

Where Risk Concentrates in an Agent Chain

Four categories cover most of what shows up once an agent has real authority to act, and each one traces back to a specific point in the chain rather than to the model itself.

Confused Deputy vs. Prompt Injection — the Distinction That Actually Matters

Both end with an agent doing something it shouldn't, which is why they're easy to lump together — but the mechanism is different and so is the fix. Prompt injection is malicious input hijacking behavior: a poisoned document or web page carries hidden instructions the model wasn't supposed to follow. The fix is treating external content as untrusted input, never as instructions. Confused deputy is something else entirely — an agent uses privilege it legitimately holds, on a request that looks exactly like the kind of request it's supposed to honor. Nothing gets injected; the agent is simply talked into misusing trust it was correctly granted. Input filtering doesn't catch this, because the input isn't malicious-looking — the fix is scoping that privilege so narrowly, and expiring it so quickly, that there's nothing valuable left to misuse even if the trick works.

How I'd Secure an Agent Chain

The four-phase shape below isn't a new lifecycle invented for AI — it's the same design/develop/deploy/operate discipline every mature security program already runs, applied to a system where the components happen to make their own decisions. What's different is what belongs in each phase.

Proving It's Actually Safe

Standard model evals answer "does this model produce good outputs." They don't answer "does this chain of agents behave safely once the agents start interacting" — that requires looking at the system, not the components. MIT's System-Theoretic Process Analysis for Security (STPA-Sec) exists for exactly this: it treats risk as something that emerges from how components interact, not as a property of any single component in isolation, which is precisely the failure mode a component-by-component review misses. Pair that with agent-specific evaluations — testing across autonomy levels and multi-agent conditions, not a single pass/fail benchmark — and with threat intelligence sharing across organizations, since no single team sees the full range of ways an agent chain can be steered wrong.

Where Agentic AI Deployments Actually Fail

Almost never at the model. The failures cluster at the seams: a team governs the model carefully and never threat-models the chain it sits in; credentials get issued once and never expire, so a compromised agent keeps its access indefinitely; logging captures each agent's own activity but nothing ties the chain together, so a bad outcome can't be traced back to the hop that actually caused it; and autonomy gets granted all at once instead of in phases, because a phased rollout feels slower than the business wants — right up until the first cascading failure makes the case for it after the fact instead of before.