AI Agent Memory
Everything an agent keeps between turns and between sessions — sliced two ways, because each type is also its own attack surface.
A model answering a single prompt has no memory — the context window is all it knows. An agent is different: it carries state forward, so it can pick up a task where it left off, remember what you prefer, and reuse an approach that worked last week. That state splits cleanly intoshort-term memory — the live context for the task in hand, usually a rolling summary of the conversation — andlong-term memory, which survives the session and is retrieved on demand.
Long-term memory is where the interesting design choices — and the interesting security problems — live. It gets categorized two different ways: by what kind of thing is being remembered (its function), and by how it's stored and retrieved (its structure). They're not alternatives; a single system has a function and a structure, and most real agents run several combinations at once.
The Two Ways to Slice Agent Memory
The top band is the cognitive taxonomy — borrowed loosely from how human memory is described. The middle band is the engineering taxonomy — the actual data structure behind the store. The legend is what production frameworks (LangChain, LlamaIndex, LangGraph, MemGPT / Letta) tend to assemble in practice.
By function — what kind of thing is remembered
| Type | What It Holds | What It's For | How It's Stored |
|---|---|---|---|
| Episodic | Records of specific past events and interactions — "the user asked me to fix bug X on Tuesday, and here is what I did." | Continuity and personalization across sessions; being able to say what happened and when. | Timestamped entries — a log, often embedded so past episodes can be recalled by similarity. |
| Semantic | General facts and knowledge not tied to any one event — "the user prefers Python over JS," "the API rate limit is 100/min." | A stable knowledge base the agent can rely on without re-deriving it every time. | Structured records or embedded notes; closer to a profile or a wiki than a log. |
| Procedural | "How to do things" — learned skills, workflows, tool-use patterns, strategies that worked before. | Reusing a known-good approach instead of re-planning from scratch. | Reusable playbooks, scripts, or prompt templates — sometimes baked into fine-tuning rather than kept as text. |
| Reflective / meta | The agent's own summary and evaluation of its past actions — "that approach failed, next time try X." | Self-correction over time; distinct from just replaying raw episodes, because the lesson is already distilled. | Periodically generated reflection notes, written back to long-term memory as their own records. |
By structure — how it's stored and retrieved
| Type | How It Works | Good At | Watch For |
|---|---|---|---|
| Vector / embedding | Text chunks stored with embeddings; retrieval is a semantic-similarity search. | Fuzzy recall over lots of unstructured history — the default for RAG-style agent memory. | Opaque (you can't easily read what it "knows"); needs strict per-user partitioning. |
| Key-value / structured | Explicit facts as structured records, like a user-profile document. | Interpretable and editable — you can see, correct, or delete a specific fact. | Someone has to decide what gets promoted to a durable fact, and keep it current. |
| Graph | Entities and the relationships between them, stored as a knowledge graph. | Reasoning over connections — "who reports to whom," "which service depends on which." | Heavier to build and maintain; extraction errors compound across edges. |
| Buffer / summary | A rolling summary of the conversation so far, regenerated as it grows. | Keeps the active context compact and cheap without dropping the thread. | Summarization is lossy — detail the agent will later need can quietly disappear. |
| External tool / database | No internal memory store at all — the agent queries a real database, ticketing system, or filesystem. | One source of truth, existing access controls, nothing new to secure or keep in sync. | Latency and availability of that system become the agent's; query injection is a risk. |
The design choice comes down to trade-offs between retrieval accuracy, latency, cost, and how much you want the memory to be interpretable and editable versus purely embedding-based. A structured profile you can read and correct is safer to operate; a vector store scales to far more history with far less curation. Most teams use both.
The Read Path, the Write Path, and Where It Breaks
Memory has two directions, and the security problems are almost all on the way in. On the read path, the agent assembles context for a turn from the short-term buffer, a similarity search over long-term memory, and any structured profile facts. On thewrite path, after the turn, some distillation of what happened — an episode, a new fact, a reflection — gets written back to long-term memory, where it will be retrieved automatically on every future turn. Anything that lands in that store is trusted by default forever, which is exactly why the write path needs a gate.
Memory Is an Attack Surface
Every memory type on this page is useful and dangerous for the same reason: it lets something from the past influence a future decision without a human in the loop. That's the whole value, and it's the whole risk. The table maps each failure mode to the control that actually addresses it — and note that several of these map straight onto the broader AI risk taxonomies: memory poisoning is the top-ranked threat in OWASP's Agentic Security Initiative work, persisted injection isprompt injectionwith a longer fuse, and data retention is adata-protectionobligation the moment personal data is involved.
| Risk | What Happens | Primary Control |
|---|---|---|
| Memory poisoning | A false or attacker-planted "fact" gets written to long-term memory and is then trusted on every future turn. OWASP's Agentic Security work ranks this as the top agentic-AI threat. | Gate writes: validate and attribute what gets persisted, prefer editable structured facts over silent embedding, and let a human review or roll back memory. |
| Persisted prompt injection | An injection the agent reads once — in a document or tool result — gets saved into memory, so it replays automatically in sessions long after the original content is gone. | Treat retrieved memory as untrusted data, not instructions: delimit it, and separate "content that can inform an answer" from "content that can trigger an action." |
| Cross-session / cross-tenant leakage | A vector store or profile table that isn't partitioned per user returns one person's history — or one customer's data — to another. | Namespace every memory by a verified identity; make the user/tenant key a mandatory filter on every read, enforced server-side, not a convention. |
| Sensitive-data retention | Secrets, credentials, or personal data land in a durable store that outlives the conversation and was never in scope for that kind of data — including under a right-to-erasure request. | Scrub or classify before persisting, set a TTL / decay on episodic memory, and keep memory addressable so a specific record can be found and deleted. |
| Integrity & auditability | No record of what the agent knew, when it learned it, or who could change it — so a bad outcome can't be traced to the memory that caused it. | Log memory writes and reads like any other privileged operation; version the store; restrict who (and which agent) can write. |
How I'd Design Agent Memory
- Decide short-term vs long-term explicitly. A rolling summary keeps the active context cheap; only a deliberate subset of that should ever be promoted to a store that persists.
- Gate every write. Nothing reaches long-term memory without passing a scrub for secrets and personal data, a check for injected instructions, and an attribution stamp — who or what wrote this, and when.
- Namespace by verified identity. The user or tenant key is a mandatory server-side filter on every read, not a query parameter the agent is trusted to set correctly.
- Give memory a lifetime. Episodic memory should decay or expire on a TTL. A store that only ever grows is a store no one can reason about — and an ever-larger erasure and breach liability.
- Prefer editable structured facts for anything that matters. If a wrong "fact" would change the agent's behavior, you need to be able to see it, correct it, and delete it — which an opaque embedding doesn't allow.
- Treat retrieved memory as data, never instructions. Delimit it in the prompt exactly like any other untrusted context, and keep "content that can inform an answer" separate from "content that can trigger an action."
- Log and version the store. Memory reads and writes are privileged operations; when an agent does something wrong, you want to trace it to the record that caused it.
Short-term memory is a context-window optimization. Long-term memory is a database the agent writes to itself and trusts unconditionally — so it needs the same things any other trusted datastore needs: an owner, access control scoped to one identity, a retention limit, an audit trail, and a gate on what's allowed in.
See Securing Agentic AI Systemsfor how a poisoned memory becomes a cascading failure across an agent chain,AI Identity & Criticality Tiersfor the verified identity that memory reads should be scoped to, andDefense in Depth for Bedrock AI Chatfor how this site's own chatbot delimits retrieved context as inert reference.