What Is Agentic Memory?

Why memory is the thing that makes agents actually useful

A single LLM call is stateless. You send a prompt, you get a response, and the model forgets everything the moment the call ends. That's fine for a one-shot question. It's a serious problem when you're building an agent that needs to book a follow-up appointment, remember a client's preferences, or pick up a workflow where it left off yesterday.

Most early AI demos skip this problem by keeping everything inside one long prompt. That works until the conversation gets long or complex, at which point you hit the model's context window limit and information starts falling off the edge. Real production agents need a memory architecture, not just a long prompt.

The four types of agentic memory, explained plainly

In-context memory is the simplest kind. It's everything currently sitting inside the active prompt window. Fast to access, zero latency, but capped by the model's context limit. GPT-4o supports roughly 128k tokens. Llama 3.1 70B, which we deploy on-premises, supports the same. When a conversation or task exceeds that limit, something has to be summarized or dropped.

External memory solves the scale problem. The agent writes facts, summaries, or structured data to a database and retrieves them later via search or lookup. This is where vector databases like Chroma or Weaviate come in. The agent embeds a query, finds semantically similar stored records, and pulls them into the current prompt. It's slower than in-context recall, but it doesn't have a size ceiling. This is also the layer that connects directly to RAG pipelines.

Procedural memory stores how-to knowledge: system prompts, tool definitions, workflow templates, and fine-tuned model weights. It doesn't change per conversation. It defines what the agent knows how to do at all. Episodic memory records what actually happened in past sessions: which actions the agent took, what the user said, what outcomes occurred. This is what lets an agent say 'last time you asked about X, here's what we found' without re-doing the work.

When memory architecture gets complicated

For simple single-agent workflows, in-context plus a small external store is usually enough. The design gets harder when you have multiple agents sharing state. If an orchestrator agent delegates to a scheduling sub-agent and a billing sub-agent, all three need consistent access to the same memory layer without overwriting each other's writes. That requires explicit memory scoping and, in regulated industries, audit logging of every read and write.

In healthcare specifically, episodic memory that stores patient interactions becomes PHI under HIPAA the moment it's tied to an identifiable individual. That changes your entire storage architecture. You can't just write session logs to a generic vector database. The store needs to sit inside your HIPAA-compliant infrastructure, covered by a signed BAA with every vendor who touches it.