Agent Memory: Architecture, Types, and Trade-Offs

Memory Is Four Things, Not One

Agent memory is the set of mechanisms by which an AI agent retains and retrieves information across turns, sessions, and other agents. It is not a single store. It comprises working memory (the live context window for the current step), episodic memory (an ordered record of past interactions and events), semantic memory (durable facts, preferences, and learned patterns), and shared memory (state that multiple concurrent agents read and write). Each has a different lifetime, a different consistency requirement, and a different way of going wrong.

Ask an engineering team what they use for agent memory and you will usually get a product name. That skips the design question. A product covers a retrieval pattern; memory is a set of requirements, and the requirements barely overlap. Sort them by two axes — scope (one turn, one agent, one tenant, everyone) and lifetime (one request, one session, indefinitely, until the next write) — and the architecture mostly falls out.

We have written elsewhere about the three-layer memory model for production agents and surveyed the tools that cover each layer. This post is the map above both: the types, what each is for, the honest trade-offs, and the case almost nobody writes about — memory shared across concurrent agents rather than private to one.

Working Memory: The Context Window Is Not Storage

Working memory is what is physically in the prompt when the model runs. It is the only memory the model can actually reason over; everything else is a retrieval system whose job is deciding what gets promoted into it this turn.

That framing kills a common confusion. Teams say “we increased the context window, so we solved memory.” A larger window changes the budget for working memory. It gives the agent no way to know which facts are current, which have been superseded, or which another agent changed thirty seconds ago.

The trade-off. Working memory has zero retrieval cost and no infrastructure. It is also the most expensive per token and the least trustworthy under pressure: when the window fills, something gets dropped, chosen by a heuristic that has no idea which fact was load-bearing. Long-horizon agents relying on it alone degrade characteristically — they stay fluent and stop being correct.

What belongs here. The current task, the current turn, the small set of facts the step depends on, and pointers to everything else. Not the transcript. Not the knowledge base.

Episodic Memory: What Happened, In Order

Episodic memory is the ordered record of events the agent observed or caused: messages, tool calls, results, actions, outcomes. It is append-only and temporally ordered, and nothing in it is revised, because it records what occurred rather than claiming what is true.

It answers questions no other layer can: what did this agent know when it decided, what has already been tried, what did we promise the customer, what is the audit trail. If you cannot reconstruct the inputs to a past decision, you cannot debug it or defend it.

The trade-off. It grows without limit and most of it is never read. Both standard responses cost something. Summarization is cheap and lossy in a way you discover later — the detail that mattered is usually the one the summarizer judged unimportant. Similarity retrieval over the raw log is lossless but retrieves by resemblance when the question was about order. “The last five things that happened to this account” is a range scan over an ordered log, not a nearest-neighbor query; an embedding search answers it plausibly and wrongly.

The rule. Keep the raw ordered log as the record of history and build summaries and embeddings as derived views over it, never as replacements. A derived view you can recompute is an optimization; one that destroyed its source is a liability.

Semantic Memory: What Is Generally True

Semantic memory holds durable facts, learned preferences, entity profiles, domain knowledge, and policy — mutable, but slowly revised. “This customer prefers email over phone” belongs here. So does “refunds above $500 require a supervisor.”

This is the layer most teams build first, usually as a vector index — and the one layer where a vector index is genuinely the right tool.

The trade-off. Semantic memory is the layer most prone to holding several versions of one fact at once. Facts are extracted from conversations and documents at different times, written as separate records, and retrieved together. Nothing in a similarity index expresses “this record supersedes that one.” The agent gets two contradictory chunks in the same retrieval and picks whichever the model finds more fluent.

Fixing that requires what a similarity index lacks: identity and revision. A fact needs a subject, a value, a validity period, and a write path that supersedes prior values instead of accumulating beside them. The structured part of semantic memory belongs in a store with real keys and updates, with the embedding index as a retrieval path over it rather than its home.

The less-discussed trade-off. This is the layer where staleness is genuinely tolerable, and teams over-engineer it anyway. A preference learned last week is fine. A policy updated this morning can propagate in minutes. Consistency budget spent here is spent in the wrong place.

Shared Memory: The Case Almost Nobody Covers

Everything above is private memory: one agent accumulating context over time. Nearly all writing about agent memory stops there, because that is the chatbot case — one assistant getting better at serving one user across sessions.

Private memory is forgiving for a rarely-stated reason: only one actor is involved. If the agent’s recollection is a few seconds behind, nothing happened in those seconds except what that same agent did. The memory is behind reality, but the agent is reality. Staleness costs a little quality and no correctness.

Shared memory breaks that. It is state more than one agent reads and writes in the same window: a spend envelope several purchasing agents draw against, a customer account touched simultaneously by a voice agent and a chat agent, an inventory position, a quota, a live configuration a control loop is adjusting while agents act on it. The staleness argument collapses here, because between one agent’s read and its write, another agent acted.

The failure is not exotic. Two agents read a remaining budget of $200. Both check their $150 request against it. Both pass. Both commit. The budget is overdrawn and neither agent did anything wrong — each decided correctly against the context it was given. No amount of prompt engineering addresses this. It is a memory architecture failure: the shared state each agent read did not reflect what the other had just done.

Three conditions have to hold together for this to be your problem:

Concurrency, not just multiplicity. A thousand agents each working on their own tenant’s data is partitioning, not contention. Ten thousand agents on ten thousand separate documents have no shared-memory problem. Three agents on one budget do.

No human gate. If a person reviews before the action commits, the review is the serialization point and ordinary propagation keeps up. Contention matters when the agent acts directly.

State that cannot merge. The condition most often missed. Concurrent coding agents working through git branches are already serialized by twenty years of merge tooling and a pull-request review — that state merges, so contention on it is handled. Money, quotas, mandates, velocity counters, seat allocations, deployed configuration do not merge; there is no three-way merge for a dollar. Ask directly: when two agents hit this state simultaneously, what resolves the conflict today? If the answer names a real mechanism, you are fine. If it is “our pipeline usually keeps up,” you have found the problem.

Where all three hold, shared memory’s consistency requirement is categorically different from the other three types. Not “fresh enough for good answers” — correct at the moment of the read, because a decision gates on it.

Memory Is Not State

A second confusion compounds the first, because the two words get used interchangeably.

Memory is what the agent recalls about the interaction. State is what is true in the business right now.

An agent that remembers a customer said their balance was $4,200 has a memory. The balance is state, and state has an owner elsewhere that changes it without telling the agent.

The error: an agent retrieves a balance, holds it in working memory, discusses options for four minutes, then authorizes a transfer against the number it recalls. In those four minutes a card settled. The agent did not read a stale value — it read a correct value and then treated a memory of a past read as a statement about the present. It is invisible in testing, because in testing nothing else moves.

The rule is unglamorous and absolute: anything the decision gates on must be read at decision time, not recalled. Memory can tell the agent what the customer intends and what was discussed. It must not supply the number the transfer is checked against. Recall is for context; a fresh read is for the gate.

That is also the real line between the stateful and stateless agent patterns. The interesting agents are hybrid on this axis: stateful about the conversation, stateless about the business facts.

What the Storage Layer Under Shared Memory Has to Provide

Working, episodic, and semantic memory are well served by existing components. Shared memory is where the composed stack runs out, so be specific about what the store underneath it has to do.

Serve every retrieval pattern the decision needs under one coherent snapshot. An agent step usually needs three things at once: structured state (the balance, the quota, the permission), derived signals (spend in the last hour, a velocity count), and semantic retrieval (similar past cases, relevant policy). In a composed stack those come from three systems at three propagation stages, so the agent reasons over a combination of values that never coexisted. Coherence across retrieval patterns is what makes the context a description of one moment rather than three.

Keep derived context current enough to matter. Counters and aggregates a gate depends on cannot be maintained on a pipeline cadence measured in seconds when the decision window is shorter than that. Maintaining them as incremental materialized views over the same underlying data, updated from change data capture rather than rebuilt on a batch schedule, narrows that lag to sub-second — asynchronously, with real lag, not zero. Usually that is enough, because at normal per-entity rates the derived value reflects the previous action before the next decision arrives.

Offer a real serialization point where the state cannot tolerate a race. Freshness alone does not solve this: if two agents must not both succeed, something has to be authoritative for that record and enforce it transactionally.

Those three requirements together describe closing the context gap: the distance between what a decision needs and what it can actually see, when the context reaching it is incomplete, inconsistent, or outdated at the moment it runs. That is what the Tacnode Context Lake™ is built for: real-time, multi-modal context infrastructure that serves structured state, derived signals, and semantic retrieval from one internally coherent view. The split between the last two requirements maps onto how it is deployed. In the primary pattern, Tacnode reads from your existing system of record asynchronously, via change data capture or streaming, and is not in your write path. What changes is the read side: every agent reads structured state, derived signals, and semantic retrieval from one place, against the same set of ingested events, instead of assembling them from stores at different propagation stages. The agent’s decision sees accurate current context, so it does not approve what it would have blocked had it seen current state.

When the contended state is agent-owned — allocations, coordination records, the decisions the agents themselves produce, with no upstream system that owns them — Tacnode can be the system of record for that dataset. That is Pattern 2, and the only place the stronger claim applies: writes are transactional, and two agents cannot both commit a conflicting action against the same record. Label that deliberately rather than assuming the read-side pattern gives you enforcement it does not.

Most agent systems need both. Getting the split right starts with sorting your memory by type and being honest about which parts are private and which are shared — and for the broader failure pattern shared memory belongs to, see why real-time decisions fail and the decision coherence architecture that addresses it.

Frequently Asked Questions