AI agent memory is not one product category. It is four — vector databases, agent memory libraries, OLTP-plus-cache stacks, and Context Lakes — each solving a different slice of the problem. Here is what each category covers and where each one falls short.
TL;DR: AI agent memory tools fall into four categories. Vector databases (Pinecone, Weaviate, Chroma, Qdrant) handle semantic retrieval — finding relevant content by similarity. Agent memory libraries (Mem0, Zep, Letta, LangMem) handle episodic memory — conversation continuity and summarization across sessions. OLTP-plus-cache stacks (Postgres + Redis + Kafka) handle state — the live operational data agents act on. Context Lakes (Tacnode) unify all three under one coherent substrate so agents read fresh, internally consistent context at decision time. The first three each solve one slice. Production agents need all three slices, and need them coherent. :::
“What should we use for our agent’s memory?” is the wrong question. It assumes memory is one thing you pick a tool for, like picking a database. It is not. AI agent memory is a set of distinct requirements that today’s market splits across four different product categories — and most teams pick one, hit a wall, and bolt on the others.
This is a survey of those four categories: what each one is, which agent memory problem it solves, and where the gap shows up when you push it past a demo. The categories are vector databases, agent memory libraries, OLTP-plus-cache stacks, and Context Lakes. The vendors in each are mostly known. The framing — that there are exactly four lanes and they cover different things — is the part teams miss.
AI agent memory tools are software systems that store and retrieve the context an agent needs to make decisions. They span four categories — vector databases for semantic retrieval, agent memory libraries for episodic continuity, OLTP-plus-cache stacks for live operational state, and Context Lakes that unify all three. Most production agents require capabilities from multiple categories, and the coordination between them is where deployments fail.
Before surveying the tools, it helps to know what they are tools for. Production agent memory has three jobs, and most teams discover them in order:
Episodic memory holds what happened. Past conversations, prior tool calls, prior decisions, prior outputs. Agents need it so they do not re-ask, re-explain, or contradict themselves across sessions. Episodic memory is append-only and time-ordered.
Semantic memory holds what is known. Domain knowledge, product docs, customer profiles, learned patterns, policies. Agents need it so reasoning is grounded in the organization’s accumulated understanding, not just the model’s pre-training. Semantic memory is governed and shared across agents.
State memory holds what is true right now. Account balances, inventory levels, in-flight orders, recent actions by other agents and services. Agents need it so the decisions they take reflect current reality, not stale snapshots. State memory is mutable and authoritative — and freshness is a correctness requirement, not a performance optimization.
You may see this same split named differently. The “short-term memory vs long-term memory” framing common in agent literature usually collapses episodic and semantic into long-term memory (everything persisted across sessions) and treats short-term memory as the active context window. State memory often gets ignored entirely in that framing, which is part of why teams keep rediscovering that operational freshness was never solved.
If you want the architectural treatment of why these three layers exist and why production agents need all of them, the companion piece is AI Agent Memory Architecture: The Three Layers Production Systems Need. The rest of this post surveys the tools.
Representative tools: Pinecone, Weaviate, Chroma, Qdrant, Milvus, pgvector
Vector databases store text or other content as numerical embeddings and retrieve by similarity. Ask a question, embed the question, find the closest matches in the index, return the underlying content. The pattern came out of retrieval-augmented generation (RAG) and remains the default first step when a team starts wiring memory into an agent.
What vector databases address: semantic retrieval. If the agent needs to recall a policy document, surface the relevant past conversation, or pull domain knowledge that matches the current query, similarity search is the right shape of operation. Vector indexes handle the dimensionality and scale that brute-force comparison cannot.
Where the gap shows up: - Vector search is not memory architecture. It is a retrieval pattern. An agent that only retrieves cannot reason about temporal ordering (“what did we discuss in the last interaction?”), cannot read live state (“what is the current balance?”), and cannot guarantee consistency when multiple agents read and write at the same time. - No transactional semantics. Vector databases optimize for fast nearest-neighbor lookup, not for ACID guarantees on writes. When two agents update overlapping context simultaneously, the database does not arbitrate; whichever write lands last wins, and the other agent may never see the change before it acts. - Freshness drift. Embeddings are computed when content is ingested. If the underlying source changes — a policy update, a corrected customer record, a refreshed metric definition — the embedding does not change until the content is re-embedded and the index is rebuilt. The retrieved match can be lexically close to a query while being semantically stale.
Vector databases are necessary when agents need to ground answers in a corpus. They are not sufficient as the agent’s memory system on their own. See What Retrieval Really Means for AI Agents for the longer breakdown.
Representative tools: Mem0, Zep, Letta (formerly MemGPT), LangMem, Cognee
This category is the newest of the four. The products are SDKs and managed services purpose-built to give LLM agents conversation continuity. They sit between the agent loop and the underlying store, summarizing prior turns, extracting facts, tagging interactions, and surfacing the relevant slice of history when the next prompt is constructed.
What agent memory libraries address: episodic memory and lightweight semantic memory. They handle the operational work that comes with running an agent across many sessions — summarization, decay, retrieval of prior interactions, fact extraction. Some also expose a knowledge graph layer for relational reasoning over extracted entities.
Where the gap shows up: - Built for a single agent, mostly for a single user. The conversational-continuity framing assumes one agent talking to one user across sessions. Multi-agent systems with shared state — fraud agents and approval agents acting on the same customer, for example — fall outside the typical interface. Coordinating memory across agents is not what these libraries were designed for. - State is somebody else’s problem. Agent memory libraries persist conversation context. They do not hold the agent’s operational state of record — balances, exposures, inventory, in-flight requests. That data lives in the application’s OLTP database, and these libraries do not arbitrate between what the agent “remembers” and what the operational systems currently show. - Backed by infrastructure they did not build. Most of these libraries are an abstraction layer over a vector database plus a relational store plus, sometimes, a graph engine. The coherence problems of the underlying systems propagate up. If the vector store and the relational store drift, the library does not detect it.
Agent memory libraries are the right pick when conversation continuity is the dominant requirement and the agent does not need to coordinate with other agents or read live operational state. They are not a substitute for an architecture that addresses all three memory layers.
Representative tools: Postgres + Redis + Kafka (and variants — MySQL, Cassandra, Memcached, Pub/Sub)
This is what teams have built for the last fifteen years and what most production agents end up bolted to. A transactional database holds the source of truth. A cache layer holds the hot, derived slice that needs to be read fast. A message bus or change-data-capture pipeline propagates updates from the database to the cache, to downstream services, to feature stores, to whatever else needs to know.
What this stack addresses: state memory, mostly. Operational data lives in the database with ACID guarantees. The cache layer makes hot reads fast. The message bus keeps downstream views eventually consistent.
Where the gap shows up: - Eventually consistent is not always consistent enough. The cache always lags the database by some interval — sometimes milliseconds, sometimes seconds, sometimes longer when the pipeline is under load or backfilling. Agents that decide inside that lag window decide against stale state. See Context Under Concurrency for the structural argument. - Each downstream view is its own pipeline. The fraud service has its own Kafka consumer and its own derived state. The auth service has another. The recommendation service has a third. Each one is at a different propagation stage at any given moment, so different services see different versions of reality. Agents that read across services compose snapshots from caches that do not agree with each other.
The OLTP-plus-cache pattern is mature, well-understood, and entirely correct for systems built around human users who can tolerate occasional staleness. It was not designed for agents making concurrent, irreversible decisions in millisecond windows.
Representative tool: Tacnode
A Context Lake is a single substrate that holds all three memory layers — episodic, semantic, and state — under one engine, with one consistency model, queryable through one interface. Episodic events are ingested via change data capture and time-ordered. Semantic context is maintained as incrementally updated materialized views (IMVs) that converge sub-second as new events land. State is queryable under the same consistent snapshot as semantic and episodic context, so an agent reading account state and a derived velocity count and a similar past case sees all three from the same moment. Vector similarity, full-text search, aggregations, and point lookups all run against the same underlying data.
What a Context Lake addresses: all three layers, plus the coherence problem between them. Because there is one substrate rather than four systems and three pipelines, agents read context that reflects the same underlying set of ingested events. The retrieval gap (different services reading different states) and the preparation gap (derived context lagging events) close together rather than separately.
This is a different product category from the first three. It is not a faster vector database, a smarter agent memory library, or a new cache. It is the substrate underneath them all, intended to remove the coordination problem rather than add another layer on top of it. The architectural argument lives in The Next Evolution of AI Infrastructure: From Data Lake to Context Lake.
Where the gap shows up: A Context Lake is the heaviest commitment of the four categories. It replaces parts of an existing data architecture rather than sitting alongside it, which is the right call when agent coordination and state freshness are correctness requirements, and the wrong call for a single-agent prototype that only needs conversation continuity.
The four categories are not directly substitutable. Picking among them is a question of which memory layer your agents actually need and how those needs compose:
A useful test: if your agents make decisions whose effects interact — two fraud models on the same account, an inventory agent and a fulfillment agent on the same SKU, a credit-line agent and a transaction agent on the same customer — and they make those decisions inside a tight validity window where stale or inconsistent context produces a concrete business consequence, then state freshness and coherence are correctness requirements, not optimizations. The first three categories will each address one slice of that problem and force coordination across the others. A Context Lake addresses the coordination directly.
If your agents are conversational assistants over documents, with no shared state and no live decisions, then a vector database plus an agent memory library is the right starting point and a Context Lake is overbuilt.
The mistake teams make is reaching for a vector database, hitting the freshness wall, bolting on an agent memory library, hitting the coordination wall, bolting on Kafka and Redis, hitting the coherence wall — and only then asking whether the architecture they assembled by accident is the architecture they would have designed on purpose.
| Your agents need... | Start with |
|---|---|
| Grounded answers from a document corpus | Vector database |
| Conversation continuity across sessions, single agent | Agent memory library |
| Operational decisions against a single source of truth, tolerable staleness | OLTP-plus-cache |
| Multi-agent coordination over live state under concurrency | Context Lake |
AI agent memory is not a product category. It is a set of requirements split across four:
Vector databases handle semantic retrieval. They are necessary when agents must ground answers in a corpus, and insufficient as the agent’s full memory system.
Agent memory libraries handle episodic memory and conversation continuity. They are the right pick for single-agent assistants, and not built for multi-agent coordination or live operational state.
OLTP-plus-cache stacks handle state memory with mature transactional semantics. They tolerate staleness that agents at millisecond decision cycles cannot, and they do not address semantic or episodic memory.
Context Lakes unify all three memory layers under one substrate with one consistency model. They are overbuilt for single-agent prototypes and the right architecture when agent coordination and state freshness are correctness requirements.
The teams that ship agents to production stop asking which memory tool to pick and start asking which slices of the memory problem they have. The answer to that question determines the architecture. The tool follows.
Former Cockroach Labs. Tells stories about infrastructure that actually make sense.
Book a demo and discover how Tacnode can power your AI-native applications.
Book a Demo