A context gap is the space between what just happened and what your decision system can see. The rule is right; the counter it reads is 800 milliseconds old, and the three systems it queries describe three different moments. Here’s why faster caches and more replicas can’t close the gap — and what actually does.
TL;DR: A context gap is when a decision system can’t get complete, consistent, and current context within the window it has to act. The decision logic is fine — the state it reads is not. The gap comes in two forms: pipeline lag, where derived context (counters, aggregates, features) trails the events it summarizes, and inconsistent reads, where context is split across systems that can’t be read as one moment in time. Under high state velocity and concurrent decisions, the gap is structural: faster caches make values older, read replicas scale the wrong side, and every added pipeline adds another seam. Closing it takes three properties in one serving layer — derived context that converges incrementally in sub-seconds, one internally coherent snapshot for every concurrent reader, and derivation that runs where the state lives.
A context gap is when a decision system cannot access complete, consistent, and current context within the window it has to act. The decision logic is correct; the state it reads — an aggregate, a velocity count, a session snapshot — either lags the events it summarizes or is scattered across systems that cannot be queried as one moment in time. The result is a correct rule evaluating a world that no longer exists.
Every automated decision is two things: logic and context. The logic is the rule, the model, the policy — “decline above five merchants in sixty seconds,” “approve if exposure is under the limit,” “route this agent action if the account is in good standing.” Teams invest enormously in the logic. But the logic only ever evaluates the context it’s handed, and in most production stacks that context is assembled from systems that each learned about reality at a different time.
That’s the gap: not between your rule and the right rule, but between what happened and what your decision could see. When real-time decisions fail, the post-mortem almost never finds a broken rule. It finds a stale counter.
Two concepts make the gap precise.
The validity window. Every automated decision has a window — typically ten milliseconds to one second — within which it must run and inside which its context must be true. A card authorization can’t wait for tonight’s batch. An agent’s tool call commits in the time it takes to execute. The window is a property of the business moment, not of your infrastructure: it’s how long the world holds still enough for the answer to stay right. Context that’s accurate but arrives after the window is indistinguishable from context that’s wrong.
Derived context. Decisions rarely read raw events. They read derived context — a velocity count, a rolling exposure sum, a risk feature, a session summary — state that must be computed from events before it can be read. That computation takes time, and it runs somewhere. The moment a decision depends on derived state, it inherits the propagation delay of whatever machinery maintains that state. This is where the gap lives. A decision that needs only a single stored record can read it directly and atomically — no gap. A decision that needs “how many times in the last sixty seconds” is at the mercy of the pipeline that maintains the count.
Pipeline lag. Derived context must be prepared before decision time — but the preparation trails the events it summarizes. The stream job is windowing and checkpointing; the batch refresh runs every fifteen minutes; the cache write lands after the fact. Eight events have happened; the counter says four. In steady state the lag is milliseconds and invisible. Under a burst — precisely when the decision matters most — the lag grows, because the same spike that makes the count decision-critical is the spike the pipeline is choking on. The prepared answer is ready, fast to read, and already wrong.
Inconsistent reads. The context a single decision needs is split across systems that can’t be queried under one consistent snapshot. Balance from the system of record, velocity from a cache, embeddings from a vector store: three reads, three propagation stages, three moments in time. The composite context your decision evaluates — balance as of 200 milliseconds ago, velocity as of 3 seconds ago, profile as of 45 seconds ago — never existed as a coherent whole. Polyglot persistence built this on purpose: each store is excellent at its job and none of them agree on what time it is.
Most production stacks have both. Pipeline lag makes each individual value stale; inconsistent reads make the assembled picture incoherent even when each value is individually fresh.
If the gap were a tuning problem, tuning would have fixed it by now. It persists because two forces compound it.
State velocity. The state a decision reads is being changed by the decisions themselves. Every authorization updates the velocity it’s checked against; every agent action changes the account state the next action reads. The faster events arrive, the faster any prepared answer decays — and the systems most in need of fresh context are exactly the ones changing state fastest.
Concurrency. Decisions don’t arrive one at a time. A card-testing burst, a flash sale, a fleet of agents acting on the same account — dozens of decisions read the same derived state in the same narrow interval, before any of them has propagated into it. Each reads the same stale value. Each passes individually. Collectively they blow through the limit the rule was written to enforce. It isn’t one stale read; it’s a cluster of concurrent decisions all reasoning from the same outdated snapshot — the pattern we walk through in Context Under Concurrency.
There’s a third, quieter failure that shows up at fleet scale: divergence. Each service maintains its own pipeline into its own state store, and each pipeline sits at a different propagation stage. The fraud model’s cache is 3 seconds behind; the auth service’s is 1 second behind. They don’t just see stale reality — they see different realities, and make conflicting decisions about the same customer in the same moment.
Not every stale value is a context gap. Four questions separate a structural gap from ordinary latency:
All four together and the gap is structural — no cache TTL setting will make it go away. And the honest disqualifiers matter just as much: batch personalization, dashboard freshness, feed ranking with no tight window — those are real problems with well-served solutions, and they are not this problem.
A faster cache. Caching cuts read latency — how fast you get the value. The gap is propagation delay — how old the value is when you get it. A cache adds its own staleness window on top of the pipeline’s: within the TTL, every reader gets the cached value no matter how far reality has moved. You’ve made the read faster and the answer older, which is precisely the wrong trade during a burst. The distinction is the subject of Data Freshness vs Latency — they’re different axes, and cache tuning only moves one of them.
More read replicas. Replicas scale read throughput. But the bottleneck is the write side — the rate at which new events are incorporated into derived state. More replicas means more places to read a stale number from, faster. The write path that’s actually behind is unchanged.
Another pipeline. The instinct when context is missing is to add a system that has it: a streaming layer, a feature store, a fresher cache. Each addition is another propagation boundary advancing on its own clock — another seam where consistency silently breaks, and more divergence between the readers that depend on it. This is the paradox of the composed stack: every system added to close a gap widens the read incoherence, because coherence is a cross-system property and no amount of per-system excellence produces it.
The fix is architectural: stop assembling decision context from chains of systems that propagate independently, and serve it from one layer with three properties.
Derived context converges incrementally, in sub-seconds. Counters, aggregates, and features are maintained as incrementally-updated views — each arriving event updates the derived state it touches, converging within a bounded sub-second SLA rather than on a refresh timer. Deliberately asynchronous, so maintenance never contends with the write path — but continuous, so lag doesn’t balloon under burst the way a batching pipeline’s does. This closes the freshness dimension: the prepared answer tracks the events it summarizes. That’s the Live property.
Every concurrent reader sees one internally coherent snapshot. All decisions — the fraud model, the auth service, every agent in the fleet — read the same layer under one consistent, point-in-time view. Concurrent decisions in a burst stop each reading a different stale value; the balance, the velocity count, and the risk feature a decision reads all reflect the same set of ingested events. This unifies reads and eliminates cross-service divergence. That’s the Shared property.
Derivation runs where the state lives. Some context is best precomputed as an incrementally-maintained view; some is best computed on demand at query time — an aggregation over recent events, a vector similarity search, an LLM-derived signal. When both run inside the same layer against the same ingested state, there’s no external derivation pipeline to fall behind, and pre-computed and on-demand context agree with each other. This closes the pipeline lag at its root. That’s the Semantic property.
This is what we built Tacnode’s Context Lake to be: real-time, multi-modal data infrastructure that sits downstream of your systems of record — ingesting via change data capture, streams, and connectors — and serves every decision one fresh, coherent view of state. It doesn’t sit in your core systems’ write path or block their transactions; it makes sure the decision that runs sees current state, so it doesn’t approve what it would have blocked if it had seen what just happened. And where the layer holds its own authoritative state — agent actions, decision logs, derived state it owns — writes are fully transactional, so concurrent updates to that state serialize correctly.
The result, back on the timeline from the top of this post: the lane the decision reads catches up to the lane where reality happens, and stays inside the validity window. The rule you already wrote finally sees the number it was always supposed to see. For the full framing, see the context gap page and decision coherence.
Former Meta and Microsoft. Built distributed query engines at petabyte scale. Author of the Composition Impossibility Theorem (arXiv:2601.17019).
Fresh Context Newsletter
The argument behind the posts — what breaks when AI acts on stale data, from the team building the Context Lake. About twice a month.
Book a demo and discover how Tacnode can power your AI-native applications.
Book a Demo