What Is a Freshness Budget?

What is a freshness budget?

The term is part of the context-gap vocabulary Tacnode uses to make the validity window operational. The window states the requirement — context must be accurate within roughly this interval for this decision. The freshness budget is that requirement decomposed across the systems that produce the context: how much of the interval each pipeline stage, cache hop, and derivation step is allowed to consume, and what the components actually spend in practice.

The concept earns its keep in ML-serving and agent systems, where it is routinely inverted in practice: teams budget model latency in detail — milliseconds per inference hop — while the features the model reads carry seconds of unmeasured pipeline age. For real-time inference inside tight decision windows, the feature freshness budget, not the model latency budget, is the binding constraint — and most systems never measure it. The full decomposition is worked through in real-time ML inference and feature freshness.

How a freshness budget works

Budgeting a card-authorization decision with a 500-millisecond validity window:

Input                    Production path                     Measured age (p99)
account balance          direct read, committed data         10 ms
merchant risk score      nightly batch                       up to 24 h
spend velocity           CDC → stream job → cache            2.8 s
device signal            session store                       150 ms

The decision's effective freshness is the maximum: it acts on a picture up to 24 hours old wherever the risk score matters, and at least ~3 seconds old whenever velocity matters — spending the entire 500 ms budget many times over, invisibly. The budget discipline is: measure every input's event-to-readable age at the tail, compare the maximum against the validity window, and either bring the over-budget components down (move derivation closer to the data, replace batch with incremental maintenance) or redesign the decision to not depend on them.

Two properties make budgets slippery. Ages are not constant — pipeline lag stretches under exactly the burst conditions that matter, so budgets must be checked at burst percentiles, not medians. And ages are not independent — a shared broker or stream job under backpressure ages several inputs at once, so the worst case arrives correlated.

Why the freshness budget matters

The budget turns "is our data fresh enough?" from a feeling into arithmetic, and its max-of-components rule redirects effort to where it counts: accelerating an already-fast input buys nothing while a slow input dominates the maximum, which is why point optimizations so often produce no decision-quality improvement. It also exposes the structural argument — when a budget is measured in hundreds of milliseconds and any component involves scheduled recomputation, the budget is unmeetable by schedule tuning — the pipeline lag has to be removed architecturally instead.

FAQ