What Is Point-in-Time Correctness?

What is point-in-time correctness?

Training data is built after the fact. By the time a team assembles a year of labeled transactions, the feature tables hold current values, many of which changed after the events being labeled. Joining labels to current features gives the model facts it could not have had at prediction time.

The Databricks feature store docs state the requirement this way: "Point-in-time correctness creates a training dataset that reflects feature values as of the time each label observation was recorded. This is important to prevent data leakage, which occurs when you use feature values for model training that were not available at the time the label was recorded." Feast says it can "reproduce the state of features at a specific point in the past." Feature stores package it, but underneath it is an ordinary temporal join; what to look for is covered in how to evaluate a feature store.

Leakage is common and hard to see. In Leakage and the Reproducibility Crisis in ML-based Science, a 2022 survey of the research literature, Sayash Kapoor and Arvind Narayanan found leakage errors in 17 fields, "collectively affecting 329 papers." A close relative is label leakage, which Google's ML glossary defines as "A model design flaw in which a feature is a proxy for the label." A feature updated after the outcome is often both.

How point-in-time correctness works

A point-in-time join needs an append-only history of timestamped feature values and a timestamp on every label. For each label, it takes the latest feature row for the same entity at or before the label's time.

-- labels:          one row per prediction event (event_id, card_id, event_ts, is_fraud)
-- feature_history: append-only, one row per feature value
--   event_ts     = when the value became true
--   available_ts = when the serving path could first read it
SELECT l.event_id, l.card_id, l.event_ts, l.is_fraud,
       f.txn_count_1h
FROM   labels l
LEFT JOIN LATERAL (
  SELECT h.txn_count_1h
  FROM   feature_history h
  WHERE  h.card_id      = l.card_id
    AND  h.event_ts    <= l.event_ts
    AND  h.available_ts <= l.event_ts   -- exclude values that arrived late
  ORDER  BY h.event_ts DESC, h.available_ts DESC   -- latest correction wins a tie
  LIMIT  1
) f ON true;

DuckDB has a native ASOF JOIN, and Databricks matches "the most recent value for a particular primary key that is not later than" the lookup timestamp. Two details decide whether the join is actually correct:

1. 1.Event time versus availability time. Feast's docs note that by default its joins "only constrain the feature's event timestamp," so "a value that was backfilled or corrected after an entity dataframe timestamp can still be returned for it." Its filter_by_created_timestamp=True option adds a created_timestamp <= entity_timestamp condition, the same job as available_ts above.
2. 2.How far back to look. Feast will "scan backward in time from the entity dataframe timestamp up to a maximum of the TTL time specified," and a label with no row in that window gets no value. Too long a window can attach stale values; too short leaves features null.

Why point-in-time correctness matters in real-time systems

A point-in-time join on event time reproduces what was true, not what the serving path could read. If the online feature trails its sources by a stream job's lag or a cache's expiry, the production model sees an older value than the one the join attached, and the training set is still optimistic. That is how training-serving skew survives a correct join. The fix is to record availability time, when each value became readable where the model reads it, and join on that, or to train on logged serving features.

Fast-changing features make this matter most. A velocity count or a balance can change several times within one pipeline delay, so "true at event time" and "readable at event time" differ most for exactly these features. Table-level reads as of a past timestamp, covered in time travel queries, answer the same question for a whole table within a retention window: useful for debugging one decision, not for months of training history.

FAQ