Two gaps in the LongMemEval harness, both of which had to close before any
recency feature could be judged.
The store was fed a synthetic counter (ts += 1.0 per turn) and the dataset's
own `haystack_dates` were ignored. Session order happened to be chronological,
so ordering was right, but the intervals were fiction — and exponential decay
is a function of the interval, so anything time-aware was being measured
against made-up ages. Dates are now parsed (civil-from-days, pinned against
reference values) and turns are spread over the minutes after their session
start; an unparseable date falls back to position so order still holds.
`newest_gold_first` measures what recall cannot. On a `knowledge-update`
question LongMemEval labels *both* the stale session and the one that
supersedes it as gold, so returning either scores as a hit even though only
one answers the question. The new metric asks whether the newest gold session
outranked the older ones. The current retriever scores 43-45% on it across
every mode — chance — which is the gap a temporal signal is supposed to close.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
