Back to News · Walkthrough

·Walkthrough

The clock is an input: how an unpinned timestamp lied to us four times

A scoring engine that weighs evidence by age has a second input besides the evidence — the clock. We hit the same bug four times before we named it, and every time it returned a plausible number instead of an error.

If a verdict depends on how old the evidence is, then the instant you evaluate "now" is an input to the score, exactly like the evidence is. The difference is that everyone pins the evidence and almost nobody pins the clock. An input nobody pins will eventually lie — and because it produces a number rather than a crash, it lies quietly.

We caught this four times across one body of work, and it took the fourth to name the class. A calibration gate scored fixed-date test evidence against the wall clock, silently applying weeks of time-decay to frozen inputs; the accuracy figure drifted with the calendar and happened to peak — 0.95 against a true 0.48 — on the exact day we looked at it. A recall measurement read near-zero when the real rate was an order of magnitude higher. An enrichment loop sampled its clock before it ran. And a frozen evidence snapshot re-scored from 314 to 49, because the clock was sampled a few seconds before enrichment stamped the evidence, so the strongest rows were dated after the scoring instant and dropped.

The tell is that none of these threw an error. The filter that dropped the strong rows wasn't broken — discarding evidence asserted after the scoring instant is exactly what "score as of time T" means, and the historical-replay machinery depends on it. The bug was upstream every time: sampling T before the work that produced the evidence. A crash gets fixed the same day. A plausible-but-wrong number gets shipped, cited, and built upon.

The durable fix isn't to catch the drop; it's to make it impossible. Derive the clock from the data instead of from a separate wall-clock read: score at the newest evidence timestamp plus a hair, and no row can ever fall after the clock. Pin the clock in tests so this week's number is comparable to last week's. The lesson travels well beyond threat intelligence — if your system has a time parameter, the clock is an input: pin it, validate it, and never sample it before the work it measures.

Related field note: what a vanished API field costs