Price History API

Store observations, not prices

A price is a mutable current-state field. An observation is an immutable fact: this source reported this value at this instant. Store observations and current price becomes a view over them. Store prices and you have thrown away the thing you were trying to build.

Each observation should carry two timestamps — when the source said it, and when you learned it. Those differ, sometimes materially, and anything time-sensitive needs both.

The series is a change log

An unchanged reading should still count as a poll — it informs how often to look again — but should write nothing. Done properly, three rows over ninety days does not mean you looked three times. It means the price changed twice.

{ "series": [
   { "observed_at": "2026-06-09T13:21:15Z", "price_minor": 39800 },
   { "observed_at": "2026-08-21T02:10:44Z", "price_minor": 31999 },
   { "observed_at": "2026-09-06T14:21:59Z", "price_minor": 19800 } ] }

// $398.00 held 73 days. $319.99 held 16. $198.00 is one day old.

Time-weight everything, or your statistics lie

Ties need handling too. A listing that has never moved sits at both its own low and its own high — midrank puts it at the fiftieth percentile, which is the honest answer. Without it, a flat price reads as either the best deal ever recorded or the worst, depending on which way the comparison happens to fall.

Never pool sellers

Compare a seller's price against that seller's own history. Pooling every retailer's series into one distribution makes the cheapest seller look merely average, because the pool contains the expensive sellers' prices too. The percentile that means something is the one scoped to where you would actually buy.

What history is actually for

It converts a reading into a judgement. “$198.00” is a fact any scraper returns. “$198.00, the fourth percentile of eighty-nine days at this retailer, lowest since March” is a decision — and it is what a shopper, or an agent acting for one, actually needs.

It is also the honest basis for a discount claim. Percent-off depends on a reference price the seller chooses; percentile depends on what the product has actually cost, which it does not.

The strategic consequence

Every other component of a price product can be built in a quarter — collection, matching, alerting, an API. History cannot be built at all; it can only be started. A competitor who decides to enter tomorrow has ninety days of history in ninety days, and no amount of funding compresses that.

Which means the correct time to start collecting is before you need it. If you are evaluating vendors, start a candidate collecting on your priority products in parallel now, so the data exists when the decision arrives.

Segments, not rows: reconstructing what a price actually was

The mechanics matter because the naive implementation is both obvious and wrong. You do not store one row per hour — you store a row when something changes. So a price that held sixty days is one row and a price that flickered for an afternoon is three, and any statistic counting rows has concluded the afternoon mattered twenty times more than the two months.

# observations -> [(price, start, end)] segments over the window
segments = to_segments(observations, window_start, window_end)
total = sum(s.end - s.start for s in segments)

below = sum(s.end - s.start for s in segments if s.price < current)
equal = sum(s.end - s.start for s in segments if s.price == current)

# midrank: half the time spent AT the current price counts as below.
# Without this term a flat price reports as both 0th and 100th percentile.
percentile = 100 * (below + 0.5 * equal) / total

The midrank term is the part that gets left out, and leaving it out produces the most confusing output a price tool can give. A product whose price has never moved is simultaneously at its lowest ever and its highest ever, and which one you report depends entirely on whether you wrote a strict or non-strict comparison. With midrank it correctly reports as the middle of its own distribution — and the honest output there is not a percentile at all but a verdict saying the price has not moved.

Windows change the answer, so state the window

A percentile is meaningless without the window it was computed over, and the same price can sit at the 5th percentile over 30 days and the 60th over a year. Neither is wrong. A system reporting a percentile without its window, or silently picking one, is hiding the parameter that most determines the answer.

The last row requires honesty. A percentile computed over three weeks of observation is not a percentile against the product's behaviour, and presenting it as one is the most common overstatement in this category. Return the observation count and the window start alongside every verdict so a consumer can judge whether to trust it.

Questions

Common questions

Related