Pagination is for interactive reads. If an analyst is building a model, they should not be paginating an API — they should be reading a file.
POST /v1/exports
{ "select": "observations",
"category": "electronics.audio",
"since": "2026-06-01",
"format": "parquet" }
→ 202 { "export_id": "exp_71a", "status": "building", "estimated_rows": 4120887 }
GET /v1/exports/exp_71a
→ 200 { "status": "ready", "url": "https://…", "expires_at": "…", "rows": 4118203 }| select | One row per |
|---|---|
| products | Canonical product |
| listings | Product at a seller |
| offers | Current purchasable offer |
| observations | Immutable reading — the full history |
| events | Derived change |
NDJSON for streaming into anything. Parquet for analysis — it compresses roughly ten to one on this shape and columnar reads make time-series aggregation dramatically faster.
Pass `since` with your last export's high-water mark. Observations are append-only, so incremental export is exact rather than approximate.