How we verify every number - checking 2.7 million data points against the publisher

A data product is only as good as its worst number. In October 2026 we stopped asking "does this value look odd?" and started asking "what does the publisher print?", for every point we hold. This post explains how, what we found, and what we changed.

#Flags are not answers

Detectors are cheap. Ours look for spikes, sign flips, values out of range, long flat runs and two series that should agree but don't. On our catalog they flagged about 24,000 points. If we had "fixed" them, we would have damaged the data. Almost all of them are real:

• remittances from a small country that jump tenfold one month and fall back the next;
• twice-yearly dividends, Paris Club repayments, a thin Islamic yield curve;
• an SBP series that really does print 3,643 between 0.017 and 0.021.

The reverse also holds. Two of the worst errors we found were invisible to the detectors: a yield curve read from the wrong column, about 0.3 percentage points off for months, and one fund's NAV history stored under another fund's id. A spike detector can't see either.

So we settle every question at the source.

#How a check works

For each publisher we re-read the original, through the same parser that loaded it, and compare every point, not just the flagged ones.

Each difference gets one of four verdicts:

1. Confirmed as published. The publisher prints exactly this. We keep it.
2. Our error. A parsing, scaling, date or mapping bug. We fix the parser, add a regression test with the real file, correct the data, and delete the fake "revision" the correction would otherwise leave in the revision history.
3. Publisher revision. The source now shows a different figure. We reload it, and the old value is kept as a dated revision.
4. Can't be checked. The source no longer hosts it. We report it rather than guess.

#What we fixed

• The PKRV yield curve, April 2022 to September 2024. Older MUFAP files are a panel of broker quotes. On 48 days we stored one broker's quote instead of MUFAP's published average. We skipped a third layout entirely on 63 days, and on 2 February 2024 showed 0.00% for every tenor from 2 to 20 years. All 1,067 files were re-read.
• Fund NAVs. About 21,000 points of another fund's history had been stored under newer funds' ids, and about 55,000 missing NAVs were filled. This was done only where MUFAP's history agrees with its own daily NAV table. For a few fund ids, MUFAP's history endpoint actually serves a different fund's series. Those funds keep the daily-table NAVs, which are right.
• Stale SBP series. Daily SCRA flows had stopped in July and are current again. The CPI 12-month moving average had silently failed on an invalid response.
• Two wrong unit labels and six bulletin-era auction points that were weighted-average yields, not cut-offs.

#When the official number looks wrong

Sometimes the publisher's figure is almost certainly a slip, for example a value exactly 1,000 times its neighbours. We still don't overwrite it: the promise of an official-data API is that you get the official figure. Instead the point carries a note, on the series page and in the API as meta.notes:

As published by SBP (EasyData …) … about 210,000 times above the neighbouring values.

We say "suggests a unit slip" only when an exact power of 1,000 puts the value between its neighbours, and we skip series that are naturally lumpy. Funds get the same treatment: three Alfalah Pension Fund II sub-funds whose published NAV fell towards zero carry a note on their fund page and in /v1/funds/{id}.

#Keeping it fixed

The checks don't stop. SBP series are re-pulled in full on a rolling cycle. Cheap consistency checks run every night and alert us: CPI growth recomputed from the index, reserves adding up, bid below offer, SPI cities against the national figure, and no NAVs dated before a fund existed. Every change lands in the changelog (RSS).

How data flows and how to trace any value to its source: sources & provenance.