Open call: review our methodology

The ask

• One to two hours, any time in the next six weeks.
• Read the methodology and the definitions timeline, poke the data, and send us what is wrong, missing, or misleading.
• Comments and our responses are published in full on this page — attributed or anonymous, your choice.
• No payment — only public acknowledgement, and a resource your students may cite made better by you.

The five-minute reviewer pack

1. What it is: a verifiable republication layer over UNHCR / IDMC / UN WPP statistics — content-addressed snapshots, a quarterly DOI (10.5281/zenodo.22087749), daily public validation, metric definitions in seven languages. It produces no data; it produces the rigour needed to cite data.
2. Self-checks: 24 daily automated checks — 18 data invariants (incl. mapped + unmappable = global total for every metric × 75 years; three golden numbers within 1%). Current snapshot b51605e2, data as of 2025-12-31.
3. Known limits (we confess first): the null≠zero chain trusts the upstream "-" convention; IDPs cover conflict/violence only; OIP creates a 2018 break; the live layer is estimates; individual cell values are not yet reconciled row-by-row against the upstream interface (only golden numbers and sum identities are).
4. Criticism we most want: legal accuracy of the definitions (in all seven languages), missing caveats, whether the responsible-citation rules hold up, and any unintentionally misleading visualisation.

How to respond

Write to [email protected] — any format: bullet points, annotated screenshots, one long e-mail. We will respond point by point, and ask how you wish to be credited before anything is published.

Reviews received

Every review and our response is published here — including the parts we cannot answer. No human review has arrived yet; below is an automated one.

2026-09-05 · Automated code review (an AI, commissioned by the maintainer)

A full pass over the source and the live site. It found no reason to rewrite the architecture, and three semantic defects affecting published claims plus four reader-facing bugs. We reproduced every one of them in the code and fixed them the same day. We had not caught any of these ourselves:

• The home page's "1 in every N people is forcibly displaced" was computed from total people of concern, which also carries stateless persons and others of concern (~6%). It is now computed from refugees + asylum-seekers + IDPs + OIP, and the published figure moves from 1 in 71 to 1 in 76. (methodology)
• The data had always declared which view each metric exists in, but nothing in the app read it — ?m=stateless&v=origin put residence figures under an origin heading. (methodology)
• Citations and downloads for derived metrics credited UNHCR alone, although their IDP component comes from IDMC. Every citation and export now lists every source actually used.
• Switching language discarded the entire query string (countries, year, metric); shared comparison links came back with the rows and colours swapped; per-1,000 rates below 0.01 printed as "0", which on this site means "the source reported zero"; and on a first visit the tour card covered the legend.

Each fix ships with a test that fails against the previous build, and the data layer gained two invariants (#17, #18) so the two semantic defects cannot return. The country search boxes are now one shared ARIA 1.2 combobox: arrow keys with wrap, Home/End, Enter, a two-stage Escape, aria-activedescendant, and options that are no longer individual tab stops.
One thing still not done: a real screen-reader pass. The behaviour above is covered by automated keyboard tests, but no one has walked it with NVDA or VoiceOver — and an automated scan missing an interaction model is exactly what happened here.