Data-driven country risk tiers: re-deriving a hand-set model input from objective signals

Every country in Voidly Atlas carries a risk tier — an integer 1-5 in country_geography.risk_tier, tier 1 = highest censorship risk, tier 5 = lowest. That tier is not a measurement: an analyst hand-set it. And it is not inert — the v3.3 censorship classifier consumes it as an input feature, the 7-day shutdown forecast consumes it, and several downstream surfaces lean on it. A stale or wrong hand-set tier is silently baked into every model that reads it. This finding ships a data-driven alternative: a tier derived from six objective signals over the trailing 365 days — confirmed censorship incident count (disruption rows excluded, same rule the forecast labeler uses), mean block rate, distinct domains blocked, distinct blocking methods, mean 7-day forecast risk, and DBSCAN anomaly frequency. Each feature is rank-transformed then z-scored; the composite is their equal-weight mean; countries are clustered with KMeans(k=5) on the composite (highest-mean cluster relabeled tier 1), with an equal-frequency quantile binning reported alongside as a cross-check. The first cut produced nonsense — the US scored 60% blocked, Sweden 100% — and three data-quality fixes each corrected a real bias: IODA alerts excluded (they are ASN connectivity outages, not censorship), only signal_level=critical counts as blocked (elevated/warning are soft early-warning levels that community probes self-report — one mis-networked UK probe had flagged DuckDuckGo and WhatsApp as blocked), and the block rate is empirical-Bayes-shrunk toward the global pooled rate (0.328) so a 29-row country like Sweden is pulled to the prior while China, with thousands of rows, barely moves. Result: of 148 countries with a hand-set tier, 107 land in a different data-derived tier and only 41 agree — the data says higher risk for 41 and lower risk for 66, with 50 large (>=2 tier) disagreements. The hand-set column parks 87 countries in a tier-3 catch-all; the data does not. The five biggest disagreements are exactly the cases that prove why this stays a PROPOSAL: Eritrea, North Korea and Turkmenistan are hand-set tier 1 (the most repressive information environments on earth) but the data drops them to tier 4-5 — not because the data is right but because it is blind, there are almost no OONI probes inside those countries so total censorship and zero measurement look identical to a feature pipeline. The UK and Canada move the other way (hand-set tier 5, data tier 2-3) because they are heavily probed and accumulate DBSCAN anomaly frequency plus minor genuine blocking. PROPOSAL ONLY — this never writes the country_geography table; the hand-set tier stays authoritative until a human reviews the diff. The artifact makes a hand-set number auditable and hands a reviewer a ranked list of exactly which country tiers to re-examine first. Live at GET /v1/atlas/risk-tiers + /v1/atlas/risk-tiers/{cc}. Built by scripts/build-data-driven-risk-tiers.py.