Voidly
All findingsPublished research

Regional contagion does not improve the 7-day forecast — within-country dynamics already capture it

Censorship spreads regionally — a neighbour's shutdown often precedes your own — so regional "contagion" features are an obvious candidate to sharpen a 7-day forecast. We tested it honestly and it did not pan out. Starting from the production honest_forecast (HistGradientBoosting, 24 within-country…

Read the full published summary

Censorship spreads regionally — a neighbour's shutdown often precedes your own — so regional "contagion" features are an obvious candidate to sharpen a 7-day forecast. We tested it honestly and it did not pan out. Starting from the production honest_forecast (HistGradientBoosting, 24 within-country features, strict temporal rolling-origin CV across 9 folds on independent OONI ground truth, AUC 0.8154 / PR-AUC 0.2934 — reproduced here exactly), we added three leakage-safe regional features: the same-region (continental, EXCLUDING the country itself) mean anomaly rate and event fraction, both as yesterday's strictly-lagged value AND as the contemporaneous value legitimately available at prediction time, plus a 7-day regional rolling mean. Across the same folds, neither variant beat the base: lagged contagion AUC 0.8140 / PR-AUC 0.2893 (−0.0015 / −0.0041), contemporaneous AUC 0.8153 / PR-AUC 0.2900 (−0.0001 / −0.0033) — flat-to-slightly-worse on every metric, with no consistent per-fold lift. The honest read: for a 7-day, country-relative anomaly-spike forecast, a country's OWN recent dynamics (anomaly lags, rolling means, z-score vs its trailing 28-day baseline, recent event history) already carry the forward signal; regional spillover at continental granularity adds noise, not skill. This does not refute contagion as a phenomenon — finer political/regime-similarity neighbourhoods, or a different target like the country-day classifier (where regime-weighted contagion did help v3.3), may differ — but at this granularity it is a clean negative, and we did not ship it. No model change: honest_forecast stays at its validated 0.815. Reproduce with the rolling-origin harness scripts/honest-forecast-backtest.py.

Censorship is contagious. When one government shows that a shutdown “works” during an election or a protest, its neighbours often follow within days, and a regional crisis can light up several countries at once. So an obvious way to sharpen Voidly’s honest 7-day forecaster is to let it see its neighbourhood: if the countries around you are spiking, maybe you are next. This finding tests that idea the only way worth testing it — leakage-free, against the same temporal cross-validation the production model is held to — and reports the result whether or not it flatters us. It does not. We did not ship it.

The baseline we had to beat

The production model (honest_forecast) is a HistGradientBoosting classifier over 24 within-country features — anomaly-rate lags and rolling means, the escalation gap and z-score versus the country’s own trailing 28-day baseline, recent event counts, data volume, calendar terms. Its label is a country-relative anomaly spike in the next 7 days, built on independent OONI ground truth, and it is scored by strict rolling-origin temporal cross-validation: each of 9 folds trains only on data strictly before its 120-day test window. Headline: AUC 0.8154, PR-AUC 0.2934, a large honest lift over the persistence baseline (AUC 0.594). We reproduced those numbers to four decimals before changing anything, so the comparison is apples to apples.

The contagion features

We grouped the 60 forecast countries into continental regions (MENA, Central Asia, South Asia, East/SE Asia, Sub-Saharan Africa, Western Europe, Eurasia, the Americas, Oceania) and added three regional signals, each computed excluding the country itself so the model can never just read its own value back:

  • region_ar_lag1 — yesterday’s mean anomaly rate across the rest of the region (strictly past).
  • region_event_rate_lag1 — the fraction of the rest of the region in an anomaly event yesterday.
  • region_ar_roll7 — the 7-day rolling regional mean.

We then ran a second, stronger variant using the contemporaneous regional value (the region’s state at time T, excluding self) — legitimately available at prediction time, since you observe other countries’ data up to T without touching the target’s future. Both variants are leakage-safe: the label lives in [T+1, T+7]; every feature is ≤ T and never uses the target country’s own forward window.

The result — a clean negative

variantmean-fold AUCmean-fold PR-AUCΔAUCΔPR-AUC
base (24 features)0.81540.2934——
+ lagged contagion (27)0.81400.2893−0.0015−0.0041
+ contemporaneous contagion (27)0.81530.2900−0.0001−0.0033

Neither variant beats the base on any metric, and the per-fold AUCs show no consistent lift — some folds tick up a thousandth, most tick down. The contemporaneous version is essentially neutral on ranking (−0.0001 AUC) but still loses a little precision-recall area. Adding regional context made the forecaster slightly worse.

What this means — carefully

The honest read is that for a 7-day, country-relative anomaly-spike forecast, a country’s own recent dynamics already carry essentially all the forward signal there is. The model knows when a country is escalating versus its own baseline; the regional average is either redundant with that (correlated neighbours move together, and the country’s own lags already encode the local onset) or too coarse to add anything (a continental average pools free and censored countries and washes out).

This is not a claim that contagion isn’t real. It is a claim about this feature, this target, this granularity. Two doors stay open: finer political or regime-similarity neighbourhoods (rather than crude continents), and a different target — the country-day classifier, where regime-weighted contagion did earn its place in v3.3. But we will not ship a feature that loses on the test that matters just because the story is appealing. The forecaster stays at its validated AUC 0.815, and this negative is on the record so the next person doesn’t have to rediscover it. Reproduce with the rolling-origin harness scripts/honest-forecast-backtest.py.

The source record.

Original source links and labels. Linked responses may have changed since publication.