Voidly
All findingsPublished research

Per-individual-domain 7-day shutdown forecast (top 42 domains × 50 countries)

Per-individual-domain shutdown forecast at /v1/forecast/domain/{domain}/{cc}. Shared XGBoost across all (domain, country) pairs with domain one-hot — LOCO median AUC 0.999 across 28 evaluable domains, temporal-holdout AUC 0.983, Brier 0.061 → 0.041 after isotonic. Caveat: 75% positive base rate…

Read the full published summary

Per-individual-domain shutdown forecast at /v1/forecast/domain/{domain}/{cc}. Shared XGBoost across all (domain, country) pairs with domain one-hot — LOCO median AUC 0.999 across 28 evaluable domains, temporal-holdout AUC 0.983, Brier 0.061 → 0.041 after isotonic. Caveat: 75% positive base rate means the model is mostly predicting block-state persistence (~85% of pair-days), not novel state transitions.

The Atlas already ships a per-country shutdown forecast (XGBoost + isotonic, AUC 0.954) and a per-platform variant covering 12 platforms. The most journalist-relevant question still hadn't shipped though — not "will Iran tighten?", but "will Iran block twitter.com in the next 7 days?" This finding closes that gap.

The build

  • Universe: top 42 domains by historical block volume across the top 50 most-active countries (the target was top-100 domains, but the Voidly evidence pipeline only tracks 42 distinct domains as of 2026-05-21 — every one of them ends up in the model).
  • Grid: dense (domain × country × date) panel, 42,425 row-days after the "≥5 observations in the trailing 90 days" active-pair filter. 31,795 positive (75% base rate).
  • Target: binary — did this (domain, country) pair see any block or elevated/critical anomaly observation in the next 7 days? Same convention as the per-platform forecast — broader than critical-only because per-individual-domain critical events are sparse.
  • Features (26 total): per-pair block_rate / anomaly_rate / n_obs / n_block at 7d / 30d / 90d lookback; country-wide block_rate at the same horizons; cross-country contagion (n_countries_blocked_30d, blocked_elsewhere_30d); persistence (blocked_here_90d); risk_tier, continent one-hot, day-of-week. Plus 42 domain one-hots — one shared XGBoost across all pairs with domain embedded as a categorical feature.
  • Algorithm: XGBoost (n_estimators=300, max_depth=5, lr=0.05, min_child_weight=3, reg_lambda=1.5) + isotonic calibration on a 30-day temporal holdout.

Results

  • LOCO median AUC across top-30 domains: 0.999 (top-50 directive floor 0.65 — passed)
  • LOCO mean AUC: 0.999 across 28 domains with both classes in held-out folds
  • Temporal holdout AUC (last 30 days): 0.983, Brier 0.061
  • Isotonic calibration: Brier 0.061 → 0.041 on the 21,551-row holdout
  • Best per-domain LOCO AUC: tiktok.com / tumblr.com / substack.com / pornhub.com / xvideos.com all at 1.000
  • Worst per-domain LOCO AUC: gemini.google.com at 0.986 (still well above floor)
  • Top features by gain: n_block_7d (35%), block_rate_7d (31%), country_block_rate_7d, blocked_here_90d

Honest caveats

The headline AUC is misleading without context. Once a domain is blocked in a country, it tends to stay blocked for the 7-day horizon — the persistence is ~85%. So the model is mostly answering "will the current block state persist?", which it does extremely well. The minority of state transitions (~15% of pair-days) are where the model has to actually generalize. Per-platform forecasts showed the same pattern (and the same honest caveat — see the per-platform sidecar).

Other limitations: (a) the cross-country contagion feature n_countries_blocked_30d is identical across all countries on a given (domain, date), so it behaves as a global "this domain is hot right now" prior — it can't discriminate WITHIN a single day. (b) Countries with zero historical evidence on a domain score the feature-vector floor (≈base rate) — treat thin-coverage pairs as low-information. (c) Only 42 of the targeted 100 domains have any evidence in the table — the missing 58 are upstream-coverage gaps, not model gaps.

Sample predictions (2026-05-21)

  • P(telegram.org blocked in CN, 7d) = 0.999 (n_pair_obs_90d=64) — model nails the persistent block
  • P(twitter.com blocked in CN, 7d) = 1.000 (n_pair_obs_90d=70) — same
  • P(twitter.com blocked in IR, 7d) = 0.056 (n_pair_obs_90d=7) — thin coverage in IR + no recent block measurements → low-information floor
  • P(twitter.com blocked in US, 7d) = 0.056 (n_pair_obs_90d=0) — correctly low

What ships

Four endpoints under /v1/forecast/domain/*: /{'{'}domain{'}'}/{'{'}cc{'}'} (single pair), /{'{'}domain{'}'} (one domain across all countries), /info (sidecar), /list (universe). Every response includes the top-3 features by gain with their current values, the pair-specific n_obs_90d, the domain's LOCO AUC, and the full honest_caveats array. No frontend pages — API + sidecar + this finding only.

Leakage audit (2026-05-21). A label-leakage audit flagged this model as LEAKED — not via a circular feature, but via target triviality. The target (any anomaly in the next 7 days on a domain/country pair) is near-constant per pair: ~75% of rows are positive and the MIN_PAIR_OBSERVATIONS>=5 filter keeps only persistently measured pairs. For a domain blocked every day (twitter in Iran), the trailing anomaly_rate_7d is ~1.0 and the future label is always 1, so the LOCO AUC of 0.999 measures "is this pair currently blocked," not forecast skill. On status-transition days the honest AUC is expected to be ~0.55-0.65. Recommended fix: re-define the target as a newly-blocked status transition (0-to-1), or report AUC restricted to transition days / non-persistent pairs.

The source record.

Original source links and labels. Linked responses may have changed since publication.