Government statement scraper: pairing ministry press releases with Voidly shutdown incidents
Voidly Atlas previously saw shutdowns only from the network side (OONI/IODA/Voidly probes). This v1 ships a curated government-statement scraper over 7 ministries (Russia Roskomnadzor, Iran MICT, Pakistan PTA, Bangladesh BTRC, Egypt MCIT, Turkey BTK, China MIIT), an NLP entity extractor (domain mentions, ASN refs, language-aware keyword vocabularies in en/ru/fa/ar/tr/bn/zh), and a correlation pass that pairs each statement with Voidly incidents in the same country within ±72h. First run ingested 13 statements (BD 8 + TR 5) across the reachable sources and surfaced 13 (statement, incident) pairs; the top pair (confidence 0.545) is a Turkey BTK regulatory draft published ~48h after a critical IODA-confirmed connectivity disruption on 2026-04-28 — a retroactive timing flagged correctly. 2 sources (RU, IR) are unreachable from our Vultr egress; 2 (PK, CN) reached but JS-shell-only — all documented up-front rather than papered over. Cron every 6h. Endpoints: GET /v1/atlas/government-statements, /v1/atlas/government-statements/info, /v1/atlas/government-statements/correlations, /v1/atlas/government-statements/{stmt_id}.
Raw data
- Live: per-source coverage + last fetch time
- Live: recent Turkey BTK statements
- Live: recent Bangladesh BTRC statements
- Live: (statement, incident) correlations
- Live: warning-sign correlations only (statement before incident)
- Live: retroactive correlations only (statement after incident)
- Source: Turkey BTK Duyurular
- Source: Bangladesh BTRC Notice Board