voidly
RUIRPKBDEGTRCN

Government statement scraper: pairing ministry press releases with Voidly shutdown incidents

Voidly Atlas previously saw shutdowns only from the network side (OONI/IODA/Voidly probes). This v1 ships a curated government-statement scraper over 7 ministries (Russia Roskomnadzor, Iran MICT, Pakistan PTA, Bangladesh BTRC, Egypt MCIT, Turkey BTK, China MIIT), an NLP entity extractor (domain mentions, ASN refs, language-aware keyword vocabularies in en/ru/fa/ar/tr/bn/zh), and a correlation pass that pairs each statement with Voidly incidents in the same country within ±72h. First run ingested 13 statements (BD 8 + TR 5) across the reachable sources and surfaced 13 (statement, incident) pairs; the top pair (confidence 0.545) is a Turkey BTK regulatory draft published ~48h after a critical IODA-confirmed connectivity disruption on 2026-04-28 — a retroactive timing flagged correctly. 2 sources (RU, IR) are unreachable from our Vultr egress; 2 (PK, CN) reached but JS-shell-only — all documented up-front rather than papered over. Cron every 6h. Endpoints: GET /v1/atlas/government-statements, /v1/atlas/government-statements/info, /v1/atlas/government-statements/correlations, /v1/atlas/government-statements/{stmt_id}.

#scraper#government#press-releases#cross-source#investigative#transparency#ml-honesty

Raw data