Forecast labels cleaned: IODA outages no longer count as confirmed censorship

The bug

Voidly's ingest pipeline promotes critical IODA ASN-level outage alerts into the incidents table with incident_type='disruption' and severity='critical'. The forecast feature builder loaded ALL incidents and labeled country-day pairs positive whenever any such incident fell in the next 7 days. The unintended consequence: IODA outages — which include fiber cuts, BGP changes, DDoS attacks, weather, and routine maintenance — were being treated as confirmed censorship.

April 2026 had 1,074 incidents across 167 countries. 1,011 were IODA disruption. Only 45 were CensoredPlanet pure-censorship and 18 were mixed. So 94% of April incidents were noise from a label perspective.

The downstream damage

• Monthly positive rate in forecast training data exploded from ~2% (Oct 2024 – Dec 2025) to 30–79% (Jan–Apr 2026).
• The retrain gate kept rejecting new models — new models correctly fit the new pathological distribution, then overpredicted positives on the frozen holdout (which had a stale 7% positive rate). Live model was 33 days stale.
• Live forecasts on real censorship-heavy countries had calibration off.

The fix

scripts/build-forecast-features.py patched to exclude incident_type = 'disruption' from the load_incidents query. Censorship + mixed incidents kept; IODA disruption excluded from the forecast target only. The disruption incidents themselves are NOT deleted from the table — they remain visible on country pages with their honest type label. Only the forecast training treats them differently.

Result: monthly positive rate before vs after

Model trained on sane labels — promoted to production

Retrained the forecast model on the sane labels. Dual-gate result:

• Legacy holdout (re-stratified from sane training data, 1,536 rows, 5.2% positive): old F1 0.469 → new F1 0.649 (+18pp)
• Temporal holdout (last 60d, 1,260 rows, 15.7% positive): old F1 0.348 → new F1 0.932 (+58pp)
• Dual-gate decision: ACCEPT. New model promoted live.

Feature importance shifted: GDELT unrest dropped from 25% to 11% (over-rewarded by disruption labels), weight redistributed to recent-shutdown, rolling block-rates, and seasonal markers — a sane attribution for a censorship forecaster.

Regression test added

New scripts/test-forecast-labels-sane.py runs after feature build in the weekly retrain. Fails the pipeline if any of the last 12 months has positive rate above 40%. The 79% explosion would have been caught instantly. Wired into weekly-retrain.sh as stage test-forecast-labels-sane.

Honest scope

This fixes the FORECAST training labels. The incidents table itself still contains the IODA disruption entries; we keep them because they are real network observations and journalists care about them. We do NOT count them as “censorship incidents” in the forecast or in canonical incident-count headlines going forward. The Atlas Score v1 and v2 weights still consider disruption signal, but with much lower weight than confirmed censorship.