voidly

Per-country forecast calibration drift monitor (auto-alert at +/-15pp)

New monitor at scripts/build-per-country-calibration-drift.py walks the top-50 most-active forecast countries daily at 05:00 UTC and computes mean predicted probability vs empirical positive rate (censorship/mixed only, IODA disruption excluded) over a trailing 30-day window. Any country crossing +/-15pp drift gets flagged and fires a calibration_drift event into the existing CenAlerts pipeline (24h dedup). Sidecar at /opt/voidly-ai/ml-deploy/calibration_drift_by_country.json. Live at GET /v1/sentinel/calibration-drift (full table) and /v1/sentinel/calibration-drift/{cc} (per-country). Why this exists: the post-refit global drift is ~0pp, but individual countries can still drift in opposite directions and cancel out in the aggregate. Honest caveats: 30-day windows are narrow, high-frequency-flip countries (VE, MM) produce noisy estimates, drift > 0 != broken model.

#monitoring#calibration#ml-honesty#transparency#cenalerts

Raw data