Voidly
All findingsPublished research

May 2026 isotonic refit audit (historical, unvalidated)

A May 2026 audit identified stale isotonic mapping and disruption-inflated outcome labels, then reported an in-sample refit on a 30-day archive. The reported Brier, ECE, and drift improvement is historical and unvalidated for current use; no present calibration or live accuracy conclusion follows…

Read the full published summary

A May 2026 audit identified stale isotonic mapping and disruption-inflated outcome labels, then reported an in-sample refit on a 30-day archive. The reported Brier, ECE, and drift improvement is historical and unvalidated for current use; no present calibration or live accuracy conclusion follows without an eligible outcome cohort.

Historical May 2026 calibration audit; reported gains unvalidated. This account describes a label and calibrator repair attempted in May. Its reported figures are not present forecast accuracy.

The two problems identified

The May audit identified a stale isotonic mapping fitted to labels that counted IODA connectivity disruptions as censorship, and an outcome join that could also count disruption-only incidents as positives. Both can make a forecast appear better or worse than it is for the censorship target.

The May refit

A script rebuilt censorship-only outcomes from the historical archive, fit a new isotonic mapping, and compared Brier and expected calibration error on the same 30-day sample. The writeup reported 720 pairs and an in-sample Brier change from 0.380 to 0.120. It also reported drift moving from +56.45pp to 0.00pp across different label views. Those values are historical, not a forward test, and the cohort has not been revalidated under current eligibility rules.

What would validate a current claim

Rebuild settled, censorship-only outcomes from a fresh eligible cohort, compare the old and new mappings on a forward holdout, and report country coverage, base rate, Brier, ECE, and missingness. An in-sample zero drift is expected from fitting and is not proof that the live forecast is calibrated. Current public intervals remain withheld pending their own coverage check.

The source record.

Original source links and labels. Linked responses may have changed since publication.