The problem: confidence numbers are everywhere, trust numbers are nowhere

Voidly Atlas surfaces model confidence in many places. The 7-day forecast ships a 90% conformal interval and an online-updated ACI alpha. The classifier ships a per-country threshold and a v3.3 stratified F1. The drift monitor flags any country whose 30-day calibration has slipped past ±15pp. Each of those is a useful number — but none of them answers the question a journalist actually has when they pick up the phone at deadline: is the IR forecast trustworthy today?

That question has no good single-number answer in the current API. The conformal width tells you about the model's spread, not whether the input data is fresh. The drift number is a 30-day average. The classifier and the forecast can disagree wildly today and you'd have to fetch both endpoints and eyeball them.

The per-day uncertainty surfacer collapses those signals into a single composite score per (country, model) per day, and ships it as a sidecar that every Atlas page and every API client can consume.

The composite

Uncertainty is computed as a weighted sum of five components, each normalized to [0, 1]:

Component	Weight	What it measures
`conformal_interval_width`	0.30	Half-width of the 90% prediction interval served by the live model. For the 7-day forecast we use the ACI `q_applied` value (online-updated; today α=0.21). For multi-horizon we use the static per-horizon halfwidth (1d / 7d / 30d). For the classifier (no native conformal) we use a proxy: distance from the per-country threshold, capped at 0.5.
`data_age_factor`	0.20	`1 - exp(-age_days / 7)` where `age_days` is days since the most recent `evidence.observed_at` row for this country. Older = more uncertain. 1 week of staleness pushes the component to ~0.63.
`calibration_drift_factor`	0.20	`abs(calibration_drift_pp) / 100` from the daily per-country drift monitor (live here). Reflects the country's actual track record on the trailing 30 days.
`sample_size_factor`	0.15	`1 / (1 + sqrt(n_observations))` where `n_observations` = count of forecasts logged for this country in the trailing 90 days. Few predictions = less battle-tested.
`cross_model_disagreement`	0.15	`1 - agreement`, where agreement = `1 - stdev(today's forecast-7day, multi-horizon-7d, classifier v3.3 today) / 0.5`. If three models disagree wildly, uncertainty rises.

Today's results (2026-05-21, 30 watched countries)

Most-uncertain: Pakistan

PK lands at mean uncertainty 0.4443, the highest of the watched set. The drivers are stacked: the calibration drift monitor has PK at +85.55pp under-prediction over the trailing 30 days (so the model is missing real positive cases), and cross-model agreement is only 0.226 — the live 7-day forecast says risk 0.95 for today, the multi-horizon 7d head says 0.59, and the v3.3 classifier says 0.010. When the three production models disagree by an order of magnitude, you should pause before quoting any one of them.

Least-uncertain: Turkey

TR lands at the bottom of the leaderboard. Data is fresh, the drift monitor has TR in spec, sample size is healthy, and the three models are clustered tightly. This does NOT mean “TR is fine” — it means TR's prediction today is the one you should second-guess least if you have to pick.

Iran — sample breakdown

IR sits mid-pack at mean uncertainty 0.2122. Components:

conformal width 0.35 (ACI α drifted upward over the spring; this is the largest single contributor)
data age 0.1412 (1.07 days since last evidence row)
calibration drift 0.1457 (-14.57pp; just inside the ±15pp gate so status is ok)
sample size 0.1446 (35 forecasts logged in 90 days)
cross-model disagreement 0.3275 (forecast=0.016, multi-horizon-7d=0.356, classifier=0.003 — multi-horizon is the outlier)

Per-model scores within IR range from 0.128 (classifier v3.3 — tightest, smallest conformal proxy) to 0.278 (multi-horizon 7d and 30d — widest static halfwidths).

Honest caveats

This is a heuristic composite, not a formal Bayesian posterior. Components are normalized to [0, 1] and combined linearly with hand-tuned weights. A proper posterior would marginalize over model parameters and would require either MCMC or variational inference per prediction.
Weights are not learned. We chose 0.30 / 0.20 / 0.20 / 0.15 / 0.15 by prior intuition: model-internal uncertainty (conformal width) gets the heaviest weight, then evidence freshness + calibration drift, then sample-size + cross-model disagreement. We did not optimize them against a held-out “was this prediction wrong?” corpus because that corpus is small and noisy.
“Uncertain” does not mean “wrong”. A high-uncertainty country may still be correct; it just lacks the signal-density, track-record, or model-agreement to back the call strongly. The score is a TRUST gauge, not a verdict.
Cross-model disagreement targets slightly different events. The 7-day forecast predicts a censorship/mixed incident within the next 7 days, the multi-horizon 7d head predicts the same target on yesterday's features, and the v3.3 classifier scores TODAY's country-day against a learned cut. They are roughly comparable in [0, 1] semantics but not identical events. Stdev of three numbers is a coarse signal — we deliberately keep it simple.
The classifier conformal is a proxy, not a true interval. The v3.3 classifier doesn't ship a native conformal half-width; we use distance-to-threshold, capped at 0.5. Adding a proper Mondrian conformal layer is a future-quarter task.
Data freshness uses evidence rows, not incidents. A country with no recent OONI/IODA evidence row gets a freshness penalty even if older measurements are still informative. This is intentional — stale ingestion is itself a trust issue.

How to use this

Two consumption patterns are wired into the API:

Per-country lookup. GET /v1/atlas/uncertainty/IR returns the full per-model breakdown, the mean score, and the five component values. Use this when a journalist asks “is today's IR forecast trustworthy?”.
Most-uncertain leaderboard. GET /v1/atlas/uncertainty/most-uncertain?limit=10 returns today's top-K countries sorted by mean uncertainty desc. Use this when you want a daily “which forecasts should we double-check before publishing?” checklist. Add ?model=classifier-v3.3 to re-rank by a specific model.

Pipeline + cron

Build script: scripts/build-per-day-uncertainty.py — pulls today's live model outputs from http://127.0.0.1:5002/v1/forecast/… and http://127.0.0.1:5000/v1/classifier/score/…, computes the composite, writes the sidecar.
Sidecar: /opt/voidly-ai/ml-deploy/per_day_uncertainty_v1.json.
Endpoint patch: scripts/patch-per-day-uncertainty-endpoint.py inserts three Flask routes into forecast_api.py before the __main__ guard (idempotent on the BEGIN marker).
Cron: 15 5 * * * on Vultr, running 15 minutes after the per-country calibration drift monitor so the drift component is fresh.