voidly
Research
Sentinel · active learning

Label uncertain forecasts

Help the model learn

The Voidly Sentinel classifier's lowest-confidence forecasts. The model is most uncertain about these 27 predictions — human labels here have the highest information value for the next retrain. Submit observed outcomes via the auth-gated /v1/sentinel/report_miss endpoint.

Method: Impact-aware ranking — Settles 2009 (UW TR-1648). Uncertainty fallback — Lewis 1994. Calibrated uncertainty — arXiv 2510.03162 (2025). · Updated every 15 min · Raw JSON

Awaiting label
27
not yet resolved
Confirmed wrong
8
false positive / negative
Total in queue
50
last 21 days
Median uncertainty
2.0%
distance from 0.5
How to submit a label

How to submit a label

  1. Get an API key — admin via Voidly, or subscriber key via /alerts.
  2. POST to /v1/sentinel/report_miss with:
curl -X POST https://api.voidly.ai/v1/sentinel/report_miss \
  -H "X-Voidly-Subscriber-Key: $YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "country_code": "IR",
    "what_happened": "Observed shutdown 2026-05-19 14:00 UTC, BGP withdrawal of AS31549. Sourced from NetBlocks twitter + my SSH probe in Tehran.",
    "source_url": "https://twitter.com/netblocks/status/..."
  }'

Rate limit: 10 / hour / key. The label enters error_queue on the Vultr ML server and is incorporated by the next nightly retrain. See /methodology for the gate logic that decides whether new models replace the serving one.

50 uncertain forecasts (lowest distance from 0.5 first)

Why active learning matters

Most ML systems retrain on random labeled samples. Active learning literature shows that labeling the lowest-confidence predictions instead gives the biggest F1 gain per labeling-budget unit.

Voidly's weekly classifier retrain has been rejecting new model candidates since 2026-05-03 because they F1-regress on a dedup'd evaluation set. The gate is working as designed, but without fresh labels we can't distinguish “real regression” from “data drift the model needs to learn.”

This page closes that loop. Journalists + censorship experts see the model's most uncertain forecasts, submit observed outcomes, and the next retrain gate uses their labels for evaluation. Slow but durable.

References

References