Help the classifier learn
These are the 44 recent forecasts the classifier is least sure about — predictions sitting near the 0.5 decision threshold. Each one is a high-value labeling opportunity: a single confirmed outcome here teaches the model more than ten labels on easy cases.
Click any candidate to see its full SHAP breakdown, evidence permalinks, and an anonymous label-suggestion form. No login. No personal data required. Three independent reviewers must agree before a label is promoted into the next retrain dataset.
Method: Impact-aware ranking — Settles 2009 (UW TR-1648). Uncertainty fallback — Lewis 1994. Calibrated uncertainty — arXiv 2510.03162 (2025). · Refreshed every 15 min · Raw JSON · Methodology
Does the loop actually improve the model? Honest answer: not yet.
Loop state: plumbed_idle. The loop is wired end to end (queue -> submit-label -> aggregate -> promote -> retrain) but 0 human labels have been submitted. Until reviewers label candidates at /atlas/active-learning the loop is idle by design — value depends on humans actually labelling. We surface the simulated lift even when it is negative.
Queue: 493 candidates · Human labels submitted: 0
- Active-learning value depends on humans actually labelling. If labels_submitted is 0 the loop is plumbed but idle.
- simulated_f1_lift uses PROXY labels (incident-table positives within +/-3 days, else negative), not human labels. It is an optimistic estimate — real human labels on uncertain days are noisier and the AL queue surfaces days the incident table is silent on. Treat the lift as 'is the queue surfacing learnable rows', not a promotion signal.
Live source: /v1/sentinel/active-learning-loop-status
How submissions become labels
- You read the evidence, pick a label, and submit anonymously.
- Submission lands in an append-only audit log (
active_learning_submissions.jsonl). Nothing is overwritten or deleted. - Once 3+ independent reviewers agree on the same candidate, the daily aggregator promotes a consensus label.
- The next weekly classifier retrain consumes the consensus file and gates promotion as usual.
Privacy: contact email is optional and only stored in the audit log — never echoed in any read endpoint and stripped from the consensus output. Your IP is hashed for spam-flood detection only (16-hex truncated SHA-256, useless for re-identification).
60 candidate forecasts — ranked by uncertainty
Related
- Impact-aware ranking methodology — why we replaced uncertainty-only sorting with a 3-factor heuristic EER
- /sentinel/labeling — read-only queue view with API-key submission flow
- /atlas/changelog — model registry / retrain history
- /methodology — retrain gate logic
- Settles 2009 — Active Learning Literature Survey (UW TR-1648)
- Calibrated Uncertainty Sampling (arXiv 2510.03162)