The gap the watchdog plugs
Voidly creates incidents through three deterministic pipelines:
scripts/create-voidly-incidents.py (Voidly
probe blocks), scripts/ingest-ooni-incidents.py
(OONI ground-truth signals), and the IODA ingest's
promote_to_incidents() function (which produces
noisy disruption labels, not censorship). Each of these is a thin rule over
one upstream source. None of them looks at the joint signal across sources,
and none of them looks at our second-opinion DBSCAN unsupervised anomaly
model. If a country-day looks like censorship but the three rules miss it,
the only way it makes it into the incident table is if a human notices.
What the watchdog does
scripts/auto-incident-watchdog.py runs every
six hours under cron and proposes draft incidents only when
four independent gates fire together:
- The DBSCAN second-opinion model flagged the country-day as anomalous
(
is_anomaly = trueon the leaderboard). - The Bayesian corroboration model puts the per-day censorship posterior at or above 0.5.
- No non-draft incident exists for the country within +/- 7 days (so we don't duplicate work the deterministic pipelines already did).
- The country-day has at least one anomalous-level evidence row from a confirmed source — OONI, CensoredPlanet, or a Voidly probe. IODA disruption rows are deliberately excluded from this gate, for the same reason the v3.3 classifier's training labels exclude them: IODA is a connectivity-disruption signal, not a censorship signal, and treating it as ground truth produces noise.
When all four hold, the watchdog writes a JSON draft to
/opt/voidly-ai/data/auto_incidents_queue/<slug>.json
with status = "draft",
created_by = "auto-watchdog",
the Bayesian posterior under
evidence_corroboration_posterior,
the DBSCAN anomaly score, the IDs of every linked confirmed-source
evidence row, and a template narrative. The drafts are exposed at
GET /v1/atlas/auto-incidents-pending for an
editor to review.
What the watchdog explicitly does not do
- It never writes to the
incidentstable. Drafts sit only in the queue dir. The 343-citable-censorship headline on the index page is unchanged by every run of the watchdog. - It does not auto-publish to
atlasFindings.ts. - It does not notify subscribers (alerts API is untouched).
- It does not estimate or report a controlled false-positive rate. The DBSCAN model's AUC versus labeled incidents is 0.6506; the Bayesian corroboration model's AUC is around 0.78. Requiring both to fire gives meaningfully higher joint precision than either alone, but we don't have a held-out test set that measures the conjunction directly. Treat this as a watchdog, not an autonomous publisher.
First-run result (2026-05-21 18:02 UTC)
Four DBSCAN flips on the leaderboard: MA, EG, IQ, IN on 2026-05-20. All four failed the Bayesian corroboration gate (posteriors 0.37, 0.37, 0.00, 0.18 — well below the 0.5 floor). All four also have existing incidents within +/- 7 days, so even with a high posterior they would have been deduped. Net: 0 drafts written. That is the honest baseline — on a normal day, the deterministic pipelines cover the obvious cases, and the watchdog stays quiet.
Operations
- Cron:
0 */6 * * * /usr/bin/python3 /opt/voidly-ai/scripts/auto-incident-watchdog.py - Log:
/var/log/voidly-auto-incident-watchdog.log - Queue dir:
/opt/voidly-ai/data/auto_incidents_queue/(one JSON per draft, plus_manifest.jsonwith per-run counts and skip reasons). - Endpoint:
GET /v1/atlas/auto-incidents-pending(supports?country=,?min_posterior=,?limit=, and?include_body=truefor the full template HTML). - Single draft:
GET /v1/atlas/auto-incidents-pending/<slug>. - Stale-draft archive: drafts not promoted within 30 days move to
auto_incidents_queue/_archive/on the next watchdog run.
Editor flow
Every draft is a single JSON object with a stable
slug, the trigger numbers, and the linked
evidence row IDs. To promote a draft, an editor confirms the underlying
numbers (each draft's raw_links field
points at the live API endpoints the trigger was computed from), then
inserts a real row into the incidents table
by copying the draft's fields into one of the existing creator scripts
(create-voidly-incidents.py or
ingest-ooni-incidents.py) with
status = "active". To dismiss a draft, the
editor deletes the JSON — no database rollback is needed because the
watchdog never touched the database.
Honest caveats
- Draft is not the same thing as confirmed. An editor must promote.
- The false-positive rate of the conjunction (DBSCAN flip AND Bayesian posterior ≥ 0.5 AND confirmed-source evidence row) is not measured against a held-out test set. We measure each component separately.
- Drafts are NEVER counted in the public 343-citable-censorship total. They live entirely in a parallel queue.
- Re-running the watchdog on the same (country, date) refreshes
source_databut preserves the originalfirst_seenso editor-facing timestamps are stable.