What this is

The Voidly probe network is 40 nodes — 15 internal (Voidly-run, datacenter-hosted) and 25 community probes (cp-* node IDs, run by volunteers on hardware Voidly does not control). The community half is the trust weak point of the entire censorship pipeline.

Probes are the ground truth. Every confirmed-censorship incident Voidly Atlas publishes is built on probe evidence. So a community probe that lies — an adversary standing one up to report “everything is blocked” and manufacture a fake shutdown, or “nothing is blocked” to mask a real one, or simply a misconfigured node that mislabels benign traffic — corrupts incidents directly. A lying ground-truth source is the worst failure mode the system has.

The probe-integrity detector (scripts/build-probe-integrity.py) exists to catch that without trusting any single probe. For every probe node over a trailing 30-day window it asks one question: when this node tested a (domain, country) target, did its verdict agree with the consensus of every other probe and upstream source that tested the same thing? A node that consistently disagrees with everyone else is either compromised, misconfigured, or sitting somewhere with genuinely different blocking — all three are worth a human look.

What a “verdict” is

Every evidence row is collapsed to a binary verdict — BLOCK (the target was censored) or CLEAR (it was reachable). The two row types encode that verdict differently, so each gets its own decoder.

Voidly probe rows. Every probe measurement lands with signal_type=block — the row is a block-test record, not a verdict — and a structured JSON blob in upstream_claim carrying the real signal, blockType:

• dns-poisoned, tcp-reset, blockpage, sni-blocked → BLOCK. Genuine DPI censorship signatures.
• http-redirect → CLEAR. A redirect such as copilot.github.com → github.com/copilot or skype.com → teams.live.com is normal product behaviour, not censorship. A probe recording an http-redirect as a block is itself misconfigured — and a probe doing that on every domain is exactly what this detector should surface.
• tcp-timeout, unknown → CLEAR (transient-leaning). A bare timeout with no DPI fingerprint is more often congestion than a block; it is not allowed to vote BLOCK.

Upstream-source rows (OONI / CensoredPlanet / IODA — no probe attribution) use signal_type as a real verdict. IODA outage rows are dropped entirely: a country-level connectivity outage is not a domain-specific censorship verdict and would wrongly drag a country pool toward BLOCK.

Consensus — a three-tier fallback

The probe_node_id column is genuinely sparse: 35 probe nodes, roughly 265 attributed rows over 30 days, and most (domain, country, day) cells are touched by only one probe. A strict “same domain, same country, same day” rule would leave almost every probe row un-scorable. So consensus is built with a three-tier fallback, and each tier carries a weight reflecting how comparable it is:

1. Tier 1 (weight 1.0) — other rows on the same domain + country + day. The gold standard: same target, same day, a different observer. Apples to apples.
2. Tier 2 (weight 0.6) — other rows on the same domain + country, a different day inside the window. Blocking policy is sticky day-to-day, so this is a decent but weaker proxy.
3. Tier 3 (weight 0.3) — the country base block rate over the window, built from upstream sources only. An adversary could stand up many probes and manufacture a fake country consensus, so the base rate is anchored only to the independently-operated OONI / CP / IODA measurement sources. Weakest tier, but it is the bridge that catches a probe whose verdict is wildly off the country norm.

For each probe row the detector takes the best available tier and asks whether the probe's verdict matches the majority verdict of that tier's pool. The node's agreement_rate is the tier-weighted mean match over its comparable rows. The probe's own rows are always excluded from the pool it is scored against — a node cannot vouch for itself.

Two secondary signals

Degenerate verdict distribution. A node reporting only blocks or only clears across at least 6 distinct targets is the “everything-is-blocked” / “nothing-is-blocked” adversary signature. It is not a standalone flag — internal probes were seeded on a low-censorship datacenter domain list (mostly CLEAR) and community probes on known-blocked domains (mostly BLOCK), so a uniform distribution is often benign. It docks the integrity score, and becomes a flag only when it corroborates a low or borderline agreement rate.

Reporting-volume outlier. A compromised node can also poison by volume — flooding the table with fake “block” reports so its evidence dominates aggregates. A node whose block-report count exceeds 5× its same-class peer median is flagged.

The integrity score and the flag rule

integrity_score (0–1) = clamp01(agreement_rate − 0.15·degenerate − 0.15·volume_outlier). A node with no comparable rows gets a neutral agreement_rate of 0.5 — the detector genuinely cannot judge it.

A node is flagged low-integrity when its agreement_rate < 0.70, or it is a volume outlier, or it has a degenerate distribution that corroborates a low / borderline agreement.

Confidence scales with how much comparable history the node has: clamp01(comparable_rows / 30). A flagged node with confidence < 0.40 is surfaced as “investigate, do not act” — there is not yet enough evidence to judge it.

First run — 35 nodes scored, 10 flagged

The first run scored all 35 probe nodes that had attributed evidence in the window and flagged 10: 8 community probes and 2 internal.

• cp-3e6ixxgs is the clearest case. It reports washingtonpost.com, whatsapp.com, messenger.com and rferl.org as blocked in Great Britain — but the GB country base rate from OONI / CP is 147 clear to 3 block. agreement_rate 0.00 across 13 comparable rows, integrity_score 0.0, flagged actionable for human review.
• Seven of the ten flags carry low confidence (below 0.40, marked “investigate, do not act”) because those community nodes have only 4–9 comparable rows — the honest signal that there is not yet enough history to judge them.
• blr, an internal node, is flagged because it classifies copilot.github.com as a benign redirect (CLEAR) in India while OONI labels the domain blocked. Here the probe is arguably correct and the consensus is wrong — which is precisely why this detector flags for review and never auto-bans. See the caveats below.

Honest caveats

1. A low-agreement node may be correct. A probe in a region with genuinely different blocking than the consensus pool will “disagree” while being right. That is why this detector flags for human review and never auto-bans or disables a probe.
2. Consensus itself can be wrong. Most probes and sources cluster in one region (heavy datacenter and Global-North skew), so the majority verdict reflects that region and an honest probe elsewhere is penalized.
3. Tier-2 and Tier-3 consensus are proxies. Different-day and country-base-rate comparisons are weaker than same-cell agreement; the tier weights down-rank them but cannot make them rigorous.
4. New and low-volume nodes have little history. A flag with confidence < 0.40 means “not enough evidence to judge” — a prompt to watch, not a verdict.
5. The probe_node_id column is sparsely populated. Most evidence rows carry no probe attribution; those rows still feed the consensus pool but cannot be scored as a probe. Coverage grows as attribution improves.
6. Degenerate verdict distribution is a weak signal in the current data because internal vs community probes were seeded with different domain lists. It nudges the score; agreement_rate is the load-bearing metric.

The detector is strictly detection plus flagging. It never auto-disables a probe. Acting on a flag is a human decision.

Endpoints

• GET /v1/atlas/probe-integrity — every probe node scored, with compact per-node rows. Supports ?flagged_only=true, ?node_class=community|internal, and ?min_score / ?max_score filters.
• GET /v1/atlas/probe-integrity/{node_id} — full integrity detail for one probe node, including the sample of (domain, country, day) cells where the node disagreed with consensus and the verdict-decoding reference.

Schema and provenance

Pipeline script: scripts/build-probe-integrity.py, shipped to Vultr at /opt/voidly-ai/scripts/. Sidecar written to /opt/voidly-ai/ml-deploy/probe_integrity_v1.json. Cron entry: 15 6 * * * (daily, 06:15 UTC). Schema: voidly-atlas-probe-integrity/v1. Endpoints served by the Atlas API and proxied through api.voidly.ai.