Voidly
All findingsPublished research

Probe-node integrity detector: a dated methodology and its limits

Correction 2026-10-10: this May 2026 finding describes a historical integrity-scoring method, not current detector or API availability. Its first recorded run scored 35 attributed nodes and flagged 10 for human review. Sparse evidence, different network vantage points, and imperfect consensus limit…

Read the full published summary

Correction 2026-10-10: this May 2026 finding describes a historical integrity-scoring method, not current detector or API availability. Its first recorded run scored 35 attributed nodes and flagged 10 for human review. Sparse evidence, different network vantage points, and imperfect consensus limit the meaning of a flag; it never proved malicious conduct or automatically disabled a node. Individual node IDs, verdicts, and operator details are withheld to protect operators. The accompanying article retains the aggregate method and limitations.

What this is

Correction, 2026-10-10. This is a dated account of the May 2026 integrity-scoring method, not evidence that the detector or its API is currently operating. Individual probe IDs, verdicts and operator details are withheld from public view to protect operators. The “community” label denotes a registration path; it does not establish independent ownership or volunteer status.

Probes are the ground truth. Every confirmed-censorship incident Voidly Atlas publishes is built on probe evidence. So a community probe that lies — an adversary standing one up to report “everything is blocked” and manufacture a fake shutdown, or “nothing is blocked” to mask a real one, or simply a misconfigured node that mislabels benign traffic — corrupts incidents directly. A lying ground-truth source is the worst failure mode the system has.

The probe-integrity detector (scripts/build-probe-integrity.py) was designed to catch that without trusting any single probe. For every probe node over a trailing 30-day window it asks one question: when this node tested a (domain, country) target, did its verdict agree with the consensus of every other probe and upstream source that tested the same thing? A node that consistently disagrees with everyone else is either compromised, misconfigured, or sitting somewhere with genuinely different blocking — all three are worth a human look.

What a “verdict” is

Every evidence row is collapsed to a binary verdict — BLOCK (the target was censored) or CLEAR (it was reachable). The two row types encode that verdict differently, so each gets its own decoder.

Voidly probe rows. Every probe measurement lands with signal_type=block — the row is a block-test record, not a verdict — and a structured JSON blob in upstream_claim carrying the real signal, blockType:

  • dns-poisoned, tcp-reset, blockpage, sni-blocked → BLOCK. Genuine DPI censorship signatures.
  • http-redirect → CLEAR. A redirect such as copilot.github.com → github.com/copilot or skype.com → teams.live.com is normal product behaviour, not censorship. A probe recording an http-redirect as a block is itself misconfigured — and a probe doing that on every domain is exactly what this detector should surface.
  • tcp-timeout, unknown → CLEAR (transient-leaning). A bare timeout with no DPI fingerprint is more often congestion than a block; it is not allowed to vote BLOCK.

Upstream-source rows (OONI / CensoredPlanet / IODA — no probe attribution) use signal_type as a real verdict. IODA outage rows are dropped entirely: a country-level connectivity outage is not a domain-specific censorship verdict and would wrongly drag a country pool toward BLOCK.

Consensus — a three-tier fallback

The probe_node_id column is genuinely sparse: 35 probe nodes, roughly 265 attributed rows over 30 days, and most (domain, country, day) cells are touched by only one probe. A strict “same domain, same country, same day” rule would leave almost every probe row un-scorable. So consensus is built with a three-tier fallback, and each tier carries a weight reflecting how comparable it is:

  1. Tier 1 (weight 1.0) — other rows on the same domain + country + day. The gold standard: same target, same day, a different observer. Apples to apples.
  2. Tier 2 (weight 0.6) — other rows on the same domain + country, a different day inside the window. Blocking policy is sticky day-to-day, so this is a decent but weaker proxy.
  3. Tier 3 (weight 0.3) — the country base block rate over the window, built from upstream sources only. An adversary could stand up many probes and manufacture a fake country consensus, so the base rate is anchored only to the independently-operated OONI / CP / IODA measurement sources. Weakest tier, but it is the bridge that catches a probe whose verdict is wildly off the country norm.

For each probe row the detector takes the best available tier and asks whether the probe's verdict matches the majority verdict of that tier's pool. The node's agreement_rate is the tier-weighted mean match over its comparable rows. The probe's own rows are always excluded from the pool it is scored against — a node cannot vouch for itself.

Two secondary signals

Degenerate verdict distribution. A node reporting only blocks or only clears across at least 6 distinct targets is the “everything-is-blocked” / “nothing-is-blocked” adversary signature. It is not a standalone flag — internal probes were seeded on a low-censorship datacenter domain list (mostly CLEAR) and community probes on known-blocked domains (mostly BLOCK), so a uniform distribution is often benign. It docks the integrity score, and becomes a flag only when it corroborates a low or borderline agreement rate.

Reporting-volume outlier. A compromised node can also poison by volume — flooding the table with fake “block” reports so its evidence dominates aggregates. A node whose block-report count exceeds 5× its same-class peer median is flagged.

The integrity score and the flag rule

integrity_score (0–1) = clamp01(agreement_rate − 0.15·degenerate − 0.15·volume_outlier). A node with no comparable rows gets a neutral agreement_rate of 0.5 — the detector genuinely cannot judge it.

A node is flagged low-integrity when its agreement_rate < 0.70, or it is a volume outlier, or it has a degenerate distribution that corroborates a low / borderline agreement.

Confidence scales with how much comparable history the node has: clamp01(comparable_rows / 30). A flagged node with confidence < 0.40 is surfaced as “investigate, do not act” — there is not yet enough evidence to judge it.

Dated aggregate result

The May 2026 first run scored 35 nodes with attributed evidence and flagged 10 for human review. Several flags had low confidence because their comparable sample was small. A flag did not establish malicious conduct or justify automatic disabling. Per-node examples and verdict details are withheld by design.

Honest caveats

  1. A low-agreement node may be correct. A probe in a region with genuinely different blocking than the consensus pool will “disagree” while being right. That is why the described detector flagged for human review without automatically disabling a probe.
  2. Consensus itself can be wrong. Most probes and sources cluster in one region (heavy datacenter and Global-North skew), so the majority verdict reflects that region and an honest probe elsewhere is penalized.
  3. Tier-2 and Tier-3 consensus are proxies. Different-day and country-base-rate comparisons are weaker than same-cell agreement; the tier weights down-rank them but cannot make them rigorous.
  4. New and low-volume nodes have little history. A flag with confidence < 0.40 means “not enough evidence to judge” — a prompt to watch, not a verdict.
  5. The probe_node_id column is sparsely populated. Most evidence rows carry no probe attribution; those rows still feed the consensus pool but cannot be scored as a probe. Coverage grows as attribution improves.
  6. Degenerate verdict distribution is a weak signal in that dated dataset because internal vs community probes were seeded with different domain lists. It nudges the score; agreement_rate is the load-bearing metric.

In the implementation described at publication, the detector flagged cases for human review and did not automatically disable a probe.

Public sources

Per-probe integrity endpoints and operator listings are withheld from public access. Aggregate probe statistics are available separately; they do not reproduce node verdicts or identify operators. The checked-in pipeline script and sidecar schema document the historical method, not current execution.

The source record.

Original source links and labels. Linked responses may have changed since publication.