Probe scheduling optimizer: a Thompson-sampling priority list for where to point the probe network next
The Voidly probe network has a hard capacity ceiling: roughly 40 nodes, 62 domains, a 5-minute cadence. Today every node runs the same fixed domain list everywhere. That spends probe attention uniformly — and uniform is wasteful. A (country, domain) pair we have measured 400 times, all of them showing the same block, has nothing left to teach us on probe 401. A pair we have seen four times, or one whose block state flipped last week, is exactly where a fresh probe adds knowledge.
The probe scheduling optimizer is a recommendation engine for that problem. It produces a ranked priority list — which (country, domain) pairs to probe next — so a scheduler, or a human running the network, can concentrate scarce probe cycles where the model is most uncertain.
The method: Thompson sampling per (country, domain)
Each (country, domain) pair is modelled as a bandit arm carrying a
Beta(alpha, beta) posterior over the latent parameter
p = P(this pair is blocked). We start from a weakly
informative Jeffreys prior Beta(0.5, 0.5).
- Accumulate evidence.
alphagains recency-weighted block observations,betagains recency-weighted unblock observations. A block is any evidence row whosesignal_typeis in the standard Atlas block set (dns-blocking,http-blocking-tcp-reset,block,interference,tor-blocking, and similar); an unblock is anokrow. Each observation is weighted byexp(-age_days / 45), so a block state that flipped a month ago is not drowned by a year of stale agreement. - Compute posterior variance. The variance of
Beta(a, b)isa·b / ((a+b)² · (a+b+1)). It is large when a pair is under-sampled or genuinely 50/50, and small when many consistent observations have piled up. That variance is the information-gain proxy — probe where the posterior is widest. - Draw a Thompson sample. One sample from each Beta. This is the bandit exploration step: it occasionally surfaces a pair even when its mean variance is modest, so the ranking never fully starves an arm.
- Combine and rank.
priority = posterior_variance × recency_weight × flip_weight × (1 + thompson_weight · (1 - |sample-0.5|·2)). The recency weight favours pairs not observed in a long time; the flip weight (1.5×) favours pairs whose block state changed in the trailing 30 days; the Thompson term is bounded small so the deterministic variance-and-recency signal leads and the sample only re-orders near-ties.
First run
The first build (21 May 2026) scored 1,054 (country, domain) pairs from a 365-day evidence window, skipping 57 cold pairs with fewer than two observations. 269 pairs showed a block-state flip in the trailing 30 days. The five highest-priority pairs to probe next were Uzbekistan / twitter.com, Iran / binance.com, Algeria / tiktok.com, Iran / tumblr.com, and Azerbaijan / telegram.org — all sparse-but-contested pairs where the posterior is wide and the last observation is months old.
Why this is hard to do honestly
Four honest caveats are baked into every API response:
- This is a recommendation, not a control loop.
It is not wired into
probe_module.pyor the live probe scheduler. Nothing changes the probe cadence yet. The pipeline produces a priority list; consuming it is a separate, out-of-scope step. - Thompson sampling assumes independent arms. Each (country, domain) pair is treated as an independent bandit. In reality DPI policy correlates domains within a country — a censor flipping one news site often flips many at once. A truly correct model would be hierarchical or contextual. Treat the ranking as a heuristic, not an optimum.
- High variance is not always censorship uncertainty. A wide posterior can mean a pair is simply intermittently reachable — a flaky DNS resolver, a congested transit link — rather than genuinely contested. The recent-flip term partly separates the two, but not perfectly.
- Cold pairs are out of scope. Pairs with fewer than two observations cannot be scored from evidence and are not ranked here. The existing fixed-list scheduler already covers them, so unseen pairs still get probed.
Endpoints
GET /v1/atlas/probe-priority— the top-K (country, domain) pairs to probe next, ranked by priority score. Supports?limit,?country,?domain,?recent_flip=true, and?min_observationsfilters.GET /v1/atlas/probe-priority/info— full methodology, the Beta prior, the priority formula, generation metadata, and the honest caveats. No row data.
Schema and provenance
Pipeline script:
scripts/build-probe-scheduling-priority.py. Endpoint
patch: scripts/patch-probe-priority-endpoint.py. Both
shipped to Vultr at /opt/voidly-ai/scripts/. Sidecar
written to
/opt/voidly-ai/ml-deploy/probe_scheduling_priority_v1.json.
Cron entry: 50 4 * * * (daily, 04:50 UTC). Schema:
voidly-atlas-probe-scheduling-priority/v1. The Thompson
draws use a fixed random seed so the daily cron is reproducible.