Daily domain delta: which domains gained or lost blocking countries overnight

Most censorship-research dashboards answer the question "is domain X blocked in country Y right now?" The Voidly Atlas daily domain delta answers a different question, the one journalists actually need at 8 AM: which domains gained or lost blocking countries between yesterday and today?

This is the "ChatGPT picked up three new blocking countries overnight" signal. It runs once a day at 04:00 UTC, diffs today's per-domain set of blocking countries against yesterday's, and ships the top 25 gainers, the top 25 losers, and a 7-day per-domain history as a pre-rendered sidecar that any client can read in one HTTP round-trip.

How it works

For every (domain, country, day) tuple in the trailing 7 days of the 85K-row evidence table, we compute the count of block measurements that day. Block measurements are rows where signal_type is one of block, dns-blocking, http-blocking-tcp-reset, http-blocking-timeout, tcp-reset, blockpage, or interference, gated by signal_value > 0.5.

We then require min_observations=3 per (domain, country, day) before that country counts as "blocking domain X today" — a single anomalous probe row does not flip the bit. Today's delta is just today_set − yesterday_set (countries_added) plus yesterday_set − today_set (countries_removed), and movers are ranked by absolute delta.

Why this is hard to do honestly

The temptation is to turn "Twitter gained Singapore overnight" into a headline. The Atlas refuses to. Probe coverage is uneven: every country's probe schedule is rotating, every six hours, on a priority-tier basis. A "new block" almost always reflects probe rotation rather than a fresh block going up. The first build of the pipeline confirmed this — the largest gainers on day one were Skype and GitHub Copilot, neither of which is the subject of a fresh multi-country censorship event. The signal was probe coverage expanding into countries we hadn't probed yesterday, exactly as the caveat predicts.

The endpoint bakes four honest caveats into every response:

1. Probe coverage is uneven — a "new block" can mean we just started probing that country.
2. Some delta is intermittent connectivity (one DNS resolver momentarily reached the domain). The min_observations=3 floor cuts most of it but treat 1-2 country deltas as noise.
3. "Lost a block" can mean a country was probed once today but three times yesterday — not that the block was lifted.
4. 24h windows have under 500 global block measurements on average. This is a leading indicator, not ground truth.

The journalist-grade workflow is: read the daily delta, find a domain that moved by ≥ 3 countries with at least one independent corroborating source per country, then click through to the per-domain 7-day history endpoint and confirm the trend isn't a single-day blip.

Endpoints

• GET /v1/atlas/domain-delta — today's top gainers and losers. Supports ?limit, ?min_delta, ?domain filters.
• GET /v1/atlas/domain-delta/<domain> — 7-day per-domain history of blocking countries (only for domains in today's top-mover set, storage budget).
• GET /v1/atlas/domain-delta/info — methodology, build time, freshness, caveats. No row data.

Schema and provenance

Pipeline script: scripts/build-daily-domain-delta.py. Endpoint patch: scripts/patch-domain-delta-endpoint.py. Both shipped to Vultr at /opt/voidly-ai/scripts/. Sidecar written to /opt/voidly-ai/ml-deploy/domain_delta.json. Cron entry: 0 4 * * *. Schema: voidly-atlas-domain-delta/v1.