A calendar-and-history shutdown hazard did not beat recurrence in development (2022–2025)

No improvement established. A calendar-and-history shutdown hazard ("arm A") was rejected in every development fold for 2022–2024. It did not beat the best simple baseline under the frozen rule, at a budget of five alerts a day. The 2025 fold is marked exposed and was rejected too. This is a negative result from offline development folds. It is not evidence that arm A can warn of shutdowns, and we make no claim that it can.

The question

Can a model that knows only calendars and a place's past shutdowns rank the places most likely to see a new shutdown start in the next one to seven days? Can it do so better than simple rules that use the same history? Arm A is that model. We built it as the reference arm that later, richer models must beat. The pre-registered expectation was that recurrence (places with recent shutdowns are likely to see more) would be hard to beat. That is what happened.

What went in

Labels. Shutdown starts come from Access Now's STOP data for 2019–2025. They pass through our frozen label projection:

• lanes come from the stated cause;
• cuts by another state's forces are quarantined;
• renewals and nearby cuts merge into one campaign.

Units. There are 152 national units and 141 state or region units in the four countries with frequent regional cuts: India, Myanmar, Pakistan and Russia. Pakistan and Russia are provisional. A ranking is issued every day at 06:00 UTC.

Features. Arm A uses only calendars and shutdown history. Examples: how many shutdowns a unit had in the past year and month, how long ago its last one was, and recent shutdowns elsewhere in the same country. Nothing about a shutdown's cause, actor or area enters as a feature.

Vintage rule. A past shutdown becomes usable 14 days after it starts. This mirrors the lag of our own adjudication. A second run used a stricter rule: a shutdown becomes usable only after the annual report covering it is published. That run gave the same verdicts.

The frozen protocol

The plan was frozen and hashed before the first fold ran (ddf5bb4464bea5112fbc0c4b46eef32105ce2de4f001f78479e45dbd00f507d0). A ruled revision fixed how cuts in Iraqi Kurdistan count, dated one measurement reading and wrote the known limits into the plan (85b70e904f3993d9e43cfd46231fbaae74cbd5052274d8639313e5a157b81991). The revision changed some numbers below and no verdict. A third revision replaced placeholder release dates for the stricter run with the reports' own published dates (e2e9b984b7f95b1d35704ecd41836affaa63663438d1bf8d0d10945fd8201de6). The re-run changed figures in that run only, and again no verdict.

• Folds. Each test year is scored by a model trained only on earlier years: 2022, 2023 and 2024. The 2025 fold is shown apart and marked exposed, because 2025 figures had already been seen.
• Budget. Each arm issues one list per day across national and regional units together. The top five count; a place alerted in the last seven days is skipped. An alert is a hit when the place's new shutdown starts one to seven days after the 06:00 UTC issue. A same-day start is not a hit.
• Baselines on the same panel:
  • local base rate (climatology);
  • recurrence (persistence);
  • a history-only logistic model;
  • a feature-free random policy.
  An always-alert arm is reported for its burden.
• Promote rule, per stratum. All three must hold:
  • the 95% interval for arm A's recall gain over the strongest baseline lies above zero;
  • its precision gain is not below zero;
  • its Brier skill score against the local base rate is above zero.
  The intervals come from 2,000 paired resamples of unit-months. In development the rule can only reject; it never promotes.

The result

The table shows the decision line (see below) for unscheduled shutdowns, in the primary 14-day-lag run. "Recall@5" is the share of new shutdown starts that one of the day's top five alerts caught one to seven days ahead.

Where arm A's recall point estimate was higher (2022 and 2024 national), the interval still included zero, so the fold was rejected. Precision@5 is small for every arm because five daily alerts over a year far outnumber the starts. The counts are small; regional starts on the decision line number 2 to 15 a year. They are development counts, not rates to expect in operation.

The label-line caveat

Prospectively, a shutdown counts on the decision line only if two independent technical sources confirm it. Many regional cuts cannot pass that test, so each one is tagged with the line it would get:

• order_documented: district cuts in India;
• report_only: Myanmar states that measurement cannot see, and mobile-only cuts in Russia;
• route_level: localized mobile cuts in Pakistan.

Only the decision line counts toward the decision. On the other lines arm A caught far fewer starts than recurrence; for example, 0.069 against 0.759 for Indian district cuts in 2024. Those lines are reported, never pooled.

Scheduled shutdowns: void, not a result

Exam and religious-calendar shutdowns need calendars as they were known at the time, with the date each timetable was published. No such announcement-dated calendar data exists for past years: the dates we have are either cut orders or the days cuts were seen. So the scheduled-track comparison is void. Our notice register now captures exam timetables, moon-sighting announcements and election schedules with their publication times. These terms will exist going forward and cannot be backfilled.

Limits that matter

• Retrospective labels. Lanes and label lines are assigned by rule from STOP fields, not by our adjudicators. Tagging lines this way approximates the prospective technical test.
• Countries without records. 68 of the 152 roster countries have no STOP records, so they have no development labels and are not scored here.
• Measurement coverage. Myanmar's coverage tiers are one reading as of 14 October 2026, applied to every past day.
• Missing terms. The 7-day contagion term was removed because it is always zero under the 14-day lag. The "national cut active" state is zero in live use until our label ledger records ends.
• Report dates. The stricter run uses each annual report's published date. The 2020 report may have been public two days earlier than the date used, which is unverified.
• Small counts. Intervals on a handful of starts are wide; a rejection here means the fold did not show a gain, not that the arm is proven useless.

What happens next (pre-registered)

Arm A is still logged as the reference arm. Its pre-registration reads: "arm A v1 – calendar + history hazard; rejected in development folds 2022–2025 against recurrence; logged prospectively as the reference arm; no claim."

It joins the daily commitment at the 17 October 2026 06:00 UTC issue, in a new file version beside the existing arms. The cutover is recorded in the commitment index and on the early-warning page the day before. The model is frozen at that point: fitted once on every label with an onset up to 31 December 2025, and never refit inside its window. A refit would be a new arm with a new clock.

Its own evaluation clock starts at the cutover. The decision comes after at least 180 days and 120 confirmed onsets on the decision line, both. The result will be published whatever it shows.

Evidence

The fold results are held privately under the STOP data licence, which allows internal development use only until a rights receipt exists. Their SHA-256 digests let the figures above be checked against them later:

• results 8cd579b125cbb65302d53befd05b83f21db4afd505345c094589c85eee4ac1e6;
• stricter-run results with the reports' published dates b63aacf5873afd657aae974aef5e07249ea071799bf79d018db41c4c3b46797a;
• run report f7ec3acdf1b3e2243ec9f4a41b4e1ef38282ac5d526a01411aa24df90394488c;
• frozen production model 1899136549782598c901f5762ad0c980f1cc0f5617c9d3984461ea597185e1bc;
• label projection episodes 9440608805f6f07ef4b03651af3f58640adfed36f0a2a89b7935dfed8144c114.

These records support the figures above. They are not a release of the underlying data, and the result has not been independently reproduced.

Paired: the conflict-news negative result, which also found history hard to beat.