Voidly
All findingsPublished research

Real-time per-day SHAP attribution for the 7-day forecast

The /v1/forecast/{cc}/7day endpoint already exposed an aggregate top_features list (3 SHAP contributions for the whole forecast). Journalists kept asking the same follow-up: "okay, but WHY is day 5 higher than day 0?" Until now the only honest answer was "an event boost gets applied outside the…

Read the full published summary

The /v1/forecast/{cc}/7day endpoint already exposed an aggregate top_features list (3 SHAP contributions for the whole forecast). Journalists kept asking the same follow-up: "okay, but WHY is day 5 higher than day 0?" Until now the only honest answer was "an event boost gets applied outside the model." This finding ships per-day SHAP attribution as a structured response field. The 7-day forecast response now carries a top_features_per_day array aligned with the 8-day forecast list, where each entry exposes the top-3 signed contributions ranked across two pools: (a) the model-side SHAP contributions from shap.Explainer permutation on the XGBoost + isotonic predict_proba (constant across days because the model takes one country feature vector), and (b) the deterministic post-model overlay (event_within_Xd, day_decay). On 2026-05-21 the IR forecast for day 7 attributes 0.121 risk to event:Iranian-Election-2026-in-4d (+0.0682) + day_decay_t+7 (+0.0350) — both overlay components — while the aggregate model-side top three (ooni_anomaly_7d -0.0253, block_rate_roll30_mean +0.0214, block_rate_lag1 -0.0175) is also returned separately. A new lean endpoint GET /v1/forecast/{cc}/7day-shap returns just the per-day attribution for callers that do not need the full forecast envelope. Cached 6h on (country, hour); permutation SHAP costs ~3ms once the explainer is warm. Latency overhead added vs raw /7day: <1ms p95 on the upstream service (within noise; explainer + cache amortize). Honest caveats baked into every response: SHAP explains the calibrated predict_proba (post-isotonic), the model features do NOT actually change per day (only the post-model overlay does — exposed honestly), and the multi-horizon model has its own per-horizon SHAP at /v1/forecast/{cc}/multi-horizon. Implementation: scripts/patch-forecast-per-day-shap.py + handleForecast7daySHAP in the API Worker.

Why per-day SHAP, and why now

The Voidly Atlas 7-day shutdown forecast at /v1/forecast/{cc}/7day has shipped probabilities since February 2026, and a single aggregate top_features field of three SHAP contributions since the sentinel trust layer landed in April. That aggregate is honest about what is driving today's risk estimate. It is not honest about why day 5 is higher than day 0.

Journalists hit that wall every time. "Iran's election is in 12 days, so the forecast trends up — show me the day that flipped." Until this finding shipped, the only answer was a hand-wave: "there's an event boost the model doesn't see, applied in the loop after the prediction." Correct, but useless for citation.

What the endpoint returns

Two things, both honest:

  1. A new top_features_per_day array on the existing /v1/forecast/{cc}/7day response, aligned 1:1 with the 8-day forecast list. Each entry exposes the top-3 signed contributions for that day, ranked across two pools:
    • Model SHAP — shap.Explainer permutation algorithm on the calibrated XGBoost + isotonic predict_proba. Constant across days because the model takes exactly one country feature vector per call.
    • Per-day overlay — the deterministic post-model boosts the forecast loop applies (event windows, day-decay). These vary per day and are the actual reason day 5 is higher than day 0.
  2. A new lean endpoint /v1/forecast/{cc}/7day-shap that returns ONLY the per-day attribution for callers that do not need the full forecast envelope.

Sample: Iran, 2026-05-21

DayDateRiskTop-3 contributions
02026-05-210.030Iranian Election in 11d (+0.035)
12026-05-220.060Iranian Election in 10d (+0.038), day_decay_t+1 (+0.005)
32026-05-240.049Iranian Election in 8d (+0.044), day_decay_t+3 (+0.015)
52026-05-260.090Iranian Election in 6d (+0.054), day_decay_t+5 (+0.025)
72026-05-280.121Iranian Election in 4d (+0.068), day_decay_t+7 (+0.035)

Notice that the model-side contributions (ooni_anomaly_7d, block_rate_roll30_mean, block_rate_lag1) do not appear in any day's top-3. That is not a bug — for Iran on this date, the deterministic event boost dominates the model contributions in absolute magnitude. Those model contributions are still returned separately as the aggregate top_features field (and as top_features_aggregate on the lean 7day-shap endpoint) so the caller can see both layers.

Sample: China, 2026-05-21 (Tiananmen anniversary)

Here the model and overlay are both visible in the per-day output, because the overlay event is further out and grows day-over-day:

DayRiskTop-3 contributions
00.031Tiananmen Anniversary in 14d (+0.019, overlay), block_rate_roll30_mean (-0.019, model), gdelt_unrest_30d (+0.016, model)
40.055Tiananmen in 10d (+0.025, overlay), day_decay_t+4 (+0.020, overlay), block_rate_roll30_mean (-0.019, model)
70.066day_decay_t+7 (+0.035, overlay), Tiananmen in 7d (+0.032, overlay), block_rate_roll30_mean (-0.019, model)

Caching and latency

The model SHAP is cached in the sentinel_trust module per (model_version, feature_vector_hash) — same key strategy as the existing aggregate top_features. The per-day attribution is cached per (country, hour) on the forecast service, capped at 256 entries with LRU eviction. The forecast itself re-runs hourly, so 6h cache TTL gives ~6 cache hits per cycle.

Latency overhead added to the upstream service: under 1ms p95 (within noise on a localhost benchmark). The permutation explainer costs ~3ms on its first call after a process restart, then caches. Per-day pool construction is pure Python list-sort with 5-10 candidates per day — negligible.

Honest caveats

Inline in every /7day-shap response, in the honest_caveats field:

  • SHAP explains the calibrated predict_proba (post-isotonic). For watched-set countries the per-country piecewise calibrator runs first; SHAP still maps onto the post-calibration probability, not the raw XGBoost output.
  • The underlying model features do not actually change per day — only the deterministic post-model overlay does. The endpoint exposes both pools honestly rather than pretending the model re-ran 8 times.
  • The multi-horizon model (1d/7d/30d) has separate per-horizon SHAP at /v1/forecast/{cc}/multi-horizon. That is a DIFFERENT codepath (three XGBoost models, one per horizon) and its attribution carries different semantics.
  • Cached 6h on (country, hour). A forecast issued at 14:30 UTC will share its per-day SHAP with all forecasts issued between 14:00 and 14:59 UTC for the same country.

Implementation

scripts/patch-forecast-per-day-shap.py — idempotent patcher that splices into forecast_api.py: captures per-day boost/decay/noise inside the existing forecast loop, computes the per-day SHAP pool after the loop using the already- computed model SHAP, and registers the new /v1/forecast/{cc}/7day-shap Flask route.

API Worker handleForecast7daySHAP() — thin proxy through the API gateway with 1h Cache-Control + 502 fallback. Registered in the Worker’s router ahead of the existing /7day$ regex with the same per-resource rate limit and the same 402-on- rate-limit conversion as the rest of the forecast family.

OpenAPI updated at /openapi.json with a new summary for /v1/forecast/{country}/7day-shap.

The source record.

Original source links and labels. Linked responses may have changed since publication.