Classifier v3.3 — the GradientBoosting model in production — has a known split personality. Its leave-country-out (LOCO) cross-validation median F1 is a healthy 0.870, but the mean is only 0.711. A ~16-country tail drags the mean down: OM, UZ, TN, LY, YE, JO, MA and similar MENA / former-Soviet states, each scoring LOCO F1 between 0.00 and 0.36. This finding is a serious, honest attempt to lift that tail with empirical-Bayes partial pooling — and a report on why it didn't work.

The hypothesis: James-Stein shrinkage toward a regional prior

Partial pooling is the textbook small-sample remedy. Instead of retraining v3.3, you wrap its output probability in a shrinkage layer:

p_final = w · p_v3.3 + (1 - w) · p_region
w       = n_country / (n_country + k)

A data-rich country (large n_country) gets w ≈ 1 and keeps its own v3.3 estimate. A data-poor country gets a small w and is pulled toward p_region, the mean v3.3 prediction across its UN-region peers. k — the sample count at which a country gets exactly 50% weight on itself — is tuned by an inner LOCO sweep. The held-out country never contributes to its own regional prior, so the cross-validation stays honest. This is the same James-Stein / empirical-Bayes shrinkage used across statistics for exactly this "noisy estimates from tiny samples" problem. It ships on top of v3.3 with no retrain — a pure post-hoc layer.

It failed the promote gate — decisively

The promote gate had three conditions, all required. The pooled layer cleared one and failed two:

The k-sweep is the tell. Across k ∈ {4, 6, 8, …, 60}, LOCO mean F1 is strictly monotonically decreasing: 0.7136 at k=4, then 0.671, 0.663, 0.652 … collapsing to 0.461 at k=60. There is no interior optimum. Every amount of shrinkage hurts; the sweep simply picks the smallest k it was offered, where shrinkage is weakest and therefore least harmful. A +0.27pp "win" at k=4 is the layer doing as close to nothing as the grid allows.

The 16 tail countries: 15 of 16 did not move at all

Only TN cleared the +3pp bar, and that is a single row flipping in a country with 4 positives — statistical noise, not a signal. The data-rich flagship countries were equally inert: IR 0.850→0.850, RU 0.613→0.613, VE 0.900→0.900, CN 0.294→0.303, EG 0.726→0.732. Nothing moved meaningfully, anywhere.

Why it didn't work: the "tail" is positive-poor, not row-poor

The shrinkage design assumes the tail countries are data-poor — the docstring imagines "a country with 8 labeled days." That assumption is wrong for these specific countries. The 16 named tail states are heavily probed MENA / post-Soviet countries: in the labeled dataset they hold 53 to 85 rows each (OM 53, LY 55, YE 53, JO 77, MA 82, DZ 81, AZ 65, KZ 85). With n_country ≥ 53, the shrinkage weight w = n / (n + k) is ≥0.93 even at k=4, and ≥0.81 even at k=12. The layer is structurally near-inert for exactly the countries the gate watches — to pull one of them halfway to its regional prior you would need k ≈ 70, and the sweep shows k≥40 craters the median F1 as the genuinely data-rich countries get over-shrunk.

What these countries actually lack is positive labels: 2-22 confirmed-censorship days apiece (YE 2, KW 2, LY 3, OM 4, TN 4). v3.3 produces honest mean probabilities of ~0.15-0.41 for them — predictions that genuinely sit below the 0.5 decision threshold because the signal is ambiguous. Their UN-region priors are also low (Western Asia 0.19, Northern Africa 0.22, Central Asia 0.22). Blending a ~0.18 prediction toward a ~0.20 prior cannot push it across 0.5. The shrinkage has nothing to shrink toward that would change a single classification. The mechanism is sound; it just cannot engage on this failure mode.

We also ruled out the obvious variants

Before calling this a dead end we tested three reasonable reformulations of the same idea, each under the identical honest LOCO protocol (scripts/diagnose-pooling-variants.py):

• V1 — positive-count weighting: use w = n_pos / (n_pos + k) so the effective sample size is the positive count (2-22), making the tail genuinely "data-poor" and the shrinkage actually engage.
• V2 — region-floor blend: p_final = max(p_raw, blend) — shrinkage may only raise a prediction, never lower it, so it can lift a borderline tail-country day across the threshold without dragging data-rich countries down.
• V3 — decoupled threshold: the V1 blend, then a per-country F1-optimal decision threshold instead of a hard 0.5, separating the shrinkage question from the threshold question.

V1 is the variant most likely to rescue the idea — switching the effective sample size to the positive count genuinely makes the tail countries "data-poor" (n_pos 2-22) so the shrinkage finally engages. It does engage, and it still fails. Under the identical honest LOCO protocol, V1's best operating point lifts the tail count from 1/16 to 4/16 — better, but nowhere near the gate's 10/16 — while LOCO mean F1 falls to 0.657 (k=4) and 0.531 (k=12), well below the 0.75 bar and the 0.711 baseline. The same monotone-in-k decay reappears: making the tail engage means the genuinely data-rich countries get over-shrunk too, and they outnumber the tail. V2's region-floor and V3's decoupled threshold are refinements layered on V1's blend; neither changes the underlying fact that V1's mechanism moves only 4 of 16 tail countries. No variant clears the gate. The full sweep is reproducible from scripts/diagnose-pooling-variants.py; machine-readable results at /opt/voidly-ai/models/experimental/pooling_variants_diagnostic.json.

Decision

The partial-pooling layer is NOT promoted. Classifier v3.3 stays in production unchanged. The pooled model bundle and full metrics are kept on the ML server for reproducibility (/opt/voidly-ai/models/censorship_classifier_pooled_v1.pkl and …_metrics.json), but no serving endpoint exposes it. /v1/classifier/score and /v1/classifier/info still report v3.3.

What would actually move the tail

This is the third Atlas experiment to converge on the same root cause. v3.4's per-regime-cluster fine-tuning held (its stack head learned to ignore the cluster heads); the forecast-contagion port held (Iran regressed 27pp). And now empirical-Bayes pooling holds. The pattern is unambiguous: no post-hoc architecture fixes a positive-label shortage. When YE has 2 confirmed-censorship days and KW has 2, F1 is a coin flip that no shrinkage, no cluster head, and no ensemble can stabilize.

The honest path forward is data, not modeling:

1. Targeted positive labeling — hand-curate 20-30 additional confirmed-censorship days for OM, LY, YE, KW, MA, TN from the existing 84K-row evidence DB. More signal beats more architecture.
2. Drop the per-country F1 framing for n_pos < 10 countries — with 2-8 positives, a ±15pp F1 swing between two models is statistically meaningless. These countries should be reported with confidence intervals or pooled into regional aggregates, not scored individually.
3. Report LOCO median alongside mean, permanently — the median (0.870) is the honest summary of v3.3's per-country skill; the mean (0.711) is mostly a measure of how many positive-starved countries are in the evaluation set.

Negative results count. Publishing a no-promote next to the promoted experiments is what keeps the methodology honest — the reader sees what we tried, what the gate demanded, and exactly why this idea, sound as it is in general, could not engage on this particular failure mode.

Reproducibility

Training script: scripts/train-hierarchical-classifier.py. Feature-enrichment pass: scripts/build-partial-pooling-features.py. Variant diagnostic: scripts/diagnose-pooling-variants.py. All run on the Vultr ML server as the service user; deterministic (seed 42). Metrics at /opt/voidly-ai/models/censorship_classifier_pooled_v1_metrics.json; variant report at /opt/voidly-ai/models/experimental/pooling_variants_diagnostic.json. Baseline v3.3: LOCO mean F1 0.7109, median 0.8696, 127 countries evaluated, 4,237 samples / 1,116 positive / 131 countries.