Three independent Atlas experiments this month converged on the same diagnosis of classifier v3.3's weak tail. v3.3 — the production GradientBoosting model — has a split personality: leave-country-out (LOCO) median F1 of 0.870, but a mean of only 0.711, dragged down by ~16 MENA / former-Soviet countries (OM, UZ, TN, LY, YE, JO, MA and similar) scoring LOCO F1 between 0.00 and 0.36. The empirical-Bayes partial-pooling finding pinned the cause: those countries are not row-poor (53–85 labeled country-days each) but positive-poor — 1–33 confirmed-censorship days apiece. Its closing line was a prescription: “no post-hoc architecture fixes a positive-label shortage — the honest path for the tail is targeted data labeling, not modeling.”

This finding is that prescription, executed and measured honestly. We mined 155 new high-confidence positive labels for the tail, retrained a v3.4 candidate on the expanded corpus, and ran the identical LOCO protocol. It failed the promote gate. More data — even genuine, multi-source-corroborated data — did not lift the tail in aggregate. v3.3 stays in production unchanged. The negative result is itself the finding, and it sharpens the diagnosis.

The labeling: 155 positives, a deliberately strict evidence bar

v3.3's corpus labels a country-day positive only when an incidents-table row has first_seen on that day. A genuine, heavily-blocked censorship day that the incidents pipeline simply never minted a row for is labeled 0. Across the 16 tail countries the corpus held 654 such currently-negative days. We recovered positives from them under a strict, documented threshold — every new positive has real corroborating evidence; no label was flipped to inflate a count; no 1→0 flip was ever made. A day is promoted to positive only if it clears a universal bar and one of three corroboration paths:

The bar is honest about what it cannot find. Of the 654 negative days examined, 246 fell below the 0.60 rate and 226 had no corroboration path — many tail countries (OM, LY, YE, TN) have plenty of 100%-blocking days, but they are OONI-single-source at only normal/warning level, not corroborated and not OONI-critical. Four countries — TN, YE, AM, GE — yielded zero new positives because the corroborating evidence genuinely does not exist at a defensible confidence bar. That is not a bug; it is the dataset honestly reporting its own limit.

The retrain failed the promote gate — decisively

v3.4 is a pure data experiment: identical 16-feature GradientBoosting architecture, identical hyperparameters, identical balanced sample weighting — the only change is the corpus (1,116→1,271 positives). The v3.3 baseline was re-run with the exact same code path, not read from a frozen file, so any library drift cancels: the re-run reproduced the published v3.3 numbers to six decimals (LOCO mean 0.710936, median 0.869565). The gate required all three conditions; v3.4 cleared one:

The third gate passing while the first two fail is the whole story. The tail did partly respond to its new labels — 8 of the 12 mined countries improved, several substantially: JO +0.31, LY +0.22, AZ +0.12, OM +0.11, AM +0.13, UZ +0.09, KZ +0.07. But that local gain was more than paid for elsewhere.

Why it failed: the new positives inflate the decision boundary

The mechanism is not subtle, and it is worth stating plainly because it is a trap for anyone who assumes “more correct labels is always better.” The model uses balanced class weights. Adding 155 positives (a 14% increase) raises the positive class's total weight, and because the new positives all sit in the high-anomaly_rate region (0.60 to 1.0), they teach the model that an elevated blocking rate is more often censorship. That is true for the tail. It is not true for a data-rich, normally-clean country that happened to log a high-measurement-noise day. Under leave-country-out the model never sees country identity — it cannot tell “UZ at anomaly_rate 0.7” (real censorship) apart from “Spain at 0.7” (measurement noise on a clean day) by anything except the 16 shared features, and in that feature space the two overlap. So the v3.4 boundary slides down, and clean head-country days cross it.

Measured directly, on TRUE-NEGATIVE country-days the model's mean predicted probability rose sharply from v3.3 to v3.4:

The count of countries scoring a perfect LOCO F1 fell from 46 (under v3.3) to 41 (under v3.4). Eleven non-tail countries regressed ≥5pp — seven of them substantively (IR −0.09, IN −0.09, ID −0.07, SA −0.09, ES, KH, CA), four small-sample noise (SC, TG, CR, GA, each ≤8 rows). And the median F1 dropping 0.870→0.800 reflects exactly this: the typical country — a cleanly-classified head country — got slightly worse so that a handful of tail countries could get better.

The two worst tail regressions sharpen the point further. TN regressed 0.333→0.000 having received zero new positives — its 4-positive evaluation was simply destabilised by the global boundary shift. BH regressed 0.500→0.133 despite gaining 4 positives, because its single original positive no longer dominates a now-noisier decision region. Neither regression is caused by a bad label; both are caused by retraining the whole model to chase the tail.

The honest conclusion

Targeted labeling was the right thing to try — the partial-pooling finding was correct that no post-hoc layer fixes a positive shortage, and this finding confirms the diagnosis was about data. But the deeper lesson is sharper than “label more”: the tail's problem is not the count of positives, it is that the tail's censorship signal is not feature-separable from head-country negatives in the 16-feature space the model is allowed to see. Pouring genuine, well-corroborated tail positives into a single global model with a single global decision threshold cannot win — it can only trade head accuracy for tail accuracy. The four countries that yielded zero new positives (TN, YE, AM, GE) also tell their own honest story: for some countries the corroborating evidence to label them well does not yet exist, full stop.

v3.4 is not promoted. Classifier v3.3 remains in production unchanged; no serving endpoint exposes the v3.4 model. The expanded corpus (labeled_incidents_v3.4.json, 1,271 positives), the mining plan with its per-day evidence trail, and the full LOCO comparison are kept on disk — they are a clean, honestly-labeled asset for the next attempt, which should be a structurally different one: per-country or per-region decision thresholds, or a tail-specialised model scored separately, rather than a single global model retrained to chase a signal it cannot isolate. This is the fourth Atlas experiment to land on the same wall from a new direction. The wall is real, and naming it precisely — the tail is not separable under LOCO with these features — is the contribution.