Every failure was the same starter

The number was fine. That was the problem.

Twenty-three of twenty-six agents completed every milestone from a cold boot: out of the bedroom, up Route 1, into the Mart, back down again, and the Pokédex in hand. The confidence interval was reported, the failures were located to a tile, and the three that stopped were noted as stopping in Oak's Lab and on Route 1.

Everything about that report was true. Nothing about it said that one entire third of the population failed every single time it was drawn.

The reason it was caught at all is that the captain watched the film rather than reading the table. Twenty-six agents on screen at once, each in its own cell, each playing its own run — and the eye does something a summary statistic does not. It groups. Three agents stuck in the same room, and all three had the same Pokémon.

One in two thousand six hundred

"All three failures were the same thing" is exactly the sort of pattern a human finds in noise, so: with 21 Bulbasaur, 2 Charmander and 3 Squirtle runs and no species effect at all, the probability that the three failures land on the three Squirtle runs is about 1 in 2,600. p ≈ 0.00038.

That is not a coincidence you shrug at. And it was the second time this exact shape of bug had appeared in this project, which is what turned it from an incident into a rule.

Why the film sees what the table cannot

Not because a film is nicer to look at. Because of what each representation can carry.

A completion table has one row per milestone and one rate per row. It is a marginal — it has already summed over the population, and a correlation between a run's hidden attribute and its outcome is exactly the information summing destroys. You cannot recover it from the table no matter how carefully you read, because it is not in the table.

The film has not summed over anything yet. Every agent is still a separate object on screen, carrying everything about itself, and the failures are visible as a group — three cells stuck in the same room at the same time is a thing you see rather than compute.

It is not free. The film that showed this takes roughly twenty-four screenshots per decision per agent — gigabytes — where the table costs nothing but the emulation. It is the expensive artefact, and this is what it is for.

Three separate causes, none of them the one first suspected

The obvious hypothesis was that the training corpus was biased toward one starter, so the student was simply worse with the others. That was checked, and it is not what was happening.

The starter choice is genuinely balanced. At the chain's own entry state the first decision carries 0.992 bits of entropy, and over 500 sampled attempts the committed student takes Bulbasaur 254 times, Squirtle 127 and Charmander 118, reaching the goal in 499 of 500. That is a real three-way split, not a collapse.

What was true, and actionable, is narrower and less obvious: two of the links on the return leg have exactly one recorded start state each, so their corpora contained zero species variation — regardless of how balanced the entry choice was. The student had simply never been shown those rooms with a Squirtle in the party.

Pulling the three failures apart gave three unrelated causes:

The fix that broke everything, and what it taught

The first wiring of the Oak-interaction goal took the whole chain from 88.5% to 0 of 150.

The trigger tile it had been trained on — one specific tile, facing one specific direction — is, independently, the chain's own entry tile for taking a starter six links earlier. Same physical room, visited three times by this chain for three different reasons. An ungated goal that fires whenever it recognises its screen fired on the very first approach to the Poké Balls and broke every single run.

The fix was structural rather than a heuristic: the late-chain goals are now only ever consulted once a return-leg stage has already latched. Not by convention — by construction. A screen that a return-leg goal happens to recognise can no longer hijack the outward leg, however visually identical it is to something further down the chain.

There is a general shape here worth naming. In a route that revisits the same rooms, a recogniser's precision is not a property of the recogniser. It is a property of the recogniser and the window of the episode it is allowed to look in. Ours was excellent at its job and catastrophic without a gate.

Where it landed

After the three fixes, an eight-agent smoke test first — 8/8 — before committing to the long run. Then a cold-boot certification at N=150:

No starter anywhere near zero, all three intervals overlapping heavily. The filmed swarm at N=26 came back 26/26 — the first time this chain's film has shown every starter completing at 100%, where the previous one showed a third of them at zero.

N=150 is not the project's 3,000-attempt convention, and bounds failure near 2%, not 0.1%. And Charmander's 13/13 has a lower interval bound of 77.2%: thirteen runs is evidence of not-obviously-broken, not of perfection.

What this actually generalises to

Very little of the above is about Pokémon.

An agent population with a self-selected hidden attribute — which tool it picked, which branch it took, which variant it was served — will produce an aggregate success rate that is arithmetically correct and behaviourally meaningless the moment that attribute correlates with outcome. The failure mode is not a wrong number. It is a right number answering a question nobody asked.

Two cheap defences came out of this. Break every rate down by the attribute, whether or not you expect it to matter. And keep one artefact in the pipeline where the population has not been collapsed yet — expensive, rarely looked at, and the only place a structural split is visible before it becomes a mystery.