AgentGB

What gets recorded, and what doesn't

Emulator instances step through real button presses against a real ROM, headless, and nothing about a run is staged afterwards. What a swarm film draws comes from one line an agent writes every decision — map, position, facing, which link, in battle or not — the instant that decision happens. A film doesn't re-simulate anything and doesn't stand a screenshot in for a run that didn't happen: the renderer draws characters at exactly the positions already on file, and that file is the output of a real playthrough, not the input to a synthetic one.

The score works the same way. Success is a byte the cartridge itself writes when a goal is actually claimed, read back out of the machine at that moment, with host-enforced no-memory-writes standing between the policy and any byte it might otherwise be tempted to fake.

The trick, and the honest version of it

None of this is affordable at real time. Every episode starts by loading a save state — about 142 KB, the whole state of the machine — which is free: deserialised and running again in under a millisecond. Headless, with the picture turned off for the paths that don't need it, one instance steps at roughly 70 times real time; fanned out across sixteen at once on one box, that measures at 31,485 frames a second. Every one of those instances is provably deterministic — this project's own emulator produces byte-identical framebuffers and save-state snapshots across four processor architectures, with a recorded match on a PSP build besides.

Save-state scripting itself is not new, and it isn't the claim here. BizHawk and PyBoy both drive an emulator this way, and PyBoy is the established Python Game Boy emulator for exactly this kind of work — it's what a 50,000-hour reinforcement learning run against this same game used, and PyBoy's own documentation states a real speed ceiling above 5×. What's specific to this project isn't the trick. It's the stack underneath it: an emulator built and pinned to a specific commit and hash rather than borrowed, a memory atlas of the cartridge as ground truth instead of a guess, and a self-test cartridge built to catch the emulator lying about hardware behaviour before any agent is trained against it.

The twenty-one links

This is the actual decomposition, not a description of one — each entry below is a single cartridge condition, checked, nothing more:

leave-the-bedroom → leave-the-house → trigger-oak-in-the-grass → follow-oak-to-the-lab → take-a-starter → battle-the-rival → out-of-the-lab → north-out-of-pallet → cross-route-1 → into-the-viridian-mart → collect-oaks-parcel → out-of-the-mart → south-out-of-viridian → back-down-route-1 → into-oaks-lab → receive-the-pokedex → out-of-the-lab-again → north-out-of-pallet-again → cross-route-1-again → into-the-viridian-mart-again → buy-pokeballs

The last three links are new since the chain last certified at eighteen: cross Route 1 a second time, back into the Viridian mart, buy the balls. The chain ends there — it does not yet reach Pewter, and it has not fought Brock.

Staying alive: not getting stuck, and healing when it doesn't need to ask

A chain scoring 594 of 600 looks like several small problems spread thin. It wasn't. Three separate 600-run sweeps produced fourteen failures between them, and every one ended the same way: twelve identical actions in a row, the same button, until the run ran out of decisions. The cause was the confidence gate itself — above 0.80 confidence the policy takes the argmax rather than sampling, and a no-op on a screen that can't change produces the exact same input on the next decision, so the same output, forever. Confidence in the wrong button doesn't wobble once it's locked; nothing about a static screen can unlock it.

The fix doesn't ban the action. It subtracts a flat amount from the repeated action's logit every time it fires without changing the frame — nothing else on the board moves. Subtracting D from a logit divides that action's odds against every alternative by e^-D per repeat: exponential decay of belief, the exact mirror of an exponential backoff, arrived at from the opposite direction. Enough repeats and the action that looked certain stops being the argmax, sampling reopens, and the agent moves on. Result: 594 of 600 to 600 of 600, and the stuck_in_battle failure mode specifically went from 4 occurrences to 0.

The same day, the same underlying machinery — noticing a condition without being told to check it constantly — became chambered goals: two conditions decoupled in time. An arm condition read from cartridge memory (the agent has taken damage) and a fire condition read from the screen, checked only once armed (a Pokémon Center is in view). Take damage, the chamber loads; later, a Center comes into view, it fires — heal, walk out, resume whatever the chain was already doing. A blackout heals the party too, so it discharges the chamber with no Center visit at all; satisfaction can arrive from any direction. Because the fire recogniser is never consulted while unarmed, it's the least-exposed recogniser in the whole system — arming is a safety property, not just a trigger. Across the current certification, 306 of 600 runs take that detour mid-chain, at zero cost to the chain itself.

How confident the fire recogniser must be isn't fixed either — it's a gradient against how much the arm condition hurt. Barely scratched, it has to be almost certain a building is a Center. Badly hurt, a glimpse is enough. The floor was found by measurement, one value at a time:

0.92 is the measured last safe rung, one notch of margin above where it breaks. The first version of this gate was tuned looser than that and fired 13,673 times over the course of testing it, trapping five runs standing at a door; gated at 0.92 the same recogniser fires 591 times.

The full story — three 600-run sweeps of the same bug, the fix, and the honest bits that came with building it — is its own post: exponential decay of belief →

The best idea in this project

Viridian City and Route 1 are each walked twice in this chain — once out, to fetch Oak's parcel, once back, to deliver it — and to a screen-only policy the two crossings look identical. A single student trained over both directions split 33.53% / 66.77% between them: two numbers that sum to a hundred, which is what a partition looks like, not a policy doing something badly.

The first fix worked. It taught the agent to open its own bag mid-route and read whether the parcel was in it — a genuine seventh button, a real recogniser, certified at 299 of 300. It was thrown out anyway, on the captain's own correction: the game already says the answer out loud. The instant the parcel is collected, Gen 1 draws the fact across the screen in plain text.

A recogniser trained on that one screen latches a stage change — a single integer, advanced once and never re-checked — instead of answering the same ambiguous question every decision for the rest of the run. Both doubling-back rooms now certify at 100.00%, against random floors under 4%.

One frozen brain, and a nudge that starts at zero

Every fix to this project since the star pupil was committed is a small, separate network bolted onto a frozen base — its own convolution-and-dense stack, sharing no weights with the policy underneath it. It never replaces the base's opinion about which button to press; it adds a nudge to the base's own pre-softmax scores before one is chosen. Goals themselves work the same way structurally: each is its own small network of the same shape, two-class, restricted to a declared screen region, with its own confidence threshold — not one head bolted onto the trunk trying to answer every question at once.

The nudge starts at exactly zero. Its last layer is initialised to a zero weight matrix and a zero bias, which return zero for any input regardless of what the layers before them computed — so on the day it's built, before a single training step has run, the base's own answer passes through unchanged. Training only ever pulls the adapter away from zero along directions its own small corpus actually visits; a situation it has never seen stays at zero, not at some arbitrary learned guess. The base network's weights are hashed before and after every adapter is trained, and the hash is required to match.

That is why a new goal — a nickname prompt, a trainer battle, the return leg of a route — costs a few hundred training episodes and a frozen-base guarantee, rather than a retrain of the hundred-thousand-parameter network everything else depends on.

The design, in one paragraph

Reinforcement learning asks one network to learn two jobs from one scalar reward: where to go, and what am I looking at. AgentGB splits them. Where to go is a search problem, and search is solved — a shortest path over a walkable graph does it. What am I looking at is a perception problem, which is what networks are good for. The student's whole job is to see, which is why it is a hundred thousand parameters and not millions.

The price is that the teacher half is allowed what the student never is: the cartridge's memory and a map walked in advance. An end-to-end agent needs neither. The split is not novel — the PokéAgent Challenge's winning entry decomposes a route into subgoals and distils the result into a network, and the third-place entry paired deterministic pathfinding with a language model. Two of the top three cut the same seam.

The engine that quietly changed

An earlier 18-link read came back at 69 of 100, and the milestone breakdown pointed straight at the return leg: every one of the 31 failures happened on the second Route 1 crossing or later, split across a battle-menu trap (28 stuck in battle mode) and a known no-op loop at two exit tiles (3 stuck in the overworld) — both failure shapes already on this project's own record, nothing new.

The student hadn't gotten worse. The picture it was being judged on had changed. One commit to this project's emulator, folded into the pinned build without anyone noticing, flipped the default render mode from the cheap engine everything here was trained and measured under to the hardware-accurate one — visually different on exactly the battle-transition frames this project's screen recognisers key on, with the underlying game state (position, mode, RNG) untouched. Nobody had told the emulator which engine to use, so it silently inherited whichever one the binary happened to be compiled with.

Same weights, same seeds, same code — nothing here was retrained or retuned. The fix is a pin: which render engine an evaluation uses is now recorded and enforced alongside the emulator's own commit and the ROM's own hash, defaulted to the cheap engine everything was actually built against, with an explicit override still available for anyone deliberately re-checking the hardware-accurate path. A guard now asserts that default on every test run and fails loudly if a future change ever moves it again without saying so.

Writing

Not gameplay: three thousand copies of one character, walked into a Pokéball shape by a separate choreography stage with no cartridge and no trained policy anywhere near it. The camera pulls back over one unbroken shot; the crowd is still moving when it ends.

The whole arc in order, with the number that was true at each point, is on the progress page.