devlog.

Pokewalker · Performance

The second core was doing nothing

The walker's chip has two cores. One of them drew the picture, encoded it, sent it to the screen and read the touch panel, one job after another. The other played the speaker and spent nine tenths of its time waiting. Now they share the frame.

13.8 → 39.7
Walking face, frames a second
18.8 → 2.7 ms
Encoding a frame
0
Pixels changed

The walker is an ESP32-S3 behind a round 412×412 screen. The software draws a 103×103 picture and the board scales it up four times, so every dot is a crisp 4×4 block. Until today that ran at about 13 frames a second on the walking face and 7 or 8 in the open world.

It now runs at 39.7 on the walking face, 36.5 with the closed ball spinning, and 14 to 19 across the open world. The drawing code did not get faster. The time went to the second core.

Frames a second on the board, before and after, for eight screens BEFORE AFTER 0 10 20 30 40 frames a second Closed ball, spinning: 13.2 fps before, 36.5 after (2.8x) Closed ball, spinning 13.2 36.5 Walking face: 13.8 fps before, 39.7 after (2.9x) Walking face 13.8 39.7 Quick battle: 12.1 fps before, 24.3 after (2.0x) Quick battle 12.1 24.3 Park plaza: 8.1 fps before, 15.1 after (1.9x) Park plaza 8.1 15.1 Gate road, standing: 7.5 fps before, 15.7 after (2.1x) Gate road, standing 7.5 15.7 Gate road, walking: 6.7 fps before, 14.1 after (2.1x) Gate road, walking 6.7 14.1 Cerulean, standing: 7.3 fps before, 15.4 after (2.1x) Cerulean, standing 7.3 15.4 Cerulean, walking: 8.7 fps before, 19.0 after (2.2x) Cerulean, walking 8.7 19.0
Frames a second on the board, three runs each. The gate road and Cerulean are two places in the open region around the park, measured standing still and walking.

Where a frame went

The first change was a line of numbers. Every few seconds the board now prints how long each part of the frame took. On the closed ball it read: 23.2 ms drawing, 18.8 ms encoding the picture into the panel's colour format, 19.2 ms sending it, and 11.3 ms reading the touch panel. All on core 0, in a row.

Core 1 ran the speaker and nothing else. It was idle about 90% of the time.

One frame on the closed ball, before and after: four jobs in a row on core 0, then core 0 drawing while core 1 sends BEFORE — 13.2 FRAMES A SECOND core 0 draw 23.2 ms encode 18.8 ms send 19.2 ms touch 11.3 ms core 1 speaker only, idle about 90% of the time AFTER — 36.5 FRAMES A SECOND core 0 draw 23.6 ms draw 23.6 ms draw 23.6 ms core 1 send 15.7 ms send 15.7 ms encode 2.7 ms touch and motion sensor read here, during the draw 0 10 20 30 40 50 60 70 80 milliseconds · a vertical line starts the next frame
One frame on the closed ball, to scale. Before, core 0 does four jobs in a row. After, it only draws, and core 1 encodes and sends the last frame while the next is drawn.

One table instead of a lookup per pixel

The canvas stores each pixel as one of 256 shades. Turning a shade into the bytes the panel wants meant working out its colour every time, for every pixel. But within one frame a shade is always the same colour.

So once a frame the walker builds a table: all 256 shades, each already encoded as the four panel dots it becomes. A row of the picture is then a run of table lookups. Encoding fell from 18.8 ms to 2.5 ms on that change alone.

Sending from the other core

This was the biggest win. When core 0 finishes a picture, it copies the canvas into a snapshot in PSRAM and posts it to core 1 through a one-slot mailbox. Then it goes straight on to the next frame. Core 1 encodes the snapshot band by band and puts it on the wire.

Neither side takes a lock. Core 0 only writes the mailbox when it is empty, and core 1 only reads it when it is full. Core 1 never prints either, so when a save to flash parks it mid-frame, the frame is late and nothing else happens.

The bytes on the wire are the same as before: the same windows, the same transactions, in the same order. Only the core sending them changed.

Then the band buffers were doubled. Each band now goes out as two halves from two pairs of DMA buffers, so the next half is encoded while the last one is on the wire.

There is a floor here. The panel's bus runs at 40 MHz on four lanes, the ceiling its datasheet gives for it, and a full 412×412 frame takes about 16 ms to cross it. The send is now 15.7 ms. That is the bus, not the code.

Sharing the ground

The open world spends most of its frame on the ground: the grass, paths and water drawn row by row in perspective. Those rows do not depend on each other, so the two cores can split them.

Both cores take the next row from one shared counter until none are left. Each row is written into a buffer, and when every row is done, core 0 copies them into the canvas in the same order, left to right and top to bottom, that the one-core drawing uses. The picture cannot change, only when it is ready.

With this and the changes above, the park plaza went from 8.1 to 15.1 frames a second. The battle arena's floor is drawn the same way now, for a smaller gain.

The one that went backwards

The spinning closed ball got the same treatment and came out slower on the board. Core 1 is busy sending the previous frame for most of the time the ball is being drawn, so handing it rows only made core 0 wait. It was taken back out.

Reading the sensors in the gaps

The touch panel and the accelerometer share one I2C bus. They used to be read between bands of the panel transfer, and the transfer had moved to core 1. Leaving the reads on core 0 meant none during a long draw, and the open world's draw is 50 ms and more. The step counter wants a sample at least 25 times a second.

So while core 0 draws, it lends core 1 the bus. Core 1 reads both sensors on the usual cadence, between its own bands and rows, and keeps each reading with the time it was taken. When the picture is done, core 0 takes the bus back and replays the readings through exactly the code it would have run had it read them itself, at those times.

Touch no longer costs core 0 anything: 11.3 ms became 0. The accelerometer is now read at about 65 times a second everywhere, up from 30 to 38 in the park.

Nothing on the screen changed

Every golden frame the test suite keeps is unchanged, and so is the hash of the whole pixel battle. Tests that draw the same pictures with the work split across two threads get identical frames. The speaker ran through all of it with no underruns.

Those checks read the canvas, the picture as the software holds it. Nobody has yet looked at the physical panel after this change, so the last step, from the bytes on the wire to the glass, is still to be seen.

What is left

  • The park's standing objects still take about 27 ms, all on core 0.
  • The walking face and the closed ball are now bound by core 0's own draw, about 23 ms.
  • The S3's vector instructions have not been tried on any of it.

The battles this speeds up are the ones in the stadium post, and the rest of the device is on the project page.


Every figure is from the real board, three runs a screen. The frames compared are the software's own canvas; the panel itself has not yet been checked by eye since the change.

← Back to devlog