A REPL agent in its smallest form: one model call writes the player

One call to a local model, 518 seconds and 5,290 tokens of reasoning, produced a 44 line Python function. That function then played an ARC-AGI-3 game at 2,880 actions a second, 60,000 actions in 21 seconds, three times over. It cleared nothing. It died 454 times on every run, at the same 454 moments, because it is deterministic and the game is too.

That is the smallest possible version of the idea that won the first ARC-AGI-3 milestone, a REPL agent where the model writes the player rather than being the player, and the smallest version is where the idea’s real requirement shows.

What a REPL agent is, and what the winning one did

The first milestone’s winning entry, per ARC Prize’s writeup, “runs Qwen 3.6 27B FP8 locally” and plays by writing and running Python in a live console, turning the board into variables and code rather than asking the model for keystrokes. It keeps playing past its context limit by evicting its oldest messages, which the writeup calls “infinite play via eviction”, and its authors reported that hand crafted tools made the model worse and letting it improvise made it better.

The design has one property that matters more than the rest: the model sees the results of its own code and revises. It is a loop of write, run, observe, rewrite, at whatever cadence the model can afford, and the console is fast enough that the observe step is nearly free.

The version tested here has the write and the run, and none of the revise. That was deliberate. It is the cheapest thing that can be called a REPL agent, and its failure is instructive.

How one model call became a 44 line policy

The model, gemma-4-31b-qat through LM Studio, received a description of the interface rather than the game: a 64 by 64 integer board, four legal actions, a history list of what each past action did, and the colour summary of the opening frame. It was asked for a Python function policy(board, history) returning one action, using only numpy, exploring systematically and avoiding moves that changed nothing.

The call took 518.0 seconds and 5,290 tokens of reasoning, and returned a clean code block. The function it wrote keeps moving in the direction that last changed the board, rotates to the next direction when a move changes nothing, and checks its last four moves for an up down up down oscillation and breaks it. It is a reasonable first policy for a maze. It is what a person would write in the first minute, before playing.

Loaded into the 40 line loop in place of the random policy, it ran without a single exception across 185,000 actions.

What the policy did at 2,880 actions a second

The ARC-AGI-3 engine does not care who is pressing the keys, and the policy pressed them at about 2,880 a second, slower than random play’s four thousand, most likely because of the numpy comparison it does every step to see whether the board changed. Sixty thousand actions took 20.8 seconds.

The three identical rows are the point. The policy has no randomness in it and the game has none either, so a deterministic policy on a deterministic game replays the same 60,000 actions and the same 454 deaths every time. The seed governs only a random fallback for illegal or crashing moves, and had it fired the three rows would have diverged; they did not.

Random play, which has no idea what it is doing, found level one in two of three runs. The policy, which has an idea, found it in none, because its idea was wrong for this game and nothing in the loop could tell it so.

Why the smallest REPL agent does worse than random

Randomness is a form of search. A random agent will eventually try the sequence the level wants, at a cost the scoring rightly treats as worthless. A deterministic policy tries one sequence forever. If the sequence is wrong, more actions do not help, and the 454 deaths per run are the same 454 deaths every time.

What the winning design adds is the thing left out here. After the policy dies for the tenth time in the same place, the model gets to see that and write a different policy. The console makes that observation cheap, the eviction trick keeps the conversation alive long enough to do it many times, and the model’s reasoning is spent on the revision, which is a question it is good at, rather than on the keystroke, which it is slow and unreliable at.

One model call bought a policy that ran 185,000 actions for free. A second call, shown the recording of the first, would have bought a better one. The first call is the whole cost of the design; the loop around it is the whole value.

What it would take to make this a real REPL agent

Three additions, each of which this run’s numbers make concrete.

Feed the results back. After each 60,000 action run, or sooner, hand the model the death count, the positions where it died, and a summary of the boards it saw, and ask for a revised policy. The recording has all of that, frameless, at 322 bytes an action.

Give it the console rather than a single function. The technical report describes a harness of this kind as one that lets the model “execute arbitrary Python code to selectively retrieve and transform information from its action history”, so it can test a hypothesis about a mechanic before committing to a policy. A function that returns one action cannot ask a question; a console can.

Manage the context. Every revision adds the previous code and its results to the conversation, and on a 64K context that fills in a handful of rounds. Eviction, or a summary the model maintains itself, is what turns a few rounds into as many as the game needs.

None of those change the economics that make the design worth building. On this machine a model call is minutes and an engine step is a third of a millisecond, and a REPL agent is the arrangement that spends the minutes on deciding what to try and the milliseconds on trying it. The smallest version proved the arrangement works and the policy does not, in that order, and only the second of those is fixable by writing more code.