The ARC-AGI-3 toolkit asked for nothing. No account, no card, no key. It printed Got anonymous API key, pulled a list of 25 environments, and wrote the first game to disk as 2,060 lines of readable Python. Downloading that one game took 1.2 seconds on a studio test machine.
Then the local model got its turn. It took 191.6 seconds and produced no move at all.
That gap is the whole story of this benchmark right now. ARC-AGI-3 launched on 25 March 2026 with humans clearing 100% of its games and the best frontier system scoring 0.51%. On 3 September, five months later, OpenAI’s GPT-6 Astra posted 99.9%. Underneath those two headline numbers sits a set of much smaller ones that nobody put in a press release, and they point somewhere useful for anyone running models on their own hardware.
How to run ARC-AGI-3 locally without an API key
The toolkit is a normal pip install. uv pip install arc-agi pulled arc-agi 0.9.9 and arcengine 0.9.3, both MIT licensed, both requiring Python 3.12 or newer. Neither needs a GPU, because the games are plain Python.
Six lines of code start a game:
“`python
import arc_agi
from arcengine import GameAction
arc = arc_agi.Arcade()
env = arc.make(“ls20”)
obs = env.step(GameAction.RESET)
print(obs.available_actions)
“`
The first call printed Got anonymous API key, fetched 25 environments, and downloaded ls20 in about one second. What lands on disk is the game itself, not a client stub: environment_files/ls20/9607627b/ls20.py, 2,060 lines of Python, with the whole game directory weighing 108 KB. Every step after that runs on your own machine.
Those 25 are the public set. The technical report counts 135 environments in total, 25 public, 55 semi private and 55 fully private, each with at least six levels, and every frame a 64×64 grid of 16 colours. The interface is deliberately small: five keys, an undo, and the ability to point at one cell. On ls20 the engine only ever offered four of them. The private sets are what the leaderboard scores. The public 25 are what you can take apart at home.
Two things cost time before the first useful move. The action enum does not accept its own values: GameAction(1) raises ValueError: 1 is not a valid GameAction even though GameAction.ACTION1.value is 1. The members are declared as tuples, (1, SimpleAction), and the class rewrites _value_ in its own __init__, so Python’s lookup table never learns the integer. The engine ships the workaround: GameAction.from_id(1). The second is worse, because it fails silently. A game must be sent RESET before it starts, and again after every GAME_OVER. A loop that skips the second reset keeps stepping a finished game, which is why an early random run here reported 352,000 actions per second and zero levels cleared. It was not playing. It was pressing buttons at a dead screen.
What RHAE measures and why random play scores zero
ARC-AGI-3 does not score you on whether you finished. It scores you on how many moves you needed compared to a person who had never seen the game either. The metric is called Relative Human Action Efficiency, and the published methodology gives the per level formula as level_score = (human_baseline_actions / ai_actions) ^ 2, capped once an agent gets within 1.15x of the human. Level scores are then averaged with the level number as the weight, so level 6 counts six times as much as level 1.
Squaring is the part that matters. Take twice as many moves as the human and you keep a quarter of the score, not half. Take ten times as many and you have nothing left.
The human number is not hidden behind an API. It ships in the game’s own metadata.json, as a list called baseline_actions. For ls20 it reads [22, 123, 73, 84, 96, 192, 186], seven levels, and the first one takes a first-time human 22 moves.
Random play was measured against that on the studio test stack, three seeds, a 60,000 action budget each, resetting after every death. Seed one cleared level 1 after 40,699 actions. Seed two took 23,136. Seed three never cleared it at all, and died 458 times trying. Put 40,699 into the formula against a baseline of 22 and the level score comes out at 0.0000003. Not low. Gone.
That is the design working. A benchmark that only asked “did you finish” would be beaten by a fast loop and a large budget, and we have measured before how easily a benchmark number can come from the harness rather than the model.
What an ARC-AGI-3 local model costs per move on a Mac
The obvious next step is to put a local model behind the same loop. The studio test stack is an M4 Pro with 48 GB of unified memory, serving gemma-4-31b-qat through LM Studio on localhost:1234 at 64K context. That is not an arbitrary choice: it is the same model that took second and third place in the first ARC-AGI-3 milestone.
The naive harness is the one everyone writes first. Serialise the frame as 64 rows of hex digits, list the legal actions, ask for one action name back. One frame arrives as 4,221 prompt tokens.
It did not answer for most of an afternoon.
The first call was capped at 16 output tokens and came back empty, because 13 of those 16 went to reasoning before a single visible character appeared. Raising the cap to 256 bought nothing: 191.6 seconds for turn one, 225.9 for turn two, both truncated. At 1,024, two more turns at 157.4 and 94.2 seconds, and in each one 1,021 of the 1,024 tokens went to reasoning while the visible reply was an empty string. At 4,096, four more failures at roughly six and a half minutes each.
The cap was the whole problem, and it was closer than it looked. At 16,384 the same board on the same model returned ACTION1 in 442.7 seconds, having spent 4,274 tokens on reasoning. It had been cut off 178 tokens short, over and over. That failure has its own writeup, because the numbers it produces on the way down are actively misleading: why LM Studio returns an empty response.
Now set the working number against the human one. A first-time player finishes ls20 level 1 in 22 moves. At 442.7 seconds a move, this stack would need two and three quarter hours to spend 22 moves on the first of seven levels, with no reason to think they would be the right 22. Downsampling the same frame to 16×16 brings a move down to 191.9 seconds, which is still seventy minutes for that one level.
The gap is not the weights. Both milestone entries that used this exact model got real scores out of it, and neither fed it a raw grid. Reki renders the last few frames as labelled images and asks for a single JSON object. The Duck gives its model a Python REPL and three views of the board at once. They spent their effort on the wrapper, not the prompt, which is the same conclusion we reached measuring how much context a local agent really needs.
Why GPT-6 Astra scored 62.7 and 99.9 on the same benchmark
When ARC-AGI-3 launched on 25 March 2026, the announcement reported humans at 100% and frontier systems at 0.51%. The technical report put the top four at Opus 4.6 on 0.50%, Gemini 3.1 Pro on 0.40%, GPT 5.4 on 0.20% and Grok 4.20 on 0.10%. Five months later, the results table reads 2.11% for Grok 4.6, 7.78% for GPT-5.6 and 30.16% for Claude Opus 5.
Then, on 3 September, ARC Prize published GPT-6 Astra: 62.7% on the semi private set for $26,098 under the standard harness, and 99.9% for $18,817 under OpenAI’s own Provider Adapter, which preserves opaque reasoning state between requests and compacts long conversations so the model can reuse earlier work. Same model, same games. A 37 point gap, and the expensive run is the low one.
ARC-AGI-3 on the semi private set, read 4 September 2026. Blue bars are the standard harness. The yellow bar is the same model under OpenAI’s Provider Adapter.
ARC Prize did not oversell it. Their post says saturating the benchmark “would not represent ‘proof of achieving AGI'”, and they now commit to reporting both harness conditions separately with each condition labelled. That is the right call, because the number that moved was not the model’s.
This is the single most useful thing to take from ARC-AGI-3 right now, and it applies to every agent you build. State carried between calls beat raw capability by a factor no model upgrade has matched this year. We found the same shape when measuring what a coding agent’s system prompt actually costs: the harness, not the weights, decided the bill.
Which local models are winning ARC-AGI-3 prize money
Here is the part that did not make the headlines. ARC Prize’s writeup of the first milestone, judged on 30 June 2026, lists the top three entries. None of them called a frontier API.
First place went to Tufa Labs with an agent called The Duck, running Qwen 3.6 27B at FP8, locally. It plays by writing Python into a live REPL and turning the board into variables, reading the game through three channels at once: a rendered image, the raw ASCII grid, and a segmentation tool. It stays alive past its context limit by evicting its oldest messages, which the writeup calls infinite play via eviction. The finding buried in that entry is worth more than the ranking: hand crafted tools made the model worse, and letting it improvise made it better.
Second place, Reki, renders recent frames as labelled images, feeds them to Gemma 4 31B locally, and asks for one JSON object back. Third place, forge, also runs Gemma 4 31B. That is the same model this stack keeps loaded in LM Studio, and the same one we put head to head against Qwen 3.5 earlier this year.
So the leaderboard and the prize table tell opposite stories. The leaderboard says you need a September frontier model and a five figure evaluation bill. The prize table says the top three finishers used models you can hold on a laptop, and gave the code away, because the competition rules require every winning method to be open sourced under a permissive public domain licence, CC0 or MIT-0, before it is eligible. Milestone #2 closes on 30 September 2026, with $25,000 for first place. Final submissions are due 2 November and results land 4 December, and the $700,000 grand prize sits unclaimed until an agent takes 100%.
The benchmark that was meant to be immune to brute force has been answered, in five months, by a harness that remembers. Whether that counts as reasoning is ARC Prize’s problem. The practical read is narrower and lands closer to home: on ARC-AGI-3, what you keep between calls is worth more than what you pay per call, and the people proving it on prize money are doing it with 27B and 31B weights on hardware you can buy. Milestone #2 closes on 30 September, and the code has to be public to win.