There are 25 ARC-AGI-3 games sitting on this machine. They arrived as ordinary Python files, 2,060 lines for the first one, and any of them can be opened in a text editor. That is the public set, and it is where most of the “ARC-AGI-3 is solved” numbers come from.
The number that actually compares one system against another is 62.7%, it was posted on 3 September, and it is not a hundred.
Three different numbers are being called saturation
They measure three different things and only one of them is a benchmark result.
100% on the public set. The public set is 25 environments and 183 levels, published so people can develop against them. Install the toolkit and they download to your disk as source. You can read the game logic, edit it, and run it as many times as you like. NVIDIA reported a perfect 100.00 RHAE on that set in August, and to their credit said so plainly: “These results cover the 25-environment ARC-AGI-3 public set. They are not results on the semi-private or fully private competition sets.”
A perfect score on a set you can read the source of is a development milestone. It is not evidence about novel environments, which is the only thing this benchmark claims to measure.
99.9% on the semi private set, with the vendor’s own harness. This is the GPT-6 Astra number from 3 September.
62.7% on the semi private set, with the shared harness. Same model, same games, same day.
The gap between the last two is 37 points, and it is the whole story.
The three ARC-AGI-3 numbers in circulation, read 4 September 2026. Only the third compares one system against another.
What the Provider Adapter harness actually changes
A harness is the code around the model: it feeds in the frame, takes back an action, and decides what the model remembers between turns. ARC-AGI-3 ships a deliberately plain one, identical for every system, so the comparison is between models rather than between engineering teams.
OpenAI’s Provider Adapter is not that. Per ARC Prize’s own writeup, it “preserves opaque reasoning state between requests and uses compaction for longer conversations, allowing the model to reuse prior work”. The model stops starting each turn cold.
The effect is not subtle, and it is not only the score. The same post reports the Provider Adapter runs were “approximately 3.66x faster by aggregate recorded elapsed time and used 49% fewer total tokens”. Faster, cheaper and vastly better, from a change to what the model is allowed to carry forward rather than to the model.
The costs make the point again. The 62.7% run cost $26,098. The 99.9% run cost $18,817. The worse score was the more expensive one.
ARC Prize has not tried to hide any of this. They now commit to reporting both conditions separately with each labelled, and the same post states outright that saturating the benchmark “would not represent ‘proof of achieving AGI'”. The caveats are in the primary source. They fall off in the retelling.
Which ARC-AGI-3 set a score is on, and why it decides everything
The benchmark is built in three layers, and the technical report counts 135 environments across them: 25 public, 55 semi private, 55 fully private.
The public 25 are a development kit. They ship with the toolkit, they run offline, and the human baseline for every level rides along in the game’s own metadata.json. Nothing about them is hidden, which is the point of them.
The semi private 55 are what ARC Prize runs models against for the leaderboard. The fully private 55 are held back for the competition, and no submission sees them before it is scored.
So a score is only meaningful with three labels attached: which set, which harness, and what it cost. Drop any one and the number stops being comparable. “ARC-AGI-3 is saturated” drops all three.
The split also explains why a public set score cannot be checked the way people assume. When the games are on your own disk as editable source, a run against them is a measurement of your development loop, not of the model’s ability to meet something new. That is not an accusation of cheating. It is what a development set is for, and it is why ARC Prize keeps two more layers behind it.
What is left to solve on ARC-AGI-3
Quite a lot, and the prize table is the cleanest evidence.
The $700,000 grand prize goes to the first agent that reaches 100%, and it is unclaimed. Submissions for ARC Prize 2026 close on 2 November with results on 4 December, and every winning method has to be open sourced under a permissive public domain licence before it is eligible. Nobody has taken the top prize on the sets that count.
Meanwhile the shared harness leaderboard, the one where every system runs the same code, tells a much more ordinary story than the headlines: Grok 4.6 at 2.11%, GPT-5.6 at 7.78%, Claude Opus 5 at 30.16%, and Astra’s 62.7%. That is real, fast progress from 0.51% in March. It is not a solved benchmark.
There is a second reason to be careful with the 100% claims. ARC Prize’s own leaderboard, per the technical report, only accepts systems “served behind a general-purpose API” that have not been specially prepared for the benchmark. Harness-based entries go on a separate community board, which explicitly disclaims them as “not evidence of AGI progress”. The organisation drawing the line is the one being cited as having had its benchmark beaten.
Why this matters if you are building agents rather than reading about them
The useful signal here is not who is winning. It is the 37 points.
The same weights, on the same games, scored 62.7 and 99.9 depending on whether the scaffolding around them kept state between calls. No model release this year moved a benchmark that far. If you are building anything agentic, that is the number to take away, and it is the cheapest improvement available to you: what your agent remembers between calls is worth more than which model you point it at.
It also happens to be what the people winning ARC-AGI-3 prize money are doing. The first milestone went to entries running 27B and 31B open weight models locally, including the Gemma 4 31B we run on this stack, and every one of them spent its effort on the wrapper rather than the weights. One writes Python into a live REPL. One renders frames as labelled images and takes back structured output. Both of them are harness work, and both exist because a raw board is expensive for a local model to read.
The lesson generalises past this benchmark, which is more than can be said for the headline. We ran into the same thing measuring what a coding agent’s system prompt really costs: the wrapper decided the bill, not the model.
The honest summary of ARC-AGI-3 in September 2026 is that a frontier model went from half a per cent to 62.7% in five months under the neutral harness, and to nearly a hundred when allowed to keep its own working memory between turns. Both facts are interesting. Only one of them is about the model, and neither of them is saturation. Ask which set, which harness, and what it cost, and most of the claims you will read this month answer none of the three.