The same 64 by 64 game board went to the same local model twice within an hour on a studio test machine, once as 4,096 hex digits and once as a 512 pixel PNG. The text cost 4,179 prompt tokens, 3,733 tokens of reasoning and 393 seconds. The image cost 187 prompt tokens, 1,691 tokens of reasoning and 159 seconds.
Image vs text is the first decision a harness makes for a vision capable local model, and it is usually made by habit. Here is what one measured pair looks like, and what it does and does not settle.
How the same board was sent as an image and as text
The model was gemma-4-31b-qat through LM Studio, which reports it as a vision language model; the endpoint accepts an image_url content part carrying a base64 PNG alongside text, in the same shape as the OpenAI API. The frame was the opening board of ls20. For the text run it was written as 64 rows of hex digits, one per cell. For the image run each cell was mapped to a colour from a fixed 16 entry palette, scaled up eight times with nearest neighbour to 512 by 512 so that every cell was an 8 by 8 block, and encoded as a PNG.
Both runs carried the same system prompt, the same four legal actions, the same instruction to pick one, an 8,192 token cap and a temperature of 0.2. The two calls were made in the same session, a few minutes apart, so the comparison is not confounded by a model reload or a different server build.
Image vs text: what the picture cost against the hex grid
| Input | Prompt tokens | Reasoning tokens | Time | Answer |
|---|---|---|---|---|
| Text, 64 rows of hex | 4,179 | 3,733 | 393.2s | ACTION1 |
| Image, 512 by 512 PNG | 187 | 1,691 | 158.8s | ACTION4 |
The prompt is 22 times smaller as an image, the thinking is less than half, and the wall clock is 40 percent of the text run. An earlier text run on the same frame in a different session, the one that found the 4,096 cap was too small, needed 4,274 tokens of reasoning and 442.7 seconds, so the text cost is consistent across sessions and the image is cheaper than both.
One pair is one pair. On this stack identical prompts have varied by more than four times in thinking, so a 2.2 times difference in reasoning between two single runs is a lead rather than a finding. The prompt token count is not subject to that noise; 187 against 4,179 is arithmetic, and it holds on every call.
Why 187 tokens for a picture and 4,179 for the same cells
Text tokenisers do not know that a grid is a grid. Sixty four rows of sixty four characters become roughly one token per cell, because hex digits do not compress, and the model then has to reconstruct the two dimensional structure from a one dimensional stream before it can reason about adjacency at all. That reconstruction is part of the 3,733 tokens of thinking.
A vision encoder takes the image as a small fixed grid of patches and hands the language model a couple of hundred embeddings that already carry position. The 187 prompt tokens include the instruction text, so the image itself was on the order of a hundred and fifty tokens for 4,096 cells. Nothing in the reasoning has to rebuild the layout, which is a plausible reason the thinking was shorter, and the reason it is only plausible is that a single run cannot separate it from chance.
This is the same effect that makes a 16 by 16 downsample cheaper than the raw grid: the model pays for how much it has to reconstruct, not for how many characters arrived. An image is the representation that asks for the least reconstruction, at the cost of precision the encoder decides for you.
What the answer changed to, and why that is not a verdict
The text run chose ACTION1. The image run chose ACTION4. Neither is known to be right, since a first move on an unknown game is exploratory by definition, and the 40 move agent that pressed ACTION1 eleven times running shows that agreeing with the text run is no recommendation.
What the different answer does show is that the model saw a different board. An image conveys colour as colour and adjacency as adjacency; a hex grid conveys both as symbols to be decoded. A model that reads those two descriptions of one frame differently is behaving as expected, and which reading is more useful is a question for a full game, not a first move. The second place entry in the first milestone, per ARC Prize’s writeup, “renders the recent frames as labeled images” and feeds them to a Gemma 4 31B, which is the same model class and the same decision, made by a team that then won money with it.
The precision question is real. A vision encoder works at a fixed patch resolution, and a single cell in a 64 by 64 board is one sixty fourth of the image edge. Whether an 8 by 8 pixel block survives the encoder as a distinct object is something to test per game with a known board, in the same way a hand built game with a one pixel sprite tests the engine.
When to send an image to a local model
Send the image when the board is large and the model is vision capable. The prompt cost drops by an order of magnitude on every call, which is the one part of this comparison that is not a single run, and the thinking and wall clock dropped with it here.
Send text when the decision turns on an exact cell value or coordinate, because the text is lossless and the image is whatever the encoder made of it. Send both when you can afford it: the image for layout, a colour summary for the exact counts, at a combined cost still well under the raw grid.
And send the image at a resolution you have checked. Scaling 64 cells to 512 pixels made each cell an 8 by 8 block, which is large enough that the model’s answer changed; scaling to 64 pixels would give the encoder one pixel per cell, and there is no reason to believe it would see anything. The technical report fixes the frame at 64 by 64 with 16 colours, and every representation of it, text, downsample, summary or picture, is a choice about which of those cells the model gets to keep.