Hermes Agent put a blue “Recommended” badge next to Qwen3.6 35B-A3B the first time we opened its new Local Models page on a 48 GB M4 Pro. One click on “Set up for me” would have downloaded 21.9 GB, installed a llama.cpp engine and made the model our default for new chats. We downloaded it, along with the Qwen3.8 27B that Hermes lists above it, and ran Qwen3.6 35B-A3B vs Gemma 4 31B on the same ARC-AGI-3 board test behind our reasoning token counts. Qwen3.6 35B-A3B wrote about 50 tokens a second, nearly five times Gemma’s speed, and gave a move in 4 of 12 runs. Gemma gave one in 11.
From here, Qwen3.6 means Qwen3.6 35B-A3B and Qwen3.8 means Qwen3.8 27B.
Why does Hermes recommend Qwen3.6 35B-A3B on a 48 GB Mac?
The badge is a speed prediction, not a quality ranking. Hermes ships a catalog of four local models, and its own quality scores rate Qwen3.8 above Qwen3.6. The rule in Hermes’s catalog.py then drops any model it predicts will generate fewer than 20 tokens a second, a figure the code calls the pleasant floor. The prediction is memory bandwidth divided by the bytes read for each token, and machines with unified memory are assumed to move 210 GB a second. The comment beside that number says it was measured on unified-memory NVIDIA hardware, not on a Mac.
Qwen3.8 is a dense model, so every token reads all 16.5 GB of its weights, and Hermes’s formula gives it about 13 tokens a second. That is below the floor, so it is out. Qwen3.6 is a mixture of experts. Its model card lists 35 billion parameters with 3 billion active per token, and Hermes’s catalog prices it at 15% of the file per token, which the same formula turns into about 62 tokens a second. So the model Hermes rates lower is the one it recommends.
Our measurements agree with the direction of that prediction. Across 12 runs per model, the median generation speed was 50.8 tokens a second for Qwen3.6, 10.9 for Gemma 4 31B and 8.7 for Qwen3.8. Qwen3.8 came out slower than Gemma even with multi-token prediction switched on.
Median generation speed over 12 ARC-AGI-3 runs per model, Hermes’s llama.cpp build b10679, 11 September 2026.
How we tested Qwen3.6, Qwen3.8 and Gemma 4 in Hermes
Hermes desktop v0.21.1, the release that made Bot Mode the default, keeps local models under Settings, Providers, Local Models. The Download buttons do nothing until the engine is installed. Install runtime fetched llama.cpp build b10679 from GitHub, which took 37 MB on disk against the few hundred MB the page describes. Qwen3.8 then arrived in about four and a half minutes, 16.46 GB plus a 0.93 GB vision projector, and Qwen3.6 took about six minutes for 22.66 GB plus 0.90 GB. Both land in ~/.hermes/models, a folder LM Studio does not scan. We used “Configure” rather than “Set up for me”, so our default model never changed.
The test is the one from our guide to running ARC-AGI-3 on a Mac. We took the first frame of the game ls20, reduced it to 8×8, 16×16, 32×32 and 64×64 grids of hex digits, listed the four legal actions and asked for one action name. Each model got three runs per board at temperature 0.2, with a cap of 16,384 output tokens and a 65,536 token context. Every run went through the llama.cpp server Hermes installs, with the launch settings Hermes’s own code writes: an 8-bit KV cache, flash attention and, for both Qwen models, multi-token prediction. The one change we made was the context, which we pinned to 65,536 for all three models so they match.
Gemma needed one more step. The memory estimator in Hermes’s estimator.py knows the attention layout of Gemma 2 and Gemma 3 but not Gemma 4. It prices Gemma 4 31B at about 72 GiB for a 64K context, so the code that writes launch settings declined it. The same engine loaded the same file at 65,536 context in 11 seconds once we gave it the settings Hermes uses for other dense models.
Two limits apply. A background job that keeps Gemma loaded in LM Studio for our assistant reloaded it at some point during the runs: when the Qwen3.8 sweep ended, an idle 18.85 GB copy sat in memory beside the test model. That does not change what a model writes, but the timings are not from a clean machine. The test also records whether a model commits to a move within budget, not whether the move is correct.
Qwen3.6 35B-A3B vs Gemma 4 on ARC-AGI-3: answers, tokens and speed
| Model | Runs with a move | Runs that hit 16,384 | Median speed | Median time per run |
|---|---|---|---|---|
| Gemma 4 31B | 11 of 12 | 1 | 10.9 tok/s | 2.7 min |
| Qwen3.6 35B-A3B | 4 of 12 | 8 | 50.8 tok/s | 5.2 min |
| Qwen3.8 27B | 0 of 12 | 10 | 8.7 tok/s | 31.5 min |
Gemma’s answers took between 666 and 3,413 tokens. Its one failure was a 64×64 run that spent the full 16,384 over 29 minutes. Qwen3.6 answered once on each board size, after 10,249, 12,756, 4,507 and 8,941 tokens. At its speed a failed run costs about five minutes rather than half an hour, but eight of its twelve runs ended the same way: 16,384 tokens and no move.
Qwen3.8 never gave a move. Ten runs hit the cap at about 30 minutes each, and two 64×64 runs stopped on their own after 3,377 and 13,616 tokens with an empty reply, the pattern we traced in LM Studio empty responses where reasoning eats the answer. The full sweep took 5.7 hours against 1.1 for Gemma and 0.9 for Qwen3.6.
Runs that ended with a move, out of 12 per model, temperature 0.2 and a 16,384 token cap, 11 September 2026.
The models also read the board differently, and Gemma’s steadiness matches what the two Gemma 4 31B ARC-AGI-3 entries built their agents on. Gemma chose ACTION1 seven times and ACTION2 four times, including all three 32×32 runs. Qwen3.6 chose ACTION1 three times and ACTION3 once.
This matches the first time we measured the family. On 9 September the dense Qwen 3.6 27B ran the same boards in LM Studio, and all six of its runs stopped at 16,384 tokens with no answer. With the cap raised to 49,152 it answered the 8×8 board after 30,837 reasoning tokens and 47.7 minutes, where Gemma had needed 684 tokens and about a minute. How the cap interacts with a reasoning model is the subject of our piece on max_tokens for reasoning models.
Why Qwen3.6 runs out of tokens: what the reasoning text shows
Token counts cannot tell careful thinking from a loop, so we reran Qwen3.6 on the 8×8 board and kept the full reasoning text. One run answered ACTION2 after 8,245 tokens and 166 seconds. Its reasoning follows one idea, that an isolated cell is the player, all the way to a move. The other run hit the cap. For each quarter of each text we measured the share of 60-character stretches that had already appeared earlier in the same text.
The answered run repeated 0%, 2.0%, 5.7% and 21.8% of itself, quarter by quarter. The capped run started the same way, at 0% and 1.8%, then rose to 26.3% and 99.5%. By its last quarter almost every stretch was a repeat, and the phrase “Maybe the actions are” had restarted 38 times against 3 in the answered run. The server’s multi-token prediction counters point the same way. On one capped Qwen3.6 run, 96.8% of drafted tokens were accepted, and text that repeats itself is easy to predict.
Share of 60-character stretches already written earlier, by quarter of the reasoning text, two Qwen3.6 runs on the 8×8 board at temperature 0.2, 11 September 2026.
Sampling is the open question. Qwen’s model card recommends temperature 1.0, top_p 0.95, top_k 20, min_p 0 and presence_penalty 1.5 for thinking on general tasks, and says a presence_penalty between 0 and 2 can “reduce endless repetitions”. Our test ran at temperature 0.2 with no presence penalty, the setting used across the whole series. On a 16×16 board, our LM Studio temperature tests found temperature made little difference to how much Gemma reasoned. Hermes’s own preset for Qwen3.6 sets temperature 1.0, top_k 20, top_p 0.95 and min_p 0 but no presence penalty. We have not measured either Hermes’s preset or the model card’s settings on this board. One looping run at 0.2 is an example of the failure, not a rate.
Which local model to pick in Hermes on an M4 Pro
For a task like this one, where a model has to read a grid and commit to a single move, Gemma 4 31B is still the model we run, as it was when we set it against Bonsai 27B on a Mac. On our test stack it committed in 11 of 12 runs at 10.9 tokens a second. Qwen3.6 makes sense where speed per token matters more than finishing, or if Qwen’s recommended sampling removes the loop, which we have not shown. Qwen3.8, at 8.7 tokens a second and no answers in 12, is not a model we would put behind an agent on a 48 GB machine.
If Hermes already talks to Gemma through LM Studio, as in our always-on assistant setup with Hermes Agent and LM Studio, open “Configure” rather than “Set up for me”, because the one-click path changes the default model for new chats. The two Qwen downloads also stay in ~/.hermes/models after testing, 41 GB with their vision projectors, and LM Studio cannot see them. GGUF builds of one Qwen model can differ by whether they carry the multi-token prediction head, so check which build a tool pulled before comparing its speed with another’s. Where that head lives also varies by publisher, as our look at how MTP draft models are packaged found.
The number still missing is Qwen3.6 with presence_penalty at 1.5, the one setting its model card recommends that Hermes’s preset leaves out.