A single ollama pull gemma4:26b command downloads 16 gigabytes of weights. Two minutes later, you are talking to a model that scores 82.6% on MMLU Pro in Google’s Gemma 4 model card, within three points of the 31B dense flagship. That combination changes the calculus for anyone running local AI on unified memory hardware. The 26B MoE from Google DeepMind, released April 2 2026 under the Apache 2.0 license, is the first open model where the efficiency argument is stronger than the raw size argument. On a system with shared CPU-GPU memory, it matters more than any dense alternative at any quantization.
What does the 26B MoE architecture do for memory efficiency?
The model has 26 billion total parameters arranged as a mixture of 128 expert sub-networks. A learned router activates only 8 experts per token, plus one shared expert, for a total of roughly 3.8 billion active parameters per forward pass. The remaining 22 billion weights sit in memory as a static lookup pool. They occupy space. They do not consume compute.
This is the distinction that matters for unified memory. On a discrete GPU system, VRAM is the scarce resource and the active-parameter count determines speed. On an Apple Silicon Mac or an NVIDIA GB10-class machine, the same memory pool serves both storage and computation. The 26B MoE needs all 26B weights in that pool at Q4 precision, which is about 16 to 18 gigabytes depending on the quantization variant. But the per-token compute is only about 4 billion parameters worth of matrix operations. That is why the model generates text at speeds that feel like a 4B dense model while answering questions at a quality level near the 31B dense flagship.
The 256K context window, listed in Google’s model card, adds KV cache overhead that grows with prompt length, but the MoE architecture keeps the compute side light even when the context is large.
What speed can you expect at each unified memory budget?
The practical question for anyone buying a machine today is what speed to expect at each memory tier. We have not measured the 26B MoE on our own hardware, so this section sets out what fits at each tier rather than how fast it runs, and it pairs well with our guide on how to pick a local model for your RAM.
At the 24 gigabyte tier, which covers the M4 Pro 24GB and any Windows laptop with 24GB of unified or shared memory, Q4_K_M is the realistic choice. The model artifact alone takes 16 to 18 gigabytes, leaving 6 to 8 gigabytes for the OS, the KV cache, and other applications. Context length needs discipline. Keep it at 4K to 8K unless you close every other application.
At the 48 gigabyte tier, the most common high-end MacBook Pro configuration, the experience changes. The model loads at Q5_K_M or even Q8_0 with full 128K context headroom. For comparison, the 31B dense model does fit on our own 48 GB M4 Pro at a 64K context and generates there at a median 11.0 tokens per second, as recorded in our test of LM Studio’s default context length. The MoE’s advantage at this tier should be speed, since it activates 3.8 billion parameters per token against the 31B’s full set, though we have not measured it.
How does quality per GB compare to the 31B dense model?
The headline benchmark numbers tell a clear story. Google’s Gemma 4 model card puts the 26B MoE at 82.6% on MMLU Pro versus 85.2% for the 31B dense, 77.1% versus 80.0% on LiveCodeBench v6, and 88.3% versus 89.2% on AIME 2026. The 31B wins every row, by less than three points each time. That is a real quality difference. The question is what it costs to get those extra points.
The runtime footprint is larger than the file size. On our own 48 GB M4 Pro, the 31B dense QAT build is an 18.85 GB file whose running process held 21.97 GB at an 8,192-token context and 25.24 GB at 65,536, as measured in our context window test. We have not measured the 26B MoE the same way, so we make no quality-per-gigabyte calculation here.
The gap stays small on coding and reasoning. On LiveCodeBench v6, the 26B scores 77.1% to the 31B’s 80.0%, a difference of 2.9 points that may matter less in daily use than the speed the MoE should have. On our own 48 GB M4 Pro, the 31B generates at 10.3 to 11.5 tokens per second, so a reply capped at 300 tokens takes 19 to 28 seconds, as measured in our prompt processing benchmark. The quality difference between the two models is smaller than the quality difference between getting an answer quickly and getting it slowly enough to break your flow.
Which quantization should you choose for your memory budget?
The 26B MoE gives back some of the memory advantage at higher quantization levels. The practical question is whether the extra precision matters for the work you do.
For users with 32 to 48 gigabytes of unified memory, Q5_K_M is a reasonable upgrade if you have the headroom and want the extra safety margin for long-context reasoning. For anyone on 24GB machines, Q4_K_M is the realistic and correct choice.
What do the benchmarks miss about daily use?
Published benchmarks measure tokens per second and MMLU points at fixed context lengths. They do not capture three factors that determine whether a model feels good to use on a unified memory machine.
The first is prompt processing speed. On our own 48 GB M4 Pro, the 31B dense processes prompts at about 65 tokens per second, so a 5,197-token prompt waited 79.56 seconds before the first token, as measured in our prompt processing benchmark. In that test, prompt processing was roughly 75 percent of the wait for a 290-token answer. For retrieval-augmented generation workflows, where the user’s question plus retrieved context can easily reach 4,000 tokens, that is most of the time before the first word appears. A model that activates fewer parameters per token should shorten it, though we have not measured the 26B MoE’s prompt processing on this machine.
The second is latency under thinking mode. The cost is in extra tokens emitted during internal reasoning. In our own runs of the 31B dense, 254 to 297 thinking tokens came before 35 to 36 visible ones, so roughly 87 percent of the generation time went to text the user never reads, as our prompt processing benchmark recorded. The practical takeaway: leave thinking mode off by default and toggle it only for genuinely difficult problems.
The third factor is the compound latency of agentic workflows. If a coding agent makes five tool calls, each requiring a model invocation, it pays the model’s generation speed five times, and on our 48 GB M4 Pro the 31B dense generates at about 11 tokens per second. The cost grows with every additional step. For anyone building local agents, autocorrect systems, or iterative code assistants on unified memory, the 26B MoE is not just a good fit. It is the only practical option until the 31B dense finds an efficiency breakthrough or unified memory capacities cross 192 gigabytes on consumer hardware. We reach a similar conclusion in our pick for the best local model for OpenCode.
The next generation of Apple Silicon, with the M4 Ultra reportedly supporting up to 192 gigabytes of unified memory, will make the 31B dense more comfortable at FP16. But the 26B MoE will still be the faster choice for interactive work, and the quality gap between them will remain small. The model that wins on quality per gigabyte and quality per second is the one that makes local inference feel like a product, not a science project. Right now, that model is the 26B MoE.
For how a mixture-of-experts model actually sizes against a dense one in unified memory, including why it can be faster and larger at the same time, see MoE on a Mac.
Disclaimer: This is a research piece. We have not run these comparisons ourselves: the benchmark scores in this piece are Google’s published figures, and the only measurements of ours are for the dense 31B on the studio M4 Pro. We will add our own measurements of the 26B MoE once we have run them.