Speculative decoding is the largest free speed win available to a local model. A small draft model proposes several tokens, the big model checks them in one pass, and the output is identical to what you would have got anyway. The catch is that it needs a draft model, and the draft model is not in the file you downloaded.
We read six Gemma 4 12B files across three publishers to find out where it actually lives. It ships in two entirely different shapes depending on the model family, and the main quantised download contains neither of them.
Where does the MTP head actually live?
In one of two places, and the difference is architectural rather than cosmetic.
For Qwen3.5, the multi-token prediction tensors sit inside the main model file as an extra block. We found that by comparing publishers: one shipped 41 blocks where another shipped 40, and the extra block held four MTP tensors. Whether you get speculative decoding depends on which publisher’s file you pulled, because the head is either in the download or it is not.
For Gemma 4 12B, it is a separate file with its own architecture. bartowski’s repository publishes mtp-gemma-4-12B-it-Q4_0.gguf alongside the ordinary quants. Reading it returns something that is not a fragment or a patch but a small complete model:
| Property | Value |
|---|---|
| Architecture | gemma4-assistant |
| Internal name | google/gemma-4-12B-it-assistant |
| Blocks | 4 |
| Tensors | 49 |
| Hidden size | 1024 |
| Vocabulary | 262,144, shared with the main model |
Two of those 49 tensors are nextn.pre_projection and nextn.post_projection, which is the mechanism: the draft model projects into and out of the main model’s representation space so its proposals can be checked cheaply.
Four blocks against the main model’s forty-eight. It is roughly a twelfth of the network, carrying the full vocabulary so it can propose real tokens.
Does the main download include it?
No, in every build we read.
We audited Gemma 4 12B from three publishers: unsloth, Google’s own quantisation-aware-trained release, and bartowski. All three produce a structurally identical main model, 48 blocks and 667 tensors, differing only in quantisation mix:
- unsloth ships Q4_K with Q6_K promotions
- Google ships Q4_0 with Q6_K promotions
- bartowski ships Q4_K, Q5_K and Q6_K
Not one of them embeds an MTP head in the main file. That includes Google’s own quantisation-aware-trained release, which is the build most likely to be treated as the reference version of the model. Of the three, only bartowski publishes the assistant model at all, and only as a separate download that you have to know exists.
So the practical situation for anyone wanting speculative decoding on Gemma is that the ordinary path gives you nothing. You need a second file, from one specific publisher, whose filename prefix is the only sign of what it is.
Why two packaging shapes for the same idea?
Because the two model families implemented it differently, and GGUF simply reflects that.
Qwen trains its MTP tensors as part of the network, so they live in the same tensor namespace and a converter either carries them across or drops them. Google trains a separate assistant model, so it converts into its own file with its own architecture string.
That has a consequence worth stating plainly: gemma4-assistant is an architecture a runtime must recognise. A loader that knows gemma4 will not necessarily know what to do with a file declaring gemma4-assistant, which puts the burden on the runtime rather than the file. This is the same layer where the interesting decisions kept turning up when we looked at how Ollama handles MTP, where the finding was that it exploits Gemma’s heads and discards Qwen’s.
Read together, the three layers now look like this. The model author decides whether to train an MTP component and how to package it. The quantiser decides whether to carry it across, and for Gemma whether to publish it at all. The runtime decides whether to use what it finds. A speedup that is real and free at every one of those layers still reaches you only if all three line up.
What to check before assuming you have it
Three checks, none costing more than a file listing.
Look for a separate file with an mtp- prefix. If the model family packages its draft separately, the main quant will never contain it no matter which publisher you choose. Its absence from the file you downloaded is not a defect.
Compare block counts between publishers. If the family embeds the head instead, a build one block shorter than another build of the same model has dropped it. Nothing in the quant label records that.
Do not read a smaller file as a worse quant. An assistant model is a twelfth of the depth and will
look, in a file listing, like an aggressively compressed version of something. It is not a quant of the
main model at all; it is a different network with its own job, and loading it as your primary model would
give you a four-block model wearing a familiar name.
Check the architecture string, not the filename. A draft model declares its own architecture, and that string is what decides whether your runtime can load it. Reading it takes a range request over the first few megabytes, the same method that shows what a quant name does and does not tell you and that we used for sizing mixture-of-experts models in memory.
The pattern across this whole series of audits holds here too. Nothing is concealed, and every one of these publishers has made a defensible choice. But the choices are invisible from a download page, and the difference between a setup that can draft tokens and one that cannot is a file most people never learn to look for.