Pull a multimodal model in GGUF form and you get two files. The one everyone talks about carries the language model and wears the quant label: q4_0, Q4_K_M, whatever you chose. The other is called mmproj, it is rarely explained, and people tend to assume it is a small adapter that inherits whatever precision the main file uses.
It does not. We read the tensor tables of mmproj files from five publishers. Four ship them at 16-bit or 32-bit precision beside a text model compressed to four bits. The fifth, the llama.cpp organisation itself, compresses the vision tower to 8-bit and stops there. Nobody takes the eye down to the precision of the model it feeds.
What is actually inside an mmproj file?
A complete vision encoder, declaring its own architecture.
Reading Google’s own Gemma 4 26B mmproj returns something structurally independent of the model it accompanies:
| Property | Value |
|---|---|
| Architecture | clip |
| Tensors | 356 |
| Vision blocks | 27 |
| Embedding length | 1152 |
| Projection dimension | 2816 |
| Image size | 224, patch size 16 |
| Types | BF16 x190, F32 x166 |
The architecture string is clip, not gemma4. That is the same pattern we found with the separate draft model for speculative decoding: a companion file is a different network with its own architecture, not a fragment of the main one.
The tensor names split into two groups, and the ratio is the interesting part. Three hundred and fifty-five carry a v prefix, the vision encoder proper. Exactly one carries mm, and that is the multimodal projector: the single matrix that maps the vision encoder’s output into the language model’s embedding space.
So the file named for the projector is 355 parts vision encoder and one part projector. The name describes the smallest thing in it.
At 224 pixels with a patch size of 16, the encoder divides an image into a 14 by 14 grid, 196 patches, and pushes each through 27 transformer blocks. That is a substantial network on its own. For comparison, the text model it accompanies has 30 blocks, so the vision tower is nearly as deep as the language model it feeds. Calling it a projector understates it by an order of magnitude.
Who quantises the vision tower, and how far?
One publisher out of five, and only halfway.
The Gemma 4 26B mmproj from Google contains 190 BF16 tensors and 166 F32 tensors. Not a single quantised tensor, in a repository whose entire purpose is quantisation-aware training at q4_0.
Checking every publisher we could find:
| Publisher | mmproj precision offered |
|---|---|
| BF16 and F32 only | |
| unsloth | BF16, F16, F32 |
| bartowski | BF16, F16 |
| lmstudio-community | BF16 |
| ggml-org | Q8_0, alongside an F16 |
unsloth publishes three variants and bartowski two, so this is not a case of nobody bothering. They made options, and every option is 16-bit or wider. The exception is ggml-org, the organisation behind llama.cpp itself, which ships a Q8_0 vision tower for several models and demonstrates using one in its own quantisation documentation.
Reading that Q8_0 file shows a genuine compression rather than a relabel: 163 Q8_0 tensors, 27 F16 and 166 F32, against the BF16 build’s 190 BF16 and 166 F32. It is 0.81 GB where Google’s is 1.19 GB, a third smaller.
But notice where it stops. Q8_0 is eight bits. The text model beside it is at four. Even the publisher most willing to compress the vision tower leaves it at twice the precision of the language model, and no one at all offers a 4-bit mmproj.
That near-uniformity is the finding. Publishers who disagree about almost everything else, as we found when the layer-promotion schedule turned out to be inherited rather than chosen, agree that the eye gets treated more gently than the brain.
What it costs you
More than the file size suggests, and it is a fixed cost.
For Gemma 4 26B, the q4_0 text model is 14.44 GB and the mmproj is 1.19 GB. The vision tower is 7.6% of the combined download, which sounds minor until you consider what it is not doing: shrinking. Choose a more aggressive quant for the text model and the mmproj stays exactly where it is, so its share of your memory budget rises with every step down you take.
Push the text model to a 2-bit build and the unquantised vision tower could approach a fifth of what you load.
This matters for the same reason the expert budget matters when sizing a mixture-of-experts model in memory: the number in the filename describes one component, and the thing you actually load is the sum.
What to do about it
Three practical points.
Do not skip the mmproj and expect vision to work. It is not optional metadata. Without it the language model has no way to receive an image at all, because the one mm tensor that bridges the two lives in that file.
Budget for it separately. Add the mmproj size to your quant’s size before deciding whether a model fits. On a memory-constrained machine, a multimodal model is meaningfully larger than its headline quant implies, and the gap widens as you quantise harder.
Read the architecture string if you are unsure what a file is. A repository listing often shows
several files whose names differ by a few characters, and mmproj is only a convention rather than a
guarantee. The clip architecture declared inside the file is the reliable signal, and it takes a range
request over the first few megabytes to read. A file declaring clip is a vision encoder whatever it is
called; a file declaring the model architecture is the model.
A quantised one exists if you need the memory. ggml-org publishes Q8_0 mmproj files for several models, roughly a third smaller than the BF16 build. It is the only source we found, so if memory is tight that is where to look, and if you are already using an mmproj from elsewhere it is almost certainly uncompressed.
The wider point is the one this series keeps arriving at. A model is not a file, it is a set of files with different rules, and the label on the one you were shown describes only that one. The text model obeys the quant you asked for. The eye it sees through was never part of that conversation.