This is the runbook. If you want to know *what the format actually changes* before you pull a tag, read what Ollama NVFP4 means for local model quality first. If you want to know *why* it only works on macOS, that is a separate piece: NVFP4 on Apple Silicon. Here we are just getting a model running.
Prerequisites
- A Mac with Apple Silicon. Not optional. NVFP4 in Ollama runs on the MLX engine, and the MLX engine is macOS-only. On Windows or Linux the catalogue returns
412: this model requires macOS. - Ollama v0.19.0 or later. That is the release that shipped the MLX engine, on 27 March 2026. NVFP4 importing landed three days earlier in the MLX path.
- Unified memory to spare. Ollama’s MLX launch guidance was “a Mac with more than 32GB of unified memory.” Below that you are likely on the llama.cpp engine instead, and NVFP4 tags may not be the right choice.
Check your version:
“bash
ollama --version
`
The tags that exist
Confirmed NVFP4 tags in the Ollama library:
Gemma 4
`bash
ollama run gemma4:12b-nvfp4
ollama run gemma4:26b-nvfp4
ollama run gemma4:31b-nvfp4
ollama run gemma4:e2b-nvfp4
ollama run gemma4:e4b-nvfp4
`
Qwen3.5
`bash
ollama run qwen3.5:4b-nvfp4
ollama run qwen3.5:9b-nvfp4
ollama run qwen3.5:27b-nvfp4
ollama run qwen3.5:35b-a3b-nvfp4
ollama run qwen3.5:35b-a3b-coding-nvfp4
`
Qwen3.6 NVFP4 tags also exist, qwen3.6:27b-coding-nvfp4 and qwen3.6:35b-a3b-coding-nvfp4, though we have seen them referenced in issue threads rather than confirmed on a library page.
If you read somewhere that Ollama has only one NVFP4 model, that was true in March and is long out of date.
Four gotchas
1. -coding- tags are not fine-tunes. Per an Ollama collaborator, the coding variants are "otherwise unmodified" models with different default sampling parameters. Do not pick one expecting a differently trained model. If you want the sampling behaviour, you can get it on any tag by setting the parameters yourself.
2. No vision on the NVFP4 coding tags. Stated reason: "the MLX backend they run on is not yet vision capable." If your pipeline sends images, this is a hard blocker, not a degradation.
3. Structured output has been silently ignored on the MLX engine. Issue #17013, 2 July 2026: format was being dropped without an error. This is the worst failure mode for agent work: your JSON schema is discarded and you get prose back. Verify structured output explicitly before trusting it in a pipeline.
4. The tag does not mean the whole model is NVFP4, and the file will not be smaller. Conversion is partial by design: NVIDIA's own recipe for Gemma 4 26B-A4B is named nvfp4_experts_only, and community builds preserve norms, SSM convolutions and router gates in BF16/F32. It shows up in Ollama's own library tags: gemma4:26b-a4b-it-q4_K_M and gemma4:26b-nvfp4 are both 18 GB. Do not expect a memory saving from the name.
Verifying you got what you asked for
Confirm the model's actual size on disk and which engine served it:
`bash
ollama list
ollama ps
“
ollama ps will show the loaded model and its size. On paper NVFP4 should be *smaller* than q4_K_M, 4.5 bits per weight against 4.89, but in practice it is not, for the reason in gotcha 4. We work through that arithmetic in NVFP4 vs Q4_K_M.
What you should expect performance-wise
Ollama’s own claim, in its MLX performance post of 11 June 2026, is that NVFP4 is “about 20% faster than q4_K_M on the updated engine” and “roughly halves the quality loss of 4-bit quantization, relative to unquantized BF16.” Measured on Gemma 4 12B, MacBook Pro M5 Max, averaged over 10 runs at 8,300 tokens of input.
No numbers accompany the quality claim. No perplexity values, no dataset, no BF16 baseline figure, so as published it cannot be checked.
The independent measurements that do exist point the other way. On a MacBook Pro M3, in Ollama issue #16127: gemma4:31b-nvfp4 at 6.29 tok/s against gemma4:31b-it-q4_K_M at 6.62. So do not assume the 20% transfers to your machine. On the one published Apple Silicon comparison it reverses.
If you are on Gemma 4, you are also getting MTP speculative decoding by default since v0.31.1, which is a much larger speed effect than the format choice. That is covered in Ollama MTP, and it means any NVFP4-versus-q4_K_M timing you take on Gemma 4 needs MTP held constant or it measures the wrong thing.
The comparison worth running on your own machine, and the one we intend to run on ours, is narrow enough to finish in an afternoon: the same model in NVFP4 and in q4_K_M, with a tensor audit, on-disk size, peak memory, tokens per second at 8K and 32K context, and perplexity against a BF16 baseline, all with draft_num_predict: 0 so MTP is held constant.
The short version
- Apple Silicon only, Ollama v0.19.0+, ideally more than 32GB unified memory.
- Gemma 4 (
12b,26b,31b,e2b,e4b) and Qwen3.5 (4b,9b,27b,35b-a3b, plus-coding) have NVFP4 tags. -coding-means different sampling defaults, not a different model.- No vision on those tags; structured output has been silently dropped on MLX.
- Check the tensor ratio and the on-disk size before believing the format is doing anything.
- Hold MTP constant if you are benchmarking the format on Gemma 4.