The format is called NVFP4. NVIDIA defined it. NVIDIA’s Blackwell tensor cores execute it in hardware. And if you try to pull an NVFP4 model in Ollama on a Windows or Linux machine with a Blackwell card in it, the server refuses you:
“
412: this model requires macOS
“
That is not a bug report we are filing. That is the documented behaviour, and it is the single most counterintuitive fact in local AI right now. NVFP4 arrived in Ollama through the MLX engine, MLX runs only on macOS, and so the NVIDIA-designed four-bit format is, inside Ollama, a Mac-only feature.
Why a format is not a chip
The name does the damage. NVFP4 sounds like a hardware capability, and half of it is: Blackwell’s fifth-generation tensor cores consume the format directly, handling element grouping, dynamic scaling and the 4-bit matrix multiply without a dequantization step. Hopper cannot. That part is silicon.
But NVFP4 is also just a layout. Four-bit E2M1 elements, an FP8 E4M3 scale for every 16 of them, an FP32 scale for the tensor. Any framework willing to write kernels can read that layout and do the arithmetic, on any GPU. Apple’s MLX did exactly that. Metal kernels over an NVIDIA-specified layout, no NVIDIA silicon involved.
So Blackwell makes NVFP4 *fast*. Blackwell is not what makes NVFP4 *possible*. Those two ideas get collapsed constantly, and the 412 error is what the collapse feels like from the outside.
The timeline, from the commits
NVFP4 entered Ollama through the MLX path, never the GGML one.
The commit mlx: add mxfp4/mxfp8/nvfp4 importing (Ollama PR #15015) merged on 24 March 2026. Three days later, v0.19.0 shipped the MLX engine “in preview”, per its release notes. By the launch post on 30 March, Ollama was already advertising ollama run qwen3.5:35b-a3b-coding-nvfp4.
It has been actively worked since. mlx: Support NVIDIA TensorRT Model Optimizer import landed on 27 April. Then v0.30.6 on 5 June: “MLX embedding layers now use NVFP4 global scale for improved quantization on Apple Silicon.” The dedicated write-up came on 11 June, in Ollama’s MLX performance post.
Ollama’s own claims in that post: NVFP4 generates roughly 20% faster than q4_K_M on the updated engine, and roughly halves the quantization quality loss against q4_K_M as measured by perplexity on Gemma 4 12B.
Both numbers are Ollama measuring Ollama. We have not reproduced either. We intend to, on the studio M4 Pro, and we will say so plainly when we do. What those claims mean for output you actually read is the subject of what Ollama NVFP4 means for local model quality.
Who says it is macOS-only
This is not inference from release notes. Ollama collaborator rick-github has stated it directly, more than once.
On issue #16056, 9 May 2026: “NVFP4 support is provided by the MLX engine. The MLX engine currently only runs on macOS, so macOS is required to run an NVFP4 model. Support for Linux and Windows in the MLX engine is a work in progress.”
Again on #15854, 15 May: “NVFP4 support is via the MLX engine. The MLX engine currently requires macOS. Transitively, running qwen3.6:27b-coding-nvfp4 requires macOS.”
The restriction is enforced server-side, at the model catalogue, not locally. Users on Ollama issues #15854, #15772 and #16019 all hit the same 412. One user on #16265 went further: they found and installed ollama-windows-amd64-mlx.zip, got the MLX engine running on Windows, and were *still* refused, because the gate is an operating-system header on the catalogue request. A collaborator’s reply: “MLX support has not been officially announced for non-macOS environments.” That issue was opened on 22 May and remains open.
The workaround, and its ceiling
There is a second, separate path to NVFP4 that does work off macOS, and conflating it with the first is the easy mistake.
llama.cpp added NVFP4 as a GGML type in PR #19769, merged 11 March 2026, CPU-only at merge with Metal, CUDA and Vulkan deliberately deferred. Native CUDA kernels followed in PR #20644, landing around build b8967 in late April. Ollama bundles llama.cpp and pulled it forward.
So third-party NVFP4 GGUFs run on Ollama’s llama.cpp engine, on hardware Ollama’s own NVFP4 tags refuse. Issue #16056, opened 9 May 2026 by Huckies after a GGML_ASSERT(type >= 0 && type < GGML_TYPE_COUNT) failed on a Gemma 4 26B NVFP4 model, asks for the gap to be closed and remains open.
What the published measurements show, though, is that the format is not winning where it is available.
On Apple Silicon, NVFP4 is slower. Ollama issue #16127, reported by johnkea on 13 May 2026 on a MacBook Pro M3:
| Tag | tok/s |
|---|---|
gemma4:31b-nvfp4 | 6.29 |
gemma4:31b-it-q4_K_M | 6.62 |
That is the MLX path, the one NVFP4 is supposed to be optimised for, losing to the format it is meant to replace. The issue was closed as a duplicate of #16030.
On Blackwell through the GGUF path, also slower on decode. From knoopx's Qwen3.6-35B-A3B-NVFP4-GGUF model card on Hugging Face: token generation 136 t/s for NVFP4 against 188 t/s for Q4_K_XL. Prompt processing goes the other way, 9767 against 9121. Note that is Q4_K_XL, not Q4_K_M.
And the files are not smaller. From Ollama's own library tags:
| Tag | Size |
|---|---|
gemma4:26b-a4b-it-q4_K_M | 18 GB |
gemma4:26b-nvfp4 | 18 GB |
gemma4:31b-it-q4_K_M | 20 GB |
gemma4:31b-nvfp4 | 19 GB |
A format that should cost 4.5 bits per weight is arriving at parity with one that costs 4.89. The community ggufbench build of Gemma 4 26B-A4B lands at roughly 5.60 bits per weight, a "4-bit" format nearly a bit and a half over budget.
Why the files do not shrink
The mechanism is documented, and it is not a bug. NVFP4 conversion is partial by design.
NVIDIA's own quantization recipe for nvidia/Gemma-4-26B-A4B-NVFP4 is named nvfp4_experts_only. Only the MoE expert parameters get converted. Everything else stays wide.
knoopx's card says the same thing from the other direction: NVFP4 is applied "on all Linear weight layers, preserving auxiliary tensors (norms, SSM convolutions, router gates) in BF16/F32."
And there is a correctness dimension. llama.cpp Discussion #22042, opened by ORippler on 17 April 2026, explains that NVFP4 is a "derived tensor" needing a per-tensor F32 scale multiplied by a per-block F8 scale: F32_act = F4 x F8 x F32. ORippler's assessment: the "current quantize_nvfp4/dequantize_row_nvfp4 functions inside GGML are incorrect when F32 != 1.0", and the dequantization responsibility is "shared between GGML and its callee... undocumented in GGML and is, in my eyes, error prone."
So the honest summary is that a model tagged NVFP4 is substantially not in NVFP4, by intent, and the GGML implementation of the parts that are has open correctness questions. We have not counted the tensors in any specific file ourselves. That would take a gguf-dump pass, and we have not run one.
What this means if you are choosing hardware
The practical reading, for a local-AI setup rather than a datacentre:
On a recent Mac, NVFP4 is a first-class path. It arrives through the MLX engine, it is what Ollama's own performance work has been aimed at, and the model library has real coverage. Confirmed NVFP4 tags today include Gemma 4 at 12b, 26b, 31b, e2b and e4b, and Qwen3.5 at 4b, 9b, 27b, 35b-a3b and 35b-a3b-coding, with Qwen3.6 NVFP4 tags also in circulation.
On Windows or Linux, Ollama's NVFP4 tags are closed to you regardless of your GPU. You can run community NVFP4 GGUFs through the llama.cpp engine instead, and on current evidence you should not expect a speed win on Blackwell.
If you are buying a card for NVFP4 specifically, note that the software lagged the silicon. Consumer and workstation Blackwell is SM 12.0, and SM 12.0 support trailed the datacentre SM 10.x parts in vLLM and CUTLASS. That is an availability gap rather than a hardware one, but it is the kind of gap that costs a quarter.
There is a plausible route to NVFP4 off macOS, and it runs through MLX rather than around it: MLX has a CUDA backend upstream, and Ollama has been wiring it. PR #16871, "mlx: update and fix CUDA JIT packaging", merged 23 June. PR #16894, "mlx: route fast quantized matmul on CUDA", is open. If that lands, the 412 stops making sense and this article needs rewriting.
Two caveats on the tags themselves
The -coding- NVFP4 tags are not Ollama fine-tunes. Per rick-github they are "otherwise unmodified" models with different default sampling parameters. Do not read coding as a distinct trained variant.
And they have no vision, "because the MLX backend they run on is not yet vision capable". If you were planning a multimodal local pipeline on an NVFP4 tag, that is the blocker.
The short version
- NVFP4 is a layout, not a chip capability. Blackwell makes it fast; Blackwell is not required to make it work.
- In Ollama, NVFP4 arrived via the MLX engine, which is macOS-only. Pulling an NVFP4 tag elsewhere returns
412: this model requires macOS, enforced at the catalogue. - Published measurements have NVFP4 losing on decode on both platforms: 6.29 against 6.62 tok/s on an M3 (MLX), 136 against 188 t/s on Blackwell (GGUF).
- The files are not smaller. 18 GB against 18 GB on Ollama's own Gemma 4 26B tags; a community NVFP4 build lands near 5.60 bpw.
- The reason is documented and deliberate: NVIDIA's recipe is named
nvfp4_experts_only, and auxiliary tensors stay BF16/F32. - MLX-on-CUDA work is in progress and would invalidate the macOS restriction.
For what the format is at the bit level, including the widely repeated range error, see NVFP4 does not range from -8 to 7.5. For where MLX and llama.cpp actually diverge on a Mac, see MLX vs llama.cpp vs MetalRT.