There is a moment, about forty minutes into a 22 GB download on home broadband, when you start wondering what is actually in the file. For NVIDIA’s Nemotron 3.5 Lightning, released on 11 August 2026 under the OpenMDW 1.1 licence, the answer turned out to be worth checking: the official 4-bit GGUF is 22,459,679,904 bytes, and 2.67 GB of that is stored at BF16, full 16-bit precision. All of it sits in one block at the end of the model, and that block is a multi-token prediction head that most people running the model locally will never turn on.
We read the tensor tables of four separate publishers’ builds directly, over HTTP range requests, without downloading the weights. The header and the tensor-info table sit at the front of a GGUF file, so roughly 8 MB of each file is enough to enumerate every tensor, its shape and its quantisation type. This is the same method behind our GGUF quant naming audit, and it answers a question the file listing cannot.
What is actually inside the Nemotron 3.5 Lightning GGUF?
The architecture string in the file is nemotron_h_moe, and the metadata describes a model that looks nothing like a standard transformer. There are 53 blocks. Only seven of them carry attention weights, at block indices 5, 12, 19, 26, 33, 42 and 52. Twenty-three blocks are Mamba-2 state-space layers, carrying ssm_conv1d, ssm_in, ssm_out and friends. Twenty-four are mixture-of-experts blocks with 128 experts each, of which 6 are active per token, plus one shared expert that always runs.
The attention spacing is worth a second look, because it is not uniform. The gaps run 7, 7, 7, 7, 9, 10. The first five attention layers arrive on a regular beat and then the spacing stretches towards the end of the model. Anyone assuming a clean “one attention layer every seven blocks” rule when planning a layer split across devices would be wrong twice, at blocks 42 and 52.
Now the quantisation. In NVIDIA’s own NVFP4 build, exactly 93 tensors are NVFP4. Those are the expert weights and the output head. Every attention weight, every SSM weight and the token embedding are BF16, untouched. That is the first thing worth knowing about this model: a “4-bit” Nemotron is 4-bit only where the experts live, which is most of the parameters but nowhere near all of the file.
Here is the size breakdown we computed from the tensor table, in gigabytes:
| Component | Size | Share of file |
|---|---|---|
| MoE experts, blocks 1 to 51 | 14.95 GB | 72.7% |
| Block 52, the MTP head | 2.67 GB | 13.0% |
| Mamba-2 SSM layers | 1.78 GB | 8.7% |
| Token embedding | 0.70 GB | 3.4% |
| Attention layers, all seven | 0.28 GB | 1.4% |
| Output head | 0.18 GB | 0.9% |
The seven attention layers cost 0.28 GB. The single MTP block costs almost ten times that.
Why does block 52 cost 2.67 GB in a 4-bit file?
Block 52 is doing three jobs at once. It holds the seventh and final attention layer, a full MoE block with all 128 experts, and four tensors named nextn.eh_proj, nextn.enorm, nextn.hnorm and nextn.shared_head_norm. The metadata key nemotron_h_moe.nextn_predict_layers is set to 1, which is the model declaring that it carries one multi-token prediction layer.
Multi-token prediction is the mechanism behind self-speculative decoding: the head guesses the next token or two ahead of the main model, and the main model verifies the guess in a single pass. When it works, you get tokens faster for free. We covered how this head ships inside Qwen builds in the Qwen GGUF MTP head piece and the two packaging patterns in MTP draft models. Nemotron 3.5 Lightning is the clearest case yet.
The reason the block is expensive is that the quantiser skipped it. Every expert tensor in blocks 1 through 51 is NVFP4. In block 52, ffn_down_exps.weight, ffn_up_exps.weight, ffn_down_shexp.weight and ffn_up_shexp.weight are all BF16. The nextn.eh_proj.weight tensor, shaped 5376 by 2688, is BF16 too. A whole 128-expert block sat out the quantisation pass.
We can prove the cost rather than assert it. A community rebuild, waseigo/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-noMTP-GGUF, is the same NVFP4 file with block 52 removed: 494 tensors instead of 510, a maximum block index of 51 instead of 52, and no nextn tensors at all. It weighs 19,788,999,872 bytes. The difference between the two files is 2,670,680,032 bytes. Our estimate from the tensor table alone was 2.67 GB. The arithmetic and the byte counts agree, which means the MTP head is 11.9 percent of what you download.
Which Nemotron 3.5 Lightning GGUF should you download?
Four publishers made four different decisions about that block, and the choice changes what you get.
NVIDIA and ggml-org keep the head inside the model at BF16. The NVFP4 file is 22.46 GB and the Q4_K_M is 25.43 GB. Bartowski keeps the head inside the main file but quantises it to Q4_0, and then also ships it separately as mtp-NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q4_0.gguf at 1.16 GB and a Q8_0 version at 2.18 GB. That separate file is 19 tensors, which is one block. Unsloth keeps the head inside and puts nextn.eh_proj at Q8_0. The waseigo rebuild deletes it.
Bartowski’s main build is the interesting one for a different reason. The file is called Q4_K_M, and its tensor table contains 12 Q4_K tensors out of 417. The expert weights are 11 Q8_0, 35 Q5_0 and 2 Q4_0. There is not a majority of Q4_K anywhere in a file named Q4_K_M, which is the same naming problem we documented in the quant naming piece and the same reason we keep reading tensor tables instead of filenames.
One trap to avoid if you repeat this audit. The general.file_type key in NVIDIA’s NVFP4 build reads 39, and 39 in the ggml tensor-type enum is MXFP4. It is a different enum. Checked against include/llama.h in llama.cpp, LLAMA_FTYPE_MOSTLY_NVFP4 is 39 and LLAMA_FTYPE_MOSTLY_MXFP4_MOE is 38, so the file is correctly labelled and the collision is a coincidence of numbering. Reading the file-type key with the tensor-type table produces a confident wrong answer.
Does Nemotron 3.5 Lightning fit on a 48 GB Mac?
On paper, comfortably. The NVFP4 build is 22.46 GB of weights and the model activates 3B parameters per token, so a studio work stack with 48 GB of unified memory has room for the weights and a working context.
The caveat is the context claim. NVIDIA’s model card gives the model up to 1M tokens and then notes that for a single H100 deployment they use 256K. The GGUF metadata declares context_length at 1048576. A hybrid Mamba and attention model does not hold context the way a pure transformer does, since the SSM layers carry a fixed-size recurrent state rather than a growing KV cache, and only seven layers here have a KV cache at all. That makes the memory curve genuinely different from the MoE sizing we measured in MoE sizing on unified memory, and it is the next thing we intend to measure rather than estimate.
If you are not running speculative decoding, the practical move today is the 19.79 GB build without the MTP block, or bartowski’s separate MTP file kept aside until you want it. If you are, keep the head, and know that NVIDIA’s version of it is the only one still at full precision.
The wider point is that “30B-A3B, 4-bit, 22.5 GB” describes almost nothing about a file whose largest single block is a speculative-decoding accessory stored at four times the precision of everything around it. The tensor table took eight megabytes to read. The download it saves is nearly three gigabytes.