Ollama MTP: the speculative decoding that is already on, and the DRAFT command nobody documented

If you run Gemma 4 on Apple Silicon through Ollama, you have been getting speculative decoding since 30 June, whether you configured it or not. Ollama’s own figure is nearly 90% faster token generation across a coding-agent benchmark, on by default, no flags.

There is also a DRAFT instruction in the Modelfile parser that works and appears nowhere in the Modelfile documentation. We will get to that.

First, a terminology correction, because the phrase people search for is not a real term.

“MTP drafter” is not a thing

MTP is real and established. Multi-token prediction. llama.cpp ships --spec-type draft-mtp, documented as “Use Multi Token Prediction (MTP) heads from the main model.” Ollama’s release notes and blog use it throughout. vLLM has a whole vllm/v1/spec_decode/ directory.

“Drafter” is informal. We grepped for it. The load-bearing terms in primary sources are draft model (llama.cpp’s --model-draft, Ollama’s DRAFT) and proposer, which is what vLLM’s code actually calls these things (ngram_proposer.py, and zero occurrences of “drafter” in eagle.py). “Drafter” lives in blog prose, not in any spec or codebase we checked.

If you are writing this up, “MTP draft head” or “MTP module” is the safe phrasing.

One more wart worth knowing so you do not repeat it: Ollama’s v0.23.1 release note heading expands MTP as “Multi-token Processing“. That is a typo on Ollama’s side. The pull request, the parameter docs and the June blog all say prediction.

What it actually does

A draft model is a small fast model that proposes several tokens ahead. The large target model then verifies the whole proposal in one batched forward pass and keeps the prefix it agrees with.

The critical property: the output does not change. This is a pure latency optimization, not a quality trade. Ollama states it directly: the speedup “does not change the model’s output.”

MTP is the variant where the draft head is not a separate model at all. It ships *inside* the main model as extra tensors, trained alongside it. Gemma 4 has them. That is why Ollama can turn this on with no configuration and no second download.

The timeline

Ollama PR #15980, merged 5 May by pdevine, is the one to read. It added the MTP path, the DRAFT Modelfile instruction, safetensors draft-model import, and --quantize-draft.

The v0.31.1 note is the one that matters for daily use: “Ollama auto-tunes how many tokens to draft as it runs, so the speedup is on by default, requires no configuration, and does not change the model’s output.”

Worth noting how it got fast: Ollama contributed a batched-draft-token matmul kernel upstream to MLX. On an M5 Max with NVFP4 weights, that makes Gemma 4’s largest matmuls 2 to 2.5x faster. This is the same MLX-only story as NVFP4 on Apple Silicon, the interesting local-inference work is landing in the Apple path first.

The undocumented part

DRAFT is a valid Modelfile instruction. We verified it in parser/parser.go, where it sits in the valid-command list alongside from, adapter and template.

It is not in the public Modelfile reference.

This is the same shape as the OpenJarvis analytics switch we found in our retest: a working control that the documentation does not mention. It is shipped, it is parsed, and you would only find it by reading the source or the pull request.

What *is* documented is the parameter, and it carries a trap:

> draft_num_predict: “Maximum number of speculative draft tokens to predict per step when a draft model is available. Separate draft models default to 4; embedded MTP tensors require setting this parameter. Set to 0 to disable speculative drafting.”

Read that second clause twice. A separate draft model gets a default of 4. Embedded MTP tensors require you to set the parameter. Since v0.31.1 auto-tunes the draft count, these two statements are in tension, and we could not resolve from the docs alone which wins on a current build. If you are relying on MTP and have not touched draft_num_predict, that is worth measuring rather than assuming.

To turn it off entirely: draft_num_predict: 0.

Which models have MTP heads

DeepSeek-V3 is the canonical originator of the sequential-MTP design, in its technical report (arXiv 2412.19437).

Qwen is the frustrating case. The Qwen3.6-27B model card lists “MTP: trained with multi-steps” and gives serving flags for other stacks: vLLM’s --speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}', SGLang’s --speculative-algo NEXTN. But Ollama issue #17019, 3 July 2026, is about Qwen3.5 and 3.6 safetensors whose mtp.* tensors were corrupting MLX imports. The fix dropped them. So Qwen ships the heads and Ollama currently discards them.

If you want MTP speedups on Ollama today, that means Gemma 4.

Lower-confidence, from survey material rather than individual model cards: GLM-5, Qwen3-Next-80B-A3B, Nemotron 3 Super, MiniMax M2.7 and MiMo are also reported to ship MTP heads. Treat those as leads, not facts.

The gap: nothing for GGUF

There is still no general speculative decoding on Ollama’s llama.cpp engine.

Ollama issue #5800, “Enable speculative decoding”, has been open since 19 July 2024 with 64 comments. The community pull request #8134 was closed unmerged in January 2025.

llama.cpp itself has the full menu: --spec-type none|draft-simple|draft-eagle3|draft-dflash|draft-mtp|ngram-*, plus -md/--model-draft, all in its docs/speculative.md. Ollama bundles llama.cpp and does not expose any of it.

So the honest state of play: MTP on Ollama is a macOS, MLX, Gemma 4 feature. Everywhere else in Ollama, speculative decoding is a two-year-old open issue. If you need it on GGUF, run llama.cpp directly.

What to check on your own machine

Five checks, in the order we intend to run them on the studio M4 Pro. None of them needs more than a Mac that already has Gemma 4 pulled.

1. Confirm it is actually on. Compare tok/s on Gemma 4 with draft_num_predict unset against draft_num_predict: 0. If there is no difference, MTP is not engaging and the auto-tune claim needs qualifying.
2. Resolve the draft_num_predict ambiguity. Does an explicit value beat auto-tune on a current build?
3. Verify output equivalence. Same seed, same prompt, MTP on and off. The claim is that output is identical; that is cheap to check and worth checking.
4. Test it under an agent, not a chat. Speculative decoding wins most on predictable token runs, and agent traffic is full of them. The tool-calling harness from our tool-calling piece is the right load.
5. Check acceptance rate, not just tok/s. A high draft count with poor acceptance can be slower than no drafting.

The short version

• MTP = multi-token prediction, a draft head shipped inside the model. “MTP drafter” is not a real term. Say “draft model” or “MTP draft head”.
• On by default since v0.31.1 (30 June 2026), auto-tuned, ~90% faster on Ollama’s coding-agent benchmark, output unchanged.
• macOS + MLX + Gemma 4 only. Qwen ships MTP tensors; Ollama drops them.
• DRAFT is a working, undocumented Modelfile instruction. draft_num_predict is documented and says embedded MTP “requires setting this parameter”, which sits oddly with auto-tune.
• Disable with draft_num_predict: 0.
• No speculative decoding on Ollama’s GGUF path. Open issue since July 2024. Use llama.cpp directly if you need it.