In September we pointed two agents at one local model and each got half of it. Two identical requests to Gemma 4 31B through LM Studio ran at half speed apiece, and the server’s total output stayed at about 11 tokens a second. On 12 October 2026 we repeated the test on LM Studio’s MLX engine and the second caller cost almost nothing: each request ran at 11.5 tokens a second and the total doubled to 23. LM Studio parallel requests, on the page that documents them, are still listed as supported on llama.cpp, with MLX to follow. This is what we measured on both engines, with two models, on a 48 GB Mac.
What the LM Studio parallel requests setting does on each engine
LM Studio loads every model with a setting called Max Concurrent Predictions. Its documentation page for parallel requests says the setting lets several requests be processed in parallel through continuous batching, where they would otherwise queue, and that the default is 4. When we read the page on 12 October 2026 it also said this is supported for LM Studio’s llama.cpp engine, “with MLX coming soon”.
Our loads matched the default. On LM Studio 0.4.25, lms ps reported a parallel value of 4 for every model in this test, on the llama.cpp engine at version 2.47.0 and on the MLX engine at version 1.11.0 alike. We changed nothing. LM Studio’s MLX engine is a separate project from its llama.cpp runtime: the mlx-engine repository lists Apple’s mlx-lm as the inference engine it is built with. So the same setting is carried out by two different pieces of software, and there is no reason to expect the same result from both.
A batch is the useful picture here. To write one token, a model passes over its weights once. If two requests can share that pass, the second request rides along and the total output rises. If they cannot share it, the two requests take turns and each gets half. Our September piece on two agents on one LM Studio endpoint found the second case on llama.cpp with Gemma 4 31B.
How we tested LM Studio parallel requests on MLX and llama.cpp
The test machine is an M4 Pro with 48 GB of unified memory on macOS 27.0.1. We used two models, each as a GGUF on llama.cpp and as a 4-bit build on MLX: Gemma 4 31B, a dense model, and Qwen3.6 35B-A3B, a mixture-of-experts model. Each build was loaded alone, because two large models generating together on 48 GB slow each other down and would have spoiled the timing.
One Python script sent a short question, about 72 prompt tokens, with a 500-token cap on the reply. It ran the question three times on its own. Then it started two copies at the same moment on two threads, each with a different random session id at the top so that neither could be answered from the other’s cache. The script streams each reply and times it from the stream, so tokens per second means the same thing on both engines. Temperature was 0.
The single-request figure we quote for each build is the middle of its three runs. The two-request figures come from one pair per build. That is a small sample, and we say below what it does not cover.
What two requests at once cost on Gemma 4 31B and Qwen3.6 35B-A3B
On MLX, close to nothing. On llama.cpp, it depended on the model.
Gemma 4 31B and Qwen3.6 35B-A3B on llama.cpp 2.47.0 and MLX 1.11.0: total tokens written per second with one request and with two at once. Tested 12 October 2026.
| Model and engine | One request | Two at once, each | Two at once, combined |
|---|---|---|---|
| Gemma 4 31B, llama.cpp | 11.7 | 5.7 | 11.4 |
| Gemma 4 31B, MLX | 12.6 | 11.5 | 23.0 |
| Qwen3.6 35B-A3B, llama.cpp | 49.4 | 39.4 | 78.8 |
| Qwen3.6 35B-A3B, MLX | 61.5 | 64.7 | 129.4 |
All figures are tokens written per second.
Gemma 4 31B on llama.cpp repeated the September result almost to the decimal. Alone it wrote 11.7 tokens a second. With a second caller each request got 5.7, and the pair took 91.5 seconds where one request had taken 43 to 45. The combined rate was 11.4, so the second caller added nothing.
The same model on MLX wrote 12.6 tokens a second alone and 11.5 for each of two callers. The pair finished in 38.2 seconds, against 34 to 36 seconds for one request. Combined output was 23.0 tokens a second, twice what llama.cpp produced with the same two requests.
Qwen3.6 35B-A3B behaved differently on llama.cpp. Alone it wrote 49.4 tokens a second. With two callers each got 39.4, so the combined rate rose to 78.8. Parallel requests do pay on llama.cpp for this model, at a cost of about a fifth of each caller’s speed. On MLX the cost was not measurable. One request ran at 61.5 tokens a second, and each of two ran at 64.8 and 64.6. The pair finished in 8.4 seconds, the same as a single request, with a combined rate of 129.4.
The wait before the first token grew a little under load on both engines. On Gemma it went from under 2 seconds to 3.4 seconds on llama.cpp and to about 2.5 seconds on MLX. On Qwen3.6 it stayed under 0.7 seconds with two callers on both engines.
We can describe the pattern, though the cause is beyond what this run shows. A dense model like Gemma 4 31B uses all of its weights for every token, and on llama.cpp two requests appear to compete for that pass. A mixture-of-experts model computes with a small active share of its weights, which may be why llama.cpp could overlap two Qwen3.6 requests and not two Gemma ones. The MLX engine overlapped both. We did not read either engine’s scheduler, so that explanation is a reading of the numbers and should be treated as one.
When to point two agents at one MLX model in LM Studio
The September result meant that a shared endpoint on a 48 GB machine only divided the same output between its callers. On the MLX engine that no longer follows from the numbers. Two callers on Gemma 4 31B got 11.5 tokens a second each, almost exactly the 11.7 that one caller gets on llama.cpp. If your work splits into two independent jobs, such as one agent that writes and one that checks, the MLX build of the model gives you the second one nearly free.
Three limits apply before anyone rebuilds a setup around this. We tested two callers and no more, although the setting allows four. We tested short prompts, and two agents that each carry a 20,000-token prompt will also hold two sets of cached context in memory, which we did not measure. And the switch to MLX changes more than the batching. Our comparisons of Qwen3.6 35B-A3B on MLX and llama.cpp and of the Gemma 4 31B MLX build and the GGUF list what else moved, including a structured output fault on one and missing reasoning on the other.
The documentation page still describes MLX support for parallel requests as on its way. On version 1.11.0 of the engine, on this machine, two requests were already sharing the model at full speed. The next thing to measure is the third and fourth caller, because the default allows them and a dense 31B model has to stop scaling somewhere.
