Two identical requests fired at one LM Studio endpoint on a studio test machine finished at 310 seconds and 725 seconds. Alone, the same request had finished in 306. The server’s total output rate was 11.2 tokens a second with two callers and 11.3 with one.
That is the whole answer to what LM Studio concurrent requests cost: two agents can share one local endpoint, and they will each get half of it. Here is the measurement, and what it means for an agent design on a 48 GB machine.
How two agents were pointed at one LM Studio endpoint
The endpoint was LM Studio’s OpenAI compatible server with gemma-4-31b-qat loaded at 64K context, the setup every measurement on this stack has used. The request was the 16 by 16 ARC-AGI-3 board from the reasoning effort tests, four legal actions, one action back, a 16,384 token cap so that nothing could be truncated.
First the request ran alone. Then two threads sent the identical request within the same millisecond and each timed its own response. Nothing else was using the model.
What LM Studio concurrent requests do to each request’s speed
| Run | Wall clock | Reasoning tokens | Output tokens a second |
|---|---|---|---|
| Solo | 305.8s | 3,450 | 11.3 |
| Paired, finished first | 310.6s | 1,803 | 5.8 |
| Paired, finished second | 725.4s | 6,343 | 8.8 |
The reasoning counts differ by a factor of three and a half between the two paired calls, which is the same run to run spread this model shows on identical input and has nothing to do with concurrency. The per second rates are the signal. The call that finished first ran its whole life alongside the other and got 5.8 tokens a second, half the solo rate. The call that finished second got half speed for its first 310 seconds and full speed for the remaining 415, which blends to 8.8.
Add the two paired outputs, 1,810 and 6,350 tokens, and divide by the 725 seconds the pair took: 11.2 tokens a second. The server did exactly as much work per second with two callers as with one.
The cap matters to reading this correctly. All three calls had 16,384 tokens of room and all three finished with finish_reason: stop, so no number above is a truncation artefact. And the wall clock of a paired call on its own says nothing: 310 seconds looks like the solo time until you notice it produced half the tokens. Divide output tokens by elapsed seconds for every call, always; it is the only figure that separates a slow model from a shared one, and it is the figure that a control ladder reads for the same reason.
Why the combined throughput did not change
A 31B model on this hardware is bound by memory bandwidth, not by how many requests are queued. Every generated token reads the whole set of weights once, and the machine can do that about eleven times a second whether the tokens belong to one conversation or two. LM Studio’s server accepted both requests and interleaved them, which is the fair thing to do, and fairness here means each caller waits twice as long.
There is no batching win to be had at this size on this machine. Batching two sequences would let each weight read serve two tokens, but only if the server does it and only if the arithmetic rather than the bandwidth were the limit; the measured rate says neither applied. LM Studio’s endpoint documentation describes the request shape and says nothing about scheduling, and the numbers are the only documentation of what it does under load.
Compare the ARC-AGI-3 API’s own rate limit, 600 requests a minute per key. A local 31B answers about one request every five minutes. The bottleneck for a local agent is never the benchmark’s server; it is the eleven tokens a second at home, and sharing them does not create more.
When two agents on one endpoint makes sense
When the two agents are not both waiting on the model at once. An agent that thinks for five minutes and then plays two thousand actions locally in a second leaves the endpoint idle for that second, and a second agent can use it. Two per move agents, each wanting a model call every turn, simply halve each other’s speed; two agents that call the model rarely and play fast between calls barely notice each other.
That is the same shape as the design the ARC-AGI-3 milestone winners used and the one a per move agent measured here argues for from the other direction. Move the expensive call out of the inner loop and the endpoint’s eleven tokens a second stretch across as many agents as have something to do while they wait.
When the agents are heterogeneous. A small model for the frequent decision and the 31B for the rare one would share the memory rather than the endpoint, and on this machine two large models do not fit at once; a small one beside the 31B is a different question that this run did not test.
What to do instead on a 48 GB machine
Serialise deliberately rather than by accident. Two agents sharing an endpoint without knowing it will each report the model as twice as slow as it is, and any latency figure taken during the overlap is wrong. If two things must share, put a queue in front of the endpoint so that each call runs at full speed and the wait is visible, rather than letting the server interleave and hide it.
Measure the solo rate first and keep it. On this stack it is 11.3 output tokens a second for the 31B, and every design decision about how often an agent may call the model is a division by that number. Two agents divide it by two. Four divide it by four. The number that does not move is the one the machine can do, and it is the one to plan around, as it is for everything else this hardware does with the benchmark.