Why your local model stops at exactly 8,192 tokens

Twelve of thirty six benchmark runs on a studio test machine stopped early. Every one of them stopped at the same number: 4,152 prompt tokens plus 4,040 generated, or 1,116 plus 7,076, or 332 plus 7,860. Three different boards, two different models, and in all twelve cases the two numbers add to exactly 8,192.

The token cap on those requests was 16,384. The cap never fired. The LM Studio context length did, and the giveaway was that a truncation limit you set yourself does not produce a round power of two when you add the prompt to it.

What the LM Studio context length actually limits

The number your request controls is max_tokens, and it bounds only what the model generates. The context window bounds the prompt and the generation together, because both live in the same buffer.

So there are two ceilings, and only one of them is in your request:

max_tokens        limits   out
context length    limits   in + out

When a run stops because of the second one, the response looks exactly like the first. finish_reason is length, the visible content may be empty, and the usage block reports a generation count that is nowhere near your cap. On a reasoning model, where the thinking is billed as output and arrives before the answer, the result is the same empty string an undersized cap produces, for a completely different reason.

Why two models hit it and one did not

The three models in that run were qwen3.5-9b, qwen3.5-27b and gemma-4-31b-qat, all Q4_K_M or Q4_0 GGUF builds from the same catalogue. Both Qwen models truncated. Gemma never did, across twelve runs on identical boards.

The difference was not the model. It was how each one got loaded.

Gemma had been loaded explicitly, some days earlier, with a 65,536 token context and left resident. The two Qwen models were never loaded by hand at all. They were requested by model id through the API, and LM Studio just-in-time loaded each one using its default context length, which on this install is 8,192.

That default is the whole story. A model you load yourself keeps the context you gave it. A model the server loads for you gets the default, and nothing in the response tells you which one you are talking to.

The documentation is half the trap. LM Studio’s page on lms load tells you the flag exists, that “You can set the context length when loading a model using the --context-length flag” and that it “determines how many tokens the model will consider as context when generating text”. What no page states is what you get when you omit it. The default is real, it is 8,192 on this install, and it is not written down anywhere you would look.

You can check directly. LM Studio’s own endpoint reports it:

GET http://localhost:1234/api/v0/models
  google/gemma-4-31b-qat    loaded_context_length=65536   state=loaded
  qwen/qwen3.5-9b           loaded_context_length=8192    state=loaded

That is the field to read before trusting any measurement, and it is not part of the OpenAI compatible /v1/models response, so a client written against the standard shape will never see it. LM Studio’s REST API documentation is explicit that its own endpoints carry capabilities the OpenAI compatible ones lack, and this is one of them: the compatibility layer gives you the request shape you already know, and quietly drops the one field that would have told you the measurement was wrong.

How to tell a context limit from a token cap

The arithmetic is the tell, and it costs nothing to check on every call.

Add the prompt tokens to the generated tokens. If the total is a round power of two, or the same number on every truncated call regardless of how large the prompt was, you are hitting the context window. If the generated count alone equals your cap, you are hitting max_tokens.

The twelve failures above make the pattern obvious once the sum is written down:

Read the generated column on its own and it looks like three different behaviours, with the largest board apparently thinking least. Read the totals and there is only one behaviour. The bigger the board, the less room was left to think in, which is the exact opposite of what the generated column appears to say and would have been a very confident wrong conclusion.

What this does to a benchmark

It ruins it quietly, which is worse than failing.

The run those numbers came from was measuring how much reasoning each model spends as a board grows. Gemma had eight times the room the Qwen models had, so the comparison was not between three models. It was between one model with space and two models in a smaller box. The apparent finding, that the Qwen models thought less on larger boards, was an artefact of the boards being larger and leaving less room.

Nothing in the data flagged it. Every request returned HTTP 200 with a well formed response and a plausible number in it. The only reason it surfaced was that two different models reported an identical 4,040 generated tokens on the same board, which is not a coincidence any sampling temperature produces.

It is worth being precise about how ordinary the mistake is. The cap was set to 16,384 deliberately, as headroom, after an earlier round of runs was truncated by a cap that was too small. Doubling the headroom fixed that problem and had no effect on this one, because the two ceilings are unrelated. A generous max_tokens cannot buy room that the context window does not have, and no amount of raising it will, which is the part that turns an afternoon into a day.

The measurements that survived are the ones that do not depend on where generation stopped. Throughput was unaffected: 32.3 tokens a second median for the 9B, 11.0 for Gemma 31B, 9.6 for the Qwen 27B, twelve runs each. Where a run ends does not change how fast the tokens came out before it ended.

What to set, and what to check

Three habits, in the order they pay off.

Load every model explicitly before you measure anything, and give it a context you chose:

lms load qwen/qwen3.5-9b --context-length 65536 --gpu max

Then verify it took, because a load that silently fell back to a default is the same failure again. On a 48 GB machine one large model at 64K is the practical limit, so a multi-model comparison means loading and unloading between models rather than keeping them all resident.

Log in + out on every call alongside finish_reason, the reasoning count and the wall clock. It is one extra field and it is the difference between a benchmark and a measurement of your own configuration.

The same care applies to any runtime with its own default. Ollama takes num_ctx per request and has its own idea of a sensible value; llama.cpp takes -c; a hosted API decides for you and usually publishes the number. In every case the thing to record is not what you asked for but what the server says it loaded.

And treat a round number with suspicion generally. 8,192 in a usage block is not a thing a model decided. It is a thing a buffer decided, and the model never had a say.