I sent Ollama a generation request with "num_ctx": 16384 in the payload and got HTTP 200 and a normal-looking completion back. The model was running with a 4,096-token context the whole time. The key was at the wrong level of the JSON, and no error surfaced anywhere.
TL;DR — Ollama’s /api/generate and /api/chat only read num_ctx and num_predict from inside the "options" object. At the root of the payload they are dropped silently: the request succeeds, the answer looks fine, and the model loads with the default 4,096-token context, so long prompts get truncated without any error. The CONTEXT column of ollama ps is where the truth shows up.
The setup
I was debugging an article-generation pipeline: a Python script that sends one large prompt (about 2,700 tokens of instructions plus source material) to a local model through Ollama, a server that runs LLMs on your own machine and exposes an HTTP API on port 11434. Two request parameters matter here. num_ctx is the context window the model gets loaded with — prompt and answer must both fit inside it, and Ollama’s default is 4,096 tokens. num_predict caps how many tokens the model may generate. The pipeline’s articles kept coming back cut short even though the payload clearly said "num_ctx": 16384. Everything below was reproduced with curl against Ollama 0.32.14 and a small local model, nemotron-3-nano:4b.
What I expected
I expected misplaced parameters to either work or fail loudly. Plenty of Ollama parameters do live at the top level of the payload — model, prompt, system, stream, format, keep_alive — so a flat payload with num_ctx next to them is a natural thing to write, and most HTTP APIs would answer a misplaced or unknown field with a 400.
What actually happens
Root-level num_ctx on a freshly started model:
curl -s http://127.0.0.1:11434/api/generate -d '{
"model": "nemotron-3-nano:4b",
"prompt": "Say OK.",
"stream": false,
"num_ctx": 16384
}'
# HTTP 200, normal completion
$ ollama ps
NAME ID SIZE PROCESSOR CONTEXT UNTIL
nemotron-3-nano:4b 6cc467f05439 2.8 GB 100% GPU 4096 4 minutes from now
The request asked for 16,384. The model loaded with 4,096. Same result with /api/chat and a root-level num_ctx: HTTP 200, CONTEXT 4096.
num_predict gave me a second observable signal, done_reason in the response body. At the root it is ignored; inside "options" it bites:
# root level: ignored
# -> "done_reason": "stop", "eval_count": 78, full four-season answer
curl -s http://127.0.0.1:11434/api/generate -d '{"model": "nemotron-3-nano:4b",
"prompt": "List the four seasons of the year.", "stream": false, "num_predict": 5}'
# inside options: applied
# -> "done_reason": "length", "eval_count": 5
curl -s http://127.0.0.1:11434/api/generate -d '{"model": "nemotron-3-nano:4b",
"prompt": "List the four seasons of the year.", "stream": false,
"options": {"num_predict": 5}}'
"done_reason": "length" means the generation cap was actually enforced; "stop" with eval_count: 78 means my cap of 5 never existed as far as the server was concerned. (Aside: the capped run returned an empty response string — this model spends its first tokens on internal reasoning, so all 5 went there. eval_count is the reliable field.)
The leniency goes one level deeper. A typo inside options is also accepted silently:
curl -s http://127.0.0.1:11434/api/generate -d '{"model": "nemotron-3-nano:4b",
"prompt": "Say OK.", "stream": false, "options": {"num_ctxx": 16384}}'
# HTTP 200 — and ollama ps shows CONTEXT 4096 again
So neither a misplaced key nor a misspelled one produces any error. Reminded me of a WordPress REST API that answered 200 while enforcing none of the capabilities I thought it did — success responses prove less than they appear to.
The fix
Put sampling and context parameters inside "options", and verify with ollama ps after a request:
# force-unload so the next request decides the loaded context
curl -s http://127.0.0.1:11434/api/generate \
-d '{"model": "nemotron-3-nano:4b", "keep_alive": 0}'
curl -s http://127.0.0.1:11434/api/generate -d '{
"model": "nemotron-3-nano:4b",
"prompt": "Say OK.",
"stream": false,
"options": {"num_ctx": 16384, "num_predict": -1}
}'
$ ollama ps
NAME ID SIZE PROCESSOR CONTEXT UNTIL
nemotron-3-nano:4b 6cc467f05439 3.1 GB 100% GPU 16384 4 minutes from now
CONTEXT 16384, and the loaded size grew from 2.8 GB to 3.1 GB — the memory for the larger attention cache is real, which is presumably why Ollama does not default to a big window. In a pipeline, two cheap assertions catch both halves of this bug: check ollama ps (or log it) after the first generation to confirm the context you asked for, and treat "done_reason": "length" as a warning that output was cut by the token cap.
Why it works
Ollama’s API distinguishes request-level fields (model, prompt, stream, …) from model runner options, and only the "options" map reaches the runner. The JSON decoding is lenient in both places: a top-level key it does not recognize is dropped rather than rejected, and unknown keys inside options are skipped the same way. That is a deliberate compatibility choice in many APIs, but combined with a defaulted num_ctx it fails dangerously: the server does not error, the model does not error, and prompt truncation at generation time is also silent from the client’s point of view. The only client-visible symptom of my original bug was output quality — the ~2,700-token prompt plus a long completion did not fit in 4,096 tokens, and the model produced truncated, degraded articles. Silent success is the failure mode that gets past tests, and here it got past three layers of them.
What I did not test
Only Ollama 0.32.14, on Linux, with one model, via curl and stream: false. I always unloaded with keep_alive: 0 between context tests, so I did not check whether changing options.num_ctx on an already-loaded model triggers a reload on its own. I did not test the OpenAI-compatible /v1/chat/completions endpoint, which has its own parameter mapping. Parameter placement is documented in the Ollama API reference — the docs are right, they just never mention what happens when you get it wrong.
Facts
context: local Ollama server (0.32.14) driven over HTTP /api/generate and /api/chat
problem: num_ctx and num_predict at the JSON root are silently ignored; HTTP 200, model loads with default 4096 context, long prompts truncate with no error
solution: put them inside the "options" object; verify with ollama ps CONTEXT column and done_reason ("length" = num_predict enforced)
verified_on: 2026-08-17
applies_to: [Ollama 0.32.14, /api/generate, /api/chat, Linux]
does_not_apply_to: [OpenAI-compatible /v1/chat/completions endpoint (not tested), streaming responses (not tested)]