I asked a local Ollama model for a two-key JSON object with format: "json" set, and got back an empty string — with done_reason: "stop", as if everything had gone fine. On another run, the same flag handed me half a JSON object, cut in the middle of a sentence. Neither reply was an error; both were format: "json" doing exactly what it promises, which turned out to be less than I thought.
TL;DR — format: "json" constrains the grammar of what is sampled; it does not promise a parseable payload. A reasoning model can return an empty response (fix: "think": false), and any num_predict cap can cut the JSON mid-string (fix: close the open string and brackets, then parse with json.loads(s, strict=False)).
The setup
I maintained two content pipelines that call a local Ollama server (POST /api/generate) for short structured answers — a title, a body, a score — and parse the reply as JSON. Three request fields matter for this story. format: "json" asks the server to constrain token sampling to a JSON grammar. think toggles the thinking channel: a reasoning model (here nemotron-3-nano:4b) first emits deliberation tokens into a separate thinking field of the reply, then writes its actual answer into response. And options.num_predict caps the number of generated tokens. Every non-streaming reply also carries done_reason: "stop" when the model chose to end its turn, "length" when the cap ended it.
What I expected
format: "json" reads like a guarantee: constrain generation to a grammar, receive a parseable object. I expected the worst case to be bad JSON — wrong keys, invented structure — never no JSON at all. And I assumed a token cap was harmless with the grammar active: if every token must be legal JSON, surely the turn ends at a valid point.
What actually happens
import json, urllib.request
def gen(payload):
req = urllib.request.Request("http://127.0.0.1:11434/api/generate",
data=json.dumps(payload).encode())
return json.load(urllib.request.urlopen(req))
d = gen({"model": "nemotron-3-nano:4b", "stream": False, "format": "json",
"prompt": "Return JSON with keys title, body. Subject: python decorators."})
print(d["done_reason"], len(d["response"]), len(d["thinking"]))
# stop 0 332
Three runs, three identical results: done_reason: "stop", response empty, thinking between 271 and 757 characters. The curious part is what the thinking contained: { "title": "python decorators", "body": "Decorators in Python are a powerful language feature… — the JSON I asked for, shaped by the grammar, written into the wrong channel. The model spent its whole turn thinking and ended without ever reaching response. From the calling code’s side this is indistinguishable from « the model is too weak for structured output »: no error, no payload, stop.
Adding "think": false fixed it instantly — response came back with 570 and 843 characters of JSON on my two runs, and both parsed.
Second failure, same request plus "think": false and "options": {"num_predict": 40}:
{
"title": "Python Decorators Explained",
"body": "Python decorators are a powerful feature that allow you to modify or enhance the behavior of functions. A decorator is
done_reason: "length", 173 characters, no closing quote, and json.loads raised Unterminated string starting at: line 3 column 11 (char 54). The grammar guarantees every sampled token extends a legal JSON prefix; when the cap hits, a legal prefix is all you have.
The fix
For the empty response: pass "think": false in the request body (top level, next to format — not inside options). For the truncated fragment, a conservative repair recovers the payload without re-running inference:
import json
def close_truncated_json(s: str) -> str:
"""Close an open string, drop dangling commas, balance brackets."""
s = s.rstrip().rstrip(",")
in_str, esc, stack = False, False, []
for ch in s:
if in_str:
if esc: esc = False
elif ch == "\\": esc = True
elif ch == '"': in_str = False
elif ch == '"': in_str = True
elif ch in "[{": stack.append(ch)
elif ch in "]}" and stack: stack.pop()
if esc: s = s[:-1] # dangling backslash at the cut
if in_str: s += '"' # unclosed string
s = s.rstrip().rstrip(",")
return s + "".join("]" if c == "[" else "}" for c in reversed(stack))
raw = d["response"] # the 173-char done_reason:"length" fragment from above
obj = json.loads(close_truncated_json(raw), strict=False)
# {'title': 'Python Decorators Explained', 'body': 'Python decorators are…'}
This returned both keys, body intact up to the cut. One of the two pipelines runs this repair in production behind ten unit tests, all green this session.
strict=False matters on its own: some models put a raw newline inside a JSON string, and the default parser rejects it:
json.loads('{"title": "line one\nline two"}')
# json.decoder.JSONDecodeError: Invalid control character at: line 1 column 20 (char 19)
json.loads('{"title": "line one\nline two"}', strict=False)
# {'title': 'line one\nline two'}
One flag, and the most common « Invalid control character » from LLM output disappears.
Why it works
format: "json" is constrained decoding: at each step, tokens that would break the JSON grammar are masked out. That constraint says nothing about which channel the tokens land in, and nothing about the turn ending at a complete document. Here the constrained tokens went to the thinking channel and the model stopped there — I can only report the observation, not Ollama’s internals. num_predict is a hard cap applied regardless of grammar state, so it cuts wherever the budget runs out, including mid-string. The practical contract is done_reason: "stop" plus a non-empty response; check both before parsing. This is the second time an Ollama request answered 200 while meaning less than it seemed — the first was num_ctx silently ignored when it sits outside options.
What I did not test
Only nemotron-3-nano:4b on Ollama 0.32.14, non-streaming /api/generate. I did not test /api/chat, other reasoning models, or format with a full JSON schema instead of "json". The repair is deliberately conservative: a fragment cut inside a key name still fails after closing brackets — cleanly, but I did not measure how often that happens.
Facts
context: local Ollama inference with format:"json" for structured output, reasoning model with a thinking channel
problem: response is empty with done_reason "stop" (model thinks its whole turn), or num_predict cuts the JSON mid-string with done_reason "length"
solution: pass "think":false for reasoning models; repair truncated payloads (close string, balance brackets) and parse with json.loads(strict=False)
verified_on: 2026-08-17
applies_to: [Ollama 0.32.14, nemotron-3-nano:4b, /api/generate non-streaming, Python 3.14]
does_not_apply_to: [/api/chat (untested), format with a JSON schema (untested), non-reasoning models (no thinking channel to steal the turn)]