Handle None content from reasoning models - #60
Open
tmartin2113 wants to merge 1 commit into
Open
Conversation
`message.content` is None when a model returns no text at all. Reasoning
models do this routinely: the reasoning tokens exhaust max_tokens before any
content is emitted, and the response comes back with finish_reason='length'
and content=None. Three call sites assume a string, and each fails
differently:
utils/llm_client.py chat()
re.sub(..., None) raises TypeError. There is no try/except in chat(),
so this propagates to the caller as an unhandled error.
services/oasis_profile_generator.py
services/simulation_config_generator.py
_fix_truncated_json(None) raises AttributeError on None.strip(). That
IS caught, but by the generic `except Exception` handler, so a
truncated response is logged as "LLM call failed" and burns every
retry with backoff sleeps before failing with a misleading error.
The irony is that finish_reason='length' is precisely the case the code just
below each of these lines is written to recover from -- the recovery path is
the one that breaks.
Defaulting to "" lets each site take the path it already has: chat() returns
an empty string, and the generators hit json.loads("") -> JSONDecodeError,
which routes into the existing JSON-repair and retry logic instead of the
wrong branch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
message.contentisNonewhenever a model returns no text at all. Reasoning models do this routinely — the reasoning tokens exhaustmax_tokensbefore any content is emitted, and the response arrives withfinish_reason='length'andcontent=None.Three call sites assume a string, and they fail in two different ways:
utils/llm_client.pychat()re.sub(..., None)→TypeError: expected string or bytes-like object, got 'NoneType'. There's notry/exceptinchat(), so it propagates to the caller.services/oasis_profile_generator.py_fix_truncated_json(None)→AttributeError: 'NoneType' object has no attribute 'strip'services/simulation_config_generator.pyThe two generator cases are arguably worse than the crash. The
AttributeErroris caught — but by the genericexcept Exceptionhandler, so a truncated response gets logged asLLM call failed, burns every retry with backoff sleeps, and finally fails with an error that points at the wrong thing entirely.The irony:
finish_reason == 'length'is precisely the case the code immediately below each of these lines is written to recover from. The recovery path is the one that breaks.Reproduction
Easiest way to hit it in practice is a reasoning model with a modest
max_tokens— the model spends the budget thinking and returns no content.Change
Default to
""at all three sites. That's deliberately the minimal fix: it lets each site take the path it already has rather than adding new branches.chat()returns an empty string, which callers already handlejson.loads("")→JSONDecodeError→ the existing JSON-repair and retry logic, which is the correct path for a truncated responseTesting
Verified both failure modes reproduce against the exact expressions in the current code, and that the guard resolves each:
re.subreturns'', and_fix_truncated_json('')returns''instead of raising.Independent of #59 — different files' concerns, no overlap.
🤖 Generated with Claude Code