Skip to content

Handle None content from reasoning models - #60

Open
tmartin2113 wants to merge 1 commit into
nikmcfly:mainfrom
tmartin2113:fix/none-content-from-reasoning-models
Open

Handle None content from reasoning models#60
tmartin2113 wants to merge 1 commit into
nikmcfly:mainfrom
tmartin2113:fix/none-content-from-reasoning-models

Conversation

@tmartin2113

Copy link
Copy Markdown

Problem

message.content is None whenever a model returns no text at all. Reasoning models do this routinely — the reasoning tokens exhaust max_tokens before any content is emitted, and the response arrives with finish_reason='length' and content=None.

Three call sites assume a string, and they fail in two different ways:

Site Failure
utils/llm_client.py chat() re.sub(..., None)TypeError: expected string or bytes-like object, got 'NoneType'. There's no try/except in chat(), so it propagates to the caller.
services/oasis_profile_generator.py _fix_truncated_json(None)AttributeError: 'NoneType' object has no attribute 'strip'
services/simulation_config_generator.py same

The two generator cases are arguably worse than the crash. The AttributeError is caught — but by the generic except Exception handler, so a truncated response gets logged as LLM call failed, burns every retry with backoff sleeps, and finally fails with an error that points at the wrong thing entirely.

The irony: finish_reason == 'length' is precisely the case the code immediately below each of these lines is written to recover from. The recovery path is the one that breaks.

Reproduction

>>> import re
>>> content = None                      # what the API returns
>>> re.sub(r'<think>[\s\S]*?</think>', '', content).strip()
TypeError: expected string or bytes-like object, got 'NoneType'

>>> content.strip()                     # inside _fix_truncated_json
AttributeError: 'NoneType' object has no attribute 'strip'

Easiest way to hit it in practice is a reasoning model with a modest max_tokens — the model spends the budget thinking and returns no content.

Change

Default to "" at all three sites. That's deliberately the minimal fix: it lets each site take the path it already has rather than adding new branches.

  • chat() returns an empty string, which callers already handle
  • the generators reach json.loads("")JSONDecodeError → the existing JSON-repair and retry logic, which is the correct path for a truncated response

Testing

Verified both failure modes reproduce against the exact expressions in the current code, and that the guard resolves each: re.sub returns '', and _fix_truncated_json('') returns '' instead of raising.

Independent of #59 — different files' concerns, no overlap.

🤖 Generated with Claude Code

`message.content` is None when a model returns no text at all. Reasoning
models do this routinely: the reasoning tokens exhaust max_tokens before any
content is emitted, and the response comes back with finish_reason='length'
and content=None. Three call sites assume a string, and each fails
differently:

  utils/llm_client.py chat()
      re.sub(..., None) raises TypeError. There is no try/except in chat(),
      so this propagates to the caller as an unhandled error.

  services/oasis_profile_generator.py
  services/simulation_config_generator.py
      _fix_truncated_json(None) raises AttributeError on None.strip(). That
      IS caught, but by the generic `except Exception` handler, so a
      truncated response is logged as "LLM call failed" and burns every
      retry with backoff sleeps before failing with a misleading error.

The irony is that finish_reason='length' is precisely the case the code just
below each of these lines is written to recover from -- the recovery path is
the one that breaks.

Defaulting to "" lets each site take the path it already has: chat() returns
an empty string, and the generators hit json.loads("") -> JSONDecodeError,
which routes into the existing JSON-repair and retry logic instead of the
wrong branch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant