Disable model reasoning for NPC turns; give structured output headroom - #449
Merged
Conversation
First real-hardware conversation testing (Windows, bundled CPU llama-server, Qwen3-4B starter model) showed every turn degrading to the "I'm not sure what to say right now" fallback: Qwen3 spends hundreds of tokens on chain-of-thought before the JSON, the 512-token default budget truncates the object, parse + repair both fail, and the player never hears the model at all — at 2-4x the latency. - The managed llama-server now starts with LLAMA_ARG_REASONING=off (env var, not CLI flag: older user-installed binaries ignore unknown env vars but would refuse to start on an unknown flag). NPC turns want fast, in-character JSON, not visible thinking. - Turn requests get max_tokens=1024: rubric-rich turn output regularly lands in the 300-600 token range and the default left no headroom. Verified on the target machine: with these two changes the same scenario turn returns a real in-character Japanese reply from the starter model (no fallback), and the engine autostart path brings the model back after a core restart. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X4GogqwrcPc5M5VkM6wMWT
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First real-hardware conversation testing (Windows, bundled CPU
llama-server, Qwen3-4B starter model) showed every turn degrading to
the "I'm not sure what to say right now" fallback: Qwen3 spends
hundreds of tokens on chain-of-thought before the JSON, the 512-token
default budget truncates the object, parse + repair both fail, and the
player never hears the model at all — at 2-4x the latency.
(env var, not CLI flag: older user-installed binaries ignore unknown
env vars but would refuse to start on an unknown flag). NPC turns
want fast, in-character JSON, not visible thinking.
lands in the 300-600 token range and the default left no headroom.
Verified on the target machine: with these two changes the same
scenario turn returns a real in-character Japanese reply from the
starter model (no fallback), and the engine autostart path brings the
model back after a core restart.
Co-Authored-By: Claude Fable 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01X4GogqwrcPc5M5VkM6wMWT