fix: send trailing system-role message as user on OpenAI-compatible wire (#1395) - #1396
Merged
Merged
Conversation
Behavioral tests added:
- TestMapMessagesNonLeadingSystemSentAsUser
- TestMapMessagesLeadingOnlySystemUnchanged (non-regression, already passes)
- TestCompleteWireBodyTrailingSystemAsUser (end-to-end wire body proof)
Test runner output (expected: two failing, one passing):
=== RUN TestMapMessagesNonLeadingSystemSentAsUser
client_trailing_system_test.go:45: trailing runtime_context message role = "system", want user (this is the #1395 fix)
--- FAIL: TestMapMessagesNonLeadingSystemSentAsUser (0.00s)
--- PASS: TestMapMessagesLeadingOnlySystemUnchanged (0.00s)
=== NAME TestCompleteWireBodyTrailingSystemAsUser
client_trailing_system_test.go:122: wire.Messages[last].Role = "system", want user (trailing system must ride as user on the wire)
--- FAIL: TestCompleteWireBodyTrailingSystemAsUser (0.00s)
FAIL
These tests will pass after the implementation in the next commit.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJGxhoFhA8JjkwZFcLGdS5
…ble wire Implementation for tests added in 5390d49. mapMessages() in internal/provider/openai/client.go now rewrites any system-role message that is not the first message to role "user" on the wire. This fixes DeepSeek (via OpenRouter), which returns an empty assistant message whenever the last message in the request has role "system" -- the harness appends the per-turn <runtime_context> block as a second, trailing system message to keep the cacheable prompt prefix unchanged (runner_step_engine.go buildTurnMessages). Position and content are unchanged; only the wire role differs. The leading system message (index 0) is untouched. Anthropic (extractSystem hoists every system message into the top-level system param) and buildTurnMessages are unaffected. Test runner output (expected: all passing): === RUN TestMapMessagesNonLeadingSystemSentAsUser --- PASS: TestMapMessagesNonLeadingSystemSentAsUser (0.00s) === RUN TestMapMessagesLeadingOnlySystemUnchanged --- PASS: TestMapMessagesLeadingOnlySystemUnchanged (0.00s) === RUN TestCompleteWireBodyTrailingSystemAsUser --- PASS: TestCompleteWireBodyTrailingSystemAsUser (0.00s) PASS ok go-agent-harness/internal/provider/openai 0.241s go test ./internal/provider/... ./internal/harness -race: all packages ok go vet ./internal/provider/... ./internal/harness/...: clean Behavioral tests covered: BT trailing-system-as-user (leading unchanged, non-leading rewritten, end-to-end wire body). Files changed: internal/provider/openai/client.go Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WJGxhoFhA8JjkwZFcLGdS5
…system-as-user Adds a regression test that would fail if the fix in 7a69e27 is reverted, proven by re-running it against the pre-fix client.go: on a real harness.Runner -> real openai.Client path, the wire body's trailing runtime_context message must be role "user", not "system". Also updates the prompts/compiled/system_prompt.txt comments to describe the OpenAI- compatible wire role vs. the harness's internal "system" role, and adds the docs/logs/engineering-log.md entry for the bug. Full targeted suite output: go test ./internal/provider/openai/... -run 'TestMapMessages|TestComplete|TestRunner' -v ok go-agent-harness/internal/provider/openai 0.235s (17/17 tests pass) go test ./internal/provider/... ./internal/harness -race ok all packages go vet ./internal/provider/... ./internal/harness/... clean (exit 0) go test ./internal/server/... -run TestRunSmoke ok go-agent-harness/internal/server 0.271s (fake-provider path unchanged) go test ./... (full suite): only pre-existing, unrelated failures in internal/acceptance/ptyrunner (real-PTY tests that fail identically on origin/main in this sandbox, confirmed by re-running one of them against a stash of these changes -- environment limitation, not caused by this PR). Regression scenarios covered: - Reverting the mapMessages fix makes TestMapMessagesNonLeadingSystemSentAsUser, TestCompleteWireBodyTrailingSystemAsUser, and TestRunnerRuntimeContextReachesOpenAIWireAsUser all fail (verified directly). - The integration test exercises the real Runner -> buildTurnMessages -> Complete -> mapMessages seam, not just a direct mapMessages() unit call. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WJGxhoFhA8JjkwZFcLGdS5
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Owner
Author
|
Live verification (coordinator, OpenRouter key): harnessd built from this branch, no proxy, model deepseek/deepseek-v4-flash via openrouter. Three consecutive runs of a write+read+reply prompt: all completed, 3 steps each, 0 llm.empty_response.retry events, file contents correct. Before this fix the same daemon config failed 3/3 with max_empty_responses. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1395
Summary
DeepSeek models via OpenRouter (and other OpenAI-compatible backends) returned an empty assistant message (1-3 tokens,
finish_reason: stop) whenever the request's last message had rolesystem, so every run died within three turns withmax_empty_responses. The harness intentionally places the per-turn<runtime_context>block last as asystem-role message (buildTurnMessagesininternal/harness/clone.go) to keep the cacheable system+tools+history prefix stable across turns. This PR changes only the OpenAI-compatible wire mapper:mapMessagesininternal/provider/openai/client.gonow sends anysystem-role message that is not the first message as roleuseron the wire, leaving content and position unchanged. The leading system message is untouched.Scope and issue reconciliation
Matches the issue's required fix exactly: mapper-level change (not
buildTurnMessages), trailing position preserved,prompts/compiled/system_prompt.txtcomment updated. Anthropic'sextractSystem(hoists every system message into the top-levelsystemparameter regardless of position) is unchanged, as specified. No model-quirk table was added -- the mapper approach worked, so the documented alternative (atrailing_system_as_userprovider quirk) was not needed.Impact analysis reconciliation
internal/provider/openai/client.gomapMessages: the only production code change. Every OpenAI-compatible-routed provider (openai, openrouter, deepseek, kimi, qwen, gemini-via-compat) shares this one client, so one fix covers all of them -- confirmed by grep, there is no secondchatMessage-mapping implementation.internal/provider/anthropic/client.go: not touched, confirmed unaffected by design (system hoisting is position-independent).internal/harness/clone.gobuildTurnMessages: not touched, per the issue's stated preference.Architecture and duplication check
Searched
rg RoleSystem internal/providerandrg "func mapMessages\|chatMessage{" internal/provider-- one mapper (openai/client.go), one hoisting function (anthropic/client.goextractSystem), no parallel catalog or duplicate wire-mapping logic exists or was introduced.Test-first evidence
Red command:
go test ./internal/provider/openai -run 'TestMapMessagesNonLeadingSystemSentAsUser|TestCompleteWireBodyTrailingSystemAsUser' -vObserved failure:
This proved the mapper forwarded the trailing system role verbatim -- the exact defect in the issue.
Green command: same command after the
mapMessagesfix -- all pass.TestMapMessagesLeadingOnlySystemUnchanged(single-leading-system case) passed before and after, guarding the non-regression contract.Regression/integration evidence:
TestRunnerRuntimeContextReachesOpenAIWireAsUserdrives a realharness.Runner(with a stubsystemprompt.Engine) through the realopenai.Client, asserting the wire body's trailing message is roleuser. Re-running this test against the pre-fixclient.go(checked out from the red commit) reproduces the failure, confirming it actually catches a revert.Verification evidence
go test ./internal/provider/openai/... -run 'TestMapMessages|TestComplete|TestRunner' -v-- 17/17 pass.go test ./internal/provider/... ./internal/harness -race-- all packages pass.go vet ./internal/provider/... ./internal/harness/...-- clean.go test ./internal/server/... -run TestRunSmoke-- pass (fake-provider path unchanged, no key/Docker needed).go test ./...: only pre-existing failures ininternal/acceptance/ptyrunner(real-PTY acceptance tests). Confirmed these fail identically onorigin/mainwithout this change (re-ran one against a stash of this branch's diff) -- a sandbox/environment limitation unrelated to this fix.Rollout and rollback
Wire-shape change only, no persistence or schema involved. Revert this PR (or revert the single
mapMessagescommit) to fully roll back; no data repair needed.Documentation
prompts/compiled/system_prompt.txt(lines ~7-10 and ~103): comments now describe the harness's internalsystemrole for the runtime_context block versus the OpenAI-compatible wire role ofuser, and note the Anthropic client is unaffected.docs/logs/engineering-log.md: new dated entry with cause, fix, and regression coverage, following the existing log format.Contract checklist
🤖 Generated with Claude Code
https://claude.ai/code/session_01WJGxhoFhA8JjkwZFcLGdS5