Skip to content

fix: send trailing system-role message as user on OpenAI-compatible wire (#1395) - #1396

Merged
dennisonbertram merged 3 commits into
mainfrom
issue-1395-trailing-system-as-user
Sep 6, 2026
Merged

fix: send trailing system-role message as user on OpenAI-compatible wire (#1395)#1396
dennisonbertram merged 3 commits into
mainfrom
issue-1395-trailing-system-as-user

Conversation

@dennisonbertram

Copy link
Copy Markdown
Owner

Closes #1395

Summary

DeepSeek models via OpenRouter (and other OpenAI-compatible backends) returned an empty assistant message (1-3 tokens, finish_reason: stop) whenever the request's last message had role system, so every run died within three turns with max_empty_responses. The harness intentionally places the per-turn <runtime_context> block last as a system-role message (buildTurnMessages in internal/harness/clone.go) to keep the cacheable system+tools+history prefix stable across turns. This PR changes only the OpenAI-compatible wire mapper: mapMessages in internal/provider/openai/client.go now sends any system-role message that is not the first message as role user on the wire, leaving content and position unchanged. The leading system message is untouched.

Scope and issue reconciliation

Matches the issue's required fix exactly: mapper-level change (not buildTurnMessages), trailing position preserved, prompts/compiled/system_prompt.txt comment updated. Anthropic's extractSystem (hoists every system message into the top-level system parameter regardless of position) is unchanged, as specified. No model-quirk table was added -- the mapper approach worked, so the documented alternative (a trailing_system_as_user provider quirk) was not needed.

Impact analysis reconciliation

  • internal/provider/openai/client.go mapMessages: the only production code change. Every OpenAI-compatible-routed provider (openai, openrouter, deepseek, kimi, qwen, gemini-via-compat) shares this one client, so one fix covers all of them -- confirmed by grep, there is no second chatMessage-mapping implementation.
  • internal/provider/anthropic/client.go: not touched, confirmed unaffected by design (system hoisting is position-independent).
  • internal/harness/clone.go buildTurnMessages: not touched, per the issue's stated preference.
  • No config, schema, persistence, or API surface changed. The only externally observable change is the wire role of one message sent to OpenAI-compatible endpoints.

Architecture and duplication check

Searched rg RoleSystem internal/provider and rg "func mapMessages\|chatMessage{" internal/provider -- one mapper (openai/client.go), one hoisting function (anthropic/client.go extractSystem), no parallel catalog or duplicate wire-mapping logic exists or was introduced.

Test-first evidence

Red command: go test ./internal/provider/openai -run 'TestMapMessagesNonLeadingSystemSentAsUser|TestCompleteWireBodyTrailingSystemAsUser' -v
Observed failure:

client_trailing_system_test.go:45: trailing runtime_context message role = "system", want user (this is the #1395 fix)
--- FAIL: TestMapMessagesNonLeadingSystemSentAsUser (0.00s)
client_trailing_system_test.go:122: wire.Messages[last].Role = "system", want user (trailing system must ride as user on the wire)
--- FAIL: TestCompleteWireBodyTrailingSystemAsUser (0.00s)

This proved the mapper forwarded the trailing system role verbatim -- the exact defect in the issue.

Green command: same command after the mapMessages fix -- all pass. TestMapMessagesLeadingOnlySystemUnchanged (single-leading-system case) passed before and after, guarding the non-regression contract.

Regression/integration evidence: TestRunnerRuntimeContextReachesOpenAIWireAsUser drives a real harness.Runner (with a stub systemprompt.Engine) through the real openai.Client, asserting the wire body's trailing message is role user. Re-running this test against the pre-fix client.go (checked out from the red commit) reproduces the failure, confirming it actually catches a revert.

Verification evidence

  • go test ./internal/provider/openai/... -run 'TestMapMessages|TestComplete|TestRunner' -v -- 17/17 pass.
  • go test ./internal/provider/... ./internal/harness -race -- all packages pass.
  • go vet ./internal/provider/... ./internal/harness/... -- clean.
  • go test ./internal/server/... -run TestRunSmoke -- pass (fake-provider path unchanged, no key/Docker needed).
  • Full suite go test ./...: only pre-existing failures in internal/acceptance/ptyrunner (real-PTY acceptance tests). Confirmed these fail identically on origin/main without this change (re-ran one against a stash of this branch's diff) -- a sandbox/environment limitation unrelated to this fix.
  • Not verified: live DeepSeek-via-OpenRouter completion. I have no provider API key in this worktree; live verification against the actual DeepSeek model is left to the coordinator/reviewer per the task instructions.

Rollout and rollback

Wire-shape change only, no persistence or schema involved. Revert this PR (or revert the single mapMessages commit) to fully roll back; no data repair needed.

Documentation

  • prompts/compiled/system_prompt.txt (lines ~7-10 and ~103): comments now describe the harness's internal system role for the runtime_context block versus the OpenAI-compatible wire role of user, and note the Anthropic client is unaffected.
  • docs/logs/engineering-log.md: new dated entry with cause, fix, and regression coverage, following the existing log format.

Contract checklist

  • Linked issue follows the current structured contract and this PR closes it
  • Issue acceptance criteria, impact map, and scope were updated when the design changed -- no scope change occurred (mapper approach worked as specified)
  • All callers, consumers, sources of truth, and similar abstractions were searched
  • No unrelated cleanup, hidden scope growth, duplicated wiring, or parallel abstraction was introduced
  • Tests were written first and the expected red failure was observed
  • Targeted checks and the repository-required full regression are green (excluding pre-existing, unrelated PTY acceptance failures)
  • Security, compatibility, lifecycle, deployment, observability, documentation, and rollback were reconciled
  • Real mouse/keyboard/API/operator behavior was exercised -- not applicable to this provider-wire change beyond the fake-provider smoke test; live DeepSeek verification is a follow-up

🤖 Generated with Claude Code

https://claude.ai/code/session_01WJGxhoFhA8JjkwZFcLGdS5

dennisonbertram and others added 3 commits September 6, 2026 08:15
Behavioral tests added:
- TestMapMessagesNonLeadingSystemSentAsUser
- TestMapMessagesLeadingOnlySystemUnchanged (non-regression, already passes)
- TestCompleteWireBodyTrailingSystemAsUser (end-to-end wire body proof)

Test runner output (expected: two failing, one passing):

=== RUN   TestMapMessagesNonLeadingSystemSentAsUser
    client_trailing_system_test.go:45: trailing runtime_context message role = "system", want user (this is the #1395 fix)
--- FAIL: TestMapMessagesNonLeadingSystemSentAsUser (0.00s)
--- PASS: TestMapMessagesLeadingOnlySystemUnchanged (0.00s)
=== NAME  TestCompleteWireBodyTrailingSystemAsUser
    client_trailing_system_test.go:122: wire.Messages[last].Role = "system", want user (trailing system must ride as user on the wire)
--- FAIL: TestCompleteWireBodyTrailingSystemAsUser (0.00s)
FAIL

These tests will pass after the implementation in the next commit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJGxhoFhA8JjkwZFcLGdS5
…ble wire

Implementation for tests added in 5390d49.

mapMessages() in internal/provider/openai/client.go now rewrites any
system-role message that is not the first message to role "user" on the
wire. This fixes DeepSeek (via OpenRouter), which returns an empty
assistant message whenever the last message in the request has role
"system" -- the harness appends the per-turn <runtime_context> block as a
second, trailing system message to keep the cacheable prompt prefix
unchanged (runner_step_engine.go buildTurnMessages). Position and content
are unchanged; only the wire role differs. The leading system message
(index 0) is untouched. Anthropic (extractSystem hoists every system
message into the top-level system param) and buildTurnMessages are
unaffected.

Test runner output (expected: all passing):

=== RUN   TestMapMessagesNonLeadingSystemSentAsUser
--- PASS: TestMapMessagesNonLeadingSystemSentAsUser (0.00s)
=== RUN   TestMapMessagesLeadingOnlySystemUnchanged
--- PASS: TestMapMessagesLeadingOnlySystemUnchanged (0.00s)
=== RUN   TestCompleteWireBodyTrailingSystemAsUser
--- PASS: TestCompleteWireBodyTrailingSystemAsUser (0.00s)
PASS
ok  	go-agent-harness/internal/provider/openai	0.241s

go test ./internal/provider/... ./internal/harness -race: all packages ok
go vet ./internal/provider/... ./internal/harness/...: clean

Behavioral tests covered: BT trailing-system-as-user (leading unchanged,
non-leading rewritten, end-to-end wire body).
Files changed: internal/provider/openai/client.go

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJGxhoFhA8JjkwZFcLGdS5
…system-as-user

Adds a regression test that would fail if the fix in 7a69e27 is reverted,
proven by re-running it against the pre-fix client.go: on a real
harness.Runner -> real openai.Client path, the wire body's trailing
runtime_context message must be role "user", not "system". Also updates
the prompts/compiled/system_prompt.txt comments to describe the OpenAI-
compatible wire role vs. the harness's internal "system" role, and adds
the docs/logs/engineering-log.md entry for the bug.

Full targeted suite output:

go test ./internal/provider/openai/... -run 'TestMapMessages|TestComplete|TestRunner' -v
  ok  go-agent-harness/internal/provider/openai  0.235s  (17/17 tests pass)

go test ./internal/provider/... ./internal/harness -race
  ok all packages

go vet ./internal/provider/... ./internal/harness/...
  clean (exit 0)

go test ./internal/server/... -run TestRunSmoke
  ok go-agent-harness/internal/server  0.271s (fake-provider path unchanged)

go test ./... (full suite): only pre-existing, unrelated failures in
internal/acceptance/ptyrunner (real-PTY tests that fail identically on
origin/main in this sandbox, confirmed by re-running one of them against
a stash of these changes -- environment limitation, not caused by this PR).

Regression scenarios covered:
- Reverting the mapMessages fix makes TestMapMessagesNonLeadingSystemSentAsUser,
  TestCompleteWireBodyTrailingSystemAsUser, and
  TestRunnerRuntimeContextReachesOpenAIWireAsUser all fail (verified directly).
- The integration test exercises the real Runner -> buildTurnMessages ->
  Complete -> mapMessages seam, not just a direct mapMessages() unit call.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJGxhoFhA8JjkwZFcLGdS5
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@dennisonbertram

Copy link
Copy Markdown
Owner Author

Live verification (coordinator, OpenRouter key): harnessd built from this branch, no proxy, model deepseek/deepseek-v4-flash via openrouter. Three consecutive runs of a write+read+reply prompt: all completed, 3 steps each, 0 llm.empty_response.retry events, file contents correct. Before this fix the same daemon config failed 3/3 with max_empty_responses.

@dennisonbertram
dennisonbertram merged commit eb18862 into main Sep 6, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant