Skip to content

test: pin the translation layer with an offline suite - #2

Merged
cainrus merged 1 commit into
mainfrom
ship/translation-tests
Sep 5, 2026
Merged

cainrus merged 1 commit into
mainfrom
ship/translation-tests

Conversation

@cainrus

@cainrus cainrus commented Sep 5, 2026

Copy link
Copy Markdown
Owner

The CI gate was a parse check standing in for tests. Replace it with 82
node:test cases over the part of the bridge that can actually be wrong:
the two translation directions and the SSE rebuild.

The suite never opens a socket or calls the upstream API. Streaming is
driven by a recorded Responses stream in test/fixtures/, re-fed at chunk
sizes 1, 7, 64 and 997 to prove that events spanning a socket boundary
reassemble identically.

What the tests hold in place, beyond the happy path:

  • tool_result leaves the user turn and becomes a top-level
    function_call_output; call ids stay paired across a full tool round
  • an empty errored tool_result still sends "error" -- Responses rejects
    an empty output string
  • the encrypted reasoning blob survives a round trip through a thinking
    block's signature, and the rebuilt reasoning item is hoisted ahead of
    the call it produced
  • a foreign thinking block or a corrupt signature decodes to nothing
    instead of throwing
  • reasoning.effort and include are sent only to gpt-5.6-*; the cheap
    fallback 400s on them
  • max_tokens: 1 is floored to 16; temperature and top_p are dropped
  • an unmapped model lands on the fallback rather than a 400
  • truncated tool arguments degrade to {} instead of crashing the response
  • a block left open by a dropped upstream connection is still closed, so
    the client cannot hang

Structural change, needed to test any of it without a live server:

Options considered

  • spawn the proxy on a port and drive it over HTTP: covers the same code
    but adds sockets, an upstream stub and flakiness to every case
  • move the translators into a second module: clean, but the package's one
    claim is that it is a single file
    Picked: export the pure functions from codex-proxy.ts and start the
    server only when the file is the process entry point. Importing it is now
    side-effect free -- no port, no exit on a missing key. A child-process
    test pins that contract, and a smoke run confirms npm start still exits
    1 with FATAL without a key and still prints the ready line with one.

Two conversions were inlined in the request handler and had to come out
to be reachable: toAnthropicMessage for the non-streaming response, and
createStreamTranslator, which takes raw SSE text and emits events through
a callback so the socket and the fixtures drive the same code. Neither
changes behaviour.

ci: run npm test instead of node --check. The job name stays test.

The CI gate was a parse check standing in for tests. Replace it with 82
node:test cases over the part of the bridge that can actually be wrong:
the two translation directions and the SSE rebuild.

The suite never opens a socket or calls the upstream API. Streaming is
driven by a recorded Responses stream in test/fixtures/, re-fed at chunk
sizes 1, 7, 64 and 997 to prove that events spanning a socket boundary
reassemble identically.

What the tests hold in place, beyond the happy path:

- tool_result leaves the user turn and becomes a top-level
  function_call_output; call ids stay paired across a full tool round
- an empty errored tool_result still sends "error" -- Responses rejects
  an empty output string
- the encrypted reasoning blob survives a round trip through a thinking
  block's signature, and the rebuilt reasoning item is hoisted ahead of
  the call it produced
- a foreign thinking block or a corrupt signature decodes to nothing
  instead of throwing
- reasoning.effort and include are sent only to gpt-5.6-*; the cheap
  fallback 400s on them
- max_tokens: 1 is floored to 16; temperature and top_p are dropped
- an unmapped model lands on the fallback rather than a 400
- truncated tool arguments degrade to {} instead of crashing the response
- a block left open by a dropped upstream connection is still closed, so
  the client cannot hang

Structural change, needed to test any of it without a live server:

Options considered
- spawn the proxy on a port and drive it over HTTP: covers the same code
  but adds sockets, an upstream stub and flakiness to every case
- move the translators into a second module: clean, but the package's one
  claim is that it is a single file
Picked: export the pure functions from codex-proxy.ts and start the
server only when the file is the process entry point. Importing it is now
side-effect free -- no port, no exit on a missing key. A child-process
test pins that contract, and a smoke run confirms `npm start` still exits
1 with FATAL without a key and still prints the ready line with one.

Two conversions were inlined in the request handler and had to come out
to be reachable: toAnthropicMessage for the non-streaming response, and
createStreamTranslator, which takes raw SSE text and emits events through
a callback so the socket and the fixtures drive the same code. Neither
changes behaviour.

ci: run npm test instead of `node --check`. The job name stays `test`.
@cainrus
cainrus enabled auto-merge (squash) September 5, 2026 15:51
@cainrus
cainrus merged commit 4e436b4 into main Sep 5, 2026
1 check passed
@cainrus
cainrus deleted the ship/translation-tests branch September 5, 2026 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant