Skip to content

stdio transport: UTF-8 double-encoding on output; silence on non-ASCII input #4

Description

@noizu

Found by The Robot Reviews' D-030 test pass (AC1 real-stdio integration test). Raw wire-capture evidence lives in the test comments: the-robot-reviews/app/test/trr_site/doc_pointers/client_stdio_integration_test.exs.

1. OUTPUT double-encodes non-ASCII

Each UTF-8 byte of a glyph is emitted as a Latin-1 codepoint: F0 93 85 9C leaves the server as C3 B0 C2 93 C2 85 C2 9C. Every client response containing a hieroglyph token is corrupted — and the tokens are the system's core citation identifier.

2. INPUT: non-ASCII requests get no response

A request carrying non-ASCII (e.g. token-keyed lookup — the core citation flow) never produces a response; the server stays silent until the client times out (30s in our transport).

3. Minor: generate omits description

Real generate stores description and lookup returns it, but the generate result omits the field; it is also required in real but optional in our FakeTransport (test-fidelity divergence).

Suggested fix direction

The transport should treat stdin/stdout as raw binary (:binary mode / port with stream lines) rather than Unicode codepoint lists — the double-encode pattern is the classic :unicode/Latin-1 list mismatch.

Repro: TRR's client_stdio_integration_test.exs auto-discovers a sibling checkout and compiles out when absent, so it doubles as a cross-repo repro harness.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions