fix(serve): an unpaired surrogate escape answered 500 "engine failed"

[kevin9327]

Symptom

A request with a JSON string that holds an unpaired UTF-16 surrogate escape gets HTTP 500 and a message that blames the engine. The escape looks like "\ud83d": a client produces it when it cuts a string between the two halves of an emoji (a JS slice on a long message is the usual way). It fails on /v1/chat/completions, /v1/completions and /v1/messages.

Against a real coli serve (tiny Qwen3.6 fixture, unfixed dev):

500 /v1/chat/completions  {"messages":[{"role":"user","content":"emoji \ud83d"}]}
500 /v1/chat/completions  {"messages":[{"role":"user","content":"x \ude00 y"}]}
500 /v1/completions       {"prompt":"\udcf0"}
500 /v1/messages          {"max_tokens":4,"messages":[{"role":"user","content":"\ud83d"}]}
200 /v1/chat/completions  {"messages":[{"role":"user","content":"ok 😀"}]}   <- a paired escape is fine

{"error":{"message":"The colibri engine failed to process the request.","type":"server_error","param":null,"code":"engine_error"}}
server log: [api] 127.0.0.1 - request failed: 'utf-8' codec can't encode character '\ud83d' in position 23: surrogates not allowed

The engine is fine; the request is invalid. OpenAI SDKs retry a 5xx, so the client also resends it before surfacing an error that points at the wrong side.

Root cause

json.loads accepts a lone surrogate escape, so read_json returns a body containing text that no UTF-8 byte sequence can represent. The first step of Engine.generate is payload = prompt.encode("utf-8"). The UnicodeEncodeError reaches do_POST's catch-all, which answers 500 engine_error.

read_json already turns undecodable UTF-8 in the raw body into a 400 ("Request body must be valid JSON."). An escaped lone surrogate is the same invalid text, spelled differently. generate also already refuses the neighbouring case, NUL bytes in a prompt, with a 400.

Fix

In read_json, after parsing, check that the body can be encoded as UTF-8 (json.dumps(body, ensure_ascii=False).encode("utf-8")). If it cannot, answer 400 with "Request body contains an unpaired UTF-16 surrogate escape; strings must be valid Unicode." The check runs once per request, and a body is at most MAX_BODY = 4 MiB.

Because this is at the request boundary, it covers every endpoint and field: prompts, messages, tool definitions, GBNF grammars, and the cluster registry, which shares read_json. Paired escapes such as 😀 decode to a real character and are unaffected.

Tests

New HTTPTest.test_unpaired_surrogate_is_a_client_error posts a lone \ud83d to all three endpoints and expects 400. FakeEngine never encodes the prompt, so the test wraps its generate with the same first step the real Engine.generate takes (prompt.encode("utf-8")). Without the fix, the server answers exactly the production 500.

Fail-before on unfixed dev:

[api] 127.0.0.1 - request failed: 'utf-8' codec can't encode character '\ud83d' in position 26: surrogates not allowed
FAIL: test_unpaired_surrogate_is_a_client_error (...) (path='/v1/chat/completions')
AssertionError: 500 != 400
FAIL: test_unpaired_surrogate_is_a_client_error (...) (path='/v1/completions')
AssertionError: 500 != 400
FAIL: test_unpaired_surrogate_is_a_client_error (...) (path='/v1/messages')
AssertionError: 500 != 400

Verification

• Every test module that imports openai_server or cluster (20 modules; test_fp8_repack_full_family left out because it needs torch here) passes with PYTHONUTF8=1: OK, 28 skipped. In the cp949 locale the only errors are the two test_openai_tools_v41_e2e cases that read their fixture with a locale-encoded read_text(). They error the same way on unfixed dev.
• End to end, same coli serve as above, after the fix:
  • The four lone-surrogate requests answer 400.
  • The paired escape and a plain request still answer 200.
  • The server log has no request failed line.
• The same happens through coli chat --attach. On a Windows console whose code page is not UTF-8, piped UTF-8 input with an emoji reaches the server as lone surrogates. Before the fix that turn failed with a 500; now the chat prints the 400 and its message.

🤖 Generated with Claude Code