server: do not abort a completion when the model emits invalid UTF-8 - #202

Open

danielhanchen wants to merge 9 commits into

fix/server-invalid-utf8-abort

Commits on Sep 6, 2026

server: do not abort a completion when the model emits invalid UTF-8
danielhanchen
committed
61726ba

Commits on Sep 9, 2026

server: flush a trailing incomplete UTF-8 sequence on the final parse
danielhanchen
committed
25fcc22
server: match the JSON serialiser's UTF-8 replacement boundaries
danielhanchen
committed
be6431c
server: apply the lead-specific first-continuation bounds
danielhanchen
committed
4470648
server: reject complete but illegal UTF-8 before copying it through
danielhanchen
committed
9b2955e
server: trim the UTF-8 sanitising comments
danielhanchen
committed
6dc4a96
tighten the comments added by this change
danielhanchen
committed
b27c0b4
server: unit test for the UTF-8 sanitising, against an independent WHATWG decoder
danielhanchen
committed
a5ac51d
server: pin the token pipeline case, streaming against non streaming
danielhanchen
committed
ff1d590