Repository navigation
Conversation
Preserve existing newline and EOF semantics while scanning only newly received text in the four SSE loop adapters. Fixes ranxianglei#6849
(ai: vllm-qwen/qwen3.8-27b) Duplicate screening: not a duplicate. No other open issue or PR targets the growing-prefix scans in the core-loop SSE readers. Related work checked and confirmed distinct: #6579 (closed) made the native-compaction terminal observer linear — different parser, different path; #2329 / #6619 / #6654 established the split-CRLF and data-field contracts this PR preserves (their tests are in the validation set and pass). The author's other open PRs (#6821/#6843/#6845/#6848/#6854) cover different mechanisms. Labels: bug, scope/wiring, p2, pr — performance defect in the wire-protocol adaptation layer; usable but degrades under large single SSE events. ReviewDirection (issue #6849). The issue identifies the right root cause: each loop reader appends the chunk to Independent verification (I re-ran everything, not just trusted the numbers):
Correctness analysis. I traced the framer's state machine against the old semantics by hand for the tricky cases — split CRLF across chunks, lone-CR terminators, CR-terminated streams, blank-line-only input, whitespace-only EOF residuals, empty chunks — and it matches. The key invariant it builds on is the #2323 hold-back in Regression review (existing behaviors on touched paths, item by item):
Test validity. The scan-bound test has teeth: it fails on master (271M chars ≫ 8× bound) and passes on the PR — a deterministic operation count, not a timing threshold, so it's CI-stable. The Unicode one-byte-split tests cover multi-byte characters across all three terminator styles. The hermetic HTTP suites cover real local responses; the PR is transparent that the live Codex four-phase E2E was not run (upstream unreachable) — acceptable given the hermetic coverage and green CI full suite. Non-blocking notes:
Auto-merge gate: not eligible — 结构化八问
判定:可合(建议先 rebase 到当前 master——落后 4 个 README-only 提交,无冲突;与 #6845 的合入顺序注意一下即可)。 |
Problem
Fixes #6849.
The OpenAI, Anthropic, Responses, and Google core-loop SSE readers normalize and search the entire unfinished event after every network chunk. A single large event, such as Gemini inline image output, repeatedly scans the same prefix and blocks the event loop for progressively longer periods.
Changes
sse-util.tsmodule and use it in the four loop SSE readers.There are no new configuration fields, dependencies, source modules, or kernel changes.
Reproduction and measured result
The parent is master
2e3cfc973b8620b1f7bc8a415ddcbd31e7f4aaee(0.1.215).The deterministic regression passes a 1 MiB image-shaped Google event through the real adapter in 4 KiB chunks and counts the characters examined by newline searches. Unchanged master examines 271,581,497 characters for 1,048,717 input bytes and fails the linear-work assertion; the candidate examines 2,097,465 characters and forwards the exact same frame. The three one-byte Unicode/LF/CRLF/CR controls also pass on the baseline.
The included benchmark measures the Google adapter with one warmup and the median of three timed passes, receiving 16 KiB chunks:
These are local adapter/framing measurements using synthetic image-shaped payloads, with no image codec, network, or model involved. CI uses the deterministic scan bound rather than a timing threshold.
Validation
137 focused tests passed:
The new framer is also compared against the previous whole-buffer framing for all short combinations of ordinary text, CR, and LF at every two-chunk split, plus one-character chunking, multiline/metadata fields, trailing CR, and EOF residuals. Existing controls cover real local HTTP responses, signed thinking replay, tool continuations, all four adapters, and hermetic Google compression.
Also passed:
tsc --noEmit --project tsconfig.jsonnpm run buildnode scripts/check-test-import-graph.mjsgit diff --checkThe live Codex four-phase E2E has not been run.
E2E_CHECK=1confirms the built distribution and installed CLI, but the configuredhttp://127.0.0.1:8199/v1/modelsupstream is unreachable. The HTTP tests above are hermetic.Related work checked: #6579 optimizes the separate native-compaction terminal observer, not these loop readers. #2329 and #6619/#6654 establish the CRLF and data-field behavior retained here.