Repository navigation
[PR] perf(stream): serialize request evidence only for filtered JSON replies - #6597
Merged
ranxianglei merged 1 commit intoNov 14, 2026
Merged
Conversation
Owner
|
Superseded by #6677 (merged as Landed into master via the real merge commit The fix for #6596 is now in master. Closing this PR as superseded — thanks for the careful, well-tested fix. |
This was referenced Nov 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #6596
Why
Each non-streaming JSON reply currently serializes
ctx.messagesin full, even for ordinary short prose and when absorb is disabled. This is synchronous history-sized work on the shared proxy process. The string is only needed by absorb's quote-preservation check when a reply actually enters the prose filter.Change
Delay request-history serialization behind the existing filter gate and absorb-enabled check across Anthropic, OpenAI Chat, Responses and Gemini. Reuse evidence across Anthropic blocks. Gemini prechecks its response parts and captures any needed request evidence before yielding proxy tools, preserving the existing snapshot timing. Filtering rules, user quotations, signed thoughts and real tool arguments retain their behavior.
The patch includes a reproducible benchmark of the real async rewriters and compatibility coverage. No new configuration or protocol behavior.
Measurements
Base
e1f31ad9d5b47500362699a9867819b861c7de96(0.1.213); Node 24.22.1/Linux. For each history size, three fresh before/after process pairs alternate ordering. Each process runs two warmups and five measured calls per protocol/mode. The 8 MiB input contains 1,024 validCoreMessageentries and the response is a fixed short answer.Median of per-process medians:
Median whole-process peak RSS for the combined eight-case run: 183,504 KiB → 128,184 KiB. Every per-protocol full-response SHA-256 digest matched across revisions and samples.
The 1 KiB control is mixed, with all medians below 0.05 ms: Gemini absorb-off 0.031 → 0.044 ms and absorb-on 0.019 → 0.027 ms. These results establish removal of the large-history serialization cost in this synthetic rewriter workload; they are not production HTTP latency measurements. Output probing/filtering still runs, and armed replies requiring quote evidence still serialize history.
Reproduce one fresh process with:
Verification
json-rewrite-history,tag-echo,absorb-emissionandexternal-summary-runtimesuites.git diff --checkpass.The linked issue records all-state duplicate screening and triage. Topic N (server response processing), cross-client; suggested
p2,scope/wiring.