Repository navigation
Conversation
ranxianglei#554 guard meter (ranxianglei#6828) A dual-shape body carries the conversation under two top-level keys: the lane's native field (messages[] on openai/anthropic, input[] on responses) plus another protocol's field (a Responses-native host such as dsh sends `input` alongside `messages`). The upstream bills only the native field, so the protocol-blind meter counted the conversation twice and the egress fuse false-tripped on the phantom total (1,815,699 measured vs ~380K billed, 4.77x), while preflight's protocol-aware estimate saw only ~135K and never compressed. Thread the lane's protocol through measuredOutbound and the preflight measured-body doom tiers so the meter skips the foreign conversation fields for that lane, top level only. Without a protocol the meter keeps its fail-closed count-everything behavior. Regression test: tests/issue6828-guard-meter-dual-shape.test.ts
…e-request guard meter (ranxianglei#6828) Review supplement to the PR: the egress fuse (relay) and the preflight measured-body tiers were made protocol-aware, but sideRequestGuard() still metered the raw body protocol-blind even though the lane protocol is a parameter of the function. A dual-shape body's foreign conversation field double-counted there too, false-blocking the compress/nudge side call — the very request that would shrink the session. - sideRequestGuard: pass the lane protocol to estimateRawBodyTokens - regression test: a dual-shape body must not false-block the side request (verified red without the fix, green with it)
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 查重: 不重复 — Searched open + closed issues/PRs for "guard meter" / "egress fuse": the only open PR fixing #6828 is this one (#6840); #4849 is closed and unrelated. Sibling #6844 is the same symptom family (the measured-body egress fuse counting a model-hidden field in the Review of PR #6840 (fixes #6828)Verdict: the fix is correct and safe. I pushed one additive commit ( What the PR does right
Root-cause judgmentThe dual-shape body is the trigger, but the defect is the meter: it is protocol-blind and counts every top-level conversation field regardless of which one the lane actually bills. The PR fixes the meter at the right layer (the estimate function + its outbound call sites), not by filtering payload in transit — that is the correct root-cause fix, not a mask. (The foreign field still rides the wire — bandwidth waste — but that is a separate wire-fidelity concern, out of scope here.) Additive commit I pushed (
|
…de-lane suite (ranxianglei#2170 lane-guard) The lane-guard gate (ranxianglei#2170) requires that a change to src/server/side-request.ts land alongside one of the dedicated side-lane suites. The previous commit put the ranxianglei#6828 dual-shape regression in tests/issue6828-guard-meter-dual-shape.test.ts (the meter's file), which does not satisfy the gate. Move it to tests/side-request-isolation.test.ts, the canonical home for sideRequestGuard tests.
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) CI 跟进(新 head 当前 CI(live 核实): |
Resolve conflicts in src/preflight.ts: keep the PR's ranxianglei#6828 foreign-conversation-field skip and main's isNonModelTopLevelKey skip side by side in estimateRawBodyTokens and estimateRawBodyBreakdown. src/server.ts and src/server/side-request.ts merged clean.
What broke
The #554 guard meter walks every string field of the outbound body, so when a body carries the conversation under two protocol shapes at once (messages plus Responses' input), it counts the history twice. The reporter's fuse reading was 1,815,699 tokens against about 380K actually billed, so the fuse blocked a body that fits.
What changed
How verified
Fixes #6828