Repository navigation
fix: arm emergency shrink on outbound payload size, not raw body (#1493) - #1498
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.1444-pr.1498.f7ff73dThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr1498.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 This is the PR for #1493 (fix landed by ework-daemon, already reported in that thread). Not re-doing it — reviewing now: pulling branch |
[bot] 🏷 Review complete — mergeable (rebase onto current master first, or merge directly; see ⑤). I did not re-do the work; this is a verification pass over branch ① Original issue (#1493) — direction check. The triage in the #1493 thread holds: root cause is ② Regression review — existing behaviors on touched paths (
No unintended regressions found. ③ Code review. Helper mirrors the preflight computation exactly; all three call sites pass identical args. Two minor, non-blocking nits: (a) the old code wrapped its estimator in try/catch; the new call sites evaluate ④ Test validity — verified locally, the test has teeth. Ran ⑤ Base freshness & full pre-flight (done on the rebased head). The PR base was stale when CI ran against master (base b371bd0; master since gained #1490/#1489 cache-ledger, release v0.1.160, etc.). Master-side Related item flagged by author: the overflow-not-parseable Verdict: 可以合并 — direction correct, regression inventory clean, test proven to bite, green on rebased head + CI. Merge itself stays human-only. 一句话中文摘要:审核了 #1493 的修复 PR(失败后按出站载荷而非原始 body 估算来武装紧急收缩),回归清点无意外行为变更、#857 证据隔离与 #604 死锁解除均保持、新测试在旧代码上确实复现 bug(44191 ≥ 30400 阈值)且在修复后通过,本地已验证 rebase 到最新 master 后全套 2912 测试全绿、CI 13/13 绿,可以合并。 |
Deep review: DO NOT MERGE as-is — two deterministic regressions, plus a compose conflict with #1503An independent reviewer pass (differential: PR head vs base, same kernel install, mutation-verified) found the refactor idea sound (single source of truth for outbound size) but the implementation breaks two things it must preserve. Details: F1 (HIGH): #604 relay-5xx self-heal is broken on the pristine head
Mechanism: the arm value is not only a preflight floor — it is the truncation-depth input (kernel F2 (HIGH): the transform-failure fallback — the accident scene named in this PR's own description — now arms at ~overhead+images onlyEvery protocol's kernel-transform-failure catch resets For a genuinely over-window raw payload on a persistent fallback, nothing ever crosses the emergency band → no rescue → the #604 deadlock class reintroduced by different means. The symptom disappears only because the poison is replaced by silence. Fix direction: mirror the existing pattern at src/server.ts:2926 ( F3 (MEDIUM): compose conflict with #1503 — #1503 is the root-cause-side fix and should merge first
Recommended path: merge #1503 first, then rework this PR on top — add the F4: commit-message evidence doesn't holdHead full suite: 2885 tests / 2864 pass / 9 fail — 2 real (F1) + 7 environmental (kernel-pin goldens, identical on base). The claimed "#604 genuine-overflow self-heal tests still pass" covers Minor
Mutation check: reverting only the arming change reddens both new tests (armed 43883 ≥ band 30400) — the tests genuinely encode the claimed fix; they just pin the wrong invariant for the fallback path. |
backstop Follow-ups from review of #1503: - F1/HIGH: estimateInputTokens floors on lastInputTokens only when lastInputTokensSource === "usage". An estimate-grade baseline (raw-view poison from a transform-failure fallback arm, #1492/#1493) would otherwise mask the payload's own estimate through the max() and let the #453 clamp fail open while the poison persists. clampOutgoingOutput now feeds the source through (#1498 F5's clamp hole, closed here instead). - F3: the loop-floor and emergency-band gates share this exact clamping mechanism, so they inherit the fix; pinned via the updated backstop test. - tests/issue453-backstop.test.ts: the real-count test now also pins the gate (estimate-grade and unknown-provenance baselines never floor).
A no-usage upstream failure (relay 5xx / network drop) armed the emergency
shrink from estimateTokensFast(<entire JSON body>) — a chars/4 count that
inflates base64 images and dense payloads to RAW-history scale. That poisoned
lastInputTokens fed the preflight trigger floor and nudge baseline, firing a
pointless 42–203s preflight on every subsequent request even though the
outbound payload fit the window ("payload fits with no compressible ranges").
Fix: measure the arm against the SAME outbound payload estimate the fit gate
uses — a new outboundPayloadBreakdown helper (post-fold content + wire
overhead + correctly-billed images) shared by preflightCompressIfNeeded and
armFailureShrink. A genuinely-overflowing payload still arms high enough to
break the #604 relay deadlock; a fitting one no longer falsely crosses the
trigger. Also downgrades the benign "payload fits, no compressible ranges"
preflight log from warn to info.
Behavior change: the local estimate used to arm the emergency shrink after a
no-usage failure changed from raw-body chars/4 (raw-history scale) to the
outbound payload estimate (what would actually be sent) — lower and more
accurate; a real usage report still overwrites it on success.
Tests: new tests/issue1493-arm-outbound-payload.test.ts reproduces the bug
(fails on old code: arms to 44191 > 30400 band) and passes on the fix. Full
suite green (2873 pass / 0 fail); the #604 genuine-overflow self-heal tests
still pass.
Fixes #1493
…arm on the text char bound Second-round rework after review (on top of #1503): - F2 (HIGH): outboundPayloadBreakdown now measures originalMessages when processedMessages is empty — every protocol's kernel-transform-failure catch leaves it empty while the outbound IS the raw client body, so the estimator previously returned wire overhead only and the rescue never armed in the very scene #1493 names. Regression test drives the real fallback (broken state seam) and pins the arm at >= the raw char bound. - F1 (HIGH): the arm is not only a preflight floor — the kernel's truncation loop consumes it as its remaining-depth input, and the optimistic chars/4 text estimate undercounts dense payloads by up to ~4x (#553), stopping the loop one candidate early (pre-rebase this deterministically re-broke the #604 relay rescue: armed 31101 vs 31691, retry 116.5KiB vs 108.7KiB, 500 vs 200). armEstimate floors the arming quantity on the text char bound (never undershoots, deliberately skips image base64 so #1493's pixel-billed fix stands); preflight keeps the billing-accurate payloadEstimate so a payload that fits does not falsely cross the trigger. - Rebased onto #1503 (ee12524): its consumer-side provenance gates are the root-cause half; this PR now supplies the producer-side fallback and depth insurance. All shared suites green together: issue1493 3/3, issue1492 2/2, relay-5xx selfheal 3/3, overflow selfheal, #453 backstop.
7402704 to
9dd8f98
Compare
|
Reworked on top of #1503 (head F2 (HIGH — fixed): F1 (HIGH — fixed by construction): the arm is not only a preflight floor — the kernel's truncation loop consumes it as its remaining-depth input, and the optimistic chars/4 text estimate undercounts dense payloads by up to ~4× (#553), stopping the loop one candidate early. The arming quantity ( Composition with #1503: rebase resolved the one conflict ( Merge after #1503. |
[bot] 🏷 Rework ( |
[bot] 🏷 Verification of reworked head Evidence on
Finding-by-finding:
Regression audit (existing behaviors touched → verdict): preflight trigger floor (intentional change: estimate-sourced baseline can no longer floor measured-payload sizing — disclosed in #1503, which this stacks on); arm value semantics (old: raw-body chars/4 → new: text char-bound floor + correctly-billed images/overhead — lower for image-heavy payloads since base64 is no longer counted at chars/4; the two self-heal e2e shapes pin the known rescue paths, and an image-dominant near-window relay case would be the residual spot to watch); nudge baseline (estimate baselines ignored on measured payloads — #1503's disclosure); transform-failure fallback arming (restored to raw-view measurement — the F2 fix); 4xx-must-not-arm preserved (relay-5xx test 3 green); wire shape, persistence format, kernel pin, config schema — untouched. Merge order: #1503 first, then this PR. No action needed from me once #1503 lands unless you want the F6 log level flipped. 中文摘要:复核了重做后的 |
…ontext (#2683) Two producers of one mechanism family - session input baselines that outlive the context that measured them: 1. armFailureShrink adopted the char-count UPPER BOUND of the outbound payload as session.stats.lastInputTokens (a 1.7-4x over-reading). The #1498-F1 bound floor predates the #1492/#1839 reader quarantine: no processTurn feed passes an estimate-grade arm into any kernel band anymore, so the bound's only live effect was phantom mass - inflated preflight budgets / output headroom / displays surviving a model or window change until the next usage report (live case: 441,193 armed against a 202,885-token payload; after a 1M->400k model switch the meter read 142% and two ~3-minute emergency folds ran on a payload already at 62% of the new window). The arm now adopts the billing- caliber reading (k-scaled chars/4 + wire overhead + image reserve), and its log names the provenance. 2. A usage-grade baseline measured under model X kept deciding requests for model Y (only ORIGIN changes demoted it, #1933 F2): after a mid-session model switch the foreign bill pinned the preflight loop's decisionTokens above the fold target forever - folding until budget exhaustion although the payload fit. A real model change (both sides known; unknown->known never counts) now retires the trusted-grade baseline at the per-request model-stamp site, before prepare sizes the nudge and before the trigger runs; the next usage report re-anchors. Estimate-grade values survive the switch (byte-scale properties of the payload); same-model window moves keep the bill valid, so genuine overflow still folds. Behavior changes (old -> new): - armed value on a no-usage failure: char-count upper bound -> billing- caliber estimate (it may now land BELOW a future bill on dense JSON/CJK routes where chars/4 undercounts, #553; acceptable because estimate-grade values are quarantined from every decision floor and k closes most of the gap where route+model provenance matches) - usage/overflow-arm baseline + usage anchor across a model switch: trusted verbatim -> retired until the next usage report (forensic state preserved: cumulative inputTokens meter, last-usage timestamp, calibration ring, cache-ledger history) Tests: tests/issue2683-cross-model-baseline.test.ts (new: G1 cross-model retirement e2e - zero summary calls + re-anchor; G2 arm scale band [chars/8, char-count) + provenance clause; G3 control - same-model over-window bill still folds via its floor), re-scaled #1498-F2 assertion in tests/issue1493-arm-outbound-payload.test.ts, and updated pins in tests/context-provenance-outbound.test.ts.
…ontext (#2683) Two producers of one mechanism family - session input baselines that outlive the context that measured them: 1. armFailureShrink adopted the char-count UPPER BOUND of the outbound payload as session.stats.lastInputTokens (a 1.7-4x over-reading). The #1498-F1 bound floor predates the #1492/#1839 reader quarantine: no processTurn feed passes an estimate-grade arm into any kernel band anymore, so the bound's only live effect was phantom mass - inflated preflight budgets / output headroom / displays surviving a model or window change until the next usage report (live case: 441,193 armed against a 202,885-token payload; after a 1M->400k model switch the meter read 142% and two ~3-minute emergency folds ran on a payload already at 62% of the new window). The arm now adopts the billing- caliber reading (k-scaled chars/4 + wire overhead + image reserve), and its log names the provenance. 2. A usage-grade baseline measured under model X kept deciding requests for model Y (only ORIGIN changes demoted it, #1933 F2): after a mid-session model switch the foreign bill pinned the preflight loop's decisionTokens above the fold target forever - folding until budget exhaustion although the payload fit. A real model change (both sides known; unknown->known never counts) now retires the trusted-grade baseline at the per-request model-stamp site, before prepare sizes the nudge and before the trigger runs; the next usage report re-anchors. Estimate-grade values survive the switch (byte-scale properties of the payload); same-model window moves keep the bill valid, so genuine overflow still folds. Behavior changes (old -> new): - armed value on a no-usage failure: char-count upper bound -> billing- caliber estimate (it may now land BELOW a future bill on dense JSON/CJK routes where chars/4 undercounts, #553; acceptable because estimate-grade values are quarantined from every decision floor and k closes most of the gap where route+model provenance matches) - usage/overflow-arm baseline + usage anchor across a model switch: trusted verbatim -> retired until the next usage report (forensic state preserved: cumulative inputTokens meter, last-usage timestamp, calibration ring, cache-ledger history) Tests: tests/issue2683-cross-model-baseline.test.ts (new: G1 cross-model retirement e2e - zero summary calls + re-anchor; G2 arm scale band [chars/8, char-count) + provenance clause; G3 control - same-model over-window bill still folds via its floor), re-scaled #1498-F2 assertion in tests/issue1493-arm-outbound-payload.test.ts, and updated pins in tests/context-provenance-outbound.test.ts. Rebased onto master d95799d (2026-10-24, reopened after the stale-ticket cleanup closed this PR while #2683's fix was absent from master): resolved text conflicts in src/server/handle.ts (#4205 window-caliber stamps — the retirement block now sits after both stamps, before the lastModel stamp) and src/server/relay.ts (#3129 v1 framer insertion, #3701 close-1009 byte-shed branch — kept deliberately un-alerted, only the calibrated arm call applied); extended the calibrated-provenance flag to the two armFailureShrink call sites master added after this branch cut (#3148 circuit-open fail-fast, #3701 oversize resend). Verified on the rebased head: typecheck clean + 49/49 across the targeted baseline/meter suites (issue2683 G1/G2/G3, #1498-F2, context-provenance-outbound and adjacent anchor/floor suites).
…amily 1M window (#5500) Two independent defects compounded into the 'context ~N tokens exceeds the model window' fail-fast loop reported in #5500: C1 (window miss): CONTEXT_LIMIT_TABLE had no step-* row, so step-family models fell to the base.modelContextLimit 2e5 default (~150K effective window) despite serving 1M. The miss stranding is the #852 anti-pattern (too-small guesses never self-heal). Add the row from field evidence (nvidia_nim/stepfun-ai/step-5-preview enforces max_input_tokens=1048576). C2 (char-bound poisoning): the #604/#1498-F1 failure arm persists an ESTIMATE-grade lastInputTokens at char-bound scale (~4x tokens) into stats_json, replayed into every later request. Every reader except the preflight ENTRY was already grade-disciplined (#1839 kill-at-reader); the entry's currentTokens init and entryTokens floor were the last ungraded consumers, inflating overshoot budgets on every request after restart. Apply the per-round floor's exact discipline (#1492) to both entry reads. usage/overflow-arm anchors still floor; unknown-baseline regime and the #604 kernel truncate rescue consumer are untouched.
…amily 1M window (#5500) Two independent defects compounded into the 'context ~N tokens exceeds the model window' fail-fast loop reported in #5500: C1 (window miss): CONTEXT_LIMIT_TABLE had no step-* row, so step-family models fell to the base.modelContextLimit 2e5 default (~150K effective window) despite serving 1M. The miss stranding is the #852 anti-pattern (too-small guesses never self-heal). Add the row from field evidence (nvidia_nim/stepfun-ai/step-5-preview enforces max_input_tokens=1048576). C2 (char-bound poisoning): the #604/#1498-F1 failure arm persists an ESTIMATE-grade lastInputTokens at char-bound scale (~4x tokens) into stats_json, replayed into every later request. Every reader except the preflight ENTRY was already grade-disciplined (#1839 kill-at-reader); the entry's currentTokens init and entryTokens floor were the last ungraded consumers, inflating overshoot budgets on every request after restart. Apply the per-round floor's exact discipline (#1492) to both entry reads. usage/overflow-arm anchors still floor; unknown-baseline regime and the #604 kernel truncate rescue consumer are untouched.
…ontext (#2683) Two producers of one mechanism family - session input baselines that outlive the context that measured them: 1. armFailureShrink adopted the char-count UPPER BOUND of the outbound payload as session.stats.lastInputTokens (a 1.7-4x over-reading). The #1498-F1 bound floor predates the #1492/#1839 reader quarantine: no processTurn feed passes an estimate-grade arm into any kernel band anymore, so the bound's only live effect was phantom mass - inflated preflight budgets / output headroom / displays surviving a model or window change until the next usage report (live case: 441,193 armed against a 202,885-token payload; after a 1M->400k model switch the meter read 142% and two ~3-minute emergency folds ran on a payload already at 62% of the new window). The arm now adopts the billing- caliber reading (k-scaled chars/4 + wire overhead + image reserve), and its log names the provenance. 2. A usage-grade baseline measured under model X kept deciding requests for model Y (only ORIGIN changes demoted it, #1933 F2): after a mid-session model switch the foreign bill pinned the preflight loop's decisionTokens above the fold target forever - folding until budget exhaustion although the payload fit. A real model change (both sides known; unknown->known never counts) now retires the trusted-grade baseline at the per-request model-stamp site, before prepare sizes the nudge and before the trigger runs; the next usage report re-anchors. Estimate-grade values survive the switch (byte-scale properties of the payload); same-model window moves keep the bill valid, so genuine overflow still folds. Behavior changes (old -> new): - armed value on a no-usage failure: char-count upper bound -> billing- caliber estimate (it may now land BELOW a future bill on dense JSON/CJK routes where chars/4 undercounts, #553; acceptable because estimate-grade values are quarantined from every decision floor and k closes most of the gap where route+model provenance matches) - usage/overflow-arm baseline + usage anchor across a model switch: trusted verbatim -> retired until the next usage report (forensic state preserved: cumulative inputTokens meter, last-usage timestamp, calibration ring, cache-ledger history) Tests: tests/issue2683-cross-model-baseline.test.ts (new: G1 cross-model retirement e2e - zero summary calls + re-anchor; G2 arm scale band [chars/8, char-count) + provenance clause; G3 control - same-model over-window bill still folds via its floor), re-scaled #1498-F2 assertion in tests/issue1493-arm-outbound-payload.test.ts, and updated pins in tests/context-provenance-outbound.test.ts. Rebased onto master d95799d (2026-10-24, reopened after the stale-ticket cleanup closed this PR while #2683's fix was absent from master): resolved text conflicts in src/server/handle.ts (#4205 window-caliber stamps — the retirement block now sits after both stamps, before the lastModel stamp) and src/server/relay.ts (#3129 v1 framer insertion, #3701 close-1009 byte-shed branch — kept deliberately un-alerted, only the calibrated arm call applied); extended the calibrated-provenance flag to the two armFailureShrink call sites master added after this branch cut (#3148 circuit-open fail-fast, #3701 oversize resend). Verified on the rebased head: typecheck clean + 49/49 across the targeted baseline/meter suites (issue2683 G1/G2/G3, #1498-F2, context-provenance-outbound and adjacent anchor/floor suites).
What
Fixes #1493 — preflight was being armed against the raw client-history size instead of the compressed outbound payload size. In very long sessions this caused a 42–203s preflight compression on every request once a no-usage upstream failure occurred, saving ~1K tokens each time and then logging that the payload fit anyway.
Root cause
When an upstream request fails without a usage report (relay 5xx or network drop),
armFailureShrinksetsession.stats.lastInputTokens = estimateTokensFast(<entire JSON body>)— a chars/4 count of the whole wire body. That inflates base64 images and dense payloads to raw-history scale (e.g. 1.27M for a ~119K outbound payload). The poisoned value fed two consumers:Math.max(lastInputTokens, payloadEstimate)→ fired preflight every request even though the outbound payload fit the window;#857 had tagged the value
"estimate"to shield evidence-grade consumers (self-heal / #496 gate / retraction), but the preflight floor and nudge baseline still consumed it — an incomplete mitigation.Fix
Measure the arm against the same outbound payload estimate the fit gate already uses. A new
outboundPayloadBreakdown()helper (post-fold content + wire overhead + correctly-billed images) is now the single source of truth shared bypreflightCompressIfNeededandarmFailureShrink.This makes the arm self-correcting: a payload that genuinely overflows the window still arms high enough to break the #604 relay-5xx deadlock (verified — those self-heal tests still pass); one that fits no longer falsely crosses the trigger.
Also downgrades the benign "payload fits with no compressible ranges" preflight log from
warn→info(the user's explicit ask; avoids warn spam in exactly this scenario).Behavior change (disclosed)
The local estimate used to arm the emergency shrink after a no-usage failure changed from raw-body chars/4 (raw-history scale) to the outbound payload estimate (what would actually be sent) — lower and more accurate. A real usage report still overwrites it on success. No change to the config schema, persistence format, wire shape, or the acp-kernel pin.
Pre-flight
npm run typecheck— cleannpm test— 2873 pass / 0 fail (incl. arm emergency shrink on upstream failure to break relay-5xx deadlock #604 genuine-overflow self-heal, image-baseline, nudge suites)npm run build— tsup + dist-import-annotations OKNew regression test
tests/issue1493-arm-outbound-payload.test.tsreproduces the bug (fails on old code: arms to 44191 > 30400 band) and passes on the fix.Related (not changed here, flagged for visibility)
The overflow-not-parseable path (
estimateTokensFast(rawBody)insrc/server.ts) uses the same chars/4 estimator class, but it is clamped tomin(declared, est)and only fires on a real upstream overflow rejection, so it over-shrinks conservatively rather than stalling. Left out of this PR to keep it single-scoped; happy to address separately if you want it tracked.Fixes #1493