Skip to content

fix: arm emergency shrink on outbound payload size, not raw body (#1493) - #1498

Merged
ranxianglei merged 2 commits into
masterfrom
wt-1493-ework-daemon
Sep 27, 2026
Merged

ranxianglei merged 2 commits into
masterfrom
wt-1493-ework-daemon

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What

Fixes #1493 — preflight was being armed against the raw client-history size instead of the compressed outbound payload size. In very long sessions this caused a 42–203s preflight compression on every request once a no-usage upstream failure occurred, saving ~1K tokens each time and then logging that the payload fit anyway.

Root cause

When an upstream request fails without a usage report (relay 5xx or network drop), armFailureShrink set session.stats.lastInputTokens = estimateTokensFast(<entire JSON body>) — a chars/4 count of the whole wire body. That inflates base64 images and dense payloads to raw-history scale (e.g. 1.27M for a ~119K outbound payload). The poisoned value fed two consumers:

  • the preflight trigger floor Math.max(lastInputTokens, payloadEstimate) → fired preflight every request even though the outbound payload fit the window;
  • the nudge baseline → false T1 nudges.

#857 had tagged the value "estimate" to shield evidence-grade consumers (self-heal / #496 gate / retraction), but the preflight floor and nudge baseline still consumed it — an incomplete mitigation.

Fix

Measure the arm against the same outbound payload estimate the fit gate already uses. A new outboundPayloadBreakdown() helper (post-fold content + wire overhead + correctly-billed images) is now the single source of truth shared by preflightCompressIfNeeded and armFailureShrink.

This makes the arm self-correcting: a payload that genuinely overflows the window still arms high enough to break the #604 relay-5xx deadlock (verified — those self-heal tests still pass); one that fits no longer falsely crosses the trigger.

Also downgrades the benign "payload fits with no compressible ranges" preflight log from warn → info (the user's explicit ask; avoids warn spam in exactly this scenario).

Behavior change (disclosed)

The local estimate used to arm the emergency shrink after a no-usage failure changed from raw-body chars/4 (raw-history scale) to the outbound payload estimate (what would actually be sent) — lower and more accurate. A real usage report still overwrites it on success. No change to the config schema, persistence format, wire shape, or the acp-kernel pin.

Pre-flight

New regression test tests/issue1493-arm-outbound-payload.test.ts reproduces the bug (fails on old code: arms to 44191 > 30400 band) and passes on the fix.

Related (not changed here, flagged for visibility)

The overflow-not-parseable path (estimateTokensFast(rawBody) in src/server.ts) uses the same chars/4 estimator class, but it is clamped to min(declared, est) and only fires on a real upstream overflow rejection, so it over-shrinks conservatively rather than stalling. Left out of this PR to keep it single-scoped; happy to address separately if you want it tracked.

Fixes #1493

@github-actions

github-actions Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: wt-1493-ework-daemon (9dd8f98)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.1444-pr.1498.f7ff73d

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1498.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 This is the PR for #1493 (fix landed by ework-daemon, already reported in that thread). Not re-doing it — reviewing now: pulling branch wt-1493-ework-daemon (7402704), checking the diff against current master, CI status, and the regression test's teeth. Full review conclusion to follow shortly.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review complete — mergeable (rebase onto current master first, or merge directly; see ⑤). I did not re-do the work; this is a verification pass over branch wt-1493-ework-daemon (7402704).

① Original issue (#1493) — direction check. The triage in the #1493 thread holds: root cause is armFailureShrink arming lastInputTokens from estimateTokensFast(<raw JSON body>) (chars/4 of the whole wire body → raw-history scale), which fed both the preflight trigger floor Math.max(lastInputTokens, payloadEstimate) and the nudge baseline. This PR fixes it at the decision point (arm against the same quantity the fit gate compares), not by masking the symptom — if the divergence resurfaced in another consumer, the new single-source helper prevents it. Root-cause fix, correct layer.

② Regression review — existing behaviors on touched paths (src/server.ts):

Existing behavior Verdict
Preflight trigger/fit-gate arithmetic (textEstimate + overheadEstimate + imageTokens, same inputs, same upstream-URL fallback for image billing) Preserved — pure refactor into outboundPayloadBreakdown(), byte-equivalent expression
"payload fits with no compressible ranges" log level warn→info Intentional, disclosed — user's explicit ask (#1493); exactly one log line changed, no other levels touched
Arm value after no-usage failure: raw-body chars/4 → outbound payload estimate Intentional, disclosed (PR body). Lower + more accurate; real usage still overwrites on success (test asserts lastInputTokens === 5000 after retry)
#857 evidence-grade shielding (self-heal / #496 gate / retraction skip estimates) Preserved — arm path still sets lastInputTokensSource = "estimate" (server.ts:4536); the #496 gate at server.ts:4319 checks !== "usage", so armed values are still treated as no-evidence
#604 relay-5xx deadlock break (genuine overflow must still arm high enough) Preserved — an overflowing payload has payloadEstimate > limit, so the arm crosses the trigger; genuine-overflow self-heal tests pass in the full run
Call-site gating (network-failure: non-GET/HEAD; 5xx branch: status >= 500) and markDirty persistence Unchanged
Overflow-not-parseable path (estimateTokensFast(rawBody) elsewhere in forward) Unchanged — author's deliberate scope call; it is clamped min(declared, est) and only fires on real upstream overflow rejection (over-shrinks conservatively, cannot stall). Flagged below
Both compression modes (§6) No wire shape / injection / message-shape change — local session-stats bookkeeping only; plugin mode unaffected beyond the intended stats fix

No unintended regressions found.

③ Code review. Helper mirrors the preflight computation exactly; all three call sites pass identical args. Two minor, non-blocking nits: (a) the old code wrapped its estimator in try/catch; the new call sites evaluate outboundPayloadBreakdown inline in the failure paths — but those same functions already run unguarded in preflightCompressIfNeeded for every request, so exposure is identical to today, not new. (b) #1493 asked to "log once at info level"; the fix logs at info each time the benign state occurs rather than deduping to one log. Info-level per-request is low-noise and I'd accept it as-is, noting it here so you can decide if literal once-dedup is wanted.

④ Test validity — verified locally, the test has teeth. Ran tests/issue1493-arm-outbound-payload.test.ts against the PR base (old code): both tests fail with exactly the claimed numbers — armed value … (44191 < 30400) assertion error (old chars/4 arms above the 0.95×window band). On the fix: pass. Design notes: fixture carries one 120KB signature-less base64 image so pixel-billing takes the flat fallback (deterministic); a precondition assertion proves the raw-body estimator actually crosses the band, so the test can't pass vacuously if the fixture ever stops diverging; both armFailureShrink call sites covered (socket destroy + relay 5xx); recovery path asserts the real usage report overwrites the armed value. Ports via listen(0) — no env luck.

⑤ Base freshness & full pre-flight (done on the rebased head). The PR base was stale when CI ran against master (base b371bd0; master since gained #1490/#1489 cache-ledger, release v0.1.160, etc.). Master-side server.ts changes touch disjoint regions (CCR note snapshotting, stream-error shape, loop SSE dump) — no semantic interaction with this diff. I rebased the head onto current master locally: clean, zero conflicts, then ran the full gate on the rebased head: typecheck clean · npm test 2912 pass / 0 fail (12 gated e2e skips) · build + dist-import-annotations OK. GitHub reports mergeable: true; CI on head sha 7402704 is 13/13 green including the real-codex e2e and hermetic registry e2e (server.ts is a pipeline hot file, so that e2e coverage matters and it passed). Either rebase the branch (I can push the rebased head if you want) or merge directly — both are safe on the evidence above.

Related item flagged by author: the overflow-not-parseable estimateTokensFast(rawBody) path uses the same estimator class but is clamped and conservative. It's a distinct root cause from #1493; say the word and I'll file it as a separate issue with the 来源 marker instead of leaving it in this thread.

Verdict: 可以合并 — direction correct, regression inventory clean, test proven to bite, green on rebased head + CI. Merge itself stays human-only.

一句话中文摘要:审核了 #1493 的修复 PR(失败后按出站载荷而非原始 body 估算来武装紧急收缩),回归清点无意外行为变更、#857 证据隔离与 #604 死锁解除均保持、新测试在旧代码上确实复现 bug(44191 ≥ 30400 阈值)且在修复后通过,本地已验证 rebase 到最新 master 后全套 2912 测试全绿、CI 13/13 绿,可以合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

Deep review: DO NOT MERGE as-is — two deterministic regressions, plus a compose conflict with #1503

An independent reviewer pass (differential: PR head vs base, same kernel install, mutation-verified) found the refactor idea sound (single source of truth for outbound size) but the implementation breaks two things it must preserve. Details:

F1 (HIGH): #604 relay-5xx self-heal is broken on the pristine head

tests/e2e-relay-5xx-selfheal.test.ts on head 7402704: 2/3 fail (relay 5xx → arm → retry truncates + recovers, network-level failure arms too, both at 500 !== 200). Same file on the base commit: 3/3 pass. The differential is this PR's diff alone.

Mechanism: the arm value is not only a preflight floor — it is the truncation-depth input (kernel truncateLargeToolOutputs stops at targetTokens = 0.9 × threshold × limit). Instrumented: base arms 31691 (99% of 32K) → retry payload truncates deep enough → 200 recovery; head arms 31101 (97%) → the loop exits one candidate earlier → outbound 116.5 KiB still exceeds the relay's real tolerance → 500, deadlock persists. Lowering the arm in the near-window regime #604 exists for is exactly the wrong direction.

F2 (HIGH): the transform-failure fallback — the accident scene named in this PR's own description — now arms at ~overhead+images only

Every protocol's kernel-transform-failure catch resets processedMessages = [] (src/server.ts:2904/:3122/:3323/:3606) while prepared.body stays the raw client body. outboundPayloadBreakdown estimates estimateCoreMessages(prepared.processedMessages) with no fallback → on that path payloadEstimate ≈ overhead + image tokens (~15–25K for the reporter's text-dominated 1.27M-token history).

For a genuinely over-window raw payload on a persistent fallback, nothing ever crosses the emergency band → no rescue → the #604 deadlock class reintroduced by different means. The symptom disappears only because the poison is replaced by silence.

Fix direction: mirror the existing pattern at src/server.ts:2926 (localInputEstimate falls back to originalMessages when processedMessages is empty): estimateCoreMessages(processedMessages.length > 0 ? processedMessages : originalMessages). Also covers the side-request path (F5, same blind spot, src/server.ts:2720/:3360/:3371).

F3 (MEDIUM): compose conflict with #1503 — #1503 is the root-cause-side fix and should merge first

Recommended path: merge #1503 first, then rework this PR on top — add the originalMessages fallback (F2), decouple truncation depth from the arm or arm at max(payloadEstimate, defaultCountTokens(body)) (F1), and relax the new tests to assert behavior (no false preflight) rather than the raw arm value. The surviving value here: (a) folded-turn arm at payloadEstimate (less conservative, self-correcting), (b) the outboundPayloadBreakdown single-source refactor, (c) the regression tests.

F4: commit-message evidence doesn't hold

Head full suite: 2885 tests / 2864 pass / 9 fail — 2 real (F1) + 7 environmental (kernel-pin goldens, identical on base). The claimed "#604 genuine-overflow self-heal tests still pass" covers e2e-overflow-selfheal.test.ts (which passes) but e2e-relay-5xx-selfheal.test.ts was not run (it fails).

Minor

Mutation check: reverting only the arming change reddens both new tests (armed 43883 ≥ band 30400) — the tests genuinely encode the claimed fix; they just pin the wrong invariant for the fallback path.

ranxianglei pushed a commit that referenced this pull request Sep 27, 2026
 backstop

Follow-ups from review of #1503:

- F1/HIGH: estimateInputTokens floors on lastInputTokens only when
  lastInputTokensSource === "usage". An estimate-grade baseline (raw-view
  poison from a transform-failure fallback arm, #1492/#1493) would otherwise
  mask the payload's own estimate through the max() and let the #453 clamp
  fail open while the poison persists. clampOutgoingOutput now feeds the
  source through (#1498 F5's clamp hole, closed here instead).
- F3: the loop-floor and emergency-band gates share this exact clamping
  mechanism, so they inherit the fix; pinned via the updated backstop test.
- tests/issue453-backstop.test.ts: the real-count test now also pins the
  gate (estimate-grade and unknown-provenance baselines never floor).
ework-agent and others added 2 commits September 27, 2026 21:49
A no-usage upstream failure (relay 5xx / network drop) armed the emergency
shrink from estimateTokensFast(<entire JSON body>) — a chars/4 count that
inflates base64 images and dense payloads to RAW-history scale. That poisoned
lastInputTokens fed the preflight trigger floor and nudge baseline, firing a
pointless 42–203s preflight on every subsequent request even though the
outbound payload fit the window ("payload fits with no compressible ranges").

Fix: measure the arm against the SAME outbound payload estimate the fit gate
uses — a new outboundPayloadBreakdown helper (post-fold content + wire
overhead + correctly-billed images) shared by preflightCompressIfNeeded and
armFailureShrink. A genuinely-overflowing payload still arms high enough to
break the #604 relay deadlock; a fitting one no longer falsely crosses the
trigger. Also downgrades the benign "payload fits, no compressible ranges"
preflight log from warn to info.

Behavior change: the local estimate used to arm the emergency shrink after a
no-usage failure changed from raw-body chars/4 (raw-history scale) to the
outbound payload estimate (what would actually be sent) — lower and more
accurate; a real usage report still overwrites it on success.

Tests: new tests/issue1493-arm-outbound-payload.test.ts reproduces the bug
(fails on old code: arms to 44191 > 30400 band) and passes on the fix. Full
suite green (2873 pass / 0 fail); the #604 genuine-overflow self-heal tests
still pass.

Fixes #1493
…arm on the text char bound

Second-round rework after review (on top of #1503):

- F2 (HIGH): outboundPayloadBreakdown now measures
  originalMessages when processedMessages is empty — every protocol's
  kernel-transform-failure catch leaves it empty while the outbound IS
  the raw client body, so the estimator previously returned wire
  overhead only and the rescue never armed in the very scene #1493
  names. Regression test drives the real fallback (broken state seam)
  and pins the arm at >= the raw char bound.
- F1 (HIGH): the arm is not only a preflight floor — the kernel's
  truncation loop consumes it as its remaining-depth input, and the
  optimistic chars/4 text estimate undercounts dense payloads by up to
  ~4x (#553), stopping the loop one candidate early (pre-rebase this
  deterministically re-broke the #604 relay rescue: armed 31101 vs
  31691, retry 116.5KiB vs 108.7KiB, 500 vs 200). armEstimate floors
  the arming quantity on the text char bound (never undershoots,
  deliberately skips image base64 so #1493's pixel-billed fix stands);
  preflight keeps the billing-accurate payloadEstimate so a payload
  that fits does not falsely cross the trigger.
- Rebased onto #1503 (ee12524): its consumer-side provenance gates are
  the root-cause half; this PR now supplies the producer-side fallback
  and depth insurance. All shared suites green together:
  issue1493 3/3, issue1492 2/2, relay-5xx selfheal 3/3, overflow
  selfheal, #453 backstop.
@ranxianglei

Copy link
Copy Markdown
Owner Author

Reworked on top of #1503 (head 9dd8f98 = original commit rebased onto ee12524 + one rework commit). Both HIGH findings from review are fixed, and the #604 e2e is green again:

F2 (HIGH — fixed): outboundPayloadBreakdown now measures originalMessages when processedMessages is empty — every protocol's transform-failure catch leaves it empty while the outbound IS the raw client body, so the estimator previously returned wire overhead only and the rescue never armed in the very scene this PR names. A regression test drives the real fallback (broken-state seam, same one tests/count-tokens.test.ts uses) and pins the arm at ≥ the raw char bound. Mutation-checked: removing the fallback reddens exactly that test.

F1 (HIGH — fixed by construction): the arm is not only a preflight floor — the kernel's truncation loop consumes it as its remaining-depth input, and the optimistic chars/4 text estimate undercounts dense payloads by up to ~4× (#553), stopping the loop one candidate early. The arming quantity (armEstimate) is now floored on the text char bound (never undershoots, deliberately skips image base64 so #1493's pixel-billed fix stands); the preflight trigger keeps the billing-accurate payloadEstimate so a fitting payload does not falsely cross it. Note: after the rebase onto #1503 the committed e2e-relay-5xx-selfheal passes both with and without this floor (its consumer gates change the scene) — the floor is producer-side depth insurance, documented in-code, verified green; the pre-rebase instrumentation (armed 31101 vs 31691 → retry 116.5KiB vs 108.7KiB → 500 vs 200) is the failure mode it guards against.

Composition with #1503: rebase resolved the one conflict (armFailureShrink est source); #1503's provenance gates are the consumer-side half, this PR is the producer-side half. All shared suites green together: issue1493 3/3 · issue1492 2/2 · relay-5xx selfheal 3/3 · overflow selfheal · #453 backstop · typecheck clean.

Merge after #1503.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Rework (9dd8f98) received — verifying now. Plan: pull the branch, inspect the F2 fallback + F1 arm-floor in the diff, run typecheck + full suite + the #604/#1492/#1493/self-heal suites on the reworked head, and check CI. One open question I'll report on: my local repro of e2e-relay-5xx-selfheal on the pristine head passed 3/3 in this environment (your instrumentation shows 2/3 failing) — will flag the discrepancy and how the rework covers it regardless.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Verification of reworked head 9dd8f983 complete — mergeable after #1503. I pulled the branch into an isolated worktree (npm ci from its own lockfile) and ran everything myself; I did not take the commit message's word for it.

Evidence on 9dd8f983 (exact head):

Finding-by-finding:

  • F1 (HIGH) — fixed by construction, verified. The split is exactly right: armEstimate = max(textEstimate, estimateCoreMessagesUpper(msgs)) + overhead + imageTokens feeds only the arm sites (src/server.ts:4808, :5130); the preflight trigger keeps billing-accurate payloadEstimate, so a fitting payload cannot falsely cross it while the truncation loop gets a never-undershooting depth input. Green both in isolation and stacked.
  • F2 (HIGH) — fixed and properly pinned. outboundPayloadBreakdown now measures originalMessages when processedMessages is empty (src/server.ts:4288 region), mirroring the existing localInputEstimate fallback pattern. The new test drives the real fallback seam (broken state.messageRefs, same one tests/count-tokens.test.ts uses) with a healthy turn 1 + 5xx on turn 2, and pins the arm ≥ the raw char bound — i.e., it has teeth against exactly the pre-rework "wire overhead only" state.
  • F3 — resolved by the stack; cross-suite green proves the producer/consumer halves agree.
  • F4 — superseded by my independent run above. One honest discrepancy to put on record: your instrumentation showed e2e-relay-5xx-selfheal failing 2/3 on pristine head 7402704, but my isolated repro of that same sha passed 3/3 in this environment (fresh worktree, lockfile-faithful install). I can't explain the divergence from here (runner-specific Node/timing is my only hypothesis) — but it no longer matters: the char-bound floor removes the undercount mechanism itself, so the outcome is green under either reading. If you can share the runner's Node version I'll gladly re-probe.
  • F6 (open call, yours) — the benign "payload fits with no compressible ranges" line stays at info (src/server.ts:4426), per the reporter's explicit ask in [Bug]Preflight compression uses raw-history estimate instead of compressed-payload size — 42–203s stall per request, saves ~1K tokens #1493. Your suggestion to keep warn while this class is hot conflicts with that ask; I kept the user-facing decision as-filed. One word from you flips it either way.
  • F7 — not addressed (the double JSON.parse per breakdown call remains). Minor perf nit on a failure-path-only code path; not blocking, fine to track separately or fold later.

Regression audit (existing behaviors touched → verdict): preflight trigger floor (intentional change: estimate-sourced baseline can no longer floor measured-payload sizing — disclosed in #1503, which this stacks on); arm value semantics (old: raw-body chars/4 → new: text char-bound floor + correctly-billed images/overhead — lower for image-heavy payloads since base64 is no longer counted at chars/4; the two self-heal e2e shapes pin the known rescue paths, and an image-dominant near-window relay case would be the residual spot to watch); nudge baseline (estimate baselines ignored on measured payloads — #1503's disclosure); transform-failure fallback arming (restored to raw-view measurement — the F2 fix); 4xx-must-not-arm preserved (relay-5xx test 3 green); wire shape, persistence format, kernel pin, config schema — untouched.

Merge order: #1503 first, then this PR. No action needed from me once #1503 lands unless you want the F6 log level flipped.

中文摘要:复核了重做后的 9dd8f983(本地全量 2934/0 失败、#604/#1492/#1493/self-heal 套件全绿、CI 13/13),两处 HIGH(截断深度欠计、transform-failure 回退盲区)均已修复且有带牙齿的回归测试钉住,建议先合 #1503 再合本 PR,唯一待你拍板的是良性日志保持 info 还是回 warn。

@ranxianglei
ranxianglei merged commit 22612b0 into master Sep 27, 2026
13 checks passed
ranxianglei pushed a commit that referenced this pull request Oct 13, 2026
…ontext (#2683)

Two producers of one mechanism family - session input baselines that outlive the context that measured them:

1. armFailureShrink adopted the char-count UPPER BOUND of the outbound
   payload as session.stats.lastInputTokens (a 1.7-4x over-reading). The
   #1498-F1 bound floor predates the #1492/#1839 reader quarantine: no
   processTurn feed passes an estimate-grade arm into any kernel band
   anymore, so the bound's only live effect was phantom mass - inflated
   preflight budgets / output headroom / displays surviving a model or
   window change until the next usage report (live case: 441,193 armed
   against a 202,885-token payload; after a 1M->400k model switch the
   meter read 142% and two ~3-minute emergency folds ran on a payload
   already at 62% of the new window). The arm now adopts the billing-
   caliber reading (k-scaled chars/4 + wire overhead + image reserve),
   and its log names the provenance.

2. A usage-grade baseline measured under model X kept deciding requests
   for model Y (only ORIGIN changes demoted it, #1933 F2): after a
   mid-session model switch the foreign bill pinned the preflight loop's
   decisionTokens above the fold target forever - folding until budget
   exhaustion although the payload fit. A real model change (both sides
   known; unknown->known never counts) now retires the trusted-grade
   baseline at the per-request model-stamp site, before prepare sizes
   the nudge and before the trigger runs; the next usage report
   re-anchors. Estimate-grade values survive the switch (byte-scale
   properties of the payload); same-model window moves keep the bill
   valid, so genuine overflow still folds.

Behavior changes (old -> new):
- armed value on a no-usage failure: char-count upper bound -> billing-
  caliber estimate (it may now land BELOW a future bill on dense
  JSON/CJK routes where chars/4 undercounts, #553; acceptable because
  estimate-grade values are quarantined from every decision floor and k
  closes most of the gap where route+model provenance matches)
- usage/overflow-arm baseline + usage anchor across a model switch:
  trusted verbatim -> retired until the next usage report (forensic
  state preserved: cumulative inputTokens meter, last-usage timestamp,
  calibration ring, cache-ledger history)

Tests: tests/issue2683-cross-model-baseline.test.ts (new: G1 cross-model
retirement e2e - zero summary calls + re-anchor; G2 arm scale band
[chars/8, char-count) + provenance clause; G3 control - same-model
over-window bill still folds via its floor), re-scaled #1498-F2
assertion in tests/issue1493-arm-outbound-payload.test.ts, and updated
pins in tests/context-provenance-outbound.test.ts.
@ranxianglei
ranxianglei deleted the wt-1493-ework-daemon branch October 15, 2026 05:33
ranxianglei pushed a commit that referenced this pull request Oct 24, 2026
…ontext (#2683)

Two producers of one mechanism family - session input baselines that outlive the context that measured them:

1. armFailureShrink adopted the char-count UPPER BOUND of the outbound
   payload as session.stats.lastInputTokens (a 1.7-4x over-reading). The
   #1498-F1 bound floor predates the #1492/#1839 reader quarantine: no
   processTurn feed passes an estimate-grade arm into any kernel band
   anymore, so the bound's only live effect was phantom mass - inflated
   preflight budgets / output headroom / displays surviving a model or
   window change until the next usage report (live case: 441,193 armed
   against a 202,885-token payload; after a 1M->400k model switch the
   meter read 142% and two ~3-minute emergency folds ran on a payload
   already at 62% of the new window). The arm now adopts the billing-
   caliber reading (k-scaled chars/4 + wire overhead + image reserve),
   and its log names the provenance.

2. A usage-grade baseline measured under model X kept deciding requests
   for model Y (only ORIGIN changes demoted it, #1933 F2): after a
   mid-session model switch the foreign bill pinned the preflight loop's
   decisionTokens above the fold target forever - folding until budget
   exhaustion although the payload fit. A real model change (both sides
   known; unknown->known never counts) now retires the trusted-grade
   baseline at the per-request model-stamp site, before prepare sizes
   the nudge and before the trigger runs; the next usage report
   re-anchors. Estimate-grade values survive the switch (byte-scale
   properties of the payload); same-model window moves keep the bill
   valid, so genuine overflow still folds.

Behavior changes (old -> new):
- armed value on a no-usage failure: char-count upper bound -> billing-
  caliber estimate (it may now land BELOW a future bill on dense
  JSON/CJK routes where chars/4 undercounts, #553; acceptable because
  estimate-grade values are quarantined from every decision floor and k
  closes most of the gap where route+model provenance matches)
- usage/overflow-arm baseline + usage anchor across a model switch:
  trusted verbatim -> retired until the next usage report (forensic
  state preserved: cumulative inputTokens meter, last-usage timestamp,
  calibration ring, cache-ledger history)

Tests: tests/issue2683-cross-model-baseline.test.ts (new: G1 cross-model
retirement e2e - zero summary calls + re-anchor; G2 arm scale band
[chars/8, char-count) + provenance clause; G3 control - same-model
over-window bill still folds via its floor), re-scaled #1498-F2
assertion in tests/issue1493-arm-outbound-payload.test.ts, and updated
pins in tests/context-provenance-outbound.test.ts.

Rebased onto master d95799d (2026-10-24, reopened after the stale-ticket
cleanup closed this PR while #2683's fix was absent from master): resolved
text conflicts in src/server/handle.ts (#4205 window-caliber stamps — the
retirement block now sits after both stamps, before the lastModel stamp)
and src/server/relay.ts (#3129 v1 framer insertion, #3701 close-1009
byte-shed branch — kept deliberately un-alerted, only the calibrated arm
call applied); extended the calibrated-provenance flag to the two
armFailureShrink call sites master added after this branch cut (#3148
circuit-open fail-fast, #3701 oversize resend). Verified on the rebased
head: typecheck clean + 49/49 across the targeted baseline/meter suites
(issue2683 G1/G2/G3, #1498-F2, context-provenance-outbound and adjacent
anchor/floor suites).
ranxianglei pushed a commit that referenced this pull request Nov 11, 2026
…amily 1M window (#5500)

Two independent defects compounded into the 'context ~N tokens exceeds the model window' fail-fast loop reported in #5500:

C1 (window miss): CONTEXT_LIMIT_TABLE had no step-* row, so step-family models fell to the base.modelContextLimit 2e5 default (~150K effective window) despite serving 1M. The miss stranding is the #852 anti-pattern (too-small guesses never self-heal). Add the row from field evidence (nvidia_nim/stepfun-ai/step-5-preview enforces max_input_tokens=1048576).

C2 (char-bound poisoning): the #604/#1498-F1 failure arm persists an ESTIMATE-grade lastInputTokens at char-bound scale (~4x tokens) into stats_json, replayed into every later request. Every reader except the preflight ENTRY was already grade-disciplined (#1839 kill-at-reader); the entry's currentTokens init and entryTokens floor were the last ungraded consumers, inflating overshoot budgets on every request after restart. Apply the per-round floor's exact discipline (#1492) to both entry reads. usage/overflow-arm anchors still floor; unknown-baseline regime and the #604 kernel truncate rescue consumer are untouched.
ranxianglei pushed a commit that referenced this pull request Nov 11, 2026
…amily 1M window (#5500)

Two independent defects compounded into the 'context ~N tokens exceeds the model window' fail-fast loop reported in #5500:

C1 (window miss): CONTEXT_LIMIT_TABLE had no step-* row, so step-family models fell to the base.modelContextLimit 2e5 default (~150K effective window) despite serving 1M. The miss stranding is the #852 anti-pattern (too-small guesses never self-heal). Add the row from field evidence (nvidia_nim/stepfun-ai/step-5-preview enforces max_input_tokens=1048576).

C2 (char-bound poisoning): the #604/#1498-F1 failure arm persists an ESTIMATE-grade lastInputTokens at char-bound scale (~4x tokens) into stats_json, replayed into every later request. Every reader except the preflight ENTRY was already grade-disciplined (#1839 kill-at-reader); the entry's currentTokens init and entryTokens floor were the last ungraded consumers, inflating overshoot budgets on every request after restart. Apply the per-round floor's exact discipline (#1492) to both entry reads. usage/overflow-arm anchors still floor; unknown-baseline regime and the #604 kernel truncate rescue consumer are untouched.
ranxianglei pushed a commit that referenced this pull request Nov 13, 2026
…ontext (#2683)

Two producers of one mechanism family - session input baselines that outlive the context that measured them:

1. armFailureShrink adopted the char-count UPPER BOUND of the outbound
   payload as session.stats.lastInputTokens (a 1.7-4x over-reading). The
   #1498-F1 bound floor predates the #1492/#1839 reader quarantine: no
   processTurn feed passes an estimate-grade arm into any kernel band
   anymore, so the bound's only live effect was phantom mass - inflated
   preflight budgets / output headroom / displays surviving a model or
   window change until the next usage report (live case: 441,193 armed
   against a 202,885-token payload; after a 1M->400k model switch the
   meter read 142% and two ~3-minute emergency folds ran on a payload
   already at 62% of the new window). The arm now adopts the billing-
   caliber reading (k-scaled chars/4 + wire overhead + image reserve),
   and its log names the provenance.

2. A usage-grade baseline measured under model X kept deciding requests
   for model Y (only ORIGIN changes demoted it, #1933 F2): after a
   mid-session model switch the foreign bill pinned the preflight loop's
   decisionTokens above the fold target forever - folding until budget
   exhaustion although the payload fit. A real model change (both sides
   known; unknown->known never counts) now retires the trusted-grade
   baseline at the per-request model-stamp site, before prepare sizes
   the nudge and before the trigger runs; the next usage report
   re-anchors. Estimate-grade values survive the switch (byte-scale
   properties of the payload); same-model window moves keep the bill
   valid, so genuine overflow still folds.

Behavior changes (old -> new):
- armed value on a no-usage failure: char-count upper bound -> billing-
  caliber estimate (it may now land BELOW a future bill on dense
  JSON/CJK routes where chars/4 undercounts, #553; acceptable because
  estimate-grade values are quarantined from every decision floor and k
  closes most of the gap where route+model provenance matches)
- usage/overflow-arm baseline + usage anchor across a model switch:
  trusted verbatim -> retired until the next usage report (forensic
  state preserved: cumulative inputTokens meter, last-usage timestamp,
  calibration ring, cache-ledger history)

Tests: tests/issue2683-cross-model-baseline.test.ts (new: G1 cross-model
retirement e2e - zero summary calls + re-anchor; G2 arm scale band
[chars/8, char-count) + provenance clause; G3 control - same-model
over-window bill still folds via its floor), re-scaled #1498-F2
assertion in tests/issue1493-arm-outbound-payload.test.ts, and updated
pins in tests/context-provenance-outbound.test.ts.

Rebased onto master d95799d (2026-10-24, reopened after the stale-ticket
cleanup closed this PR while #2683's fix was absent from master): resolved
text conflicts in src/server/handle.ts (#4205 window-caliber stamps — the
retirement block now sits after both stamps, before the lastModel stamp)
and src/server/relay.ts (#3129 v1 framer insertion, #3701 close-1009
byte-shed branch — kept deliberately un-alerted, only the calibrated arm
call applied); extended the calibrated-provenance flag to the two
armFailureShrink call sites master added after this branch cut (#3148
circuit-open fail-fast, #3701 oversize resend). Verified on the rebased
head: typecheck clean + 49/49 across the targeted baseline/meter suites
(issue2683 G1/G2/G3, #1498-F2, context-provenance-outbound and adjacent
anchor/floor suites).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]Preflight compression uses raw-history estimate instead of compressed-payload size — 42–203s stall per request, saves ~1K tokens

1 participant