Repository navigation
fix: arm emergency shrink on upstream failure to break relay-5xx deadlock (#11) - #229
ranxianglei wants to merge 1 commit into
Conversation
…lock (#11) A failed request (relay 5xx, network error) never reports usage, so lastInputTokens stays frozen and every retry re-sends the same oversized payload — the relay fails it again. On 5xx and network-level failures, arm the emergency shrink with a local estimate of the body just sent: the next turn's processTurn fires the kernel's emergency nudge and tool-result truncate server-side, shrinking the payload without model cooperation. 4xx responses are excluded (auth/quota errors must not distort the usage signal).
📦 Built Package ArtifactBranch: Option A — Install from npm PR tag (recommended)npm install -g billion-context@pr-229Each push to this PR publishes a new version under the Option B — Download artifact
tar xzf billion-context-pr229.tgz
npm install -g packageThis comment is automatically updated on each push. |
|
Review vs master 3cd85b6: What holds:
What changed on master since (rebase must account for):
Not mergeable as-is; keep + rework per #604. (tracking: #604) |
|
Closing — superseded by #611 (merged in v0.1.90), which arms |
Closes #604 — tracking issue for this PR.
Problem (#11)
zcode user on a justwoker relay channel: every request for claude-opus-5 returns 500 after installing bili. Log shows:
{"error":{"type":"new_api_error","message":"upstream error: do request failed"}}— the relay's own error (relay → its upstream failed), fixed ~31s timeout, plus one Cloudflare 522Deadlock: compression triggering depends on the upstream usage report (
lastInputTokens). A failed request never reports usage → the proxy re-sends the same ~250K payload on every retry → the relay fails it again → forever. The session can never recover through bili.Fix
Arm the emergency shrink on failures with a local estimate of the body actually sent:
armFailureShrink()(src/server.ts) —estimateTokensFast(body)raisessession.stats.lastInputTokensif larger, thenmarkDirty(error paths return before forward()'s trailing save). Deliberate exception to the "tokenCount must be real usage" invariant: it only RAISES the value (lower bound → compress earlier, never later), the kernel no-ops below truncate.threshold, and the next successful usage report overwrites it.Next turn,
processTurnsees usage ≥100% → the kernel's emergency nudge +emergency-truncatenode truncate large old tool results server-side (no model cooperation — critical, because the model never responds in this state). The resent payload shrinks below the relay's tolerance and the session recovers.Tests
tests/e2e-relay-5xx-selfheal.test.ts(2 new e2e tests, mirroring the overflow-selfheal harness):[truncated for context space], shrank, upstream 200 → real usage report overwrites the armed valuelastInputTokensuntouchedFull suite: 567 tests pass, typecheck + build clean.