Skip to content

track: plugin-mode proxy-side emergency truncation (last-resort backstop, deferred from #467) #470

Closed

Description

@ranxianglei

来源: #453 (#453) 分析时发现;owner 在 #453 分阶段方案确认中明确要求「另开 issue 单独跟踪」;PR #467 (#467) 有意未包含此项。

Background

#453: in plugin mode (bili pi) bili trusts the ACP agent to run compress. When a weak model ignores the nudge, the payload grows past the window and the upstream 400s on input+output (vLLM rejects an oversized total instead of clamping) — then the client falls back to native compaction and loses all working state.

PR #467 ships two host-side backstops that cover the common path:

  1. Clamp the outgoing output budget — max_tokens/max_completion_tokens/max_output_tokens capped to window − estimated_input − margin, so input+output can no longer exceed the window on request-rebuilding upstreams (both modes).
  2. Escalate the nudge — force-inject every turn once usage ≥ EMERGENCY_NUDGE_ESCALATION_PCT (0.70), filling the sub-0.75 cadence-silent zone (kernel's pressure branch ≥0.75 already force-injects; verified against decideNudge).

This issue tracks the third line of defense deliberately NOT included in #467: proxy-side emergency truncation in plugin mode.

Why it is deferred (carrier-consistency risk)

The existing preflight/self-heal truncation path builds tool-call-less blocks whose acp_summary is re-voiced as a user message by systemToUser — that is proxy-mode semantics, where the proxy owns compression and the ephemeral tool call never enters the client's history.

In plugin mode the contract is inverted: the ACP-native agent OWNS compression — the compress call + result live in the agent's own re-sent history, and the wire carrier is the tool call itself (proxy suppresses tool/nudge injection of its own; the agent's view never renders the kernel's acp_summary). If the proxy unilaterally compresses/truncates ranges the agent has not compressed:

So this needs a design pass before any implementation, e.g.:

  • can the proxy restrict emergency truncation to ranges the agent provably cannot claim (already-blocked / protected-only zones)?
  • or notify/reserve the ranges in the agent's ref space so its next re-send stays consistent?
  • what happens to the agent's pending compress calls that targeted now-truncated refs?

Expected shape (for discussion, not a commitment)

  • Last resort only: engage when input alone nearly fills the window (the territory where the output-clamp floor no-ops) and preflight/self-heal did not already fire.
  • Must preserve consistency between the agent's re-sent history and the proxy's block/state store.
  • Strictly lower priority than fix(#453): hard backstop for plugin-mode context overflow #467's backstops: the clamp guarantees no more input+output 400s, the escalation nudge gives the agent repeated chances to self-compress; truncation is the final fallback for models that ignore both.

References

Activity

  1. ranxianglei commented on Sep 2, 2026

    @ranxianglei
    OwnerAuthor

    🤖 ework agent · qwen3.8-27b

    [bot] 🏷 Design pass complete. I traced the actual code paths (master@60e76f6) instead of taking the framing at face value, and one finding reframes the whole issue.

    Ground truth: plugin-mode proxy-side compression ALREADY EXISTS

    The issue frames "proxy-side emergency truncation in plugin mode" as a new third line of defense to design. It already ships today — via preflightCompressIfNeeded, which runs unconditionally in both modes:

    • src/server.ts:1273 — called inside if (!countTokens && !responsesCompact), with no pluginMode gate.
    • src/server.ts:2180-2268 — takes no pluginMode param; fires purely on tokenCount >= limit where tokenCount = max(lastInputTokens, estimateCoreMessages(processedMessages)) (line 2200). On fire it calls preflightCompress → applyRanges(...) (src/preflight.ts:321), creating persistent, tool-call-less blocks.
    • src/server.ts:1338 (comment): "Blocks created without a tool call (preflight compression…) have NO other carrier… it must survive."
    • src/server.ts:1549 + 1551 — stripKernelSummaries + systemToUser run in prepareOpenai regardless of mode (pluginMode only gates injectTools, line 1518). A preflight block has no compressCallId, so its acp_summary is not stripped (server.ts:1349-1357 only strips carried summaries) and is re-voiced as a user message by systemToUser.

    Corroborated by tests/plugin-nudge-injection.test.ts:13-19: plugin mode's only backstop is literally described as "only preflight at the hard limit."

    So the exact carrier-inconsistency the issue warns about — a proxy-created, agent-unproduced, user-voiced acp_summary over a range the agent still holds unfolded — is live behavior whenever preflight fires in plugin mode, not a hypothetical cost of a new feature. (The README table cell "Preflight blocks (no tool call)? No" for plugin mode appears inaccurate.)

    What the hazard actually is (softer than stated)

    Tracing the fold mechanics, the divergence is milder than the OP suggests:

    • Refs are content-fingerprinted and stable across turns. When the agent re-sends the still-unfolded range R next turn, the kernel folds R into the proxy's block on the wire — the model sees the summary, not both. No double-render; orphan-gc only reaps fully orphaned blocks (src/orphan-gc.ts:43), and R's ids are still present so the block stays anchored.
    • The real cost is view divergence: the agent believes R is still in context, but the upstream model only sees the summary. If the agent later cites R's verbatim content or compresses a range overlapping R, it works from a stale mental model → fidelity loss / mild confusion. The [Bug] ACP: ≥2 compressed blocks render as multiple system messages → SGLang (vLLM-compatible) upstreams return 400 #377 single-system constraint is already handled by systemToUser.

    Net: existing plugin-mode preflight is "works but degrades fidelity," not "broken."

    The residual gap after #467

    #467 is still open (not merged); its clamp kills the common input+output 400. The only input-reduction territory left is input alone ≈ full window (where the clamp's OUTPUT_CLAMP_FLOOR=1024 no-ops). Two sub-problems sit there:

    1. Preflight's trigger still undershoots. plugin mode: no hard backstop when the agent ignores the compress nudge — context grows past the window and upstream 400s #453 found estimateCoreMessages excludes system+tools (~10-20k). fix(#453): hard backstop for plugin-mode context overflow #467 fixed that undershoot for the clamp (estimateInputTokens) but not for preflight's trigger (src/server.ts:2200 still uses estimateCoreMessages). So the mechanism that should catch input-alone overflow still fires late or misses in exactly this territory.
    2. When preflight does fire here, it emits the fidelity-divergence folds above.

    How deep is the tail? Input alone must approach the entire window (not just window−maxOutput) before the clamp can't save us — much rarer than the #453 repro.

    Recommendation (clear bottom line, not a menu)

    1. Ship now — small, both modes, high leverage: fix preflight's trigger undershoot by reusing fix(#453): hard backstop for plugin-mode context overflow #467's estimateInputTokens (or folding system+tools into the estimate at src/server.ts:2200). This makes the backstop that already exists reliably engage for input-alone overflow, in both modes, minimal risk. Arguably a completion of plugin mode: no hard backstop when the agent ignores the compress nudge — context grows past the window and upstream 400s #453/fix(#453): hard backstop for plugin-mode context overflow #467 (they fixed the clamp side of the same undershoot and left this side).
    2. Recommend AGAINST a second persistent-fold path. Naive "emergency truncation that creates blocks behind the agent's back" adds divergence cost with zero new capability — preflight already provides persistent folds. Don't build it.
    3. Only if you want divergence-free host-side terminal behavior for the deep-tail case: the principled design is a stateless, request-scoped drop — plugin mode only, engage when input-alone ≥ window AND preflight didn't already bring it under; drop the oldest safe-boundary whole exchanges (complete user→assistant turns, no dangling tool-call/result pairs) from the outgoing payload for THIS request only; mutate no persistent state (no blocks/carriers/refs). By construction it can't fork the agent's view: nothing recorded, refs stable, the agent's next re-send re-includes the dropped exchanges (re-dropped next turn if still over; blind spot closes once the agent finally compresses). This answers the OP's three sub-questions directly: (1) no need for "zones the agent can't claim" — we act transiently on the full history, leaving no claim; (2) no need to notify/reserve refs — nothing persistent changes; (3) the agent's pending compress calls targeting dropped refs still resolve, since refs are stable and we recorded no competing block. Treat as optional polish for a rare case, not a priority.
    4. Bottom line: I'd resolve this as "by-design: preflight is the plugin-mode last resort; ship the trigger fix (item 1); the extreme input-alone case falls through to the client's native compaction by design" — unless you specifically want item 3, in which case the spec above is ready to implement.

    Happy to (a) open the trigger-fix as a small PR (standalone, or folded into #467 before it merges), and (b) correct the README cell. Say which way you want to take it and I'll go.

  2. ranxianglei commented on Sep 6, 2026

    @ranxianglei
    OwnerAuthor

    Fixed in PR #576 (branch 2026-09-06_preflight-wire-overhead, commit 8e0a6ad).

    Root cause

    The preflight trigger and both post-fold fit gates estimated the payload with estimateCoreMessages (messages only). But the billed input also carries the system prompt and tool definitions — including the proxy-injected compress system prompt (~1.9k tokens) and the ACP tool definitions (~0.5k), plus whatever the client sends (~10-20k on agent clients). So the trigger fired late: the text alone "fit" the window while the real input already overflowed it, guaranteeing a 400 from the upstream. #467 fixed the clamp side only.

    Fix

    • src/server.ts: new estimateWireOverhead(protocol, body) — parses the rebuilt wire body and counts system/instructions text (anthropic string + array blocks, openai hoisted system/developer messages, responses instructions) + tools JSON.
    • Trigger (preflightCompressIfNeeded) and both post-fold fit gates now add the overhead term.
    • src/preflight.ts: PreflightDeps.wireOverhead — the fold loop's budget accounts for it too, otherwise it stops at "text is small enough" while the caller's fit gate still overflows → 502.
    • README plugin-mode preflight cell corrected: it's a last-resort backstop that fires in both modes.

    Tests

    • tests/wire-overhead.test.ts (6 unit tests: per-protocol system extraction, bad JSON → 0, empty payload).
    • tests/preflight-fail-fast.test.ts e2e: text alone fits a 10k window, overhead pushes it to 14130 → preflight folds and forwards.
    • tests/issue453-backstop.test.ts clamp e2e redesigned (window 400k / max_tokens 150k) so the clamp band and preflight band are properly separated; asserts max_tokens < 150k when only the clamp sees the collision.
    • tests/issue330-soft-zone.test.ts big1/big2 resized 32000→24000 chars (old size forwarded a payload whose billed input exceeded the window — the new fit gate correctly rejects it).

    Pre-flight: typecheck ✓, npm test 1070/1072 (2 known env fails in plugin-agent.test.ts), build ✓.

    Out of scope (not done): stateless request-scoped drop — deferred; the persistent-fold alternative from the design discussion was rejected.

  3. ranxianglei commented on Sep 6, 2026

    @ranxianglei
    OwnerAuthor

    🤖 ework agent · qwen3.8-27b

    [bot] 🏷 Independently verified PR #576 (fetched 2026-09-06_preflight-wire-overhead @ 8e0a6ad, reviewed the diff, ran typecheck + the touched tests on that commit). It matches the recommendation from my design pass — item 1 — and goes further than my sketch in good ways.

    What I checked on the actual code:

    • Ordering is sound. prepared = runPrepare() (src/server.ts:1497) builds the full rebuilt body before preflightCompressIfNeeded runs (src/server.ts:1499), so estimateWireOverhead(prepared.protocol, prepared.body) counts the real forwarded system+tools, not a pre-injection estimate.
    • No per-protocol double-count. OpenAI: injectOpenaiSystem (from acp-kernel/wire) returns a messages array with system prepended as role=system/developer entries and there's no top-level system field → only the in-array branch counts. Anthropic: top-level system field (src/server.ts:1717) is read directly; Responses: instructions. Each protocol's overhead counted exactly once. Bad JSON → 0 (no crash), empty payload → ~0.
    • Consistent across all four decision sites. The overhead term is added at the trigger (payloadEstimate, src/server.ts:~2563), passed into the fold loop via PreflightDeps.wireOverhead (src/preflight.ts), and applied to both post-fold fit gates (src/server.ts:~2658). Your extra catch on the fold-loop budget is the important one — without it the loop stops at "text fits" while the caller's gate still overflows → the 502 you called out. That path wasn't in my original analysis; nice find.
    • Parsed-the-actual-body approach is more robust than my "reuse estimateInputTokens" sketch because it counts what's really on the wire rather than re-deriving it.

    Verification results: npm run typecheck ✓ clean; touched tests wire-overhead / preflight-fail-fast / issue453-backstop / issue330-soft-zone → 21/21 pass. README plugin-mode preflight cell corrected as flagged in the design pass.

    Scope: agree the stateless request-scoped drop stays deferred and the persistent-fold alternative is correctly rejected — consistent with recommendations #2/#4. Net resolution of this tracking issue: preflight IS the plugin-mode last resort; #576 makes that backstop reliable by counting wire overhead; the extreme input-alone case falls through to client native compaction by design.

    #576 is open and mergeable (mergeable_state: clean). I can't merge it (human-only). Once it lands I'd close #470 as resolved-by-#576 — say the word after merge and I'll close it, or close it now if you prefer tracking closed ahead of merge.

  4. added a commit that references this issue on Sep 6, 2026
  5. added a commit that references this issue on Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions