Skip to content

fix: isolate side requests (title-gen) from main session kernel state - #398

Merged
ranxianglei merged 3 commits into
masterfrom
2026-08-31_side-request-isolation
Sep 4, 2026
Merged

ranxianglei merged 3 commits into
masterfrom
2026-08-31_side-request-isolation

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Fixes #388

Side requests (title-gen / small utility calls) share the main session key but were running the full prepare pipeline. A 56-token title-gen request could clobber the main session's acp_status view, nudge baseline, and per-block survivedCount.

Root cause

  • rememberPluginMessages(...) ran for every forwarded request (incl. side requests) — acp_status reads that "last prepare view".
  • prepareAnthropic/prepareOpenai/prepareResponses had no side-request guard → side requests ran processTurn (assignRefs/advanceSurvival), rewrote session.stats.contextTokens (nudge baseline), and called snapshotMessages.
  • isTitleGen existed but only gated tool injection, never the snapshot/state.

Change

  • isSideRequest(parsed) (src/server.ts): tiny output budget — max_tokens / max_completion_tokens / max_output_tokens ≤ 200 (same heuristic as prepareOpenai's isTitleGen). Missing/non-positive budget is never a side request.
  • handle(): side requests (non-count-tokens, non-responses-compact) are routed as pure passthrough (forward with prepared=null) — no processTurn, no snapshot, no usage capture, no stats mutation, no rememberPluginMessages. The main session's kernel state is driven only by main requests.
  • Regression test tests/side-request-isolation.test.ts: unit test for isSideRequest + e2e asserting a side request leaves kernel state / stats / snapshot / nudge baseline byte-identical while main requests advance it (monotonic).
  • launcher-plugin-mode tests used an unrealistic max_tokens:10 that collided with the heuristic; bumped to 8192 (they verify binding, not budget).

Pre-flight

  • npm run typecheck clean
  • npm test 870/870 pass
  • npm run build OK
  • E2E codex suite skipped locally (no codex binary / local upstream); runs in CI

Closes #388

…#388)

Side requests (title-gen / small utility calls) share the main session key
but were running the full prepare pipeline: processTurn (assignRefs /
advanceSurvival), session.stats.contextTokens rewrite, snapshotMessages, and
rememberPluginMessages. A 56-token title-gen request could clobber the main
session's acp_status view, nudge baseline, and per-block survivedCount.

- Add isSideRequest(parsed): tiny output budget (max_tokens /
  max_completion_tokens / max_output_tokens <= 200, same heuristic as
  prepareOpenai's isTitleGen). Missing/non-positive budget is never a side req.
- In handle(), route side requests (non-count-tokens, non-responses-compact)
  as pure passthrough (forward with prepared=null): no processTurn, no
  snapshot, no usage capture, no stats mutation, no rememberPluginMessages.
- Regression test tests/side-request-isolation.test.ts: unit test for
  isSideRequest + e2e asserting a side request leaves kernel state, stats,
  snapshot and nudge baseline byte-identical while main requests advance it.
- launcher-plugin-mode tests used an unrealistic max_tokens:10 that collided
  with the heuristic; bumped to 8192 (they verify binding, not budget).

Pre-flight: typecheck clean, 870/870 tests pass, build OK. E2E codex suite
skipped locally (no codex binary / local upstream); runs in CI.
@github-actions

github-actions Bot commented Aug 31, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-08-31_side-request-isolation (8b88162)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-398

Each push to this PR publishes a new version under the pr-398 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr398.tgz
npm install -g package

This comment is automatically updated on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 ework agent · vllm-qwen/qwen3.8-27b

[bot] 🏷 已审查本 PR(分支 2026-08-31_side-request-isolation,单 commit 10c872a)。结论:修复正确、最小、有回归测试覆盖,四项验收标准全满足。建议合并前在 CI 跑一次 E2E codex 套件(见末尾)。

正确性(src/server.ts,已对上下文代码逐行核对,非只看 diff):

  • isSideRequest(parsed)(:499):非对象 / 缺失 / 非正数 / 字符串预算 → false;max_tokens / max_completion_tokens / max_output_tokens ≤ 200 → true。与 prepareOpenai 的 isTitleGen 同阈值,边界(200 是、201 否)正确。
  • 守卫(:1109):!countTokens && !responsesCompact && protocol !== null && isSideRequest(parsed) → forward(..., null, ...) + return。位置在 getSession 之后(日志需要 session.id)、在 prepare 管线(:1114–1237)之前 → 侧请求跳过 processTurn(assignRefs/advanceSurvival)、snapshotMessages、rememberPluginMessages(:1232)、usage 采集、stats 变更。forward 签名与调用一致,prepared=null 走纯透传,不碰内核状态。

pre-flight(本地实跑,非引用 PR 作者结论):

  • npm run typecheck clean
  • npm test 870/870 通过
  • npm run build OK
  • 新增 tests/side-request-isolation.test.ts 2/2 通过

测试质量:

  • 单测:三个协议字段名、边界 200/201、非对象 / null / 字符串 / 0 / 负数全覆盖。
  • e2e:主请求1(max_tokens=1024)→ 断言内核 state 变更 + refs 分配 + lastInputTokens=50000 + snapshot 设置;侧请求(max_tokens=100,同一 session key)→ 断言 state / stats / lastMessages / lastInputTokens / requests 逐字节不变;主请求2 → 断言 state 单调推进。mock upstream 对侧请求回 tiny input_tokens(56)、主请求回 large(50000),直接复现污染场景。
  • launcher-plugin-mode.test.ts 把不真实的 max_tokens:10 改为 8192(这些测试验证绑定而非预算)——合理,避免与新启发式冲突。

边界/注意点(不阻塞):

  • 启发式(max_tokens ≤ 200)可能把"用户显式设小 max_tokens 的正常轮次"(如 max_tokens=100 问个简单问题)误判为侧请求 → 该轮跳过压缩。罕见、低影响,可接受(否则无法修 bug)。
  • 守卫在 plugin 绑定逻辑(:1066–1089)之后,侧请求也会跑那段;实际无害——title-gen 不携带 plugin header / prompt_cache_key,不会误消费 register。

合规:分支名 YYYY-MM-DD_short-title ✓、单 fix: commit ✓、version 未动 ✓。

建议(合并前):AGENTS.md 要求改动 request pipeline(server.ts)合并前跑 E2E codex 套件。本沙箱无 codex 二进制 / 本地 upstream,跑不了(与 PR 作者说明一致)。请人工触发 manual-dispatch CI(ci-e2e.yml)跑一次 4-phase core 后再合并。

验收标准对照(来自 #388):✅ 侧请求前后主会话 acp_status 桶 / ranges / nudge 基线 / 各块 survivedCount 不变;✅ stats.inputTokens 不被 6-token 级侧请求覆盖;✅ bili export 视图不冻结;✅ 混合主/侧流内核 state 单调仅由主请求驱动。

LGTM,待人工合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

Review 结果 + 彻底修复已推送(8b88162)

原实现(forward(prepared=null) 纯透传)方向正确、#388 验收全部达成,但丢了两样真实功能,5 个既有测试挂:

失败测试 原因
#460 openai title-gen strip(流式/非流式 ×2) title-gen 透传后不再剥模型回显的 acp 渲染标签 —— 而 #460 本来就是「title-gen 泄漏标签」的真实报告
#473 fake-completion retry 夹具 max_tokens:64 被判为侧请求,跳过重试
#411 abort-usage ×2 夹具 max_tokens:10 同上
#475 G3 夹具 max_tokens:100,fake-completion 缓冲路径不再被覆盖

修复(本次提交)

效果

预检(合并后重跑)

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 ework agent · qwen3.8-27b

[bot] 🏷 独立复核了跟进提交 8b88162(rebase 到 v0.1.82 之后),结论:LGTM。你指出的「纯透传丢 #460」是真实回归——我第一轮 LGTM 基于旧 base(v0.1.67),那时 #460 的完整测试集还没进来,rebase 后才暴露;你的修复方向正确。

核心改动(逐行追了 forward())

夹具分离干净

我的预检(8b88162,自跑)

合并前建议:AGENTS.md 要求请求管线改动(server.ts 命中)合并前跑 E2E codex 套件。本地无配置好的 upstream,E2E 跑不了 → 请手动触发 CI(.github/workflows/ci-e2e.yml,需 E2E_UPSTREAM_URL/KEY secret)再合。

按规则我不合并(PR merge 仅人工)。整体:修复正确、最小、契约齐备,可进 CI。

@ranxianglei
ranxianglei merged commit 80ba8c3 into master Sep 4, 2026
6 checks passed
@github-actions github-actions Bot mentioned this pull request Sep 25, 2026
antclt pushed a commit to antclt/billion-context that referenced this pull request Sep 25, 2026
…ess (kernel ranxianglei#398/ranxianglei#403)

Kernel 0.0.90 ships the refold-in-place stack (ranxianglei#398/ranxianglei#400/ranxianglei#402) but
markBlockRestoredInline had no host caller — the capability was inert in
bili's proxy lane: a decompress→recompress loop could never refold, and
the kernel's blocked-refold guidance promised an inline-restore path
that never materialized on this wire.

Wiring: resolveDecompress flips restoredInline (and markDirty) on a
successful FULL-BLOCK restore (default one-level view or full) — the
tool result stays in the client's re-sent history, so the model holds
the re-summarization material. Range restores (startId/endId) return
partial content and must NOT flip the flag; inactive blocks are skipped.

Tests (tests/refold-wiring.test.ts): flag flip + idempotence, full:true
parity, range-restore exclusion, end-to-end proxy-shaped refold (empty
view, in-place update, marker cleared, no second block), and a negative
control pinning the honest already-compressed rejection.

Downstream verification against the published 0.0.90: typecheck ✓,
full suite 2559 pass / 0 fail, build ✓.
ranxianglei pushed a commit that referenced this pull request Oct 14, 2026
…refs

When the proxy holds no live message view of a session (in-memory copy
dropped between model requests by resident-pool eviction or restart) but
the refs table is intact, applyRanges fed an empty message list into the
kernel: every requested ref was judged dangling, marked DEAD in deadRefs
(committed + persisted even on total failure), and receipts blamed
content-changed / substrate-destruction while the breaker armed with a
false cause. The refs were never actually changed — the next model
request restores the view and clears the tombstones anyway.

- new gate in applyRanges (+ external-summary lane): refuse with reason
  gate:no-visible-context and an honest receipt (refs intact, nothing
  recorded as dead, do-not-retry-until-next-model-request); writes no
  deadRefs; feeds the loop breaker the true cause
- the gate YIELDS when any requested range overlaps an active
  compression block (#398/#403 refold works from state alone under a
  proxy-shaped empty view) and for fresh ref-less sessions (unchanged
  kernel diagnostics)
- noteCompressLoopFailure gains forcedCause so the breaker records the
  honest cause without re-running failure-cause inference
- resident-pool evictions are now logged (session id, weight, budget) so
  the empty-view window is observable instead of silent
@ranxianglei
ranxianglei deleted the 2026-08-31_side-request-isolation branch October 15, 2026 05:32
aolshevskiy pushed a commit to aolshevskiy/billion-context that referenced this pull request Oct 16, 2026
…ianglei#3371)

screenRecommendedUnits advertised a message-ref unit whose entire span
is already owned by an active, NOT restored-inline block — the exact
shape applySingleRange's livelock guard (pi#199) is guaranteed to reject
('Range .. contains no new compressible messages'), unless the ranxianglei#398
refold verdict classifies it a genuine in-place refold. Recommend and
apply disagreed on one shape: advertise -> reject -> re-advertise, the
ranxianglei#3236 loop family reached through the consumed-unit corner.

The verdict (evaluateRefold + requestedRefoldSpan + RefoldDecision) moves
verbatim from compress.ts to refold.ts so both lanes consult the SAME
definition — the projectUnit discipline. The screen now mirrors the
apply-side guard exactly: filteredIds === 0 && consumedBlockIds > 0 &&
!isBlockBoundary survives only when evaluateRefold says 'refold'.
Block-boundary units (bN..bM promote/merge) stay exempt, same as apply.
No public kernel surface change; the golden ledger is untouched.

Also bumps kernel to 0.0.127 and regenerates package-lock.json, which
had drifted to 0.0.123 through six version bumps (ranxianglei#3331 review C-H1) —
npm ci in a clean checkout would have failed.

Kernel boundary: owner-approved via ranxianglei#3371.

Tests: kernel suite 1279/1279 incl. new issue3371 fixture (screened-out
shape + apply agreement, genuine refold stays advertised and updates b1
in place, fresh spans unaffected); bili-side compress-reject-reasons 4/4,
compress-recovery-ranges 10/10, e2e-compress-cascade 7/7; root tsc clean.

Closes ranxianglei#3371
ranxianglei pushed a commit that referenced this pull request Oct 18, 2026
The fold→inline-decompress→refold cycle (#398) updates a block in place
with its originals off the wire, so live derivation finds nothing and the
carried-from-own-previous-summary branch is the ONLY source of the
folded-images footer. None of the original five lanes exercised it —
teeth-check: with that branch disabled, this lane is the one that fails.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

标题生成等侧请求污染主会话状态:prepareAnthropic 缺 isTitleGen 守卫、快照无条件覆盖、survivedCount 每请求空转

1 participant