Repository navigation
feat: codex compact family — combined #322/#323/#324/#325/#317/#319 + UA unification - #336
Conversation
Codex (>=0.147) reuses the root session-id header on subagent threads, so the legacy identity chain mapped every thread of one task onto a single session — subagents inherited the root's compression state, breaking #150's isolation (a guardian subagent must read the user's original authorization verbatim, never a compressed summary). Add codexTurnIdentity(): when x-codex-turn-metadata is present and cross-checked (metadata.thread_id === thread-id header), partition by thread_source — "user" keeps the session-id header (current root semantics, stable across turns), "subagent" gets its own thread-id (fresh independent state per thread). Untrusted metadata (absent / unparseable / mismatched / unknown thread_source) falls through to the legacy chain unchanged, so no-metadata clients (omp pck, etc.) see zero regression. The trusted id enters the verbatim session chain directly (no hashing, no new dimensions) and bypasses subagentNamespace, leaving the kernel's empty-instructions non-anchoring path untouched for metadata-less clients. Tests: unit coverage for both thread_source values, cross-check mismatch, JSON parse failure, unknown thread_source, missing headers, omp pck chain unaffected; plus a server-level test proving subagent and root threads land in distinct isolated sessions.
…ty (#316 PR-B) A client that drops its oldest messages (truncation) previously failed the full-depth prefix match and was orphaned into a fresh pfa- session, stranding the old session's compression blocks. resolve() now falls back to a tail-window comparison: the incoming's leading items are matched against each stored chain's trailing items (position-independent per-item hashes), so a truncated replay reattaches its original session and the kernel reconciles by message id (verified: blocks deactivate gracefully on truncation and revive on full replay, no throw). New pfa- sessions also record a best-effort lineage (discarded candidates + reason truncated/forked) in session metadata for UI/debug only — never used for matching. Live-edge overwrite semantics unchanged. - src/prefix-affinity.ts: TAIL_WINDOW/MIN_TAIL_MATCH/MAX_STORED_ITEMS/ MIN_FORK_PREFIX; perItemHashes + lcpLength; resolve() gains tail-window reattach + lineage; note() stores per-item hashes; AnonymousAffinity gains via + itemHashes + lineage. - src/server.ts: tail-window reattach log line; note() 4th arg; metadata records via + lineage. - tests/prefix-affinity.test.ts: note() call sites updated to 4 args. - tests/tail-window-reconcile.test.ts (new): tail-window reattach, prefix/new regressions, fork + truncated lineage, and a kernel reconcile anchor test. Independent of PR-A (codex turn-metadata, #317). Pre-flight: typecheck OK, 746/746 tests, build OK.
guardian_review / guardian_classifier (Feature string) / memory_consolidation threads previously fell back to the legacy chain and landed on the root session — the exact #150 isolation violation. Invert the discrimination: only "user" joins the root session; every other thread_source with a valid thread_id cross-check partitions by thread-id.
…tail (PR review) Tail-pinned matching only reattached when the client retained <= TAIL_WINDOW (8) items; a rolling-window client keeping 20-30 items orphaned on every slide. Search the incoming's leading window across the stored chain's per-item hashes at any offset (<= 57 offsets x 8 compares, trivial). MAX_STORED_ITEMS 64 -> 128 so deeper windows stay searchable.
… claude env) (#321 PR-D)
…R-C) The rebase marker fired on ANY 2xx upstream response, so a failed, incomplete, or truncated compaction stream scheduled a rebase on top of a compaction that never happened (the #249 false-rebase bug): bili discarded valid blocks and mis-tracked the ledger. - new src/stream-terminal.ts: terminal-state observer for Responses responses. SSE: response.completed -> completed, response.failed / response.incomplete -> failed, no terminal event before EOF -> unknown. JSON: output array -> completed, error field -> failed, unparseable / empty / oversized -> unknown. unknown is the safe default (no rebase; next request self-heals via kernel message-id deactivation). - server.ts: both markNativeCompactionBoundary call sites now gate on observed terminal === completed. 2xx empty body no longer rebases. - trigger-path hardening in prepareResponses: a trailing compaction_trigger request is detected BEFORE injection — no compress system prompt, no compress tools, renderTags none, compressInjected false (response flows through plain passthrough instead of the rewriter), and resetAfterSuccess is set so its terminal state gates the rebase. Nudge skip now keys off the same pre-injection detection. - tests: 16 unit tests for the observer + e2e through the proxy covering completed / failed / truncated / 2xx-error-JSON / empty-body / JSON-output and the clean-forward assertions; finality-test control moved to a separate session (a completed trigger now rebases the compacting session, legitimately disarming its nudge).
…321 PR-E2) When BILI_CODEX_COMPACT=intercept, the client is codex (UA codex_cli_rs/), and the ACP state is healthy (transform ok + steady-state < 90% of window + an active block to hand off), bili forges a success response for codex's native compaction requests and never contacts upstream — a deterministic handoff to the ACP state: - trigger form (input ends with compaction_trigger): forged 2-frame SSE (response.output_item.done with one {type:compaction, id:fc_bili_*, encrypted_content:bili:acp:<summary>} + response.completed with honest usage). Matches codex's collect_compaction_output contract (exactly one compaction item + response.completed; all other events ignored). - endpoint form (POST /responses/compact): forged JSON {output:[...]} where output is the rebuilt history (recent items + ACP summary), matching codex's CompactHistoryResponse parser. Otherwise (kill-switch off by default, ACP not keeping up >= 90%, or nothing compressed yet) the request passes through to upstream and native compaction backstops (rebased by the PR-C semantic gate). Bili-generated compaction items (fc_bili_ prefix / bili:acp: sentinel) are stripped from subsequent requests so codex's echoed item does not double-count. Pre-flight: typecheck clean; 752/752 tests (7 unit + 5 e2e new); build ok; forge code verified inlined in dist.
The /bili/ prefix section had no Claude Code entry. Add it (claude's undici fetch ignores HTTPS_PROXY, so the /bili/ URL form via ANTHROPIC_BASE_URL is the only manual option) plus a note that manual mode must set CLAUDE_CODE_AUTO_COMPACT_WINDOW itself — the bili claude launcher does this automatically (PR-D) but manual /bili/ users do not get the injection. claude clamps the value down to its perceived model window, so over-setting is safe. EN + zh-CN.
…/0.147.0, so the intercept never fired for real exec-mode clients (verified by real-device e2e: isCodex=true, gate=true, forged SSE accepted)
…t' into 2026-08-29_codex-compact-family
… into 2026-08-29_codex-compact-family
…' into 2026-08-29_codex-compact-family
…pt' into 2026-08-29_codex-compact-family # Conflicts: # src/server.ts
…into 2026-08-29_codex-compact-family
… into 2026-08-29_codex-compact-family
… by codex-models.ts)
📦 Built Package ArtifactBranch: Option A — Install from npm PR tag (recommended)npm install -g billion-context@pr-336Each push to this PR publishes a new version under the Option B — Download artifact
tar xzf billion-context-pr336.tgz
npm install -g packageThis comment is automatically updated on each push. |
[bot] Picked up. Reviewing PR #336 now: fetching the combined tree, re-running typecheck/test/build, and auditing each of the six included PRs plus the UA-unification commit. Will report findings here. |
|
Comprehensive cross-PR review (delegated reviewer, on this exact tree) Verified clean: conflict resolution (both 1. MAJOR — trigger-form intercept loses the pre-compaction summary. Forge puts block summaries into the opaque Minors: 2) Nits: 8) forged SSE ignores Top action: resolve #1 (+ e2e assertion recalling a value that was compressed pre-forge) before enabling |
[bot] Independent verification of the delegated review complete — all on this exact tree ( Verdict per finding: 1. MAJOR — trigger-form summary loss: CONFIRMED, reproduced empirically.
2. Minor (gate-fail forwards unstripped body): CONFIRMED. 3. Minor (endpoint Prepared lacks stream/protocol): FALSE POSITIVE on this tree. 4. Minor (tail-window cross-merge): CONFIRMED, reproduced. Two anonymous conversations sharing a 3-item templated head: the new conversation reattaches to the old one's session ( 5. Minor (clamp overrides operator limit): CONFIRMED. 6. Minor (turn-identity "user" path precedence): CONFIRMED (structural, narrow). 7. Minor (pending register steal): CONFIRMED (structural, narrow). Nits: 8 ✅ (forge always SSE, ignores Action — implementing now on this PR:
#5/#6/#7 are design trade-offs of the family rather than defects in the combined tree — leaving them for your triage (happy to take any of them as follow-up issues). Will report back with the diff + full re-verification. |
The trigger-form forge put block summaries into the opaque compaction item, but the kernel deactivates blocks whose messages vanish from the truncated replay (syncBlocks keeps a block only while one of its effectiveMessageIds or its acp_summary_ message is present) — so everything that lived only in compressed blocks disappeared from the model's view on the first post-forge turn. Fix: capture the active blocks' summaries in the kernel's render format at forge time and at echo-strip time (upgrade path for sessions forged by an older build) into session.metadata.codexForgedSummaries (persisted, exact-text dedup), and re-inject them into the developer message on every /responses turn. Tests: unit coverage for renderForgedSummary/mergeForgedSummaries; mock-upstream integration test asserting the post-forge turn strips the echoed fc_bili_ item and re-injects the summary as a developer message; e2e forge phase now also recalls a sentinel that was compressed pre-forge (archive #6, not in codex's retained tail).
…g path; tail-window offset-0 guard prepareResponsesCompact mutated state and then returned the original (unstripped) body on gate-fail/catch — the echoed fc_bili_ item reached upstream. Strip before the early return so pass-mode, gate-fail, and catch paths all forward a clean body. Tail-window reattach now excludes offset 0: a head-to-head match is either a full-prefix case (step 1) or a distinct conversation sharing a templated head — never a truncation reattach (the retained suffix of a truncation starts strictly inside the stored chain). Closes the cross-merge of two anonymous conversations with identical templated heads.
[bot] Top action resolved — pushed 3 commits to the PR head (
Verification on the pushed tree: Left for your triage (confirmed earlier, not fixed here): #5 clamp overriding an operator-tuned |
ework's pass (656f90d..ccd7036) and my pass (d4fe1ba) fixed the same findings concurrently. This reconciles them, best mechanism per finding: - MAJOR (summary loss after trigger-form forge): hybrid. The echoed fc_bili_ item is REPLACED by a summary-carrying user message (history-borne: kernel-compressible, kept by codex's user-message retention) — and ework's forge-time capture into session.metadata.codexForgedSummaries stays as the fallback: when the echo does NOT come back, the captured summaries are re-injected into the developer message. Exactly one path is active per turn (no duplication), and neither an echoed-item loss nor unbounded permanent injection can drop or bloat the handoff. - tail-window cross-merge: keep ework's offset-0 guard (a head-to-head match is never a truncation reattach) over the byte-size heuristic. - endpoint compact: strip-on-all-paths (ework) + all-or-nothing state commit with restore on gate-fail/rebuild-failure/error (mine). - codexTurnIdentity: validated 'user' turns fall through to the legacy precedence chain (stronger headers may own the id). - pending plugin register can no longer be claimed by a subagent thread. - codex window clamp exempts an operator's explicit compress.modelContextLimit (operator tuning is never floored/clamped). - trigger-form forge honors parsed.stream (JSON when not streaming) and mints usage from max(lastInputTokens, payload estimate, 1). - isCompactionTrigger reuses hasCompactionTrigger. 823/823 unit tests; full e2e incl. forge + pre-forge deep recall pass against local sglang.
|
Review findings 全部修复完毕(两次并行修复已协调为一次混合方案) 发现 ework 与我同时对审查发现做了修复并推到同一分支。
验证:typecheck ✓ · 单测 823/823 ✓ · build ✓ · e2e 全套含 forge + 压缩前深度回忆(档案#6 post-forge 可召回)pass 1 fail 0(本地 sglang,~7min)。 新 e2e 单测覆盖:echo 在场→handoff user 消息且 dev 注入被抑制;echo 缺席→dev 兜底注入生效。 |
[bot] 收到 — 独立复核了 独立验证(本环境,pr-336 @ 1c6a72c): 代码层面确认的几个协调点(不止跑测试,逐条看了机制):
两个已知残留边界(都不值得现在修,记录在案):
关于撞车:你贴出 top action 后我直接开工推了修复,没有先确认你是否已动手 — 下次我先在 issue 里认领再动分支。混合方案比任何单方案都好(history-borne 可被 kernel 再压缩,dev 兜底覆盖回显缺席),#249 家族可以合了。合并后记得关 #317/#319/#322/#323/#324/#325。 |
codex 0.147 ships MCP tools to the model as a namespace tool type; self-hosted servers (sglang/vllm/ollama/llama.cpp) do not parse it, so the injected bili MCP tools become silently invisible and the model fumbles for them. Users had no way to discover BILI_LAUNCHER_PLUGIN=0. launcherInjectMcp now falls back to wire-injected flat tools when the codex upstream host is loopback/RFC1918/ULA/link-local/.local and BILI_LAUNCHER_PLUGIN is unset; BILI_LAUNCHER_PLUGIN=1 forces plugin mode. claude is unaffected (it flattens MCP tools itself).
Model: glm-5.3 (via pi)
What
One combined PR of the six downstream PRs of #249 (via the #321 layered plan), so they land, get reviewed, and get tested as the unit they were designed to be:
resolveCodexBudgetArgs)response.completed/response.failed), not HTTP statussrc/codex-models.ts: codex perceived-window table (bundled snapshot + 272k fallback) +codexAlignedWindowmin() clampsrc/codex-compact.ts: intercept codex native compaction, forge 2-event SSE (fc_bili_ACP summary), strip echo next request; kill-switch `BILI_CODEX_COMPACT=interceptx-codex-turn-metadatapartition:thread_source=="user"→ session-id, any other value → thread-id (guardian family included)codex-models.tshad its ownisCodexClientmatching onlycodex_cli_rs/— missedcodex_exec/(the exec-mode UA, the same dead-code class fixed on #325 in c507e0c). Now imports the singleCODEX_UA_PREFIXESimplementation fromcodex-compact.ts(0074964)The one merge conflict (
src/server.tsPrepared object, #324×#325) was resolved keeping both fields —resetAfterSuccessandcodexForge.Why one PR
Verification (all on this exact tree)
npm run typecheck✅npm test816/816 ✅npm run build✅qwen3.8-27b, ~8 min):pass 1 fail 0. Evidence from the run log:preflight compressed 3 range(s) … (34384 → 32013)codex compact intercepted (trigger); forged SSE with 2 block summary(s), upstream not contactedstripped 1 bili compaction item(s) echoed by codex(echo stripping verified on subsequent turns)Supersedes
#317 #319 #322 #323 #324 #325 (close after this merges; #249 closes as superseded by the family, #326 stays closed-recommendation).