Repository navigation
[PR] POC: carry fold summaries as CCR placeholders (not inline prose) — weak-model confusion, from #6025 (#6111) - #6120
ranxianglei wants to merge 10 commits into
Conversation
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.9486-pr.6120.0e757f5This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr6120.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Dup check: no duplicate — searched open+closed issues and PRs for fold-summary / CCR-placeholder topics; nothing else tracks this POC. Live state (verified against GitHub just now): branch head is now unused-exports root cause (reproduced locally via Review so far: the PR body's mechanism claims check out against real code — synthetic ref band Pushing 3 labeled commits onto this PR branch (adding to the existing PR, not opening a parallel one):
Full-suite verdict = remote CI on the pushed head. Final review conclusion (8问) follows in-thread once CI reports. Holding until you rule on the A/B/C data — no merge action from me. |
(ai: vllm-qwen/qwen3.8-27b) CI re-verified live — everything is green, no new work needed this run. What I found (all verified against live sources in this run, not memory):
This run made zero code changes and zero pushes: both reds were resolved by prior runs of this lane before my verification completed, and CI on the current head is fully green. Nothing left for the CI lane to do. 结构化八问(本轮结论):
判定:live head |
e61ce19 to
992a0bc
Compare
… seam (#6111) The CCR placeholder transform now lives in ONE helper (applyFoldSummaryCcr) that owns the gate (featureEnabled('foldSummaryCcrPlaceholder') && ccrEnabled) and the store adopt, used by BOTH egress seams: - prepare-openai.ts (chat-completions shape, the original #6120 site) - prepare-responses.ts (responses shape — BsoBird's endpoint wire_api='responses' never hit the openai seam, so it saw inline prose; H2 from the plan) No behavior change to the default-off POC: same gate, same transform, same placeholder bytes. The helper is now unit-tested both directions (gate on/off x ccr on/off) per gate discipline #3617.
🔶 需要决策 State this run (verified live, not from memory): the merge conflict is already resolved — live merge state is Root cause (verified, single): commit I regenerated the goldens locally to measure the blast radius, then reverted: the diff is exactly and only that one description sentence — 4 format entries in Why this is a decision, not a mechanical fix: a tool-surface change that trips the golden gate is stop-the-line per the wire-contract rule (#1304) — the sanctioned path is regeneration backed by a stated decision, not an auto rubber-stamp. And there's a real design question underneath: should a default-off POC change the wire surface for users outside the rollout? Options:
Recommendation: B. A default-off POC should not change the wire for users outside the rollout — advertising a capability that errors for them is misleading, and it needlessly churns the stable tool-definition prefix. Gating the note is the root-cause fix (make the surface respect the gate) rather than blessing a changed surface, and it keeps the goldens stable. Default if undecided: B (conservative: preserves the default-off invariant, no wire change for non-rollout users, no golden churn). If you'd rather take the one-line path, A is a trivial approval. I have not pushed anything this run — the goldens were regenerated locally only to measure the diff, then reverted; the working tree is clean. Awaiting your call on A / B / C. |
(接上一条 🔶 需要决策 — 按决策请求体例补结构化八问,供拍板)
选项对比:
默认(不拍板时):B —— 保守、保默认关不变量、非 rollout 零 wire/缓存影响、golden 稳定。若你想走一行路,A 也是轻量批准。 本轮未 push 任何改动(golden 仅在本地重生成以量 diff,已还原,工作树干净);已打 |
进度汇报(响应卡住重启提示): 本轮工作其实已经完成,"卡住 30 分钟"是误报 —— 我在收尾后进程处于空闲状态,被看门狗重启了,不是卡在某个操作上。刚已重新 live 核验(非凭记忆):
当前无阻塞、无需我继续动手 —— 按 auto-fix lane 规则停在决策点,等你 A/B/C 拍板(不拍板默认 B:保默认关不变量、非 rollout 用户零 wire/缓存影响、golden 稳定)。收到决策我再继续。 |
Re-verified live this run (mirror cache + one direct check-runs read on the current head
No changes pushed this run. Stopping — the reported check is green and the only open item is the pending A/B/C decision. |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) PR #6120 审核进度(中途汇报,最终结论随后) 查重: 不重复(首轮已核)。本轮是 rebase 后 head
行为变更披露(我的修复 1):feature 开启时 fold head 从实际 96 → 设计值 256(旧值→新值);feature 默认关闭,对现有用户零影响。 当前状态:修复 commit(head 边界 + cap 计数 + 测试保真度 + cap 漂移回归测试 + 文档补回)正在推 PR 分支,定向测试 + CI 绿后我贴最终八问结论。不执行合并(等 owner 看 A/B/C 数据拍板)。 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Closing the loop on this run (the full confirmation is already posted above):
No changes pushed this run; working tree clean. Stopping at the decision point. |
… review) Review fixes for the #6111 fold-summary CCR placeholder POC (PR #6120): 1. Head bound: applyFoldSummaryCcr consulted effectiveCcr(session)?.maxHeadChars with a 256 fallback, but the resolved CCR config always carries a filled-in maxHeadChars (default 96 from the kernel DEFAULT_CCR_CONFIG), so the 256 fold bound never fired in production and fold heads silently inherited the 96-char tool-result bound — the fragment size the module measured and rejected. Use FOLD_SUMMARY_DEFAULT_HEAD_CHARS directly; the test now stores the production (resolved) config shape and pins head > 200. 2. Cap count: mergeFoldIndex incremented the anchor count for every processed anchor, including re-merged existing ones. A long session re-merges the whole view's anchors on every egress, so the count inflated per turn and the 16384 cap stopped the index from growing long before it was reached. The cap now counts distinct anchors only; regression test seeds the index at cap-1 with a re-merged + a new anchor (fails pre-fix, passes post-fix).
Re-verified live this run (mirror cache + one direct check-runs read on the current head
No changes pushed this run (nothing to fix mechanically — the golden mismatch is a decision, not a mechanical fix). Stopping at the decision point: the only open item is the pending A/B/C ruling on the |
#6111) as the current question in proxy mode. Replace each rendered fold summary's outbound text with a deterministic CCR placeholder whose full prose is stored under a stable synthetic ref for on-demand acp_retrieve. - New DEFAULT-OFF rollout key foldSummaryCcrPlaceholder (rollout.features; env BILI_ROLLOUT_FOLD_SUMMARY_CCR_PLACEHOLDER). No existing default changes. - New module src/fold-summary-ccr.ts: syntheticSummaryRef(blockId) mints a canonical m-form ref in reserved band [5_000_000..9_999_998] (far above counter refs); foldSummariesToCcr swaps outbound copies only — session state / message records untouched, so the next compression is unaffected. - Wired into the openai main egress path (prepare-openai.ts), gated on featureEnabled && ccrEnabled so retrieval never points at an absent tool. - Placeholder byte-stable across turns -> prefix-cache friendlier than prose. EXPERIMENTAL: default off; not for merge until owner rules on A/B/C data. Follow-up sites (responses-tool-output, adapter-openai) deferred to keep the diff on the repro path. Pure host, zero kernel changes.
The type is only used as foldSummariesToCcr's return annotation inside src/fold-summary-ccr.ts; no other file references the name. Dropping the `export` satisfies the unused-exports static gate (INTERNAL_ONLY) without a waiver — callers consume the return structurally via inference.
… seam (#6111) The CCR placeholder transform now lives in ONE helper (applyFoldSummaryCcr) that owns the gate (featureEnabled('foldSummaryCcrPlaceholder') && ccrEnabled) and the store adopt, used by BOTH egress seams: - prepare-openai.ts (chat-completions shape, the original #6120 site) - prepare-responses.ts (responses shape — BsoBird's endpoint wire_api='responses' never hit the openai seam, so it saw inline prose; H2 from the plan) No behavior change to the default-off POC: same gate, same transform, same placeholder bytes. The helper is now unit-tested both directions (gate on/off x ccr on/off) per gate discipline #3617.
… index Per the #6111 conclusion (comment 6796731303): the grep-like keyword collision works for code-dense folds but is blind to prose folds, so the two mechanisms are complementary, not either/or. - Bounded head: the fold-summary placeholder head now defaults to 256 chars (was the 96 tool-result default) so a prose fold — which has no literals for the index to catch — still carries a real topical hint. Still bounded: it is a hint, the full prose stays one acp_retrieve away. - Inverted index: extractFoldAnchors pulls the grep-able literals (backtick, CJK 「」, paths, hex, #refs) out of each fold summary and maps them to the summary's synthetic ref, persisted in session.metadata.foldSummaryIndex (bounded, idempotent across turns). - Keyword query: executeRetrieve now treats a non-mNNNNN arg as a keyword — it searches the index (exact anchor, then substring) and returns the matching block refs + heads so the model localizes the block and then retrieves it in full by ref. Gated on the rollout feature; when off a non-ref arg is a plain miss. The retrieve tool description gains a conditional note so the model knows the capability exists. Pure host, zero kernel, default-off (rollout foldSummaryCcrPlaceholder). 17 unit tests (anchor extraction, bounded head, index build/idempotence, keyword hit/miss/off, exact-ref no-regression); 77 targeted regression green.
#6111) The type is only referenced inside fold-summary-ccr.ts (readFoldSummaryIndex is the sole accessor; the keyword query in store.ts reads session.metadata directly to avoid a circular import), so the unused-exports gate flags it as INTERNAL_ONLY. Dropping the leading export keeps the gate green without changing behavior.
… review) Review fixes for the #6111 fold-summary CCR placeholder POC (PR #6120): 1. Head bound: applyFoldSummaryCcr consulted effectiveCcr(session)?.maxHeadChars with a 256 fallback, but the resolved CCR config always carries a filled-in maxHeadChars (default 96 from the kernel DEFAULT_CCR_CONFIG), so the 256 fold bound never fired in production and fold heads silently inherited the 96-char tool-result bound — the fragment size the module measured and rejected. Use FOLD_SUMMARY_DEFAULT_HEAD_CHARS directly; the test now stores the production (resolved) config shape and pins head > 200. 2. Cap count: mergeFoldIndex incremented the anchor count for every processed anchor, including re-merged existing ones. A long session re-merges the whole view's anchors on every egress, so the count inflated per turn and the 16384 cap stopped the index from growing long before it was reached. The cap now counts distinct anchors only; regression test seeds the index at cap-1 with a re-merged + a new anchor (fails pre-fix, passes post-fix).
…ed (fixes docs-sync, #6111 POC) The previous docs commit hand-edited the generated config tables, which the docs-sync lane (gen-config-docs generate + drift check) rejects. Document the new rollout key through the real source of truth: add the rollout.features.foldSummaryCcrPlaceholder entry to website/config-reference/rollout.yaml and regenerate the en + zh tables. check-coverage stays green (the env name is dynamic, same as the sibling rollout keys).
fd0d323 to
150eb97
Compare
(ai: vllm-qwen/qwen3.8-27b) Conflict resolved — PR #6120 rebased onto current master and force-pushed. 七问 summary of this run:
|
…ookup description (#6111) The POC's RETRIEVE_TOOL_DESCRIPTION change (fold-block keyword lookup) drifted the pinned bili.json + manifest-optin.json snapshots. Regenerated via the official script; diff is exactly the new description sentence on each wire shape, nothing else.
(ai: vllm-qwen/qwen3.8-27b) Red CI fixed — the 3 failing test shards were one root cause: the PR's own
|
Summary
POC for #6111 (root-cause tracking stays in #6025): in proxy mode, fold summaries are re-voiced inline as
usermessages (systemToUser(isRenderedSummaryMessage)), and weak models misread a wall of that summary prose as the current question (答非所问). This branch tests the hypothesis by replacing each rendered summary's outbound text with a deterministic CCR placeholder, storing the full prose under a stable synthetic ref so the model can pull it on demand viaacp_retrieve.EXPERIMENTAL — default OFF. Do not merge until the owner rules on the A/B/C experiment data.
What changed (pure host, zero kernel)
src/rollout.ts: new default-off keyfoldSummaryCcrPlaceholderunderrollout.features.src/fold-summary-ccr.ts(new):syntheticSummaryRef(blockId)mints a canonicalmNNNNNref from a reserved band[5_000_000..9_999_998];foldSummariesToCcr(...)swaps only outbound copies.src/server/prepare-openai.ts: gated transform inserted on the openai egress path between image-rescale and thesystemToUserre-voice. Gate =featureEnabled("foldSummaryCcrPlaceholder") && ccrEnabled(session).tests/fold-summary-ccr.test.ts(new): 6 targeted tests.Why a synthetic ref (the crux)
Empirically verified with a scratch probe: rendered fold summaries get no kernel ref in proxy mode — the pipeline runs
assignRefsNodebeforepruneNode, so a summary created byrenderSummarynever enters ref assignment, and the proxy client never round-trips it. The placeholder therefore mints its own stable ref. Retrieval stays safe:executeRetrieve→retrieveByRefresolves purely offstore.byRef(independent ofmessageRefs), first-write-wins, andrestampContentStoreleavesacp_summary_*raw ids frozen (unmapped).blockId → summaryis immutable (a re-fold makes a successor block), so first-write-wins is correct.Behavior change
None. Default off; zero existing defaults touched. Only when both the new rollout key AND
ccr.enabledare on does any byte change — and then only the outbound view ofisRenderedSummaryMessagerows (session state / message records untouched, so the next compression is unaffected). The placeholder is byte-stable across turns → prefix-cache friendlier than inline prose.Config surface
Adds ONE default-off behavior key under the existing
rollout.featuresmechanism (envBILI_ROLLOUT_FOLD_SUMMARY_CCR_PLACEHOLDER). Not a new top-level section. The existing surface cannot express this: CCR placeholders today only cover tool results / compressed blocks keyed by real refs; a per-session, per-block synthetic-ref placeholder for render-time summaries is a distinct opt-in behavior. Migration: n/a (off by default, additive).Experiment design (A/B/C, weak-model live test — run after CI green)
Metrics: misread (答非所问) rate ·
acp_retrievecall count × retrieved size · cache-hit delta · whether weak models actually know to callacp_retrieve.Follow-up (deferred, kept out of this diff)
Same-shape seams not yet wired — this POC targets the openai repro path where the experiment runs:
src/responses-tool-output.ts:108,125andsrc/loop/adapter-openai.ts:247. Plugin mode never rendersacp_summary(KDD#7), so no plugin-mode seam is needed.Verification
Local: typecheck clean, build OK,
tests/fold-summary-ccr.test.ts6/6, plus targeted regression suites green (rollout/store/storage-ccr/ccr-v2 73/73; openai-seam/ccr-restamp 9/9). Full-suite verdict = remote CI on this head.Closes #6111