Skip to content

[PR] fix: hash-free lite fold anchors for past-cap tool cores (#3752) - #3753

Merged
ranxianglei merged 5 commits into
masterfrom
2026-10-19_lite-fold-anchors-past-cap
Oct 20, 2026
Merged

ranxianglei merged 5 commits into
masterfrom
2026-10-19_lite-fold-anchors-past-cap

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #3752

What & why

The MAX_ANCHORS = 16384 seed-loop cap (#2334) dropped every covered id past the window entirely. Ids past the cap therefore had exactly ONE recovery channel left — the positional scan — and that scan dies after MAX_RESYNC_GAPS (8) accumulated fingerprint breaks. On a long session with weeks of genuine drift the whole tail region is positional-dead, so the 0.1.200→0.1.201 canonical-identity re-key (#3658) mass-unfolded thousands of folded messages per session in BOTH directions (the field incident: one Anthropic session 125K→998K input tokens in one turn, cache hit 2%; downgrade re-churned identically). Repro harness confirmed the mechanism before the fix: with 17.4K covered ids + scattered genuine edits, the capped ledger lost 660 of 8.5K missing ids while an uncapped control lost only the genuinely-edited cells (honest drift).

Past-cap TOOL cores now persist a hash-free lite anchor ({r, c, t, b} — role, content class, protocol tool id, text length). Pass 1's toolCallId channel consumes exactly those fields, so they re-anchor at any session length with zero added hashing and no new misattribution surface (claim discipline unchanged: unique unclaimed candidate per (toolCallId, role)).

Scope decisions:

Behavior changes (disclosed per owner policy 2026-09-22)

  1. Additive persistence: foldAnchors records gain lite entries {r,c,t,b} for covered tool-core ids beyond index 16384 in block order (~60–80 B each; a 17K-core session adds ~1MB worst case vs today's zero). Old builds reading them degrade gracefully: their Pass 1 already consumes exactly {t,r}, norm-keyed passes skip the absent n.
  2. Gap-tier bracket eligibility extended: lite-anchored past-cap tool cores can now be claimed via the order-bracket tier (previously no anchor → skipped). Same sanctioned tier and evidence class as today's anchored ids (verified claims fencing both flanks + role + length + tool-consistency guard); strictly more recovery, doctrine unchanged.
  3. Known residual on the downgrade path: an OLD build reading NEW state classifies lite anchors (no sh/wh) as textless in its shape-census predicate, so a fully-fenced count-exact run of ≤2 consecutive tool cells could be k-th paired by the old build's textless tier (that tier has no role check — pre-existing looseness). Rare, bounded, and confined to the downgrade scenario where the old build is already re-churning the rest of the session. Not fixed here (would mean editing released code paths); noted for the record.
  4. Under-cap behavior is byte-identical (cap logic untouched; proven by the targeted suite below).

Tests

  • New tests/issue3752-lite-anchors.test.ts (3 tests): mint shape (full == cap exactly, lite field-shape exact, non-tool past-cap stays anchor-less), channel proof (asserts on the reconciler's own by toolCallId counter, not just outcomes, plus no-cross-pairing), and the incident-trajectory convergence replay (true pre-fix ledger → re-keyed drifted resend → refold → frozen steady-state ledger). All three RED without the fix, GREEN with it (3.7 s).
  • Targeted local: all 16 existing fold-reconcile-related suites — 224/224 green. Full-suite verdict comes from CI on this head.

Config surface

None — no new env vars, config keys, or defaults.

Kernel change

None — kernel/ untouched (host-side reconcile only).

@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-19_lite-fold-anchors-past-cap (31e19c2)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4589-pr.3753.bdbce66

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3753.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix: hash-free lite fold anchors for past-cap tool cores (#3752) [PR] fix: hash-free lite fold anchors for past-cap tool cores (#3752) Oct 18, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复 —— 按失败签名检索过 open+closed 的 issue 与 PR(past-cap / lite anchor / MAX_ANCHORS / re-key mass-unfold / downgrade churn):无同类在途条目。本 PR 是 #3752 止血方案的直接实现(Option A lite anchors)。唯一同文件在途项是 #3257(verified-incremental prefix fast path,#3248 phase 1)——不同机制、不同目的,非重复,但两者都改 src/fold-reconcile.ts,合并顺序上需留意热文件语义冲突(review 里会核)。

标签: bug + opencode + scope/kernel + p1(缺陷类;宿主 = OpenCode 2.0.28;失败域 = 上下文管理/reconcile 核心逻辑;严重度 = 缓存命中崩坏→成本爆炸+周级折叠状态丢失,正中 p1)。

说明:本 PR 是我上一会话(ses_7fa25c47d66eFc7UirP74g5dDz)在 #3752 分析后开的止血 PR(Option A)。现在被唤醒做独立复核——不复读 issue 侧结论,重新逐段读 diff、审回归与测试有效性。CI 静态门禁全绿(kernel-guard / config-surface-guard / version-guard / docs-sync / unused-exports 等);9 个 test 分片(ubuntu/windows × node 22/24 × 3 shard)仍在跑。完整八问复核结论待 test 分片落地后给出。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

独立复核完成(不复读 issue 结论,重读了 diff、逐点核了回归面与测试有效性)。先给判定:代码与测试质量达标、机制正确、回归面已逐项核过 → 可合;唯一前置是主 CI 的均衡 test 分片在当前 head 上还没跑出一次「跑完到绿」(详见第 5 问后),合并前请确认一次干净 re-run。

结构化八问

1. 是什么问题:超长会话在 bili 侧 identity re-key(0.1.200→0.1.201 = #3658 canonical-identity 换键)后发生大规模 unfold——超过 MAX_ANCHORS=16384 窗口的已折叠 id 丢了除「位置扫描」外的所有重锚通道,而位置扫描累计 MAX_RESYNC_GAPS=8 次断点后死亡,导致窗口外的 tool core 无法恢复、prefix-cache 命中崩到 ~2%。

2. 什么原因导致的:根因分两层——R1:#2334 的性能窗口把窗口外 id 整个丢弃(旧代码 if (anchorCount>=MAX_ANCHORS) break),只留会死的位置扫描;R2:identity epoch 切换无迁移层。本 PR 针对 R1 的 tool-core 主类止血。

3. 影响范围与严重程度:触发条件 = 会话覆盖 id 数 > 16384 AND 发生 bili 侧 identity re-key(或等量漂移)。严重度 p1:缓存命中崩坏→成本爆炸 + 周级折叠状态丢失(事故实测单会话 125906→997959 input tokens)。仅命中「超长会话 + re-key」组合,非全量用户。

4. 怎么修复的:Option A —— 为窗口外 tool core 铸 hash-free lite anchor {r,c,t,b}(liteToolAnchorFrom,src/fold-reconcile.ts:689),让 Pass 1 的 toolCallId 精确通道(src/fold-reconcile.ts:907-917)在任何会话长度都能重锚它们;非 tool core 保持原状(positional-only,其通道依赖文本 hash,正是窗口要限的成本);full-anchor 预算不变(anchorCount 只对 full anchor 自增,#2334 契约保住)。附带一个正确的自愈:当容量回升时,lite anchor 经既有 #2396 backfill 分支(src/fold-reconcile.ts:1615)自动升级回 full。

5. 是否有回归问题:我把 FoldAnchor 每个字段消费点全列出来核了一遍(n/m/sh/wh/p/x/t/r/c):lite anchor 缺字段处全部有 undefined 守卫,不会崩溃、也不会经任何内容通道(byNorm/twin/turn-key/image-norm/simhash)误配——这些通道对 lite 一律 skip。行为变更(已披露):窗口外 tool core 从「整段丢弃」变「lite 可经 toolCallId 精确配」——这是本 PR 目的,old→new 明确。次要发现(非阻塞):#3641 shape-census 的 textless 路径(src/fold-reconcile.ts:1295-1364)没有 role 检查,而 lite anchor(no sh/wh)在新 build 和旧 build 都会落进这个桶——即 PR 披露的「downgrade-only 残留」实际范围略大(新 build 也受同一纪律约束)。但它被 mass-event(≥4)+ 精确计数相等 + 双侧稳定 flank 三重门限卡死,属既有 #3641 文本less 纪律,低危。建议在本 issue 记一条 checklist,不单独拆片。

6. 是否破坏缓存:不破坏,方向相反——修复本身恢复缓存命中(减少 unfold→减少 prefix 失效)。full-anchor 预算、steady-state 零工作契约(#2334/#1930-2)保持不变;持久化是 session.metadata.foldAnchors 附加式条目,不动既有 fold 游标/去重键,无 schema/version bump。

7. 是否改了设计方向:否。仍是 order-centric reconciliation 框架内补一条 hash-free 通道,未改架构/边界/数据流。R2 的完整迁移层(Option C,identityEpoch + sticky alias table)仍待 owner 决策,本 PR 明确不越界、诚实标注了边界(首个 re-key pass 的死区 core 仍会 unfold 一次,下次 fold 收敛,之后对该长度的 tool core drift-proof)。

8. 是否引入了新的配置:否。无新 env/config/默认值。

测试有效性(④)

三个测试都有牙、非恒真:

  • T1 钉住 mint 形状(full==CAP 恰好、lite 字段精确、非-tool 窗口外仍无 anchor——含负向断言);
  • T2 用 reconciler 自己的 byTool 计数证明走的是 toolCallId 通道(我核过该字符串确实来自 repair-mode drift log 行 src/fold-reconcile.ts:1964 ${plan.byTool} by toolCallId,不是幽灵串),并断言无跨配、host 编辑不泄漏进 tool 通道;
  • T3 事故轨迹收敛(真实 pre-fix ledger → refold → 冻结稳态,且不断言 foldDriftEscalated latch)。
    确定性 OK(逻辑路径无端口/随机)。作者声称 3 RED→GREEN + 相邻 224/224。诚实声明:我通过读码核对了机制与断言有效性,但本地无 node_modules/build 产物、且 CI 分片未跑完,我没有独立执行到 RED-without-fix / GREEN-with-fix——执行级证明目前依赖作者声明 + 待 CI。

diff 整洁度 / 代码整洁(⑤/⑤b)

3 文件 +326/-15,全部与本 PR 目的相关(CHANGELOG 条目含一处标注的 #3642 措辞勘误——合规的顺手小修)、fix、回归测试。命名达意、注释讲 why(带 incident 溯源)、无死代码/魔法数(复用 CAP 常量)、风格一致、无新依赖。达标。

CI 现状(诚实)

head a2badbcae(2 commit,mergeable=True,merge_state=None 未定)上:

越界发现(不在本 PR 范围,已点名不留空)

收工判定

可合,前提是先拿到主 CI 均衡 test 矩阵的一次干净绿色(触发 re-run 或等其自然落定)。PR 链接:#3753。

awork added 2 commits October 19, 2026 11:18
The MAX_ANCHORS=16384 seed-loop cap (#2334) dropped every covered id past
the window entirely, leaving the positional scan as their only recovery
channel — and that scan dies after MAX_RESYNC_GAPS accumulated breaks. On
long sessions with weeks of genuine drift, the 0.1.200->0.1.201 identity
re-key (#3658) therefore mass-unfolded thousands of folded messages per
session in both directions (upgrade and downgrade).

Past-cap tool cores now persist a hash-free lite anchor ({r,c,t,b}): Pass
1's toolCallId channel consumes exactly those fields, so they re-anchor at
any session length with zero added hashing and no new misattribution
surface (claim discipline unchanged). Non-tool past-cap cores keep the
positional-only status quo; the full-anchor cap and the #2334 zero-work
steady-state contract are untouched (lite anchors reuse verbatim).

FoldAnchor.n becomes optional (norm-keyed passes skip absent n); the
seed loop continues past the cap for tool cores instead of breaking.

Regression tests pin mint shape, the channel (via the reconciler's own
counter), and the pre-fix-ledger convergence replay; all red without the
fix, green with it.
@ranxianglei

Copy link
Copy Markdown
Owner Author

红了

awork added 2 commits October 20, 2026 03:54
Merging master into this branch pulled in #3811, which added
maskUrlForPublish to src/log-mask.ts. It is called only by
maskUrlsForPublish in the same file; #3811's external consumer
(src/web/diagnostics.ts) imports the wrapper, never this helper.
The unused-exports gate therefore classifies it INTERNAL_ONLY.
Drop the leading export per the gate's prescription. Behavior-neutral:
the internal caller is unaffected and no dist export surface changes.
The master merge left two duplicated config-table rows in
CONFIGURATION.md / CONFIGURATION.zh-CN.md: a stale 'storage.unified
defaults to shadow' row alongside the current '#3818 flip -> full' row,
and an older 'storage.sqliteAutoDownload' row lacking the #3788
retry-cooldown sentence alongside the current one. Re-running the
single-source generator (tools/gen-config-docs.mjs) collapses each key
to its current seed row. Retained rows carry the newest semantics; no
knob is lost. Satisfies the docs-sync drift guard.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

合入 master 后有两个静态门禁红了。都是合并产物,不是 lite-anchor 逻辑本身的问题——两个我已经在分支上修好、对应门禁本地也过了,但这个环境当前连不上 github.com,推不上去,先把状态说清。

两个红灯 + 修法:

  1. unused-exports —— master 的 [security][web] Diagnostics blob prints private relay hosts verbatim in the config section — the "secrets masked" header only masks values, not hosts | 诊断包 config 段原样输出内网中转地址:标题说"密钥已遮蔽",但只遮值不遮主机 #3811 给 src/log-mask.ts 加了 maskUrlForPublish,它只被同文件的 maskUrlsForPublish 调用([security][web] Diagnostics blob prints private relay hosts verbatim in the config section — the "secrets masked" header only masks values, not hosts | 诊断包 config 段原样输出内网中转地址:标题说"密钥已遮蔽",但只遮值不遮主机 #3811 的外部消费方 src/web/diagnostics.ts import 的是外层 wrapper,从不直接引用这个 helper)。门禁判它 INTERNAL_ONLY 过度导出。修法:按门禁自己的处方去掉那行 export(不加 waiver)。行为零变化:内部调用不受影响,dist 导出面不变。本地 check-unused-exports 已转绿。

  2. docs-sync —— 这次 master 合并把 CONFIGURATION.md / CONFIGURATION.zh-CN.md 各留了两行重复配置表项:一条旧 storage.unified(默认 shadow,Storage wrap-up: 6 fixes + switch default mode to full (single PR) | 存储收尾六修 + 缺省模式翻 full(一个 PR 统一处理) #3818 翻转前版本)+ 现版本(默认 full)并存;一条旧 storage.sqliteAutoDownload(缺 SQLite unified storage degraded to legacy-only (bili v0.1.201, node v20.19.5, win32-x64) #3788 重试冷却句)+ 现版本并存。重跑单源生成器 tools/gen-config-docs.mjs 后每个 key 收敛回当前行,重复行被删。没丢任何配置项——留下的是带最新语义那条;生成幂等(再跑一次无 diff)。

八问:

  1. 什么问题:合并 master 后 unused-exports / docs-sync 两个静态门禁红。
  2. 什么原因:都是合并引入的产物——[security][web] Diagnostics blob prints private relay hosts verbatim in the config section — the "secrets masked" header only masks values, not hosts | 诊断包 config 段原样输出内网中转地址:标题说"密钥已遮蔽",但只遮值不遮主机 #3811 的内部 helper 被过度导出;合并把两侧相邻编辑拼出了重复表行。与 fold-anchor 改动无关。
  3. 影响范围/严重度:仅 CI 门禁层,不影响任何运行时行为;不修则 PR 无法合并。
  4. 怎么修的:去一个 export(仅可见性收窄);重跑文档生成器收敛重复行。两个独立标注 commit。
  5. 回归风险:无。de-export 不动任何调用路径(dist surface 不变);文档只删重复旧行、保留最新语义行。
  6. 是否破坏缓存:不涉及。
  7. 是否改设计方向:不涉及。
  8. 是否引入新配置:不涉及。

行为变更声明:无 —— 两处都是可见性/产物级,无阈值/默认值/时序/输出格式变化。

推送状态:受阻 —— 这个环境此刻连不上 git push 用的主机(github.com:直连超时、走代理被重置;api.github.com 反而是通的)。两个 commit 已在你的合并提交之上做好(干净 fast-forward),egress 一恢复我就推上去并确认这两个门禁在 CI 上转绿。不用你做什么,先同步。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

egress 恢复了,已推上去。PR #3753 新 head:31e19c298(在你那个合并提交之上,干净 fast-forward)。

CI 上两个红灯都转绿了:

  • unused-exports ✅ success
  • docs-sync ✅ success

其余静态门禁(kernel-guard / config-surface-guard / version-guard / docs-coverage / lane-guard / lock-sync / build-artifact …)也全绿。剩下的均衡 test 分片(ubuntu/windows × node 22/24)和真客户端 e2e 车道还在跑——那是全量回归的最终判词,跑完即定论。

两个 commit(fix: 去过度导出 / docs: 收敛合并出的重复表行)都在分支上、独立标注,diff 干净、无行为变更。全绿后就可以合了(合并按惯例你来)。

@ranxianglei
ranxianglei merged commit fa47f78 into master Oct 20, 2026
49 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.631-master.fa47f78 (rolling master tag).

npm install -g billion-context@0.0.631-master.fa47f78  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ci:full Run the FULL CI matrix (all e2e lanes + 12-leg test matrix) on this PR opencode p1 scope/kernel 发版

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0.1.200 → 0.1.201 upgrade (#3658 identity change) mass-unfolds persisted sessions incl. Anthropic lane; downgrade re-churns

1 participant