Repository navigation
[PR] #4316 Layer 3 (number directly mapped to canonical seq) PR 1/3: Fold-overwrite into real seq range | #4316 层3(编号直连正史 seq)PR 1/3:折叠覆盖写入真实 seq 区间 - #4335
Conversation
…ipt coverage Pins the storage-side contract for "编号直连正史 seq": a block whose covered messages exist as append-only rows carries their real min/max seq; unresolvable coverage (imported pre-fold) keeps the 0 placeholder. Red against the current hard-wired 0 at the single write site (src/storage/ingest.ts).
…ranscript Resolve each block's covered ids to canonical [start_seq, end_seq] bounds at ingest time (store.ts sqlite + mysql.ts), after final absolute seqs are known (delta placeholders renumber in-transaction, so the pure planner cannot compute them). Unresolvable coverage keeps the 0 placeholder — honest gap, never a fabricated bound. Zero compression-behavior change: nothing reads these columns yet; only export surfaces them (diagnostic 0 -> real).
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.5141-pr.4335.7307a36This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr4335.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复。 这是父 issue #4316「切模型/客户端重写历史后消息身份全换血」三层定案的层3 PR 1/3;近期无其他 PR 在填 标签: 我把 diff、双后端上下文、planner、列的读者、以及 PR head 与 master 两边的 CI 都过了一遍。结论:代码正确、范围克制、披露完整、方向对,有一处性能点建议合入前顺手改掉。逐问如下。 1. 是什么问题 — #4316 层3 要"折叠记账从覆盖的消息 h_id 集合迁到正史 2. 什么原因导致的 — "恒 0"的根因是 planner( 3. 影响范围与严重程度 — 当前没有任何压缩路径读这两列(grep 全仓确认),只有 4. 怎么修复的 — sqlite( 5. 是否有回归问题(重点)—
6. 是否破坏缓存 — 不涉及。纯存储侧记账列填充,不碰前缀缓存/重试/增量游标/去重键。 7. 是否改了设计方向 — 不转向,是既定三层方案的落地第一步,边界清晰(host 8. 是否引入新配置 — 无。无 env/config/CLI/默认值变化。 测试有效性 — 两个用例都有牙:正向断言 CI 状态澄清(关键) — PR head diff 整洁度 / 代码质量 — 双后端对称镜像符合本文件既有惯例(store.ts/mysql.ts 本就处处互镜);命名达意( 合并判定:LGTM(带一个建议项)。 代码正确、范围克制、披露到位、方向对。建议合入前:①把全表扫描改成 性能那条要不要我直接在这个分支上补 commit?因为是你自己按"PR 1/3 只做填边界"刻意收着的最小范围,我没擅自动你的分支——你点头我就推(独立标注 commit);或者留到 PR2 开始真正读这两列时一并优化也行。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.793-master.8735ac4 # this exact build
npm install -g billion-context@master # always the newest merged build
|
) #4335's block start_seq/end_seq fill ran `SELECT msg_id, seq FROM messages WHERE session_id=?` on EVERY persist whenever blocks existed — an O(full transcript) scan plus full-table Map allocation, real cost on the flagship marathon sessions where messages is huge. Collect the union of every block's effective+direct covered ids first, then fetch only those rows via a chunked `msg_id IN (...)` (500/batch). The (session_id, msg_id) index makes each batch targeted lookups; chunking keeps a heavily-folded session under the driver's bind-parameter cap so an over-limit query cannot throw and feed record()'s 3-strike self-disable. The min/max span accumulation and [0,0] placeholder semantics are unchanged — restricting the map to covered rows is byte-equivalent because boundsForCovered only ever looks up covered ids. Applied identically to the sqlite (store.ts) and mysql.ts twins. Adds tests/issue4512-ingest-span-in-clause.test.ts pinning the two paths the old code never exercised: a covered-id union spanning multiple query batches, and directMessageIds feeding the union. Existing #4316 seq-bounds regression tests still pass.
) #4335's block start_seq/end_seq fill ran `SELECT msg_id, seq FROM messages WHERE session_id=?` on EVERY persist whenever blocks existed — an O(full transcript) scan plus full-table Map allocation, real cost on the flagship marathon sessions where messages is huge. Collect the union of every block's effective+direct covered ids first, then fetch only those rows via a chunked `msg_id IN (...)` (500/batch). The (session_id, msg_id) index makes each batch targeted lookups; chunking keeps a heavily-folded session under the driver's bind-parameter cap so an over-limit query cannot throw and feed record()'s 3-strike self-disable. The min/max span accumulation and [0,0] placeholder semantics are unchanged — restricting the map to covered rows is byte-equivalent because boundsForCovered only ever looks up covered ids. Applied identically to the sqlite (store.ts) and mysql.ts twins. Adds tests/issue4512-ingest-span-in-clause.test.ts pinning the two paths the old code never exercised: a covered-id union spanning multiple query batches, and directMessageIds feeding the union. Existing #4316 seq-bounds regression tests still pass.
) #4335's block start_seq/end_seq fill ran `SELECT msg_id, seq FROM messages WHERE session_id=?` on EVERY persist whenever blocks existed — an O(full transcript) scan plus full-table Map allocation, real cost on the flagship marathon sessions where messages is huge. Collect the union of every block's effective+direct covered ids first, then fetch only those rows via a chunked `msg_id IN (...)` (500/batch). The (session_id, msg_id) index makes each batch targeted lookups; chunking keeps a heavily-folded session under the driver's bind-parameter cap so an over-limit query cannot throw and feed record()'s 3-strike self-disable. The min/max span accumulation and [0,0] placeholder semantics are unchanged — restricting the map to covered rows is byte-equivalent because boundsForCovered only ever looks up covered ids. Applied identically to the sqlite (store.ts) and mysql.ts twins. Adds tests/issue4512-ingest-span-in-clause.test.ts pinning the two paths the old code never exercised: a covered-id union spanning multiple query batches, and directMessageIds feeding the union. Existing #4316 seq-bounds regression tests still pass.
) #4335's block start_seq/end_seq fill ran `SELECT msg_id, seq FROM messages WHERE session_id=?` on EVERY persist whenever blocks existed — an O(full transcript) scan plus full-table Map allocation, real cost on the flagship marathon sessions where messages is huge. Collect the union of every block's effective+direct covered ids first, then fetch only those rows via a chunked `msg_id IN (...)` (500/batch). The (session_id, msg_id) index makes each batch targeted lookups; chunking keeps a heavily-folded session under the driver's bind-parameter cap so an over-limit query cannot throw and feed record()'s 3-strike self-disable. The min/max span accumulation and [0,0] placeholder semantics are unchanged — restricting the map to covered rows is byte-equivalent because boundsForCovered only ever looks up covered ids. Applied identically to the sqlite (store.ts) and mysql.ts twins. Adds tests/issue4512-ingest-span-in-clause.test.ts pinning the two paths the old code never exercised: a covered-id union spanning multiple query batches, and directMessageIds feeding the union. Existing #4316 seq-bounds regression tests still pass.
背景
#4316 三层定案的第 3 层(owner 拍板 2026-10-22):折叠记账从 h_id 集合迁到正史
[start_seq, end_seq]区间。本 PR 是该层的 PR 1/3,只做存储侧「填 seq 边界」——为后续 reconcile 区间证据层(PR 2)与内核编号直连(PR 3)打地基。改动
blocks.start_seq/end_seq(两列早存在、NOT NULL、此前恒为占位 0)现在 ingest 时从 append-only 正史解析出真实值:块覆盖消息 id → 其 seq 的 min/max。ingestSession(src/storage/store.tssqlite +src/storage/mysql.ts双后端),而非纯 planner——最终绝对 seq 在事务内重编(delta 占位 seq 于事务里 renumber),planner 拿不到。行为变更披露
start_seq/end_seq从「恒 0」变为「可解析时 = 覆盖消息真实 seq 区间」。当前无任何压缩路径读取这两列,仅export.ts诊断面会看到真实值(0 → 真值,属诊断修正非回归)。压缩行为零变化。无 schema 迁移(列本就存在)、无 kernel 改动(纯 hostsrc/storage)、无新增配置/env/CLI。验证
tests/issue4316-seq-bounds.test.ts(红桩→绿)+ 整个storage-*测试簇 28/28 通过。npm run typecheck通过。Part of #4316(layer 3, PR 1/3)。