Repository navigation
[PR] feat(#4313): 每条消息和每个压缩块带上不可变本地时间戳——模型从此分清什么早发生、什么刚发生(默认关闭) - #5236
Conversation
…tsStamps mint, <acp> time attribute rendering (no wall-clock read on the render path), fold-receipt time spans, prompt ordering rules (smaller ref/time = earlier)
…unified schema v10 adds nullable messages.ts (sqlite+mysql, probe-first ALTER, ingest-stamp-once, NO backfill: pre-feature rows render byte-identical without the attribute), hydrate onto CoreMessage.ts for snapshot render/incremental lane/shadow reads, and append covered [start – end] span to compression summary fingerprint lines when resolvable
…print spans; storage v10 migration/watermark/hydration assertions; regenerate storage-render goldens (tsStamps/ts column in state digest) with wall-clock display normalization in the golden drive harness alongside the existing port/sid/tz masking
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.7384-pr.5236.160e879This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5236.tgz
npm install -g packageInstall instructions are refreshed on each push. |
The timestamps feature bumped kernel/package.json 0.0.170 -> 0.0.171 but left kernel/package-lock.json at 0.0.170, so the lock-sync job failed on the branch. Lockfile now matches; only the two version fields change.
… under the global legacy test baseline its sidecar never opens and the index reconstruction gate can never pass
The stamp node and tag/summary time rendering now key off a ports flag
(createCore({timestamps})); with the default core nothing is minted or
rendered, so default-off bytes stay identical to pre-feature master.
Prompt defaults revert to the exact pre-feature texts; the timestamped
wordings survive as exported variants hosts select behind their own
flag. Version bump 0.0.171 -> 0.0.172 (kernel-change: host-side opt-in
toggle for PR #5236 requires per-instance pipeline gating).
…feature wire bytes compress.messageTimestamps / BILI_MESSAGE_TIMESTAMPS gate the display side end-to-end: core pipeline ports (per-instance), fingerprint-span receipt lines at the single applyRanges site, and the timestamped prompt variants selected at every host prompt exit via messageTimestampSections. Off is the default and stays byte-identical to pre-feature master. Docs (CONFIGURATION.md + zh, website seeds regenerated by the official script, coverage ok), storage-render goldens regenerated off-default (shape equals pre-feature master; only state digests drift), new host toggle suite, and config-surface seeds for both this key and sessions.hardMaxBytes.
…try, lockfile regen - schema head is v10 (#4313 adds the immutable message-ts column); the master merge brought back v9 pins in issue4666/issue4677/issue4796 test files (mysql-contract 'expected 9 actual 10' + latent test-shard failures) — re-pin to 10 - register BILI_MESSAGE_TIMESTAMPS in LIFECYCLE.md §1 (opt-in feature flag, default off) — lifecycle-ledger-guard - regenerate root package-lock.json with the exact CI command; the 0.0.172 bump commit missed the workspace version entry (0.0.171 → lock-sync red)
ranxianglei
left a comment
There was a problem hiding this comment.
Review #5236(feat #4313:消息时间戳)
结论:功能代码审查通过;合并前必须先把分支同步到当前 master(见下,是真实的语义冲突,不是机械版本号冲突)。正式验证留给 CI。
合入前必须处理:分支落后当前 master(88 文件 / 19 个 commit)
master 已合入 2026-10-30 owner 版 #5188/#5140 sidecar 恢复(MaybeOpen 对瞬时环境故障会抛错、hadIntegrityQuickCheckFailure 熔断)与 #4665 archive/restore 携带 identity/retired_at。本分支携带的是同一 #5188 恢复工作的更早并行版(tryRecoverSidecar / isTransientContentionError / "MaybeOpen never throws" 假设)。本地尝试 merge origin/master 在下列文件出现真实内容冲突,已 abort,未推送任何改动:
src/persist.ts(同一段tryRecoverSidecar,两侧注释与错误处理模型不同)src/storage/store.ts/src/storage/mysql.ts/src/storage/shadow.tskernel/package.json、kernel/package-lock.json、根package-lock.json
同步建议:rebase(或 merge)当前 master 后,丢弃本分支自带的旧版 #5188 代码(以 master 为准),只保留 #4313 的改动;最终 kernel 版本按 master 基线(现为 0.0.171)再 +1,并再生成 lockfile。
小问题
- PR 描述写
kernel/package.json 0.0.170→0.0.171,实际提交 bump 到 0.0.172(commit 9c83b0a「sync package-lock for the 0.0.172 bump」)。同步后请统一描述与实际版本。 - (随上解决)master 侧 kernel 已到 0.0.171,本分支从 0.0.170 直跳 0.0.172,lock 三方冲突即源于此。
功能审查(通过项)
- 铸戳一次、永不回填:stamp-timestamps 节点只对真正新到的 id 铸
Date.now();已在 tsStamps 的 id 永不改写(持久化值 > carried 值);已知 id(byRaw 中已存在但无戳)保持无戳——不为老消息编造"今天"(kernel/src/compress.ts createStampTimestampsNode)。渲染路径只读存储值、从不读墙钟——前缀缓存不变量成立,且 verif 例外登记表给了同类于 createdAt 的理由(verif/exceptions.ts)。 - 重启不重铸:cloneState / syncBlocks(kernel/src/sync.ts)/ 水合(src/state-hydration.ts mergeState)三处都携带 tsStamps;丢任何一处都会让重启后首轮全量重铸或静默掉属性,这里都防住了。
- 存储层字节纪律:
serializeForStore先剥掉 ts 再算 identity/content_hash(src/storage/ingest.ts)——加时间戳不改内容身份;v10 可空列(sqlite+mysql 均 probe-first ALTER),存量行不回填(INSERT OR IGNORE 从不 UPDATE),新行缺逐条戳时取批内单一样例 batchTs;fork/archive 克隆行携带 ts(store.ts / mysql.ts 的 INSERT…SELECT 都含 ts 列)。 - 回执跨度不谎报:
blockCoverageTimes(src/stream.ts)要求覆盖区间全部 id(含边界 ref 解析)都有戳,任一缺失整体回退旧格式而不是收窄范围。 - 默认关 = 字节兼容:提示词 DEFAULT/TIMESTAMPED 双变体、createCore 每实例一次性解析(#394)、render 节点独立门控、receipt span 同旗控制;存储层无条件记录所以后开可见完整历史——与描述一致。
- 测试覆盖充分:铸一次不复铸 / 渲染确定性 / 老行原字节 / 跨块继承(kernel/tests/issue4313-message-timestamps.test.ts)+ toggle/fingerprint/migration v10/shadow/golden(golden drive 的墙钟掩码与既有 port/sid/tz 同类)。
本地定向验证(正式留给 CI,惯例)
kernel 定向套件(issue4313-message-timestamps + packs + pipeline)51/51 绿;宿主侧定向批(issue4313-toggle / issue4313-fingerprint / storage-unified / snapshot-shadow)33/33 绿。
是否可合并:暂缓——分支同步到 master(#5188 以 master 版为准)+ CI 绿之后再合;功能实现本身质量过关,未见逻辑缺陷。
… into pr5236w # Conflicts: # kernel/package-lock.json # kernel/package.json # package-lock.json # src/persist.ts # src/storage/mysql.ts # src/storage/shadow.ts # src/storage/store.ts # tests/golden/storage-render/anthropic.json # tests/golden/storage-render/chat.json # tests/golden/storage-render/google.json # tests/golden/storage-render/responses.json # tests/issue5140-contention-not-disabling.test.ts # tests/issue5188-sidecar-recoverable-open.test.ts # tests/plugin-fork.test.ts
|
跟进:分支已同步 master,冲突已解决,重新审查通过(2026-10-31) 做了什么
合并后验证(本地,正式 CI 在跑)
结论更新:上一条评论中的同步前置条件已满足,等本分支 CI 全绿即可合并。 |
|
这个很重要但是是新功能,现在暂缓合并,需要再审查一次 从实证角度说明 关闭开关的时候 渲染是不变的 |
…tatusLine (kernel-guard Rule 2) kernel/src changed five files; the PR body's "0.0.170 → 0.0.171" line was stale — master is already at 0.0.171 (#5182) and this branch carried no bump, which trips ci.yml kernel-guard Rule 2 (src changed without version bump → exit 1). Bump to 0.0.172 and re-sync both lockfiles. If #5236 lands first with 0.0.172, re-sync to 0.0.173 on merge.
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复——但发现 #4313 存在两个并行实现 PR,需要你裁决二选一:
两者设计互相冲突(格式/配置键/存储位置/摘要载体都不同),只能合一个。按先来后到 #5183 在先且仍是 draft、设计更贴近 #4313 讨论收敛的结论;#5236 已非 draft 且声明了 owner 拍板。我不关闭任何一个(关闭权在你)。 标签:pr、scope/kernel(主改面是 kernel 渲染与压缩提示词)、p3(opt-in 特性,默认关零影响)。 正在对 #5236 做完整代码审查(kernel + host + 存储迁移 + 测试有效性 + 回归清点),结论会带两 PR 对比与合并建议一并给出。 |
… 0.0.172 (order coupling noted in PR 5253)
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.940-master.e193861 # this exact build
npm install -g billion-context@master # always the newest merged build
|
…reign writer holds index.db After owner ruling 3a6ab62 (a live second writer never blocks startup), per-operation guards are the sole defense against an O(db-size) exclusive VACUUM starving a co-resident instance. storage gc kept its #4612 up-front refusal (CLI + panel endpoint); sessions rm's default full vacuum had none — while a proxy held index.db it could win the exclusive lock and starve that proxy's writes across the whole rewrite. runRm now checks store.unifiedActiveWriter() (same source of truth as runGc's refusal) before the shrink: a live foreign writer ⇒ skip ONLY the shrink with an actionable pid-named note; the deletion is a short transaction and stays committed (same graceful shape as the pre-existing SQLITE_BUSY path). No-writer and --no-vacuum paths unchanged. All four full-VACUUM sites are now guarded: runGc, runRm, admin.ts panel endpoint (#4612 409), store.gc.
ranxianglei#5128) The trae wire family builds on the model-endpoint-patterns lane, whose stage-1 added the commandcode wire selector, while master carries the hybrid message-timestamp prompt wrapping (ranxianglei#4313/ranxianglei#5236). After both land, ONE relay round-2 code path serves all three at once: src/server/relay.ts:2678-2682 prepared.commandcode ? createCommandcodeAdapter(...) : prepared.traeWire ? createTraeAdapter(...) : pickAdapter(...) commandcode and trae both keep protocol "openai" internally, so both land in that same block — and their stream dialects are different (bare JSONL vs the event:/data: SSE envelope without a [DONE] sentinel). Neither per-lane suite covered the two together, and nothing covered them together with the timestamped lean pack, which is exactly the state a second declared endpoint creates on one proxy instance. The test runs one proxy with both patterns declared, drives both lanes through it, and asserts: each keeps its own wire dialect on the way back, the timestamped lean pack still carries its marker, and a request to the third path stays a verbatim relay. It asserts coexistence only — per-lane wire contracts stay in trae-endpoint / adapter-trae / adapter-commandcode. Held locally before the rebase landed so it would pin the merged line numbers rather than a lane checkout's; verified against 1dea3d9: 44/44 pass across this file plus trae-wire, trae-plugin-pipe, trae-endpoint, adapter-trae and model-endpoints under the CI loader chain, and typecheck is clean.
…reign writer holds index.db After owner ruling 3a6ab62 (a live second writer never blocks startup), per-operation guards are the sole defense against an O(db-size) exclusive VACUUM starving a co-resident instance. storage gc kept its #4612 up-front refusal (CLI + panel endpoint); sessions rm's default full vacuum had none — while a proxy held index.db it could win the exclusive lock and starve that proxy's writes across the whole rewrite. runRm now checks store.unifiedActiveWriter() (same source of truth as runGc's refusal) before the shrink: a live foreign writer ⇒ skip ONLY the shrink with an actionable pid-named note; the deletion is a short transaction and stays committed (same graceful shape as the pre-existing SQLITE_BUSY path). No-writer and --no-vacuum paths unchanged. All four full-VACUUM sites are now guarded: runGc, runRm, admin.ts panel endpoint (#4612 409), store.gc.
…reign writer holds index.db After owner ruling 3a6ab62 (a live second writer never blocks startup), per-operation guards are the sole defense against an O(db-size) exclusive VACUUM starving a co-resident instance. storage gc kept its #4612 up-front refusal (CLI + panel endpoint); sessions rm's default full vacuum had none — while a proxy held index.db it could win the exclusive lock and starve that proxy's writes across the whole rewrite. runRm now checks store.unifiedActiveWriter() (same source of truth as runGc's refusal) before the shrink: a live foreign writer ⇒ skip ONLY the shrink with an actionable pid-named note; the deletion is a short transaction and stays committed (same graceful shape as the pre-existing SQLITE_BUSY path). No-writer and --no-vacuum paths unchanged. All four full-VACUUM sites are now guarded: runGc, runRm, admin.ts panel endpoint (#4612 409), store.gc.
…reign writer holds index.db After owner ruling 3a6ab62 (a live second writer never blocks startup), per-operation guards are the sole defense against an O(db-size) exclusive VACUUM starving a co-resident instance. storage gc kept its #4612 up-front refusal (CLI + panel endpoint); sessions rm's default full vacuum had none — while a proxy held index.db it could win the exclusive lock and starve that proxy's writes across the whole rewrite. runRm now checks store.unifiedActiveWriter() (same source of truth as runGc's refusal) before the shrink: a live foreign writer ⇒ skip ONLY the shrink with an actionable pid-named note; the deletion is a short transaction and stays committed (same graceful shape as the pre-existing SQLITE_BUSY path). No-writer and --no-vacuum paths unchanged. All four full-VACUUM sites are now guarded: runGc, runRm, admin.ts panel endpoint (#4612 409), store.gc.
Resolves #4313(owner 2026-10-30 拍板实现)。
默认关闭(opt-in):本特性由
compress.messageTimestamps开关控制(配置顶层 compress 块布尔值,或环境变量BILI_MESSAGE_TIMESTAMPS;env 优先:0=强制关,其他任意值=开;未设置看配置文件,都没有=关)。关闭时线上出站字节与升级前逐字节一致。开启需重启 bili 实例。存储层无条件记录每条消息时间戳,所以先运行积累的历史在开启后完整浮现。这是啥问题?
模型在线上看得到每条消息的
mNNNNN标号,但看不到任何时间。两次实战后果:怎么解决?
每条消息盖一次不可变时间戳,渲染进现有
<acp>标签;压缩块指纹行自动追加覆盖时间范围。<acp tokens="2.1K" type="tool:bash" time="2026-10-30 09:02:11">m00175</acp>(完整 yyyy-mm-dd hh:mm:ss 本地时间,约 +8 token/条)[起 – 止]范围(覆盖区间任一消息无时间则整体回退旧格式,绝不谎报范围)测试依据
一个一次性代价
升级后新消息首次渲染会带上 time 属性(前缀与旧字节不同 → 该会话当轮缓存自然重建一次);之后的请求全部命中稳定值。存量历史消息不受影响。
kernel-change: #4313 消息时间戳特性 — kernel/src/{compress-tools,compress,index,packs,render-refs,sync,types}.ts 新增不可变 ts 列铸造/渲染/跨块继承 + kernel/tests/*(issue4313 回归套件);kernel/package.json 0.0.171→0.0.173(基线为合并当前 master 后;master 已经由 #5253 占用 0.0.172,按两 PR 约定顺序规则顺延一号),lock 已同步再生成。owner 2026-10-30 拍板(#4313)。2026-10-31 已合并 master 解决冲突(#5188 sidecar 恢复以 master owner 版为准,本 PR 仅保留 #4313 改动),golden storage-render 按官方脚本重新生成。 2026-10-31 再次同步 master(其后新增的 #5253/#5292/#5294/#5163 等合并):零冲突,kernel 1507/1507 + host 存储/compress 定向批全绿,随本次推送。