Skip to content

[PR] fix(fastpath): #5643 附录二 — the v6 render snapshot evolves BY EVENTS (sync first-scene journal + evidence-addressed apply) - #5732

Merged
ranxianglei merged 1 commit into
2026-11-04_fastpath-v6-msgidfrom
2026-11-05_event-addressed-snapshot
Nov 6, 2026
Merged

ranxianglei merged 1 commit into
2026-11-04_fastpath-v6-msgidfrom
2026-11-05_event-addressed-snapshot

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

(ai: sglang-gpu0/qwen3.8-27b)

What this changes

Post-#5721 supplement on the frozen ordinal-deviation line: the v6 render snapshot table is now the owner-ruling LSM snapshot itself (#5643 附录二) — a compression fold lands as a diff written straight into the table, never as a snapshot replacement:

Moment What happens
First scene (the compress tool endpoint, synchronous) recordServerSideFold journals the fold's covered absolute positions + live block signature into the checkpoint meta in ONE transaction. Row bytes stay untouched — on plugin lanes the agent owns summary rendering, so bili-side folds usually change no wire bytes at all. Idempotent per signature; fail-soft per sid — the tool response never depends on this plane.
Reconcile (the next model turn, forced exactly ONCE) The journal red-gates the round as structural-pending; the full pass classifies every captured cell against the kernel's OWN output (block effectiveMessageIds ↔ pf_ledger.core_id ↔ absolute pos): covered cells upsert in one tx (op apply, prefix bytes + S4 checksums untouched); count-changing witnessed layout churn rides the coarse rebase tier (contiguous replaceAll); any other changed/lost cell kills the plane with its byte-for-byte pre-existing verdict.
Steady Appends continue exactly as before; the gate's strict row-count equality became an abs-pos span check (ascending-distinct positions, holes do not red-gate).

Supersedes #5721's whole-table witnessed rebase for the fine-grained case (it survives verbatim as the coarse tier for native-compaction-class events). Unwitnessed drift stays a permanent kill, semantics unchanged. Store layer unchanged — existing pfRenderCommit already covers arbitrary-pos upserts + S4 checkpoints.

Honest V1 boundaries (flagged, follow-ups)

  • Coarse-tier rebuild reindexes contiguously (pos-preserving reconciliation is the documented follow-up).
  • Physical purge is deferred: journal entries ride as audit trail absorbed by the checkpoint; covered cells are overwritten, not deleted, in V1.
  • Client-native compaction that drops identities leaves dead cores in the append-only identity plane — green window-pass would resurrect them → permanent tripwire-red degradation for that sid; availability intact by design (availability > performance). Alive-filter fix belongs to a separate issue.

Stacking note

This branch stacks on #5721 (036719c21) until it merges; after that I rebase onto the lane tip (2026-11-04_fastpath-v6-msgid) so this PR carries only its own commit. Do not review #5721's two commits here as part of this change.

Verification

Local targeted suites all green (typecheck clean, build implied by typecheck over tsc): issue5664-render-snapshot 18/18 (incl. 6 new event-semantics pins: first-scene journal→red→zero-event apply loop, unwitnessed-covered kill, fine-grained apply prefix-stability, coarse-tier routing both directions, boundCoveredPositions binding rules, sparse-hydrate/non-ascending-kill), fastpath-v6-msgid 14/14, issue5403-watermark-fastpath, lane-decisions-matrix 99/99, e2e-plugin-tool-routing, plugin-agent 76, openai-plugin-postfold-seam, plugin-render-endpoint. Full-suite verdict = CI lanes on the pushed head.

Closes nothing (the scope issue is #5643; progress reported there per owner request).

@ranxianglei ranxianglei changed the title fix(fastpath): #5643 附录二 — the v6 render snapshot evolves BY EVENTS (sync first-scene journal + evidence-addressed apply) [PR] fix(fastpath): #5643 附录二 — the v6 render snapshot evolves BY EVENTS (sync first-scene journal + evidence-addressed apply) Nov 5, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 open [PR] 行里没有第二个针对本线程(v6 render snapshot 事件化演进,#5643 附录二)的修复 PR。唯一相关的是 #5721(witnessed rebase,mirror 行 5723)——它是本 PR 声明的叠层底座,不是平行修复:本 PR 在 fine-grained case 上 supersede 它、保留其 coarse tier,属同一修复线的纵深推进,不是拆片重复(机制同源、症状同源,按「连贯修复不拆片」归并在线内管理)。#5643 是 scope issue,进度按 owner 要求在那里汇报,本 PR 不 close 任何东西——一致。

标签: bug + scope/wiring + p2(缺陷线 #5643 ordinal-deviation 的服务端快路径/渲染快照面修复;最坏情形是 plane kill 降级回 legacy 全量轮,可用性有界,非 p1 类)。

状态同步(开工前快照):

开始逐段读 58bb5b3 的 diff,结论稍后按八问体例给出。

@ranxianglei ranxianglei added bug Something isn't working p2 scope/wiring labels Nov 5, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

结论先行:可合 —— 机制正确、语义收紧是有意的、失败路径全部保可用性。唯一前置条件是你 stacking note 里已承诺的:#5721 合并后把本分支 rebase 到车道 tip(带入 b54b0bd21),让本 PR 只带自己的一个 commit。我核过该交互:b54b0bd21(estimate 只计 F 之外新尾)动的是 prepare-openai ~L701–770 的 clamp/stats 段,本 commit 动的是 ~L804–860 的 station-2 capture 段,无行重叠、语义正交;apply 路径对 rw.rowsParsed/*Cum 的重定基与尾价算法自洽。

结构化八问

  1. 是什么问题:v6 render 快照表必须随压缩事件演进(owner 裁定 Snapshot-Centric Compression Foundation: Single-Row Table as Source of Truth + Compression = Insert/Delete/Update Events | Snapshot中心化的压缩底座:单行表真源 + 压缩=对表的增删改事件(非增量车道,#5652/v6 线定位纠正) #5643 附录二)。[PR] fix(fastpath): witnessed rebase — the v6 render snapshot evolves with compression (#5689 post-merge supplement, #5664) #5721 的「任何 witnessed 分歧 → 整表替换」粒度太粗——O(全表) 抖动、丢位置稳定性、且折叠事件没有第一现场记录。本 PR 把演进做成事件寻址:折叠在发生地同步记账,核对轮按引擎自己的证据逐格分类落表。
  2. 什么原因导致的:快照与账本之间缺一条「事件→绝对位置」的可寻址通道。修法是把压缩引擎自身输出绑回账本:block effectiveMessageIds ↔ pf_ledger.core_id ↔ 绝对 pos(boundCoveredPositions),只有引擎能作证的格子允许移动,其余一律不猜。
  3. 影响范围与严重程度:仅 watermarkV6Render rollout 门内的 v6 render 车道会话;所有失败路径(kill/soft-fail/journal 失败)都降级到字节正确的 legacy 全量路径,可用性无损失。纯内部机制,无外部可见行为变化。
  4. 怎么修复的:三时刻机制——① 压缩工具端点同步单事务 journal(覆盖位置 + 活块签名入 ckpt meta,行字节不动,按签名幂等,per-sid fail-soft,工具响应不依赖它);② journal 把下一轮红门为 structural-pending,强制走一次 legacy 全量核对:零事件 → 单事务清旗(多数插件车道场景,agent 自持摘要渲染、线上字节不变);等量分歧且改动全在覆盖内且有见证 → 细粒度 apply(只 upsert 覆盖格,前缀字节+S4 不动,O(diff));变长布局事件 → 粗粒度 witnessed rebase([PR] fix(fastpath): witnessed rebase — the v6 render snapshot evolves with compression (#5689 post-merge supplement, #5664) #5721 逻辑原样抽出为 rebaseOnWitness);任何覆盖外的改动/丢失 → 永久 kill(判定串逐字不变);③ 稳态追加照旧,门的严格行数相等改为 abs-pos span 检查。
  5. 是否有回归问题:逐项审过既有行为(清单见下)。两处有意语义变更已披露(见下节);另有一处窄化值得点名:witnessed + 等量 + 部分格子在覆盖外被改——[PR] fix(fastpath): witnessed rebase — the v6 render snapshot evolves with compression (#5689 post-merge supplement, #5664) #5721 会整表 rebase 收养新字节,本 PR 改为 kill。这是刻意的:见证只证明「有结构事件」,不能给覆盖外的外来字节背书(代码注释原话 "it would bless foreign bytes"),kill 后 legacy 路径照常服务,可用性无损。经验上该切片很窄(宿主中途改写历史字节本来就罕见,且宿主自压缩通常改变长度会走粗粒度层)。
  6. 是否破坏缓存:不涉及缓存键/重试/增量游标/去重键。apply 刻意保持前缀字节与校验和不动(O(diff) 正是为了前缀稳定);kill 降级 legacy,字节正确、无污染风险。journal 不碰线上字节。
  7. 是否改了设计方向:没有转向——是沿 [PR] fix(fastpath): witnessed rebase — the v6 render snapshot evolves with compression (#5689 post-merge supplement, #5664) #5721 的 owner 裁定方向(快照随压缩演进、禁 recast、不猜)深化到事件粒度。存储层 LSM 形状(行表=L0 日志=S4 ckpt)零改动,我核过 store 实现:checkpoint 在同一事务内吸收自己的日志条目(写 ckpt 后 DELETE FROM pf_render_log),pfRenderMeta 只重放 ls_seq > ckpt_ls_seq 的条目 ⇒ 重启后 pendingDeletes 不会双重计数(journal 事务的 meta 含 n、其 delete 条目同事务被吸收;hydrate 里的累加是纯防御性的)。
  8. 是否引入了新的配置:无。无新 env/config/CLI 面,全部受既有 watermarkV6Render rollout 特性门控。

既有行为清单审查结果(② 回归关)

既有行为 结论
无见证分歧 → 永久 kill(判定串、dead 持久化、行不可变) 保持,测试钉住(KILLED PERMANENTLY + dead 条目 + 行不变)
witnessed 变长事件 → 整表 rebase(连续重编号、S4、audit 重置) 保持,逻辑从 #5721 原样抽为 rebaseOnWitness,双向钉住
witnessed 等量分歧且改动越出覆盖 有意变更:rebase → kill(理由见第 5 问)
steady append / cast 提交路径 保持,仅加 positions 簿记(cast 稠密、append 推 prevLen+i)
门的对齐不变式 有意变更:严格 rows.length === maxPos+1 → span 检查(首≥0、尾≤maxPos、hydrate 时严格升序去重)。滞后/空洞表不再自动红门,但每轮绿门前有 verifyRenderWindow 字节级绊线(存表末 F 行 vs 本轮新算窗口,不等即回 legacy)+ 周期 audit 全量轮 + station-2 分类(愈合或 kill)三重兜底。残留边界:滞后表遇上「移位窗口恰好逐字节相同」的退化巧合可能放行一轮错位形态,直到 audit 轮纠正——需要软失败滞后 + 连续 F 个字节跨区间重复的双重巧合,概率极窄且可用性无损,记录在案不拦
hydrate 一致性 严格连续 → 升序去重(防御性接受稀疏;V1 所有 op 都保稠密,稀疏只可能是未来 op 或损坏);乱序读形仍永久 kill(stub reader 钉住)
S4 校验和纪律 每次 meta 写入重算校验和(测试钉 pfRenderCkptIntact),store 层未动
witness 条件 ①–④ 保持;新增 ⑤(structural-pending 即证据)是纯加法,只在 journal 强制的那一轮为真
压缩工具端点 新增调用 try/catch-warn 包裹、按签名幂等、平面未武装时 no-op(下一轮直接以折叠后形态 cast 武装)、失败压缩不会误记(签名未动→幂等跳过);工具响应零依赖
双折叠先于核对轮到达 核过:pendingDeletes 计数可能重复计同一批位置,但它只是红门布尔量(>0 即红),一次核对轮整体清零,无功能危害

非阻塞观察项(不拦合,供后续参考)

  • 测试缺口一:journal 之后、materialize 之前重启(崩溃窗口)没有专门 pin——「hydrate 读到 pending journal → pendingDeletes == n(而非 2n)→ 红门 → 零事件 apply 清零」。当前由 store 吸收语义保证(我已读码核实),补一个 pin 能把这条闭环钉死。
  • 测试缺口二:append + pendingDeletes>0(增长叠加未确认 journal → 粗粒度层)这条具体分支没有独立 pin,目前靠与 shrink 共享 reconcileLegacyDiverged 入口间接覆盖;那是一行 fallthrough,风险低。
  • PR 正文已诚实登记的三个 V1 边界(粗粒度层连续重编号、物理 purge 延后、客户端原生压缩丢弃身份后的死核复活 tripwire)我都认可其处置:前两个是文档化的后续,第三个按你的说法归另一个 issue(存活过滤修复),本线程不重复立项。

收工判定

可合。diff 干净(4 文件、每一行都在本 PR 目的内,无 kernel/、无配置面、无锁文件漂移)、CI 在推送头 58bb5b3ca 全绿(8 成功 / 9 跳过 / 0 失败)、六枚新 pin 都有牙(断言具体 op 码、精确行字节/位置、持久化死判、S4 完整性、重启再水合)。待 #5721 落地后完成你承诺的 rebase,即可进合并流程。

…sync first-scene journal + evidence-addressed apply)

Post-#5721 supplement on the frozen ordinal-deviation line: the row table
is now the owner-ruling LSM snapshot itself — a compression fold lands as
a DIFF written straight into it, never as a snapshot replacement:

* FIRST SCENE (synchronous, 压缩请求内同步落表): recordServerSideFold runs
  from the compress TOOL endpoint the moment a fold lands — ONE transaction
  journals the covered absolute positions + the live block signature into
  the checkpoint meta; row bytes stay untouched (plugin-lane folds usually
  change no wire bytes). Idempotent per signature; fail-soft per sid —
  the tool response never depends on this plane.
* RECONCILE (下一轮只做核对): the journal red-gates exactly ONE forced
  full pass ('structural-pending'); its capture classifies every cell
  against the kernel's OWN output (block effectiveMessageIds ↔
  pf_ledger.core_id ↔ absolute pos): fold-covered cells upserted in one
  tx ('apply', prefix bytes + checksums untouched); count-changing
  witnessed layout churn rides the COARSE rebase tier (contiguous
  reindex = documented V1 boundary); ANY other changed/lost cell kills
  the plane with its byte-for-byte pre-existing verdict.
* Gate vocabulary gains 'structural-pending'; the rows/count strict
  equality becomes an abs-pos span check (ascending distinct, holes do
  not red-gate); hydrate accepts the sparse shape defensively; debug
  state exposes pendingDeletes. Store layer unchanged (existing
  pfRenderCommit covers arbitrary-pos upserts + S4 checkpoints).
@ranxianglei
ranxianglei force-pushed the 2026-11-05_event-addressed-snapshot branch from 58bb5b3 to 0f3c77c Compare November 5, 2026 15:25
@ranxianglei ranxianglei added the ci:full Run the FULL CI matrix (all e2e lanes + 12-leg test matrix) on this PR label Nov 5, 2026
ranxianglei added a commit that referenced this pull request Nov 6, 2026
…5729)

The new §7.2 bullet anchored the no-CI window on #5732 (an unrelated
fastpath snapshot PR). The thread that documents this window — and that
this PR refs for option C — is #5729. One-token docs correction; no
code/config/wire change.
@ranxianglei
ranxianglei merged commit 33c9e3b into 2026-11-04_fastpath-v6-msgid Nov 6, 2026
35 checks passed
ranxianglei pushed a commit that referenced this pull request Nov 7, 2026
…er merge (#5826)

The merged fastpath-v6 wave (#5652/#5664/#5732) changed two pinned behaviors: PF_SLOT_UNKNOWN moved client->server (import fix), v6 F1 emits a fixed 64-id tail window instead of slicing at learned K, and v6 F2 makes any id-less/duplicate slot clear the whole captured table ('?' never travels). Re-pinned groups 3+4 of tests/msgid-rule-parity.test.ts to the post-merge verdicts with the ruling citations; added an over-cap case for the raised MAX_IDS (16384).
@github-actions github-actions Bot mentioned this pull request Nov 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ci:full Run the FULL CI matrix (all e2e lanes + 12-leg test matrix) on this PR p2 scope/wiring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant