From 82654242888738f205e059df723138f87e7fa27a Mon Sep 17 00:00:00 2001 From: ranxianglei Date: Mon, 19 Oct 2026 11:03:51 +0800 Subject: [PATCH 1/5] #3751: payload-addressed image message ids (media fingerprint seed slot) Image cores now seed their id with the image's own fingerprint (bytes hash for data images, URL string for URL references, ordered join for the legacy plural shape). One image changing re-mints only its own core instead of shifting the whole family's _N ordinals; the ordinals return to their #459/#463 role of disambiguating true duplicates. Text/tool seeds are untouched (non-image ids stay byte-identical); cross-protocol same-image-same-id and responses legacy/split shape equivalence are pinned in the cross-protocol golden. kernel-change: image media fingerprint slot in canonical identity (deriveCanonicalMessageId options.mediaFingerprint + 4 codec sites); kernel 0.0.136. --- CHANGELOG.md | 2 + MESSAGE-IDENTITY.md | 21 ++++ MESSAGE-IDENTITY.zh-CN.md | 17 ++++ kernel/package.json | 2 +- kernel/src/wire/anthropic.ts | 23 +++-- kernel/src/wire/google.ts | 5 + kernel/src/wire/identity.ts | 24 ++++- kernel/src/wire/openai.ts | 37 +++++-- kernel/src/wire/responses.ts | 33 +++++- kernel/tests/cross-protocol-identity.test.ts | 100 +++++++++++++++++++ package-lock.json | 2 +- 11 files changed, 244 insertions(+), 22 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 60b22f8433..f4e6dc9e60 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,8 @@ Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag ### Changes +- **图片消息 id 按载荷寻址(#3751)**:图片 core 的身份种子增加媒体指纹槽——数据图取其字节的 sha256(URL 引用图取 URL 字符串),一张图变化只重铸它自己那条 core,不再把全族 `_N` 序号整体位移(该位移曾是每轮断缓存与 #3748 写放大事故的喂数机)。`_N` 序号回归本职:仅区分字节完全相同的重复图。文本/工具消息种子不变,id 逐字节保持;跨协议同图同 id、responses legacy 单图形状与拆分形状同 id(金桩钉死)。存量图片会话一次性 re-key,由 #3641 判决分类与 #3756 快照渲染兜底吸收。 + - **Fork 子会话不再因前缀漂移永久 409(#3741,owner 决策:门降 advisory)**:dsh/客户端 fork 出的子会话,其首个请求如果与继承的有序前缀不匹配(升级换代重排消息 id、宿主改写 body 等),过去直接 409 且被拒请求不计数——重试永远撞同一扇门,子会话确定性卡死、无任何自愈或清除路径。现在改为**照常收养**:session-id 绑定本来就是真正的唤醒路由,折叠覆盖差异由 reconcile(#3641 order-over-hash)兜底修复;漂移首请求付出一次性未折叠 token 峰,正确性无损。每次收养打一条带分类的 warn(reason 沿用 #3743 诊断:id-drift / hash-drift-under-same-id / prefix-longer-than-incoming 等),`publicForkPrefix` 在漂移时保持 false,side-lane 降级启发照常发言。测试:三种漂移形态全部 200+转发+父会话不动+恰一条 warn(one-shot:首请求后 stats.requests>0 绕过检查)。 - **快照渲染:proxy 车道的出站前缀改由服务端正史渲染,存量会话对内核升级免疫(#3756,根治 #3752/#3751)。** 折叠覆盖此前锚定在"客户端本轮重发的字节"上——消息 id 由内容哈希现算,身份函数一变(协议切换 #3640、#3642 的一次性 re-key、dsh 图片 offload #3588、宿主自查点)覆盖名单就与历史失联:轻则整段历史因 MAX_ANCHORS=16384 封顶失锚(#3752),重则图片族 `_N` 后缀全体位移重铸、每轮 fold 断缓存(#3751)。新引擎 `src/server/snapshot-render.ts`(灰度开关 `BILI_ROLLOUT_SNAPSHOT_RENDER`,默认 0%):当会话有活跃折叠块且统一存储可读时,prepare 层的出站消息序列 = 服务端快照行前缀(覆盖区由块的 seq/signature 覆盖集自拥抑制 + `acp_summary_*` 锚原位重渲,身份跟随块而非客户端字节)+ 客户端尾巴(分歧点 join 之上逐字保留,rewind 裁剪不复活)。链回溯用 id 快路径 + 内容签名兜底(#3658 canonical 种子同构,升级后同内容仍可链上);任何环节失步整轮降级回 legacy 路径,零行为面风险。验收:身份函数原地翻盐后出站前缀逐字节不变(老病当场钉死)、协议中途切换会话不分叉、客户端任意删/checkpoint 覆盖区不复活(一轮采用滞后,模块头有注释)、稳态轮轮前缀字节稳。观测闩 `metadata.snapshotRenderTurns`;kernel 加 `__setMessageIdSaltForTests` 测试缝(kernel 0.0.134,仅测试可达,生产零调用)。 diff --git a/MESSAGE-IDENTITY.md b/MESSAGE-IDENTITY.md index 226d5c935b..2cef137514 100644 --- a/MESSAGE-IDENTITY.md +++ b/MESSAGE-IDENTITY.md @@ -281,6 +281,27 @@ the lanes where D is off, unavailable, or degraded — and the observation channel (`reconcileFoldCoverage`) still watches the CLIENT view (render and observation are deliberately separate inputs). +## Images are payload-addressed, not family-numbered (#3751) + +The `_N` instance discipline above assumes the base id already separates +*which content* this is. Images broke that assumption until #3751: every +image core carried the same canonical text (`[image]`), so all images in a +conversation hashed to ONE shared base and the `_N` ordinals did all the +separating — positionally. One image changing (offload placeholder, re-encode, +delete) shifted every sibling's ordinal, re-minting the whole family on every +turn (the #3748 write-amplification feeder). + +The fix keeps the doctrine and closes the gap: the canonical seed grows a +sixth slot, but only for media cores — `hash(imageBase64)` for data images +(the bytes both codecs and the content CAS already hold), the URL string for +URL-referenced images (no bytes on the wire — the URL is the only identity +available), and the ordered join of part fingerprints for the rare legacy +plural-sidecar core. Byte-identical images sharing a base id is intended: +`_N` stays as the ordinal for *true duplicates* (the #459/#463 instance +discipline, applied to what it was designed for). One image changing now +re-mints only its own core. Text cores never gain the slot, so non-image ids +stay byte-identical across the upgrade. + ## Known cost of the hash axiom — and how it was fixed Same bytes ⇒ same identity has a cost face: #1476 — a user re-sends a short diff --git a/MESSAGE-IDENTITY.zh-CN.md b/MESSAGE-IDENTITY.zh-CN.md index 71ba0675e6..8e09568cb7 100644 --- a/MESSAGE-IDENTITY.zh-CN.md +++ b/MESSAGE-IDENTITY.zh-CN.md @@ -100,6 +100,23 @@ bili 的应对遵循上文的失败模式公理:**不猜** old→new 配对(猜 #3641 的 reconcile/判决层全部保留:它们仍是 D 关闭/不可用/降级时的安全网;观测通道(`reconcileFoldCoverage`)继续观察**客户端**视图(渲染与观测刻意分离)。 +## 图片按载荷寻址,不再按家族编号(#3751) + +上面的 `_N` 实例纪律有一个前提:base id 本身已经区分了「这是哪份内容」。 +图片在 #3751 之前打破了这个前提——每条图片 core 的规范文本都是同一个 +`[image]` 标记,一个会话里所有图片哈希出**同一个共享 base**,全靠 `_N` +序号做区分——而序号是**位置敏感**的。一张图变化(offload 占位、重编码、 +删除)会把全族序号整体位移,每轮把整个家族的 id 重铸一遍(正是 #3748 +写放大事故的喂数机)。 + +修法保持教义、补上缺口:规范种子增加第六个槽位,但**只有媒体 core 才有** +——数据图取 `hash(imageBase64)`(字节在 codec 与内容 CAS 两边都已经有), +URL 图取 URL 字符串(wire 上没有字节,URL 是唯一可用的身份),罕见的 +legacy 复数 sidecar core 取各部件指纹的有序串。字节相同的图片共享 base +id 是有意的:`_N` 保留给**真正的重复图**(回到 #459/#463 实例纪律的本职)。 +一张图变化现在只重铸它自己那条 core。文本 core 永不加该槽位,非图片 id +跨本次升级逐字节不变。 + ## hash 公理的已知代价——以及修复方向 同字节 ⇒ 同身份有代价面:#1476——用户原样重发与已折叠内容相同的短文本;裸 `h_` id 重新派生;`isCovered()` 吞掉新消息。注意**修复方向**:kernel #459 加实例重编号(`_1/_2…`),#463 加 `lastPassIds` 快照区分折叠后**回声**(id ∈ covered ∧ ∈ 上一轮 → 保持 id,prune 吞掉)与真**新实例**(id ∈ covered ∧ ∉ 上一轮 → 重编号到空闲 `_k`)。content hash 始终是基座,只在其上叠加实例判别——连 hash 公理自己的 bug 都靠**保留 hash** 修,而不是退回位置 id。 diff --git a/kernel/package.json b/kernel/package.json index 432699877c..18e5d6febe 100644 --- a/kernel/package.json +++ b/kernel/package.json @@ -1,6 +1,6 @@ { "name": "acp-kernel", - "version": "0.0.135", + "version": "0.0.136", "description": "Framework-agnostic context-compression engine (model-driven, 3-tier LSM). Pure core: no host dependency.", "license": "MIT", "author": "ranxianglei", diff --git a/kernel/src/wire/anthropic.ts b/kernel/src/wire/anthropic.ts index e0d3274f5c..8871425579 100644 --- a/kernel/src/wire/anthropic.ts +++ b/kernel/src/wire/anthropic.ts @@ -4,6 +4,7 @@ import { CANONICAL_REDACTED_THINKING_TEXT, canonicalToolArgs, deriveCanonicalMessageId, + imageMediaFingerprint, } from "./identity.js"; import { hashId } from "./util.js"; import { ClusterCounter } from "./message-id.js"; @@ -192,20 +193,11 @@ export function anthropicToCore(body: AnthropicRequestBody): Flat { break; } case "image": { - const base = deriveCanonicalMessageId( - m.role, - "text", - CANONICAL_IMAGE_TEXT, - ); - // #3642: parse the base64 source into the shared media sidecars so - // every dialect presents the same image core (marker text + - // imageMediaType/imageBase64); the raw block keeps verbatim - // round-trip. Other source shapes keep the raw-only sidecar. const source = b.source && typeof b.source === "object" && !Array.isArray(b.source) - ? (b.source as { type?: unknown; media_type?: unknown; data?: unknown }) + ? (b.source as { type?: unknown; media_type?: unknown; data?: unknown; url?: unknown }) : undefined; const mediaSidecar = source?.type === "base64" && @@ -216,6 +208,17 @@ export function anthropicToCore(body: AnthropicRequestBody): Flat { imageBase64: source.data, } : {}; + const base = deriveCanonicalMessageId(m.role, "text", CANONICAL_IMAGE_TEXT, { + // #3751: payload-addressed ids — base64 source hashes its bytes, + // URL source degrades to the URL string, unknown shapes keep the + // shared legacy base. + mediaFingerprint: + source?.type === "base64" && typeof source.data === "string" + ? imageMediaFingerprint(source.data) + : typeof source?.url === "string" + ? imageMediaFingerprint(source.url) + : undefined, + }); msgs.push({ id: clusters.next(base), role: m.role, diff --git a/kernel/src/wire/google.ts b/kernel/src/wire/google.ts index f85db1266b..2fd8970814 100644 --- a/kernel/src/wire/google.ts +++ b/kernel/src/wire/google.ts @@ -4,6 +4,7 @@ import { CANONICAL_IMAGE_TEXT, canonicalToolArgs, deriveCanonicalMessageId, + imageMediaFingerprint, } from "./identity.js"; import type { BiliMessage } from "./bili-message.js"; @@ -216,6 +217,10 @@ export function googleToCore(body: GoogleRequestBody): GoogleFlat { isModel ? "assistant" : "user", "text", CANONICAL_IMAGE_TEXT, + { + // #3751: payload-addressed ids — inlineData hashes its bytes. + mediaFingerprint: imageMediaFingerprint(inline.data), + }, ); msgs.push({ id: clusters.next(base), diff --git a/kernel/src/wire/identity.ts b/kernel/src/wire/identity.ts index 1f67201be4..890637d563 100644 --- a/kernel/src/wire/identity.ts +++ b/kernel/src/wire/identity.ts @@ -93,12 +93,32 @@ export function deriveCanonicalMessageId( role: string, contentType: string, text: string, - options: { toolName?: string } = {}, + options: { toolName?: string; mediaFingerprint?: string } = {}, ): string { - const seed = `${idSalt ? idSalt + "|" : ""}${role}|${contentType}||${options.toolName ?? ""}|${text}`; + const seed = + `${idSalt ? idSalt + "|" : ""}${role}|${contentType}||${options.toolName ?? ""}|${text}` + + (options.mediaFingerprint !== undefined ? `|${options.mediaFingerprint}` : ""); return "h_" + hashId(seed); } +/** #3751 — media fingerprint for image-bearing cores. + * + * Pre-#3751 every image core shared the same canonical text ("[image]"), so + * all images in a conversation hashed to ONE base id and were told apart + * only by the ClusterCounter's positional `_N` ordinals — one image changing + * (offload placeholder, re-encode, delete) shifted the whole family's + * ordinals and re-minted every sibling id on every turn. + * + * The fingerprint makes the id content-addressed per image: data images hash + * their decoded bytes; URL-referenced images (no bytes on the wire) degrade + * to the URL string; the rare legacy plural-sidecar core (pre-#3642 shape) + * joins its parts' fingerprints in order. Byte-identical images sharing a + * base id is INTENDED — `_N` stays as the ordinal for true duplicates only. + * Non-image ids never gain the sixth slot, so they stay byte-identical. */ +export function imageMediaFingerprint(base64OrUrl: string): string { + return hashId(base64OrUrl); +} + /** Stable seed for opaque wire items (responses reasoning items without * text, computer/file/web/image/code/mcp call items): hash the item with * volatile lifecycle fields stripped — `id` (provider-minted, changes on diff --git a/kernel/src/wire/openai.ts b/kernel/src/wire/openai.ts index 03cbd6158e..412fa640dc 100644 --- a/kernel/src/wire/openai.ts +++ b/kernel/src/wire/openai.ts @@ -3,8 +3,27 @@ import { CANONICAL_IMAGE_TEXT, canonicalToolArgs, deriveCanonicalMessageId, + imageMediaFingerprint, } from "./identity.js"; import { hashId } from "./util.js"; + +/** #3751: ordered image fingerprints for a legacy plural-sidecar part list — + * data-URL images hash their bytes, URL images hash the URL, non-image + * parts contribute nothing (the legacy id of an image-free opaque core is + * preserved verbatim). */ +function imageFingerprintsOfParts(parts: unknown[]): string[] { + const out: string[] = []; + for (const part of parts) { + if (!part || typeof part !== "object") continue; + const p = part as { type?: unknown; image_url?: { url?: unknown } }; + if (p.type !== "image_url") continue; + const url = typeof p.image_url?.url === "string" ? p.image_url.url : undefined; + if (!url) continue; + const parsed = parseDataUrl(url); + out.push(imageMediaFingerprint(parsed ? parsed.base64 : url)); + } + return out; +} import { ClusterCounter } from "./message-id.js"; import { parseDataUrl, type BiliMessage } from "./bili-message.js"; @@ -145,11 +164,13 @@ export function openaiToCore(body: OpenAIRequestBody): Flat { (part as { image_url?: { url?: unknown } }).image_url?.url; const parsed = typeof url === "string" ? parseDataUrl(url) : undefined; - const base = deriveCanonicalMessageId( - "user", - "text", - CANONICAL_IMAGE_TEXT, - ); + const base = deriveCanonicalMessageId("user", "text", CANONICAL_IMAGE_TEXT, { + // #3751: payload-addressed ids — data images hash their bytes, + // URL images degrade to the URL string. + mediaFingerprint: imageMediaFingerprint( + parsed ? parsed.base64 : typeof url === "string" ? url : "", + ), + }); msgs.push({ id: clusters.next(base), role: "user", @@ -196,7 +217,11 @@ export function openaiToCore(body: OpenAIRequestBody): Flat { ...(host.rawOpenaiContentParts ?? []), ]; } else { - const base = deriveCanonicalMessageId("user", "text", ""); + const base = deriveCanonicalMessageId("user", "text", "", { + // #3751: the legacy plural core gets the ordered join of its + // image fingerprints (if any) — zero images keeps the legacy id. + mediaFingerprint: imageFingerprintsOfParts(leadingOpaque).join(",") || undefined, + }); msgs.push({ id: clusters.next(base), role: "user", diff --git a/kernel/src/wire/responses.ts b/kernel/src/wire/responses.ts index 7ee68ab03b..17b01e33c7 100644 --- a/kernel/src/wire/responses.ts +++ b/kernel/src/wire/responses.ts @@ -6,6 +6,7 @@ import { canonicalOpaqueSeed, canonicalToolArgs, deriveCanonicalMessageId, + imageMediaFingerprint, } from "./identity.js"; import type { ConversationIdentity } from "./util.js"; import { hashId } from "./util.js"; @@ -309,12 +310,18 @@ export function responsesToCore( part.type === "input_image" && typeof part.image_url === "string" ) { + const image = parseDataUrl(part.image_url); const base = deriveCanonicalMessageId( "user", "text", CANONICAL_IMAGE_TEXT, + { + // #3751: payload-addressed ids — bytes when present, else URL. + mediaFingerprint: imageMediaFingerprint( + image ? image.base64 : part.image_url, + ), + }, ); - const image = parseDataUrl(part.image_url); msgs.push({ id: clusters.next(base), role: "user", @@ -374,8 +381,30 @@ export function responsesToCore( role === "user" ? userDisplayText(message.content, effText) : effText; + // #3751: payload-addressed ids — every input_image in the merged + // shape contributes its fingerprint in order (single image ⇒ one + // fingerprint, byte-identical to the split path's image core so + // both forms of one logical message share one id; text-only legacy + // cores keep their ids verbatim). + const mediaFingerprint = Array.isArray(message.content) + ? message.content + .flatMap((part) => { + if ( + part?.type === "input_image" && + typeof part.image_url === "string" + ) { + const raw = part.image_url; + const decoded = parseDataUrl(raw); + return [imageMediaFingerprint(decoded ? decoded.base64 : raw)]; + } + return []; + }) + .join(",") || undefined + : undefined; coreId = clusters.next( - deriveCanonicalMessageId(role, "text", displayText), + deriveCanonicalMessageId(role, "text", displayText, { + mediaFingerprint, + }), ); const imageUrl = Array.isArray(message.content) ? message.content.find( diff --git a/kernel/tests/cross-protocol-identity.test.ts b/kernel/tests/cross-protocol-identity.test.ts index 32c0ea28b1..7d3b927d89 100644 --- a/kernel/tests/cross-protocol-identity.test.ts +++ b/kernel/tests/cross-protocol-identity.test.ts @@ -240,3 +240,103 @@ test("cross-protocol golden: deleting one of two sibling tool calls only renumbe assert.equal(bothCalls[1]!.replace(/_1$/, ""), soloCall[0]!); assert.notEqual(bothCalls[1], soloCall[0], "suffix renumbers when the earlier twin disappears"); }); + +/** + * #3751 — payload-addressed image ids. Pre-#3751 every image shared one + * base id (the "[image]" marker hash) and ClusterCounter `_N` ordinals told + * them apart positionally: one image changing re-minted the whole family. + * Now the seed carries the image's own fingerprint (bytes hash, URL + * fallback), so family ordinals only disambiguate true duplicates. + */ + +test("#3751: one image changing re-mints only its own core, never the sibling", () => { + const conv = (imgB: string) => + openaiToCore({ + model: "gpt-x", + messages: [ + { + role: "user", + content: [ + { type: "image_url", image_url: { url: `data:image/png;base64,${IMG_DATA}` } }, + { type: "image_url", image_url: { url: `data:image/png;base64,${imgB}` } }, + ], + }, + ], + }).msgs; + + const before = conv("c2hvcA=="); + const after = conv("c2hvcEch"); + assert.equal(before.length, 2); + assert.equal(after.length, 2); + assert.equal(before[0]!.imageBase64, IMG_DATA); + // THE invariant: the untouched sibling keeps its id byte-identical. + assert.equal(after[0]!.id, before[0]!.id, "sibling image id stable"); + assert.notEqual(after[1]!.id, before[1]!.id, "changed image re-mints only itself"); + // No ordinals in either family (distinct fingerprints ⇒ distinct bases). + for (const m of [...before, ...after]) assert.match(m.id, /^h_[0-9a-f]+$/); +}); + +test("#3751: byte-identical images share the base id and split via the _N ordinal", () => { + const msgs = anthropicToCore({ + model: "claude-x", + messages: [ + { + role: "user", + content: [ + { type: "image", source: { type: "base64", media_type: "image/png", data: IMG_DATA } }, + { type: "image", source: { type: "base64", media_type: "image/png", data: IMG_DATA } }, + ], + }, + ], + }).msgs; + assert.equal(msgs.length, 2); + assert.match(msgs[0]!.id, /^h_[0-9a-f]+$/, "first duplicate: bare base"); + assert.match(msgs[1]!.id, /^h_[0-9a-f]+_1$/, "second duplicate: _1 ordinal"); +}); + +test("#3751: URL-referenced images degrade to the URL fingerprint (stable, distinct)", () => { + const conv = (url: string) => + openaiToCore({ + model: "gpt-x", + messages: [{ role: "user", content: [{ type: "image_url", image_url: { url } }] }], + }).msgs; + const a1 = conv("https://cdn.example.com/a.png"); + const a2 = conv("https://cdn.example.com/a.png"); + const b = conv("https://cdn.example.com/b.png"); + assert.equal(a1[0]!.id, a2[0]!.id, "same URL ⇒ same id across turns"); + assert.notEqual(a1[0]!.id, b[0]!.id, "different URL ⇒ different id"); + // And a URL image never collides with the same-name data image (no bytes + // on the wire — the URL is the only identity we have). + const data = openaiToCore({ + model: "gpt-x", + messages: [ + { role: "user", content: [{ type: "image_url", image_url: { url: `data:image/png;base64,${IMG_DATA}` } }] }, + ], + }).msgs; + assert.notEqual(a1[0]!.id, data[0]!.id); +}); + +test("#3751: responses legacy single-image shape and split shape share one id", () => { + const legacy = responsesToCore({ + model: "gpt-x", + input: [ + { type: "message", role: "user", content: [{ type: "input_image", image_url: `data:image/png;base64,${IMG_DATA}` }] }, + ], + }).msgs; + const split = responsesToCore({ + model: "gpt-x", + input: [ + { + type: "message", + role: "user", + content: [ + { type: "input_text", text: "" }, + { type: "input_image", image_url: `data:image/png;base64,${IMG_DATA}` }, + ], + }, + ], + }).msgs; + assert.equal(legacy.length, 1); + assert.equal(split.length, 1, "empty text part folds away"); + assert.equal(legacy[0]!.id, split[0]!.id, "same logical image ⇒ same id via both paths"); +}); diff --git a/package-lock.json b/package-lock.json index bd7e652912..3ee9749c1c 100644 --- a/package-lock.json +++ b/package-lock.json @@ -48,7 +48,7 @@ }, "kernel": { "name": "acp-kernel", - "version": "0.0.135", + "version": "0.0.136", "dev": true, "license": "MIT", "devDependencies": { From f296b0e0d771b0c82e23bf985db80cbd3ccf36d8 Mon Sep 17 00:00:00 2001 From: ranxianglei Date: Mon, 19 Oct 2026 11:28:45 +0800 Subject: [PATCH 2/5] snapshot-render harness: settle on the store tail view instead of a fixed 60ms nap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The first #3767 full run tripped the D-lane tests on loaded CI runners: the save→ingest roundtrip outlived the nap, the D engine degraded to the legacy path (surface asserts still passed — prune renders the same anchor), and only the snapshotRenderTurns latch exposed the miss. Poll until the session's tail view CHANGES (not grows — a rewind turn legitimately net-shrinks the pointer sequence) before the next turn. --- tests/snapshot-render.test.ts | 31 ++++++++++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-) diff --git a/tests/snapshot-render.test.ts b/tests/snapshot-render.test.ts index cc8611d78f..706fc581da 100644 --- a/tests/snapshot-render.test.ts +++ b/tests/snapshot-render.test.ts @@ -62,6 +62,7 @@ type Harness = { bodies: string[]; openaiUrl: string; anthropicUrl: string; + store: { unifiedTailSeqs(sessionId: string): number[] | null } | null; sessionOf: (label: string) => { meta: { label: string }; state: { blocks: { id: string; active: boolean }[] }; metadata: Record } | undefined; }; @@ -128,6 +129,7 @@ async function withHarness(fn: (h: Harness) => Promise, storeEnabled: bool bodies, openaiUrl: `http://127.0.0.1:${port}/bili/http://127.0.0.1:${upstreamPort}/v1/chat/completions`, anthropicUrl: `http://127.0.0.1:${port}/bili/http://127.0.0.1:${upstreamPort}/v1/messages`, + store: storeEnabled ? store : null, sessionOf: (label) => listSessions().find((x) => x.meta.label === label) as never, }; try { @@ -155,7 +157,7 @@ async function turn(h: Harness, url: string, label: string, messages: unknown[], body: JSON.stringify({ model: "gpt-test", messages, stream: true }), }); const body = await r.text(); - await new Promise((res) => setTimeout(res, 60)); // let debounced=0 writes land + await settleRows(h, label); // debounced=0 write + ingest must LAND before the next turn's D view build void history; // (history appended below after body parse) // realistic client: fold the assistant's visible reply into the transcript if (history) { @@ -164,6 +166,33 @@ async function turn(h: Harness, url: string, label: string, messages: unknown[], return { status: r.status ?? 0, body, idx: h.bodies.length - 1 }; } +/** Wait until the unified store's row count for the session GROWS past the + * count observed after the previous turn (first turn: ≥1) — the D engine + * reads rows DURING the next turn's prepare, and on a loaded CI runner the + * save→ingest roundtrip can outlive a fixed 60ms nap (the exact flake + * #3767's first full run tripped: surface asserts passed on the legacy + * path, only the D latch exposed that rows weren't there yet). */ +const rowsSeen = new Map(); +async function settleRows(h: Harness, label: string): Promise { + if (!h.store) { + await new Promise((res) => setTimeout(res, 60)); // D off: legacy path, no rows needed + return; + } + const prev = rowsSeen.get(label) ?? 0; + for (let i = 0; i < 200; i++) { + const n = h.store.unifiedTailSeqs(label)?.length ?? 0; + // The tail view is the CURRENT observation selector — a rewind turn + // legitimately NET-SHRINKS it (dropped pointers), so wait for any + // CHANGE, not growth. + if (n !== prev && n > 0) { + rowsSeen.set(label, n); + return; + } + await new Promise((res) => setTimeout(res, 25)); + } + throw new Error(`settleRows: store tail view never changed for ${label} (still ${prev})`); +} + /** One assistant transcript entry per response: all plain-text content deltas * concatenated (cache-friendly-proxy client discipline — the stateless client * sees the model text plus any streamed 📦 receipt line as one reply). */ From d8310e4938af8e451e4005fc8ed085a091708ed4 Mon Sep 17 00:00:00 2001 From: awork Date: Mon, 19 Oct 2026 11:45:01 +0800 Subject: [PATCH 3/5] docs(kernel): image fingerprint comments hash the base64 payload, not decoded bytes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review nit on #3751 (no behavior change): imageMediaFingerprint() hashes the base64 string carried on the wire (design m36538: sha256(imageBase64)), but four code comments described the input as decoded bytes. Wording only — identity.ts, openai.ts, responses.ts, google.ts. --- kernel/src/wire/google.ts | 2 +- kernel/src/wire/identity.ts | 2 +- kernel/src/wire/openai.ts | 6 +++--- kernel/src/wire/responses.ts | 2 +- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/kernel/src/wire/google.ts b/kernel/src/wire/google.ts index 2fd8970814..13595e0f27 100644 --- a/kernel/src/wire/google.ts +++ b/kernel/src/wire/google.ts @@ -218,7 +218,7 @@ export function googleToCore(body: GoogleRequestBody): GoogleFlat { "text", CANONICAL_IMAGE_TEXT, { - // #3751: payload-addressed ids — inlineData hashes its bytes. + // #3751: payload-addressed ids — inlineData hashes its base64 payload. mediaFingerprint: imageMediaFingerprint(inline.data), }, ); diff --git a/kernel/src/wire/identity.ts b/kernel/src/wire/identity.ts index 890637d563..6f9da6d30b 100644 --- a/kernel/src/wire/identity.ts +++ b/kernel/src/wire/identity.ts @@ -110,7 +110,7 @@ export function deriveCanonicalMessageId( * ordinals and re-minted every sibling id on every turn. * * The fingerprint makes the id content-addressed per image: data images hash - * their decoded bytes; URL-referenced images (no bytes on the wire) degrade + * their base64 payload; URL-referenced images (no bytes on the wire) degrade * to the URL string; the rare legacy plural-sidecar core (pre-#3642 shape) * joins its parts' fingerprints in order. Byte-identical images sharing a * base id is INTENDED — `_N` stays as the ordinal for true duplicates only. diff --git a/kernel/src/wire/openai.ts b/kernel/src/wire/openai.ts index 412fa640dc..bb0176ff83 100644 --- a/kernel/src/wire/openai.ts +++ b/kernel/src/wire/openai.ts @@ -8,9 +8,9 @@ import { import { hashId } from "./util.js"; /** #3751: ordered image fingerprints for a legacy plural-sidecar part list — - * data-URL images hash their bytes, URL images hash the URL, non-image - * parts contribute nothing (the legacy id of an image-free opaque core is - * preserved verbatim). */ + * data-URL images hash their base64 payload, URL images hash the URL, + * non-image parts contribute nothing (the legacy id of an image-free opaque + * core is preserved verbatim). */ function imageFingerprintsOfParts(parts: unknown[]): string[] { const out: string[] = []; for (const part of parts) { diff --git a/kernel/src/wire/responses.ts b/kernel/src/wire/responses.ts index 17b01e33c7..946754ce4a 100644 --- a/kernel/src/wire/responses.ts +++ b/kernel/src/wire/responses.ts @@ -316,7 +316,7 @@ export function responsesToCore( "text", CANONICAL_IMAGE_TEXT, { - // #3751: payload-addressed ids — bytes when present, else URL. + // #3751: payload-addressed ids — base64 payload when present, else URL. mediaFingerprint: imageMediaFingerprint( image ? image.base64 : part.image_url, ), From 5ee2586f8b06d5db97dbae65721314337981fb70 Mon Sep 17 00:00:00 2001 From: awork Date: Mon, 19 Oct 2026 11:45:01 +0800 Subject: [PATCH 4/5] style(kernel): keep openai.ts import block contiguous Review nit on #3751 (no behavior change): imageFingerprintsOfParts was inserted between two import groups. Move it below the imports so the file matches the repo's all-imports-first convention. --- kernel/src/wire/openai.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/src/wire/openai.ts b/kernel/src/wire/openai.ts index bb0176ff83..ff20584f31 100644 --- a/kernel/src/wire/openai.ts +++ b/kernel/src/wire/openai.ts @@ -6,6 +6,8 @@ import { imageMediaFingerprint, } from "./identity.js"; import { hashId } from "./util.js"; +import { ClusterCounter } from "./message-id.js"; +import { parseDataUrl, type BiliMessage } from "./bili-message.js"; /** #3751: ordered image fingerprints for a legacy plural-sidecar part list — * data-URL images hash their base64 payload, URL images hash the URL, @@ -24,8 +26,6 @@ function imageFingerprintsOfParts(parts: unknown[]): string[] { } return out; } -import { ClusterCounter } from "./message-id.js"; -import { parseDataUrl, type BiliMessage } from "./bili-message.js"; export type OpenAIContentPart = | { type: "text"; text: string } From 4c9123fc918d9a986e842ed7e3c4d02a7db4cdb1 Mon Sep 17 00:00:00 2001 From: awork Date: Mon, 19 Oct 2026 12:16:51 +0800 Subject: [PATCH 5/5] test: port-zone #3044 leg accepts the #3654 twin-adopt exit code 3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CI-red fix (labeled, unrelated to #3751): the 'unmarked lane'd spawn against a LIVE same-build twin' live test pinned exitCode === 1, but since #3654 a twin-collision loser exits TWIN_ADOPT_EXIT_CODE (3) as the spawner's adopt signal (src/listen-policy.ts) — every other listen failure keeps 1. Which code surfaces depends on a socket-level race (clean EADDRINUSE + registry hit → 3; concurrent-bind-tolerant bind losing at the post-bind claim → 1), so the pin flaked per tree on the ubuntu shards (master run on this PR's exact base failed identically). Both outcomes satisfy the test's invariant — lost loudly without hopping — which the hop/port/twin-alive asserts keep carrying. --- tests/port-zone-live.test.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tests/port-zone-live.test.ts b/tests/port-zone-live.test.ts index e0de8db6a4..69421f03fd 100644 --- a/tests/port-zone-live.test.ts +++ b/tests/port-zone-live.test.ts @@ -495,7 +495,10 @@ test( child.once("exit", (code) => resolve(code)); setTimeout(() => resolve(undefined), 120_000).unref(); }); - assert.equal(exitCode, 1, "the duplicate must fail fast with exit code 1"); + // #3654: a twin-collision loser exits TWIN_ADOPT_EXIT_CODE (3, the spawner's adopt signal); + // a loser that loses at the post-bind claim exits 1. Both = lost loudly without hopping — + // the hop/port asserts below carry the real invariant. + assert.ok(exitCode === 1 || exitCode === 3, `the duplicate must fail fast (exit 1 or the #3654 adopt signal 3), got ${String(exitCode)}`); assert.equal(await portOpen(ZONE_PORT_BASE + 1), false, "nothing may have hopped onto the next port — the hopper IS the orphan shape"); assert.equal(await portOpen(ZONE_PORT_BASE), true, "the serving twin keeps its port"); assert.ok(daemon1 !== undefined && isPidAlive(daemon1), "the serving twin must still be alive");