feat(desktop): preserve transcript continuity across bounded ranges

[Astro-Han]

Problem

#4206 replaced Renderer turn virtualization with a bounded Runtime Host range. That change substantially improved long-transcript performance and removed a second scroll and layout authority.

One problem remains. Opening an older range replaces the current resident range. Turns between those ranges remain durable in Storage, but Desktop no longer has enough position information to represent them. This can make separated ranges appear adjacent and makes navigation across unloaded history incomplete.

The previous plan in this issue was written before #4206 established the current architecture. Its Renderer virtualizer and PR1-PR4 implementation plan are superseded.

Current boundary

flowchart LR
    S[Storage<br/>complete transcript] --> P[Runtime Host pager<br/>bounded bodies]
    P --> D[Desktop<br/>bounded projection]
    D --> R[Renderer<br/>single scroll authority]

Loading

Direction

Add the smallest position contract needed by a real Desktop consumer:

• Maintain lightweight Turn positions incrementally.
• Page positions with a simple keyset cursor and stable revision or watermark.
• Continue loading message bodies through the existing session.transcript.page path.
• Let Desktop represent unloaded ranges without creating another scroll authority.
• Deliver the first change as an end-to-end Desktop slice.

Do not introduce materialized full-history snapshots, durable generations or leases, a second body protocol, or a custom byte scanner unless the Desktop consumer demonstrates that they are necessary.

Acceptance

• Navigating tail to old Turn to intermediate Turn to tail does not create false adjacency.
• Durable Turns remain discoverable while message bodies stay bounded.
• Appending a Turn does not rebuild an O(N) position snapshot.
• The existing transcript scroll authority remains the only writer of scroll position.
• Large-transcript performance does not regress from perf(desktop): bound transcript rendering and eliminate scroll LoAF #4206.

Related: #4206, #4244, #4295.

Codex helped analyze the architecture and draft this correction. I verified the issue against the current implementation and recent performance changes.

[Sun-GLiang]

Thank you for your invitation. I will continue to promote this work.

[Sun-GLiang]

take

[Sun-GLiang]

English

Additional regression scenario: large first turn with an active live tail

This was the condition under which the original bug was first observed and would be useful to preserve as an explicit acceptance/regression case.

Given:

• the transcript is larger than the bounded range budget;
• the first turn is large enough to consume most or all of the historical resident range;
• intermediate turns are durable but non-resident;
• the final turn is still streaming through the active overlay.

After navigating from the live tail to the first turn, the current Renderer projection can effectively become:

resident durable range: Turn 1
active overlay:         Turn N
non-resident history:   Turn 2 ... Turn N-1

Because logical turns are derived from resident messages plus the active overlay, the Renderer can make Turn 1 and Turn N appear adjacent. The intermediate durable turns are still present in storage, but their logical positions disappear along with their non-resident bodies.

The unified logical turn-position model should instead preserve:

Turn 1        loaded historical content
Turn 2...N-1  unloaded logical positions / gaps
Turn N        active overlay at the live tail

Suggested regression coverage:

1. Start with a transcript larger than twice the range budget, with a large first turn and an active final turn.
2. Navigate from the live tail to the first turn while streaming continues.
3. Verify that intermediate durable turns retain logical positions and that the first and active final turns never become logically adjacent.
4. Navigate or scroll back to the live tail.
5. Complete the active-overlay-to-durable handoff and verify stable identity/order, no duplication, no missing turns, and no scroll jump.

This is part of the same paging/virtualization boundary problem, but explicitly covering it would protect the original discovery condition.

简体中文

补充回归场景：超大的首个 turn 与仍处于 active 状态的 live tail

这是最初发现该 bug 时的触发条件，建议作为明确的验收和回归测试场景保留下来。

条件如下：

• transcript 大于有界 range budget；
• 第一个 turn 很大，占用历史驻留范围的大部分或全部预算；
• 中间 turns 已持久化，但当前未驻留；
• 最后一个 turn 仍通过 active overlay 流式生成。

从 live tail 跳转到第一个 turn 后，Renderer 当前的投影实际上可能变成：

驻留的 durable range：Turn 1
active overlay：       Turn N
未驻留历史：           Turn 2 ... Turn N-1

由于逻辑 turns 是从驻留消息与 active overlay 推导的，Turn 1 和 Turn N 可能被错误地表现为相邻。中间 turns 仍存在于 storage，但其逻辑位置会随着未驻留正文一起从 Renderer 中消失。

统一后的逻辑 turn-position 模型应当保留：

Turn 1        已加载的历史正文
Turn 2...N-1  未加载的逻辑位置 / gaps
Turn N        位于 live tail 的 active overlay

建议测试：在流式生成期间跳转到超大的首个 turn，验证中间 turn 的逻辑位置始终存在；然后返回 live tail 并完成 active-overlay 到 durable message 的交接，验证顺序与身份稳定、不重复、不缺失且滚动位置不跳动。

[Sun-GLiang]

English

Controlled Desktop E2E reproduction — rerun at 01369b0

I reran the same large-first-turn + active-live-tail experiment at 01369b08248d8b66b6c5a1a6204f9683397a10a1 (fix(desktop): simplify partial history notice (#4140)). All other conditions were kept the same.

The run used an isolated E2E user-data directory and scratch SQLite workspace for every launch; it did not read or modify a developer database. The fixture Session was bound to the fixture Connection through the current llmConnectionId, so the result was not obscured by the legacy-account health banner.

The fixture constructed this durable transcript:

sequence 0-1: Turn 1 request + assistant plan (> 512 KiB)
sequence 2-3: completed Turn 2 request + reply
sequence 4-5: completed Turn 3 request + reply
sequence 6:   Turn 4 request

It then loaded the durable range around sequence 0 and supplied only the Turn 4 assistant response through the active overlay. The Runtime snapshot identified Turn 4 as the running root turn with one active text stream.

Observed state:

durable watermark:          6
resident durable sequences: [0, 1]
hasNewer:                   true
active overlay:             [Turn 4 assistant]

durable turn index:         [Turn 1, Turn 2, Turn 3, Turn 4]
prompt residency:           [true, false, false, false]
Renderer source count:      2
Renderer logical turn ids:  [Turn 1, Turn 4]
mounted turn ids:           [Turn 1, Turn 4]

The result is unchanged from the earlier baseline: the rendered transcript contains the Turn 1 request/reply followed immediately by the Turn 4 assistant overlay. It does not contain the Turn 2 request/reply, the Turn 3 request/reply, or the durable Turn 4 request. Storage and the landmark index still contain all four Turns, but the non-resident Turns have no logical positions in the Renderer virtualizer.

At this commit, the partial-history UI is visibly simpler: the page shows only “正在查看较早的消息” and “返回最新消息”. The legacy “需要选择账号” banner is absent. This improves the notice but does not change the sparse Renderer projection or restore a continuous logical transcript.

The focused Playwright case passed in two independent launches, each with a fresh isolated database and a different generated Host/session identity. The existing transcript replica/range-store verification also remained green (20/20 tests). All temporary fixture hooks and the temporary spec were removed after capture, and the worktree was restored without a commit.

As before, this is controlled state injection at the DesktopTranscriptReplica preparation boundary. It exercises the production Desktop storage index, transcript range store, Renderer, prompt rail, and turn virtualizer, while deliberately not claiming to reproduce every detail of a natural provider-stream lifecycle. The active-overlay condition is established by the replica/runtime probe; this injected seed does not set the final Turn's data-live-streaming DOM attribute at this commit.

简体中文

受控 Desktop E2E 复现实验——基于 01369b0 重跑

我在 01369b08248d8b66b6c5a1a6204f9683397a10a1（fix(desktop): simplify partial history notice (#4140)）上重新运行了同一个“超大首个 turn + 仍处于 active 状态的 live tail”实验，其他条件保持不变。

每次启动都使用独立的 E2E user-data 目录和临时 SQLite workspace，没有读取或修改开发者数据库。Fixture Session 通过当前的 llmConnectionId 绑定到 fixture Connection，因此结果没有受到旧版账号健康提示条的干扰。

Fixture 构造了以下持久化 transcript：

sequence 0-1：Turn 1 请求 + assistant plan（> 512 KiB）
sequence 2-3：已完成的 Turn 2 请求 + 回复
sequence 4-5：已完成的 Turn 3 请求 + 回复
sequence 6：  Turn 4 请求

随后，实验围绕 sequence 0 加载 durable range，并只通过 active overlay 提供 Turn 4 的 assistant 回复。Runtime snapshot 将 Turn 4 标记为正在运行的 root turn，并包含一个 active text stream。

观察到的状态：

durable watermark：          6
驻留 durable sequences：     [0, 1]
hasNewer：                   true
active overlay：             [Turn 4 assistant]

durable turn index：         [Turn 1, Turn 2, Turn 3, Turn 4]
prompt residency：           [true, false, false, false]
Renderer source count：      2
Renderer logical turn ids：  [Turn 1, Turn 4]
mounted turn ids：           [Turn 1, Turn 4]

结果与先前基线一致：渲染后的 transcript 中，Turn 1 的请求/回复之后直接出现 Turn 4 的 assistant overlay。其中没有 Turn 2 的请求/回复、Turn 3 的请求/回复，也没有已持久化的 Turn 4 请求。Storage 和 landmark index 中仍然存在全部四个 Turns，但未驻留 Turns 在 Renderer virtualizer 中没有逻辑位置。

在这个 commit 上，partial-history UI 明显更简洁：页面只显示“正在查看较早的消息”和“返回最新消息”；旧的“需要选择账号”提示条没有出现。这改善了提示方式，但没有改变稀疏的 Renderer 投影，也没有恢复连续的逻辑 transcript。

聚焦的 Playwright 用例在两次独立启动中均通过；每次都使用全新的隔离数据库和不同的动态 Host/session identity。现有 transcript replica/range-store 验证也保持通过（20/20 tests）。截图完成后，所有临时 fixture hooks 和临时 spec 均已移除，worktree 已恢复且没有提交。

与之前相同，本次复现是在 DesktopTranscriptReplica 准备边界注入受控状态。它覆盖了生产实现中的 Desktop storage index、transcript range store、Renderer、prompt rail 和 turn virtualizer，但不声称复现自然 provider streaming 生命周期的每一个细节。Active overlay 条件由 replica/runtime probe 确认；在此 commit 上，这个注入 seed 不会给最终 Turn 设置 data-live-streaming DOM 属性。

[Sun-GLiang]

I reproduced the user-visible symptom with a real previously persisted Desktop session.

Observed behavior

Opening the existing session 描述一下这个issue。 shows only the final small segment of the conversation (the last tool calls and final answer). The earlier prompts and answers appear to be gone from the transcript, while the prompt rail still shows landmarks for the earlier prompts.

This looks like Renderer-side history loss, but the durable transcript is intact:

• 87 durable records are still present, with a complete sequence range from 0 through 86.
• The stored transcript is about 208,052 bytes.
• The first prompt and its answer, as well as the later intermediate prompts, are all still present in storage.
• The visible UI segment corresponds to the final records near sequence 63–83.

So this is not deletion or storage corruption. The missing content is outside the Renderer’s current resident transcript range.

Important size-boundary detail

This session is smaller than the 512 KiB transcript page/range limit, but it still reproduces the problem because the initial subscription bootstrap is separately limited to 16 KiB:

SESSION_TRANSCRIPT_BOOTSTRAP_MAX_BYTES = 16 * 1024;
SESSION_TRANSCRIPT_PAGE_MAX_BYTES = 512 * 1024;

On open, DesktopTranscriptReplica.prepare() installs only the durable bootstrap and records that older content exists. The Renderer range store then exposes only those resident messages to ChatView, so the turn virtualizer can build logical positions only for that tail range.

Navigating to an unloaded prompt follows the same architectural path described in this issue: DesktopTranscriptReplica.#replaceWithRange() clears the current durable range, installs the target range, and publishes the old sequences as evicted. The Renderer deletes those sequences from its range store. Consequently, paging changes the set of logical turns instead of preserving one continuous transcript with unloaded gaps.

The prompt rail remains populated because it comes from the separate transcript-wide landmark index (listTurnLandmarks), which explains the confusing combination of multiple rail ticks and only one visible transcript segment.

Conclusion

This is a direct real-session reproduction of #4123. The specific initial-open trigger here is the 16 KiB bootstrap boundary rather than the 512 KiB page boundary, but the root cause is the same: non-resident durable turns have no stable logical positions in Renderer state.

The desired fix in this issue—keeping a lightweight transcript-wide turn-position index, representing unloaded ranges as gaps/spacers, and paging message bodies around the viewport—would address this case as well.

[Sun-GLiang]

Implementation checklist

Controller packet: issue-4123-pr1-failure-pair-matrix-v1.3.5-controller-approved-2026-08-31

Current execution packet: issue-4123-pr2-semantic-transcript-v1.4-controller-approved-2026-08-31

• PR1 — Storage semantic position index

  • PR URL: feat(storage): add semantic transcript position snapshots #4258
  • Current PR state: open, Draft; implementation complete at the controller-approved exact head, but review/merge completion is intentionally not claimed
  • Delivery gate: keep Draft and do not merge until PR2 provides the first production Runtime Host consumer and the cumulative PR1 + PR2 composition review passes
  • Final amendment state: v1.3.1 legacy bodyless admission recovery; v1.3.2 persisted bounded admission fixed point; v1.3.3 exact purge invalidation; v1.3.4 persisted failure provenance; v1.3.5 database-enforced origin/reason pair matrix
  • Exact head: 1c79bad38d12cffabfd329be8bfd83baf8728561
  • Controller decision: one exact generation atomically materializes owner/shared dense projections; explicit tagged turn | note | empty keys; projection-bound leases; shared scalar visibility policy reused through Core
  • Review: independent fixed-point Standards review PASS with 0 findings; independent fixed-point Spec review PASS with 0 findings; delivery-boundary P1 at feat(storage): add semantic transcript position snapshots #4258 (review) remains open until PR2 composition proof
  • CI: all checks for exact head 1c79bad38d12cffabfd329be8bfd83baf8728561 passed; main test 25m06s, Windows package 23m27s, dependency audit, macOS/Windows owner, Windows recovery, direct-peer builds, immutable tarball, and Linux/macOS/Windows installed-CLI validations green; conditional installed-CLI Eval job skipped as designed
  • Merge: not-merged; review required
  • Evidence: original base 8c491e64b35cd7af66437efc8020a88b171e2bb6; local npm test exit 0; Storage 1072 total / 1065 pass / 7 skip; Runtime Host 1423 total / 1414 pass / 9 skip; format/lint/typecheck/ASF headers/notices/diff-check passed; worktree and remote branch clean at exact head
  • normalized Turn metadata 与 additive final-v35 schema migration
  • explicit tagged Turn/note/empty position identity，无 prefix guessing
  • bounded legacy/imported/chunked recovery
  • exact snapshot、authorityRevision、snapshotGeneration
  • same-generation owner/shared dense projections 与 shared visibility policy versioning
  • projection-bound lease、ABA、capacity/backpressure、reopen recovery
  • indexed owner/shared pages 与 raw-capacity-accounted body lookup
  • Storage/Core/Host-projector parity verification gate passed
  • PR remains Draft before composition
  • PR1 + PR2 composition gate passed before PR1 returns to Ready for Review

• PR2 — Runtime Host semantic transcript protocol

  • PR URL: feat(runtime-host): page semantic transcript turns #4296
  • Superseded fork-local comparison PR: feat(runtime-host): page semantic transcript turns Sun-GLiang/maka-agent#1
  • Delivery topology: upstream cumulative Draft targeting apache/maka:main; apache/maka lacks the fork-only PR1 base ref, so the Draft temporarily shows PR1 + PR2 and must be narrowed/revalidated after PR1 merges
  • Current PR state: OPEN, Draft; URL/base/head/title/body independently read back; mergeStateStatus=DIRTY against the moving main base is recorded as a pre-PR1-merge condition, not resolved in this step
  • Primary implementation branch: codex/4123-pr2-host-semantic-pager
  • Read-only upstream delivery alias: codex/4123-pr2-host-semantic-pager-upstream-draft at the same exact head
  • Session: 01a052e2-f65e-7f13-abdd-f93611ea2665
  • Worktree: /Users/sun/.codex/worktrees/1497/maka-agent; clean at the exact head
  • Approved packet: issue-4123-pr2-semantic-transcript-v1.4-controller-approved-2026-08-31
  • Logical PR1 predecessor: exact PR1 head 1c79bad38d12cffabfd329be8bfd83baf8728561; GitHub PR base at creation: apache/maka:main at ac59b47b72cdfd63b737dac6946715789ce277c5
  • Exact head: 9d9c67d73313000869a452180654eb27169f3c35
  • Predecessor: upstream PR1 feat(storage): add semantic transcript position snapshots #4258 remains Draft and unmerged; PR2 must not merge first
  • Protocol surface: additive session.transcript.positions.query and session.transcript.turn_window.page; dormant until the later rollout decision
  • Reuse proof: existing transcript reader adapter, shared projector, HMAC token machinery, buffer slicing, client fragment assembler, operation lane, and retained/preparation ledger were extended rather than duplicated
  • Privacy/bounds: dense owner/shared semantic positions; no raw snapshot triple, lease, projection, sequence, hidden count, or recovery detail on wire; complete-position windows only
  • Verification: Storage 1072 total / 1065 pass / 7 skip; Runtime Host 1449 total / 1440 pass / 9 skip; focused Storage 55/55 and focused Runtime Host 184/184; final npm test, build, format, lint, typecheck, ASF headers, notices, CLI notices, epoch-73 compatibility guard, and diff checks passed
  • Benchmark: 10,000 owner/shared positions, warm exact snapshot/WAL invariants, 15 MiB single-position 31-fragment one-read path, 17 MiB typed oversized zero-fragment path, and 17/64 MiB multi-position bounded-window fixtures passed the frozen structural budgets
  • Review: independent Standards and Spec/Composition reviews passed the final exact head with no blocking or high-severity findings; upstream maintainer review remains pending
  • Compatibility: epoch remains 73; additive compatible-change declaration included; no Desktop/Renderer production activation
  • Merge: not-merged; upstream cumulative Draft must not merge or become Ready before PR1, and post-PR1 base narrowing/conflict resolution requires a new controller verification
  • Evidence receipt: upstream PR feat(runtime-host): page semantic transcript turns #4296 OPEN/Draft; base main; delivery alias and primary branch both resolve to exact head 9d9c67d73313000869a452180654eb27169f3c35; PR body, test/benchmark summary, and cumulative PR1→PR2 consumer-invariant matrix recorded; initial PR effort-label checks passed
  • subscription-scoped positions query
  • semantic Turn-window pager
  • shared projection 与 signed cursor
  • shared fragment assembler reuse 与 complete-Turn limits；不得创建第二套 transcript body assembler/decoder
  • oversized Turn typed failure
  • production Runtime Host path directly consumes the amended PR1 Storage APIs
  • no alternate Turn index、body store、full-history scan/cache 或 test-only consumer
  • exact snapshot triple + projection survives Storage query → opaque signed token → continuation
  • token tamper、cross-domain/subscription replay、stale snapshot、ABA fail closed
  • snapshot acquire/page/replace/release、双 generation 与 capacity/backpressure verified through Runtime Host lifecycle
  • legacy/imported/chunked recovery exercised through Runtime Host, including bounded progress and reopen/resume
  • semantic pager returns complete Turns only under frozen position/body/resource limits
  • oversized Turn returns frozen typed failure without partial Turn output
  • PR1 + PR2 cumulative latency、migration/open、database/WAL growth thresholds passed
  • every retained PR1 mechanism maps to a demonstrated PR2 consumer invariant
  • unused PR1 machinery removed or explicitly justified
  • delivery-boundary P1 resolved by independent composition review
  • cumulative PR1 + PR2 verification gate passed
  • Runtime Host verification gate passed

• PR3 — Desktop continuous transcript

  • PR URL: not-created
  • State: not-started; prohibited until PR1 and PR2 pass review, tests, merge/checklist synchronization
  • Review: not-started
  • Merge: not-merged
  • Evidence receipt: not-available
  • bounded Main semantic body window
  • Preload/IPC snapshot fencing
  • Renderer sparse gaps 与稳定 scrollbar
  • 首次打开 >2× range budget 的 session 时，未驻留历史仍保留连续逻辑位置与 scrollbar 几何
  • 连续向上滚动加载相邻 semantic body windows 时，保持当前可见 Turn anchor ≤4px，不得跳到新 range 首 Turn 或 Turn 1
  • single scroll authority 与统一导航
  • active overlay durable handoff
  • keyboard/screen-reader acceptance
  • Desktop verification gate passed

• PR4 — Complete export and release

  • PR URL: not-created
  • Review: not-started
  • Merge: not-merged
  • Evidence receipt: not-available
  • streaming copy/save 与 overlay snapshot
  • compatibility epoch 与 mixed-version handshake
  • continuous transcript 默认启用
  • 旧 partial-range/export 路径清理
  • release/rollback verification passed

• Final end-to-end acceptance

  • tail → oldest → intermediate → tail 保持同一 logical transcript
  • >2×512 KiB 与 >16 KiB bootstrap fixtures通过
  • mounted durable Turns ≤10，active overlay ≤1
  • position/body/Main/Renderer缓存预算通过
  • scroll anchor误差 ≤4px
  • DOM/heap第二轮增长 ≤10%
  • shared projection无identity/count泄漏
  • keyboard、screen reader、焦点恢复通过
  • complete copy/save包含oldest/intermediate/tail且不扩张活动cache
  • PR1–PR4均已merge且无未解决review thread