Replies: 2 comments 2 replies
EnglishI agree with the central diagnosis and direction here. I implemented the original Runtime Host Artifact authority in #1503. That work established a single authenticated, lease-bound, crash-recoverable owner during the Host migration, but it did not establish that everything called an “Artifact” forms one product domain. The producer audit here is a good reason to revisit that boundary. The publication and purge-intent protocols protect one specific situation: metadata and filesystem bytes committed across two durability domains. They provide crash recovery, exact stable-ID replay, and safe cleanup after an uncertain commit. Operational state such as Session recap intent/result does not need that protocol. Session-owned blobs may still need equivalent guarantees, but they do not necessarily need the current generic Artifact authority or its shared source taxonomy. My preferred decomposition is:
I would not make “serve Tool Results directly from the RuntimeEvent log” part of this decision yet. #3630 identified valid reasons to keep the immutable execution ledger separate from the model-facing serving materialization: bounded reads, authorization, copied Sessions, remote Host placement, and independent retention. Retiring ArtifactStore does not require collapsing those responsibilities. Logical ownership and physical deduplication can be evaluated independently. A few details matter for sequencing:
That does not mean we need a long-lived compatibility mode. Our development workflow is AI-agent-driven, so re-homing these paths and completing a cutover can be substantially faster than a conventional migration. I would prefer a short, decisive migration over dual writes, parallel authorities, or an indefinitely retained legacy reader. Development speed, however, does not by itself make durable user state disposable. I suggest a one-time storage transition:
If the project deliberately decides that some Artifact classes are disposable, we can skip migration for those classes. That should be an explicit data-lifecycle decision, not an accidental consequence of removing enum values. I am comfortable with having no ongoing compatibility state; I am not comfortable with silently making an existing workspace prevent Host startup or lose user-visible deliverables. Removing the unused derivation producer can therefore be an independent, revertible first change. Removing legacy source values should happen as part of the one-time cutover. For this direction to close #4027 rather than merely relocate the cost, I think the cutover needs explicit performance invariants:
We can satisfy these requirements without a long-lived compatibility mode: migrate the load-bearing records during a versioned cutover, atomically retire the legacy authority, and reclaim obsolete payloads in bounded batches. The acceptance test should include a workspace with approximately 10–12k legacy Artifacts and verify that Host readiness and an unrelated Session retirement do not scale with the total Artifact count. So I support taking this direction to the dev list, framed as:
That preserves the useful invariants from #1503 without requiring unrelated data to continue sharing one lifecycle protocol. 中文我认同这里的核心诊断和总体方向。 我实现了 #1503 中最初的 Runtime Host Artifact authority。那项工作是在 Host 迁移期间建立一个经过认证、受 lease 约束、可崩溃恢复的统一 owner，但它并没有证明所有被称为 “Artifact” 的值属于同一个产品领域。这次 producer 审计足以说明我们应该重新审视这个边界。 publication 与 purge-intent 协议保护的是一个特定场景：metadata 与文件系统字节需要跨两个持久化域提交。它提供崩溃恢复、精确 stable-ID replay，以及不确定提交后的安全清理。Session recap intent/result 这类 operational state 不需要这套协议。Session-owned blob 可能仍需要同等保证，但不一定需要当前通用的 Artifact authority，也不应继续共享同一套 source 分类。 我倾向的拆分方式是：
我暂时不会把“直接从 RuntimeEvent 日志服务 Tool Result”纳入本次决策。#3630 已指出保持不可变执行账本与模型读取 materialization 分离的合理原因：有界读取、权限、Session 复制、远端 Host 部署以及独立 retention。退役 ArtifactStore 并不要求合并这些职责。逻辑 ownership 与底层物理去重可以分别评估。 推进时有几个细节很重要：
但这不意味着我们需要长期保留兼容模式。 我们的开发流程由 AI Agent 驱动，因此迁移这些路径并完成切换，速度可以明显快于传统开发。我更倾向于一次短促、明确的迁移，而不是引入 dual write、并行 authority 或无限期保留 legacy reader。 不过，开发速度本身并不会使用户磁盘上的持久化状态自动变得可丢弃。我建议执行一次性 storage transition：
如果项目明确决定某些 Artifact 类别本来就是可丢弃的，我们可以跳过这些类别的迁移。但这应当是显式的数据生命周期决策，而不是删除枚举值后意外产生的结果。我可以接受不保留持续运行的兼容态，但不能接受现有 workspace 因此无法启动 Host，或静默丢失用户可见的交付物。 因此，删除无人使用的派生 producer 可以作为第一个独立、可 revert 的改动；删除历史 source 值则应当与一次性 cutover 同时完成。 为了让这个方向真正关闭 #4027，而不是仅仅转移成本，我认为 cutover 还需要明确以下性能不变量：
这些要求不需要长期兼容模式：可以在 versioned cutover 中迁移仍然承重的记录，原子退役旧 authority，再以有界批次回收无用 payload。验收测试应包含约 10–12k 个历史 Artifact，并验证 Host Ready 与无关 Session 的退休耗时不会随 Artifact 总量增长。 因此，我支持把这个方向带到 dev 邮件列表，并建议将决策表述为：
这样既能保留 #1503 中仍然有价值的不变量，也不再迫使互不相关的数据继续共享同一套生命周期协议。 |
|
I think this discussion currently couples two decisions that should be separated. For #4027, the evidence supports a narrower diagnosis: That is primarily an
None of those steps requires first deciding the final product surface for Deep Research, uploads, or subagent write-backs. There is a separate and useful domain-boundary question here. Session recap state should not use a file publication protocol, and Write/Edit/Bash derivations without a reader should not be produced. But the presence of those misuses does not by itself prove that a shared physical blob engine must be retired. Logical ownership, typed facades, and physical storage are separate decisions. If the intended role of So I suggest splitting the scope:
The second effort is worthwhile, but it should not be a prerequisite for fixing the first, and #4027 alone does not justify a product-surface redesign. |
EnglishThanks — I think this is the missing distinction. I agree that we should separate:
The already-merged #4031 and #4033 are useful mitigations, but #4027 remains open and the measured residual still points to global metadata operations and readiness-path work. That evidence justifies indexed row/set operations, removing global scans from Host Ready, and bounding recovery work. It does not by itself justify redesigning Deep Research, uploads, or other product surfaces. I would push the clean target one step further, though. A bounded recovery journal is appropriate for repairing the current mutable, path-oriented store, but it may not be necessary in the zero-history design. With immutable blobs and typed references, the lifecycle can be:
A crash before the reference commit leaves a harmless orphan. A committed reference whose blob is missing is corruption and fails closed. Deletion no longer needs a cross-metadata/filesystem purge transaction, so publication and purge-intent journals may disappear rather than be carried into the replacement. I also think the resulting contract matters more than its name. A refactored Quota and retention should also be class-specific:
So I agree with splitting the work. #4027 should not wait for product-surface decisions. At the same time, I would avoid investing deeply in recovery machinery that the immutable target could eliminate. A useful first proof would be to put Read image snapshots and Tool Result archives behind the narrowed blob capability and verify:
Would this match what you mean by narrowing 中文感谢，这正是当前方案缺失的关键区分。我认同应该拆开三个决策：
已经合并的 #4031 和 #4033 是有价值的缓解，但 #4027 仍未关闭，测得的残留成本依然指向全局 metadata operation 与 readiness-path 工作。这些证据足以支持 indexed row/set operation、从 Host Ready 中移除全局扫描，以及限制 recovery 工作量；但它本身并不能推出必须重新设计 Deep Research、上传或其他产品界面。 不过，我想把干净终局再向前推一步。 若 blob 不可变，并通过 typed reference 使用，其生命周期可以简化为：
引用提交前崩溃只会留下无害 orphan；已经提交的引用找不到 blob，则属于数据损坏并 fail closed。删除不再需要跨 metadata/filesystem 的 purge transaction，因此 publication 与 purge-intent journal 可以被删除，而不是迁入新实现。 我也认为最终契约比名称更重要。无论叫重构后的 Quota 与 retention 也应该按 durability class 区分：
因此我同意拆分工作。#4027 不应等待产品界面决策；但同时也应避免在终局可以删除的 recovery 机制上投入过深。 一个合适的第一阶段证明，是让 Read 图片快照和 Tool Result archive 使用收窄后的 blob capability，并验证：
这是否符合你所说的将 |
|
Yes — this matches what I mean by narrowing the current authority, with one further simplification: before building another filesystem/S3-like immutable blob engine, I think we should test whether SQLite itself can be the physical substrate for the context-offload domain. Problem modelThe cohesive domain is not “Artifact”. It is Agent context offload:
Under that model, Read image snapshots and Tool Result archives belong together. Session recap effect state does not. Write/Edit/Bash copies without a reader should not exist. Uploads and durable user deliverables should not be forced into this store merely because they are bytes. The measured workspace is about 11.7k files / 6 GB, roughly 0.5 MB per payload on average. That is not a scale at which SQLite or a normal filesystem should require ten minutes to become ready. The failure comes from the access pattern: Proposed targetUse a dedicated CREATE TABLE context_blobs (
blob_id BLOB PRIMARY KEY, -- SHA-256
payload BLOB NOT NULL,
size_bytes INTEGER NOT NULL,
created_at INTEGER NOT NULL
);
CREATE TABLE context_refs (
ref_id TEXT PRIMARY KEY,
session_id TEXT NOT NULL,
owner_type TEXT NOT NULL, -- read_snapshot / tool_result_archive
owner_id TEXT NOT NULL, -- eventId or attachmentId
blob_id BLOB NOT NULL REFERENCES context_blobs(blob_id),
retention_class TEXT NOT NULL,
expires_at INTEGER,
created_at INTEGER NOT NULL,
UNIQUE(owner_type, owner_id)
);A put becomes one SQLite transaction:
SQLite WAL, The RuntimeEvent can commit its Reads remain bounded and authorized through This lets SQLite own the exact machinery we should not implement ourselves:
BoundaryThe material unknown is the payload size distribution, not the record count. We should measure p50/p95/p99/max and whether any context-offload producer requires streaming writes. Node If real payloads exceed that boundary, the credible fallback is hybrid storage: SQLite-inline context blobs below a threshold and an external immutable blob substrate only for proven large/streaming objects. I would not introduce that second durability domain before the distribution requires it. So my preferred framing is:
This is the same logical-ownership split you described, but it first asks whether we need to own a physical BlobStore implementation at all. |
Uh oh!
There was an error while loading. Please reload this page.
Context
ArtifactStoreis a Runtime Host authority spanning 27 files / ~7.5k lines (excluding tests), centered on a 1888-line class that owns a SQLite metadata table, a filesystem layout, a writer lock, a bootstrap lock, a purge-intent marker protocol, publication staging with crash recovery, cross-record referential integrity, IPC, and a Desktop preview registry.#4027 reports the symptom: cold-starting a Host against 11.7k artifacts blocks readiness for ~10 minutes. Before optimizing that path, I'd like to ask what the authority is for — because reading the producers, "Artifact" turns out not to name one need.
Four unrelated needs currently share it
1. Model-replay bytes.
createReadImageSnapshotterstores image bytes theReadtool must re-feed to the model on replay;tool_result_archivestores oversized tool results so pruned turns can be re-read. Both are load-bearing. But the archive's content isserializeToolResultForArchiveover a RuntimeEvent payload that is already durably stored incore_agent_run_events— the same bytes, written twice, kept forever. #3630 asks the narrower version of this question.2. An effect journal that contains no file.
session-effect-coordinatorwritessession-recap-intent.jsonandsession-recap-result.jsonas artifacts, then recovers by reading the result artifact back whencreatefails. That is idempotent-transaction bookkeeping. It wants a row next to the other coordinator state, not staging files and a purge-intent protocol.3. Write amplification with no reader.
deriveToolArtifactCandidatescopies the entire file of everyWrite, a synthesized diff of everyEdit, and the redirect target of everyBash > fileinto the artifact store. Greppingsource: 'tool_result'across the repo finds write sites only — no consumer reads these back, andartifact-visibility.tshides them from the pane.provider_request_captureis in the same class. These are duplicates of bytes that already exist in the user's workspace, retained indefinitely, that nothing will ever open.4. User-facing deliverables.
deep_research,subagent_writeback,user_upload. The only ones a human sees.Two supporting facts:
ArtifactSourcevalues have no producer at all:synthesis_cache_block,history_compact_block,history_compact_source,export,snapshot,fixture— and three of those six are marked user-visible.ARCHITECTURE.mdmentions artifacts once, inside the eval result kernel. There is no stated product role for the Artifact store as a Host authority.Proposed direction
Retire
ArtifactStoreas an authority rather than optimize it.If this holds, most of the recovery protocol in
artifact-store.tsdisappears rather than getting faster, and #4027 becomes unreachable instead of mitigated.Questions to settle
tool_resultderivations that I failed to find?This retires a Host authority, so it is material. If the direction holds here, I will take the decision to
dev@maka.apache.orgbefore any implementation issue is opened; #4027 stays open in the meantime as the user-visible bug.AI assistance disclosure: I used Maka to audit the artifact implementation — producer/consumer greps, source-visibility mapping, and the recovery and purge paths. I reviewed every claim above against the source on current
mainand own its accuracy.中文
背景
ArtifactStore是 Runtime Host 的一个 authority，横跨 27 个文件 / 约 7.5k 行（不含测试），核心是一个 1888 行的类，同时持有：SQLite 元数据表、文件系统布局、writer lock、bootstrap lock、purge-intent 标记文件协议、带崩溃恢复的 publication staging、跨记录引用完整性校验、IPC，以及 Desktop 预览注册表。#4027 记录了症状：对着 11.7k 个 artifact 冷启动 Host，会阻塞就绪约 10 分钟。在优化这条路径之前，我想先问这个 authority 究竟为何存在——因为把生产者读一遍会发现，"Artifact" 并不指代同一个需求。
四个互不相干的需求共用了它
1. 模型重放字节。
createReadImageSnapshotter保存Read工具在重放时必须重新喂给模型的图像字节；tool_result_archive保存超大工具结果，好让被裁剪的轮次可以再读回来。两者都是承重的。但归档的内容是对 RuntimeEvent payload 做serializeToolResultForArchive的产物，而该 payload 已经durable 地存在core_agent_run_events里——同样的字节写两遍，且永久保留。#3630 问的是这个问题的窄版本。2. 一份不含文件的效果事务日志。
session-effect-coordinator把session-recap-intent.json和session-recap-result.json当作 artifact 写入，并在create失败时回读结果 artifact 来恢复。这是幂等事务记账，它要的是与其他 coordinator 状态并列的一行记录，而不是 staging 文件和 purge-intent 协议。3. 无人读取的写放大。
deriveToolArtifactCandidates会把每次Write的整个文件、每次Edit合成出的 diff、以及每条Bash > file的重定向目标复制进 artifact store。在全仓库 grepsource: 'tool_result'只能找到写入点——没有任何消费者读回它们，artifact-visibility.ts也明确把它们从面板隐藏。provider_request_capture属于同一类。它们是用户工作区里本就存在的字节的副本，被无限期保留，且永远不会被打开。4. 面向用户的交付物。
deep_research、subagent_writeback、user_upload。只有这一类是人真正会看到的。两条支撑事实：
ArtifactSource中有 6 个完全没有生产者：synthesis_cache_block、history_compact_block、history_compact_source、export、snapshot、fixture——而这 6 个里有 3 个被标记为用户可见。ARCHITECTURE.md只提过一次 artifact，在 eval result kernel 里。Artifact store 作为 Host authority 的产品职责从未被记录。主张的方向
退役
ArtifactStore这个 authority，而不是优化它。若此方向成立，
artifact-store.ts里大部分恢复协议是被删掉而不是被加速，#4027 也随之变为不可达，而非被缓解。需要定夺的问题
tool_result派生物是否存在我没找到的消费者？这会退役一个 Host authority，属于 material 变更。若方向在此处成立，我会在开出任何实现 issue 之前把决策带去
dev@maka.apache.org；在此期间 #4027 作为用户可见的 bug 保持开启。AI 使用披露：我借助 Maka 审计了 artifact 的实现——生产者/消费者 grep、source 可见性映射，以及恢复与清理路径。以上每一条论断我都已对照当前
main的源码核验，并对其准确性负责。All reactions