Repository navigation
RFC-0048: 录制 schema 3 与传输回放 - #325
Conversation
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
对照 RFC-0044(#279,
Q1、Q2、Q3 按待讨论保留原提案,这里不替维护者下结论。 Generated by Claude Code |
…position Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
三处都已改,见最新提交。
Generated by Claude Code |
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Q1 的补充事实按 li 的要求挪到了 issue:#270 (comment) 。RFC 正文的 Q1 恢复为开 PR 时的写法;上一条评论表格里 rebuild_provider 和“草稿”两处修正保留。 Generated by Claude Code |
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
更正上一条:RFC 还开着,新发现直接进正文。Q1 的补充事实已放回 §11 Q1( Generated by Claude Code |
eric8810
left a comment
There was a problem hiding this comment.
Review
方向正确,但当前文本还不可实现:
- 阻塞 Fetch 层缺少模型调用身份:
DiagnosticScope只有call_id,Exchange却要(seq, attempt, parent);重试和 MoA 子调用无法在叶子 Fetch 匹配。 - 重要
modelRef用"<key>:<modelId>"再按第一个:拆,和 registry 的可配置 separator 冲突,也丢了 namespace/method;建议直接用结构化引用。 - 重要(安全) 响应分块原样落盘,但 RFC-0041 §5 要求在进入 writer 前做资格判定/脱敏;流式错误体里的凭证可能被持久化。
- 重要 单次 exchange 上限 8 MiB 原始字节,JSONL 单行上限也是 8 MiB,base64 后必然超限;writer 合法的记录读不回来。
- 重要 input 脱敏后仍标
live=true,live 回放会发送[REDACTED]。 - 重要
abandoned映射为可重试的FetchError::Io,回放可能意外重试,改变录制的 attempt 序列。 - 重要 缺少 RFC-0041 要求的 terminal/phase 校验;迁移步骤没列全 schema-2 消费方(
MockDecisionReplayModel、Web traces),且replay.create在 step 4 才引入的ReplaySet上被提前使用。
DiagnosticScope now carries (seq, attempt, parent), the per-attempt exchange counter and observers, so §3, §4.1, §8 and Q3 cite it instead of restating it. Provider tests that replay through ReplaySession move to ops integration tests, since providers and devtools do not depend on each other (RFC-0044 §8.1, RFC-0046 §5.9). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
c494403:按 RFC-0044 §4.2(adf6c618)与 RFC-0046 §5.9(fe5f2f07)更新。§3、§4.1、§8、Q3 的 (seq, attempt, parent) 与交换序号改为引用 |
eric8810
left a comment
There was a problem hiding this comment.
评审:RFC-0048 录制 schema 3 与传输回放 —— 批准(附 4 个待决问题的建议)
设计核验通过的关键点:
- 与 RFC-0041 的分工干净:格式约束/匹配语义/离线鉴权/错误码不重设计,本文只定接口、切片与顺序;schema 3 完整形状与 §4.1 逐字段吻合(Captured 联合、exchangeId 三元组、offsetUs 原点、replayability 写时算读时重算、metadata/payload 策略、8 MiB/4096 上限)。
- "回放就是一个 Fetch" 是正确解法:修复了 mock 只认 OpenAI chat 的根因(跳过 provider 解析器),与上游测试在 Fetch 层喂预制响应(msw)的做法同构;Exact/Sequential 的因果消费序((seq,attempt) 约束)正确处理重试与 MoA 并发;ReplayAmbiguous 同响应取最早、header 排除集合合理;429/5xx 作为普通响应走 provider 错误解析而非引擎错误——与 RFC-0041 §8.3 一致。
- live 回放改 modelRef(删除 provider() 截断猜测)正确关闭"重建目标靠猜"问题;ops 改动表完整(replay.offline 关 G-k、capabilities Future→Current 关 G-c、recorder.create 策略化)。
- Q1 的分析质量高:http.rs:375 与 Fetch 观察点两种位置的签名头不一致分析(SigV4 的 x-amz-date/x-amz-content-sha256)是决定性论据——观察点方案使录制与回放看到同一份请求。§4.1 按 RFC-0044 写是对的。
- 实施计划诚实:3→4 之间 master 无 mock 回放的缺口如实登记且给出合并选项;每个 PR 只做一个方向。
维护者对四个待决问题的建议(按 §11):
- Q1:采用 RFC-0044 的 Fetch 观察点(即本文 §4.1 现状),删除原提案。理由如上第 4 点。
- Q2:按 §10 排序(先建 aimux-devtools 再写 schema 3),不允许临时位置再搬。
- Q4:接受 (b)——确定性一次性转换脚本不进仓库、在 PR 里列出供审核。转换是公开的、符合 RFC-0041 §2"不偷偷转存"的要求;脱敏头缺值在 §5.2 中已排除出比较,可回放。
- Q5:保持 3、4 分开。缺口在两个未发布提交之间,不构成用户可见回归;换来两个可评审的中型 PR。
两个补充要求(随实施 PR 处理,不阻塞本 RFC):
- PR 3 需明确 schema-2 测试 fixture 的去向清单(aimux-core/tests/fixtures/master_*_recordings.jsonl 等 4 个 jsonl),不能只处理 Jev(Q4)。
- "outcome 不包 Captured"是对 RFC-0041 §4.1 字面的简化,落地时在 RFC-0041 加一行修订引用(与此前 §0.1.7/§7.1 修订同惯例)。
批准。
|
§11 四个待决问题的决定(接上条评审,作为维护者侧结论):
另请实施 PR 3 时附上 schema-2 测试 fixture 的去向清单(master_*_recordings.jsonl 等),并在落地时于 RFC-0041 补"outcome 不包 Captured"的修订行。 |
…schema-3-and-transport-replay
…review §11 becomes the decided questions (Q1 observer, Q2 devtools first, Q4 one-off conversion, Q5 separate PRs). The 10-20 review items land in the text: modelRef is the structured ops ModelRef; response chunks are judged before the writer queue; one line limit shared by writer and reader with a base64 budget; sensitive input is forbidden, so live is off; abandoned replays as non-retryable ReplayBeyondRecording; reading validates modelCalls, exchange ids, terminals and offsets; ReplaySet moves to step 3; step 3 lists every schema-2 consumer and fixture. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5j9hYv7hDeYgMvvRu2F9T
|
10-20 评审的七条
10-21 的决定与两个补充要求
这次改动都在正文范围内,没有新的待定项,请再看一眼。 Generated by Claude Code |
eric8810
left a comment
There was a problem hiding this comment.
复审后仍有阻塞项:1) terminal.error.message / outcome.error 仍是字符串,未走 RFC-0041 §5 的脱敏,provider body/prompt 可能落盘;2) SSE 判定把 OpenAI 的 data: [DONE] 当 unparsed,所有 OpenAI-compatible 流会不可回放;应把空 data、[DONE]、注释/event/id 当控制行;3) offline provider 默认取 modelRef.key,但 registry alias 不一定等于 model.provider(),应取 target.model.provider 或要求显式 provider;4) schema-2 迁移仍漏 jev_test.rs 的 jev_systemone_live.jsonl;5) 与 alignment doc 的字符串 modelRef/ per-call model 字段冲突未登记;6) MoA/router 子模型没有 offline 构造路径;7) prepare/auxiliary exchange 可能阻塞 model exchange 匹配;8) 录下的 connect/timeout/io 被假定会重试,最后一次 attempt 不成立。请修后再复审。
Error text follows the payload rules: outcome keeps name and code, errorDetail and decisionResult are Captured and redacted, terminal messages are policy-gated. SSE control lines ([DONE], comments, event, id, retry, empty data) are not parsed. The offline package comes from an exact table of each package's provider ids, never the registry alias. Each model call records its model; replay.session and replay.offlineModel let hosts compose router and MoA offline. Only model exchanges are ordered. Retries follow the recorded options, including the last attempt. The Jev fixture's three readers and the alignment doc differences are listed. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5j9hYv7hDeYgMvvRu2F9T
|
10-25 复审的八条在
Generated by Claude Code |
eric8810
left a comment
There was a problem hiding this comment.
新 head f74d6a4 仍有三类阻塞:1) reader 要求 complete=true 时 status ∈ success/error/cancelled,但 writer 会合法写 incomplete+complete=true,ReplaySet::load 会拒绝合法录制;2) Exact 仍忽略 name-only header、比较敏感 query、对 multipart 做字节比较,和 RFC-0041 §7 不符;3) terminal.error.message 仍只做 URL 清理,未走 RFC-0041 §5 的 body/secret 策略。另有 deny_unknown_fields 与“后续加可选字段”冲突、Captured 缺少 not-applicable、ops-only error code/modelRef DTO 泄漏到 devtools、G-c 过度声明。请修后再复审。
- complete describes record completeness: any outcome status may pair with complete=true; reader no longer rejects legal incomplete records - Exact follows RFC-0041 §7: per-package ExchangePolicy header sets, name-only non-auth headers make Exact unavailable, auth query dropped on both sides, multipart parsed, lossless JSON numbers - error text follows RFC-0041 §5: errorDetail captured only when every field maps to captured sources; transport error stores kind only - schema 3 field set closes at first release (deny_unknown_fields); WebSocket support bumps the schema - Captured gains not-applicable; devtools owns RecordedModelRef and drops the ops error code; G-c stays open for credential.create Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5j9hYv7hDeYgMvvRu2F9T
…schema-3-and-transport-replay
|
回复 @eric8810 10-25 08:29Z 的评审,改动在
另外核对了 Q4:Jev 的请求头只有 Generated by Claude Code |
eric8810
left a comment
There was a problem hiding this comment.
仍有阻塞:reader 仍无条件要求 exchange terminal/timing,但 writer 合法写 complete=false 的 unfinished exchange;Jev 转换缺 transport-added content-type,Exact 会 miss;metadata 与 not-applicable 优先级冲突;replayability.mock 未覆盖 replay.offline 需要 captured input。请修后再复审。
- terminal and terminalOffsetUs are required only when complete=true; complete=false records may carry unfinished exchanges - Captured state precedence: not-applicable before capture policy - mock means transport replay (RFC-0041 §4.3); replay.offline needs a captured input or an explicit one, reported as operation: reasons - Jev conversion restores the leaf headers build_fetch_request adds (content-type, user-agent); step 4 verifies it with an Exact replay Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G5j9hYv7hDeYgMvvRu2F9T
|
@eric8810 你 11:06Z 那次评审 的四条已经改好,在
Generated by Claude Code |
eric8810
left a comment
There was a problem hiding this comment.
上一轮四个阻塞点都已修:complete=false 允许 unfinished、Jev Q4 恢复 content-type/user-agent、metadata 优先级排序、replay.offline 的 input。剩余均为实现前置/措辞 nit。Approve。
Requested by li
Summary
本 RFC 是 RFC-0041 G1–G3 的落地设计,接替 #270:录制只写 schema 3,在传输叶子记录原始字节与分块;回放是一个
Fetch,离线组装的 provider 照常编码和解析,每个 provider 都能离线回放;删除 schema 1/2 读取、ProviderRecord、MockReplayModel、rebuild_provider。crate 归属以 RFC-0044(#279)为准,本文不定。下面先是录制与回放子系统的现状,再是本 RFC 设计的改动。录制与回放子系统现状
行号是 master
b91829af。上游没有录制格式:
@ai-sdk/devtools在模型调用层用 middleware 和 telemetry 记步骤(devtools/src/middleware.ts:101、integration.ts:113);上游测试用createTestServer在fetch处喂响应(test-server/src/create-test-server.ts:112),provider 照常编码解析。本 RFC 的改动
RFC 还写了:schema 3 的完整形状与读取规则(§3)、匹配与响应回放规则(§5)、离线组装与
ExchangePolicy登记(§6)、ops 每个 op 的改动和错误码、绑定与工具(§7)、与 RFC-0041 / RFC-0044 的对照(§8)、本期不做的项与登记差异(§9)、实施计划与 schema 2 fixture 去向(§10),以及维护者已定的五个问题(§11)。How
只新增
rfc/0048-recording-schema-3-and-transport-replay.md,不改代码。🤖 Generated with Claude Code
https://claude.ai/code/session_01G5j9hYv7hDeYgMvvRu2F9T