Skip to content

fix(chat): preserve terminal worker errors when control reads fail - #7903

Merged
loopx-agent merged 1 commit into
mainfrom
codex/chat-terminal-failure-20261102
Nov 2, 2026
Merged

loopx-agent merged 1 commit into
mainfrom
codex/chat-terminal-failure-20261102

Conversation

@loopx-agent

Copy link
Copy Markdown
Collaborator

An upstream worker error can be lost when the failure handler makes another control-plane eligibility read and that read fails. The queue worker exits with the original Turn still active, preventing later independent requests from running.

Record terminal failures through the existing status CAS and Turn-bound claim release. Keep current eligibility checks around provider cancellation and retry; an unavailable retry read refuses replay while preserving the original error.

Validation: 13 focused regression cases, 473 tests in the changed agent module, and 259 tests in six related queue, steering, recovery and Antigravity suites. The initial broader run exposed six overly strict timestamp comparisons in the new fixtures; the final module run includes the corrected assertions. Baseline reproduction fails the three unavailable-control failure cases. Ruff, compilation, packaged Chat build and exact-head premerge checks pass.

This prevents the demonstrated failure path. It does not replay previously failed work or certify recovery of a running older service; installation retains the existing runtime activation guard.

Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent | GPT-6 | OpenAI | self_reported (host execution observation unavailable)

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

动机 / Motivation

用户在原会话连续发送请求时,上游连接失败恰逢控制面读取不可用。 修复前原错误没有落盘,下一条请求停在队列;修复后保留原失败,下一条独立请求继续执行。 后续请求已持久化却无法执行,用户反复追问仍被同一占用挡住。原请求只执行一次,错误可独立读回;后续请求完成,停止和已完成状态保持。 本PR不重放旧执行,不改变账号、授权或既有安装重启门禁。 运行旧版本时已经遗留的活动占用及其原业务尚需受控恢复;真实安装与全部历史消息送达未由本PR证明。

改动思路 / Approach

没有阻塞性发现。最强反对理由是删掉执行资格检查可能让迟到错误覆盖停止或已完成结果;这里复用了既有状态比较写入(CAS),只允许 starting/running 进入失败终态,并只释放相同 Turn 的占用。真正可能产生上游效果的取消、重试仍要检查当前资格;无法核验时不会取消或重放。失败落盘属于现有生命周期保证,应由原状态CAS决定;取消与重试仍由当前执行资格和原生效果证明决定。 只修复已复现的失败收尾与后续独立队列推进;存量遗留执行的恢复、安装采用和业务补答仍留在原恢复任务。 对照增加外层通用 catch、第二个守护线程或第二份状态,现有 _fail_turn 已能完成这项责任,删除冗余读取更直接。

具体改动 / Changes

关键代码讲解

  1. _run_started_turn 的三种错误分支改为直接交给原失败写入 owner。超时路径仍在 try 内先确认执行资格才调用 interrupt,检查失败也保留原超时;正常成功路径不变。
  2. _prepare_transport_retry 将既有 typed control transport/startup 读取异常解释为拒绝自动重试,继续保留第一错误。重试次数、退避、原生无工具证明及原会话绑定均不变。
  3. _fail_turn 的持久化实现不变:状态 CAS 先落盘,再写原错误事件、历史并精确释放占用。本次说明为何这一步不应再依赖一轮远程资格查询。新增13个回归场景覆盖三类失败、控制面可用/不可用、迟到停止/完成和不可用重试读。

规格依据为 docs/architecture/rfcs/app-conversation-and-async-inbox-v0.md,spec_revision fb16c6cb1369efce52a673030c914b38e1dac166(固定文本)。下列标签用于映射该文段落,并非新协议:C-FIFO 对应原 Session FIFO 后续请求,公开 enqueue_turn 对照通过;C-OWNER 对应单消费者及现有权威,两个请求各执行一次且无新增执行者;C-STOP 对应精确 Turn 停止边界,六个迟到错误负例保留既有停止/完成结果。三项均 implemented。主干推进到6e947c79期间,相关 runtime、store、typed admission 与规格内容没有变化。

对主干的风险 / Risks

PASS:同一公开 enqueue_turn 脚本、真实文件存储/Core 与异步队列,对照基线 fb16c6cb1369efce52a673030c914b38e1dac166 和当前 head。健康三场景两侧均通过;额外资格读取失效时,基线三场景为 running/queued 且原错误缺失,当前三场景均保存 failed/timed_out、后续 completed、原请求执行一次。fixture SHA256 791f2688904911b0cf1e7bf1b607d1ba2bd00f8e13e1aa6807ad8139959ad5fe。只模拟 provider 与故障读取,未模拟终态或队列推进后置条件。

PASS:变更模块473项、六个相邻 owner 模块259项,Ruff、编译、打包 Chat、精确提交 premerge 的6个 direct及2个 selected检查。初次组合测试的六个失败是新负例过度比较活动时钟,原记录保留;只调整这些时钟断言后,完整变更模块473项通过,状态、答案、消息、事件和占用断言保留。浏览器对真实打包 HTTP/Core 读回一次原失败和一次后续答案,活动占用为空。CI按当前 Goal规则不查询、不等待。Change-quality 当前 policy disabled,没有伪造启用回执。

已有停止或已完成状态由 CAS 拒绝覆盖;错误收尾不授予账号、受众或重放权限。主要剩余风险是运行旧版本已遗留的活动占用,以及文件写入自身故障的既有恢复边界;本PR不证明生产旧实例恢复或全部历史消息送达,也不绕过安装门禁。回滚为这两个文件的单一提交。

我的整体评价 / Verdict

这是 justified_increment:失败收尾属于现有生命周期保证,应由原状态CAS决定;取消与重试仍由当前执行资格和原生效果证明决定。long_horizon improved:已复现的原错误不会再被一次辅助读取失败替换,后续独立工作继续。user_experience improved:失败可读回,后续消息不增加确认或重填资料。loopx/chat_runtime.py未修改渲染、布局或浏览器资源,visual_review not_applicable;未验证范围为生产旧实例恢复和真实消息补办,没有把合成HTTP回读冒称生产界面全量验收。净生产代码减少一行,无新版本分支或迁移,复用现有语义词汇和单一队列 owner。当前 head 为 aedd4c14c72e213de3d092f1ad19a3b02b2fd57e;部署与存量业务验收保留为独立后续责任。

English verdict: APPROVE aedd4c1. No blocking finding; terminal errors and independent FIFO continuation verified. Live legacy recovery and backlog delivery remain outside this qualification.

@loopx-agent
loopx-agent merged commit d77aeb8 into main Nov 2, 2026
4 of 6 checks passed
@loopx-agent
loopx-agent deleted the codex/chat-terminal-failure-20261102 branch November 2, 2026 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant