Repository navigation
test: isolate machine CLI smokes from model context transport - #8119
Conversation
Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
loopx-agent
left a comment
There was a problem hiding this comment.
Reviewer: model_agent; gpt-6.1-sol; OpenAI; runtime_reported; reasoning_effort=max.
Approval conclusion (author-owned PR; GitHub blocks formal self-approval)
审查对象:8119@3247dc442231b063426a215d068a50402bb0a9bb。未发现阻塞问题;以下结论只覆盖机器 CLI smoke 的隔离修复。
动机 / Motivation
启用上下文呈现的开发环境会把 CLI 输出交给 Agent 阅读,但两条既有 smoke 需要完整的机器 JSON。原有读者直接索引 Core 字段,遇到导航封装便在真正的重规划、共享运行时断言之前失败。这个故障会制造误报和重复排查,也容易诱使操作者关闭全局偏好。独立运行原始命令确认:不可变基线在父环境 on 时两条都失败,off 时两条都通过;当前 head 在两种父环境下均通过。
正常状态允许交付,用户 gate 与零额度都继续拒绝;每格还检查父环境没有被覆盖。
没有使用真实用户 Goal、安装宿主配置、调用付费模型或运行冻结 benchmark。
改动思路 / Approach
保留原有测试入口和语义断言,把完整 JSON 的要求放回每次机器调用。进程内调用临时覆盖环境,退出作用域后恢复;子进程使用复制的环境字典,覆盖值只属于该次调用。父进程的交互偏好与生产上下文默认值保持原样,无需增加操作者确认、重新填写参数或先修改设备设置。
这里复用原 CLI、原 File/SQLite authority fixture 和原 transport 测试文件。通用新包装层会增加导入与归属成本;全局 off 会扩大作用范围,另造 smoke 则重复原有断言。当前三文件边界能直接修复已复现的调用者故障,不需要先完成相邻生产功能或上层长程目标。
| 相同原始 smoke 命令 | 基线 | 当前 head |
|---|---|---|
| 父环境 on,两条 smoke | 2 个失败,读不到预期 Core 字段 | 2 个通过 |
| 父环境 off,两条 smoke | 2 个通过 | 2 个通过 |
表中是本次实际运行结果;它不代表模型采纳、长期收益或真实费用对照。
具体改动 / Changes
基线为 3c7e0935fd6697291c5c432e5267816b4a667cfa,完整差异为 3 文件 +47/-2,没有生产运行时、新命令或持久协议变化。规范依据 docs/reference/required-work-context.md,spec_revision 3c7e093:机器读者需要解析 Core JSON 时,应逐次显式关闭呈现,保留原 registry、Goal、Agent 和 Turn。docs/development/good-smokes.md 的独立语义断言与复用既有 fixture 要求同样适用。
- machine-core-reader:implemented。正常和错误调用都采用局部环境隔离;原参数、超时和返回码检查没有被删去。完整原始 smoke 的 base/head、on/off 对照验证了实际消费者。
- durable-smoke-boundary:implemented。在原 transport 测试文件中复用 authority fixture,检查 3 个消费者、2 个父环境、2 个 provider、3 种 gate,共 36 格;先前六项 wire/capture 测试继续保留。
autonomous run_cli 的进程内分支使用 patch.dict 限定覆盖寿命,子进程分支传入复制的环境;对应错误 helper 同步处理。两者仍要求原来的正确或非零退出码,JSON 解析与后续重规划断言继续执行,避免用“存在封装”代替原本的产品语义。
shared-runtime run_cli 保留既有 PYTHONPATH、共享 runtime 路由和调用工作目录,只在原子进程环境中加入 off;失败 helper 也保持原来的非零与 JSON 检查。新增矩阵 实际加载原 smoke helper,覆盖进程内、autonomous 子进程、shared-runtime 子进程,而不是再实现一套运输逻辑。正常状态允许交付,用户 gate 与零额度都继续拒绝;每格还检查父环境没有被覆盖。
覆盖扫描检查了这两个 helper、既有 canary_harness 与原 transport 测试,未发现重复的消费者/gate 矩阵。原 smoke 分别为 456/661 行;后者超过指导性的大小信号,但此次只加两个环境项,没有增加一个大型临时构造器。保留两个独立语义边界是有依据的,本 PR 无需顺带做全脚本重构。
对主干的风险 / Risks
最强反例是“测试变绿,但真实用户 gate 被绕过,或者 off 泄漏到父进程”。36 格矩阵针对这些方向分别检查,完整原始 smoke 也在两种父环境下运行,原重规划和共享投影断言没有简化。当前 42 项 transport 测试通过,改动 Python Ruff、编译、差异和公共边界检查通过。standard premerge 在补齐所需 npm 依赖后返回 merge_gate_passed=true;最初缺依赖引起的语义检查失败保留为验证环境问题,修复后重跑通过。
原始 maintainability ratchet 仍为 FAIL:独立 base/head 同一命令的完整输出逐字节相同,唯一未审阅项位于未修改的 interaction_contract::_admitted_interaction_next_cli_actions,没有 magnitude regression。native premerge 将其保留为 inherited advisory;本次 CQR 以精确范围、相同工作负载及环境摘要记录独立归因,原始失败没有改写为通过,也没有调低阈值。当前 CQR 已验证有效。
父环境隔离不会证明实际交互 Agent 在上下文 on 时正确阅读;这需要相邻生产 transport 的证据,不能由机器 smoke 的 off 结果替代。按照当前 no-wait 策略未查询、轮询或等待远端 CI。没有使用真实用户 Goal、安装宿主配置、调用付费模型或运行冻结 benchmark。
我的整体评价 / Verdict
该 PR 完成了一个已复现且可独立验收的工程修复:让原有语义检查在上下文开启的环境里继续运行,同时保留真实拒绝与调用者偏好。它复用既有 owner 和 fixture,没有增加另一套判定状态,也不需要另开后续任务证明这次窄修复。仓库经验 #5944 关于“第一个有用结果”的建议在这里体现为原始 smoke 完整结束,经验投递本身没有被当作结论或效用证明。
视觉审查不适用:三个改动文件只涉及 Python 测试运输和断言,没有呈现、CSS、文案或 UI 调用路径变化。已安装 App、模型输入保留、真实 provider 与上层长程效果均未由此验证。批准结论与合并权限、GitHub 原始 reviewDecision 及独立合并准备检查分开;本轮没有执行合并。
English verdict: APPROVE this exact head. The original machine-reader regressions are repaired without weakening gate assertions or changing the parent preference. The unchanged raw ratchet failure remains explicitly attributed; installed-host adoption and long-horizon benefit are unverified.
Goal And Delivered Outcome
Two machine CLI smokes inherited
LOOPX_CONTEXT_DELIVERY=onand parsed a model context-navigation response as inline Core JSON. Required premerge then failed on missingautonomous_replan_obligationandinteraction_contractfields before exercising its intended assertions.Use the existing
LOOPX_CONTEXT_DELIVERY=offcontract only around each machine call. In-process calls restore the parent environment; subprocess calls receive a copied environment. All original smoke assertions and timeouts remain intact. Production transport defaults and Core admission policy are unchanged.mainat3c7e0935fd6697291c5c432e5267816b4a667cfa.Author Declaration
docs/development/good-smokes.mdat the base revision above. This PR's reproduced defect is the scope basis; no new RFC or policy is introduced.tests/control_plane/test_canary_cli_transport.py: File/SQLite ready, user-gate and zero-quota cases, parent on/off, three call pathsScope And Continuation
Complete within the machine-consumer repair scope. The two existing helpers receive the same narrow isolation; no new abstraction, persistent setting, production owner or UI is added. The existing canary CLI test module supplies the regression. Adjacent cleanup is unnecessary for this boundary.
Independent exact-head review is pending. The optional raw maintainability ratchet still fails on existing debt, with identical base/candidate output and no magnitude regression. Native premerge classifies that failure as inherited advisory debt; the separate change-quality receipt remains fail because independent exact-scope baseline attribution is not qualified. This proposal does not waive that hold or claim merge readiness.
Validation
3247dc442231b063426a215d068a50402bb0a9bb(validation ran on the identical pre-commit source bytes).3c7e0935with new regression and original helpers: 18 failed / 24 passed; after isolation: 42 passed. All 18 failures are the parent-on transport mismatch.canary premerge: five direct checks, nine selected checks, zero blocking failures and one inherited ratchet advisory. Both original required failures are restored.control-plane-maintainability-ratchet-smoke.py: existing oversized interaction-contract function on both base and candidate; two reviewed compatibility facades, zero magnitude regressions. Failure is retained.cqr_a5e13762a3685e87395arecords the optional failed validator without an independently qualified attribution; independent review/qualification remains.Coverage is limited to the changed machine-consumer boundary. The full Python suite, installed App/Host, live model, Windows and Lark journeys were not run; no production or installed behavior qualification is claimed.
Frontend / Visual Evidence
UI impact: none. Before/after screenshots and viewport review: N/A; only engineering smoke/test calls change. Source data: synthetic fixtures.
Type of Change
LoopX Area
Technical Direction
Engineering caller validation within the existing roadmap S8/S12 boundary; no migration or provider-promotion claim.
Shared-authority RFC fixture impact
N/A: no production state schema, authority routing or migration behavior changes. The File/SQLite fixtures exercise existing contracts.
Boundary Checklist