Skip to content

feat(todos): wire caller-approved validation_command into todo completion - #3142

Merged
huangruiteng merged 1 commit into
huangruiteng:mainfrom
NIU-123370:feat/todo-completion-validation
Aug 14, 2026
Merged

feat(todos): wire caller-approved validation_command into todo completion#3142
huangruiteng merged 1 commit into
huangruiteng:mainfrom
NIU-123370:feat/todo-completion-validation

Conversation

@NIU-123370

Copy link
Copy Markdown
Contributor

Implements #3082 along the shape @huangruiteng sketched: reuse the existing acceptance-loop validation handler (lifted into a shared module) rather than adding a new validator, and gate durable writeback on a typed receipt.

What this does

  • A todo may declare a caller-approved validation_command (+ optional validation_label) at loopx todo add time, stored on the todo metadata.
  • complete_goal_todo runs that command before the durable writeback commits. Pass → completion commits as usual; fail → returns ok=False with a typed validation receipt and leaves the state unchanged. Because complete_task keys quota spend off the completion ok marker, a blocked completion also blocks the spend for free.
  • Todos without a declared command keep the current fast path unchanged (no global forcing).

How it maps to the agreed shape

  • Handler reuse, not a new validator: the _run_caller_validation / _require_passed pair from capabilities/issue_fix/acceptance_loop.py is lifted verbatim into control_plane/runtime/validation_command.py — privacy invariant preserved (stdout/stderr/local_path_captured stay False; schema stays issue_fix_validation_command_v0). acceptance_loop.py now imports them from there (capability → control_plane; correct layering).
  • Validation runs outside the mutation lock: the declared command is pre-read and run before exclusive_file_lock(..., operation="todo_complete") is acquired, so a multi-second validation does not block concurrent todo operations on the same goal (the MUTATION lock deadline is 5s). Skipped on dry_run and on terminal replay.
  • Timeout under the outer budget: inner validation timeout is 20s, below the 30s outer CLI/MCP subprocess budget, so a timed-out command still yields a typed receipt.
  • No new settlement adapter — formal SettlementStep composition and the journaled replay-reuse fixture are deferred to land with fix(turn): fence remote execution and terminal writeback #3074, per "no new generic validator or framework is needed… a data field plus a caller for an existing step."

Two decisions I'd like your call on (happy to adjust)

  1. Command form: stored validation_command as a plain string (e.g. pytest -q tests/test_x.py), fed to the existing shlex-based handler. Your run-once precedent uses --validation-command-json (JSON argv). String was simpler for the metadata schema and matches the handler directly; switching to JSON argv is a small change if you prefer it.
  2. Receipt schema: reused issue_fix_validation_command_v0 (the handler's native schema). Easy to switch to loopx_turn_task_validation_v0 if you want a single shared id.

Tests

tests/control_plane/test_todo_completion_validation.py — 7 cases: positive (with an execution spy so it fails if the gate is silently skipped), negative (fail → blocked, state unchanged, no spend), no-command parity, timeout, terminal-replay (no re-run), missing executable, malformed command. The typed failure is also surfaced in render_todo_markdown (a Validation section with command_label / passed / status / exit_code / summary). Maintainability ratchet baseline for loopx/todos.py bumped 2142 → 2318 to register the intentional module growth.

Heads-up: 9 pre-existing failures on main (not from this PR)

While verifying I found 9 failing tests on upstream main that are unrelated to this change — a --required-read-id drift between production (agent_scoped_evidence_log.py / should_run_packet.py append --required-read-id <id> to the evidence-log required-read command) and the replan/portfolio test fixtures (which assert the command without that flag). I confirmed they fail identically on the base commit with my changes excluded. Happy to file a separate issue if useful.

Non-goals (deferred)

Refs #3082.

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

详细中文评审

精确评审头: 3142@a56fd4512cde6e2f206e215fd8da8e83ba78d121

动机

本 PR 实现 #3082(P0):把“caller-approved validation_command”接入 todo completion——完成前运行调用方声明的验证命令,验证未通过则不提交写回、不产生完成状态、不消耗 quota。这是把“done 必须被验证”从口号变成机器门禁的核心改动。

改动思路

复用 issue_fix acceptance-loop 的验证执行器(提升到 control_plane/runtime/validation_command.py,行为逐字保留、wire schema issue_fix_validation_command_v0 不变),在 complete_goal_todo 的 mutation 锁之前预读并运行验证:声明了命令的 todo 走“验证门禁”,未声明的 todo 保持原快速路径。验证失败返回 ok=False + typed receipt + validation_blocked_completion=True,状态不变,spend 也随之被挡住。

具体改动(关键代码讲解)

  • run_caller_validation(validation_command.py)shlex.split(无 shell)、cwd=workspace、只记录 exit_code/passed,stdout/stderr/本地路径均不采集;超时抛 TimeoutExpired 由调用方转 typed failure receipt。隐私边界保持。
  • _read_declared_validation / _run_declared_completion_validation(todos.py):命令只在 todo add 时写入、无 update 路径(不可漂移);workspace 缺失/超时/命令无法启动/格式错误全部转 passed=False 的 typed receipt,而不是让异常破坏完成路径。
  • 门禁位置:验证在 exclusive_file_lock 之前运行(避免多秒 subprocess 占用 5s mutation 锁);complete_goal_todopassed is not True 时直接返回,未触碰状态文件。dry-run 与已完成的终端重放跳过验证。
  • 契约与展示contract.py 增加 validation_command/validation_label 元数据字段;markdown.py 对被拦截的完成渲染 ## Validation 段(仅公开安全字段)。
  • acceptance_loop 重构:本地 _run_caller_validation/_require_passed 删除并改为从 control_plane 导入,行为逐字一致(capability → control_plane 分层正确)。

对主干的风险

风险低且方向正确:默认快速路径不变(无全局强制);新字段可选;验证失败 fail closed(不提交、不 spend)。两点 P2(非阻塞):(1) todos.py 增至 2318 行(+178),建议后续把“读取→运行→门禁”编排收敛进 bounded 模块;(2) canary baseline 的 todos.py 行数已同步更新(2142→2318),与 diff 一致,无作弊。

我的整体评价

APPROVE。 scope fit 成立(complete_goal_todo 是真实生产调用点);typed receipt 与 spend 阻断闭环完整;测试覆盖正/负/超时/命令缺失/快速路径;隐私边界保持。这是对 #3082 的一次扎实落地。


English Verdict

APPROVE — exact head a56fd4512cde6e2f206e215fd8da8e83ba78d121.

Implements #3082: caller-approved validation_command runs before todo-completion writeback, gating state changes and quota spend on a typed receipt; the no-command fast path is unchanged. Validation executes outside the mutation lock, failure paths are typed and fail-closed, the issue_fix handler is reused verbatim with a shared wire schema, and the canary baseline matches the diff. Verified: 7 new focused tests, 51 todo regression tests, turn fake-host smoke, and py_compile all pass. P2 only: consider moving the orchestration out of the now-2318-line todos.py in a follow-up. No blockers.

huangruiteng added a commit that referenced this pull request Aug 12, 2026
Security hardening release: fixes GHSA-2225/GHSA-c42j/GHSA-hfmf/
GHSA-vx2m/GHSA-p7c9 via #3137-#3140, caller-approved completion
validation (#3142), and the durable-smoke review gate (#3134).
…tion

When a todo declares a caller-approved `validation_command` (set at
`todo add` time), `complete_goal_todo` now runs it independently and
requires a passing receipt before the durable writeback commits; on
failure it returns ok=False with a typed validation receipt and the
state is left unchanged (the MCP complete_task quota spend, which keys
off the ok marker, is blocked for free). Todos without a declared
command keep the current fast path unchanged.

Implements huangruiteng#3082 per the maintainer's shape: reuse the existing
acceptance_loop handler (lifted to a shared
control_plane.runtime.validation_command module) rather than adding a
new validator, and model the result as a typed receipt.

Design notes (for review):
- validation_command / validation_label are stored on the todo metadata
  as plain string fields (JSON-argv form is a documented follow-up).
- The validation subprocess runs BEFORE the exclusive mutation lock is
  acquired (pre-read of the declared command), so a slow command does
  not block concurrent todo operations on the same goal; it is skipped
  on dry_run and on terminal replay.
- Inner validation timeout is 20s, kept under the 30s outer CLI/MCP
  subprocess budget so a timed-out command still yields a typed receipt.
- No new settlement adapter: formal SettlementStep composition and the
  journaled replay-reuse fixture are deferred to land with huangruiteng#3074.
- Maintainability ratchet baseline for loopx/todos.py bumped 2142 -> 2318
  to register the intentional module growth from this feature.

Tests: positive (with execution spy), negative (fail -> blocked),
no-command parity, timeout, terminal-replay (no re-run), missing
executable, and malformed command.

Refs huangruiteng#3082.
@NIU-123370
NIU-123370 force-pushed the feat/todo-completion-validation branch from a56fd45 to f371a0a Compare August 13, 2026 07:28

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

详细中文评审(exact head f371a0a

动机

#3082 要求 caller-approved 验证进入 todo 完成路径:todo add 声明验证命令,完成时独立执行,失败则阻断提交。此前 issue_fix 已有私有实现;本 head 把执行器抽到 control_plane/runtime/validation_command.py 复用,并在 todo complete 前跑验证,失败返回 typed 阻断结果而不落库。

改动思路

validation_command.py 集中 run_caller_validation/require_validation_passedshlex.split 无 shell、仅记录 exit_code/passed、不捕获 stdout/stderr/本地路径,保持 issue_fix 原有 wire schema issue_fix_validation_command_v0)。todos.pycomplete_goal_todo获取 state 锁之前预读 todo 声明的 validation 并执行(20s 超时,外层 30s 预算内),dry_run 与已完成的 replay 跳过;失败/超时/无 workspace/命令缺失/畸形命令都生成 passed=False typed receipt 并返回 validation_blocked_completion: True,不提交。

具体改动

  • loopx/control_plane/runtime/validation_command.py(新,58 行):共享 runner + require 助手。
  • loopx/capabilities/issue_fix/acceptance_loop.py:删除私有实现,改为导入共享 runner(行为与 wire 兼容)。
  • loopx/cli_commands/todo.py / todo_argument_validation.pytodo add 增加 --validation-command/--validation-label
  • loopx/control_plane/todos/contract.pymarkdown.py:metadata 字段与 Validation 渲染。
  • loopx/todos.py(+178):_read_declared_validation(锁外只读预读)、_resolve_goal_repo_workspace_run_declared_completion_validation(六种失败都 typed 化)、complete_goal_todo 集成。
  • loopx/canary/module_metric_baseline.json:todos.py 2318 行基线更新。
  • tests/control_plane/test_todo_completion_validation.py(新,277 行):覆盖成功/失败/超时/无 workspace/畸形命令/replay/dry_run。

对主干的风险

执行模型正确:验证在锁外跑,避免 5s mutation 锁被 subprocess 占用;无 shell 注入面(shlex.split+argv);隐私安全(不落 stdout/stderr/路径)。信任模型:--validation-command 由操作者声明,在 goal repo 内执行——这是有意授权,但 P2 建议明确该选项只对可信 CLI/operator 开放(若未来暴露给 MCP/untrusted caller 需加门禁);超时 20s 与 issue_fix 的调用语义一致。验证:test_todo_completion_validation.py 7/7、test_cli_output_budget.py 15/15、diff-check 干净。

我的整体评价

比旧 head 更干净的实现:复用、锁外执行、六类失败全部 typed 化、replay/dry_run 语义正确。当前结论:APPROVE。

English verdict

APPROVE (exact head f371a0a). Caller-approved validation is now a shared privacy-safe runner wired into todo completion: declared at add, executed before the state lock, and any failure (non-zero, timeout, missing workspace, unlaunchable, malformed) blocks completion with a typed receipt; dry-run and terminal replay skip validation. 7/7 new validation tests and 15/15 CLI budget tests pass; diff-check clean. P2: document/restrict that --validation-command is operator-only. No blockers.

@NIU-123370

Copy link
Copy Markdown
Contributor Author

已 rebase 到最新 main(head f371a0ac)。此前失败的 replan_evidence / model_behavior 那批测试现已通过;当前 CI 剩余的 ratchet 失败指向 loopx/state_refresh.py 的 module_metric 债务,#3142 并未改动该文件,应该是 main 上 fix(refresh-state) 带入的,供参考。

liubf21 pushed a commit to liubf21/loopx that referenced this pull request Aug 14, 2026
Security hardening release: fixes GHSA-2225/GHSA-c42j/GHSA-hfmf/
GHSA-vx2m/GHSA-p7c9 via huangruiteng#3137-huangruiteng#3140, caller-approved completion
validation (huangruiteng#3142), and the durable-smoke review gate (huangruiteng#3134).
@huangruiteng
huangruiteng merged commit 8b8e7b2 into huangruiteng:main Aug 14, 2026
4 of 5 checks passed
huangruiteng pushed a commit that referenced this pull request Aug 15, 2026
…ounded module

Addresses the P2 follow-up from #3142's review: move the caller-approved
completion-validation orchestration (read declared command -> run -> gate)
out of the 2318-line loopx/todos.py and into a new bounded module,
loopx/control_plane/todos/completion_validation.py.

complete_goal_todo now calls a single run_completion_validation_gate(...)
before acquiring the mutation lock; the gate, the pre-read, the runner, and
the typed-failure receipt construction all live in the new module. No
behavior change — this is a pure move (the privacy invariant, the
outside-the-lock placement, the dry_run/terminal-replay skip, and the 20s
timeout are all preserved).

Effect: loopx/todos.py 2318 -> 2167 lines; canary ratchet baseline for
loopx/todos.py ratcheted back 2318 -> 2167 accordingly. The new module
follows its control_plane/todos/* siblings (not individually budgeted).

Stacked on #3142 (branch based on feat/todo-completion-validation); rebase
onto main and open as a standalone follow-up PR once #3142 merges.
@huangruiteng

Copy link
Copy Markdown
Owner

这个 feature 挺好用,我今天跑 benchmark,loopx 自动用上它了
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants