fix(benchmark): keep native Goal alive through continuations

[huangruiteng]

Summary

• make the canonical native Codex Goal runtime wait through Codex-owned automatic continuation turns until the Goal leaves active
• allow an isolated runner's process working directory to differ from the task-visible Goal working directory
• route the runnable DeepSWE adapter through the terminal Goal API while keeping its no-model attachment preflight
• add state-machine, stdio-process, timeout, path-separation, receipt, and runnable-example coverage

Motivation

A benchmark adapter could observe one completed task turn and close its app-server session even though the attached Goal was still active. That loses the continuation work the Goal is meant to schedule. Keeping a second copy of this lifecycle in each runner also lets the real product path and benchmark treatment drift apart.

The benchmark toolkit now owns the complete Goal transaction: start exactly one initial task turn, observe its completion, read Goal status, and—only while the Goal remains active—drain the next automatic continuation under one total deadline.

Boundaries

The benchmark runner still owns process and filesystem isolation, environment policy, task bridges, control ledgers, verifier execution, and scoring. This change does not alter benchmark tasks, scoring, permission boundaries, upload/submission behavior, or launch a benchmark job.

Validation

• python -m pytest benchmark/tests/test_native_codex_goal.py tests/capabilities/test_benchmark_toolkit.py -q — 27 passed
• Ruff check and format check — passed
• real Codex 0.144.1 no-model Goal attachment preflight — passed; no turn/start
• loopx canary premerge --from-git-diff --goal-id loopx-deepswe-bench — 4 direct checks and 18 selected canary/risk/boundary checks passed with no failures or skips
• changed-file public/private boundary scan — clean

The canary reports the expected generic benchmark_sensitive manual hold. The repository owner explicitly authorized self-merge for this narrow helper/runtime repair; it does not change scoring, task semantics, permissions, or launch behavior.

[huangruiteng]

Pre-merge validation receipt:

• Changed surfaces: canonical native Goal runtime/API, compatibility exports, runnable DeepSWE adapter, focused tests, and benchmark-toolkit documentation.
• Direct checks: committed/staged/unstaged diff hygiene plus changed-Python compilation; 4/4 passed.
• Catalog canaries: 9/9 passed, including benchmark permission, candidate-source, artifact-path, evidence, and maintainability boundaries.
• Risk-profile smokes: 8/8 passed.
• Focused validation: 27 tests passed; Ruff check/format passed; live Codex 0.144.1 no-model Goal attachment passed without turn/start.
• Public/private boundary: changed-file scan passed; no credentials, private paths, internal labels, raw task content, trajectories, logs, or verifier output are included.
• Failures/skips: none.
• Manual hold: benchmark_sensitive, expected from path classification. Owner authorization is explicit. This is a narrow helper/runtime repair; it does not change task semantics, scoring, verifier behavior, permissions, upload/submission policy, or launch a benchmark job.
• Coverage rationale: the pure state machine proves repeated continuation and timeout semantics; the stdio-process fixture proves transport ordering and process/Goal cwd separation; the live preflight proves the real app-server attachment path. A model turn or benchmark job is intentionally outside this PR's validation boundary.

[huangruiteng]

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

精确评审头（Exact Head）: 3273@e4914fef987516fa633811dcb08d95182fe96841
Base: main | 类型: native Goal continuation 生命周期修复（7 files, +654）
作者: huangruiteng（owner）

---

详细中文评审

动机

上一轮 #3271 的 native Goal runtime 只跑一个初始 task turn：adapter 可能看到一次完成就关闭 app-server，丢掉 Goal 继续调度的自动 continuation 工作。本 PR 让已安装 runtime 拥有完整 Goal 事务——直到 Goal 离开 active 才停。

改动思路

新增 run_native_goal_process_until_terminal / run_native_goal_until_terminal：启动恰好一个初始 turn → 观察其完成事件 → 读 Goal 状态 → 只要仍 active 就继续 drain 下一条自动 continuation，受单一总 deadline（超时 fail-closed goal_timeout_before_terminal）。同时允许 runner 的进程 process_cwd 与 Goal 可见的 cwd 分离（隔离 runner 控制目录 vs 任务可见目录）。Runnable DeepSWE adapter 改走 terminal Goal API，保留 no-model 预检。

具体改动

• loopx/capabilities/benchmark_toolkit/native_codex_goal.py：continuation drain + terminal 判定 + process_cwd 分离 + receipt 增加 post_goal_status/goal_status_poll_count。
• benchmark/native_codex_goal.py：facade 同步导出新入口。
• benchmark/deepswe/run_native_codex_goal.py：goal_until_terminal 模式 + 预检保留。
• 测试：状态机等待、active-forever fail-closed、真实 stdio 进程 cwd 分离、真实 stdio terminal 等待。

关键内容讲解

1. 完整生命周期：turn/start 只发生一次；后续 continuation 由 Goal 状态驱动 drain，避免 adapter 各自复制生命周期。
2. fail-closed 超时：总 deadline 到期仍未 terminal 即抛 goal_timeout_before_terminal，不静默关闭。
3. cwd 分离：process_cwd 用于隔离控制目录，Goal 线程仍用任务可见 cwd，两者解耦且有真实 stdio 测试覆盖。

对主干的风险

无阻断项。benchmark helper/runtime seam：不改变任务、评分、权限、上传/提交行为，不启动 benchmark job；runner 仍拥有隔离/环境/verifier/评分。CI：Sign-off、dependency-review SUCCESS；pytest IN_PROGRESS；build×2/deploy 显示 CANCELLED（force-push 前的旧 run，非失败，合并前确认新 run 绿）。

验证矩阵（exact head 实测/CI）

场景	结果
pytest benchmark/tests/test_native_codex_goal.py tests/capabilities/test_benchmark_toolkit.py	27 passed
作者 no-model Goal attachment preflight（Codex 0.144.1）	PASS
canary premerge（作者，goal-id loopx-deepswe-bench）	4 direct + 18 selected 全过
Sign-off / dependency-review	SUCCESS
pytest（CI）	IN_PROGRESS（评审时刻）
build ×2 / deploy	CANCELLED（旧 run，非失败）

我的整体评价

方向正确：把“Goal 生命周期”收进已安装单实现，adapter 不再自己决定何时关闭会话；超时 fail-closed、cwd 分离与真实 stdio 测试都到位。结论：APPROVE（author-owned PR，以 COMMENTED 记录批准结论；合并前确认 CI pytest 绿且 build/deploy 有非取消的新 run）。

---

English Verdict

Verdict: APPROVE (author-owned PR; recorded as COMMENTED because GitHub blocks formal self-approval). The installed native Goal runtime now owns the complete transaction: one initial turn, then draining Codex-owned automatic continuations under a single deadline until the Goal leaves active, failing closed on timeout. process_cwd is decoupled from the task-visible Goal cwd, and the runnable DeepSWE adapter uses the terminal API while keeping the no-model preflight. 27 focused tests pass at the exact head; Sign-off and dependency-review are green, pytest was in progress, and the cancelled build/deploy runs are stale pre-force-push artifacts to be re-confirmed before merge.