Skip to content

fix(codex): support strict Turn result schemas - #7976

Merged
huangruiteng merged 1 commit into
loopx-project:mainfrom
zxdvd:codex/fix-codex-strict-output
Nov 3, 2026
Merged

huangruiteng merged 1 commit into
loopx-project:mainfrom
zxdvd:codex/fix-codex-strict-output

Conversation

@zxdvd

@zxdvd zxdvd commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Goal And Delivered Outcome

Closes #7975. Base: loopx-project/loopx:main.

Codex rejected the Turn result schema before execution because evidence properties were optional and profile requests added unsupported conditional keywords. Require nullable evidence on the Codex wire and remove the conditionals. Both CLI and managed app-server operation results now pass through one small Codex normalizer: stop null placeholders become internal absence, while profile material results cannot omit evidence or use null. Non-null evidence reaches the existing validator unchanged, including invalid/empty arrays. Legacy material results without a profile retain optional evidence.

Author Declaration

Criterion Disposition Implementation / evidence
Strict schema accepted with/without execution profile implemented Required properties, nullable evidence, no conditional schema; deterministic schema tests and native CLI probe.
Stop results remain non-material implemented Both Codex transports normalize only null evidence; real/non-null and malformed evidence still fail internal validation.
Preserve profile and legacy evidence rules implemented Canonical material/stop kind sets; profile null/missing refusal, existing array validation, legacy absence tests.

Self-check: reviewed the complete diff; reused the existing result vocabulary and executor validation. No shared executor or other host implementation changed.

Scope And Continuation

Complete within this repair. Python remains the existing Codex transport boundary; no new policy owner, module, configuration, scheduler or persisted format is introduced. The two native paths share the same conversion and prompt. The owning protocol and a self-repair pattern record the compatibility rule.

Two deterministic Codex fixtures previously omitted evidence despite carrying an execution profile. They now provide their synthetic artifact/check references; the marker fixture also reads back its actual file. Their initial 19 failures were repaired and rerun, without weakening validation.

Validation

  • Tested revision: ce3515ea3d41773609d9662c9f67db0c3d6f8545 (parent 1b37e2e6e0ea77b9bd405cf9a5ef5fcd8557e92e).
  • Run state: finished. Input classes: synthetic.
Check kind Result Public-safe evidence / limitation
regression_parity passed Four focused cases fail on the original code: required fields with/without profile, and app-server stop decoding under two deadlines. They pass after repair.
unit / integration passed pytest -q tests/test_loopx_turn_codex_cli.py tests/test_codex_operation_host.py tests/test_dsh_goal_mode.py tests/test_loopx_turn_executor.py tests/control_plane/test_reward_memory_host_isolation.py: 732 passed, 2 explicit live-model qualifications skipped. Covers all six result kinds, both evidence fields, profile/legacy inputs, invalid evidence, both transports and optional-capability isolation.
real_entrypoint passed Native Codex CLI 0.163.0, gpt-6-luna, effort high, isolated temporary home and empty read-only workspace: original schema rejected with invalid_json_schema; corrected schema accepted. The wait response passes JSON Schema and internal validation after normalization; zero tools called. Accepted request reports 18,433 input and 201 output tokens. The schema/prompt are unchanged at the tested revision. This is format acceptance, not a production Goal or material-outcome qualification.
integration passed Included deterministic public CLI smoke executes three Turns on one resumable fixture session, validates marker postconditions, settles and checks idempotent replay. File/SQLite optional-capability settlement cases also pass. No live model or production store is used by these fixtures.
static passed Changed-file Ruff; configured mypy (17 files); diff-scoped semantic advisory (no supported new vocabulary carriers); bilingual RFC/diff checks; public-boundary scan.
integration not_run Full repository suite and live app-server model qualification were not run locally.
integration blocked Hosted Python Tests, DCO and the other PR workflows report action_required for this fork contribution and await upstream approval. No hosted test pass or failure is claimed.

Coverage and gaps: native schema acceptance and both deterministic adapter boundaries are covered. No PostgreSQL or frontend behavior changed or is claimed. Review and merge remain with maintainers.

Frontend / Visual Evidence

UI impact: none. Before/after/first viewport/states/attention/craft: N/A; this repairs the existing Codex transport contract and changes no screen or control. Source data: synthetic fixtures only.

Type / Area / Direction

Bug fix, documentation and focused tests; host/runtime integration and core control-plane hardening. Shared-authority RFC fixture impact: N/A; no authority-provider migration or promotion.

Boundary Checklist

Signed-off-by: zxdvd <107175+zxdvd@users.noreply.github.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · gpt-6.1-sol · OpenAI · runtime_reported · reasoning_effort=max

精确评审 head:ce3515ea3d41773609d9662c9f67db0c3d6f8545;不可变差异基线:1b37e2e6e0ea77b9bd405cf9a5ef5fcd8557e92e。未发现当前改动的阻塞问题;独立结论为 APPROVE。

动机 / Motivation

通过 Codex CLI 或受管 app-server 执行一次工作、随后返回结果的 Agent。

旧版在模型开始工作前就可能因结果格式被拒绝;修复后格式满足严格 schema,停止结果也能通过内部校验,而缺少实际证据的材料结果仍被拒绝。

本轮独立验证了两个 Codex 传输入口的停止结果和恢复会话,并确认空数组、错误类型以及带执行配置的 null 材料证据不会被接受。

本评审不认证真实模型质量、安装态采用、长期净收益,也不执行合并或切换用户宿主。

改动思路 / Approach

严格格式转换属于 Codex 传输边界,两条入口复用一个小转换器;内部证据和结算校验继续由既有 executor 拥有。

当前切片覆盖 Codex CLI 与受管 app-server 的 schema、结果转换和失败校验,不改其他 Host 或持久化格式。

只改 schema 会让停止结果在内部继续失败,只改 CLI 会遗漏 app-server;把 nullable 语义放进共享 executor 又会改变其他 Host。当前边界保留一个 Codex converter,以 canonical result kind 集合分支,不从 prose 猜状态。最强反例是“格式接受了,但无证据的材料工作也被放行”;本轮同时验证这两个方向。沿用现有入口,无需新增配置或人工确认。

具体改动 / Changes

接受依据:https://github.com/loopx-project/loopx/issues/7975,spec_revision 2026-11-03T07:41:16Z。下列 criterion_id 是本次映射标签,不是新协议。另对照未被本 PR 重写的原 Turn 规范 1b37e2e6e0,及 OpenAI Structured Outputs 的 required、nullable 与不支持条件关键字规则;未根据更新后的文档反向制造验收条件。

  • STRICT-WIRE:implemented。Strict schema accepted with and without an execution profile.
  • STOP-ABSENCE:implemented。Stop results represent absent evidence without violating the internal contract.
  • MATERIAL-EVIDENCE:implemented。Profile material results require actual nonempty evidence.

关键代码讲解

  • codex_cli_result_schema:Optional/conditional evidence becomes required nullable; conditionals deleted. 输入和关键分支:Request with/without nonempty profile. Every property required, no allOf/if/then. 现有调用:codex_cli.py:855; codex_operation_host.py:217。
  • normalize_codex_result:Stops drop only null evidence; profile materials reject missing/null; other evidence preserved. 输入和关键分支:All six exposed result kinds and legacy/profile variants. Non-null invalid evidence cannot disappear. 现有调用:codex_cli.py:952; codex_operation_host.py:222。
  • run_codex_operation_host:Same converter now used after real session.send. 输入和关键分支:Authenticated synthetic app-server start_new/resume; deadlines None and5. No bypass of original launch/profile/operation authority. 现有调用:executor._run_host -> managed operation host。

完整差异9文件 +135/−70,生产2文件 +35/−28、测试/既有演练5文件 +90/−42、规范/按需修复知识2文件 +10。新增测试补真实 fixture 的证据和 marker readback,没有改变共享 executor。独立 owning 五模块732 PASS、2项显式 live qualification SKIP;Ruff、配置的17文件 mypy、diff advisory、完整 semantic 与9路径 public scan 均通过。

同一冻结 fixture 的50个 schema/真实 CLI 观察,head全部满足 oracle;基线39个完整观察逐字同义、11个明确反例失败。另在真正 synthetic app-server 子进程上,start_new→resume × deadline None/5 四个校验均通过,基线四个都因停止证据存在而被拒绝。模型只提供合成协议响应,未调用真实模型;作者的付费 schema-acceptance probe 保持作者来源,不冒称本轮独立重跑。

对主干的风险 / Risks

正常结果之外,profile 材料 null/missing 必须失败;stop 的非 null 数组(包括空数组)、布尔和材料错误类型也不能在转换中消失。完整参数测试及真实 CLI 对照覆盖此边界,其他 Host 的构造/验证来源未改;reward-memory 关闭路径隔离模块也通过。停止结果的空路径字段、原状态/扣额 authority 及 session/profile 拒绝保持原 owner。

标准 premerge:6项直接检查,8项 catalog 与8项 risk checks,没有阻断失败/人工 hold;其中一个 catalog 项保留 inherited maintainability advisory。原 ratchet FAIL 在基线/当前均输出相同393字节、空stderr,报错 render_canonical_todo_sections 的完整因果 source 未改;不是本 PR 引入的错误。严格质量回执 cqr_6e34d5fd6a56f9db93e6 仍为 FAIL,没有借本次批准清除或豁免;合并/质量门保持独立。

初次私有 app-server 探针误用了未定义的 start 动作,在两版都先于结果校验失败;只把 fixture 修正为已有 start_new,重跑四个受影响案例后得到上述差异。首轮 public scan 的默认路由/空临时registry调用失败保留,补全隔离registry后扫描通过。没有修改产品、降低断言或把调用失败重标为通过。

语义与 CI 对齐

复用既有 material/stop kind,不新增词汇或第二验证 owner。Prompt 的“stop省略”改成“wire null”,经实际转换仍表达内部无材料证据;material 明示 non-empty 和 legacy 规则。其他工作、必读、path delta、权限与结算句未缩短。diff advisory零candidate不代替语义判断。

本轮未查询、轮询或等待远端 CI;仓库全量测试、原生Windows、安装态/实际模型采用与持续完整成本未验。评审不把未知CI视为缺证,也不把本地通过写成远端CI通过。

我的整体评价 / Verdict

APPROVE 当前精确头。long_horizon=improved、user_experience=improved 仅限这条已验证的失败/恢复边界;普通既有任务零额外确认轮次,错误可读且能沿原 owner 继续。严格格式转换属于 Codex 传输边界,两条入口复用一个小转换器;内部证据和结算校验继续由既有 executor 拥有。 当前切片覆盖 Codex CLI 与受管 app-server 的 schema、结果转换和失败校验,不改其他 Host 或持久化格式。

已检查 CLI/受管结果入口及完整差异,没有渲染页面、首屏、布局或控件改动,视觉评审不适用。 视觉未验证范围:none;本次没有渲染表面变化。

已应用的有界简化是删除重复/错误的分支并复用当前 owner,不需要新的通用框架或仪式性后续任务。回忆经验中“先验证可用结果与恢复”的建议在这里得到局部执行;它既不追加父路线门禁,也不证明 memory utility 或长程净收益。没有当前P0–P2 finding;保留上述质量FAIL和未验边界,本结论不授予合并、安装或外发权限。

English verdict: APPROVE the exact head. Strict Codex wire evidence is normalized consistently in both real synthetic transports; required material evidence and invalid-candidate rejection remain enforced. Independent focused tests and base/head counterexamples pass at head. Raw inherited ratchet and strict quality FAIL remain separate merge holds; live model/adoption is unqualified.

@huangruiteng
huangruiteng merged commit 65bde47 into loopx-project:main Nov 3, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Codex Turn result schema is rejected by strict Structured Outputs

3 participants