Enforce deterministic Python 3.12+ runtime and adoption readiness (#710)

[jeffhuber]

Resolves #710 (child of #708, part of epic #709).

Summary of Changes

1. Merged origin/main:

   • Merged origin/main (including Release campaign: preserve retry chronology and upload terminal failure evidence #711 campaign retry chronology and terminal failure evidence upload) into current branch without rebasing.
   • Preserved both Release campaign: deterministic supported runtime and truthful adapter readiness #710 deterministic runtime/readiness behavior and Release campaign: preserve retry chronology and upload terminal failure evidence #711 retry chronology/evidence behavior.

2. Addressed Codex Audit Finding (codex:10ca06eef8f192c3a8b3):

   • check_campaign_auth_readiness in src/code_mower/doctor_checks/campaign_auth.py evaluates campaign_auth_probe_requested() before the Antigravity and Muse ambient-home inspection branches.
   • When CODE_MOWER_CAMPAIGN_AUTH_PROBE=0, both providers immediately return status="skip" (state="skipped"), bypassing missing-opt-in warnings and preserving readiness.
   • Added focused regression tests in tests/test_doctor_campaign_auth.py and tests/test_doctor_campaign_readiness.py verifying that disabled probing skips ambient checks, emits no provider warnings, and keeps providers in ready_providers.

3. Addressed Claude Audit Finding (claude:478292eabccf944c5cbd):

   • On the audited head, _detect_runtime_class was already imported from release_qualify at the top of src/code_mower/release_campaigns.py across both package execution modes.
   • Preserved top-of-file imports and original location of resolve_supported_runtime in src/code_mower/release_campaigns.py without code duplication or relocation.
   • Added direct regression test test_build_adapter_argv_default_and_empty_target_runtime_resolves_bounded_runtime in tests/test_release_campaigns.py verifying that calling _build_adapter_argv with target_runtime omitted or empty string evaluates the imported deterministic fallback rather than raising NameError.

4. Deterministic Python 3.12+ Runtime Resolution & Fail-Closed Behavior:

   • Added resolve_supported_runtime(*, environ, which_fn, runner) in src/code_mower/release_campaigns.py to probe CODE_MOWER_PYTHON, running interpreter sys.executable, and versioned candidates (python3.12, python3.13, python3.14, python3) for a Python 3.12+ runtime, resolving relative paths to absolute paths.
   • Updated _invoke_local_adapter to resolve the runtime deterministically before invoking maintained adapters and pass exact --python-bin {python} --target-runtime {target_runtime} arguments, ensuring providers never fall back to ambient unvalidated python3.
   • Maintained adapter template in src/code_mower/provider_registry.py updated to include --python-bin and --target-runtime.
   • Registered python_runtime_unavailable as a bounded safe error code mapped to unavailable state with actionable remediation.

5. Result Schema Enforcement:

   • Enforced runtime_class >= python_3.12 in validate_adoption_result_payload in src/code_mower/release_qualify.py.

6. Offline Structured-Result Capability Probe:

   • Implemented check_structured_result_capability(provider) in src/code_mower/campaign_adapters.py using a bounded in-memory fixture (zero token spend, zero network) to verify provider result extraction and schema validation.
   • Updated doctor.campaign.readiness detail to distinguish command (executable readiness), auth (auth readiness), and structured_result (structured-result capability) per provider without leaking output or paths.

7. Verification Evidence:

   • Focused unit tests:
     • tests/test_doctor_campaign_auth.py: 19 tests OK
     • tests/test_doctor_campaign_readiness.py: 35 tests OK
     • tests/test_release_campaigns.py: 324 tests OK
   • Full test suite: 1445 tests OK (0 failures, 0 errors)
   • Code formatting and linter: ruff check . passed (clean)
   • Privacy scan: scripts/privacy_scan.py passed (clean)
   • Bytecode compilation: python -m compileall src tests passed (clean)
   • Git diff hygiene: git diff --check clean (0 errors)

[jeffhuber]

Codex audit (merge-authority lane)

Head SHA: 939b26d32d3f9c71d078944f5ecb5ba144c0937f
Findings: P0=0, P1=0, P2=2, P3=0 (blocker policy: any P0/P1/P2 → BLOCKED)

Codex Audit: BLOCKED

Summary:

Two P2 findings are present: the readiness check can falsely report Muse authentication for unusable key files, and unknown runtimes can bypass the Python 3.12 minimum requirement.

Findings:

• [P2] Validate the Muse key file before reporting auth ready -- /private/var/folders/f2/g9zqfdjx7z3ckcsx_f46htwm0000gn/T/codex-audit-trqv5e08/wt/src/code_mower/doctor_checks/campaign_auth.py:289
  Finding ID: codex:13f56468483aa8a8009f
  When META_API_KEY_FILE names a missing, unreadable, empty, or invalid file, this check reports Muse as authenticated and includes it in ready_providers, but the adapter's resolve_muse_api_key() reads and parses the file and will then fail for missing authentication. Validate the referenced file using the same resolution logic, rather than treating any non-empty path as a key.
• [P2] Reject unknown runtimes under the new minimum requirement -- /private/var/folders/f2/g9zqfdjx7z3ckcsx_f46htwm0000gn/T/codex-audit-trqv5e08/wt/src/code_mower/release_qualify.py:443
  Finding ID: codex:f629ee3b49e9e3cff040
  For any hosted, manual, or custom-adapter result that reports runtime_class: "unknown", this branch skips the newly added Python 3.12 minimum entirely, so an unqualified result is still accepted despite the runtime-readiness contract. Require a concrete python_<major>.<minor> value before applying the minimum-version check.

[jeffhuber]

Claude audit (merge-authority lane)

Head SHA: 939b26d32d3f9c71d078944f5ecb5ba144c0937f
Findings: P0=0, P1=1, P2=1, P3=0 (blocker policy: any P0/P1/P2 -> BLOCKED)

Claude Audit: BLOCKED

Summary:

Found a real regression in the Claude adapter result-extraction fallback that undermines the campaign system's core anti-fabrication guarantee, plus a lesser observability gap in the new readiness aggregation. Everything else (deterministic Python 3.12+ runtime gating, runtime_class >= 3.12 schema enforcement, new antigravity/muse auth checks) is well-tested and internally consistent.

Findings:

• [P1] _extract_claude_result no longer rejects is_error:true Claude envelopes -- src/code_mower/campaign_adapters.py:668
  Finding ID: claude:5810e3d615e079c0331d
  Previously _extract_claude_result explicitly returned None when the parsed envelope was not a dict or had is_error is True (if not isinstance(envelope, dict) or envelope.get("is_error") is True: return None), which is the documented fail-closed behavior for a Claude run that exited 0 but internally errored (budget exceeded, refusal, tool failure, etc.). The rewritten version only special-cases the successful branch (if isinstance(envelope, Mapping) and envelope.get("is_error") is not True: ... ) and, whenever that condition is false — including the is_error: true case — falls through unconditionally to return code_mower_gemini_cli.parse_response_json(stdout), parsing the entire raw envelope string (not the discarded/error path) for a JSON candidate. This means a Claude CLI invocation that completed with exit code 0 but reported is_error: true can still produce a non-null 'candidate' if the raw stdout (e.g. an error transcript containing a fenced JSON block in the result text) happens to contain something parse_response_json can extract. validate_bound_result still enforces schema/identity, narrowing but not eliminating the exposure — this directly weakens the 'Never fabricated' invariant this same PR's docs describe at length (a provider must only reach complete when its adapter 'actually ran' and produced a genuine result). This fallback appears to be an unintended side effect of adding fenced-JSON support for the new offline check_structured_result_capability fixture (which only needs the json.loads failure path, not the is_error bypass). Recommend restoring an explicit early return when isinstance(envelope, Mapping) and envelope.get('is_error') is True.
• [P2] Providers can silently drop out of ready_providers with no actionable/warn signal -- src/code_mower/doctor_checks/adoption.py:1071
  Finding ID: claude:c7cd448185560490ba7c
ready_providers is now additionally gated on provider_readiness.get(check.lane, {}).get('structured_result') is True, but when check_structured_result_capability() returns False for an otherwise-PASS provider (adapter installed, command found, argv configured, auth authenticated), no corresponding WARN/actionable DoctorCheck is emitted to explain why. Every other reason a provider is excluded from readiness (missing command, missing config, missing auth, missing credentials, missing repo slug) produces an explicit WARN check with actionable/remediation fields feeding actionable_providers/optional_providers. A structured_result failure instead makes the provider vanish from ready_providers while remaining absent from both actionable_providers and optional_providers, and the failure is discoverable only by reading the nested provider_readiness dict inside doctor.campaign.readiness.detail. This degrades the observability contract this exact function otherwise upholds everywhere else and could mask a real extraction/schema bug for a provider as an unexplained readiness gap.

[github-actions]

Code Mower notice: previously audited head 939b26d32d3f is no longer an ancestor of current head f51bdc02e929; commits may have been dropped. Only current-head audit verdicts count.

[jeffhuber]

Codex audit (merge-authority lane)

Head SHA: f51bdc02e9293df26168065ac48794a87c2ebada
Findings: P0=0, P1=0, P2=1, P3=0 (blocker policy: any P0/P1/P2 → BLOCKED)

Codex Audit: BLOCKED

Summary:

The runtime override is incorrectly reused as the interpreter launching Code Mower's adapter module, which can cause local campaigns to fail immediately.

Findings:

• [P2] Keep the adapter launcher on the installed interpreter -- /private/var/folders/f2/g9zqfdjx7z3ckcsx_f46htwm0000gn/T/codex-audit-erbbzf17/wt/src/code_mower/release_campaigns.py:1289
  Finding ID: codex:154f5bc434dca5f88a7f
  When CODE_MOWER_PYTHON points to a supported interpreter other than the one running Code Mower, this substitution also changes the maintained adapter's leading {python} -m code_mower.campaign_adapters. That interpreter may not have code_mower installed, so the documented override causes every local campaign to exit before qualification. Keep {python} bound to sys.executable and use the separate {target_python} placeholder for --python-bin.

[jeffhuber]

Claude audit (merge-authority lane)

Head SHA: f51bdc02e9293df26168065ac48794a87c2ebada
Findings: P0=0, P1=0, P2=1, P3=1 (blocker policy: any P0/P1/P2 -> BLOCKED)

Claude Audit: BLOCKED

Summary:

The PR adds deterministic Python 3.12+ runtime resolution for campaign adapters plus a new offline "structured-result capability" doctor probe. Most of the change (runtime resolution, schema tightening, muse/antigravity auth checks) is well tested and self-consistent. One correctness bug was found in the new structured-result capability check: it silently exercises the wrong extraction path for the Claude lane, undermining the check's stated purpose.

Findings:

• [P2] check_structured_result_capability never exercises Claude's real extraction path -- src/code_mower/campaign_adapters.py:597
  Finding ID: claude:a1b18f48030d39c2a7d5
  The function branches on canonical in {"claude", "claude_review", "claude_code"} to route the offline fixture through _extract_claude_result. But the real canonical provider id produced by resolve_provider_lane/PROVIDER_ALIAS_MAP for the local-CLI Claude lane is claude_audit (see provider_registry.py, lane_id="claude_audit"), which is never in that set. claude_review is a manual-driver lane that never reaches this code path at all, and claude_code is not a real canonical id anywhere. As a result, every real invocation from check_adoption_campaign_readiness (which calls check_structured_result_capability(canonical) with canonical="claude_audit") falls through to the generic code_mower_gemini_cli.parse_response_json(payload_str) branch instead of _extract_claude_result. The doctor.campaign.readiness/structured_result check therefore always reports structured_result: True for claude by validating a code path that Claude's real adapter output never goes through (envelope parsing, is_error handling, structured_output/result field extraction). If a future regression broke _extract_claude_result (e.g. the is_error short-circuit), this doctor check would keep passing and give false confidence that the merge-authority Claude lane is campaign-ready, defeating the stated purpose of the new check ('verify that Code Mower can correctly parse and validate adoption results from this provider'). Fix by matching on the actual canonical id (claude_audit) rather than claude/claude_review/claude_code.
• [P3] Runtime-unavailable remediation branch keyed on a duplicated free-text string -- src/code_mower/release_campaigns.py:1247
  Finding ID: claude:148adc31e933836afabc
_provider_next_action matches error == "supported Python 3.12+ runtime is unavailable" to select the Python-specific remediation message, duplicating the exact string literal produced independently in _invoke_local_adapter's python_runtime_unavailable branch. If either string is edited without the other, the tailored remediation silently falls back to the generic 'configure prerequisites...' message instead of failing a test loudly. Consider keying off the python_runtime_unavailable error code (already a registered SAFE_ERROR_CODES value) instead of a duplicated detail string.

[jeffhuber]

Claude audit (merge-authority lane)

Head SHA: 523a09aefec0ed97c5aed583fa620f32630aa8b5
Findings: P0=0, P1=0, P2=0, P3=1 (blocker policy: any P0/P1/P2 -> BLOCKED)

Claude Audit: PASS

Summary:

PR adds deterministic Python 3.12+ runtime resolution for local campaign adapters, structured-result capability probing, and Antigravity/Muse auth-readiness checks in the doctor adoption flow, plus tightened runtime_class validation in release_qualify. Reviewed the fail-closed runtime resolution (resolve_supported_runtime), the readiness/ready_providers aggregation logic, the Claude/Muse/Antigravity result-extraction refactors, and the new tests, all of which are internally consistent and match the documentation updates. No correctness, security, or data-loss blockers found.

Findings:

• [P3] Unverified helper reference _detect_runtime_class -- src/code_mower/release_campaigns.py:1330
  Finding ID: claude:d79114591a174511553c
  _build_adapter_argv now calls release_campaigns._detect_runtime_class() as a fallback when target_runtime is empty, and a new test (test_build_adapter_argv_default_and_empty_target_runtime_resolves_bounded_runtime) calls it directly. This function's definition is not visible anywhere in the diff. In the real dispatch path (_invoke_local_adapter) target_runtime is always populated by resolve_supported_runtime before _build_adapter_argv is called, so this fallback is unlikely to be hit in production, but if the helper does not already exist elsewhere in the module it would break direct/test callers with a NameError. Worth a quick confirmation that this helper pre-exists in the base tree (not part of this diff) before merge; not blocking since normal execution paths don't reach it and any omission would be caught by the new unit test in CI.

[github-actions]

Code Mower notice: previously audited head 939b26d32d3f is no longer an ancestor of current head 225ba4f10f39; commits may have been dropped. Only current-head audit verdicts count.

[github-actions]

Code Mower notice: previously audited head 939b26d32d3f is no longer an ancestor of current head 225ba4f10f39; commits may have been dropped. Only current-head audit verdicts count.

[jeffhuber]

Codex audit (merge-authority lane)

Head SHA: 225ba4f10f39583dee4e220594cf68d27c7f6e74
Findings: P0=0, P1=1, P2=0, P3=0 (blocker policy: any P0/P1/P2 → BLOCKED)

Codex Audit: BLOCKED

Summary:

The Antigravity and Muse authentication readiness checks compare aliases instead of canonical provider IDs, making them unreachable and allowing unauthenticated providers to appear ready.

Findings:

• [P1] Match auth checks against canonical provider IDs -- /private/var/folders/f2/g9zqfdjx7z3ckcsx_f46htwm0000gn/T/codex-audit-d56t4ixr/wt/src/code_mower/doctor_checks/campaign_auth.py:257
  Finding ID: codex:627f9fca38fd6f40c8be
  When invoked through check_adoption_campaign_readiness, canonical is antigravity_cli or muse_cli, so comparisons against antigravity and muse never match. Because these lanes have no command-based auth probe, the function returns None, allowing both providers to appear campaign-ready without their required ambient-home opt-in or Muse API key; use the registry's canonical identifiers or normalize aliases for these branches.

[jeffhuber]

Audit disposition for Codex finding codex:627f9fca38fd6f40c8be

The reported alias mismatch is not present. resolve_provider_lane returns lane.provider as the canonical value: antigravity_cli resolves to antigravity, and muse_cli resolves to muse. Those are the values checked by check_campaign_auth_readiness.

Head 9b09804 adds an integration regression that invokes doctor with the lane-ID aliases themselves and proves both provider-specific auth checks are reached, both missing-auth states warn, and neither provider enters ready_providers. The focused test passes for both subtests. No production behavior was changed for this disposition.

[github-actions]

Code Mower notice: previously audited head 939b26d32d3f is no longer an ancestor of current head 9b098049aee5; commits may have been dropped. Only current-head audit verdicts count.

[github-actions]

Code Mower notice: previously audited head 939b26d32d3f is no longer an ancestor of current head 9b098049aee5; commits may have been dropped. Only current-head audit verdicts count.

[gitar-bot]

Code Review 👍 Approved with suggestions 2 resolved / 3 findings

Enforces deterministic Python 3.12+ runtime resolution with fail-closed behavior when unavailable, validates adoption result schemas, models ambient-home provider auth readiness, and adds offline structured-result capability probing. All audit findings have been addressed.

Note: list_discovered_campaigns blanks the entire Board when any campaign id collides across directories, hiding all healthy unambiguous campaigns. If this fail-closed-everything behavior is unintended, consider projecting non-colliding campaigns while surfacing collisions as a warning instead.

💡 Edge Case: Board hides all healthy campaigns when one id collides

📄 src/code_mower/release_campaigns.py:4142-4156

list_discovered_campaigns returns non-colliding campaigns in campaigns and colliding ids separately, but when collisions is truthy the board returns the empty payload (0 campaigns) regardless of how many healthy, unambiguous campaigns exist. A single duplicated campaign id across two discovered directories therefore blanks the entire Board (showing only "resolve ambiguous campaigns") and hides every unrelated in-flight campaign. If this fail-closed-everything behavior is intended, it matches the docs; otherwise consider projecting the non-colliding campaigns while surfacing the collision as a warning next_action instead of discarding them.

✅ 2 resolved

✅ Bug: Offline capability probe uses clock-sensitive hardcoded timestamp

📄 src/code_mower/campaign_adapters.py:610-624 📄 src/code_mower/release_qualify.py:463-471 📄 src/code_mower/doctor_checks/adoption.py:1111-1119
check_structured_result_capability() builds a fixture with execution_state="executed" and a hardcoded timestamp_utc of "2026-09-04T08:00:00Z", then calls validate_adoption_result_payload(), which for executed results runs _validate_executed_timestamp_bounds() and rejects any timestamp more than 300s in the future (release_qualify.py:366-370). On any host whose clock is before 2026-09-04T08:05:00Z the probe raises, the except returns False, and because ready_providers now requires structured_result is True (adoption.py:1112-1119), every otherwise-ready provider is silently dropped from ready_providers. Even setting aside clock skew, pinning the fixture to a near-present calendar date makes an offline self-test time-dependent. Generate the timestamp dynamically relative to now (a small past delta) so the probe is deterministic regardless of host clock.

✅ Quality: structured_result probe marks owner_action even for optional/unenabled lanes

📄 src/code_mower/doctor_checks/adoption.py:794-808 📄 src/code_mower/doctor_checks/adoption.py:934-948
The two new doctor.campaign.structured_result WARN blocks hardcode actionable=True, optional=False, owner_action=True regardless of is_enabled, unlike the sibling adapter/credentials checks which set actionable=is_enabled and optional=not is_enabled. Because check_structured_result_capability is an offline probe of Code Mower's OWN parsing/validation code (not provider/owner state), a failure indicates a Code Mower regression, so labeling it owner_action=True and forcing an otherwise-optional provider out of campaign-ready is slightly misleading. Consider aligning actionable/optional/owner_action with is_enabled and rewording the remediation as developer-facing, and factoring the duplicated block into a helper.

🤖 Prompt for agents

Code Review: Enforces deterministic Python 3.12+ runtime resolution with fail-closed behavior when unavailable, validates adoption result schemas, models ambient-home provider auth readiness, and adds offline structured-result capability probing. All audit findings have been addressed.
  
  Note: `list_discovered_campaigns` blanks the entire Board when any campaign id collides across directories, hiding all healthy unambiguous campaigns. If this fail-closed-everything behavior is unintended, consider projecting non-colliding campaigns while surfacing collisions as a warning instead.

1. 💡 Edge Case: Board hides all healthy campaigns when one id collides
   Files: src/code_mower/release_campaigns.py:4142-4156

   `list_discovered_campaigns` returns non-colliding campaigns in `campaigns` and colliding ids separately, but when `collisions` is truthy the board returns the `empty` payload (0 campaigns) regardless of how many healthy, unambiguous campaigns exist. A single duplicated campaign id across two discovered directories therefore blanks the entire Board (showing only "resolve ambiguous campaigns") and hides every unrelated in-flight campaign. If this fail-closed-everything behavior is intended, it matches the docs; otherwise consider projecting the non-colliding `campaigns` while surfacing the collision as a warning next_action instead of discarding them.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Auto-apply	Compact
gitar auto-apply:on	gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Gitar}

[jeffhuber]

Claude audit (merge-authority lane)

Head SHA: 225ba4f10f39583dee4e220594cf68d27c7f6e74

Head SHA changed during review (225ba4f1 -> 9b098049). Skipping this verdict and requeuing for re-review of the new head.

[jeffhuber]

Codex audit (merge-authority lane)

Head SHA: 9b098049aee5251ee8dc4dbabf7fcade4b06a7aa
Findings: P0=0, P1=0, P2=0, P3=0 (blocker policy: any P0/P1/P2 → BLOCKED)

Codex Audit: PASS

Summary:

No discrete, actionable regressions were identified in the changed code.

Findings: none.

[jeffhuber]

Claude audit (merge-authority lane)

Head SHA: 9b098049aee5251ee8dc4dbabf7fcade4b06a7aa
Findings: P0=0, P1=0, P2=0, P3=2 (blocker policy: any P0/P1/P2 -> BLOCKED)

Claude Audit: PASS

Summary:

Reviewed the deterministic Python 3.12+ runtime enforcement and adoption-readiness PR. The changes (runtime resolution/binding, structured-result capability probing, Antigravity/Muse auth checks) are extensively tested and internally consistent with the updated docs. No P0/P1/P2 correctness, security, or data-loss blockers were found. Two non-blocking observations are noted for awareness.

Findings:

• [P3] Global runtime_class>=3.12 requirement not reflected in hosted-provider dispatch instructions -- docs/release-qualification.md:140
  Finding ID: claude:cf4b7154df40fb3fde26
  validate_adoption_result_payload now rejects runtime_class == 'unknown' and requires python_. >= 3.12 for every adoption result, including hosted_bridge/saas_event (Devin, Cursor BugBot) and manual --record-result submissions. The doc updates in this PR only explain the {target_python}/{target_runtime} binding for the four maintained local-CLI adapters; the Devin/Cursor BugBot setup sections and dispatch-comment instructions are unchanged and give those hosted agents no guidance that they must now report an exact Python 3.12+ runtime_class instead of 'unknown'. Confirm hosted providers can reliably satisfy this before relying on it, or extend their dispatch instructions accordingly.
• [P3] Unconditional Python 3.12+ resolution gate applies even to fully custom adapter overrides -- src/code_mower/release_campaigns.py:1670
  Finding ID: claude:d136152884a829abc09d
  _invoke_local_adapter() now calls resolve_supported_runtime() and fails closed with python_runtime_unavailable for every local_cli provider before even resolving the provider command — including adopters who fully replace campaign_adapter_argv with a custom template that never references {target_python} or {target_runtime}. This is documented as intentional ('Before invoking local provider adapters, the campaign runner resolves a supported Python 3.12+ executable...'), but it silently narrows the previously-documented adopter override freedom ('campaign_adapter_argv replaces the maintained template wholesale') by adding an implicit new host prerequisite unrelated to the adopter's own command. Worth confirming this is the intended scope.