Skip to content

fix: address late-landing bot review findings on PR #1038 and #1041#1052

Merged
joeharris76 merged 2 commits into
developfrom
chore/pr-review-followup-sweep-3
Jul 8, 2026
Merged

fix: address late-landing bot review findings on PR #1038 and #1041#1052
joeharris76 merged 2 commits into
developfrom
chore/pr-review-followup-sweep-3

Conversation

@joeharris76

Copy link
Copy Markdown
Owner

Pull Request

Description

Follow-up sweep for chatgpt-codex-connector review comments that landed on two already-merged PRs shortly after they merged: #1038 (2 findings) and #1041 (1 finding).

  • fix: address late-landing bot review findings across 11 merged PRs #1038plan_metadata_utils.py: update_plan_versions fell into the fingerprint-version-mismatch branch for a brand-new query_id (one absent from prev_metadata entirely), which fell back to prev_metadata.plan_versions.get(query_id, 0) — writing an invalid version 0 (validate_plan_metadata rejects versions < 1) whenever a newly-added query's captured fingerprint_version differed from prev's default. Fixed by bypassing that branch entirely for query_ids not in prev_metadata.plan_fingerprints — there's no prior recording to be "not comparable" to, so a new query always starts at version 1.
  • fix: address late-landing bot review findings across 11 merged PRs #1038schema.py: plan_capture_error was added to ResultBuilder's in-memory query_results dict (from PR fix: address late-landing bot review findings across 11 merged PRs #1038's own fix(qpc-05): surface plan capture/load failures instead of converting them to absence #1032 fix), but the actual JSON export goes through schema.py's _build_query_results_section, a separate serialization path that copies only specific fields onto the compact queries[] entry. The real DataFrame plan-capture-error cause was reaching the in-memory representation but getting silently dropped on export/reload. Added plan_capture_error to _build_query_results_section (and to schema_specs.yaml's query_key_order for stable placement).
  • feat(governance): enforce the vendor-supplied label via the submission gate #1041validate-submission.yml: the "Reject non-maintainer vendor/ additions" step's OWNER | MEMBER | COLLABORATOR allowlist didn't account for the trusted same-repo auto/results-mirror-* PRs that sync-results-data-to-published.yml opens with secrets.GITHUB_TOKEN (authored by github-actions[bot]), so a maintainer's vendor bundle published through that path fell through to exit 1. Added the same two-signal trust check (head.repo.fork == false AND the mirror branch pattern) the "Validate bundles" step's manifest waiver already uses — head.repo.fork is the unforgeable, load-bearing signal since github.head_ref alone is attacker-controlled on fork PRs.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Refactor / chore

Testing

Every fix has a dedicated regression test, verified to fail before the fix and pass after via git stash:

uv run -- python -m pytest tests/unit/core/manifest/ tests/unit/core/results/ tests/unit/workflows/ \
  tests/unit/test_release_infrastructure.py -q
# 873 passed, 3 skipped

Full tests/unit/ -m fast suite (23,323 tests) also run: 3 pre-existing failures, confirmed to reproduce identically on a clean develop checkout (unrelated to this PR — sandbox runs as root, bypassing two filesystem-permission-error tests, and the zstd CLI binary is absent for a third).

ruff check / ruff format --check / ty check all clean on every changed file. .github/workflows/validate-submission.yml YAML-parse verified.

Public Contract Check

  • I updated docs/reference/public-contracts.md, or this PR does not change public, beta-public, deprecated, generated, experimental, or repo-only contract surfaces.
  • I checked result schema fields, MCP tool parameters, platform support status, wrapper facades, adapter subclassing hooks, and generated docs for contract-map impact. plan_capture_error on the exported queries[] entry is additive; existing consumers are unaffected.
  • Any platform/benchmark count claim in docs is generated/checked from registry metadata, or explicitly marked editorial/non-authoritative.

Documentation

  • I updated the relevant user-facing docs (N/A — no doc-facing behavior changed here beyond code comments).
  • I added regression notes for behavior changes (inline comments explain each fix's "why").
  • I confirmed API contract wording remains accurate.

Code Quality

  • I ran formatting and lint/type checks.
  • I reviewed impacted modules for backwards compatibility and migration impact.
  • I added/updated tests for behavior changes and error paths.
  • I confirmed public contracts and release checklists are still valid.

Artifact Hygiene

  • I did not commit raw screenshots, browser reports, generated logs, benchmark outputs, or temporary binary artifacts.
  • No binary/raw evidence files added.

Notes

Not CODEOWNERS-gated (no changes under benchbox/core/query_plans/parsers/**, benchbox/core/expected_results/**, or the other soundness-critical paths) — squash auto-merge will be enabled.

🤖 Generated with Claude Code


Generated by Claude Code

- #1038 (plan_metadata_utils.py): a brand-new query_id whose fingerprint_version
  differs from prev's default fell into the version-comparability-mismatch
  branch, which fell back to prev_metadata.plan_versions.get(query_id, 0) -
  writing an invalid version 0 (validate_plan_metadata rejects versions < 1).
  Bypass that branch entirely for query_ids absent from prev_metadata: there is
  no prior recording to be "not comparable" to, so a new query always starts
  at version 1.
- #1038 (schema.py): plan_capture_error was added to ResultBuilder's in-memory
  query_results dict, but the actual JSON export goes through schema.py's
  _build_query_results_section, a separate serialization path that copies only
  specific fields onto the compact queries[] entry. Added plan_capture_error
  there too, so the real DataFrame capture-failure cause reaches the persisted
  bundle, not just the in-memory representation.
- #1041 (validate-submission.yml): the "Reject non-maintainer vendor/
  additions" step's OWNER/MEMBER/COLLABORATOR allowlist didn't account for the
  trusted same-repo auto/results-mirror-* PRs that
  sync-results-data-to-published.yml opens with secrets.GITHUB_TOKEN (authored
  by github-actions[bot]), so a maintainer's vendor bundle published through
  that path was blocked. Added the same two-signal trust check
  (head.repo.fork == false AND the mirror branch pattern) the "Validate
  bundles" step's manifest waiver already uses.

Each fix has a regression test verified to fail before the fix and pass after.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SKw2UhgHuzYPo3y5MfarD6
@joeharris76
joeharris76 enabled auto-merge (squash) July 8, 2026 11:34

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 00a76cf4b7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread benchbox/core/results/schema.py
The export fix alone was incomplete: _reconstruct_query_results() copied
iter/stream/run_type/test_type from the compact queries[] entry but not
plan_capture_error, so an export -> load -> re-export round trip silently
dropped the real DataFrame capture-failure cause again even though it
survived the first export.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SKw2UhgHuzYPo3y5MfarD6
@joeharris76
joeharris76 merged commit 24e2fee into develop Jul 8, 2026
22 checks passed
@joeharris76
joeharris76 deleted the chore/pr-review-followup-sweep-3 branch July 8, 2026 12:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants