Skip to content

fix(ci): cover stacked Python and runtime reviews - #2004

Draft
seonghobae wants to merge 2 commits into
fix/scheduler-prove-workflow-starting-tokenfrom
fix/stacked-python-runtime-review-triggers
Draft

fix(ci): cover stacked Python and runtime reviews#2004
seonghobae wants to merge 2 commits into
fix/scheduler-prove-workflow-starting-tokenfrom
fix/stacked-python-runtime-review-triggers

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

Root cause

Stacked pull request #2003 created Security Scan, SAST Semgrep, and CodeQL PR runs but no Python Security or Agent Review Runtime Quality CI run. The two missing workflows restricted pull_request to default-like base branches, and the permanent stacked-PR contract covered only the other two workflows.

RED → GREEN

  • RED: 890bac2f69ff1a51f774ddf5d6c5d819afed4ac9
    • extends the existing stacked-PR workflow contract to Python Security and Runtime Quality
  • GREEN: 14f7c85ca56be3297fa4d090d39d487d7be9bf14
    • removes only the two pull_request base-branch filters
    • preserves Python Security event types, Runtime Quality path filtering, and all push/schedule behavior
    • updates CHANGELOG, APA-style doctoring, and docs/product-technical-gap-baseline.md

Stack and authority

@coderabbitai

coderabbitai Bot commented Sep 7, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae seonghobae added bug Something isn't working priority: medium Normal-priority or P2 work type: bug Defect or incorrect behavior labels Sep 7, 2026 — with ChatGPT Codex Connector

Copy link
Copy Markdown
Contributor Author

Runtime-quality selector coverage finding — canonical owner follow-up

Authoritative consumer evidence from .github#1996@9f08ed08594f17d66f802c556c0573f6d3fae21a shows Agent Review Runtime Quality CI run 34079536693, job 101612067564, terminal SUCCESS, but the actual hosted pytest log executed only:

  • Noema token-lifetime suite: 35 passed / 1.30s
  • consolidation suite: 16 passed / 0.23s

It did not select/execute these changed surfaces from #1996:

  • tests/test_noema_orchestrator_workflow_contract.py
  • tests/test_required_workflow_queue_contract.py
  • scripts/ci/noema_review_gate.py

Therefore this hosted SUCCESS is a 51-test selector receipt, not hosted validation of the entire #1996 changed surface. Keep it distinct from #1996's local full 2996 passed, 1 skipped, 21 subtests evidence.

Historical selector/consolidation authority is merged .github#1836 (87fde45666156a0d2beae7c19914d324428da66d), which explicitly owns agent-review-runtime-quality-ci.yml affected-suite selection and consolidated contract execution. This current open #2004 already owns restoring Runtime Quality workflow coverage without adding a second workflow, so please carry the minimum current-main RED→GREEN successor here (or an explicitly linked bounded successor if overlap requires it):

  1. RED: a changed-surface fixture shaped like fix(noema): skip sidecar for ineligible reviews #1996 must prove the actual selector omits the three paths above.
  2. GREEN: extend the existing affected-suite selector / execution list minimally so the relevant Noema/review-contract suite actually runs when those paths change.
  3. Preserve the single consolidated runner/job, current path filtering, PR-stable concurrency, permissions, and no model timeout changes.
  4. Do not interpret the 51 hosted passes as coverage of omitted files, and do not replace the local full-suite evidence with this narrower receipt.
  5. Re-run the actual workflow on an unchanged exact head and inspect the hosted log to prove the added tests really executed, not merely that a structural string contract passed.

No source mutation is requested on #1996 itself by this finding. No new duplicate writer/workflow is needed.

Separate evidence boundary: the existing Noema claim-level verification gap remains independently tracked by Concept's comment 5565280099; do not conflate that semantic verification issue with this CI selector omission.

seonghobae commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

Owner coordination — preserve #2004 and #1996 as separate required deltas

Updated after #1996 advanced to 66854e8286af2f469e8cb7c02ca62b08fd4fea35.

Current coordination boundary:

Evidence boundary:

  • Full pytest session 78385 terminated successfully on exact 32bb4f62: 3002 passed, 1 skipped, 21 subtests / 643.71s / exit 0. This is owner-reported local evidence, not independently reproduced hosted evidence.
  • Do not transfer that full-suite result to 66854e82. The later head has its own focused evidence only: 37 passed with -W error.

Integration rule:

  1. fix(ci): cover stacked Python and runtime reviews #2004 and the bounded fix(noema): skip sidecar for ineligible reviews #1996 quality delta are complementary, not replacements.
  2. If fix(ci): cover stacked Python and runtime reviews #2004 owner is willing to absorb the quality repair, carry only the exact three quality commits/delta above, then prove patch/semantic equivalence and regenerate combined exact-head hosted Runtime Quality evidence.
  3. If fix(ci): cover stacked Python and runtime reviews #2004 owner does not absorb it, preserve the quality repair through an explicitly linked bounded successor/restack; do not simple-close fix(noema): skip sidecar for ineligible reviews #1996 or claim whole-PR succession.
  4. The original Noema 4 commits + 66854e82 stay in fix(noema): skip sidecar for ineligible reviews #1996 and proceed through their own review/check/merge path.
  5. After integration, inspect hosted logs to prove both (a) stacked-PR workflow admission from fix(ci): cover stacked Python and runtime reviews #2004 and (b) actual selector→pytest execution from the bounded quality delta.
  6. Ordinary merge/non-force restack only; no force push/destructive rebase.

This is coordination only; no source mutation is requested.

Copy link
Copy Markdown
Contributor Author

Single-writer correction / valid-delta handoff. Do not discard, revert, force-push, or duplicate this work.

A DeviceCheck failure hid #2004 ownership from the other session, so PR #1996 was advanced normally with three quality-only commits after its original Noema-admission head. These commits are valid delta and need to be absorbed by this existing Runtime Quality owner through an ordinary merge/non-force reconciliation, or by an explicitly bounded successor if direct integration is structurally cleaner:

  • 3170932b726dc0e7be5ba83a12d9a533ac40d543 — adds missing trigger → selector → pytest coverage for scripts/ci/noema_review_gate.py, tests/test_noema_review_gate.py, tests/test_noema_orchestrator_workflow_contract.py, and tests/test_required_workflow_queue_contract.py; also adds the consolidated-CI recurrence rule in AGENTS and executable selector regression.
  • a216b526e2bc8d0279870eac1099cec34176b6f4 — extends the same queue-suite ownership to tests/test_current_head_coalescer_self_cancellation.py and strengthens the executable selector assertion so noema/queue selection cannot both silently pass.
  • 32bb4f62f46c7cb57e56fdff6db702d2a12eb4aa — fixes the process-substitution failure hole: capture git diff --name-only "$BASE_SHA...$HEAD_SHA" before iteration so an unreadable base fails the step instead of publishing all-false suite outputs; adds a real invalid-base regression and AGENTS guidance.

Measured handoff evidence from the originating session: 3 commits total = workflow +19 lines, AGENTS +9, regression test +60; five-path trigger→selector→pytest contract initially RED with all-false selection / missing execution; command-substitution change then repaired the hidden git-diff failure path; latest focused result 23 passed, actionlint exit 0, independent Concept read review PASS. A much larger full suite is currently in flight (~59% at handoff) and must not be promoted as GREEN until terminal on the unchanged source.

Important boundary: #1996's original Noema admission optimization remains its own valid delta and must stay intact. These three later commits only repair the shared agent-review-runtime-quality-ci.yml ownership/coverage gap. Preserve both histories. Please compare #2004 14f7c85c... and #1996 32bb4f62... from their merge bases and integrate the quality subset non-destructively; if overlap exists, carry semantics + regressions, not just line text. Regenerate exact-head focused/full/hosted evidence after integration. No new workflow, no duplicate writer, no error-ignore/suppression, no predecessor evidence transfer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working priority: medium Normal-priority or P2 work type: bug Defect or incorrect behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant