-
Notifications
You must be signed in to change notification settings - Fork 1
fix(governance): supersede stale review decisions safely #1438
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Changes from all commits
8d341b2
2047371
a9f9b1e
dc1155a
df1e5a8
4ccee12
5cc9ca3
899493f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| # ADR-0006: Let exact-head robot evidence supersede stale aggregate review state | ||
|
|
||
| - Status: Accepted | ||
| - Date: 2026-08-20 | ||
| - Decision owners: Naruon maintainers | ||
|
|
||
| ## Context | ||
|
|
||
| GitHub exposes `reviewDecision=CHANGES_REQUESTED` as an aggregate pull-request | ||
| field. The field can remain `CHANGES_REQUESTED` after the requesting review was | ||
| submitted against an older commit and a later exact-head CodeRabbit or | ||
| structured OpenCode review has passed. Treating that stale aggregate as a | ||
| current blocker stranded protected merges even when current review threads and | ||
| required checks were clean. | ||
|
|
||
| The GitHub REST review response includes each review's `state` and `commit_id`, | ||
| so the gate can distinguish a current request from a request attached to an | ||
| older head (GitHub, 2026). | ||
|
|
||
| ## Decision | ||
|
|
||
| The metadata-only gate will: | ||
|
|
||
| 1. Read all pull-request review metadata when the aggregate decision is | ||
| `CHANGES_REQUESTED`. | ||
| 2. For each reviewer, consider only the latest submitted review. Keep the | ||
| blocker when that effective review is `CHANGES_REQUESTED` on the current | ||
| head, when review metadata cannot be read, or when current-head robot review | ||
| evidence is absent or pending. | ||
| 3. Treat the aggregate decision as superseded only when all requested reviews | ||
| target older commits and current-head CodeRabbit or structured OpenCode | ||
| evidence passes. | ||
| 4. Never dismiss reviews, rewrite review state, use an administrator merge, or | ||
| bypass required checks. | ||
|
|
||
| ## Consequences | ||
|
|
||
| Protected merges can proceed after a later exact-head review supersedes stale | ||
| aggregate state. A current requested change remains a hard blocker, and an API | ||
| failure fails closed. The gate performs an additional read-only reviews API | ||
| call only for pull requests whose aggregate decision is `CHANGES_REQUESTED`; | ||
| the CodeRabbit-absent path may already have read the same endpoint while | ||
| checking for structured OpenCode approval. | ||
|
|
||
| ## Verification | ||
|
|
||
| `bash scripts/ci/test_pr_governance_gate.sh` proves stale-review supersession, | ||
| blocking of current-head requested changes, and supersession by a later | ||
| same-head approval from the same reviewer. | ||
|
|
||
| ## Reference | ||
|
|
||
| GitHub. (2026). *REST API endpoints for pull request reviews*. GitHub Docs. | ||
| https://docs.github.com/en/rest/pulls/reviews?apiVersion=2022-11-28 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| # Current-head review evidence and stale aggregate state | ||
|
|
||
| Naruon's protected merge gate evaluates evidence for one immutable pull-request | ||
| head. GitHub's aggregate `reviewDecision` can remain `CHANGES_REQUESTED` after | ||
| the review was submitted on an older commit. The gate therefore reads the | ||
| review-level `commit_id`: a current-head request blocks, while only stale | ||
| requests may be superseded by passing exact-head robot evidence. | ||
|
|
||
| This preserves the operational action for the customer: fix the request when it | ||
| targets the current code; otherwise continue the protected merge loop after the | ||
| new head's Checks and robot evidence pass. The gate does not dismiss reviews or | ||
| use an administrative bypass. | ||
|
|
||
| ## APA 7 reference | ||
|
|
||
| GitHub. (2026). *REST API endpoints for pull request reviews*. GitHub Docs. | ||
| https://docs.github.com/en/rest/pulls/reviews?apiVersion=2022-11-28 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -273,10 +273,6 @@ if [ "$MERGE_STATE" = "UNKNOWN" ]; then | |
| add_waiting "Merge state is still UNKNOWN after 4 attempts on ${HEAD_REF_OID}; waiting for GitHub to refresh mergeability." | ||
| fi | ||
|
|
||
| if [ "$REVIEW_DECISION" = "CHANGES_REQUESTED" ]; then | ||
| add_blocker 'Review decision is CHANGES_REQUESTED; address requested changes before merge.' | ||
| fi | ||
|
|
||
| # shellcheck disable=SC2016 # GraphQL variables must remain literal. | ||
| THREADS_JSON="$(gh api graphql \ | ||
| -F owner="$OWNER" \ | ||
|
|
@@ -343,6 +339,7 @@ CODERABBIT_ISSUE_SUBSTANTIVE_BLOCKING_PATTERN='pre[- ]merge[^\n]*(blocking|failu | |
| CODERABBIT_NO_ACTIONABLE_PATTERN='no actionable comments? (were )?generated' | ||
| CHECK_RUNS="$(gh api "repos/${GITHUB_REPOSITORY}/commits/${HEAD_SHA}/check-runs?per_page=100")" | ||
| COMMIT_STATUS_JSON='{"statuses":[]}' | ||
| CURRENT_ROBOT_REVIEW_READY=false | ||
| if ! COMMIT_STATUS_JSON="$(gh api "repos/${GITHUB_REPOSITORY}/commits/${HEAD_SHA}/status" 2>"$COMMIT_STATUS_ERROR_FILE")"; then | ||
| printf 'commit status lookup failed:\n' | ||
| printf '%s\n' "$(<"$COMMIT_STATUS_ERROR_FILE")" | sed 's/^/ /' | ||
|
|
@@ -386,6 +383,7 @@ if [ "$CODERABBIT_COUNT" = "0" ]; then | |
| if [ "$OPENCODE_ADVERSARIAL_APPROVAL_COUNT" = "0" ]; then | ||
| add_waiting "Waiting for current-head CodeRabbit evidence or a structured OpenCode App adversarial approval on ${HEAD_REF_OID}." | ||
| else | ||
| CURRENT_ROBOT_REVIEW_READY=true | ||
| printf 'CodeRabbit check is absent; accepted current-head OpenCode App adversarial approval on %s.\n' "$HEAD_REF_OID" | ||
| fi | ||
| fi | ||
|
|
@@ -423,6 +421,8 @@ else | |
| : | ||
| elif [ "$CODERABBIT_PENDING" != "0" ] || [ "$CODERABBIT_STATUS_PENDING" != "0" ]; then | ||
| add_waiting "Waiting for current-head CodeRabbit evidence on ${HEAD_REF_OID}." | ||
| else | ||
| CURRENT_ROBOT_REVIEW_READY=true | ||
|
seonghobae marked this conversation as resolved.
|
||
| fi | ||
| fi | ||
|
|
||
|
|
@@ -476,6 +476,41 @@ else | |
| fi | ||
| fi | ||
|
|
||
| # GitHub's aggregate reviewDecision remains CHANGES_REQUESTED after the | ||
| # requested review's commit becomes stale. Treat that aggregate as blocking | ||
| # only when a current-head request still exists, review metadata is unavailable, | ||
| # or no current-head robot evidence supersedes the stale decision. This keeps | ||
| # the gate aligned with its exact-head evidence contract without dismissing any | ||
| # review object or bypassing a current request. | ||
| if [ "$REVIEW_DECISION" = "CHANGES_REQUESTED" ]; then | ||
| if ! REVIEW_METADATA_JSON="$(gh api --paginate "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews" 2>"$OPENCODE_REVIEWS_ERROR_FILE")"; then | ||
|
seonghobae marked this conversation as resolved.
|
||
| printf 'Review metadata lookup failed:\n' | ||
| printf '%s\n' "$(<"$OPENCODE_REVIEWS_ERROR_FILE")" | sed 's/^/ /' | ||
| add_blocker 'Review decision metadata could not be read; see the workflow run log.' | ||
| else | ||
| CURRENT_CHANGES_REQUESTED_COUNT="$(printf '%s' "$REVIEW_METADATA_JSON" | jq -s --arg head_sha "$HEAD_SHA" ' | ||
| [ .[] | ||
| | .. | ||
| | objects | ||
| | select(has("state") and has("commit_id")) | ||
|
Comment on lines
+493
to
+495
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When GitHub reports aggregate AGENTS.md reference: AGENTS.md:L190-L194 Useful? React with 👍 / 👎. |
||
| ] | ||
| | sort_by([(.submitted_at // .created_at // ""), (.id // 0)]) | ||
| | group_by(.user.login // .user.id // "") | ||
| | map(last) | ||
| | map(select((.state // "" | ascii_upcase) == "CHANGES_REQUESTED")) | ||
|
Comment on lines
+499
to
+500
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When a reviewer requests changes on the current head and later submits a AGENTS.md reference: AGENTS.md:L190-L194 Useful? React with 👍 / 👎. |
||
| | map(select((.commit_id // "") == $head_sha)) | ||
| | length | ||
| ')" | ||
|
seonghobae marked this conversation as resolved.
seonghobae marked this conversation as resolved.
|
||
| if [ "$CURRENT_CHANGES_REQUESTED_COUNT" != "0" ]; then | ||
| add_blocker 'Review decision is CHANGES_REQUESTED; address current-head requested changes before merge.' | ||
| elif [ "$CURRENT_ROBOT_REVIEW_READY" != true ]; then | ||
| add_blocker 'Review decision remains CHANGES_REQUESTED; await current-head robot review evidence before merge.' | ||
| else | ||
| printf 'Stale CHANGES_REQUESTED review decision is superseded by current-head robot evidence on %s.\n' "$HEAD_REF_OID" | ||
|
Comment on lines
+508
to
+509
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When a pending review is started on an older commit but submitted after the current-head CodeRabbit/OpenCode run has already passed, its AGENTS.md reference: AGENTS.md:L190-L194 Useful? React with 👍 / 👎. |
||
| fi | ||
| fi | ||
| fi | ||
|
seonghobae marked this conversation as resolved.
seonghobae marked this conversation as resolved.
|
||
|
|
||
| if ! pr_snapshot_is_current; then | ||
| exit 0 | ||
| fi | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.