Skip to content

ci: make reviewer merge-holds stick against auto-merge re-arming - #313

Open
dieterolson wants to merge 2 commits into
mainfrom
ci/merge-hold-guard
Open

ci: make reviewer merge-holds stick against auto-merge re-arming#313
dieterolson wants to merge 2 commits into
mainfrom
ci/merge-hold-guard

Conversation

@dieterolson

Copy link
Copy Markdown
Contributor

Problem

Fleet automation re-arms GitHub auto-merge on pull requests that a reviewer has deliberately disarmed, which defeats the only lightweight mechanism available for holding a dangerous PR back.

Measured in Gasification_Model on 2026-08-14 (all events attributed to dieterolson, type=User, i.e. the owner's credentials rather than a bot):

PR disarmed re-armed gap
#4692 04:15:08Z 04:15:16Z 8s
#4710 04:15:39Z 04:15:45Z 6s
#4711 04:15:50Z 04:15:56Z 6s
#4709 03:38:35Z 04:02:09Z 24m — closed outright to make the block stick

PR #4709 is why this matters rather than merely annoying: it had auto-merge armed on a diff that deleted 13 files present on main and grew SPEC.md from 5,084 to 60,863 lines.

The re-armer is not a workflow in this repository. No workflow in Gasification_Model, Tools, Tools_Private, Drake_Models, MuJoCo_Models, Controls or Maxwell_Daemon calls gh pr merge --auto or enablePullRequestAutoMerge. The arming comes from agent sessions running under the owner's gh credentials, driven by the fleet-pr-queue automation in Repository_Management/config/codex_fleet_automations.json, whose prompt instructs them to "enable auto-merge for the highest-confidence item". Because those sessions live outside any single repo, a repo-side guard is the only enforcement point that cannot be bypassed by editing an agent prompt.

What this adds

.github/workflows/Merge-Hold-Guard.yml. It only ever removes auto-merge — it never merges, never arms, never pushes.

Hold signals (any one is sufficient):

  1. the do-not-merge or blocked label
  2. the PR is a draft
  3. auto-merge was disabled by a non-bot account more recently than the head commit — a reviewer said no and nobody has pushed since
  4. the diff deletes tracked files with no acknowledgement (deletions-acknowledged label, or a Deletions-Acknowledged: yes line in the PR body)

Signal 3 deliberately ignores bot actors so the guard's own revocations can never manufacture a hold, and a genuine push clears the hold naturally.

Triggers: pull_request_target including the auto_merge_enabled activity type, so a held PR is disarmed seconds after any re-arm; plus a 20,50 * * * * sweep over armed PRs as a backstop, offset off the hour so it does not collide with the fleet pass.

Escalation. A plain revoke cannot win a 6-second re-arm race on its own. After 2 revocations against the same head commit, the guard converts the PR to a draft. This was verified empirically against Tools#4453:

$ gh pr merge 4453 --repo D-sorganization/Tools --squash --auto
GraphQL: Pull Request is still a draft (mergePullRequest)

GitHub refuses to arm auto-merge on a draft, so the hold becomes enforceable at the API level rather than advisory. gh pr ready <n> reverses it deliberately.

The guard also applies do-not-merge to deletion PRs, giving label-aware automation a signal it can honour before ever attempting to arm.

Labels

do-not-merge and deletions-acknowledged did not exist in this repo (gh pr edit --add-label do-not-merge failed with 'do-not-merge' not found). Both have been created, and the workflow re-creates them if missing so it stays drop-in for new repos.

Verification

The detection logic was dry-run against real PRs before this workflow was written. On #4709 it reports:

VERDICT: HELD -> a reviewer disabled auto-merge at 2026-08-14T04:14:21Z,
                 after the head commit (2026-08-13T20:35:02Z);
                 diff deletes 13 tracked file(s) with no acknowledgement

and on #4692/#4710/#4711 it correctly identifies all three as held with auto-merge currently armed. No false positives were found: every open PR in this repo is armed, and the only ones flagged are those the reviewer had disarmed.

YAML parses, all three run: blocks pass bash -n, and the file satisfies this repo's own lint-workflow-files.yml rules (concurrency, cancel-in-progress: true, per-job timeout-minutes, no hosted runners, no protected-branch push).

Companion change

Repository_Management needs the arming side fixed too, so agents stop attempting the arm in the first place rather than relying on the guard to undo it.

Fleet automation re-arms auto-merge on PRs a reviewer disarmed (measured at
6-8s in Gasification_Model), so a manual disarm cannot hold a PR back. Adds a
guard that revokes auto-merge on held PRs, refuses PRs deleting tracked files
without acknowledgement, and converts to draft after repeated re-arms.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

💡 Codex Review

PRS=$(gh pr list --repo "$REPO" --state open --limit 200 \
--json number,autoMergeRequest \
--jq '.[] | select(.autoMergeRequest != null) | .number')

P1 Badge Replace the scheduled bulk gh pr list sweep

In scheduled and input-less dispatch runs, this enumerates up to 200 open PRs via gh pr list every 30 minutes; because the workflow is intended for fleet-wide deployment, this becomes a recurring GraphQL sweep in every repository and directly violates the repository's prohibition on bulk gh pr list scans. Use event-driven enforcement or a focused REST-based backstop that does not enumerate the open-PR set.

AGENTS.md reference: AGENTS.md:L96-L96


gh api "repos/$REPO/pulls/$PR/files?per_page=100" --paginate \
--jq '.[] | select(.status == "removed") | .filename' \
> "$REMOVED_FILE" 2>/dev/null || : > "$REMOVED_FILE"

P1 Badge Fail closed when deletion enumeration fails

If the files endpoint returns a transient, authentication, or rate-limit error, this replaces the result with an empty file and interprets the PR as having no deletions. On an armed PR with no other signal, the workflow then leaves auto-merge enabled, defeating the deletion guard during exactly the event that must revoke it; record the failure and fail the enforcement run rather than treating it as an empty result. The suppression also continues issuing API calls after a possible rate-limit response, contrary to the repository's required handling.

AGENTS.md reference: AGENTS.md:L110-L110


HEAD_DATE="$(gh api "repos/$REPO/commits/$HEAD_SHA" \
--jq '.commit.committer.date' 2>/dev/null)" || HEAD_DATE=""
if [ -n "$LAST_DISARM" ] && [ -n "$HEAD_DATE" ] && [[ "$LAST_DISARM" > "$HEAD_DATE" ]]; then

P2 Badge Reset holds from push events rather than commit timestamps

A commit's committer timestamp is not the time it was pushed to the PR branch. If a contributor responds to a hold by pushing or force-updating to an existing older commit, HEAD_DATE still predates LAST_DISARM, so the old reviewer decision remains active even though the documented contract says any new push supersedes it; the guard can repeatedly revoke auto-merge and eventually draft the updated PR. Track the latest synchronize/head-change event or the head SHA associated with the disarm instead.


PRIOR_REVOCATIONS="$(awk -F'\t' -v cutoff="$HEAD_DATE" \
'$1 == "Bot" && $2 > cutoff { n++ } END { print n + 0 }' "$TIMELINE")"

P2 Badge Count only this guard's revocations before drafting

This counter treats every auto_merge_disabled event whose actor type is Bot as a revocation by this guard. If another GitHub App or bot has disabled auto-merge twice since the head commit, a later held run immediately satisfies the escalation threshold and converts the PR to draft even though this guard has never encountered a re-arm loop. Persist guard-specific evidence, such as actor identity plus a guard marker tied to the head SHA, before escalating.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@dieterolson
dieterolson enabled auto-merge (squash) August 14, 2026 08:01
`do-not-automate` is already the fleet-wide convention for "this work must not
be automated" (shared_scripts/agent_identity.DO_NOT_AUTOMATE_LABEL). Honouring
it here keeps one vocabulary instead of a parallel one, and collapses the
per-label checks into a single list.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant