Skip to content

bug: tickets reach ImplApproved with CI green against a stale base — mergeStateStatus=CLEAN checks for conflicts, not for whether the merge result passes #613

Description

@wrenrichley

A ticket reaches ImplApproved with a green quality run, and that run may
have executed against a base commit several merges old. Nothing in the flow
refreshes the impl branch or re-runs CI against current main before the PR is
presented as ready.

Measured today (2026-08-10, bookwright, 8 merges in ~2 hours)

PR ticket behind_by at ImplApproved what had landed underneath it
#284 103263 (#261) 2 #260 removed the interiority_ratio metric — and #284's test_dialogue.py still called get_registry()["interiority_ratio"]
#282 66521 (#229) 5 #262 made confidence= a required field on every @metric; #264 changed sensory_distribution's return shape; #260 removed a metric; #261 changed another

Both were reported mergeStateStatus=CLEAN with quality=SUCCESS.

⇒ CLEAN means "no merge conflict", not "the merge result passes." Those
are different claims and only one of them was checked. A textual merge succeeds
happily when one side deletes a function and the other side keeps calling it —
git has no opinion about that, and the green check was computed before the
deletion existed.

I merged main into both branches by hand and re-ran; both then passed. So no
breakage reached main — but that was an operator noticing, not the loop.

Why it is worse than it looks on a busy day

The hazard scales with merge rate on the project, not with anything about
the ticket. On a quiet day behind_by is 0 and the check is honest. On a day
like today, with a required-field change landing in the metric registry, a
five-commit-stale green is close to meaningless — and it is exactly the day when
an operator is moving fastest and least likely to look.

It also composes badly with the duplicate-dispatch bug (#612): a second
ImplReview box that started earlier can approve against an even older base.

Asked for

Before a ticket enters ImplApproved, either:

  1. Refresh and re-verify — merge (or rebase onto) current main, re-run
    the gate, and only then approve; or
  2. Refuse to approve a stale branch — treat behind_by > 0 as a
    not-ready condition and send the ticket back for a refresh, which is
    cheaper but stalls on a busy repo.

(1) is the better shape. (2) is acceptable and much simpler.

Acceptance, as a property

No ticket may enter ImplApproved while its impl branch is behind its base,
and the approving check run must be one that executed against a commit whose
tree includes current main. Asserting "the reviewer refreshes the branch"
would pass while an already-in-flight box approves an old one; the invariant has
to be about the ticket's state at the moment of approval, not about what any
particular role does.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions