Repository navigation
π fix(github): treat a head time ahead of the host clock as skew, not a fresh push, in the min_head_age gate - #16146
Conversation
β¦ a fresh push, in the min_head_age gate Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hive status β updated in place; edits do not notify. Full history is in the Activity Monitor. hive/awaiting-ci: CI pending β sweep will merge when green. π Hive Agent: β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 Hive filed review findings that do not block this PR as backlog items:
Cap: 10 backlog item(s) per PR per day. Review findings from Read the diff against the tree at Verified:
One docs-currency note (info, not blocking): the sweep path bounds the time before Looks correct to me. Review findings from Style/conventions review (head 292531f) The new skew reason string escapes the repo's existing min_head_age reason classifiers, which key on the literal prefix
Suggested fix: add one shared predicate (e.g. Minor: Review findings from Read the diff against the tree at What I verified:
One docs-currency gap, low: |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
There was a problem hiding this comment.
Security perspective β no blocker. Reviewed head 3260437.
Checked whether the new fail-open branch (skewed := age < 0, src/pkg/github/commit_ci.go:657-659) is reachable by a PR author who sets GIT_COMMITTER_DATE in the future. It is not: both callers only let the committer date replace the server-side updated_at pin when it is earlier (src/pkg/github/merge_ci_gate.go:194 at.Before(minHeadAgeFrom); src/pkg/github/automerge/automerge_head_age.go:181 committed.Before(pinned)), so a future HeadPushedAt can only come from GitHub's own clock. Fail-closed for age >= 0 is preserved.
Advisory (low, in-scope, not blocking): the skip has no tolerance band. Any negative age β including a few seconds of ordinary NTP drift between this host and GitHub β now disables the gate for a head evaluated in the first moments after push, which is exactly the window min_head_age exists for (commit_ci.go:386-392). The configRequiredAllGreen escape and ExpectedChecks "has not started" block cover most of that, but when the required set is unknown and expected checks are empty, a sweep landing seconds after push with no check runs yet would pass. Consider treating only age < -tolerance (e.g. -MinHeadAge or a fixed 30s) as skew and keeping the clamp-to-0 hold for small negatives.
β hive: agent=ci-maintainer backend=copilot model=claude-fable-5.1 copilot=1.0.88
β¦n when the head time is ahead of the host Review follow-up for #16146: skipping the hold on skew disabled the only cover for slow check-runs when the required set is unknown. The automerge sweep now pins the host clock at each head's first observation and feeds it to the gate when GitHub's head time is in the future; the merge-request watcher passes its PendingSince the same way. The hold lasts one min_head_age window from first observation and then clears instead of clamping to "pushed 0s ago" for the whole skew. Tests assert the hold, its expiry, and that a sane head time is unaffected. Hive-Fix-Attempt: 1/6 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Read the diff against v5 plus the callers (automerge_sweep_queued.go:129/362, automerge_sweep_trusted.go:148,676, automerge_contributor_lane.go:1420, automerge_held_runs.go:532, merge_request_watcher.go:465-485). The fix looks correct to me:
headPushedAtWith(automerge_head_age.go:196-212): a futurepinnedmakesat.Sub(pinned)negative, so the early return is skipped and every exit now goes throughhostClockBound, which returns the host-clock first observation (ornowon the very first observe).observerecordshostFirstSeenbefore the zero-candidate early return and now callstouchLockedthere, so the new map is pruned with the others (touchLocked, line 116).EvaluateCommitCI(commit_ci.go:654-681):skewedwith zeroHeadFirstSeenAtholds (age 0); with a non-zero one it measuresnow - HeadFirstSeenAt. Theage >= 0path is unchanged.- Watcher:
PendingSinceis zero on the first poll and set tonowon the first pending verdict (merge_request_watcher.go:477-485), so the second poll measures from it β exactly one window, as the body says.verifyMergeRequestCIhas a single caller, updated.
One documentation inaccuracy, low severity:
src/docs/operator-reference/README.md:453andchangelog.d/fixed-16143.mdsay the automerge sweep emitspending: head time ahead of host clock; head first seen β¦ ago. It does not: the sweep passes the host-clock time asHeadPushedAt(hostClockBoundβcommitGreenForPRSkippingβHeadPushedAt:atautomerge_sweep_queued.go:362), soEvaluateCommitCIcomputesage >= 0,skewedis false, and the sweep still logspending: head pushed Ns ago (< min_head_age)(with N measured from first observation). Only thehive-mergewatcher path, which passesHeadFirstSeenAtseparately, produces the new reason. The hold behaviour is right either way; an operator grepping the sweep log for the documented string would not find it. Suggest either wording the doc as "the sweep measures the hold from first observation (reason still readshead pushed β¦ ago)" or havingheadPushedAtWithalso surface first-seen tocommitGreenForPRSkippingso both paths share the reason.
Not a blocker. I have not approved or merged anything.
β hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
There was a problem hiding this comment.
docs-currency β read head 292531f.
Docs/changelog for the new behavior are present and accurate (src/docs/operator-reference/README.md:453, changelog.d/fixed-16143.md). Two places that render or classify the min_head_age reason still match only the old string and so miss the new one:
-
Relay refusal text now blames author tier under skew β
src/pkg/github/automerge/automerge_relay_gate.go:134-141.relayMergeRefusalonly recognizesstrings.HasPrefix(reason, "pending: head pushed "); the new reasonpending: head time ahead of host clock; head first seen β¦ ago (< min_head_age)(src/pkg/github/commit_ci.goin this diff) falls todefaultand the operator sees(author tier "β¦", require_tiers β¦)β the wrong explanation for a min_head_age hold. Fix: match the new prefix too (or match on the shared(< min_head_age)suffix asstage_tracker_caption.go:413already does) and keep theauto_merge.min_head_agehint. -
isFreshHeadOrMissingExpectedReasonmisses the skew reason βsrc/pkg/github/automerge/automerge_sweep_queued.go:401-403. Same prefix-only match; a skewed hold is no longer treated as a fresh-head pending reason atautomerge_sweep_queued.go:385, sorememberEvaluatedHeadruns for it. Suggest the same suffix match. I did not trace every reader ofevaluatedHeads, so I'm flagging the divergence rather than a specific consequence.
The dashboard caption (src/pkg/dashboard/stage_tracker_caption.go:413) already keys on the (< min_head_age) suffix and renders the new reason fine.
β hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
GitHub's server-generated timestamps (
created_at/updated_at/pushed_at) were observed running +1h ahead of the host clock.HeadPushedAtthen landed in the future,EvaluateCommitCIclamped the negative age to 0, and every green head read as "pushed 0s ago" for an hour, stalling the merge sweep.A head time ahead of the host clock is now treated as clock skew rather than a fresh push, but the
min_head_agehold is not skipped (when the required set is unknown it is the only cover for slow check-runs that have not registered yet). Instead the hold is measured from the host-clock time the hive first observed the head:EvaluateCommitCIgainsCommitCIOptions.HeadFirstSeenAt; whenHeadPushedAtis ahead ofnow()the age isnow - HeadFirstSeenAt(zero = first observation β holds), with a distinct reasonpending: head time ahead of host clock; head first seen β¦ ago (< min_head_age).HeadFirstSeenTrackerpins the host clock at each head's first observation andheadPushedAtWithreturns it when the server time (pinnedupdated_atorcreated_at-clamped commit time) is in the future.PendingSinceasHeadFirstSeenAt.Net effect: under skew a head is held for at most one
min_head_agewindow from first observation and then clears. Behaviour forage >= 0is unchanged (fail-closed).Not changed here: contributor soak and
headObservedAt(approval post-dating). Soak compares the server-side green time against localnow, so skew only lengthens the soak (fail-closed over-hold, bounded by the skew);headObservedAtcompares server timestamps against server timestamps (approvalsubmitted_at), which skew uniformly. Neither stalls a merge indefinitely, so they are left as-is.Closes #16143
β hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88