Skip to content

fix(drivers/github): make walkNewEvents resilient to pagination drift, add a completeness backstop - #45

Merged
jfallows merged 2 commits into
developfrom
claude/ecs-metadata-eks-probe-ej30ee
Jul 25, 2026
Merged

fix(drivers/github): make walkNewEvents resilient to pagination drift, add a completeness backstop#45
jfallows merged 2 commits into
developfrom
claude/ecs-metadata-eks-probe-ej30ee

Conversation

@jfallows

@jfallows jfallows commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Summary

aklivity/zilla-plus#1073 and #1075 both squash-merged into support/1.x before the 1.4.3 release, both are correctly contained by the 1.4.3 tag (confirmed directly with git tag --contains), and neither ever showed up in the fetched entries or any warning when 1.4.3 was cut. CHANGELOG.md shipped silently missing both, on an all-green workflow. (Fixed by hand there in zilla-plus#1078.)

This PR fixes the actual root cause, not just a detector for it:

1. walkNewEvents pagination-drift fix (src/drivers/github.ts) — the real fix

walkNewEvents computes the newest page once (via the initial request's Link header) and walks backward toward page 1, stopping at the first page with nothing newer than the cached watermark — an optimization that avoids re-walking a large repo's full history every run. On a live, active repo, that "newest page" figure can change during the walk: if any new event lands between the initial request and a later one in the same walk, the true page count grows, page boundaries shift, and whatever falls in the newly-revealed range is silently never fetched at any page number at all. This is almost certainly what happened to #1073/#1075.

Every page fetched during the walk now re-checks its own Link header against the highest page number seen so far; if it's grown, the newly-revealed pages are walked first (recursively, same early-stop rule) before the original walk continues — covering drift discovered at any point mid-walk, not only drift already present when the walk started. Deduplicates via a seen-id set.

Adds dedicated walkNewEvents test coverage (previously zero): single-page, multi-page with early-stop (asserting pages beyond the stop point are never even requested), the drift regression case itself, and a repeated-drift dedup case.

2. Completeness backstop (src/completeness.ts) — for whatever #1 doesn't anticipate

Fixing the one known mechanism doesn't rule out every possible way a driver could someday drop an entry, so every run also cross-checks its own output against an independent ground truth: git log itself, over the exact range about to be rendered. Any commit matching GitHub's own "this commit is PR #NNN" conventions (squash-merge's trailing (#NNN), or Merge pull request #NNN from ...) must have a corresponding number in the driver's fetched entries. A gap fails the run (core.setFailed / exit 1) rather than warning — a changelog known to be wrong is worse than a failed build, since a failed build can't be missed. The file is still written either way.

Test plan

  • npm run typecheck — clean
  • npx vitest run — 249/249 passing (238 pre-existing + 4 new walkNewEvents tests + 7 new completeness.test.ts tests)
  • New walkNewEvents tests in test/github-driver.test.ts: single page, multi-page early-stop (with a call-count assertion that stale pages are never fetched), the pagination-drift regression itself, repeated-drift dedup
  • New test/completeness.test.ts: squash-merge/merge-commit subject matching, bare #NNN prose correctly ignored, range-scoping, dedup
  • New describe('run — completeness check', ...) in test/run.test.ts: end-to-end wiring through run()

Known limitations (documented in README)

  • The completeness backstop doesn't distinguish "never fetched" from "fetched but excluded by label" — an intentionally-excluded PR matching the squash/merge subject convention will also be flagged (accepted false positive).
  • The completeness backstop only recognizes GitHub's own squash-merge and merge-commit subject conventions.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KGFBsfF1cCVcXVyDivDapK

claude added 2 commits July 25, 2026 17:13
…never fetched

Every existing failure mode in resolveHashes produces a visible warning
when an entry can't be placed. There was no equivalent for the driver
never fetching the entry in the first place: walkNewEvents walks a
repo-wide, paginated /issues/events feed with a persisted watermark, and
has zero test coverage of its page-boundary "stop at the first empty
page" logic. If that walk ever drops a merged PR's event - pagination
drift, an eventually-consistent read racing a just-merged PR, a
corrupted or unexpectedly-shared cache - the PR simply never appears
anywhere: not in the changelog, not in a warning. The run still reports
success.

This happened for real: aklivity/zilla-plus#1073 and #1075 both
squash-merged into support/1.x before the 1.4.3 release, both are
correctly contained by the 1.4.3 tag (confirmed directly with
`git tag --contains`), and neither ever showed up in the fetched
entries or any warning when 1.4.3 was cut. CHANGELOG.md shipped
silently missing both, on an all-green workflow.

Adds checkCompleteness (src/completeness.ts): an independent
ground-truth cross-check using `git log` directly over the range a run
is about to render, matching GitHub's own "this commit is PR #NNN"
subject conventions (squash-merge's trailing "(#NNN)", or a real merge
commit's "Merge pull request #NNN from ..."). Deliberately narrower
than scanCommitsReferencingNumbers (used elsewhere for a different,
already-known-entry lookup) - a bare "#NNN" anywhere in a commit body
matches ordinary prose ("Fixes #1070", "related to #929") too, which
would false-positive constantly.

Wires it into run.ts via a new `completenessIssues` field on
RunResult - action.ts/cli.ts still write out whatever changelog was
generated, but now core.setFailed / exit 1 when it's non-empty, since a
changelog known to be missing entries is worse than a failed build (a
failed build can't be missed).
The completeness check added in the previous commit only detects a
dropped entry after the fact and fails the run loudly - real resilience
means the driver doesn't drop it in the first place. This fixes the
actual root cause.

walkNewEvents computes the newest page once (via the initial request's
Link header) and walks backward toward page 1, stopping at the first
page with nothing newer than the cached watermark - an optimization
that avoids re-walking a large repo's full history every run. On a
live, active repo, that "newest page" figure can change *during* the
walk: if any new event lands between the initial request and a later
one in the same walk, the true page count grows, page boundaries
shift, and whatever falls in the newly-revealed range is silently
never fetched at any page number at all. This is almost certainly what
happened to aklivity/zilla-plus#1073/#1075: both squash-merged into
support/1.x in the same rough window of repo activity as the release
run that missed them entirely, no error or warning anywhere.

Every page fetched during the walk now re-checks its own Link header
against the highest page number seen so far; if it's grown, the
newly-revealed pages are walked first (recursively, same early-stop
rule) before the original walk continues - covering drift discovered
at any point mid-walk, not only drift already present when the walk
started. Deduplicates via a seen-id set, since a page revealed by drift
detection and a page from the original walk can now overlap.

Adds dedicated test/github-driver.test.ts coverage for walkNewEvents,
previously untested entirely: single-page, multi-page with early stop
(and an assertion that pages beyond the stop point are never
requested), the drift regression case itself, and a repeated-drift
dedup case.

README: reframes the "completeness check" section - the pagination
fix is now the primary defense, the completeness check (previous
commit) the backstop for whatever this doesn't anticipate.
@jfallows jfallows changed the title feat(run): fail loudly when git history shows a merged PR the driver never fetched fix(drivers/github): make walkNewEvents resilient to pagination drift, add a completeness backstop Jul 25, 2026
@jfallows
jfallows merged commit 65d3260 into develop Jul 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants