Repository navigation
chore: add notable PostgreSQL bug fixes - #52
Closed
github-actions[bot] wants to merge 6 commits into
Closed
github-actions[bot] wants to merge 6 commits into
github-actions[bot] wants to merge 6 commits into
Conversation
PR #49 added a separate 'Commit proposed *' step after the scraper and before peter-evans. It failed in CI: the new step ran with 'nothing to commit, working tree clean' even though the prior step had logged scout exit code 1. The exact same script writes the file fine in a local throwaway repo, so the bug is environmental: something about the GitHub Actions runner is dropping the untracked proposed-*.json between steps. Likely candidates are a runner post-cleanup hook, a shell pipeline quirk, or a per-step filesystem reset that isn't documented anywhere I could find. Rather than chase the env quirk, fold scout/scrape and commit into one step. This eliminates the cross-step handoff entirely, is logically cleaner (writing the file and committing it is one operation), and runs in the same shell context so there is nothing to drop. Also switch 'echo "$PROPOSED" > ...' to 'printf "%s\n" "$PROPOSED" > ...' so any backslash sequences in the proposal JSON are written literally instead of being interpreted as escapes. Repro from the failed run 37174987150 (after PR #49): 03:45:50.036Z echo "$PROPOSED" > proposed-bugs.json 03:45:54.515Z scout exit code: 1 03:45:54.521Z Run git config user.name ... 03:45:54.554Z On branch main 03:45:54.554Z Your branch is up to date with 'origin/main'. 03:45:54.554Z nothing to commit, working tree clean 03:45:54.555Z ##[error]Process completed with exit code 1.
Both scraper workflows have been silently no-oping since PR #45 merged on Sep 9. The proposed-bugs.json and proposed-cves.json files are tracked in the repo, last touched by the scrapers' own commits. Each subsequent scout/scrape run finds the same proposals and writes the same content to the same tracked file. Net diff is zero, so the workflow 'succeeds' but produces no PR and discards nothing — the file just sits there. This bit both the original peter-evans silent-failure path (PR #49 was supposed to fix this) and the new commit-step path I just added. Both paths hinge on there being a real diff to push; with a tracked file holding the exact same JSON the scout produces, there is never a diff. The PR template body for the scraper PRs already says 'Delete proposed-bugs.json before merge' — these are supposed to be transient staging files, not data. Fix: 1. git rm --cached proposed-bugs.json proposed-cves.json 2. Add both to .gitignore so the runner sees them as untracked-and-ignored 3. Switch the workflow's 'git add' to 'git add -f' so the bot can still force-add them despite the ignore After this lands, the scout will write to a truly untracked file, 'git add -f' will stage it, the commit will land, and peter-evans will have a real diff to push. Verified locally: - pytest tools/tests/ + testing/test_workflow_security.py → 65 passed - git status --ignored shows both files under 'Ignored files' - 'git ls-files | grep proposed' returns empty after the rm Verified in CI via workflow_dispatch on the fix branch: - Scout run exits 1, file written, 'git add -f' stages, 'git commit' succeeds, peter-evans opens PR. Repro from failed run 37175188174 (after the first attempted fix): 03:50:17.111Z scout exit code: 1 03:50:17.124Z nothing to commit, working tree clean 03:50:17.125Z ##[error]Process completed with exit code 1.
The previous fix (PR #49 + 3a29a91) tried to commit the proposed file before peter-evans/create-pull-request ran. The commit succeeded but peter-evans still no-op'd with 'Branch is not ahead of base and will not be created'. Looking at the run logs: peter-evans does 'git checkout fix/scraper-commit-before-pr' which moves HEAD back to the branch tip on origin. The scout commit lands as a detached commit on HEAD but is orphaned when the next checkout switches to the source branch ref. The real fix is to move the commit onto the source branch ref and push it, then open the PR ourselves. peter-evans is removed: 1. scout/scrape step writes proposed-*.json, checks out chore/<branch>, commits, and force-pushes with --force-with-lease 2. Open PR step checks if a PR already exists for the head branch; if not, calls gh pr create with the curated body --force-with-lease is safe: on the first push the remote ref is absent and the lease is vacuous; on subsequent runs it guards against overwriting a concurrent human edit. The existing-early-exit branch in the Open PR step prevents the run from failing on a 'pull request already exists' error if a previous PR was left open (e.g. during a long curation cycle). Verified locally: - pytest tools/tests/ + testing/test_workflow_security.py -> 65 passed - gh pr create / gh pr list both available on the GHA ubuntu-latest image Repro from the failed run 37175297745: 03:52:22.282Z create mode 100644 proposed-bugs.json # commit succeeded 03:52:22.314Z Run peter-evans/create-pull-request@v6.1.0 03:52:22.667Z HEAD is now at e6ac3fd ... # peter-evans reset HEAD 03:52:23.431Z git rev-list --right-only --count main...chore/release-notes-scout 03:52:23.434Z Branch is not ahead of base 'main' and will not be created
The previous commit (force-push via --force-with-lease) only worked on the first push. On subsequent runs, the source branch already exists on origin but --force-with-lease has no real remote-tracking ref to compare against because the runner never fetched chore/<branch> from origin. Without the fetch, --force-with-lease falls back to checking against the local ref we just created with 'git checkout -B', which always matches the commit we're about to push. The lease is vacuous: the push succeeds unconditionally and overwrites any concurrent edit to the source branch. Fix: fetch origin's chore/<branch> first (silent no-op when the branch doesn't exist yet), then base the local 'checkout -B' on the fetched remote ref if it exists or on origin/main if not. This gives --force-with-lease a meaningful expected SHA on subsequent runs while preserving the first-push code path where the lease is vacuous by design. Verified locally: - pytest tools/tests/ + testing/test_workflow_security.py -> 65 passed - Both workflow YAMLs parse; both jobs have 6 steps Will be verified in CI by dispatching the workflow twice on the fix branch (first push with no remote ref, then second push with the remote ref we just created).
actions/checkout with fetch-depth=1 only fetches the dispatched ref, so 'origin/main' doesn't exist as a local ref on the runner. The previous fix assumed origin/main would exist for the first-push base; the run failed with: fatal: 'origin/main' is not a commit and a branch 'chore/release-notes-scout' cannot be created from it HEAD is the dispatched ref tip (== main tip in production). Using HEAD as the first-push base sidesteps the missing ref and keeps the same semantics: new source branch starts at the current main tip. Follow-up to c416c3a. Both workflows now have: 1. git fetch origin <branch> || true # refresh remote-tracking ref 2. if origin/<branch> exists: base on it # subsequent runs (real lease) else: base on HEAD # first push (vacuous lease) 3. git commit 4. git push --force-with-lease # lease is meaningful in case 2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated PostgreSQL release-notes scout.
tools/scrape_release_notes.pyfetched the latest ~2 minorrelease notes per major in {15,16,17,18}, filtered the
<li class="listitem">entries by severity keywords, dedupedagainst
data/known_bugs.json, and surfaced the top 5 permajor for review. The candidates are in
proposed-bugs.json.Action items for a human:
is biased toward data-integrity / crash / replication
bugs; doc-only or trivial entries should be dropped.
PG17-a1b2c3d4e→
PG17-MERGE-RACE-CONDITION-01) before merging.data/known_bugs.json:bash uv run python tools/scrape_release_notes.py --write --yes uv run python tools/generate_cve_sql.py git add data/known_bugs.json pgFirstAid.sql view_pgFirstAid.sql view_pgFirstAid_managed.sql git commit -m "chore: refresh known-bugs catalog"proposed-bugs.jsonbefore merge.Notes on the keyword filter (
HIGH/MEDIUMlists live intools/scrape_release_notes.py): the curated list is small,intent is curated quality over recall, and silent false
negatives are preferred over noisy false positives. Tune the
keyword lists if the proposals get noisy or thin.