Skip to content

ci: retry the weaver download on transient 5xx - #187

Merged
eyelock merged 1 commit into
developfrom
ci/retry-weaver-download
Nov 7, 2026
Merged

eyelock merged 1 commit into
developfrom
ci/retry-weaver-download

Conversation

@eyelock

@eyelock eyelock commented Nov 7, 2026

Copy link
Copy Markdown
Owner

Gahan · ynw worker seat

The registry job's weaver download now retries on transient 5xx errors, so an upstream 504 no longer fails the job.

Cause: ynf run 42056792864 failed on develop (2026-11-05 15:37 UTC) when GitHub release downloads returned 504. No later ci run on develop has happened, so the failure is not disproved. The check job's 504 came from golangci-lint-action, which already retries (it gave up after 3 tries); that is outside this repository's code. The registry job's bare curl | tar had no retry at all.

Change: The registry step in .github/workflows/ci.yml downloads to a file with --retry 5 --retry-delay 10 --retry-all-errors, then extracts, so a retry never feeds tar a partial stream.

Not verified locally: this is workflow-only, and the first CI run on this PR is the test. The check job's golangci-lint download can still fail in a long outage; no change was made there.

Closes #186

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@eyelock eyelock added this to the v0.4.1 milestone Nov 7, 2026

@eyelock eyelock left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hannett · ynw review seat

Verdict: ✅ ready to merge
Merge with: squash

The weaver step now retries 5xx with --retry-all-errors, and it downloads to a file before tar reads it, which removes the partial-stream failure seen in the red run. All four jobs pass on this PR (ynf run 42360850691).

On the golangci-lint download: I don't think it leaves the issue's "done" unmet. The failed check job's log shows the action already retried, with three 504s over about 26 seconds before it gave up. So that job didn't fail on a single upstream timeout, which is the bar ynf #186 sets. It would take a longer outage to break it, and that seems worth a follow-up only if it happens again. No fix bead.

The other half of "done" is ci on develop going green, and that comes from the first develop run after this merges. Develop's only red run, 42056792864, was never re-run.

Something I noticed, not blocking: the conformance job's three gh release download calls fetch from the same GitHub release hosting with no retry. Could they hit the same 504 one day?

@eyelock eyelock added the merge: squash Merge this pull request with squash label Nov 7, 2026
@eyelock
eyelock marked this pull request as ready for review November 7, 2026 09:47
@eyelock
eyelock merged commit e5acfec into develop Nov 7, 2026
4 checks passed
@eyelock
eyelock deleted the ci/retry-weaver-download branch November 7, 2026 10:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge: squash Merge this pull request with squash

Projects

None yet

Development

Successfully merging this pull request may close these issues.

develop CI fails: registry and check jobs get HTTP 504 fetching downloads

1 participant