Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions .github/actions/go-mod-download-retry/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: Download Go modules with retries
description: >-
Runs `go mod download` with bounded retries and exponential backoff so a
transient proxy.golang.org stream reset ("stream error: stream ID N;
INTERNAL_ERROR; received from peer", #14976) fills the module cache instead
of failing the job. Every later `go build`/`go test`/golangci-lint/
govulncheck/gosec step in the job then reads a warm GOMODCACHE and never
touches the proxy for the module graph. Mirrors the retry #14990 added to
the Docker builder stages (src/Dockerfile*). Go verifies every module
against go.sum regardless of how many attempts it took, so a retry can never
substitute a different artifact. A still-failing final attempt fails the
step exactly like the unguarded download did.
inputs:
working-directory:
description: Directory holding go.mod (and go.work, when present).
required: false
default: src
attempts:
description: Maximum number of `go mod download` attempts.
required: false
default: '5'
runs:
using: composite
steps:
- name: go mod download (retry transient proxy resets)
shell: bash
working-directory: ${{ inputs.working-directory }}
env:
ATTEMPTS: ${{ inputs.attempts }}
GOFLAGS: -mod=mod
run: |
set -uo pipefail
case "$ATTEMPTS" in
''|*[!0-9]*|0) echo "::error::attempts must be a positive integer, got '$ATTEMPTS'"; exit 2 ;;
esac
delay=2
for i in $(seq 1 "$ATTEMPTS"); do
rc=0
go mod download || rc=$?
if [ "$rc" -eq 0 ]; then
echo "go mod download succeeded on attempt $i/$ATTEMPTS"
exit 0
fi
if [ "$i" -eq "$ATTEMPTS" ]; then
echo "::error::go mod download failed after $ATTEMPTS attempts (last rc=$rc)"
exit "$rc"
fi
echo "::warning::go mod download failed (attempt $i/$ATTEMPTS, rc=$rc); retrying in ${delay}s"
sleep "$delay"
delay=$((delay * 2))
done
20 changes: 20 additions & 0 deletions .github/workflows/go-security-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,11 @@ jobs:
go-version-file: src/go.mod
- name: Guard shared Go cache (fall back when the runner volume is full)
run: bash "$GITHUB_WORKSPACE/.github/scripts/go-cache-guard.sh"
# Fill GOMODCACHE up front with bounded retries so a transient
# proxy.golang.org stream reset (INTERNAL_ERROR, #14976) cannot fail the
# job's first module-fetching step; see .github/actions/go-mod-download-retry.
- name: Download Go modules (retry transient proxy resets)
uses: ./.github/actions/go-mod-download-retry

# Installed by version rather than via a marketplace action so the tool
# itself is pinned and the supply chain is one hop shorter.
Expand Down Expand Up @@ -200,6 +205,11 @@ jobs:
go-version-file: src/go.mod
- name: Guard shared Go cache (fall back when the runner volume is full)
run: bash "$GITHUB_WORKSPACE/.github/scripts/go-cache-guard.sh"
# Fill GOMODCACHE up front with bounded retries so a transient
# proxy.golang.org stream reset (INTERNAL_ERROR, #14976) cannot fail the
# job's first module-fetching step; see .github/actions/go-mod-download-retry.
- name: Download Go modules (retry transient proxy resets)
uses: ./.github/actions/go-mod-download-retry

# Version floor is load-bearing: gosec v2.21.4 pins golang.org/x/tools
# v0.25.0, which does not COMPILE under Go 1.25 ("invalid array length").
Expand Down Expand Up @@ -254,6 +264,11 @@ jobs:
go-version-file: src/go.mod
- name: Guard shared Go cache (fall back when the runner volume is full)
run: bash "$GITHUB_WORKSPACE/.github/scripts/go-cache-guard.sh"
# Fill GOMODCACHE up front with bounded retries so a transient
# proxy.golang.org stream reset (INTERNAL_ERROR, #14976) cannot fail the
# job's first module-fetching step; see .github/actions/go-mod-download-retry.
- name: Download Go modules (retry transient proxy resets)
uses: ./.github/actions/go-mod-download-retry

# skip-cache: the action's own cache save/restore is the same multi-GB
# tar problem as setup-go's; the runners already keep GOCACHE warm.
Expand Down Expand Up @@ -343,6 +358,11 @@ jobs:
go-version-file: src/go.mod
- name: Guard shared Go cache (fall back when the runner volume is full)
run: bash "$GITHUB_WORKSPACE/.github/scripts/go-cache-guard.sh"
# Fill GOMODCACHE up front with bounded retries so a transient
# proxy.golang.org stream reset (INTERNAL_ERROR, #14976) cannot fail the
# job's first module-fetching step; see .github/actions/go-mod-download-retry.
- name: Download Go modules (retry transient proxy resets)
uses: ./.github/actions/go-mod-download-retry

- name: Regenerate NOTICE from the current module graph
run: src/scripts/generate-notice.sh /tmp/NOTICE.generated
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/testutil-guard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,11 @@ jobs:
go-version-file: src/go.mod
- name: Guard shared Go cache (fall back when the runner volume is full)
run: bash "$GITHUB_WORKSPACE/.github/scripts/go-cache-guard.sh"
# Fill GOMODCACHE up front with bounded retries so a transient
# proxy.golang.org stream reset (INTERNAL_ERROR, #14976) cannot fail the
# job's first module-fetching step; see .github/actions/go-mod-download-retry.
- name: Download Go modules (retry transient proxy resets)
uses: ./.github/actions/go-mod-download-retry

- name: internal/testutil coverage floor
working-directory: src
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/v2-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,11 @@ jobs:
go-version-file: src/go.mod
- name: Guard shared Go cache (fall back when the runner volume is full)
run: bash "$GITHUB_WORKSPACE/.github/scripts/go-cache-guard.sh"
# Fill GOMODCACHE up front with bounded retries so a transient
# proxy.golang.org stream reset (INTERNAL_ERROR, #14976) cannot fail the
# job's first module-fetching step; see .github/actions/go-mod-download-retry.
- name: Download Go modules (retry transient proxy resets)
uses: ./.github/actions/go-mod-download-retry

# Warm compile cache (#7480 follow-up): the warm-build-cache job on every
# v4 push compiles the whole tree (race + norace) and saves GOCACHE keyed
Expand Down
46 changes: 46 additions & 0 deletions .github/workflows/v2-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,12 @@ jobs:
- name: Guard shared Go cache (fall back when the runner volume is full)
if: steps.select.outputs.run == 'true'
run: bash "$GITHUB_WORKSPACE/.github/scripts/go-cache-guard.sh"
# Fill GOMODCACHE up front with bounded retries so a transient
# proxy.golang.org stream reset (INTERNAL_ERROR, #14976) cannot fail the
# job's first module-fetching step; see .github/actions/go-mod-download-retry.
- name: Download Go modules (retry transient proxy resets)
if: steps.select.outputs.run == 'true'
uses: ./.github/actions/go-mod-download-retry

- name: Install gotestsum
if: steps.select.outputs.run == 'true'
Expand Down Expand Up @@ -600,6 +606,12 @@ jobs:
- name: Guard shared Go cache (fall back when the runner volume is full)
if: steps.select.outputs.run == 'true'
run: bash "$GITHUB_WORKSPACE/.github/scripts/go-cache-guard.sh"
# Fill GOMODCACHE up front with bounded retries so a transient
# proxy.golang.org stream reset (INTERNAL_ERROR, #14976) cannot fail the
# job's first module-fetching step; see .github/actions/go-mod-download-retry.
- name: Download Go modules (retry transient proxy resets)
if: steps.select.outputs.run == 'true'
uses: ./.github/actions/go-mod-download-retry

- name: Install gotestsum
if: steps.select.outputs.run == 'true'
Expand Down Expand Up @@ -871,6 +883,12 @@ jobs:
- name: Guard shared Go cache (fall back when the runner volume is full)
if: steps.select.outputs.run == 'true'
run: bash "$GITHUB_WORKSPACE/.github/scripts/go-cache-guard.sh"
# Fill GOMODCACHE up front with bounded retries so a transient
# proxy.golang.org stream reset (INTERNAL_ERROR, #14976) cannot fail the
# job's first module-fetching step; see .github/actions/go-mod-download-retry.
- name: Download Go modules (retry transient proxy resets)
if: steps.select.outputs.run == 'true'
uses: ./.github/actions/go-mod-download-retry

- name: Install gotestsum
if: steps.select.outputs.run == 'true'
Expand Down Expand Up @@ -1107,6 +1125,11 @@ jobs:
go-version-file: src/go.mod
- name: Guard shared Go cache (fall back when the runner volume is full)
run: bash "$GITHUB_WORKSPACE/.github/scripts/go-cache-guard.sh"
# Fill GOMODCACHE up front with bounded retries so a transient
# proxy.golang.org stream reset (INTERNAL_ERROR, #14976) cannot fail the
# job's first module-fetching step; see .github/actions/go-mod-download-retry.
- name: Download Go modules (retry transient proxy resets)
uses: ./.github/actions/go-mod-download-retry

# Warm compile cache (#7480 follow-up): the warm-build-cache job on every
# v4 push compiles the whole tree (race + norace) and saves GOCACHE keyed
Expand Down Expand Up @@ -1444,6 +1467,12 @@ jobs:
- name: Guard shared Go cache (fall back when the runner volume is full)
if: steps.select.outputs.run == 'true'
run: bash "$GITHUB_WORKSPACE/.github/scripts/go-cache-guard.sh"
# Fill GOMODCACHE up front with bounded retries so a transient
# proxy.golang.org stream reset (INTERNAL_ERROR, #14976) cannot fail the
# job's first module-fetching step; see .github/actions/go-mod-download-retry.
- name: Download Go modules (retry transient proxy resets)
if: steps.select.outputs.run == 'true'
uses: ./.github/actions/go-mod-download-retry

- name: Install gotestsum
if: steps.select.outputs.run == 'true'
Expand Down Expand Up @@ -1659,6 +1688,12 @@ jobs:
- name: Guard shared Go cache (fall back when the runner volume is full)
if: steps.select.outputs.run == 'true'
run: bash "$GITHUB_WORKSPACE/.github/scripts/go-cache-guard.sh"
# Fill GOMODCACHE up front with bounded retries so a transient
# proxy.golang.org stream reset (INTERNAL_ERROR, #14976) cannot fail the
# job's first module-fetching step; see .github/actions/go-mod-download-retry.
- name: Download Go modules (retry transient proxy resets)
if: steps.select.outputs.run == 'true'
uses: ./.github/actions/go-mod-download-retry

- name: Install gotestsum
if: steps.select.outputs.run == 'true'
Expand Down Expand Up @@ -2028,6 +2063,12 @@ jobs:
cache-dependency-path: |
src/go.sum
src/crux/go.sum
# Fill GOMODCACHE up front with bounded retries so a transient
# proxy.golang.org stream reset (INTERNAL_ERROR, #14976) cannot fail the
# job's first module-fetching step; see .github/actions/go-mod-download-retry.
- name: Download Go modules (retry transient proxy resets)
if: ${{ steps.cov-scope.outputs.score == 'true' }}
uses: ./.github/actions/go-mod-download-retry

- name: List packages present in checkout
if: ${{ steps.cov-scope.outputs.score == 'true' }}
Expand Down Expand Up @@ -2664,6 +2705,11 @@ jobs:
go-version-file: src/go.mod
- name: Guard shared Go cache (fall back when the runner volume is full)
run: bash "$GITHUB_WORKSPACE/.github/scripts/go-cache-guard.sh"
# Fill GOMODCACHE up front with bounded retries so a transient
# proxy.golang.org stream reset (INTERNAL_ERROR, #14976) cannot fail the
# job's first module-fetching step; see .github/actions/go-mod-download-retry.
- name: Download Go modules (retry transient proxy resets)
uses: ./.github/actions/go-mod-download-retry

- name: Restore warm Go build cache
if: ${{ !contains(vars.HIVE_RUNNER_LABELS, 'hive-runners') }}
Expand Down
1 change: 1 addition & 0 deletions changelog.d/fixed-14976-ci-go-mod-download-retry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- CI Go jobs (`v2-ci`, `v2-tests`, `go-security-analysis`, `testutil-guard`) now pre-fill the module cache through a shared `go-mod-download-retry` action that retries `go mod download` with exponential backoff, so a transient `proxy.golang.org` stream reset (`INTERNAL_ERROR`) no longer fails golangci-lint, govulncheck, gosec or the sharded tests; this completes the Docker builder-stage retry from #14990 ([#14976](https://github.com/hivecommons/hive/issues/14976)).
Loading