From 142375c0e0972f664b37d9a9f10e06a38c040174 Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Mon, 10 Aug 2026 15:53:06 -0400 Subject: [PATCH 01/19] fix(ci): cache Go modules once, not once per compile flavor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ~/go/pkg/mod is a pure function of go.sum — byte-identical for every compile flavor — but it was cached together with ~/.cache/go-build under a key partitioned by go-cache-suffix. That stored the same 1.6 GB five times (-lint, -unit, -integration, -e2e-cov, -cov), ~5 GB per go.sum generation. Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat at ~10 GB against GitHub's hard 10 GB cap; #438's 24-module bump tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. Split into gomod-v1 (~/go/pkg/mod, unsuffixed, shared by every Go job) and gobuild-v3 (~/.cache/go-build only, still per flavor). ~5 GB -> ~2 GB per generation, so two generations fit with headroom. The v3 bump is required: saves fire only on an exact-key miss, so without it the old v2 entry — still carrying the module cache — would exact-hit forever and the smaller content would never be saved. gobuild-v3 also drops the bare-prefix restore-key, which existed only to borrow another flavor's copy of the module cache; that job is now gomod-v1's, and another flavor's build objects aren't reusable. Documents the sizing constraint in the workflows README so the next cache addition budgets for two generations against the 10 GB cap. Closes #443 Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/actions/setup-env/action.yml | 81 ++++++++++++++++++---------- .github/workflows/README.md | 26 ++++++++- 2 files changed, 79 insertions(+), 28 deletions(-) diff --git a/.github/actions/setup-env/action.yml b/.github/actions/setup-env/action.yml index 42d9c6b6..4efd9376 100644 --- a/.github/actions/setup-env/action.yml +++ b/.github/actions/setup-env/action.yml @@ -14,29 +14,30 @@ # # The cache inventory (full architecture: .github/workflows/README.md): # -# 1. Go module + build cache keyed on every go.sum, `gobuild-v2-` key -# family. `restore-keys` keeps the build cache warm across go.sum -# bumps — avoids setup-go's exact-key-only cache (actions/setup-go#357). -# `go-cache-suffix` partitions the cache per job: the unit, -# integration, and e2e jobs compile with different flags -# (-race/-cover/-coverpkg/-tags), so a shared entry would only ever -# be warm for whichever job saved it. Cross-suffix restore-keys -# still share the (identical) module cache on a cold start. -# 2. golangci-lint binary + analysis cache keyed on Makefile + +# 1. Go modules, `gomod-v1-` key family, keyed on every go.sum and NOT +# partitioned per job — ~/go/pkg/mod is a pure function of go.sum, so +# one entry serves every Go job (~1.6 GB stored once, not per flavor). +# 2. Go build objects, `gobuild-v3-` key family, keyed on every go.sum +# and partitioned by `go-cache-suffix`: the unit, integration and e2e +# jobs compile with different flags (-race/-cover/-coverpkg/-tags), so +# a shared entry would only ever be warm for whichever job saved it. +# `restore-keys` keeps it warm across go.sum bumps — avoids setup-go's +# exact-key-only cache (actions/setup-go#357). +# 3. golangci-lint binary + analysis cache keyed on Makefile + # .golangci.yml. Analysis cache is the win (~10s warm vs ~90s). # Only the lint job needs it. -# 3. pnpm store (path resolved at runtime) keyed on the root +# 4. pnpm store (path resolved at runtime) keyed on the root # lockfile — the pnpm workspace projects share one lockfile + # store. Path is dynamic because pnpm's documented default # ~/.local/share/pnpm/store only applies when $HOME and the project # tree share a mount; on some runners it falls back to a # workspace-relative path. Hard-coding the default silently fails # the save with a Path Validation Error. -# 4. Playwright browser cache (~/.cache/ms-playwright) keyed on the +# 5. Playwright browser cache (~/.cache/ms-playwright) keyed on the # root lockfile. ~130 MB Chromium download otherwise re-fetched # every docs build (rehype-mermaid renders via headless Chrome). # Only the docs-build job needs it. See #132. -# 5. Astro content-collection cache (docs/.astro/) keyed on the root +# 6. Astro content-collection cache (docs/.astro/) keyed on the root # lockfile + astro.config.mjs. Speeds up warm `astro check` / # `astro build` — unchanged content skips the parse + transform # pipeline. See #132. @@ -75,26 +76,52 @@ outputs: runs: using: composite steps: - # Key version (v2): bumped when the cache's expected CONTENTS change - # shape — v2 added the go toolchain itself (~/go/pkg/mod/golang.org/ - # toolchain, see the GOTOOLCHAIN note below). Saves only happen on an - # exact-key miss, so without a bump the pre-change entry would - # exact-hit forever and the new content would never be saved. + # The module cache is UNSUFFIXED on purpose. ~/go/pkg/mod is a pure + # function of go.sum — byte-identical for every compile flavor — so + # folding it into the suffixed build cache stored the same ~1.6 GB + # five times (once per -lint/-unit/-integration/-e2e-cov/-cov job). + # That is ~5 GB per go.sum generation against a 10 GB repo cap, so two + # live generations overflowed it and GitHub started LRU-evicting warm + # entries mid-run. Splitting it out stores it once (#443). + # + # All Go jobs miss this key together on a go.sum bump and all try to + # save; the backend keeps the first and the rest log a benign "already + # exists" (same trade-off as the rest of the inventory, see header). + - uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 + if: ${{ inputs.go == 'true' }} + id: gomod-cache + with: + path: ~/go/pkg/mod + # v1: the GOTOOLCHAIN=auto toolchain rides in here too (no + # setup-go) — that content was what forced the old v2 bump. + key: gomod-v1-${{ runner.os }}-${{ hashFiles('**/go.sum') }} + # A stale generation is still worth restoring: go.sum bumps move a + # handful of modules, so most of the tree is reusable and `go mod + # download` fetches only the delta. + restore-keys: | + gomod-v1-${{ runner.os }}- + + # Build objects only. These genuinely differ per job — unit, + # integration and e2e compile with different flags + # (-race/-cover/-coverpkg/-tags) — so a shared entry would only ever be + # warm for whichever job saved it last. + # + # Key version (v3): bumped because the cache's CONTENTS changed shape + # (~/go/pkg/mod moved out, above). Saves only happen on an exact-key + # miss, so without a bump the old v2 entry would exact-hit forever and + # the new, smaller content would never be saved. - uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 if: ${{ inputs.go == 'true' }} id: gobuild-cache with: - path: | - ~/go/pkg/mod - ~/.cache/go-build - key: gobuild-v2-${{ runner.os }}-go${{ inputs.go-cache-suffix }}-${{ hashFiles('**/go.sum') }} - # Same-suffix prefix first (this job's flavor across go.sum bumps), - # then the bare prefix as a cold-start fallback — it matches any - # other job's suffixed entry, which still carries the shared module - # cache even if its build objects don't apply. + path: ~/.cache/go-build + key: gobuild-v3-${{ runner.os }}-go${{ inputs.go-cache-suffix }}-${{ hashFiles('**/go.sum') }} + # Same-suffix only. The bare-prefix fallback the v2 key carried + # existed to pick up the shared module cache from another job's + # entry; that job is now gomod-v1's, and another flavor's build + # objects are not reusable here. restore-keys: | - gobuild-v2-${{ runner.os }}-go${{ inputs.go-cache-suffix }}- - gobuild-v2-${{ runner.os }}-go- + gobuild-v3-${{ runner.os }}-go${{ inputs.go-cache-suffix }}- - uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 if: ${{ inputs.golangci == 'true' }} diff --git a/.github/workflows/README.md b/.github/workflows/README.md index e6867cd8..84b08212 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -183,7 +183,8 @@ Queue settings live in the `main branch protection` ruleset's | Cache | Key | Saved by | Notes | |---|---|---|---| -| Go modules + build | `gobuild-v2--go-` | every Go job (own suffix) | Suffix partitions by compile flavor (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`). v2 = the GOTOOLCHAIN=auto toolchain rides in `~/go/pkg/mod` (no setup-go). | +| Go modules | `gomod-v1--` | every Go job (shared) | `~/go/pkg/mod`, **unsuffixed** — a pure function of `go.sum`, so one ~1.6 GB entry serves every job. The GOTOOLCHAIN=auto toolchain rides in here too (no setup-go). | +| Go build objects | `gobuild-v3--go-` | every Go job (own suffix) | `~/.cache/go-build` only. Suffix partitions by compile flavor (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`), which compile with different flags. | | golangci binary + analysis | `golangci--` | lint | Analysis cache: ~10s warm vs ~90s. `.bin` also carries shellcheck + actionlint. | | pnpm store | `pnpm--` | any node job on miss | Store path resolved from pnpm at runtime. docs-build prunes before its save on a key rotation. | | Playwright Chromium | `playwright--` | docs-build | rehype-mermaid renders via headless Chrome at docs build. | @@ -195,6 +196,29 @@ so without a bump the old entry exact-hits forever and the new content is never captured. Keep the old prefixes as transitional restore-keys, then delete them once main has saved the new version. +**Sizing policy — the repo cache budget is 10 GB, hard.** Past it GitHub +LRU-evicts, so warm entries disappear mid-run and builds silently get +slower. Budget for **two live generations**: a `go.sum` or lockfile bump +mints a whole new set while the previous one is still warm, so the steady +state is ~2× a single generation. That is why `~/go/pkg/mod` is cached +**once** (`gomod-v1`) rather than folded into each suffixed build cache — +doing the latter stored the same ~1.6 GB five times, ~5 GB per +generation, and #438's 24-module bump pushed the repo to 10.53 GB +([#443](https://github.com/Wave-RF/WaveHouse/issues/443)). + +Before adding a cache or widening an existing `path:`, check the current +footprint and confirm two generations still fit: + +```bash +gh api repos/Wave-RF/WaveHouse/actions/cache/usage \ + -q '"\(.active_caches_size_in_bytes/1073741824*100|round/100)GB / 10GB"' +gh api repos/Wave-RF/WaveHouse/actions/caches --paginate \ + -q '.actions_caches[]|"\(.size_in_bytes)\t\(.key)"' | sort -rn | head +``` + +Never add a per-job copy of content that is a pure function of a lockfile +— key it once, unsuffixed, and let every job share it. + ## Timing (steady state, full pipeline) The non-gating **Timing summary** job writes a per-job wall-clock table From a39dc71b16cac716a1056d908120f7b2e0ecdb93 Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Mon, 10 Aug 2026 16:17:41 -0400 Subject: [PATCH 02/19] fix(ci): stop setup-go minting a sixth copy of the module cache MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review follow-up. The split consolidated five copies of ~/go/pkg/mod but left a sixth outside setup-env: actions/setup-go caches by default, and publish-dev.yml runs on every push to main, so its go.sum-keyed entry was ~1 GB live — more than the gomod-v1 entry this PR consolidates to, and a direct violation of the sizing rule the PR adds. release.yml carries the same default. Both now pass cache: false, matching the call goreleaser-validate.yml already made. Also documents what the inventory was hiding: the CodeQL caches GHAS default setup mints outside this repo's workflows (~0.4 GB), so a maintainer doing the two-generations check from the table sees the whole budget rather than 85% of it. Adds the narrowing-rotation exception to the key-versioning policy. The policy said to keep old prefixes as transitional restore-keys; gobuild-v3 deliberately does not, because a v2 archive still carries ~/go/pkg/mod and restoring it would re-materialize exactly what the rotation removes. The doc now prescribes what the code does. CHANGELOG entry for the whole change per AGENTS.md Documentation Sync. Pre-existing docs drift the gate surfaced is tracked in #444, not folded in here. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/workflows/README.md | 16 ++++++++++++++++ .github/workflows/publish-dev.yml | 9 +++++++++ .github/workflows/release.yml | 5 +++++ CHANGELOG.md | 2 ++ 4 files changed, 32 insertions(+) diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 84b08212..f6cd9ca8 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -189,6 +189,14 @@ Queue settings live in the `main branch protection` ruleset's | pnpm store | `pnpm--` | any node job on miss | Store path resolved from pnpm at runtime. docs-build prunes before its save on a key rotation. | | Playwright Chromium | `playwright--` | docs-build | rehype-mermaid renders via headless Chrome at docs build. | | Astro content collections | `astro--` | lint / docs-build | Warm `astro check`/`build` skip unchanged content. | +| CodeQL DB + deps | `codeql-dependencies-*`, `codeql-overlay-base-database-*` | GHAS default setup | **Not ours** — minted by GitHub's default CodeQL setup, not by any workflow in this repo, and not configurable here. ~0.4 GB. Listed so the budget arithmetic below is honest. | + +Deliberately **not** cached: `actions/setup-go`'s bundled cache in +`publish-dev.yml`, `release.yml` and `goreleaser-validate.yml` (`cache: false` +on each). It stores its own go.sum-keyed copy of `~/go/pkg/mod` + +`~/.cache/go-build` — ~1 GB, i.e. a seventh copy of what `gomod-v1` already +holds once — for release builds dominated by cross-compiling and multi-arch +docker rather than by `go mod download`. Key-versioning policy: bump the `v` prefix whenever the cache's expected *contents* change shape — saves only fire on an exact-key miss, @@ -196,6 +204,14 @@ so without a bump the old entry exact-hits forever and the new content is never captured. Keep the old prefixes as transitional restore-keys, then delete them once main has saved the new version. +**Exception — a rotation that *narrows* `path:` carries no transitional +restore-key.** The old archive still contains the paths you just removed, +so restoring it would re-materialize exactly the content the rotation was +meant to stop storing (and, for `~/go/pkg/mod`, extract 0444 module files +over an already-restored tree). Drop the old prefix and purge the stale +entries instead — they hold budget the new keys need. `gobuild-v3` is the +worked example: it kept only its own same-suffix prefix. + **Sizing policy — the repo cache budget is 10 GB, hard.** Past it GitHub LRU-evicts, so warm entries disappear mid-run and builds silently get slower. Budget for **two live generations**: a `go.sum` or lockfile bump diff --git a/.github/workflows/publish-dev.yml b/.github/workflows/publish-dev.yml index 97497b15..c326c337 100644 --- a/.github/workflows/publish-dev.yml +++ b/.github/workflows/publish-dev.yml @@ -50,6 +50,15 @@ jobs: - uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0 with: go-version-file: "go.mod" + # setup-go's bundled cache stores ~/go/pkg/mod + ~/.cache/go-build + # under its own go.sum-keyed entry — a private ~1 GB copy of the + # module tree that ci.yml already caches once as `gomod-v1`, and + # that this workflow mints afresh on every push to main. Two live + # generations is ~2 GB of the repo's hard 10 GB cache budget, for a + # release build that runs once per push and is dominated by + # cross-compiling + multi-arch docker, not by `go mod download`. + # Same call as goreleaser-validate.yml. See #443. + cache: false # dockers_v2 builds the linux/amd64+arm64 manifest via `docker buildx`; # the GitHub-hosted runner's default docker driver can't build diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 9a313b46..dfc337cb 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -24,6 +24,11 @@ jobs: - uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0 with: go-version-file: "go.mod" + # Tag-triggered, so a warm cache is almost never there to hit + # anyway; opting out keeps a ~1 GB go.sum-keyed copy of the module + # tree out of the 10 GB repo budget. Same call as publish-dev.yml + # and goreleaser-validate.yml. See #443. + cache: false # dockers_v2 builds the linux/amd64+arm64 manifest via `docker buildx`; # the GitHub-hosted runner's default docker driver can't build diff --git a/CHANGELOG.md b/CHANGELOG.md index 3db0d718..fdd5f430 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,6 +50,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.sum` and byte-identical for every flavor — so the same ~1.6 GB was stored five times (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`), ~5 GB per `go.sum` generation. Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. ~5 GB → ~2 GB per generation. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was minting a sixth ~1 GB `go.sum`-keyed copy of the same module tree on every push to main. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. + - **Live demo hero feed renders in `event_ts` order instead of SSE arrival order** (`docs/src/components/LiveDemo.astro`): the landing-page live activity feed prepended each streamed row to the top in the order the SSE stream delivered it, but a producer's webhook burst (a single merge-queue cycle fires ~20 events) arrives in no guaranteed order and the stream relays it in ingest order — so a late or out-of-order delivery landed above newer rows (e.g. a `pushed 12m ago` sitting on top of `reviewed a pull request 9m ago`). `addRow` now keeps the feed sorted by `event_ts` descending — it slots each row in before the first strictly-older sibling rather than blind-prepending — so the live tail matches the already-sorted `gh_activity_recent` backfill. The zone-less-SSE-timestamp normalization the sort relies on (`normTs`) was already in place; equal-second rows keep arrival order (`gh_events.event_ts` is only second-granular for CI/checks, so there's no finer tiebreak), and dedup + the `MAX_ROWS` trim are unchanged. Surfaced in dogfooding on `wavehouse.dev`; the client-side analog of the ingest-order reality the SSE stream can't reorder. - **SSE streams emit a periodic keepalive comment so quiet connections survive proxy and tunnel idle timeouts** (`internal/stream/` (new package: `subscriber.go`, `bucket.go`, `heartbeat.go` + tests), `internal/api/{stream,stream_test}.go`, `internal/config/{config,config_test}.go`, `cmd/wavehouse/main.go`, `config.yaml`, `docs/src/content/docs/{reverse-proxy.mdx,api.md,configuration.mdx,architecture.md,deployment.md}`): closes #226. The stream handler (`internal/stream/metrics.go`, `internal/api/{hub,hub_test}.go` also touched) wrote a single `: connected` comment on open and then sent nothing until an event arrived, so on a quiet table an intermediary's idle timeout reset the connection — Cloudflare's edge (and a Cloudflare Tunnel) dropped quiet streams about every two minutes in dogfooding, and every `curl`/server-side reconnect re-ran NATS gap-fill (browser `EventSource` masked it by auto-reconnecting). A single shared `Heartbeater` goroutine now drives keepalives for every live connection: connections are spread across a ring of buckets and one bucket is pushed a minimal `:` SSE keepalive comment per tick — the writes don't all fire at the same instant, and the per-connection period comes from one timer instead of a `time.Ticker` per connection. The user-facing knob is **`stream.keepalive_interval`** (`WH_STREAM_KEEPALIVE_INTERVAL`, default **30s**) — the longest a quiet stream goes without a write — chosen to clear the common 55–60s idle windows (nginx/ingress-nginx `proxy_read_timeout`, AWS ALB, Heroku) with ~2× margin while still clearing Cloudflare's ~120s edge; `stream.keepalive_buckets` (`WH_STREAM_KEEPALIVE_BUCKETS`, default 3) is an advanced load-spreading knob and the wheel ticks every `keepalive_interval ÷ keepalive_buckets`, so one rotation always spans exactly the interval regardless of bucket count. The wheel lives in a new `internal/stream` package (groundwork for #294, which will move the broadcast hub in alongside it) and exposes a `Bucket` interface (push a byte slice to a set of connections) so the #294 delivery-path throughput work can reuse it to serialize once per (role, table) rather than per subscriber. The keepalive is a standard SSE comment (ignored by `EventSource` and spec-compliant parsers; `curl` just prints it), and a failed keepalive write doubles as a liveness check that ends the handler once a connection has gone away. The reverse-proxy guide gains a per-provider/per-software idle-timeout reference table (with source links and how-to-change notes, plus the absolute-cap exceptions a keepalive can't fix, e.g. Envoy's 15s route timeout), and `-race` tests cover the concurrent connect/disconnect teardown path (the wheel pushing to a connection that is mid-teardown); raising the proxy idle timeout for `/v1/stream` is now optional rather than required. Streams are observed through metrics rather than per-event traces: the handler records `wavehouse_sse_active_streams`, `wavehouse_sse_stream_duration_seconds`, and `wavehouse_sse_frames_sent_total` / `wavehouse_sse_bytes_sent_total` (labeled by `kind`: `keepalive`, `event`, `replay`), and the per-event `SSE.PushEvent` span was dropped — the router already excludes `/v1/stream` from HTTP tracing, and a span per delivered event per subscriber is high-volume, low-value, and existed only to read the hub's `trace_headers` envelope, now collapsed so the hub broadcasts the raw event bytes instead of base64-wrapping them. From 1f18ead03c8118105f01b058a0d52f114057dce8 Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Mon, 10 Aug 2026 16:33:38 -0400 Subject: [PATCH 03/19] docs(ci): state cache sizes in stored-archive units MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review follow-up. The sizing policy mixed on-disk size with stored-archive size, so the arithmetic it tells the next maintainer to perform didn't close: ~1.6 GB (a du of ~/go/pkg/mod) times five entries is 8 GB, not the ~5 GB quoted. What counts against the 10 GB cap is the compressed entry. Live API data: the five gobuild-v2 entries measure 0.97-1.27 GB each (5.57 GB total) and the setup-go entry holding the same module tree is 1.045 GB. So the tree is ~1.6 GB on disk and ~1 GB stored, and a generation is five entries of ~1.1 GB — ~5.5 GB. Budget statements now quote stored sizes, with the on-disk figure kept only where it explains what the directory actually contains. Same wording in the action header, the README table and policy, and the CHANGELOG. Also rewords "a seventh copy" of the setup-go cache, which contradicted the "sixth copy" in the CHANGELOG and the previous commit subject; the ordinal depended on whether you counted pre- or post-split, so it now just says it duplicates what gomod-v1 holds once. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/actions/setup-env/action.yml | 10 ++++++---- .github/workflows/README.md | 13 +++++++------ CHANGELOG.md | 2 +- 3 files changed, 14 insertions(+), 11 deletions(-) diff --git a/.github/actions/setup-env/action.yml b/.github/actions/setup-env/action.yml index 4efd9376..a0ac3263 100644 --- a/.github/actions/setup-env/action.yml +++ b/.github/actions/setup-env/action.yml @@ -16,7 +16,8 @@ # # 1. Go modules, `gomod-v1-` key family, keyed on every go.sum and NOT # partitioned per job — ~/go/pkg/mod is a pure function of go.sum, so -# one entry serves every Go job (~1.6 GB stored once, not per flavor). +# one entry serves every Go job, stored once rather than per flavor +# (~1.6 GB on disk, ~1 GB as a stored archive). # 2. Go build objects, `gobuild-v3-` key family, keyed on every go.sum # and partitioned by `go-cache-suffix`: the unit, integration and e2e # jobs compile with different flags (-race/-cover/-coverpkg/-tags), so @@ -78,9 +79,10 @@ runs: steps: # The module cache is UNSUFFIXED on purpose. ~/go/pkg/mod is a pure # function of go.sum — byte-identical for every compile flavor — so - # folding it into the suffixed build cache stored the same ~1.6 GB - # five times (once per -lint/-unit/-integration/-e2e-cov/-cov job). - # That is ~5 GB per go.sum generation against a 10 GB repo cap, so two + # folding it into the suffixed build cache stored that tree five times + # over (once per -lint/-unit/-integration/-e2e-cov/-cov job) — five + # entries of ~1.1 GB stored each, ~5.5 GB per go.sum generation. + # Against a 10 GB repo cap that means two # live generations overflowed it and GitHub started LRU-evicting warm # entries mid-run. Splitting it out stores it once (#443). # diff --git a/.github/workflows/README.md b/.github/workflows/README.md index f6cd9ca8..16e680c6 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -183,7 +183,7 @@ Queue settings live in the `main branch protection` ruleset's | Cache | Key | Saved by | Notes | |---|---|---|---| -| Go modules | `gomod-v1--` | every Go job (shared) | `~/go/pkg/mod`, **unsuffixed** — a pure function of `go.sum`, so one ~1.6 GB entry serves every job. The GOTOOLCHAIN=auto toolchain rides in here too (no setup-go). | +| Go modules | `gomod-v1--` | every Go job (shared) | `~/go/pkg/mod`, **unsuffixed** — a pure function of `go.sum`, so one entry serves every job (~1.6 GB on disk, ~1 GB stored). The GOTOOLCHAIN=auto toolchain rides in here too (no setup-go). | | Go build objects | `gobuild-v3--go-` | every Go job (own suffix) | `~/.cache/go-build` only. Suffix partitions by compile flavor (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`), which compile with different flags. | | golangci binary + analysis | `golangci--` | lint | Analysis cache: ~10s warm vs ~90s. `.bin` also carries shellcheck + actionlint. | | pnpm store | `pnpm--` | any node job on miss | Store path resolved from pnpm at runtime. docs-build prunes before its save on a key rotation. | @@ -194,9 +194,9 @@ Queue settings live in the `main branch protection` ruleset's Deliberately **not** cached: `actions/setup-go`'s bundled cache in `publish-dev.yml`, `release.yml` and `goreleaser-validate.yml` (`cache: false` on each). It stores its own go.sum-keyed copy of `~/go/pkg/mod` + -`~/.cache/go-build` — ~1 GB, i.e. a seventh copy of what `gomod-v1` already -holds once — for release builds dominated by cross-compiling and multi-arch -docker rather than by `go mod download`. +`~/.cache/go-build` — ~1 GB duplicating what `gomod-v1` already holds once — +for release builds dominated by cross-compiling and multi-arch docker rather +than by `go mod download`. Key-versioning policy: bump the `v` prefix whenever the cache's expected *contents* change shape — saves only fire on an exact-key miss, @@ -218,8 +218,9 @@ slower. Budget for **two live generations**: a `go.sum` or lockfile bump mints a whole new set while the previous one is still warm, so the steady state is ~2× a single generation. That is why `~/go/pkg/mod` is cached **once** (`gomod-v1`) rather than folded into each suffixed build cache — -doing the latter stored the same ~1.6 GB five times, ~5 GB per -generation, and #438's 24-module bump pushed the repo to 10.53 GB +doing the latter stored the module tree five times over — five entries of +~1.1 GB each, ~5.5 GB per generation — and #438's 24-module bump pushed +the repo to 10.53 GB ([#443](https://github.com/Wave-RF/WaveHouse/issues/443)). Before adding a cache or widening an existing `path:`, check the current diff --git a/CHANGELOG.md b/CHANGELOG.md index fdd5f430..9a0232da 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,7 +50,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed -- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.sum` and byte-identical for every flavor — so the same ~1.6 GB was stored five times (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`), ~5 GB per `go.sum` generation. Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. ~5 GB → ~2 GB per generation. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was minting a sixth ~1 GB `go.sum`-keyed copy of the same module tree on every push to main. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. +- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~1.1 GB stored each, ~5.5 GB per `go.sum` generation (the tree is ~1.6 GB on disk, ~1 GB as a stored archive). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. ~5.5 GB → ~2 GB per generation. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was minting a sixth ~1 GB `go.sum`-keyed copy of the same module tree on every push to main. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. - **Live demo hero feed renders in `event_ts` order instead of SSE arrival order** (`docs/src/components/LiveDemo.astro`): the landing-page live activity feed prepended each streamed row to the top in the order the SSE stream delivered it, but a producer's webhook burst (a single merge-queue cycle fires ~20 events) arrives in no guaranteed order and the stream relays it in ingest order — so a late or out-of-order delivery landed above newer rows (e.g. a `pushed 12m ago` sitting on top of `reviewed a pull request 9m ago`). `addRow` now keeps the feed sorted by `event_ts` descending — it slots each row in before the first strictly-older sibling rather than blind-prepending — so the live tail matches the already-sorted `gh_activity_recent` backfill. The zone-less-SSE-timestamp normalization the sort relies on (`normTs`) was already in place; equal-second rows keep arrival order (`gh_events.event_ts` is only second-granular for CI/checks, so there's no finer tiebreak), and dedup + the `MAX_ROWS` trim are unchanged. Surfaced in dogfooding on `wavehouse.dev`; the client-side analog of the ingest-order reality the SSE stream can't reorder. From b475d94d5383caa15260ff1f58c448b9efa37e11 Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Mon, 10 Aug 2026 16:41:46 -0400 Subject: [PATCH 04/19] docs(ci): re-flow the two paragraphs the units edit left ragged No content change. 1f18ead inserted the stored-archive figures without re-wrapping, leaving a 39-char orphan line mid-paragraph in the setup-env comment block and a 20-char one in the README sizing policy, both inside paragraphs that otherwise wrap at ~72 cols. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/actions/setup-env/action.yml | 8 ++++---- .github/workflows/README.md | 5 ++--- 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/.github/actions/setup-env/action.yml b/.github/actions/setup-env/action.yml index a0ac3263..cb88cbb5 100644 --- a/.github/actions/setup-env/action.yml +++ b/.github/actions/setup-env/action.yml @@ -81,10 +81,10 @@ runs: # function of go.sum — byte-identical for every compile flavor — so # folding it into the suffixed build cache stored that tree five times # over (once per -lint/-unit/-integration/-e2e-cov/-cov job) — five - # entries of ~1.1 GB stored each, ~5.5 GB per go.sum generation. - # Against a 10 GB repo cap that means two - # live generations overflowed it and GitHub started LRU-evicting warm - # entries mid-run. Splitting it out stores it once (#443). + # entries of ~1.1 GB stored each, ~5.5 GB per go.sum generation. Against + # a 10 GB repo cap that means two live generations overflowed it, and + # GitHub started LRU-evicting warm entries mid-run. Splitting it out + # stores it once (#443). # # All Go jobs miss this key together on a go.sum bump and all try to # save; the backend keeps the first and the rest log a benign "already diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 16e680c6..80648de8 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -219,9 +219,8 @@ mints a whole new set while the previous one is still warm, so the steady state is ~2× a single generation. That is why `~/go/pkg/mod` is cached **once** (`gomod-v1`) rather than folded into each suffixed build cache — doing the latter stored the module tree five times over — five entries of -~1.1 GB each, ~5.5 GB per generation — and #438's 24-module bump pushed -the repo to 10.53 GB -([#443](https://github.com/Wave-RF/WaveHouse/issues/443)). +~1.1 GB each, ~5.5 GB per generation — and #438's 24-module bump pushed the +repo to 10.53 GB ([#443](https://github.com/Wave-RF/WaveHouse/issues/443)). Before adding a cache or widening an existing `path:`, check the current footprint and confirm two generations still fit: From 425197afb8925cfaca808c87087909c2498546ef Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Mon, 10 Aug 2026 17:09:10 -0400 Subject: [PATCH 05/19] fix(ci): hash go.mod into both Go cache keys, not just go.sum MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CodeRabbit review. GOTOOLCHAIN=auto lands the toolchain in ~/go/pkg/mod/golang.org/toolchain (we run no setup-go in ci.yml), and go.sum records no entry for it — this repo's go.sum has zero golang.org/toolchain lines. So raising go.mod's `go` directive changes which toolchain belongs in the cache while leaving go.sum byte-identical. Verified: with go 1.26.5 -> 1.27.0, sha256 of go.sum alone is unchanged (bcc16701da2c001f both before and after) while go.mod+go.sum rotates. Under the old key that is a permanent regression, not a one-off — the stale key exact-hits, and because saves fire only on an exact-key MISS the freshly fetched toolchain is never saved, so every subsequent run re-downloads it. Both Go keys now hash ('**/go.mod', '**/go.sum'). Costs no extra rotation in practice: dependency bumps already touch both files, so the only newly rotating case is precisely the toolchain one this fixes. Also corrects the setup-go rationale in publish-dev.yml and the CHANGELOG: actions/cache saves on a cache MISS, so that entry is re-saved on each dependency bump, not "minted afresh on every push to main". The budget argument is unchanged — one live ~1 GB entry per generation. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/actions/setup-env/action.yml | 49 +++++++++++++++------------- .github/workflows/README.md | 4 +-- .github/workflows/publish-dev.yml | 12 +++---- CHANGELOG.md | 2 +- 4 files changed, 36 insertions(+), 31 deletions(-) diff --git a/.github/actions/setup-env/action.yml b/.github/actions/setup-env/action.yml index cb88cbb5..8ee8d192 100644 --- a/.github/actions/setup-env/action.yml +++ b/.github/actions/setup-env/action.yml @@ -14,16 +14,16 @@ # # The cache inventory (full architecture: .github/workflows/README.md): # -# 1. Go modules, `gomod-v1-` key family, keyed on every go.sum and NOT -# partitioned per job — ~/go/pkg/mod is a pure function of go.sum, so -# one entry serves every Go job, stored once rather than per flavor -# (~1.6 GB on disk, ~1 GB as a stored archive). -# 2. Go build objects, `gobuild-v3-` key family, keyed on every go.sum -# and partitioned by `go-cache-suffix`: the unit, integration and e2e -# jobs compile with different flags (-race/-cover/-coverpkg/-tags), so -# a shared entry would only ever be warm for whichever job saved it. -# `restore-keys` keeps it warm across go.sum bumps — avoids setup-go's -# exact-key-only cache (actions/setup-go#357). +# 1. Go modules, `gomod-v1-` key family, keyed on every go.mod + go.sum +# and NOT partitioned per job — ~/go/pkg/mod is a pure function of +# those files, so one entry serves every Go job, stored once rather +# than per flavor (~1.6 GB on disk, ~1 GB as a stored archive). +# 2. Go build objects, `gobuild-v3-` key family, same key inputs, +# partitioned by `go-cache-suffix`: the unit, integration and e2e jobs +# compile with different flags (-race/-cover/-coverpkg/-tags), so a +# shared entry would only ever be warm for whichever job saved it. +# `restore-keys` keeps it warm across dependency bumps — avoids +# setup-go's exact-key-only cache (actions/setup-go#357). # 3. golangci-lint binary + analysis cache keyed on Makefile + # .golangci.yml. Analysis cache is the win (~10s warm vs ~90s). # Only the lint job needs it. @@ -78,15 +78,15 @@ runs: using: composite steps: # The module cache is UNSUFFIXED on purpose. ~/go/pkg/mod is a pure - # function of go.sum — byte-identical for every compile flavor — so - # folding it into the suffixed build cache stored that tree five times - # over (once per -lint/-unit/-integration/-e2e-cov/-cov job) — five - # entries of ~1.1 GB stored each, ~5.5 GB per go.sum generation. Against + # function of go.mod + go.sum — byte-identical for every compile flavor + # — so folding it into the suffixed build cache stored that tree five + # times over (once per -lint/-unit/-integration/-e2e-cov/-cov job) — + # five entries of ~1.1 GB stored each, ~5.5 GB per generation. Against # a 10 GB repo cap that means two live generations overflowed it, and # GitHub started LRU-evicting warm entries mid-run. Splitting it out # stores it once (#443). # - # All Go jobs miss this key together on a go.sum bump and all try to + # All Go jobs miss this key together on a dependency bump and all try to # save; the backend keeps the first and the rest log a benign "already # exists" (same trade-off as the rest of the inventory, see header). - uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 @@ -94,12 +94,17 @@ runs: id: gomod-cache with: path: ~/go/pkg/mod - # v1: the GOTOOLCHAIN=auto toolchain rides in here too (no - # setup-go) — that content was what forced the old v2 bump. - key: gomod-v1-${{ runner.os }}-${{ hashFiles('**/go.sum') }} - # A stale generation is still worth restoring: go.sum bumps move a - # handful of modules, so most of the tree is reusable and `go mod - # download` fetches only the delta. + # go.mod is in the key, not just go.sum, because the GOTOOLCHAIN=auto + # toolchain rides in ~/go/pkg/mod/golang.org/toolchain (no setup-go) + # and go.sum records no entry for it. Raising go.mod's `go` directive + # therefore changes which toolchain belongs in this cache while + # leaving go.sum untouched — a go.sum-only key would exact-hit, and + # because saves fire only on an exact-key MISS the freshly fetched + # toolchain would never be saved, re-downloading on every run. + key: gomod-v1-${{ runner.os }}-${{ hashFiles('**/go.mod', '**/go.sum') }} + # A stale generation is still worth restoring: a bump moves a handful + # of modules, so most of the tree is reusable and `go mod download` + # fetches only the delta. restore-keys: | gomod-v1-${{ runner.os }}- @@ -117,7 +122,7 @@ runs: id: gobuild-cache with: path: ~/.cache/go-build - key: gobuild-v3-${{ runner.os }}-go${{ inputs.go-cache-suffix }}-${{ hashFiles('**/go.sum') }} + key: gobuild-v3-${{ runner.os }}-go${{ inputs.go-cache-suffix }}-${{ hashFiles('**/go.mod', '**/go.sum') }} # Same-suffix only. The bare-prefix fallback the v2 key carried # existed to pick up the shared module cache from another job's # entry; that job is now gomod-v1's, and another flavor's build diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 80648de8..ba61137b 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -183,8 +183,8 @@ Queue settings live in the `main branch protection` ruleset's | Cache | Key | Saved by | Notes | |---|---|---|---| -| Go modules | `gomod-v1--` | every Go job (shared) | `~/go/pkg/mod`, **unsuffixed** — a pure function of `go.sum`, so one entry serves every job (~1.6 GB on disk, ~1 GB stored). The GOTOOLCHAIN=auto toolchain rides in here too (no setup-go). | -| Go build objects | `gobuild-v3--go-` | every Go job (own suffix) | `~/.cache/go-build` only. Suffix partitions by compile flavor (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`), which compile with different flags. | +| Go modules | `gomod-v1--` | every Go job (shared) | `~/go/pkg/mod`, **unsuffixed** — a pure function of `go.mod` + `go.sum`, so one entry serves every job (~1.6 GB on disk, ~1 GB stored). The GOTOOLCHAIN=auto toolchain rides in here too (no setup-go), which is why `go.mod` is in the key — a `go` directive bump changes the required toolchain without touching `go.sum`. | +| Go build objects | `gobuild-v3--go-` | every Go job (own suffix) | `~/.cache/go-build` only. Suffix partitions by compile flavor (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`), which compile with different flags. | | golangci binary + analysis | `golangci--` | lint | Analysis cache: ~10s warm vs ~90s. `.bin` also carries shellcheck + actionlint. | | pnpm store | `pnpm--` | any node job on miss | Store path resolved from pnpm at runtime. docs-build prunes before its save on a key rotation. | | Playwright Chromium | `playwright--` | docs-build | rehype-mermaid renders via headless Chrome at docs build. | diff --git a/.github/workflows/publish-dev.yml b/.github/workflows/publish-dev.yml index c326c337..4f71cde6 100644 --- a/.github/workflows/publish-dev.yml +++ b/.github/workflows/publish-dev.yml @@ -52,12 +52,12 @@ jobs: go-version-file: "go.mod" # setup-go's bundled cache stores ~/go/pkg/mod + ~/.cache/go-build # under its own go.sum-keyed entry — a private ~1 GB copy of the - # module tree that ci.yml already caches once as `gomod-v1`, and - # that this workflow mints afresh on every push to main. Two live - # generations is ~2 GB of the repo's hard 10 GB cache budget, for a - # release build that runs once per push and is dominated by - # cross-compiling + multi-arch docker, not by `go mod download`. - # Same call as goreleaser-validate.yml. See #443. + # module tree that ci.yml already caches once as `gomod-v1`. It + # saves a fresh archive on each cache miss (i.e. every dependency + # bump), so two live generations is ~2 GB of the repo's hard 10 GB + # budget — for a release build dominated by cross-compiling and + # multi-arch docker, not by `go mod download`. Same call as + # goreleaser-validate.yml. See #443. cache: false # dockers_v2 builds the linux/amd64+arm64 manifest via `docker buildx`; diff --git a/CHANGELOG.md b/CHANGELOG.md index 9a0232da..c03de553 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,7 +50,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed -- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~1.1 GB stored each, ~5.5 GB per `go.sum` generation (the tree is ~1.6 GB on disk, ~1 GB as a stored archive). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. ~5.5 GB → ~2 GB per generation. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was minting a sixth ~1 GB `go.sum`-keyed copy of the same module tree on every push to main. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. +- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~1.1 GB stored each, ~5.5 GB per generation (the tree is ~1.6 GB on disk, ~1 GB as a stored archive). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. ~5.5 GB → ~2 GB per generation. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB `go.sum`-keyed copy of the same module tree, re-saved on every cache miss. Both Go keys now hash `go.mod` alongside `go.sum`: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. - **Live demo hero feed renders in `event_ts` order instead of SSE arrival order** (`docs/src/components/LiveDemo.astro`): the landing-page live activity feed prepended each streamed row to the top in the order the SSE stream delivered it, but a producer's webhook burst (a single merge-queue cycle fires ~20 events) arrives in no guaranteed order and the stream relays it in ingest order — so a late or out-of-order delivery landed above newer rows (e.g. a `pushed 12m ago` sitting on top of `reviewed a pull request 9m ago`). `addRow` now keeps the feed sorted by `event_ts` descending — it slots each row in before the first strictly-older sibling rather than blind-prepending — so the live tail matches the already-sorted `gh_activity_recent` backfill. The zone-less-SSE-timestamp normalization the sort relies on (`normTs`) was already in place; equal-second rows keep arrival order (`gh_events.event_ts` is only second-granular for CI/checks, so there's no finer tiebreak), and dedup + the `MAX_ROWS` trim are unchanged. Surfaced in dogfooding on `wavehouse.dev`; the client-side analog of the ingest-order reality the SSE stream can't reorder. From 2370e05fdec12d50868864ba0f45c203806cde4c Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Mon, 10 Aug 2026 17:19:39 -0400 Subject: [PATCH 06/19] docs(ci): say why go.mod keys the build cache, not just the module one MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review follow-up on the go.mod key change. The gobuild-v3 block never stated its own reason for hashing go.mod — the header just said "same key inputs", pointing at a rationale (the toolchain rides in ~/go/pkg/mod) that doesn't apply to ~/.cache/go-build. The real reason is stronger and is what stops someone simplifying this key back to go.sum: the compiler's build ID feeds every action hash, so a toolchain bump invalidates every object in the build cache. Keyed on go.sum alone that bump exact-hits, nothing restored is reusable, and the new objects are never saved — every run recompiles cold and drags in ~1 GB of dead objects until an unrelated go.sum change rotates the key. Also updates the sizing policy's trigger list: after this change a go.mod-only bump mints a new generation too, and that paragraph is the one a maintainer reads before doing budget arithmetic. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/actions/setup-env/action.yml | 7 +++++++ .github/workflows/README.md | 15 ++++++++------- 2 files changed, 15 insertions(+), 7 deletions(-) diff --git a/.github/actions/setup-env/action.yml b/.github/actions/setup-env/action.yml index 8ee8d192..3bdf0697 100644 --- a/.github/actions/setup-env/action.yml +++ b/.github/actions/setup-env/action.yml @@ -122,6 +122,13 @@ runs: id: gobuild-cache with: path: ~/.cache/go-build + # go.mod belongs in this key for its own reason, not just symmetry + # with gomod-v1: the compiler's build ID feeds every action hash, so + # a toolchain bump invalidates every object in here. Keyed on go.sum + # alone that bump exact-hits, nothing restored is reusable, and the + # freshly built objects are never saved (saves fire only on a miss) + # — so every run recompiles cold *and* drags in ~1 GB of dead + # objects, until some unrelated go.sum change rotates the key. key: gobuild-v3-${{ runner.os }}-go${{ inputs.go-cache-suffix }}-${{ hashFiles('**/go.mod', '**/go.sum') }} # Same-suffix only. The bare-prefix fallback the v2 key carried # existed to pick up the shared module cache from another job's diff --git a/.github/workflows/README.md b/.github/workflows/README.md index ba61137b..d41fa01b 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -214,13 +214,14 @@ worked example: it kept only its own same-suffix prefix. **Sizing policy — the repo cache budget is 10 GB, hard.** Past it GitHub LRU-evicts, so warm entries disappear mid-run and builds silently get -slower. Budget for **two live generations**: a `go.sum` or lockfile bump -mints a whole new set while the previous one is still warm, so the steady -state is ~2× a single generation. That is why `~/go/pkg/mod` is cached -**once** (`gomod-v1`) rather than folded into each suffixed build cache — -doing the latter stored the module tree five times over — five entries of -~1.1 GB each, ~5.5 GB per generation — and #438's 24-module bump pushed the -repo to 10.53 GB ([#443](https://github.com/Wave-RF/WaveHouse/issues/443)). +slower. Budget for **two live generations**: a `go.mod`/`go.sum` or +lockfile bump mints a whole new set while the previous one is still warm, +so the steady state is ~2× a single generation. That is why `~/go/pkg/mod` +is cached **once** (`gomod-v1`) rather than folded into each suffixed +build cache — doing the latter stored the module tree five times over, +five entries of ~1.1 GB each, ~5.5 GB per generation, and #438's 24-module +bump pushed the repo to 10.53 GB +([#443](https://github.com/Wave-RF/WaveHouse/issues/443)). Before adding a cache or widening an existing `path:`, check the current footprint and confirm two generations still fit: From 6b9d145e856a3d04e6037ad7cc92dab13c060af4 Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Mon, 10 Aug 2026 17:21:10 -0400 Subject: [PATCH 07/19] docs(ci): give the narrowing-rotation policy its delete command MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The exception prescribes purging the stale entries but only shipped inspect commands, so a maintainer following it post-rotation had to improvise the delete. Adds the snippet, the after-not-before ordering (main still restores the old keys until the rotation lands, so an early purge just forces a cold repopulate), and the reminder to include every family the rotation orphans — turning on cache: false strands that job's setup-go-* entry as well as the renamed one. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/workflows/README.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/.github/workflows/README.md b/.github/workflows/README.md index d41fa01b..6cf1338b 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -212,6 +212,19 @@ over an already-restored tree). Drop the old prefix and purge the stale entries instead — they hold budget the new keys need. `gobuild-v3` is the worked example: it kept only its own same-suffix prefix. +Purge **after** the rotation is on `main`, not before — until then `main` +still restores the old keys, so an early delete just forces a cold +repopulate of caches you are about to abandon: + +```bash +gh api repos/Wave-RF/WaveHouse/actions/caches --paginate \ + -q '.actions_caches[]|select(.key|startswith(""))|.id' \ + | xargs -I{} gh api -X DELETE repos/Wave-RF/WaveHouse/actions/caches/{} +``` + +Include every family the rotation orphans, not just the renamed one — e.g. +turning on `cache: false` strands that job's `setup-go-*` entry too. + **Sizing policy — the repo cache budget is 10 GB, hard.** Past it GitHub LRU-evicts, so warm entries disappear mid-run and builds silently get slower. Budget for **two live generations**: a `go.mod`/`go.sum` or From abf3b6e4c8f73912f597d6c5fd24f1eadea5c149 Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Mon, 10 Aug 2026 17:31:17 -0400 Subject: [PATCH 08/19] docs(ci): use measured post-split cache sizes, note the residual gap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review follow-up. The split has now minted real entries, so the estimates are replaced with readings from the live cache API: gomod-v1 493 MB gobuild-v3 unit/integration/lint/e2e-cov/cov 152 / 148 / 140 / 120 / 25 MB generation total 1.05 GB (was 5.56 GB) Two numbers were wrong. "~1 GB stored" for ~/go/pkg/mod was inferred from the combined v2 entries rather than read; a cold save is 0.49 GB, drifting up as superseded versions accumulate through the restore -> save chain, so both halves of that are now stated. And "~1 GB of dead objects" in the build-key comment overstated ~/.cache/go-build by 6-40x — that figure belonged to the pre-split combined entry, and in the failure mode being described the key exact-hits forever so the entry can't grow into it. The win is larger than documented (1.05 GB, not ~2 GB), so the old figures erred conservative, but they broke the PR's own arithmetic. Also records the residual gap hashing go.mod does NOT close: GOTOOLCHAIN =auto uses the local toolchain whenever it satisfies the directive, so a runner image bumping its bundled Go swaps the compiler with neither go.mod nor go.sum moving — same stale-exact-hit pathology, past both keys. Left as a documented cause to check rather than plumbed into the key, since resolving the toolchain before the cache restore would forfeit the property that the restore captures the toolchain download. And fixes the go-cache-suffix description, which still said "Empty = the shared default key" — pre-split wording. Post-split an empty suffix yields a restore-key that prefix-matches every flavor, the exact cross-flavor restore this split removes. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/actions/setup-env/action.yml | 20 +++++++++++++++----- .github/workflows/README.md | 4 ++-- CHANGELOG.md | 2 +- 3 files changed, 18 insertions(+), 8 deletions(-) diff --git a/.github/actions/setup-env/action.yml b/.github/actions/setup-env/action.yml index 3bdf0697..fee6ac7d 100644 --- a/.github/actions/setup-env/action.yml +++ b/.github/actions/setup-env/action.yml @@ -17,7 +17,9 @@ # 1. Go modules, `gomod-v1-` key family, keyed on every go.mod + go.sum # and NOT partitioned per job — ~/go/pkg/mod is a pure function of # those files, so one entry serves every Go job, stored once rather -# than per flavor (~1.6 GB on disk, ~1 GB as a stored archive). +# than per flavor (~1.6 GB on disk; 0.49 GB as a stored archive on a +# cold save, drifting up as superseded module versions accumulate +# through the restore -> save chain). # 2. Go build objects, `gobuild-v3-` key family, same key inputs, # partitioned by `go-cache-suffix`: the unit, integration and e2e jobs # compile with different flags (-race/-cover/-coverpkg/-tags), so a @@ -50,7 +52,7 @@ inputs: description: "Set up the Go toolchain + the Go module/build cache" default: "true" go-cache-suffix: - description: "Per-job Go build-cache partition, e.g. '-unit' (different jobs compile with different flags). Empty = the shared default key." + description: "Per-job Go build-cache partition, e.g. '-unit' (different jobs compile with different flags). Always pass one for a Go job: an empty suffix yields a restore-key that prefix-matches every other flavor's entry, which is the cross-flavor restore the gomod-v1/gobuild-v3 split exists to avoid. The shared part of the cache is gomod-v1, which needs no suffix." default: "" golangci: description: "Cache the golangci-lint binary + analysis cache (lint job only)" @@ -84,7 +86,8 @@ runs: # five entries of ~1.1 GB stored each, ~5.5 GB per generation. Against # a 10 GB repo cap that means two live generations overflowed it, and # GitHub started LRU-evicting warm entries mid-run. Splitting it out - # stores it once (#443). + # stores it once: a measured post-split generation is ~1.05 GB + # (0.49 module + 0.59 across the five build entries) (#443). # # All Go jobs miss this key together on a dependency bump and all try to # save; the backend keeps the first and the rest log a benign "already @@ -127,8 +130,15 @@ runs: # a toolchain bump invalidates every object in here. Keyed on go.sum # alone that bump exact-hits, nothing restored is reusable, and the # freshly built objects are never saved (saves fire only on a miss) - # — so every run recompiles cold *and* drags in ~1 GB of dead - # objects, until some unrelated go.sum change rotates the key. + # — so every run recompiles cold *and* restores objects nothing can + # hit, until some unrelated go.sum change rotates the key. + # + # This closes the go-directive half only. GOTOOLCHAIN=auto uses the + # LOCAL toolchain whenever it satisfies the directive, so a runner + # image bumping its bundled Go swaps the compiler with neither + # go.mod nor go.sum moving — same pathology, past both keys. If you + # see unexplained cold build caches, that's the cause: rotate the + # v prefix to force a save. key: gobuild-v3-${{ runner.os }}-go${{ inputs.go-cache-suffix }}-${{ hashFiles('**/go.mod', '**/go.sum') }} # Same-suffix only. The bare-prefix fallback the v2 key carried # existed to pick up the shared module cache from another job's diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 6cf1338b..3b33182f 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -183,8 +183,8 @@ Queue settings live in the `main branch protection` ruleset's | Cache | Key | Saved by | Notes | |---|---|---|---| -| Go modules | `gomod-v1--` | every Go job (shared) | `~/go/pkg/mod`, **unsuffixed** — a pure function of `go.mod` + `go.sum`, so one entry serves every job (~1.6 GB on disk, ~1 GB stored). The GOTOOLCHAIN=auto toolchain rides in here too (no setup-go), which is why `go.mod` is in the key — a `go` directive bump changes the required toolchain without touching `go.sum`. | -| Go build objects | `gobuild-v3--go-` | every Go job (own suffix) | `~/.cache/go-build` only. Suffix partitions by compile flavor (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`), which compile with different flags. | +| Go modules | `gomod-v1--` | every Go job (shared) | `~/go/pkg/mod`, **unsuffixed** — a pure function of `go.mod` + `go.sum`, so one entry serves every job (~1.6 GB on disk; 0.49 GB stored on a cold save, drifting up as superseded versions accumulate). The GOTOOLCHAIN=auto toolchain rides in here too (no setup-go), which is why `go.mod` is in the key — a `go` directive bump changes the required toolchain without touching `go.sum`. | +| Go build objects | `gobuild-v3--go-` | every Go job (own suffix) | `~/.cache/go-build` only — 25–152 MB stored per flavor. Suffix partitions by compile flavor (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`), which compile with different flags. `go.mod` is in the key for its own reason — the compiler's build ID keys every object, so a toolchain bump invalidates all of them. | | golangci binary + analysis | `golangci--` | lint | Analysis cache: ~10s warm vs ~90s. `.bin` also carries shellcheck + actionlint. | | pnpm store | `pnpm--` | any node job on miss | Store path resolved from pnpm at runtime. docs-build prunes before its save on a key rotation. | | Playwright Chromium | `playwright--` | docs-build | rehype-mermaid renders via headless Chrome at docs build. | diff --git a/CHANGELOG.md b/CHANGELOG.md index c03de553..ed35ed8b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,7 +50,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed -- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~1.1 GB stored each, ~5.5 GB per generation (the tree is ~1.6 GB on disk, ~1 GB as a stored archive). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. ~5.5 GB → ~2 GB per generation. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB `go.sum`-keyed copy of the same module tree, re-saved on every cache miss. Both Go keys now hash `go.mod` alongside `go.sum`: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. +- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~1.1 GB stored each, ~5.5 GB per generation (the tree is ~1.6 GB on disk; 0.49 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.49 GB module + 0.59 GB across the five build entries), down from 5.56 GB. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB `go.sum`-keyed copy of the same module tree, re-saved on every cache miss. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. - **Live demo hero feed renders in `event_ts` order instead of SSE arrival order** (`docs/src/components/LiveDemo.astro`): the landing-page live activity feed prepended each streamed row to the top in the order the SSE stream delivered it, but a producer's webhook burst (a single merge-queue cycle fires ~20 events) arrives in no guaranteed order and the stream relays it in ingest order — so a late or out-of-order delivery landed above newer rows (e.g. a `pushed 12m ago` sitting on top of `reviewed a pull request 9m ago`). `addRow` now keeps the feed sorted by `event_ts` descending — it slots each row in before the first strictly-older sibling rather than blind-prepending — so the live tail matches the already-sorted `gh_activity_recent` backfill. The zone-less-SSE-timestamp normalization the sort relies on (`normTs`) was already in place; equal-second rows keep arrival order (`gh_events.event_ts` is only second-granular for CI/checks, so there's no finer tiebreak), and dedup + the `MAX_ROWS` trim are unchanged. Surfaced in dogfooding on `wavehouse.dev`; the client-side analog of the ingest-order reality the SSE stream can't reorder. From 5bc361c69e0dd960b5f16dc0c64c2618b73fe19f Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Mon, 10 Aug 2026 17:40:22 -0400 Subject: [PATCH 09/19] docs(ci): put every cache size on one unit convention MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review follow-up, and a repeat of the defect the previous commit set out to fix. The measured figures were each read from the live API but converted three different ways: 0.49/0.59 were MiB/1000, 1.05 was bytes/2^30, 5.56 was bytes/10^9. So the decomposition disagreed with its own total (0.49 + 0.59 = 1.08, not 1.05) and the improvement read as 5.3x when it is 4.9x. Everything is now stored-archive bytes / 2^30 — the unit the README's own usage check prints, so a reader running the documented command sees the number the docs quote: gomod-v1 516,549,867 B = 0.48 GB gobuild-v3 x5 613,276,117 B = 0.57 GB generation 1,129,825,984 B = 1.05 GB gobuild-v2 x5 5,564,533,024 B = 5.18 GB (0.9-1.2 each) 0.48 + 0.57 = 1.05, and 5.18 / 1.05 = 4.9x. Also corrects what the setup-go entry actually holds. Four places called it a ~1 GB copy of the module tree; setup-go caches GOMODCACHE *and* GOCACHE, and .goreleaser.yaml cross-compiles 8 targets, so roughly half of it is release build objects nothing else caches. Only the other half duplicates gomod-v1. The budget conclusion is unchanged — dropping it frees the whole ~1 GB — but a change whose thesis is precise byte accounting should not misattribute the bytes. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/actions/setup-env/action.yml | 11 +++++++---- .github/workflows/README.md | 7 ++++--- .github/workflows/publish-dev.yml | 5 +++-- .github/workflows/release.yml | 7 ++++--- CHANGELOG.md | 2 +- 5 files changed, 19 insertions(+), 13 deletions(-) diff --git a/.github/actions/setup-env/action.yml b/.github/actions/setup-env/action.yml index fee6ac7d..c153f5c8 100644 --- a/.github/actions/setup-env/action.yml +++ b/.github/actions/setup-env/action.yml @@ -17,7 +17,7 @@ # 1. Go modules, `gomod-v1-` key family, keyed on every go.mod + go.sum # and NOT partitioned per job — ~/go/pkg/mod is a pure function of # those files, so one entry serves every Go job, stored once rather -# than per flavor (~1.6 GB on disk; 0.49 GB as a stored archive on a +# than per flavor (~1.6 GB on disk; 0.48 GB as a stored archive on a # cold save, drifting up as superseded module versions accumulate # through the restore -> save chain). # 2. Go build objects, `gobuild-v3-` key family, same key inputs, @@ -83,11 +83,14 @@ runs: # function of go.mod + go.sum — byte-identical for every compile flavor # — so folding it into the suffixed build cache stored that tree five # times over (once per -lint/-unit/-integration/-e2e-cov/-cov job) — - # five entries of ~1.1 GB stored each, ~5.5 GB per generation. Against + # five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation. + # Against # a 10 GB repo cap that means two live generations overflowed it, and # GitHub started LRU-evicting warm entries mid-run. Splitting it out - # stores it once: a measured post-split generation is ~1.05 GB - # (0.49 module + 0.59 across the five build entries) (#443). + # stores it once: a measured post-split generation is 1.05 GB + # (0.48 module + 0.57 across the five build entries) (#443). + # Sizes here are stored-archive bytes / 2^30, the same unit the + # README's usage check prints. # # All Go jobs miss this key together on a dependency bump and all try to # save; the backend keeps the first and the rest log a benign "already diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 3b33182f..b89efea9 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -183,7 +183,7 @@ Queue settings live in the `main branch protection` ruleset's | Cache | Key | Saved by | Notes | |---|---|---|---| -| Go modules | `gomod-v1--` | every Go job (shared) | `~/go/pkg/mod`, **unsuffixed** — a pure function of `go.mod` + `go.sum`, so one entry serves every job (~1.6 GB on disk; 0.49 GB stored on a cold save, drifting up as superseded versions accumulate). The GOTOOLCHAIN=auto toolchain rides in here too (no setup-go), which is why `go.mod` is in the key — a `go` directive bump changes the required toolchain without touching `go.sum`. | +| Go modules | `gomod-v1--` | every Go job (shared) | `~/go/pkg/mod`, **unsuffixed** — a pure function of `go.mod` + `go.sum`, so one entry serves every job (~1.6 GB on disk; 0.48 GB stored on a cold save, drifting up as superseded versions accumulate). The GOTOOLCHAIN=auto toolchain rides in here too (no setup-go), which is why `go.mod` is in the key — a `go` directive bump changes the required toolchain without touching `go.sum`. | | Go build objects | `gobuild-v3--go-` | every Go job (own suffix) | `~/.cache/go-build` only — 25–152 MB stored per flavor. Suffix partitions by compile flavor (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`), which compile with different flags. `go.mod` is in the key for its own reason — the compiler's build ID keys every object, so a toolchain bump invalidates all of them. | | golangci binary + analysis | `golangci--` | lint | Analysis cache: ~10s warm vs ~90s. `.bin` also carries shellcheck + actionlint. | | pnpm store | `pnpm--` | any node job on miss | Store path resolved from pnpm at runtime. docs-build prunes before its save on a key rotation. | @@ -194,7 +194,8 @@ Queue settings live in the `main branch protection` ruleset's Deliberately **not** cached: `actions/setup-go`'s bundled cache in `publish-dev.yml`, `release.yml` and `goreleaser-validate.yml` (`cache: false` on each). It stores its own go.sum-keyed copy of `~/go/pkg/mod` + -`~/.cache/go-build` — ~1 GB duplicating what `gomod-v1` already holds once — +`~/.cache/go-build` — a ~1 GB entry holding the module tree `gomod-v1` +already keeps once, plus this job's own 8-target cross-compile objects — for release builds dominated by cross-compiling and multi-arch docker rather than by `go mod download`. @@ -232,7 +233,7 @@ lockfile bump mints a whole new set while the previous one is still warm, so the steady state is ~2× a single generation. That is why `~/go/pkg/mod` is cached **once** (`gomod-v1`) rather than folded into each suffixed build cache — doing the latter stored the module tree five times over, -five entries of ~1.1 GB each, ~5.5 GB per generation, and #438's 24-module +five entries of ~0.9-1.2 GB each, ~5.2 GB per generation, and #438's 24-module bump pushed the repo to 10.53 GB ([#443](https://github.com/Wave-RF/WaveHouse/issues/443)). diff --git a/.github/workflows/publish-dev.yml b/.github/workflows/publish-dev.yml index 4f71cde6..ce8db093 100644 --- a/.github/workflows/publish-dev.yml +++ b/.github/workflows/publish-dev.yml @@ -51,8 +51,9 @@ jobs: with: go-version-file: "go.mod" # setup-go's bundled cache stores ~/go/pkg/mod + ~/.cache/go-build - # under its own go.sum-keyed entry — a private ~1 GB copy of the - # module tree that ci.yml already caches once as `gomod-v1`. It + # under its own go.sum-keyed entry — a private ~1 GB archive: the + # module tree ci.yml already caches once as `gomod-v1`, plus this + # job's own 8-target cross-compile objects. It # saves a fresh archive on each cache miss (i.e. every dependency # bump), so two live generations is ~2 GB of the repo's hard 10 GB # budget — for a release build dominated by cross-compiling and diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index dfc337cb..83eb3c0e 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -25,9 +25,10 @@ jobs: with: go-version-file: "go.mod" # Tag-triggered, so a warm cache is almost never there to hit - # anyway; opting out keeps a ~1 GB go.sum-keyed copy of the module - # tree out of the 10 GB repo budget. Same call as publish-dev.yml - # and goreleaser-validate.yml. See #443. + # anyway; opting out keeps a ~1 GB go.sum-keyed archive — the module + # tree plus this job's cross-compile objects — out of the 10 GB + # repo budget. Same call as publish-dev.yml and + # goreleaser-validate.yml. See #443. cache: false # dockers_v2 builds the linux/amd64+arm64 manifest via `docker buildx`; diff --git a/CHANGELOG.md b/CHANGELOG.md index ed35ed8b..13269c81 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,7 +50,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed -- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~1.1 GB stored each, ~5.5 GB per generation (the tree is ~1.6 GB on disk; 0.49 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.49 GB module + 0.59 GB across the five build entries), down from 5.56 GB. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB `go.sum`-keyed copy of the same module tree, re-saved on every cache miss. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. +- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation (the tree is ~1.6 GB on disk; 0.48 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.48 GB module + 0.57 GB across the five build entries), down from 5.18 GB — a 4.9x reduction. All sizes are stored-archive bytes / 2^30, the unit the README's usage check prints. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB `go.sum`-keyed entry — the module tree `gomod-v1` already keeps once, plus that job's own 8-target cross-compile objects — re-saved on every cache miss. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. - **Live demo hero feed renders in `event_ts` order instead of SSE arrival order** (`docs/src/components/LiveDemo.astro`): the landing-page live activity feed prepended each streamed row to the top in the order the SSE stream delivered it, but a producer's webhook burst (a single merge-queue cycle fires ~20 events) arrives in no guaranteed order and the stream relays it in ingest order — so a late or out-of-order delivery landed above newer rows (e.g. a `pushed 12m ago` sitting on top of `reviewed a pull request 9m ago`). `addRow` now keeps the feed sorted by `event_ts` descending — it slots each row in before the first strictly-older sibling rather than blind-prepending — so the live tail matches the already-sorted `gh_activity_recent` backfill. The zone-less-SSE-timestamp normalization the sort relies on (`normTs`) was already in place; equal-second rows keep arrival order (`gh_events.event_ts` is only second-granular for CI/checks, so there's no finer tiebreak), and dedup + the `MAX_ROWS` trim are unchanged. Surfaced in dogfooding on `wavehouse.dev`; the client-side analog of the ingest-order reality the SSE stream can't reorder. From 32438bcfb0f9832851430a73d48391fa38660822 Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Mon, 10 Aug 2026 17:48:29 -0400 Subject: [PATCH 10/19] docs(ci): stop the job checklist contradicting the suffix rule MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review follow-up. The biggest of these is a contradiction I introduced. "Adding a job" step 3 still read "use setup-env with a fresh go-cache-suffix IF it compiles Go with new flags", while the input description now says always pass one. A new Go job whose flags match an existing flavor reads the checklist as "no new flags, no suffix needed", omits it, and gets key gobuild-v3--go- with restore-key gobuild-v3--go- — which prefix-matches every flavor's entry and mints a sixth build cache. That is precisely the cross-flavor restore this PR removes and the sixth entry its sizing policy forbids, reachable by following the checklist 60 lines below the policy. AGENTS.md sends people to this README before editing ci.yml, so the rule has to live here too, not only in the input description. Also splits the "deliberately not cached" rationale, which applied the two release jobs' reasoning to goreleaser-validate.yml as well. That job runs build --single-target with no dockers_v2, and its own comment gives the real reason: the snapshot is fast enough that the post-step save costs more than a cold go mod download. A reader auditing whether cache: false still earns its place there was being handed the wrong justification. Re-flows the module-cache comment, which the units edit left ragged. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/actions/setup-env/action.yml | 19 +++++++++---------- .github/workflows/README.md | 22 +++++++++++++++------- 2 files changed, 24 insertions(+), 17 deletions(-) diff --git a/.github/actions/setup-env/action.yml b/.github/actions/setup-env/action.yml index c153f5c8..b19ee496 100644 --- a/.github/actions/setup-env/action.yml +++ b/.github/actions/setup-env/action.yml @@ -80,17 +80,16 @@ runs: using: composite steps: # The module cache is UNSUFFIXED on purpose. ~/go/pkg/mod is a pure - # function of go.mod + go.sum — byte-identical for every compile flavor - # — so folding it into the suffixed build cache stored that tree five - # times over (once per -lint/-unit/-integration/-e2e-cov/-cov job) — + # function of go.mod + go.sum — byte-identical for every compile + # flavor — so folding it into the suffixed build cache stored that tree + # five times over, once per -lint/-unit/-integration/-e2e-cov/-cov job: # five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation. - # Against - # a 10 GB repo cap that means two live generations overflowed it, and - # GitHub started LRU-evicting warm entries mid-run. Splitting it out - # stores it once: a measured post-split generation is 1.05 GB - # (0.48 module + 0.57 across the five build entries) (#443). - # Sizes here are stored-archive bytes / 2^30, the same unit the - # README's usage check prints. + # Against a 10 GB repo cap that means two live generations overflowed + # it, and GitHub started LRU-evicting warm entries mid-run. Split out, + # it is stored once and a measured generation is 1.05 GB — 0.48 module + # plus 0.57 across the five build entries (#443). Sizes here are + # stored-archive bytes / 2^30, the unit the README's usage check + # prints. # # All Go jobs miss this key together on a dependency bump and all try to # save; the backend keeps the first and the rest log a benign "already diff --git a/.github/workflows/README.md b/.github/workflows/README.md index b89efea9..9484a3ce 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -193,11 +193,14 @@ Queue settings live in the `main branch protection` ruleset's Deliberately **not** cached: `actions/setup-go`'s bundled cache in `publish-dev.yml`, `release.yml` and `goreleaser-validate.yml` (`cache: false` -on each). It stores its own go.sum-keyed copy of `~/go/pkg/mod` + -`~/.cache/go-build` — a ~1 GB entry holding the module tree `gomod-v1` -already keeps once, plus this job's own 8-target cross-compile objects — -for release builds dominated by cross-compiling and multi-arch docker rather -than by `go mod download`. +on each) — for two different reasons. In `publish-dev.yml` / `release.yml` it +stores its own go.sum-keyed copy of `~/go/pkg/mod` + `~/.cache/go-build`: a +~1 GB entry holding the module tree `gomod-v1` already keeps once, plus that +job's own 8-target cross-compile objects, for builds dominated by +cross-compiling and multi-arch docker rather than by `go mod download`. +`goreleaser-validate.yml` opts out on its own grounds — its +`--single-target` snapshot is fast enough that the post-step save costs more +than a cold `go mod download`. Key-versioning policy: bump the `v` prefix whenever the cache's expected *contents* change shape — saves only fire on an exact-key miss, @@ -297,8 +300,13 @@ suite's wall-clock becomes a problem again, start here: 2. Gate on the change set via `needs: changes` + `if:` on its outputs — never with workflow-level `paths` filters (they'd orphan the required check, invariant 1). -3. Use `setup-env` with a fresh `go-cache-suffix` if it compiles Go with - new flags; never add cache save steps (invariant 6). +3. Use `setup-env`, and **always** pass a `go-cache-suffix` if the job + compiles Go — a fresh one for new flags, an existing flavor's if it + compiles identically. Never leave it empty: the resulting + `gobuild-v3--go-` restore-key prefix-matches every flavor's entry + (the cross-flavor restore the split exists to avoid) and mints a sixth + build entry against the sizing policy above. Never add cache save + steps (invariant 6). 4. Need a build product / data from another job? Upload it as an artifact there, then either `needs` the producer + `download-artifact` (simple, but serializes this job's setup behind the producer), or — when this From 4a48e03bd5343bb726cbe8b4e18fc708301e722b Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Mon, 10 Aug 2026 17:54:47 -0400 Subject: [PATCH 11/19] fix(ci): keep publish-dev's cross-compile cache, drop only the duplicate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review follow-up, and a correction to this PR's own change. Blanket cache: false on publish-dev.yml measurably regressed it. Across its last 20 runs, GoReleaser splits cleanly on whether setup-go's cache hit: warm (post-step 0-1s): 36 37 39 47 50 54 188 224 241 242 245 246 s cold (post-step 12-15s): 401 414 417 420 431 432 446 s No overlap. Run 29029026651 is the control — docs-only change, go.sum unchanged, entry evicted: 431s against 188-246s for its warm neighbours. So the opt-out cost 3-6.5 minutes on every push to main, and the value sits in ~/.cache/go-build (8 targets: 4 goos x 2 goarch), not the module tree. The rationale this PR shipped — "dominated by cross-compiling and multi-arch docker, not by go mod download" — argued for keeping it. setup-go's bundled cache is still wrong here because it stores the module tree alongside, re-duplicating what gomod-v1 holds once. So: keep cache: false, and cache ~/.cache/go-build alone under gobuild-v3--go-release- (~0.5 GB vs the bundled ~1 GB). The duplication #443 is about is gone; the cross-compile stays warm. The -release suffix is load-bearing: those objects are cross-compiled for 8 GOOS/GOARCH pairs and share nothing with ci.yml's native-only flavors, so neither side restores bytes it cannot use. It also satisfies the always-pass-a-suffix rule this branch added. release.yml keeps the plain opt-out — tag-triggered, so a warm entry is rarely there to hit. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/workflows/README.md | 34 ++++++++++++++++++++----------- .github/workflows/publish-dev.yml | 33 ++++++++++++++++++++++-------- CHANGELOG.md | 2 +- 3 files changed, 47 insertions(+), 22 deletions(-) diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 9484a3ce..baced8da 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -189,18 +189,25 @@ Queue settings live in the `main branch protection` ruleset's | pnpm store | `pnpm--` | any node job on miss | Store path resolved from pnpm at runtime. docs-build prunes before its save on a key rotation. | | Playwright Chromium | `playwright--` | docs-build | rehype-mermaid renders via headless Chrome at docs build. | | Astro content collections | `astro--` | lint / docs-build | Warm `astro check`/`build` skip unchanged content. | +| Go build objects (release) | `gobuild-v3--go-release-` | publish-dev | `~/.cache/go-build` from GoReleaser's 8-target cross-compile (~0.5 GB). Same family and key inputs as the CI flavors, `-release` suffix because cross-compiled objects share nothing with the native-only ones. Worth 3–6.5 min on every push to main. | | CodeQL DB + deps | `codeql-dependencies-*`, `codeql-overlay-base-database-*` | GHAS default setup | **Not ours** — minted by GitHub's default CodeQL setup, not by any workflow in this repo, and not configurable here. ~0.4 GB. Listed so the budget arithmetic below is honest. | -Deliberately **not** cached: `actions/setup-go`'s bundled cache in -`publish-dev.yml`, `release.yml` and `goreleaser-validate.yml` (`cache: false` -on each) — for two different reasons. In `publish-dev.yml` / `release.yml` it -stores its own go.sum-keyed copy of `~/go/pkg/mod` + `~/.cache/go-build`: a -~1 GB entry holding the module tree `gomod-v1` already keeps once, plus that -job's own 8-target cross-compile objects, for builds dominated by -cross-compiling and multi-arch docker rather than by `go mod download`. -`goreleaser-validate.yml` opts out on its own grounds — its -`--single-target` snapshot is fast enough that the post-step save costs more -than a cold `go mod download`. +Deliberately **not** cached: `actions/setup-go`'s bundled cache +(`cache: false` in `publish-dev.yml`, `release.yml` and +`goreleaser-validate.yml`) — for different reasons per job. + +It stores `~/go/pkg/mod` **and** `~/.cache/go-build` under one go.sum-keyed +entry (~1 GB live), so roughly half re-stores the module tree `gomod-v1` +already keeps once. `publish-dev.yml` opts out of that entry and caches the +half that pays for itself on its own key (`gobuild-v3--go-release-`, +~0.5 GB): its GoReleaser step takes 36–246 s warm versus 401–446 s cold, so +dropping the build objects outright would cost 3–6.5 minutes on every push +to main. + +`release.yml` keeps the plain opt-out — it is tag-triggered, so a warm entry +is rarely there to hit. `goreleaser-validate.yml` opts out on its own +grounds: its `--single-target` snapshot is fast enough that the post-step +save costs more than a cold `go mod download`. Key-versioning policy: bump the `v` prefix whenever the cache's expected *contents* change shape — saves only fire on an exact-key miss, @@ -240,8 +247,11 @@ five entries of ~0.9-1.2 GB each, ~5.2 GB per generation, and #438's 24-module bump pushed the repo to 10.53 GB ([#443](https://github.com/Wave-RF/WaveHouse/issues/443)). -Before adding a cache or widening an existing `path:`, check the current -footprint and confirm two generations still fit: +Steady state after the split is roughly 5 GB of the 10 — two generations +of `gomod-v1` + the five `gobuild-v3` flavors + the release build cache, +plus the node-side caches and CodeQL. Before adding a cache or widening an +existing `path:`, check the current footprint and confirm two generations +still fit: ```bash gh api repos/Wave-RF/WaveHouse/actions/cache/usage \ diff --git a/.github/workflows/publish-dev.yml b/.github/workflows/publish-dev.yml index ce8db093..6f17dec0 100644 --- a/.github/workflows/publish-dev.yml +++ b/.github/workflows/publish-dev.yml @@ -50,17 +50,32 @@ jobs: - uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0 with: go-version-file: "go.mod" - # setup-go's bundled cache stores ~/go/pkg/mod + ~/.cache/go-build - # under its own go.sum-keyed entry — a private ~1 GB archive: the - # module tree ci.yml already caches once as `gomod-v1`, plus this - # job's own 8-target cross-compile objects. It - # saves a fresh archive on each cache miss (i.e. every dependency - # bump), so two live generations is ~2 GB of the repo's hard 10 GB - # budget — for a release build dominated by cross-compiling and - # multi-arch docker, not by `go mod download`. Same call as - # goreleaser-validate.yml. See #443. + # Opt out of setup-go's bundled cache and cache the half that + # earns its keep, below. The bundled one stores ~/go/pkg/mod AND + # ~/.cache/go-build under one go.sum-keyed entry (~1 GB live), so + # roughly half of it re-stores the module tree ci.yml already + # keeps once as `gomod-v1` — the duplication #443 is about. cache: false + # The other half is the reason this job is fast, so cache it on its + # own. GoReleaser cross-compiles 8 targets here (4 goos × 2 goarch), + # and the timings split cleanly on whether this cache hit: warm runs + # finish GoReleaser in 36-246s, cold ones in 401-446s. Dropping it + # outright would cost 3-6.5 minutes on every push to main. + # + # Release-scoped suffix: these objects are cross-compiled for 8 + # GOOS/GOARCH pairs and share nothing with ci.yml's native-only + # flavors, so `-release` keeps both sides from restoring bytes the + # other can't use. Same `gobuild-v3` family and key inputs as + # setup-env's (see .github/workflows/README.md); ~0.5 GB rather than + # the ~1 GB the bundled cache held. + - uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 + with: + path: ~/.cache/go-build + key: gobuild-v3-${{ runner.os }}-go-release-${{ hashFiles('**/go.mod', '**/go.sum') }} + restore-keys: | + gobuild-v3-${{ runner.os }}-go-release- + # dockers_v2 builds the linux/amd64+arm64 manifest via `docker buildx`; # the GitHub-hosted runner's default docker driver can't build # multi-platform images, so create a docker-container builder. No QEMU diff --git a/CHANGELOG.md b/CHANGELOG.md index 13269c81..733bce31 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,7 +50,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed -- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation (the tree is ~1.6 GB on disk; 0.48 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.48 GB module + 0.57 GB across the five build entries), down from 5.18 GB — a 4.9x reduction. All sizes are stored-archive bytes / 2^30, the unit the README's usage check prints. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB `go.sum`-keyed entry — the module tree `gomod-v1` already keeps once, plus that job's own 8-target cross-compile objects — re-saved on every cache miss. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. +- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation (the tree is ~1.6 GB on disk; 0.48 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.48 GB module + 0.57 GB across the five build entries), down from 5.18 GB — a 4.9x reduction. All sizes are stored-archive bytes / 2^30, the unit the README's usage check prints. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB `go.sum`-keyed entry — the module tree `gomod-v1` already keeps once, plus that job's own 8-target cross-compile objects — re-saved on every cache miss. `publish-dev.yml` re-caches only the half that pays for itself, under `gobuild-v3--go-release-` (~0.5 GB): measured across its last 20 runs, GoReleaser takes 36–246 s with that cache warm and 401–446 s cold, so dropping the cross-compile objects outright would have cost 3–6.5 minutes on every push to main. The `-release` suffix keeps those 8-target objects from being restored by CI's native-only flavors and vice versa. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. - **Live demo hero feed renders in `event_ts` order instead of SSE arrival order** (`docs/src/components/LiveDemo.astro`): the landing-page live activity feed prepended each streamed row to the top in the order the SSE stream delivered it, but a producer's webhook burst (a single merge-queue cycle fires ~20 events) arrives in no guaranteed order and the stream relays it in ingest order — so a late or out-of-order delivery landed above newer rows (e.g. a `pushed 12m ago` sitting on top of `reviewed a pull request 9m ago`). `addRow` now keeps the feed sorted by `event_ts` descending — it slots each row in before the first strictly-older sibling rather than blind-prepending — so the live tail matches the already-sorted `gh_activity_recent` backfill. The zone-less-SSE-timestamp normalization the sort relies on (`normTs`) was already in place; equal-second rows keep arrival order (`gh_events.event_ts` is only second-granular for CI/checks, so there's no finer tiebreak), and dedup + the `MAX_ROWS` trim are unchanged. Surfaced in dogfooding on `wavehouse.dev`; the client-side analog of the ingest-order reality the SSE stream can't reorder. From ab9cfe7f1b73ea10aae9911122b511c3ad093487 Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Mon, 10 Aug 2026 18:07:25 -0400 Subject: [PATCH 12/19] docs(ci): scope invariant 6 to ci.yml, name the one cache outside it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review follow-up. 4a48e03 added the repo's only bare actions/cache outside setup-env, contradicting invariant 6 ("Caches are owned end-to-end by setup-env ... No save steps") and the checklist item that enforces it, without updating either. Invariant 6 is now scoped to ci.yml and names publish-dev.yml's release build cache as the deliberate exception — hand-rolled because that workflow doesn't use setup-env at all. The inventory row says so too, since its "Saved by" cell was the only one naming a workflow rather than a ci.yml job, sending a reader to setup-env to find nothing. Three accuracy fixes alongside: - "~1 GB live" reintroduced the unit ambiguity 5bc361c removed. That figure is stored-archive; read as on-disk it contradicts the table 13 lines above, where the module half alone is 1.6 GB on disk. Now "stored". - release.yml's stated reason was wrong. Cache reads fall back to the default branch's scope, so a tag run CAN hit publish-dev's key — which this PR keeps hot on every push to main. The real reason is that a tagged release is rare and not latency-sensitive. Also notes it could restore that key at zero budget cost, since an exact hit never saves. - "mints a sixth build entry" became wrong the moment this PR added the release cache as the sixth. Now ordinal-free so it stays right. And attributes the timing measurements honestly: they were taken on setup-go's bundled entry, which carried the same ~/.cache/go-build tree. The gobuild-v3--go-release- key is new here, so someone auditing those runs for hits on it would find none. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/workflows/README.md | 27 ++++++++++++++++++--------- .github/workflows/publish-dev.yml | 8 +++++--- .github/workflows/release.yml | 10 ++++++---- CHANGELOG.md | 2 +- 4 files changed, 30 insertions(+), 17 deletions(-) diff --git a/.github/workflows/README.md b/.github/workflows/README.md index baced8da..994b5996 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -119,13 +119,17 @@ Break one of these knowingly or not at all. runs the PR tree with no secrets beyond a read-mostly `GITHUB_TOKEN`. Fork PRs: secrets are absent and `docs-preview` skips itself. -6. **Caches are owned end-to-end by `setup-env`** +6. **`ci.yml`'s caches are owned end-to-end by `setup-env`** ([.github/actions/setup-env](../actions/setup-env/action.yml)): each cache is a nested `actions/cache` step that restores inline and saves automatically at job end on an exact-key miss. No save steps in `ci.yml`. Trade-offs accepted: failed jobs don't save (restore-keys cushion the next run), and concurrent same-key misses produce benign - "already exists" warnings. + "already exists" warnings. **One cache lives outside it**: + `publish-dev.yml`'s release build cache is a bare `actions/cache`, + because that workflow doesn't use `setup-env` at all (it runs + GoReleaser, not the test suites). It is the only `actions/cache@` in + the repo outside the composite — keep it that way. ## Coverage publishing @@ -189,7 +193,7 @@ Queue settings live in the `main branch protection` ruleset's | pnpm store | `pnpm--` | any node job on miss | Store path resolved from pnpm at runtime. docs-build prunes before its save on a key rotation. | | Playwright Chromium | `playwright--` | docs-build | rehype-mermaid renders via headless Chrome at docs build. | | Astro content collections | `astro--` | lint / docs-build | Warm `astro check`/`build` skip unchanged content. | -| Go build objects (release) | `gobuild-v3--go-release-` | publish-dev | `~/.cache/go-build` from GoReleaser's 8-target cross-compile (~0.5 GB). Same family and key inputs as the CI flavors, `-release` suffix because cross-compiled objects share nothing with the native-only ones. Worth 3–6.5 min on every push to main. | +| Go build objects (release) | `gobuild-v3--go-release-` | publish-dev (hand-rolled, not `setup-env`) | `~/.cache/go-build` from GoReleaser's 8-target cross-compile (~0.5 GB). Same family and key inputs as the CI flavors, `-release` suffix because cross-compiled objects share nothing with the native-only ones. Worth 3–6.5 min on every push to main. | | CodeQL DB + deps | `codeql-dependencies-*`, `codeql-overlay-base-database-*` | GHAS default setup | **Not ours** — minted by GitHub's default CodeQL setup, not by any workflow in this repo, and not configurable here. ~0.4 GB. Listed so the budget arithmetic below is honest. | Deliberately **not** cached: `actions/setup-go`'s bundled cache @@ -197,15 +201,19 @@ Deliberately **not** cached: `actions/setup-go`'s bundled cache `goreleaser-validate.yml`) — for different reasons per job. It stores `~/go/pkg/mod` **and** `~/.cache/go-build` under one go.sum-keyed -entry (~1 GB live), so roughly half re-stores the module tree `gomod-v1` +entry (~1 GB stored), so roughly half re-stores the module tree `gomod-v1` already keeps once. `publish-dev.yml` opts out of that entry and caches the half that pays for itself on its own key (`gobuild-v3--go-release-`, ~0.5 GB): its GoReleaser step takes 36–246 s warm versus 401–446 s cold, so dropping the build objects outright would cost 3–6.5 minutes on every push to main. -`release.yml` keeps the plain opt-out — it is tag-triggered, so a warm entry -is rarely there to hit. `goreleaser-validate.yml` opts out on its own +`release.yml` keeps the plain opt-out. Not because a warm entry is missing +— a tag run can restore from the default branch's scope, and `publish-dev` +keeps that key hot on every push to `main` — but because a tagged release +is rare and not latency-sensitive, so it isn't worth the budget. If release +wall-clock ever does matter, it can restore `publish-dev`'s key at zero +budget cost: an exact hit never saves. `goreleaser-validate.yml` opts out on its own grounds: its `--single-target` snapshot is fast enough that the post-step save costs more than a cold `go mod download`. @@ -314,9 +322,10 @@ suite's wall-clock becomes a problem again, start here: compiles Go — a fresh one for new flags, an existing flavor's if it compiles identically. Never leave it empty: the resulting `gobuild-v3--go-` restore-key prefix-matches every flavor's entry - (the cross-flavor restore the split exists to avoid) and mints a sixth - build entry against the sizing policy above. Never add cache save - steps (invariant 6). + (the cross-flavor restore the split exists to avoid) and mints an extra + build entry against the sizing policy above. Never add cache save steps to + `ci.yml` (invariant 6); a workflow outside it that needs a cache + hand-rolls one, as `publish-dev.yml` does. 4. Need a build product / data from another job? Upload it as an artifact there, then either `needs` the producer + `download-artifact` (simple, but serializes this job's setup behind the producer), or — when this diff --git a/.github/workflows/publish-dev.yml b/.github/workflows/publish-dev.yml index 6f17dec0..7ff5754b 100644 --- a/.github/workflows/publish-dev.yml +++ b/.github/workflows/publish-dev.yml @@ -52,15 +52,17 @@ jobs: go-version-file: "go.mod" # Opt out of setup-go's bundled cache and cache the half that # earns its keep, below. The bundled one stores ~/go/pkg/mod AND - # ~/.cache/go-build under one go.sum-keyed entry (~1 GB live), so + # ~/.cache/go-build under one go.sum-keyed entry (~1 GB stored), so # roughly half of it re-stores the module tree ci.yml already # keeps once as `gomod-v1` — the duplication #443 is about. cache: false # The other half is the reason this job is fast, so cache it on its # own. GoReleaser cross-compiles 8 targets here (4 goos × 2 goarch), - # and the timings split cleanly on whether this cache hit: warm runs - # finish GoReleaser in 36-246s, cold ones in 401-446s. Dropping it + # and the last 20 runs split cleanly on whether these objects were + # warm: GoReleaser finished in 36-246s with them, 401-446s without. + # (Measured on setup-go's bundled entry, which carried this same + # ~/.cache/go-build tree — this key is new here.) Dropping them # outright would cost 3-6.5 minutes on every push to main. # # Release-scoped suffix: these objects are cross-compiled for 8 diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 83eb3c0e..06fe6259 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -24,10 +24,12 @@ jobs: - uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0 with: go-version-file: "go.mod" - # Tag-triggered, so a warm cache is almost never there to hit - # anyway; opting out keeps a ~1 GB go.sum-keyed archive — the module - # tree plus this job's cross-compile objects — out of the 10 GB - # repo budget. Same call as publish-dev.yml and + # Opting out keeps a ~1 GB go.sum-keyed archive — the module tree + # plus this job's cross-compile objects — out of the 10 GB repo + # budget. A tag run could hit it (cache reads fall back to the + # default branch's scope, and publish-dev keeps an equivalent key + # hot), but a tagged release is rare and not latency-sensitive, + # so it isn't worth the budget. Same call as publish-dev.yml and # goreleaser-validate.yml. See #443. cache: false diff --git a/CHANGELOG.md b/CHANGELOG.md index 733bce31..c5e1fea8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,7 +50,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed -- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation (the tree is ~1.6 GB on disk; 0.48 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.48 GB module + 0.57 GB across the five build entries), down from 5.18 GB — a 4.9x reduction. All sizes are stored-archive bytes / 2^30, the unit the README's usage check prints. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB `go.sum`-keyed entry — the module tree `gomod-v1` already keeps once, plus that job's own 8-target cross-compile objects — re-saved on every cache miss. `publish-dev.yml` re-caches only the half that pays for itself, under `gobuild-v3--go-release-` (~0.5 GB): measured across its last 20 runs, GoReleaser takes 36–246 s with that cache warm and 401–446 s cold, so dropping the cross-compile objects outright would have cost 3–6.5 minutes on every push to main. The `-release` suffix keeps those 8-target objects from being restored by CI's native-only flavors and vice versa. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. +- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation (the tree is ~1.6 GB on disk; 0.48 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.48 GB module + 0.57 GB across the five build entries), down from 5.18 GB — a 4.9x reduction. All sizes are stored-archive bytes / 2^30, the unit the README's usage check prints. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB `go.sum`-keyed entry — the module tree `gomod-v1` already keeps once, plus that job's own 8-target cross-compile objects — re-saved on every cache miss. `publish-dev.yml` re-caches only the half that pays for itself, under `gobuild-v3--go-release-` (~0.5 GB — the bundled entry minus `gomod-v1`'s share): across its last 20 runs GoReleaser takes 36–246 s with the cross-compile objects warm and 401–446 s cold (measured on `setup-go`'s bundled cache, which carried the same `~/.cache/go-build` tree), so dropping the cross-compile objects outright would have cost 3–6.5 minutes on every push to main. The `-release` suffix keeps those 8-target objects from being restored by CI's native-only flavors and vice versa. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. - **Live demo hero feed renders in `event_ts` order instead of SSE arrival order** (`docs/src/components/LiveDemo.astro`): the landing-page live activity feed prepended each streamed row to the top in the order the SSE stream delivered it, but a producer's webhook burst (a single merge-queue cycle fires ~20 events) arrives in no guaranteed order and the stream relays it in ingest order — so a late or out-of-order delivery landed above newer rows (e.g. a `pushed 12m ago` sitting on top of `reviewed a pull request 9m ago`). `addRow` now keeps the feed sorted by `event_ts` descending — it slots each row in before the first strictly-older sibling rather than blind-prepending — so the live tail matches the already-sorted `gh_activity_recent` backfill. The zone-less-SSE-timestamp normalization the sort relies on (`normTs`) was already in place; equal-second rows keep arrival order (`gh_events.event_ts` is only second-granular for CI/checks, so there's no finer tiebreak), and dedup + the `MAX_ROWS` trim are unchanged. Surfaced in dogfooding on `wavehouse.dev`; the client-side analog of the ingest-order reality the SSE stream can't reorder. From b2682dbfe383820af3539289c80a4b1243af7382 Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Mon, 10 Aug 2026 18:18:30 -0400 Subject: [PATCH 13/19] =?UTF-8?q?fix(ci):=20correct=20release.yml's=20cach?= =?UTF-8?q?e=20reasoning=20=E2=80=94=20the=20key=20stops=20existing?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review follow-up, and an error introduced while fixing the previous one. ab9cfe7 replaced release.yml's wrong reason ("a warm cache is almost never there to hit" — false, tag runs read the default branch's scope) with a second wrong reason: that a tag run "could hit it". It can't. "It" is setup-go's bundled archive, and this PR makes that key cease to exist — publish-dev.yml now opts out, goreleaser-validate.yml already did, and ci.yml runs no setup-go at all. The README even instructs purging the stranded entry. So flipping cache: true there would be a cold miss AND a fresh ~1 GB save, the opposite of what the comment implied. The parenthetical also contradicted itself: an "equivalent" key is a different key, and setup-go only looks up its own. Both the comment and the README paragraph now say what is actually true: nothing mints that key any more; what IS warm is publish-dev's gobuild-v3--go-release- entry, restorable from the default branch's scope; a tagged release is rare and not latency-sensitive, so it isn't worth a hand-rolled restore step — but that is the lever if it ever is. Also stops "Same call as publish-dev.yml" flattening a distinction the README is careful about: same cache: false, different follow-up, since publish-dev re-caches the useful half. And names the key in the README rather than saying "that key", whose antecedent resolved the wrong way in the release.yml comment — the ambiguity was not hypothetical. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/workflows/README.md | 20 ++++++++++++-------- .github/workflows/release.yml | 17 ++++++++++++----- 2 files changed, 24 insertions(+), 13 deletions(-) diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 994b5996..6823e201 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -208,14 +208,18 @@ half that pays for itself on its own key (`gobuild-v3--go-release-`, dropping the build objects outright would cost 3–6.5 minutes on every push to main. -`release.yml` keeps the plain opt-out. Not because a warm entry is missing -— a tag run can restore from the default branch's scope, and `publish-dev` -keeps that key hot on every push to `main` — but because a tagged release -is rare and not latency-sensitive, so it isn't worth the budget. If release -wall-clock ever does matter, it can restore `publish-dev`'s key at zero -budget cost: an exact hit never saves. `goreleaser-validate.yml` opts out on its own -grounds: its `--single-target` snapshot is fast enough that the post-step -save costs more than a cold `go mod download`. +`release.yml` keeps the plain opt-out — no re-cache. After this change +nothing mints a `setup-go-*` key at all, so turning its bundled cache back +on would be a cold miss *and* a fresh ~1 GB save rather than a hit. What is +warm is `publish-dev`'s `gobuild-v3--go-release-` entry, which a tag run +could restore from the default branch's scope — but a tagged release is rare +and not latency-sensitive, so it isn't worth a hand-rolled restore step. +That is the lever if release wall-clock ever does matter, and it costs no +budget: an exact hit never saves. + +`goreleaser-validate.yml` opts out on its own grounds: its `--single-target` +snapshot is fast enough that the post-step save costs more than a cold +`go mod download`. Key-versioning policy: bump the `v` prefix whenever the cache's expected *contents* change shape — saves only fire on an exact-key miss, diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 06fe6259..01fd8967 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -26,11 +26,18 @@ jobs: go-version-file: "go.mod" # Opting out keeps a ~1 GB go.sum-keyed archive — the module tree # plus this job's cross-compile objects — out of the 10 GB repo - # budget. A tag run could hit it (cache reads fall back to the - # default branch's scope, and publish-dev keeps an equivalent key - # hot), but a tagged release is rare and not latency-sensitive, - # so it isn't worth the budget. Same call as publish-dev.yml and - # goreleaser-validate.yml. See #443. + # budget. After this change nothing mints that key at all + # (publish-dev.yml and goreleaser-validate.yml opt out too, and + # ci.yml runs no setup-go), so turning it back on here would be a + # cold miss AND a fresh ~1 GB save, not a hit. + # + # What IS warm is publish-dev's gobuild-v3--go-release- entry, + # restorable from the default branch's scope. A tagged release is + # rare and not latency-sensitive, so it isn't worth a hand-rolled + # restore step here — but that's the lever if it ever is. + # + # Same cache: false as publish-dev.yml and goreleaser-validate.yml; + # different follow-up (publish-dev re-caches the useful half). #443. cache: false # dockers_v2 builds the linux/amd64+arm64 manifest via `docker buildx`; From 6f1ee3aba31f5179b2f1ae94c7701898c3286ffd Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Tue, 11 Aug 2026 07:03:27 -0400 Subject: [PATCH 14/19] fix(ci): setup-go v7 keys on go.mod, and tag saves are unreadable MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two corrections from review, both checkable and both wrong in exactly the dimension this PR argues about. setup-go's bundled cache is NOT go.sum-keyed at the pinned v7 — it keys on the root go.mod. actions/setup-go#705 (Jan 2026) changed dependencyFilePattern from 'go.sum' to 'go.mod', and the live entry proves it: its key ends 9e56ecf5..., and sha256(sha256(go.mod)) = 9e56ecf5... exactly, while sha256(sha256(go.sum)) = 136f5059... — the hash in every gobuild-v2 key. Four places said go.sum-keyed. Worse, the CHANGELOG contrasted "both Go keys now hash go.mod alongside go.sum" against that "go.sum-keyed" entry, which reads as upstream having the toolchain staleness bug this PR fixes, when upstream fixed precisely that and errs the other way. And the release.yml argument stopped a step short. Cache writes are scoped to the ref that made them, so a save from refs/tags/v1.0.0 can never be read by refs/tags/v1.0.1 — or by anything else, ever. Re-enabling the bundled cache there isn't just a cold miss plus a save; it is a ~1 GB write-only entry burned on every tagged release, permanently unreadable. That is the strongest argument for the opt-out and it was missing. Same fact sharpens the escape hatch: "an exact hit never saves" holds only for an exact hit — a restore-keys prefix match would save, tag-scoped and unreadable. The README now prescribes actions/cache/restore (restore-only) for that lever, which has no save step at all. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/workflows/README.md | 17 ++++++++++++----- .github/workflows/publish-dev.yml | 3 ++- .github/workflows/release.yml | 19 +++++++++++++------ CHANGELOG.md | 2 +- 4 files changed, 28 insertions(+), 13 deletions(-) diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 6823e201..f7bef546 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -200,9 +200,10 @@ Deliberately **not** cached: `actions/setup-go`'s bundled cache (`cache: false` in `publish-dev.yml`, `release.yml` and `goreleaser-validate.yml`) — for different reasons per job. -It stores `~/go/pkg/mod` **and** `~/.cache/go-build` under one go.sum-keyed -entry (~1 GB stored), so roughly half re-stores the module tree `gomod-v1` -already keeps once. `publish-dev.yml` opts out of that entry and caches the +It stores `~/go/pkg/mod` **and** `~/.cache/go-build` under one entry +(~1 GB stored, keyed on the root `go.mod` — setup-go hashed `go.sum` +through v6, `go.mod` from v7, [actions/setup-go#705]), so roughly half +re-stores the module tree `gomod-v1` already keeps once. `publish-dev.yml` opts out of that entry and caches the half that pays for itself on its own key (`gobuild-v3--go-release-`, ~0.5 GB): its GoReleaser step takes 36–246 s warm versus 401–446 s cold, so dropping the build objects outright would cost 3–6.5 minutes on every push @@ -214,8 +215,14 @@ on would be a cold miss *and* a fresh ~1 GB save rather than a hit. What is warm is `publish-dev`'s `gobuild-v3--go-release-` entry, which a tag run could restore from the default branch's scope — but a tagged release is rare and not latency-sensitive, so it isn't worth a hand-rolled restore step. -That is the lever if release wall-clock ever does matter, and it costs no -budget: an exact hit never saves. + +Re-enabling the bundled cache there would be strictly negative, not merely +unhelpful: cache writes are scoped to the ref that made them, so a save from +`refs/tags/v1.0.0` can never be read by `refs/tags/v1.0.1` or anything else +— a ~1 GB write-only entry per release, permanently unreadable. If release +wall-clock ever does matter, the lever is `actions/cache/restore` on +`publish-dev`'s key: restore-only, so it reads `main`'s warm entry and never +writes a tag-scoped one. `goreleaser-validate.yml` opts out on its own grounds: its `--single-target` snapshot is fast enough that the post-step save costs more than a cold diff --git a/.github/workflows/publish-dev.yml b/.github/workflows/publish-dev.yml index 7ff5754b..e1c8ccab 100644 --- a/.github/workflows/publish-dev.yml +++ b/.github/workflows/publish-dev.yml @@ -52,7 +52,8 @@ jobs: go-version-file: "go.mod" # Opt out of setup-go's bundled cache and cache the half that # earns its keep, below. The bundled one stores ~/go/pkg/mod AND - # ~/.cache/go-build under one go.sum-keyed entry (~1 GB stored), so + # ~/.cache/go-build under one entry (~1 GB stored), keyed on the root + # go.mod at setup-go >= v7 (go.sum through v6, actions/setup-go#705), so # roughly half of it re-stores the module tree ci.yml already # keeps once as `gomod-v1` — the duplication #443 is about. cache: false diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 01fd8967..da239673 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -24,12 +24,19 @@ jobs: - uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0 with: go-version-file: "go.mod" - # Opting out keeps a ~1 GB go.sum-keyed archive — the module tree - # plus this job's cross-compile objects — out of the 10 GB repo - # budget. After this change nothing mints that key at all - # (publish-dev.yml and goreleaser-validate.yml opt out too, and - # ci.yml runs no setup-go), so turning it back on here would be a - # cold miss AND a fresh ~1 GB save, not a hit. + # Opting out keeps a ~1 GB archive — the module tree plus this + # job's cross-compile objects — out of the 10 GB repo budget. + # (setup-go >= v7 keys that entry on the root go.mod, not go.sum; + # it hashed go.sum through v6 — actions/setup-go#705.) + # + # Turning it back on here would be strictly negative. Nothing + # mints that key any more — publish-dev.yml and + # goreleaser-validate.yml opt out too, and ci.yml runs no + # setup-go — so the restore is a guaranteed miss. Worse, cache + # writes are scoped to the ref that made them: a save from + # refs/tags/v1.0.0 can never be read by refs/tags/v1.0.1, or by + # anything else. It would be a ~1 GB write-only entry burned on + # every tagged release, permanently unreadable. # # What IS warm is publish-dev's gobuild-v3--go-release- entry, # restorable from the default branch's scope. A tagged release is diff --git a/CHANGELOG.md b/CHANGELOG.md index c5e1fea8..7bf8d367 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,7 +50,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed -- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation (the tree is ~1.6 GB on disk; 0.48 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.48 GB module + 0.57 GB across the five build entries), down from 5.18 GB — a 4.9x reduction. All sizes are stored-archive bytes / 2^30, the unit the README's usage check prints. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB `go.sum`-keyed entry — the module tree `gomod-v1` already keeps once, plus that job's own 8-target cross-compile objects — re-saved on every cache miss. `publish-dev.yml` re-caches only the half that pays for itself, under `gobuild-v3--go-release-` (~0.5 GB — the bundled entry minus `gomod-v1`'s share): across its last 20 runs GoReleaser takes 36–246 s with the cross-compile objects warm and 401–446 s cold (measured on `setup-go`'s bundled cache, which carried the same `~/.cache/go-build` tree), so dropping the cross-compile objects outright would have cost 3–6.5 minutes on every push to main. The `-release` suffix keeps those 8-target objects from being restored by CI's native-only flavors and vice versa. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. +- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation (the tree is ~1.6 GB on disk; 0.48 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.48 GB module + 0.57 GB across the five build entries), down from 5.18 GB — a 4.9x reduction. All sizes are stored-archive bytes / 2^30, the unit the README's usage check prints. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB entry — the module tree `gomod-v1` already keeps once, plus that job's own 8-target cross-compile objects — re-saved on every cache miss. (That entry is keyed on the root `go.mod`: setup-go hashed `go.sum` through v6 and `go.mod` from v7, [actions/setup-go#705](https://github.com/actions/setup-go/pull/705).) `publish-dev.yml` re-caches only the half that pays for itself, under `gobuild-v3--go-release-` (~0.5 GB — the bundled entry minus `gomod-v1`'s share): across its last 20 runs GoReleaser takes 36–246 s with the cross-compile objects warm and 401–446 s cold (measured on `setup-go`'s bundled cache, which carried the same `~/.cache/go-build` tree), so dropping the cross-compile objects outright would have cost 3–6.5 minutes on every push to main. The `-release` suffix keeps those 8-target objects from being restored by CI's native-only flavors and vice versa. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. - **Live demo hero feed renders in `event_ts` order instead of SSE arrival order** (`docs/src/components/LiveDemo.astro`): the landing-page live activity feed prepended each streamed row to the top in the order the SSE stream delivered it, but a producer's webhook burst (a single merge-queue cycle fires ~20 events) arrives in no guaranteed order and the stream relays it in ingest order — so a late or out-of-order delivery landed above newer rows (e.g. a `pushed 12m ago` sitting on top of `reviewed a pull request 9m ago`). `addRow` now keeps the feed sorted by `event_ts` descending — it slots each row in before the first strictly-older sibling rather than blind-prepending — so the live tail matches the already-sorted `gh_activity_recent` backfill. The zone-less-SSE-timestamp normalization the sort relies on (`normTs`) was already in place; equal-second rows keep arrival order (`gh_events.event_ts` is only second-granular for CI/checks, so there's no finer tiebreak), and dedup + the `MAX_ROWS` trim are unchanged. Surfaced in dogfooding on `wavehouse.dev`; the client-side analog of the ingest-order reality the SSE stream can't reorder. From ba37815af0a6cd62cc1cdc0d88777f9b4fb38634 Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Tue, 11 Aug 2026 07:17:17 -0400 Subject: [PATCH 15/19] fix(ci): setup-go switched key file in v6.3.0, not v7 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Third pass at the same sentence, and the version boundary was wrong. Verified against the upstream tree rather than inferred this time: v6.2.0 dependencyFilePattern: 'go.sum' v6.3.0 dependencyFilePattern: 'go.mod' <- the switch v7.0.0 dependencyFilePattern: 'go.mod' actions/setup-go#705 merged 2026-01-26 and shipped in v6.3.0. Saying "go.sum through v6" is worse than vague: the floating v6 tag resolves to v6.5.0 today, which keys on go.mod, so it tells a reader that pinning @v6 gets go.sum keying. All four sites now say v6.2.0 / v6.3.0. Also drops an absolute that isn't true. "A save from refs/tags/v1.0.0 can never be read by anything else, permanently unreadable" overstates the scoping rule it cites: a re-run of that same tag's workflow runs at the same ref and does restore it — and a re-run is the realistic case here, since release.yml chains buildx, GHCR login, GoReleaser and attestation. Now: unreadable by another tag, by main, or by a PR — only by a retry. The conclusion is unchanged; the claim is now falsifiable-proof. And fixes a link that rendered as literal text: [actions/setup-go#705] was a shortcut reference with no definition in the file. markdownlint doesn't catch it (MD052's shortcut_syntax defaults false), and every other external reference in that README is inline. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/workflows/README.md | 19 +++++++++++-------- .github/workflows/publish-dev.yml | 9 +++++---- .github/workflows/release.yml | 11 ++++++----- CHANGELOG.md | 2 +- 4 files changed, 23 insertions(+), 18 deletions(-) diff --git a/.github/workflows/README.md b/.github/workflows/README.md index f7bef546..8bd9754d 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -201,10 +201,12 @@ Deliberately **not** cached: `actions/setup-go`'s bundled cache `goreleaser-validate.yml`) — for different reasons per job. It stores `~/go/pkg/mod` **and** `~/.cache/go-build` under one entry -(~1 GB stored, keyed on the root `go.mod` — setup-go hashed `go.sum` -through v6, `go.mod` from v7, [actions/setup-go#705]), so roughly half -re-stores the module tree `gomod-v1` already keeps once. `publish-dev.yml` opts out of that entry and caches the -half that pays for itself on its own key (`gobuild-v3--go-release-`, +(~1 GB stored), keyed on the root `go.mod` — setup-go hashed `go.sum` +through v6.2.0 and `go.mod` from v6.3.0, see +[actions/setup-go#705](https://github.com/actions/setup-go/pull/705) — so +roughly half of it re-stores the module tree `gomod-v1` already keeps once. +`publish-dev.yml` opts out of that entry and caches the half that pays for +itself on its own key (`gobuild-v3--go-release-`, ~0.5 GB): its GoReleaser step takes 36–246 s warm versus 401–446 s cold, so dropping the build objects outright would cost 3–6.5 minutes on every push to main. @@ -217,10 +219,11 @@ could restore from the default branch's scope — but a tagged release is rare and not latency-sensitive, so it isn't worth a hand-rolled restore step. Re-enabling the bundled cache there would be strictly negative, not merely -unhelpful: cache writes are scoped to the ref that made them, so a save from -`refs/tags/v1.0.0` can never be read by `refs/tags/v1.0.1` or anything else -— a ~1 GB write-only entry per release, permanently unreadable. If release -wall-clock ever does matter, the lever is `actions/cache/restore` on +unhelpful: cache writes are scoped to the ref that made them, so a save +from `refs/tags/v1.0.0` can never be read by `refs/tags/v1.0.1`, by `main`, +or by a PR — only by a re-run of that same tag. It would be a ~1 GB entry +per release that nothing but a retry can ever read. If release wall-clock +ever does matter, the lever is `actions/cache/restore` on `publish-dev`'s key: restore-only, so it reads `main`'s warm entry and never writes a tag-scoped one. diff --git a/.github/workflows/publish-dev.yml b/.github/workflows/publish-dev.yml index e1c8ccab..fe0cc369 100644 --- a/.github/workflows/publish-dev.yml +++ b/.github/workflows/publish-dev.yml @@ -52,10 +52,11 @@ jobs: go-version-file: "go.mod" # Opt out of setup-go's bundled cache and cache the half that # earns its keep, below. The bundled one stores ~/go/pkg/mod AND - # ~/.cache/go-build under one entry (~1 GB stored), keyed on the root - # go.mod at setup-go >= v7 (go.sum through v6, actions/setup-go#705), so - # roughly half of it re-stores the module tree ci.yml already - # keeps once as `gomod-v1` — the duplication #443 is about. + # ~/.cache/go-build under one entry (~1 GB stored), keyed on the + # root go.mod — setup-go hashed go.sum through v6.2.0 and go.mod + # from v6.3.0 (actions/setup-go#705). So roughly half of it + # re-stores the module tree ci.yml already keeps once as + # `gomod-v1` — the duplication #443 is about. cache: false # The other half is the reason this job is fast, so cache it on its diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index da239673..74b1ebb2 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -26,17 +26,18 @@ jobs: go-version-file: "go.mod" # Opting out keeps a ~1 GB archive — the module tree plus this # job's cross-compile objects — out of the 10 GB repo budget. - # (setup-go >= v7 keys that entry on the root go.mod, not go.sum; - # it hashed go.sum through v6 — actions/setup-go#705.) + # (That entry is keyed on the root go.mod, not go.sum: setup-go + # hashed go.sum through v6.2.0 and go.mod from v6.3.0 — + # actions/setup-go#705.) # # Turning it back on here would be strictly negative. Nothing # mints that key any more — publish-dev.yml and # goreleaser-validate.yml opt out too, and ci.yml runs no # setup-go — so the restore is a guaranteed miss. Worse, cache # writes are scoped to the ref that made them: a save from - # refs/tags/v1.0.0 can never be read by refs/tags/v1.0.1, or by - # anything else. It would be a ~1 GB write-only entry burned on - # every tagged release, permanently unreadable. + # refs/tags/v1.0.0 can never be read by refs/tags/v1.0.1, by main, + # or by a PR — only by a re-run of that same tag. It would be a + # ~1 GB entry per release that nothing but a retry can ever read. # # What IS warm is publish-dev's gobuild-v3--go-release- entry, # restorable from the default branch's scope. A tagged release is diff --git a/CHANGELOG.md b/CHANGELOG.md index 7bf8d367..4eafe4d7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,7 +50,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed -- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation (the tree is ~1.6 GB on disk; 0.48 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.48 GB module + 0.57 GB across the five build entries), down from 5.18 GB — a 4.9x reduction. All sizes are stored-archive bytes / 2^30, the unit the README's usage check prints. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB entry — the module tree `gomod-v1` already keeps once, plus that job's own 8-target cross-compile objects — re-saved on every cache miss. (That entry is keyed on the root `go.mod`: setup-go hashed `go.sum` through v6 and `go.mod` from v7, [actions/setup-go#705](https://github.com/actions/setup-go/pull/705).) `publish-dev.yml` re-caches only the half that pays for itself, under `gobuild-v3--go-release-` (~0.5 GB — the bundled entry minus `gomod-v1`'s share): across its last 20 runs GoReleaser takes 36–246 s with the cross-compile objects warm and 401–446 s cold (measured on `setup-go`'s bundled cache, which carried the same `~/.cache/go-build` tree), so dropping the cross-compile objects outright would have cost 3–6.5 minutes on every push to main. The `-release` suffix keeps those 8-target objects from being restored by CI's native-only flavors and vice versa. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. +- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation (the tree is ~1.6 GB on disk; 0.48 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.48 GB module + 0.57 GB across the five build entries), down from 5.18 GB — a 4.9x reduction. All sizes are stored-archive bytes / 2^30, the unit the README's usage check prints. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB entry — the module tree `gomod-v1` already keeps once, plus that job's own 8-target cross-compile objects — re-saved on every cache miss. (That entry is keyed on the root `go.mod`: setup-go hashed `go.sum` through v6.2.0 and `go.mod` from v6.3.0, [actions/setup-go#705](https://github.com/actions/setup-go/pull/705).) `publish-dev.yml` re-caches only the half that pays for itself, under `gobuild-v3--go-release-` (~0.5 GB — the bundled entry minus `gomod-v1`'s share): across its last 20 runs GoReleaser takes 36–246 s with the cross-compile objects warm and 401–446 s cold (measured on `setup-go`'s bundled cache, which carried the same `~/.cache/go-build` tree), so dropping the cross-compile objects outright would have cost 3–6.5 minutes on every push to main. The `-release` suffix keeps those 8-target objects from being restored by CI's native-only flavors and vice versa. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. - **Live demo hero feed renders in `event_ts` order instead of SSE arrival order** (`docs/src/components/LiveDemo.astro`): the landing-page live activity feed prepended each streamed row to the top in the order the SSE stream delivered it, but a producer's webhook burst (a single merge-queue cycle fires ~20 events) arrives in no guaranteed order and the stream relays it in ingest order — so a late or out-of-order delivery landed above newer rows (e.g. a `pushed 12m ago` sitting on top of `reviewed a pull request 9m ago`). `addRow` now keeps the feed sorted by `event_ts` descending — it slots each row in before the first strictly-older sibling rather than blind-prepending — so the live tail matches the already-sorted `gh_activity_recent` backfill. The zone-less-SSE-timestamp normalization the sort relies on (`normTs`) was already in place; equal-second rows keep arrival order (`gh_events.event_ts` is only second-granular for CI/checks, so there's no finer tiebreak), and dedup + the `MAX_ROWS` trim are unchanged. Surfaced in dogfooding on `wavehouse.dev`; the client-side analog of the ingest-order reality the SSE stream can't reorder. From dd0e7ee5e4bda4bd61a0c072f990d296e1ba0692 Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Tue, 11 Aug 2026 07:29:57 -0400 Subject: [PATCH 16/19] fix(ci): keep the shared module cache complete, enforce the suffix MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CodeRabbit's re-review, and the second finding is a real bug this PR introduced. `make cov` never declared the go-mod-download prerequisite its siblings have (lint-go, vulncheck, test-unit, test-integration all do; test-e2e gets it via $(COVER_BINARIES)). That was harmless under gobuild-v2, where every job had its own suffixed entry — a partial module tree only affected the job that saved it. It is not harmless now: gomod-v1 is a single unsuffixed entry every Go job races to save on a key rotation, and the coverage job runs only `go run ./scripts/cov report`, which fetches just the modules that one program imports. If it won that race it would store a PARTIAL ~/go/pkg/mod under the shared key, which then exact-hits for every other job — and because saves fire only on a miss, nothing would ever repair it until the next rotation. Worth noting an earlier review asserted the opposite ("every Go-toolchain make target declares the go-mod-download prereq, so the shared entry is complete no matter which job wins"). Only four targets do, and cov is not one of them. The coverage job is also the likeliest winner: it starts at run creation with no `needs` on the suites. Also enforces the suffix contract instead of only documenting it. setup-env now fails a go: true job that passes no go-cache-suffix, since an empty one yields restore-key gobuild-v3--go-, prefix-matching every flavor's entry and publish-dev's -release one. And tightens ownership wording: gomod-v1 and the gobuild-v3 flavors belong to ci.yml jobs going through setup-env; publish-dev owns only -release, and release.yml caches nothing. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/actions/setup-env/action.yml | 17 +++++++++++++++++ .github/workflows/README.md | 14 ++++++++++---- CHANGELOG.md | 2 +- Makefile | 11 ++++++++++- 4 files changed, 38 insertions(+), 6 deletions(-) diff --git a/.github/actions/setup-env/action.yml b/.github/actions/setup-env/action.yml index b19ee496..151680ff 100644 --- a/.github/actions/setup-env/action.yml +++ b/.github/actions/setup-env/action.yml @@ -79,6 +79,23 @@ outputs: runs: using: composite steps: + # Fail loudly on the one way to misuse the split. An empty suffix on a Go + # job yields restore-key `gobuild-v3--go-`, which prefix-matches every + # flavor's entry AND publish-dev's `-release` one — restoring cross-compiled + # or wrong-flag objects nothing can hit, then saving a sixth entry against + # the sizing policy. Documented in the input description and the README's + # "Adding a job" checklist; enforced here so a future Go job can't acquire + # it silently. + - name: Require a go-cache-suffix on Go jobs + if: ${{ inputs.go == 'true' && inputs.go-cache-suffix == '' }} + shell: bash + run: | + echo "::error title=setup-env::go-cache-suffix is required when go: true." \ + "An empty suffix cross-matches every other flavor's build cache." \ + "Pass a fresh suffix for new compile flags, or an existing flavor's" \ + "if this job compiles identically (see .github/workflows/README.md)." + exit 1 + # The module cache is UNSUFFIXED on purpose. ~/go/pkg/mod is a pure # function of go.mod + go.sum — byte-identical for every compile # flavor — so folding it into the suffixed build cache stored that tree diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 8bd9754d..c941ded1 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -187,8 +187,8 @@ Queue settings live in the `main branch protection` ruleset's | Cache | Key | Saved by | Notes | |---|---|---|---| -| Go modules | `gomod-v1--` | every Go job (shared) | `~/go/pkg/mod`, **unsuffixed** — a pure function of `go.mod` + `go.sum`, so one entry serves every job (~1.6 GB on disk; 0.48 GB stored on a cold save, drifting up as superseded versions accumulate). The GOTOOLCHAIN=auto toolchain rides in here too (no setup-go), which is why `go.mod` is in the key — a `go` directive bump changes the required toolchain without touching `go.sum`. | -| Go build objects | `gobuild-v3--go-` | every Go job (own suffix) | `~/.cache/go-build` only — 25–152 MB stored per flavor. Suffix partitions by compile flavor (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`), which compile with different flags. `go.mod` is in the key for its own reason — the compiler's build ID keys every object, so a toolchain bump invalidates all of them. | +| Go modules | `gomod-v1--` | every `ci.yml` Go job via `setup-env` (shared) | `~/go/pkg/mod`, **unsuffixed** — a pure function of `go.mod` + `go.sum`, so one entry serves every job (~1.6 GB on disk; 0.48 GB stored on a cold save, drifting up as superseded versions accumulate). The GOTOOLCHAIN=auto toolchain rides in here too (no setup-go), which is why `go.mod` is in the key — a `go` directive bump changes the required toolchain without touching `go.sum`. | +| Go build objects | `gobuild-v3--go-` | every `ci.yml` Go job via `setup-env` (own suffix) | `~/.cache/go-build` only — 25–152 MB stored per flavor. Suffix partitions by compile flavor (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`), which compile with different flags. `go.mod` is in the key for its own reason — the compiler's build ID keys every object, so a toolchain bump invalidates all of them. | | golangci binary + analysis | `golangci--` | lint | Analysis cache: ~10s warm vs ~90s. `.bin` also carries shellcheck + actionlint. | | pnpm store | `pnpm--` | any node job on miss | Store path resolved from pnpm at runtime. docs-build prunes before its save on a key rotation. | | Playwright Chromium | `playwright--` | docs-build | rehype-mermaid renders via headless Chrome at docs build. | @@ -337,9 +337,15 @@ suite's wall-clock becomes a problem again, start here: compiles identically. Never leave it empty: the resulting `gobuild-v3--go-` restore-key prefix-matches every flavor's entry (the cross-flavor restore the split exists to avoid) and mints an extra - build entry against the sizing policy above. Never add cache save steps to + build entry against the sizing policy above. `setup-env` now fails the + job outright if `go: true` is passed without a suffix, so this can't be + acquired silently. Never add cache save steps to `ci.yml` (invariant 6); a workflow outside it that needs a cache - hand-rolls one, as `publish-dev.yml` does. + hand-rolls one, as `publish-dev.yml` does. And make sure the job's make + target reaches `go-mod-download`: every Go job races to save the shared + unsuffixed `gomod-v1`, so a job that only fetches the modules it happens + to import can store a partial tree that then exact-hits for everyone + until the next rotation. 4. Need a build product / data from another job? Upload it as an artifact there, then either `needs` the producer + `download-artifact` (simple, but serializes this job's setup behind the producer), or — when this diff --git a/CHANGELOG.md b/CHANGELOG.md index 4eafe4d7..d39eca53 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,7 +50,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed -- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation (the tree is ~1.6 GB on disk; 0.48 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every Go job, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.48 GB module + 0.57 GB across the five build entries), down from 5.18 GB — a 4.9x reduction. All sizes are stored-archive bytes / 2^30, the unit the README's usage check prints. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB entry — the module tree `gomod-v1` already keeps once, plus that job's own 8-target cross-compile objects — re-saved on every cache miss. (That entry is keyed on the root `go.mod`: setup-go hashed `go.sum` through v6.2.0 and `go.mod` from v6.3.0, [actions/setup-go#705](https://github.com/actions/setup-go/pull/705).) `publish-dev.yml` re-caches only the half that pays for itself, under `gobuild-v3--go-release-` (~0.5 GB — the bundled entry minus `gomod-v1`'s share): across its last 20 runs GoReleaser takes 36–246 s with the cross-compile objects warm and 401–446 s cold (measured on `setup-go`'s bundled cache, which carried the same `~/.cache/go-build` tree), so dropping the cross-compile objects outright would have cost 3–6.5 minutes on every push to main. The `-release` suffix keeps those 8-target objects from being restored by CI's native-only flavors and vice versa. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. +- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation (the tree is ~1.6 GB on disk; 0.48 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every `ci.yml` Go job that goes through `setup-env`, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.48 GB module + 0.57 GB across the five build entries), down from 5.18 GB — a 4.9x reduction. All sizes are stored-archive bytes / 2^30, the unit the README's usage check prints. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB entry — the module tree `gomod-v1` already keeps once, plus that job's own 8-target cross-compile objects — re-saved on every cache miss. (That entry is keyed on the root `go.mod`: setup-go hashed `go.sum` through v6.2.0 and `go.mod` from v6.3.0, [actions/setup-go#705](https://github.com/actions/setup-go/pull/705).) `publish-dev.yml` re-caches only the half that pays for itself, under `gobuild-v3--go-release-` (~0.5 GB — the bundled entry minus `gomod-v1`'s share): across its last 20 runs GoReleaser takes 36–246 s with the cross-compile objects warm and 401–446 s cold (measured on `setup-go`'s bundled cache, which carried the same `~/.cache/go-build` tree), so dropping the cross-compile objects outright would have cost 3–6.5 minutes on every push to main. The `-release` suffix keeps those 8-target objects from being restored by CI's native-only flavors and vice versa. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. Two guards come with the shared entry: `setup-env` now fails a `go: true` job that passes no `go-cache-suffix` (an empty one yields a restore-key prefix-matching every other flavor), and `make cov` gains the `go-mod-download` prerequisite its siblings already had — CI's coverage job shares the unsuffixed `gomod-v1` and races to save it, but ran only `go run ./scripts/cov report`, so winning that race would have stored a partial `~/go/pkg/mod` that then exact-hit for every other job until the next rotation. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. - **Live demo hero feed renders in `event_ts` order instead of SSE arrival order** (`docs/src/components/LiveDemo.astro`): the landing-page live activity feed prepended each streamed row to the top in the order the SSE stream delivered it, but a producer's webhook burst (a single merge-queue cycle fires ~20 events) arrives in no guaranteed order and the stream relays it in ingest order — so a late or out-of-order delivery landed above newer rows (e.g. a `pushed 12m ago` sitting on top of `reviewed a pull request 9m ago`). `addRow` now keeps the feed sorted by `event_ts` descending — it slots each row in before the first strictly-older sibling rather than blind-prepending — so the live tail matches the already-sorted `gh_activity_recent` backfill. The zone-less-SSE-timestamp normalization the sort relies on (`normTs`) was already in place; equal-second rows keep arrival order (`gh_events.event_ts` is only second-granular for CI/checks, so there's no finer tiebreak), and dedup + the `MAX_ROWS` trim are unchanged. Surfaced in dogfooding on `wavehouse.dev`; the client-side analog of the ingest-order reality the SSE stream can't reorder. diff --git a/Makefile b/Makefile index de1ebea2..f1472bfd 100644 --- a/Makefile +++ b/Makefile @@ -734,7 +734,16 @@ test-all: ## Run all suites sequentially + one consolidated Go + TS coverage rep # (collect-only). Standalone `make cov` is "show me the numbers without # re-running tests." Fails if NO suite has data (a stray `make cov`). .PHONY: cov -cov: ## Consolidated coverage report (Go + TS) + gate against thresholds (auto-runs after test-all / ci) +# go-mod-download is not optional here even though `go run ./scripts/cov` +# would fetch what it needs on its own. CI's coverage job shares the +# unsuffixed gomod-v1 cache with every other Go job (.github/actions/ +# setup-env), and all of them race to save it on a key rotation. `go run` +# populates only the modules that one program imports, so if this job won +# that race it would store a PARTIAL ~/go/pkg/mod under the shared key — +# which then exact-hits for every other job, forever, until the next +# rotation. Downloading the full graph first keeps the shared entry +# complete whoever wins. See #443. +cov: go-mod-download ## Consolidated coverage report (Go + TS) + gate against thresholds (auto-runs after test-all / ci) @go run ./scripts/cov report ##@ CI From 3c14a9ffde5fd8172ba29feb1908e23cd1167612 Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Tue, 11 Aug 2026 08:04:04 -0400 Subject: [PATCH 17/19] fix(ci): restore the module tree in publish-dev, tighten the guard's advice MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two reviews on dd0e7ee, and the load-bearing item is that the job would have shipped slower than its own documentation claims. The 36-246s "warm" figure justifying publish-dev's hand-rolled cache was measured on setup-go's bundled entry, which carried ~/go/pkg/mod as well. Caching only the build half leaves the module tree cold on every push — ~112 MB re-downloaded — so the job would land above the documented range and read as an unexplained regression to whoever checked next. It now also restores gomod-v1 from main's scope, which ci.yml keeps warm there. Restore-only, deliberately. A full actions/cache adds a post-step save, and this job must never write the entry every ci.yml Go job shares: it runs no full `go mod download`, so a save from here is exactly the partial tree the cov fix in the previous commit exists to prevent. Restore-only writes nothing and costs 0 GB. The guard's error message gave wrong advice to the person likeliest to hit it. `go` defaults to true, so the realistic trigger is a NEW NON-GO job that uses setup-env with no `with:` block — and the message told them to pass a suffix. It now names `go: "false"` as the fix for that case, and the input description states the contract ("REQUIRED when go is true") rather than advising it, since an empty value is now exit 1. Also: the ownership tightening in dd0e7ee missed two sentences written in that same commit (the checklist's "every Go job races to save" and the Makefile's "every other Go job") — both now ci.yml-scoped; CHANGELOG's file list gains Makefile, which that commit modified; the guard comment's "a sixth entry" is off by one now that -release exists; and checklist item 3, having grown to four rules in one paragraph, is now three sub-bullets so the go-mod-download requirement is findable. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/actions/setup-env/action.yml | 17 +++++++------ .github/workflows/README.md | 38 +++++++++++++++++----------- .github/workflows/publish-dev.yml | 17 +++++++++++++ CHANGELOG.md | 2 +- Makefile | 12 ++++----- 5 files changed, 56 insertions(+), 30 deletions(-) diff --git a/.github/actions/setup-env/action.yml b/.github/actions/setup-env/action.yml index 151680ff..2f6665a1 100644 --- a/.github/actions/setup-env/action.yml +++ b/.github/actions/setup-env/action.yml @@ -52,7 +52,7 @@ inputs: description: "Set up the Go toolchain + the Go module/build cache" default: "true" go-cache-suffix: - description: "Per-job Go build-cache partition, e.g. '-unit' (different jobs compile with different flags). Always pass one for a Go job: an empty suffix yields a restore-key that prefix-matches every other flavor's entry, which is the cross-flavor restore the gomod-v1/gobuild-v3 split exists to avoid. The shared part of the cache is gomod-v1, which needs no suffix." + description: "REQUIRED when `go` is true (the default) — the action fails the job without one. Per-job Go build-cache partition, e.g. '-unit' (different jobs compile with different flags); an empty suffix yields a restore-key that prefix-matches every other flavor's entry, the cross-flavor restore the gomod-v1/gobuild-v3 split exists to avoid. A job that doesn't build Go passes `go: \"false\"` instead. The shared part of the cache is gomod-v1, which needs no suffix. The default stays empty because a composite action's `required:` isn't enforced by the runner — the guard step is." default: "" golangci: description: "Cache the golangci-lint binary + analysis cache (lint job only)" @@ -82,18 +82,19 @@ runs: # Fail loudly on the one way to misuse the split. An empty suffix on a Go # job yields restore-key `gobuild-v3--go-`, which prefix-matches every # flavor's entry AND publish-dev's `-release` one — restoring cross-compiled - # or wrong-flag objects nothing can hit, then saving a sixth entry against - # the sizing policy. Documented in the input description and the README's - # "Adding a job" checklist; enforced here so a future Go job can't acquire - # it silently. + # or wrong-flag objects nothing can hit, then saving an extra entry + # against the sizing policy. Documented in the input description and the README's + # "Adding a job" checklist; enforced here so a new job can't inherit the + # empty default silently. - name: Require a go-cache-suffix on Go jobs if: ${{ inputs.go == 'true' && inputs.go-cache-suffix == '' }} shell: bash run: | - echo "::error title=setup-env::go-cache-suffix is required when go: true." \ + echo "::error title=setup-env::go-cache-suffix is required when go is true (the default)." \ "An empty suffix cross-matches every other flavor's build cache." \ - "Pass a fresh suffix for new compile flags, or an existing flavor's" \ - "if this job compiles identically (see .github/workflows/README.md)." + "If this job builds Go: pass a fresh suffix for new compile flags, or an" \ + "existing flavor's if it compiles identically. If it does not build Go:" \ + "pass go: \"false\". See .github/workflows/README.md." exit 1 # The module cache is UNSUFFIXED on purpose. ~/go/pkg/mod is a pure diff --git a/.github/workflows/README.md b/.github/workflows/README.md index c941ded1..7262bbb1 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -194,6 +194,7 @@ Queue settings live in the `main branch protection` ruleset's | Playwright Chromium | `playwright--` | docs-build | rehype-mermaid renders via headless Chrome at docs build. | | Astro content collections | `astro--` | lint / docs-build | Warm `astro check`/`build` skip unchanged content. | | Go build objects (release) | `gobuild-v3--go-release-` | publish-dev (hand-rolled, not `setup-env`) | `~/.cache/go-build` from GoReleaser's 8-target cross-compile (~0.5 GB). Same family and key inputs as the CI flavors, `-release` suffix because cross-compiled objects share nothing with the native-only ones. Worth 3–6.5 min on every push to main. | +| Go modules (release read) | `gomod-v1--` | nobody — **restore-only** | `publish-dev` reads `ci.yml`'s shared entry from `main`'s scope via `actions/cache/restore`, so its cross-compile isn't slowed by a cold module tree. No post-step save, so 0 GB of budget and no risk of a partial write to the shared key. | | CodeQL DB + deps | `codeql-dependencies-*`, `codeql-overlay-base-database-*` | GHAS default setup | **Not ours** — minted by GitHub's default CodeQL setup, not by any workflow in this repo, and not configurable here. ~0.4 GB. Listed so the budget arithmetic below is honest. | Deliberately **not** cached: `actions/setup-go`'s bundled cache @@ -209,7 +210,10 @@ roughly half of it re-stores the module tree `gomod-v1` already keeps once. itself on its own key (`gobuild-v3--go-release-`, ~0.5 GB): its GoReleaser step takes 36–246 s warm versus 401–446 s cold, so dropping the build objects outright would cost 3–6.5 minutes on every push -to main. +to main. Those timings were measured with setup-go's bundled entry, which +also held `~/go/pkg/mod` — so `publish-dev` additionally *restores* (never +saves) `gomod-v1` from `main`'s scope, keeping the module tree warm without +which it would re-download ~112 MB per push and land above that range. `release.yml` keeps the plain opt-out — no re-cache. After this change nothing mints a `setup-go-*` key at all, so turning its bundled cache back @@ -332,20 +336,24 @@ suite's wall-clock becomes a problem again, start here: 2. Gate on the change set via `needs: changes` + `if:` on its outputs — never with workflow-level `paths` filters (they'd orphan the required check, invariant 1). -3. Use `setup-env`, and **always** pass a `go-cache-suffix` if the job - compiles Go — a fresh one for new flags, an existing flavor's if it - compiles identically. Never leave it empty: the resulting - `gobuild-v3--go-` restore-key prefix-matches every flavor's entry - (the cross-flavor restore the split exists to avoid) and mints an extra - build entry against the sizing policy above. `setup-env` now fails the - job outright if `go: true` is passed without a suffix, so this can't be - acquired silently. Never add cache save steps to - `ci.yml` (invariant 6); a workflow outside it that needs a cache - hand-rolls one, as `publish-dev.yml` does. And make sure the job's make - target reaches `go-mod-download`: every Go job races to save the shared - unsuffixed `gomod-v1`, so a job that only fetches the modules it happens - to import can store a partial tree that then exact-hits for everyone - until the next rotation. +3. Use `setup-env`. Three rules come with it: + - **Pass a `go-cache-suffix` if the job compiles Go** — a fresh one for + new flags, an existing flavor's if it compiles identically. Never + empty: the resulting `gobuild-v3--go-` restore-key prefix-matches + every flavor's entry (the cross-flavor restore the split exists to + avoid) and mints an extra build entry against the sizing policy above. + `setup-env` fails the job outright whenever `go` is true — which is + the **default** — and no suffix is passed, so a new job can't inherit + the empty default silently. A job that doesn't build Go passes + `go: "false"` instead, as the docs jobs do. + - **Make sure the job's make target reaches `go-mod-download`.** Every + `ci.yml` Go job races to save the shared unsuffixed `gomod-v1`, so a + job that only fetches the modules it happens to import can store a + partial tree that then exact-hits for everyone until the next + rotation. This is why `cov` carries the prerequisite. + - **Never add cache save steps to `ci.yml`** (invariant 6). A workflow + outside it that needs a cache hand-rolls one, as `publish-dev.yml` + does. 4. Need a build product / data from another job? Upload it as an artifact there, then either `needs` the producer + `download-artifact` (simple, but serializes this job's setup behind the producer), or — when this diff --git a/.github/workflows/publish-dev.yml b/.github/workflows/publish-dev.yml index fe0cc369..47cd363b 100644 --- a/.github/workflows/publish-dev.yml +++ b/.github/workflows/publish-dev.yml @@ -80,6 +80,23 @@ jobs: restore-keys: | gobuild-v3-${{ runner.os }}-go-release- + # And read — never write — ci.yml's shared module tree, so dropping + # setup-go's bundled cache doesn't leave this job re-downloading ~112 MB + # of modules on every push. This workflow runs on main, the same scope + # ci.yml saves `gomod-v1` into, so the entry is there to hit. + # + # restore, not cache: a full actions/cache would add a post-step save, + # and this job has no business writing the entry every ci.yml Go job + # shares — it never runs `go mod download` for the full graph (see the + # cov note in the Makefile for why a partial save there is corrosive). + # Restore-only writes nothing, so it costs 0 GB of the budget. + - uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 + with: + path: ~/go/pkg/mod + key: gomod-v1-${{ runner.os }}-${{ hashFiles('**/go.mod', '**/go.sum') }} + restore-keys: | + gomod-v1-${{ runner.os }}- + # dockers_v2 builds the linux/amd64+arm64 manifest via `docker buildx`; # the GitHub-hosted runner's default docker driver can't build # multi-platform images, so create a docker-container builder. No QEMU diff --git a/CHANGELOG.md b/CHANGELOG.md index d39eca53..6081e543 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,7 +50,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed -- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation (the tree is ~1.6 GB on disk; 0.48 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every `ci.yml` Go job that goes through `setup-env`, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.48 GB module + 0.57 GB across the five build entries), down from 5.18 GB — a 4.9x reduction. All sizes are stored-archive bytes / 2^30, the unit the README's usage check prints. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB entry — the module tree `gomod-v1` already keeps once, plus that job's own 8-target cross-compile objects — re-saved on every cache miss. (That entry is keyed on the root `go.mod`: setup-go hashed `go.sum` through v6.2.0 and `go.mod` from v6.3.0, [actions/setup-go#705](https://github.com/actions/setup-go/pull/705).) `publish-dev.yml` re-caches only the half that pays for itself, under `gobuild-v3--go-release-` (~0.5 GB — the bundled entry minus `gomod-v1`'s share): across its last 20 runs GoReleaser takes 36–246 s with the cross-compile objects warm and 401–446 s cold (measured on `setup-go`'s bundled cache, which carried the same `~/.cache/go-build` tree), so dropping the cross-compile objects outright would have cost 3–6.5 minutes on every push to main. The `-release` suffix keeps those 8-target objects from being restored by CI's native-only flavors and vice versa. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. Two guards come with the shared entry: `setup-env` now fails a `go: true` job that passes no `go-cache-suffix` (an empty one yields a restore-key prefix-matching every other flavor), and `make cov` gains the `go-mod-download` prerequisite its siblings already had — CI's coverage job shares the unsuffixed `gomod-v1` and races to save it, but ran only `go run ./scripts/cov report`, so winning that race would have stored a partial `~/go/pkg/mod` that then exact-hit for every other job until the next rotation. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. +- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`, `Makefile`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation (the tree is ~1.6 GB on disk; 0.48 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every `ci.yml` Go job that goes through `setup-env`, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.48 GB module + 0.57 GB across the five build entries), down from 5.18 GB — a 4.9x reduction. All sizes are stored-archive bytes / 2^30, the unit the README's usage check prints. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB entry — the module tree `gomod-v1` already keeps once, plus that job's own 8-target cross-compile objects — re-saved on every cache miss. (That entry is keyed on the root `go.mod`: setup-go hashed `go.sum` through v6.2.0 and `go.mod` from v6.3.0, [actions/setup-go#705](https://github.com/actions/setup-go/pull/705).) `publish-dev.yml` re-caches only the half that pays for itself, under `gobuild-v3--go-release-` (~0.5 GB — the bundled entry minus `gomod-v1`'s share): across its last 20 runs GoReleaser takes 36–246 s with the cross-compile objects warm and 401–446 s cold (measured on `setup-go`'s bundled cache, which carried the same `~/.cache/go-build` tree), so dropping the cross-compile objects outright would have cost 3–6.5 minutes on every push to main. The `-release` suffix keeps those 8-target objects from being restored by CI's native-only flavors and vice versa. Because those timings were taken with setup-go's bundled entry (which also held `~/go/pkg/mod`), `publish-dev` additionally *restores* `gomod-v1` from `main`'s scope via `actions/cache/restore` — read-only, so it costs no budget and cannot write a partial tree to the key every `ci.yml` Go job shares — without which it would re-download ~112 MB of modules per push. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. Two guards come with the shared entry: `setup-env` now fails a `go: true` job that passes no `go-cache-suffix` (an empty one yields a restore-key prefix-matching every other flavor), and `make cov` gains the `go-mod-download` prerequisite its siblings already had — CI's coverage job shares the unsuffixed `gomod-v1` and races to save it, but ran only `go run ./scripts/cov report`, so winning that race would have stored a partial `~/go/pkg/mod` that then exact-hit for every other job until the next rotation. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. - **Live demo hero feed renders in `event_ts` order instead of SSE arrival order** (`docs/src/components/LiveDemo.astro`): the landing-page live activity feed prepended each streamed row to the top in the order the SSE stream delivered it, but a producer's webhook burst (a single merge-queue cycle fires ~20 events) arrives in no guaranteed order and the stream relays it in ingest order — so a late or out-of-order delivery landed above newer rows (e.g. a `pushed 12m ago` sitting on top of `reviewed a pull request 9m ago`). `addRow` now keeps the feed sorted by `event_ts` descending — it slots each row in before the first strictly-older sibling rather than blind-prepending — so the live tail matches the already-sorted `gh_activity_recent` backfill. The zone-less-SSE-timestamp normalization the sort relies on (`normTs`) was already in place; equal-second rows keep arrival order (`gh_events.event_ts` is only second-granular for CI/checks, so there's no finer tiebreak), and dedup + the `MAX_ROWS` trim are unchanged. Surfaced in dogfooding on `wavehouse.dev`; the client-side analog of the ingest-order reality the SSE stream can't reorder. diff --git a/Makefile b/Makefile index f1472bfd..ead64f89 100644 --- a/Makefile +++ b/Makefile @@ -736,12 +736,12 @@ test-all: ## Run all suites sequentially + one consolidated Go + TS coverage rep .PHONY: cov # go-mod-download is not optional here even though `go run ./scripts/cov` # would fetch what it needs on its own. CI's coverage job shares the -# unsuffixed gomod-v1 cache with every other Go job (.github/actions/ -# setup-env), and all of them race to save it on a key rotation. `go run` -# populates only the modules that one program imports, so if this job won -# that race it would store a PARTIAL ~/go/pkg/mod under the shared key — -# which then exact-hits for every other job, forever, until the next -# rotation. Downloading the full graph first keeps the shared entry +# unsuffixed gomod-v1 cache with every other ci.yml Go job (via +# .github/actions/setup-env), and all of them race to save it on a key +# rotation. `go run` populates only the modules that one program imports, +# so if this job won that race it would store a PARTIAL ~/go/pkg/mod under +# the shared key — which then exact-hits for every other job, forever, +# until the next rotation. Downloading the full graph first keeps the shared entry # complete whoever wins. See #443. cov: go-mod-download ## Consolidated coverage report (Go + TS) + gate against thresholds (auto-runs after test-all / ci) @go run ./scripts/cov report From 11ea5633d6638cf16e3f040d5c96a4771dc5cf30 Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Tue, 11 Aug 2026 08:18:43 -0400 Subject: [PATCH 18/19] docs(ci): invariant 6 now covers both cache steps outside the composite MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The branch drifted against itself. ab9cfe7 amended invariant 6 specifically to name the ONE cache living outside setup-env — "It is the only actions/cache@ in the repo outside the composite — keep it that way" — and 3c14a9f added a second four commits later. The literal string actions/cache@ is still unique only because the new one is actions/cache/restore@, which is a pun rather than a contract: a maintainer grepping actions/cache finds two steps and no way to tell whether the second violates the rule they were just given. Invariant 6 is the normative list, so it now names both — the bare actions/cache owning the release build cache, and the actions/cache/restore that reads gomod-v1 and owns nothing — and states the principle behind the asymmetry: a workflow that needs the shared module tree reads it restore-only; writing it belongs to the ci.yml jobs that run a full go mod download. Verified there are exactly two `uses: actions/cache*` outside the composite and six inside. Also re-flows the two comment blocks the previous commit's rewording left one line wide (Makefile 82 cols in a 63-74 paragraph, action.yml 85 in a 71-82 one). Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/actions/setup-env/action.yml | 10 +++++----- .github/workflows/README.md | 13 ++++++++----- Makefile | 4 ++-- 3 files changed, 15 insertions(+), 12 deletions(-) diff --git a/.github/actions/setup-env/action.yml b/.github/actions/setup-env/action.yml index 2f6665a1..e7b1b76b 100644 --- a/.github/actions/setup-env/action.yml +++ b/.github/actions/setup-env/action.yml @@ -81,11 +81,11 @@ runs: steps: # Fail loudly on the one way to misuse the split. An empty suffix on a Go # job yields restore-key `gobuild-v3--go-`, which prefix-matches every - # flavor's entry AND publish-dev's `-release` one — restoring cross-compiled - # or wrong-flag objects nothing can hit, then saving an extra entry - # against the sizing policy. Documented in the input description and the README's - # "Adding a job" checklist; enforced here so a new job can't inherit the - # empty default silently. + # flavor's entry AND publish-dev's `-release` one — restoring + # cross-compiled or wrong-flag objects nothing can hit, then saving an + # extra entry against the sizing policy. Documented in the input + # description and the README's "Adding a job" checklist; enforced here + # so a new job can't inherit the empty default silently. - name: Require a go-cache-suffix on Go jobs if: ${{ inputs.go == 'true' && inputs.go-cache-suffix == '' }} shell: bash diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 7262bbb1..e21f788d 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -125,11 +125,14 @@ Break one of these knowingly or not at all. automatically at job end on an exact-key miss. No save steps in `ci.yml`. Trade-offs accepted: failed jobs don't save (restore-keys cushion the next run), and concurrent same-key misses produce benign - "already exists" warnings. **One cache lives outside it**: - `publish-dev.yml`'s release build cache is a bare `actions/cache`, - because that workflow doesn't use `setup-env` at all (it runs - GoReleaser, not the test suites). It is the only `actions/cache@` in - the repo outside the composite — keep it that way. + "already exists" warnings. **Two cache steps live outside it**, both in + `publish-dev.yml`, because that workflow doesn't use `setup-env` at all + (it runs GoReleaser, not the test suites): a bare `actions/cache` owning + the release build cache, and an `actions/cache/restore` that *reads* + `gomod-v1` and owns nothing. Those two are the only `actions/cache*` + steps outside the composite — keep it that way. A workflow that needs + the shared module tree reads it restore-only; writing it belongs to the + `ci.yml` jobs that run a full `go mod download`. ## Coverage publishing diff --git a/Makefile b/Makefile index ead64f89..e49fa3b2 100644 --- a/Makefile +++ b/Makefile @@ -741,8 +741,8 @@ test-all: ## Run all suites sequentially + one consolidated Go + TS coverage rep # rotation. `go run` populates only the modules that one program imports, # so if this job won that race it would store a PARTIAL ~/go/pkg/mod under # the shared key — which then exact-hits for every other job, forever, -# until the next rotation. Downloading the full graph first keeps the shared entry -# complete whoever wins. See #443. +# until the next rotation. Downloading the full graph first keeps the +# shared entry complete whoever wins. See #443. cov: go-mod-download ## Consolidated coverage report (Go + TS) + gate against thresholds (auto-runs after test-all / ci) @go run ./scripts/cov report From 7aa23264509d76347e006fe8657f6cb6a0ac7254 Mon Sep 17 00:00:00 2001 From: Eric Andrechek Date: Tue, 11 Aug 2026 08:22:25 -0400 Subject: [PATCH 19/19] docs(ci): the 3-6.5 min claim doesn't follow from its own numbers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The warm/cold clusters are 36-246s and 401-446s, so the envelope between them is 401-246 = 155s to 446-36 = 410s: roughly 2.5-7 minutes, mean delta ~4.8. "3-6.5" is a narrower claim than the data supports, and the sentence invites the subtraction by saying "so". Corrected in all four places it appears (README table row and prose, CHANGELOG, publish-dev.yml comment). Also unpicks a dangling "without which" in the README and CHANGELOG. The CHANGELOG version put twenty-odd words and a plausible-but-wrong antecedent between the pronoun and its referent, so it first parses as "without the key every ci.yml Go job shares" — the opposite of the point. Both now end the clause and start a sentence. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QFfYUaPp3Pv3gLjniiHCpm --- .github/workflows/README.md | 11 ++++++----- .github/workflows/publish-dev.yml | 3 ++- CHANGELOG.md | 2 +- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/.github/workflows/README.md b/.github/workflows/README.md index e21f788d..cc5c4d62 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -196,7 +196,7 @@ Queue settings live in the `main branch protection` ruleset's | pnpm store | `pnpm--` | any node job on miss | Store path resolved from pnpm at runtime. docs-build prunes before its save on a key rotation. | | Playwright Chromium | `playwright--` | docs-build | rehype-mermaid renders via headless Chrome at docs build. | | Astro content collections | `astro--` | lint / docs-build | Warm `astro check`/`build` skip unchanged content. | -| Go build objects (release) | `gobuild-v3--go-release-` | publish-dev (hand-rolled, not `setup-env`) | `~/.cache/go-build` from GoReleaser's 8-target cross-compile (~0.5 GB). Same family and key inputs as the CI flavors, `-release` suffix because cross-compiled objects share nothing with the native-only ones. Worth 3–6.5 min on every push to main. | +| Go build objects (release) | `gobuild-v3--go-release-` | publish-dev (hand-rolled, not `setup-env`) | `~/.cache/go-build` from GoReleaser's 8-target cross-compile (~0.5 GB). Same family and key inputs as the CI flavors, `-release` suffix because cross-compiled objects share nothing with the native-only ones. Worth ≈2.5–7 min on every push to main (mean delta ≈4.8 min). | | Go modules (release read) | `gomod-v1--` | nobody — **restore-only** | `publish-dev` reads `ci.yml`'s shared entry from `main`'s scope via `actions/cache/restore`, so its cross-compile isn't slowed by a cold module tree. No post-step save, so 0 GB of budget and no risk of a partial write to the shared key. | | CodeQL DB + deps | `codeql-dependencies-*`, `codeql-overlay-base-database-*` | GHAS default setup | **Not ours** — minted by GitHub's default CodeQL setup, not by any workflow in this repo, and not configurable here. ~0.4 GB. Listed so the budget arithmetic below is honest. | @@ -212,11 +212,12 @@ roughly half of it re-stores the module tree `gomod-v1` already keeps once. `publish-dev.yml` opts out of that entry and caches the half that pays for itself on its own key (`gobuild-v3--go-release-`, ~0.5 GB): its GoReleaser step takes 36–246 s warm versus 401–446 s cold, so -dropping the build objects outright would cost 3–6.5 minutes on every push -to main. Those timings were measured with setup-go's bundled entry, which +dropping the build objects outright would cost roughly 2.5–7 minutes on +every push to main (mean delta ≈4.8 min across those runs). Those timings were measured with setup-go's bundled entry, which also held `~/go/pkg/mod` — so `publish-dev` additionally *restores* (never -saves) `gomod-v1` from `main`'s scope, keeping the module tree warm without -which it would re-download ~112 MB per push and land above that range. +saves) `gomod-v1` from `main`'s scope, keeping the module tree warm too. +Without that restore the job would re-download ~112 MB per push and land +above the warm range this table quotes. `release.yml` keeps the plain opt-out — no re-cache. After this change nothing mints a `setup-go-*` key at all, so turning its bundled cache back diff --git a/.github/workflows/publish-dev.yml b/.github/workflows/publish-dev.yml index 47cd363b..a09e6002 100644 --- a/.github/workflows/publish-dev.yml +++ b/.github/workflows/publish-dev.yml @@ -65,7 +65,8 @@ jobs: # warm: GoReleaser finished in 36-246s with them, 401-446s without. # (Measured on setup-go's bundled entry, which carried this same # ~/.cache/go-build tree — this key is new here.) Dropping them - # outright would cost 3-6.5 minutes on every push to main. + # outright would cost roughly 2.5-7 min per push to main (the envelope + # between those two clusters; mean delta ~4.8 min). # # Release-scoped suffix: these objects are cross-compiled for 8 # GOOS/GOARCH pairs and share nothing with ci.yml's native-only diff --git a/CHANGELOG.md b/CHANGELOG.md index 6081e543..d1594f03 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -50,7 +50,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed -- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`, `Makefile`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation (the tree is ~1.6 GB on disk; 0.48 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every `ci.yml` Go job that goes through `setup-env`, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.48 GB module + 0.57 GB across the five build entries), down from 5.18 GB — a 4.9x reduction. All sizes are stored-archive bytes / 2^30, the unit the README's usage check prints. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB entry — the module tree `gomod-v1` already keeps once, plus that job's own 8-target cross-compile objects — re-saved on every cache miss. (That entry is keyed on the root `go.mod`: setup-go hashed `go.sum` through v6.2.0 and `go.mod` from v6.3.0, [actions/setup-go#705](https://github.com/actions/setup-go/pull/705).) `publish-dev.yml` re-caches only the half that pays for itself, under `gobuild-v3--go-release-` (~0.5 GB — the bundled entry minus `gomod-v1`'s share): across its last 20 runs GoReleaser takes 36–246 s with the cross-compile objects warm and 401–446 s cold (measured on `setup-go`'s bundled cache, which carried the same `~/.cache/go-build` tree), so dropping the cross-compile objects outright would have cost 3–6.5 minutes on every push to main. The `-release` suffix keeps those 8-target objects from being restored by CI's native-only flavors and vice versa. Because those timings were taken with setup-go's bundled entry (which also held `~/go/pkg/mod`), `publish-dev` additionally *restores* `gomod-v1` from `main`'s scope via `actions/cache/restore` — read-only, so it costs no budget and cannot write a partial tree to the key every `ci.yml` Go job shares — without which it would re-download ~112 MB of modules per push. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. Two guards come with the shared entry: `setup-env` now fails a `go: true` job that passes no `go-cache-suffix` (an empty one yields a restore-key prefix-matching every other flavor), and `make cov` gains the `go-mod-download` prerequisite its siblings already had — CI's coverage job shares the unsuffixed `gomod-v1` and races to save it, but ran only `go run ./scripts/cov report`, so winning that race would have stored a partial `~/go/pkg/mod` that then exact-hit for every other job until the next rotation. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. +- **Go module cache stored once instead of once per compile flavor** (`.github/actions/setup-env/action.yml`, `.github/workflows/README.md`, `.github/workflows/publish-dev.yml`, `.github/workflows/release.yml`, `Makefile`): closes [#443](https://github.com/Wave-RF/WaveHouse/issues/443). `setup-env` cached `~/go/pkg/mod` together with `~/.cache/go-build` under a key partitioned by `go-cache-suffix`, but the module cache is a pure function of `go.mod` + `go.sum` and byte-identical for every flavor — so that tree was stored five times over (`-lint`, `-unit`, `-integration`, `-e2e-cov`, `-cov`) — five entries of ~0.9-1.2 GB stored each, ~5.2 GB per generation (the tree is ~1.6 GB on disk; 0.48 GB as a stored archive on a cold save, drifting up as superseded versions accumulate). Two live generations is the steady state (a bump mints a new set while the previous is still warm), so the repo sat near GitHub's hard 10 GB cache cap; the 24-module go-deps bump ([#438](https://github.com/Wave-RF/WaveHouse/pull/438)) tipped it to 10.53 GB and GitHub began LRU-evicting warm entries mid-run. The one cache is now two: `gomod-v1--` on `~/go/pkg/mod`, **unsuffixed** and shared by every `ci.yml` Go job that goes through `setup-env`, and `gobuild-v3--go-` on `~/.cache/go-build` only, still per flavor. Measured after the split: **1.05 GB per generation** (0.48 GB module + 0.57 GB across the five build entries), down from 5.18 GB — a 4.9x reduction. All sizes are stored-archive bytes / 2^30, the unit the README's usage check prints. The `v3` bump is load-bearing — saves fire only on an exact-key miss, so without it the old `v2` entry (still carrying the module cache) would exact-hit forever and the smaller content would never be saved — and `gobuild-v3` drops the bare-prefix restore-key, which existed solely to borrow another flavor's copy of the module cache. Separately, `publish-dev.yml` and `release.yml` now pass `cache: false` to `actions/setup-go` (matching `goreleaser-validate.yml`), which was holding a sixth ~1 GB entry — the module tree `gomod-v1` already keeps once, plus that job's own 8-target cross-compile objects — re-saved on every cache miss. (That entry is keyed on the root `go.mod`: setup-go hashed `go.sum` through v6.2.0 and `go.mod` from v6.3.0, [actions/setup-go#705](https://github.com/actions/setup-go/pull/705).) `publish-dev.yml` re-caches only the half that pays for itself, under `gobuild-v3--go-release-` (~0.5 GB — the bundled entry minus `gomod-v1`'s share): across its last 20 runs GoReleaser takes 36–246 s with the cross-compile objects warm and 401–446 s cold (measured on `setup-go`'s bundled cache, which carried the same `~/.cache/go-build` tree), so dropping the cross-compile objects outright would have cost roughly 2.5–7 minutes on every push to main (mean delta ≈4.8 min). The `-release` suffix keeps those 8-target objects from being restored by CI's native-only flavors and vice versa. Because those timings were taken with setup-go's bundled entry (which also held `~/go/pkg/mod`), `publish-dev` additionally *restores* `gomod-v1` from `main`'s scope via `actions/cache/restore` — read-only, so it costs no budget and cannot write a partial tree to the key every `ci.yml` Go job shares. Without that restore the job would re-download ~112 MB of modules per push and land above the warm range quoted above. Both Go keys now hash `go.mod` alongside `go.sum`, for a different reason each: the GOTOOLCHAIN=auto toolchain lives in `~/go/pkg/mod` and `go.sum` records no entry for it, so a `go`-directive bump would otherwise exact-hit a toolchain-less archive and — saves firing only on an exact-key miss — re-download it every run; and the compiler's build ID keys every build object, so the same bump invalidates `~/.cache/go-build` too, where the failure mode is a permanent cold recompile rather than a re-download. Two guards come with the shared entry: `setup-env` now fails a `go: true` job that passes no `go-cache-suffix` (an empty one yields a restore-key prefix-matching every other flavor), and `make cov` gains the `go-mod-download` prerequisite its siblings already had — CI's coverage job shares the unsuffixed `gomod-v1` and races to save it, but ran only `go run ./scripts/cov report`, so winning that race would have stored a partial `~/go/pkg/mod` that then exact-hit for every other job until the next rotation. The workflows README gains a sizing policy — the 10 GB cap, the two-generations rule, how to check the current footprint, and the rule that lockfile-derived content is keyed once and shared — plus the narrowing-rotation exception to the key-versioning policy. - **Live demo hero feed renders in `event_ts` order instead of SSE arrival order** (`docs/src/components/LiveDemo.astro`): the landing-page live activity feed prepended each streamed row to the top in the order the SSE stream delivered it, but a producer's webhook burst (a single merge-queue cycle fires ~20 events) arrives in no guaranteed order and the stream relays it in ingest order — so a late or out-of-order delivery landed above newer rows (e.g. a `pushed 12m ago` sitting on top of `reviewed a pull request 9m ago`). `addRow` now keeps the feed sorted by `event_ts` descending — it slots each row in before the first strictly-older sibling rather than blind-prepending — so the live tail matches the already-sorted `gh_activity_recent` backfill. The zone-less-SSE-timestamp normalization the sort relies on (`normTs`) was already in place; equal-second rows keep arrival order (`gh_events.event_ts` is only second-granular for CI/checks, so there's no finer tiebreak), and dedup + the `MAX_ROWS` trim are unchanged. Surfaced in dogfooding on `wavehouse.dev`; the client-side analog of the ingest-order reality the SSE stream can't reorder.