Skip to content

ci(root): stop a cancelled run from publishing a half-built turbo cache - #667

Merged
mobeenabdullah merged 2 commits into
mainfrom
ci/turbo-cache-integrity
Aug 11, 2026
Merged

mobeenabdullah merged 2 commits into
mainfrom
ci/turbo-cache-integrity

Conversation

@mobeenabdullah

@mobeenabdullah mobeenabdullah commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

CI is failing PRs on Lint / Typecheck / Test / Build with errors like:

Cannot find module '.../packages/adapter-drizzle/dist/version-check.mjs'
  imported from '.../packages/nextly/dist/chunk-EQL5KHCP.mjs'
Cannot find module '.../packages/nextly/dist/chunk-5YJCDDRX.mjs'
  imported from '.../packages/nextly/dist/index.mjs'

in packages the pull request never touched.

The mechanism, with the evidence

Six jobs across four workflows cache .turbo under one key, turbo-${{ runner.os }}-${{ github.sha }}, with restore-keys: turbo-${{ runner.os }}-. On a pull request, ci (×2 jobs), integration (×2) and preview all run concurrently on the same commit — and they build different filter sets:

job builds
ci ./packages/* and ./apps/*
e2e ./packages/*
preview ./packages/*
integration, integration-sqlite five named packages

One key, five concurrent writers, first one wins.

The timestamps prove it happened. #661's turbo cache was created at 11:12:36. The Preview release workflow for that same commit ran 11:06:05→11:12:39 — three seconds later. CI was still running (11:06:05→11:32:02) and had not reached its post step. The cache that ci later restored was written by preview.

A job then restores a .turbo shaped by someone else's build, reports cache hit, replaying logs, and lays down a dist/ that does not contain what its own tests import. The failing build never ran: the log says nextly:build → cache hit, replaying logs c587c5cd93b644a4. Every package sets clean: true, so a build that runs cannot leave a partial dist/ — which is what pointed at the cache in the first place.

And it cannot be cleared by hand: the key is the commit, so re-running the failed job scores an exact hit on the same foreign entry. That is the first thing anyone tries, and it is the one thing that cannot work.

The fix

Scope the key to the jobturbo-v2-${{ runner.os }}-${{ github.job }}-${{ github.sha }}, restore-keys likewise. Each job restores its own lineage. Nothing is lost: a job's own previous runs are what its cache was ever useful for, and turbo entries are content-addressed, so a narrower cache is a slower run and never a wrong one.

Split restore from save and gate the save on success(). actions/cache saves in a post step that runs however the job ended; splitting is the supported way to make it conditional, now that save-always is deprecated. Independent of the sharing bug, and cheap.

Bump the prefix to turbo-v2-. Keys are immutable and restore-keys prefers the newest match, so the entries already written under the shared scheme would keep being selected. This retires them in one step and unblocks the PRs that are red right now.

The Playwright browser cache in ci.yml is deliberately untouched — it is keyed on the lockfile hash and is a different concern.

A correction to my first pass

The first version of this PR blamed cancel-in-progress cancelling runs mid-build, and gated the save on that basis. Correlating every cache entry against its run showed caches being created mid-run, which cancellation does not explain and concurrent writers do. The save gating stayed because it is independently correct; the key scoping is the fix.

Verification

  • All four workflows parse. A structural check asserts, for each of the six jobs: exactly one restore and one save, save last in the job, identical keys, path: .turbo, if: success() present, and ${{ github.job }} in both key and restore-keys.
  • No monolithic actions/cache@ remains for .turbo.

No changeset — CI-only, per AGENTS.md.

@mobeenabdullah

Copy link
Copy Markdown
Collaborator Author

@codex please review this PR

@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@mobeenabdullah, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 2 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: f297d5b2-6bc1-4d6a-9e26-43dbf441d88e

📥 Commits

Reviewing files that changed from the base of the PR and between f054383 and 1dee827.

📒 Files selected for processing (4)
  • .github/workflows/ci.yml
  • .github/workflows/integration.yml
  • .github/workflows/preview.yml
  • .github/workflows/release.yml

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pkg-pr-new

pkg-pr-new Bot commented Aug 11, 2026

Copy link
Copy Markdown

Open in StackBlitz

@nextlyhq/adapter-drizzle

npm i https://pkg.pr.new/@nextlyhq/adapter-drizzle@1dee827

@nextlyhq/adapter-mysql

npm i https://pkg.pr.new/@nextlyhq/adapter-mysql@1dee827

@nextlyhq/adapter-postgres

npm i https://pkg.pr.new/@nextlyhq/adapter-postgres@1dee827

@nextlyhq/adapter-sqlite

npm i https://pkg.pr.new/@nextlyhq/adapter-sqlite@1dee827

@nextlyhq/admin

npm i https://pkg.pr.new/@nextlyhq/admin@1dee827

@nextlyhq/admin-css

npm i https://pkg.pr.new/@nextlyhq/admin-css@1dee827

@nextlyhq/blocks-engine

npm i https://pkg.pr.new/@nextlyhq/blocks-engine@1dee827

@nextlyhq/blocks-react

npm i https://pkg.pr.new/@nextlyhq/blocks-react@1dee827

@nextlyhq/builder

npm i https://pkg.pr.new/@nextlyhq/builder@1dee827

create-nextly-app

npm i https://pkg.pr.new/create-nextly-app@1dee827

nextly

npm i https://pkg.pr.new/nextly@1dee827

@nextlyhq/plugin-form-builder

npm i https://pkg.pr.new/@nextlyhq/plugin-form-builder@1dee827

@nextlyhq/plugin-page-builder

npm i https://pkg.pr.new/@nextlyhq/plugin-page-builder@1dee827

@nextlyhq/plugin-sdk

npm i https://pkg.pr.new/@nextlyhq/plugin-sdk@1dee827

@nextlyhq/plugin-seo

npm i https://pkg.pr.new/@nextlyhq/plugin-seo@1dee827

@nextlyhq/storage-s3

npm i https://pkg.pr.new/@nextlyhq/storage-s3@1dee827

@nextlyhq/storage-uploadthing

npm i https://pkg.pr.new/@nextlyhq/storage-uploadthing@1dee827

@nextlyhq/storage-vercel-blob

npm i https://pkg.pr.new/@nextlyhq/storage-vercel-blob@1dee827

@nextlyhq/ui

npm i https://pkg.pr.new/@nextlyhq/ui@1dee827

commit: 1dee827

@mobeenabdullah

Copy link
Copy Markdown
Collaborator Author

@codex please review this PR

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: 1dee827573

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mobeenabdullah
mobeenabdullah merged commit 8a82d3e into main Aug 11, 2026
13 of 14 checks passed
@mobeenabdullah

Copy link
Copy Markdown
Collaborator Author

Holding this as a draft — the premise in the body is falsified.

This PR argued the red was caused by the shared turbo cache key. That is wrong, and the evidence is this PR's own CI run:

Cache not found for input keys: turbo-v2-Linux-ci-166f5874…, turbo-v2-Linux-ci-

No .turbo was restored at all, and the job still failed with the identical error (Cannot find module .../adapter-drizzle/dist/version-check.mjs). A GitHub Actions cache that was never restored cannot be the cause.

Two further theories died the same way:

  • Cancelled runs poisoning the cache — refuted by timestamps: fix(blocks-react): stop publishing the rules of a block that draws nothing #661's entry was written at 11:12:36 while preview ended 11:12:39 and CI ran to 11:32:02, so a concurrent workflow wrote it, not a cancelled one.
  • tsup clean: true racing a concurrent test — refuted: the adapter-drizzle:build during the Test step is cache hit, replaying logs 5cd8dc6af6888427, so tsup never ran and nothing was deleted.

What is established: version-check.mjs is produced (logged at 12:09:45 and again at 12:10:00) and is absent when plugin-sdk#test runs at 12:14:54; a cold local build produces it correctly; main passes the same job; and the nextly → @nextlyhq/adapter-drizzle dependency edge that should order the builds does exist.

The cache-budget finding stands on its own and is worth acting on regardless — turbo-Linux-* holds 7.09GB of the repo's 9.69GB/10GB, continuously evicting the pnpm-store and Playwright caches — but that is a performance and hygiene problem, not this red.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant