Skip to content

feat(gcp): add a GCS state store with a per-stage lock - #2303

Closed
duailibe wants to merge 1 commit into
alchemy-run:mainfrom
duailibe:feat/gcs-state-store
Closed

duailibe wants to merge 1 commit into
alchemy-run:mainfrom
duailibe:feat/gcs-state-store

Conversation

@duailibe

@duailibe duailibe commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Alchemy has no state store for Google Cloud Storage, so a GCP-only team has to keep its state in S3, R2 or Postgres.

This adds GCP.state(). It stores state as JSON objects in a bucket, with the same layout as the S3 store:

Alchemy.Stack(
  "MyApp",
  {
    providers: GCP.providers(),
    state: GCP.state({ bucketName: "my-company-infra-state", prefix: "alchemy" }),
  },
  Effect.gen(function* () {
    // ...
  }),
);
gs://{bucket}/{prefix}{stack}/{stage}/{fqn}.json
gs://{bucket}/{prefix}{stack}/{stage}/__stack_output__.json
gs://{bucket}/{prefix}__locks__/{stack}/{stage}

Each (stack, stage) is locked so two deploys of the same stage cannot interleave writes, like the AWS.state() lease from #2145. It uses GCS generations instead of ETags:

  • The first state operation creates the lock object with ifGenerationMatch=0. If another deploy holds it, the operation fails right away with "another deploy holds the GCS state lock", as postgresState() does.
  • The holder refreshes the lock's expiry every 20 seconds (60 second TTL) and deletes the lock when the store closes. Both requests match the generation it created, so they never touch a lock another deploy has since taken.
  • When a deploy crashes, its lock expires. The next deploy deletes that exact version (generation and metageneration match) and takes the lock.
  • If the holder cannot refresh, it stops running state operations 15 seconds before its lock would expire. This check uses a local deadline, so it does not read the lock object on every operation.
  • The lock lives under __locks__/ instead of the stage directory, so a crashed deploy's lock never keeps a deleted stage in listStages.

If the bucket is missing, the store creates it with versioning, uniform bucket-level access and public access prevention. It leaves an existing bucket's settings unchanged.

GCP.Storage.ObjectMedia gets a new create method that uploads an object only if none exists. It fails with ObjectAlreadyExists on 412. The typed 412 on objects.delete and objects.patch comes from alchemy-run/distilled#877.

CI doesn't run the GCP live tests, so I ran test/GCP/StateStore/State.test.ts against a personal GCP project. All 4 cases pass and leave no buckets behind.

Alchemy had no state store for Google Cloud Storage. `GCP.state()` stores
state as JSON objects in a bucket, laid out like the S3 store, and locks
each (stack, stage) so two deploys cannot interleave writes.

- The lock is an object at `{prefix}__locks__/{stack}/{stage}`, created
  with `ifGenerationMatch=0`. Contention fails immediately with
  "another deploy holds the GCS state lock", as with postgresState().
- The holder refreshes the lock's expiry every 20 seconds (60 second TTL)
  and deletes it on close. Both writes match the generation it created.
- A crashed deploy's lock expires; the next deploy deletes exactly the
  expired version (generation + metageneration match) and takes over.
- The holder stops 15 seconds before its lock would expire, and reports
  the lock lost when a refresh finds another generation.
- A missing bucket is created with versioning, uniform bucket-level
  access and public access prevention. Existing buckets are not changed.

ObjectMedia gains `create`, a create-if-absent upload that fails with
`ObjectAlreadyExists` on 412.
@duailibe
duailibe marked this pull request as ready for review October 16, 2026 08:30
@alchemy-version-bot

Copy link
Copy Markdown
Contributor

Install the packages built from this commit:

alchemy

pnpm install https://pkg.alchemy.run/alchemy/pr:2303:5b8dd23
@alchemy.run (6)
pnpm install https://pkg.alchemy.run/@alchemy.run/better-auth/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@alchemy.run/cloudflare-runtime/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@alchemy.run/frontend-frameworks/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@alchemy.run/node-utils/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@alchemy.run/floci/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@alchemy.run/pkg/pr:2303:5b8dd23
Transitive Dependencies (19)
pnpm install https://pkg.alchemy.run/@distilled.cloud/acme/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/aws/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/axiom/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/cloudflare/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/core/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/doppler/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/fly-io/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/gcp/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/hetzner/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/iceberg/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/infisical/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/neon/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/planetscale/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/prisma/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/railway/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/stripe/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/turso/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/zerossl/pr:2303:5b8dd23
pnpm install https://pkg.alchemy.run/@distilled.cloud/github/pr:2303:5b8dd23

Published Oct 16, 2026, 4:39 PM UTC. Expires Oct 23, 2026, 4:39 PM UTC, extended while this pull request is open.

@sam-goodwin

Copy link
Copy Markdown
Contributor

Thanks @duailibe for this. A GCS state store has since landed on main in #2379 as GCP.state(), with per-stage locking. Closing this one as superseded. If anything in this PR is still missing from #2379, please open a follow-up.

@duailibe
duailibe deleted the feat/gcs-state-store branch October 27, 2026 07:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants