diff --git a/examples/README.md b/examples/README.md index 9c6edec2e6..43f163a3f4 100644 --- a/examples/README.md +++ b/examples/README.md @@ -9,6 +9,7 @@ Use the [SQL documentation](https://alchemy.run/sql) to choose a client or | Runtime | Database | Client | Runnable project | | --- | --- | --- | --- | +| AWS Lambda | Aurora PostgreSQL | Drizzle | [aws-aurora-drizzle](./aws-aurora-drizzle) | | AWS Lambda | Aurora PostgreSQL | `pg` | [aws-rds](./aws-rds) | | Cloudflare Workers | Neon Postgres through Hyperdrive | Drizzle | [cloudflare-neon-drizzle](./cloudflare-neon-drizzle) | | Cloudflare Workers | PlanetScale Postgres through Hyperdrive | Drizzle | [cloudflare-planetscale-postgres-drizzle](./cloudflare-planetscale-postgres-drizzle) | @@ -20,10 +21,11 @@ Use the [SQL documentation](https://alchemy.run/sql) to choose a client or ### Looking for Drizzle + Aurora? -The current `aws-rds` application uses **plain `pg`**, not Drizzle. -An [Aurora PostgreSQL + Drizzle IAM test fixture](../packages/alchemy/test/AWS/RDSData/drizzle-iam-handler.ts) -exists, but depends on the surrounding test infrastructure and database-user -bootstrap. It is not a standalone example. +Start with [aws-aurora-drizzle](./aws-aurora-drizzle): a standalone Lambda +application with private Aurora PostgreSQL, IAM authentication, verified TLS, +and deployment-time schema setup. Its [integration guide](https://alchemy.run/aws/data/drizzle-aurora) +explains the connection and migration placement. The separate `aws-rds` +application uses **plain `pg`**, not Drizzle. The [AWS full-stack Drizzle guide](https://alchemy.run/aws/frontend/full-stack-tanstack-rpc-drizzle) uses **Aurora DSQL**, a different database and deployment from Aurora diff --git a/examples/aws-aurora-drizzle/README.md b/examples/aws-aurora-drizzle/README.md new file mode 100644 index 0000000000..1541d1e46d --- /dev/null +++ b/examples/aws-aurora-drizzle/README.md @@ -0,0 +1,71 @@ +# Drizzle + Aurora PostgreSQL + AWS Lambda + +A standalone example using `alchemy/Drizzle/Postgres` with Aurora PostgreSQL 17, IAM database authentication, verified TLS, and a Lambda HTTP handler. This is **Aurora PostgreSQL**, not Aurora DSQL or Drizzle over the RDS Data API. + +[Integration guide](https://alchemy.run/aws/data/drizzle-aurora/) · [SQL on AWS](https://alchemy.run/sql/providers/aws/) + +## Prerequisites + +- Install the repository dependencies with `pnpm install` from the repository root. +- Configure an [Alchemy AWS profile](https://alchemy.run/aws/setup/) with deployment permissions for VPC, RDS, Secrets Manager, IAM, Lambda, CloudWatch Logs, and the RDS Data API. +- Use **us-west-2**: the two private subnets are in `us-west-2a` and `us-west-2b`. Changing region also requires changing those availability zones and checking Aurora version/Data API availability. +- Allow several minutes for Aurora creation and deletion. The cluster incurs charges until destroyed (minimum 0.5 Aurora capacity units). This example is not configured for auto-pause or production retention. + +## Deploy + +From this directory: + +```sh +pnpm deploy --profile testing --stage aurora-example +``` + +`src/database.ts` creates an isolated VPC, two subnets, security groups, a generated admin secret, and an Aurora writer. Only the Lambda security group may connect to TCP 5432. There is no public database endpoint, NAT gateway, or internet gateway. + +`src/bootstrap.ts` runs a deployment Action using the Data API and the deployment identity. Its atomic, replay-safe initial migration creates the table and `app_iam` login, grants `rds_iam`, and grants only table CRUD and schema usage. The Action waits for the writer and its output orders Lambda deployment after setup. It runs again when its input changes or deployment is forced; it is not a general Drizzle migration runner. For schema evolution, add reviewed, versioned migrations with a history table rather than editing an already-applied initial migration. + +The deployment machine needs access to AWS HTTPS APIs, **not** a route into the VPC. Runtime queries use a private PostgreSQL socket, not the Data API. The application role has `rds-db:connect` for `app_iam`; it does not receive the admin secret or Data API grants. + +## Call the HTTP API + +The output `url` uses AWS IAM authentication. Unsigned HTTP requests are rejected. Use a SigV4-capable HTTP client authorized for both `lambda:InvokeFunctionUrl` and `lambda:InvokeFunction` on this function. Alternatively, invoke the same HTTP handler using the AWS CLI and the output `functionName`: + +```sh +aws lambda invoke --region us-west-2 \ + --function-name '' \ + --cli-binary-format raw-in-base64-out \ + --payload file://health-event.json response.json +``` + +The AWS CLI uses its own configured credentials, not Alchemy's `--profile` setting. `response.json` contains an HTTP response whose JSON body reports `app_iam`, TLS enabled, INSERT allowed, and schema CREATE denied. + +| Method | Path | Body | +| --- | --- | --- | +| GET | `/health` | — | +| GET | `/todos` | — | +| POST | `/todos` | `{"id":"aaaaaaaa-0000-4000-8000-000000000001","title":"Ship it"}` | +| PATCH | `/todos/` | `{"done":true}` | +| DELETE | `/todos/` | — | + +For direct invocation, change `rawPath` and `requestContext.http.path` in the event, set `requestContext.http.method`, and encode the request JSON as the event's `body` string. There is no `/setup` or `/migrate` route. + +## TLS and connection lifetime + +The function sets `NODE_EXTRA_CA_CERTS=/var/runtime/ca-cert.pem`, loading the Amazon CAs supplied by the managed Node.js Lambda runtime. It changes the connection URL's current `sslmode=no-verify` default to `verify-full`. Both hostname and certificate-chain verification remain enabled. This certificate path is specific to the managed Lambda runtime, not a portable container path. + +The connection Effect generates a fresh IAM token when Drizzle first opens a pool in each invocation. Queries in that invocation share the pool; the invocation scope closes it afterward. No password or token is logged or returned by the API. + +## Live integration test + +```sh +AWS_TEST_SLOW=1 ALCHEMY_PROFILE=testing bun test test/integ.test.ts +``` + +This opt-in test destroys any previous test-stage deployment, deploys the whole example including schema setup, redeploys without schema changes, rejects unsigned requests, calls the IAM-authenticated Function URL with SigV4-signed HTTP, and checks CRUD/validation/repeated invocations and SQL privileges, then destroys the stack. Without `AWS_TEST_SLOW=1`, the suite skips without provisioning. Provisioning and cleanup have finite 25-minute budgets; query assertions have a two-minute budget. + +## Destroy + +```sh +pnpm destroy --profile testing --stage aurora-example +``` + +This deletes the example database and its data. Do not use the example's disposable retention settings for production. Wait for cleanup to finish; if interrupted, rerun the same destroy command with the same profile and stage. diff --git a/examples/aws-aurora-drizzle/alchemy.run.ts b/examples/aws-aurora-drizzle/alchemy.run.ts new file mode 100644 index 0000000000..923336ae0d --- /dev/null +++ b/examples/aws-aurora-drizzle/alchemy.run.ts @@ -0,0 +1,21 @@ +import * as Alchemy from "alchemy"; +import * as AWS from "alchemy/AWS"; +import * as Effect from "effect/Effect"; +import Api from "./src/Api.ts"; +import { database } from "./src/database.ts"; + +export default Alchemy.Stack( + "aws-aurora-drizzle", + { providers: AWS.providers(), state: Alchemy.localState() }, + Effect.gen(function* () { + const api = yield* Api; + const db = yield* database; + return { + url: api.functionUrl, + functionName: api.functionName, + clusterId: db.cluster.dbClusterIdentifier, + writerId: db.writer.dbInstanceIdentifier, + vpcId: db.vpc.vpcId, + }; + }), +); diff --git a/examples/aws-aurora-drizzle/health-event.json b/examples/aws-aurora-drizzle/health-event.json new file mode 100644 index 0000000000..877804b980 --- /dev/null +++ b/examples/aws-aurora-drizzle/health-event.json @@ -0,0 +1,17 @@ +{ + "version": "2.0", + "routeKey": "$default", + "rawPath": "/health", + "rawQueryString": "", + "headers": { "host": "example.invalid" }, + "requestContext": { + "http": { + "method": "GET", + "path": "/health", + "protocol": "HTTP/1.1", + "sourceIp": "127.0.0.1", + "userAgent": "aws-cli" + } + }, + "isBase64Encoded": false +} diff --git a/examples/aws-aurora-drizzle/package.json b/examples/aws-aurora-drizzle/package.json new file mode 100644 index 0000000000..129029b349 --- /dev/null +++ b/examples/aws-aurora-drizzle/package.json @@ -0,0 +1,22 @@ +{ + "name": "aws-aurora-drizzle-example", + "version": "0.0.0", + "private": true, + "license": "Apache-2.0", + "scripts": { + "build": "tsc -b", + "deploy": "alchemy deploy", + "destroy": "alchemy destroy", + "test": "bun test" + }, + "dependencies": { + "@distilled.cloud/aws": "workspace:*", + "@effect/platform-node": "catalog:", + "@effect/sql-pg": "catalog:", + "@types/pg": "catalog:", + "alchemy": "workspace:*", + "drizzle-orm": "catalog:", + "effect": "catalog:", + "pg": "catalog:" + } +} diff --git a/examples/aws-aurora-drizzle/src/Api.ts b/examples/aws-aurora-drizzle/src/Api.ts new file mode 100644 index 0000000000..1587462d43 --- /dev/null +++ b/examples/aws-aurora-drizzle/src/Api.ts @@ -0,0 +1,115 @@ +import * as AWS from "alchemy/AWS"; +import * as Drizzle from "alchemy/Drizzle/Postgres"; +import * as Duration from "effect/Duration"; +import { eq, sql } from "drizzle-orm"; +import * as Effect from "effect/Effect"; +import * as Redacted from "effect/Redacted"; +import * as Schema from "effect/Schema"; +import { HttpServerRequest } from "effect/unstable/http/HttpServerRequest"; +import * as HttpServerResponse from "effect/unstable/http/HttpServerResponse"; +import { bootstrap } from "./bootstrap.ts"; +import { database } from "./database.ts"; +import { todos } from "./schema.ts"; + +export default class Api extends AWS.Lambda.Function()( + "Api", + Effect.gen(function* () { + const schemaVersion = yield* bootstrap; + return { + main: import.meta.url, + runtime: "nodejs22.x" as const, + functionUrl: { authType: "AWS_IAM" as const }, + timeout: Duration.seconds(30), + build: { install: ["pg"] }, + env: { + SCHEMA_VERSION: schemaVersion, + NODE_EXTRA_CA_CERTS: "/var/runtime/ca-cert.pem", + }, + }; + }), + Effect.gen(function* () { + const { cluster, subnetIds, lambdaSecurityGroup } = yield* database; + const connect = yield* AWS.RDS.Connect(cluster, { + auth: "iam", + username: "app_iam", + database: "app", + subnetIds, + securityGroupIds: [lambdaSecurityGroup.groupId], + }); + const db = yield* Drizzle.Postgres( + connect.pipe( + Effect.map((connection) => { + const url = new URL(Redacted.value(connection.url)); + // Connect currently emits no-verify; require full certificate verification. + url.searchParams.set("sslmode", "verify-full"); + return Redacted.make(url.toString()); + }), + ), + ); + return { + fetch: Effect.gen(function* () { + const request = yield* HttpServerRequest; + const url = new URL(request.originalUrl); + if (request.method === "GET" && url.pathname === "/health") { + const rows = yield* db.execute(sql`SELECT current_user AS username, + (SELECT ssl FROM pg_stat_ssl WHERE pid = pg_backend_pid()) AS tls, + has_table_privilege(current_user, 'todos', 'INSERT') AS can_insert, + has_schema_privilege(current_user, 'public', 'CREATE') AS can_create`); + return yield* HttpServerResponse.json(rows); + } + if (url.pathname === "/todos" && request.method === "GET") { + return yield* HttpServerResponse.json(yield* db.select().from(todos)); + } + if (url.pathname === "/todos" && request.method === "POST") { + const input = yield* request.json.pipe( + Effect.flatMap( + Schema.decodeUnknownEffect( + Schema.Struct({ + id: Schema.String.check(Schema.isUUID()), + title: Schema.NonEmptyString, + }), + ), + ), + ); + return yield* HttpServerResponse.json( + yield* db.insert(todos).values(input).returning(), + { status: 201 }, + ); + } + if (url.pathname.startsWith("/todos/")) { + const id = yield* Schema.decodeUnknownEffect( + Schema.String.check(Schema.isUUID()), + )(url.pathname.slice(7)); + if (request.method === "PATCH") { + const input = yield* request.json.pipe( + Effect.flatMap( + Schema.decodeUnknownEffect( + Schema.Struct({ done: Schema.Boolean }), + ), + ), + ); + return yield* HttpServerResponse.json( + yield* db + .update(todos) + .set(input) + .where(eq(todos.id, id)) + .returning(), + ); + } + if (request.method === "DELETE") { + yield* db.delete(todos).where(eq(todos.id, id)); + return HttpServerResponse.empty({ status: 204 }); + } + } + return HttpServerResponse.text("Not found", { status: 404 }); + }).pipe( + Effect.catchTag("SchemaError", () => + Effect.succeed( + HttpServerResponse.text("Invalid request", { status: 400 }), + ), + ), + Effect.orDie, + ), + }; + }).pipe(Effect.provide(AWS.RDS.ConnectHttp)), +) {} diff --git a/examples/aws-aurora-drizzle/src/bootstrap.ts b/examples/aws-aurora-drizzle/src/bootstrap.ts new file mode 100644 index 0000000000..4cc6b8d6a5 --- /dev/null +++ b/examples/aws-aurora-drizzle/src/bootstrap.ts @@ -0,0 +1,52 @@ +import * as Alchemy from "alchemy"; +import * as AWS from "alchemy/AWS"; +import * as Effect from "effect/Effect"; +import * as Schedule from "effect/Schedule"; +import { database } from "./database.ts"; + +// One atomic, replay-safe initial migration. Application roles never own DDL. +const initialMigration = `DO $$ +BEGIN + IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'app_iam') THEN + CREATE ROLE app_iam LOGIN; + END IF; + GRANT rds_iam TO app_iam; + REVOKE CREATE ON SCHEMA public FROM PUBLIC; + CREATE TABLE IF NOT EXISTS public.todos ( + id uuid PRIMARY KEY, + title text NOT NULL, + done boolean NOT NULL DEFAULT false + ); + GRANT CONNECT ON DATABASE app TO app_iam; + GRANT USAGE ON SCHEMA public TO app_iam; + GRANT SELECT, INSERT, UPDATE, DELETE ON public.todos TO app_iam; +END $$`; + +export const bootstrap = Effect.gen(function* () { + const db = yield* database; + const Bootstrap = Alchemy.Action( + "BootstrapDatabase", + Effect.gen(function* () { + const execute = yield* AWS.RDSData.ExecuteStatement(db.cluster, { + secret: db.secret, + database: "app", + }); + return Effect.fn(function* (input: { writer: string; sql: string }) { + yield* execute({ sql: input.sql }).pipe( + Effect.retry({ + while: (error) => + error._tag === "DatabaseUnavailableException" || + error._tag === "DatabaseResumingException", + schedule: Schedule.spaced("5 seconds"), + times: 8, + }), + ); + return "1"; + }); + }).pipe(Effect.provide(AWS.RDSData.ExecuteStatementHttp)), + ); + return yield* Bootstrap({ + writer: db.writer.dbInstanceArn, + sql: initialMigration, + }); +}); diff --git a/examples/aws-aurora-drizzle/src/database.ts b/examples/aws-aurora-drizzle/src/database.ts new file mode 100644 index 0000000000..b8b03edb2d --- /dev/null +++ b/examples/aws-aurora-drizzle/src/database.ts @@ -0,0 +1,54 @@ +import * as AWS from "alchemy/AWS"; +import * as Effect from "effect/Effect"; + +export const database = Effect.gen(function* () { + const vpc = yield* AWS.EC2.Vpc("Vpc", { cidrBlock: "10.62.0.0/16" }); + const subnetA = yield* AWS.EC2.Subnet("SubnetA", { + vpcId: vpc.vpcId, + cidrBlock: "10.62.0.0/24", + availabilityZone: "us-west-2a", + }); + const subnetB = yield* AWS.EC2.Subnet("SubnetB", { + vpcId: vpc.vpcId, + cidrBlock: "10.62.1.0/24", + availabilityZone: "us-west-2b", + }); + const lambdaSecurityGroup = yield* AWS.EC2.SecurityGroup( + "LambdaSecurityGroup", + { + vpcId: vpc.vpcId, + description: "Aurora example application; no inbound connections", + }, + ); + const dbSecurityGroup = yield* AWS.EC2.SecurityGroup( + "DatabaseSecurityGroup", + { + vpcId: vpc.vpcId, + description: "Postgres from the application security group only", + ingress: [ + { + ipProtocol: "tcp", + fromPort: 5432, + toPort: 5432, + referencedGroupId: lambdaSecurityGroup.groupId, + }, + ], + }, + ); + const subnetIds = [subnetA.subnetId, subnetB.subnetId]; + const aurora = yield* AWS.RDS.Aurora("Database", { + databaseName: "app", + engine: "aurora-postgresql", + engineVersion: "17.5", + subnetIds, + securityGroupIds: [dbSecurityGroup.groupId], + secret: { username: "dbadmin" }, + dataApi: true, + cluster: { + enableIAMDatabaseAuthentication: true, + serverlessV2ScalingConfiguration: { MinCapacity: 0.5, MaxCapacity: 1 }, + }, + instance: { dbInstanceClass: "db.serverless", publiclyAccessible: false }, + }); + return { ...aurora, vpc, subnetIds, lambdaSecurityGroup }; +}); diff --git a/examples/aws-aurora-drizzle/src/schema.ts b/examples/aws-aurora-drizzle/src/schema.ts new file mode 100644 index 0000000000..59677270d2 --- /dev/null +++ b/examples/aws-aurora-drizzle/src/schema.ts @@ -0,0 +1,7 @@ +import { boolean, pgTable, text, uuid } from "drizzle-orm/pg-core"; + +export const todos = pgTable("todos", { + id: uuid("id").primaryKey(), + title: text("title").notNull(), + done: boolean("done").notNull().default(false), +}); diff --git a/examples/aws-aurora-drizzle/test/integ.test.ts b/examples/aws-aurora-drizzle/test/integ.test.ts new file mode 100644 index 0000000000..69669a1324 --- /dev/null +++ b/examples/aws-aurora-drizzle/test/integ.test.ts @@ -0,0 +1,119 @@ +import { Credentials } from "@distilled.cloud/aws/Credentials"; +import { sign } from "@distilled.cloud/aws/SigV4"; +import * as Alchemy from "alchemy"; +import * as AWS from "alchemy/AWS"; +import * as Test from "alchemy/Test/Bun"; +import { describe, expect } from "bun:test"; +import * as Effect from "effect/Effect"; +import * as Redacted from "effect/Redacted"; +import * as HttpClient from "effect/unstable/http/HttpClient"; +import * as HttpClientRequest from "effect/unstable/http/HttpClientRequest"; +import Stack from "../alchemy.run.ts"; + +// Aurora provisioning/deletion is opt-in, as in the RDSData integration suite. +describe.skipIf(!process.env.AWS_TEST_SLOW)( + "Drizzle + Aurora PostgreSQL", + () => { + const { test, beforeAll, afterAll, deploy, destroy } = Test.make({ + providers: AWS.providers(), + state: Alchemy.localState(), + profile: process.env.ALCHEMY_PROFILE, + }); + const stack = beforeAll( + Effect.gen(function* () { + yield* destroy(Stack); + const outputs = yield* deploy(Stack); + yield* deploy(Stack); + return outputs; + }), + { timeout: 1_500_000 }, + ); + afterAll(destroy(Stack), { timeout: 1_500_000 }); + + test.provider( + "CRUD, TLS, least privilege, and repeated invocations", + () => + Effect.gen(function* () { + const { url } = yield* stack; + if (!url) + return yield* Effect.fail(new Error("Missing Function URL")); + const client = yield* HttpClient.HttpClient; + const getCredentials = yield* Credentials; + expect((yield* client.get(`${url}health`)).status).toBe(403); + const request = Effect.fn(function* ( + method: "GET" | "POST" | "PATCH" | "DELETE", + path: string, + body?: object, + ) { + const credentials = yield* getCredentials; + const payload = + body === undefined ? undefined : JSON.stringify(body); + const signed = yield* sign({ + method, + url: new URL(path, url).toString(), + body: payload, + service: "lambda", + region: credentials.region, + accessKeyId: Redacted.value(credentials.accessKeyId), + secretAccessKey: credentials.secretAccessKey, + sessionToken: credentials.sessionToken, + }); + const base = HttpClientRequest.make(method)(signed.url).pipe( + HttpClientRequest.setHeaders(signed.headers), + ); + const response = yield* client.execute( + payload === undefined + ? base + : base.pipe( + HttpClientRequest.bodyText(payload, "application/json"), + ), + ); + return { statusCode: response.status, body: yield* response.text }; + }); + const health = yield* request("GET", "/health"); + expect(health.statusCode).toBe(200); + expect(JSON.parse(health.body).rows).toEqual([ + { + username: "app_iam", + tls: true, + can_insert: true, + can_create: false, + }, + ]); + const id = "aaaaaaaa-0000-4000-8000-000000000001"; + yield* request("DELETE", `/todos/${id}`); + expect((yield* request("GET", "/todos")).body).toBe("[]"); + const created = yield* request("POST", "/todos", { + id, + title: "Ship Aurora guide", + }); + expect(created.statusCode).toBe(201); + expect(JSON.parse(created.body)).toEqual([ + { id, title: "Ship Aurora guide", done: false }, + ]); + const updated = yield* request("PATCH", `/todos/${id}`, { + done: true, + }); + expect(updated.statusCode).toBe(200); + expect(JSON.parse(updated.body)[0].done).toBe(true); + for (let i = 0; i < 3; i++) { + const listed = yield* request("GET", "/todos"); + expect(listed.statusCode).toBe(200); + expect(JSON.parse(listed.body)).toEqual([ + { id, title: "Ship Aurora guide", done: true }, + ]); + } + expect( + (yield* request("POST", "/todos", { id: "invalid", title: "" })) + .statusCode, + ).toBe(400); + expect((yield* request("GET", "/setup")).statusCode).toBe(404); + expect((yield* request("DELETE", `/todos/${id}`)).statusCode).toBe( + 204, + ); + expect((yield* request("GET", "/todos")).body).toBe("[]"); + }), + { timeout: 120_000 }, + ); + }, +); diff --git a/examples/aws-aurora-drizzle/tsconfig.json b/examples/aws-aurora-drizzle/tsconfig.json new file mode 100644 index 0000000000..9418efbd54 --- /dev/null +++ b/examples/aws-aurora-drizzle/tsconfig.json @@ -0,0 +1,13 @@ +{ + "extends": "../../tsconfig.base.json", + "include": ["alchemy.run.ts", "src/**/*.ts", "test/**/*.ts"], + "compilerOptions": { + "noEmit": true, + "rootDir": ".", + "module": "Preserve", + "moduleResolution": "Bundler", + "customConditions": ["bun"], + "target": "ESNext" + }, + "references": [{ "path": "../../packages/alchemy/tsconfig.json" }] +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index e678d842e5..e55286bfc7 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -642,6 +642,33 @@ importers: specifier: 4.0.0-rc.115 version: 4.0.0-rc.115 + examples/aws-aurora-drizzle: + dependencies: + '@distilled.cloud/aws': + specifier: workspace:* + version: link:../../submodules/distilled/packages/aws + '@effect/platform-node': + specifier: 'catalog:' + version: 4.0.0-rc.115(effect@4.0.0-rc.115)(redis@6.2.1) + '@effect/sql-pg': + specifier: 'catalog:' + version: 4.0.0-rc.115(effect@4.0.0-rc.115) + '@types/pg': + specifier: 'catalog:' + version: 8.20.0 + alchemy: + specifier: workspace:* + version: link:../../packages/alchemy + drizzle-orm: + specifier: 1.0.0-rc.5-ab785fc + version: 1.0.0-rc.5-ab785fc(@cloudflare/workers-types@5.20260918.1)(@effect/sql-d1@4.0.0-rc.115(effect@4.0.0-rc.115))(@effect/sql-mysql2@4.0.0-rc.115(@types/node@26.2.0)(effect@4.0.0-rc.115))(@effect/sql-pg@4.0.0-rc.115(effect@4.0.0-rc.115))(@effect/sql-sqlite-do@4.0.0-rc.115(effect@4.0.0-rc.115))(@electric-sql/pglite@0.3.15)(@libsql/client@0.17.4)(@neondatabase/serverless@1.1.0)(@types/pg@8.20.0)(arktype@2.2.3)(bun-types@1.3.14)(effect@4.0.0-rc.115)(mysql2@3.24.2(@types/node@26.2.0))(pg@8.23.0)(valibot@1.2.0(typescript@7.0.2))(zod@4.4.3) + effect: + specifier: 4.0.0-rc.115 + version: 4.0.0-rc.115 + pg: + specifier: 'catalog:' + version: 8.23.0 + examples/aws-bedrock-ai: dependencies: '@distilled.cloud/aws': diff --git a/tsconfig.json b/tsconfig.json index 2e78533d15..36cf911e4d 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -118,6 +118,9 @@ { "path": "./examples/aws-rds/tsconfig.json" }, + { + "path": "./examples/aws-aurora-drizzle/tsconfig.json" + }, { "path": "./examples/aws-static-site/tsconfig.json" }, diff --git a/website/astro.config.ts b/website/astro.config.ts index 1331f31e74..08ad0e8c94 100644 --- a/website/astro.config.ts +++ b/website/astro.config.ts @@ -752,6 +752,10 @@ export default defineConfig({ { label: "DynamoDB", link: "/aws/data/dynamodb" }, { label: "S3", link: "/aws/data/s3" }, { label: "RDS & Aurora", link: "/aws/data/rds" }, + { + label: "Drizzle + Aurora PostgreSQL", + link: "/aws/data/drizzle-aurora", + }, ], }, { diff --git a/website/public/llms.txt b/website/public/llms.txt index 783c55c980..c1eb3a7d59 100644 --- a/website/public/llms.txt +++ b/website/public/llms.txt @@ -314,6 +314,7 @@ A linear five-part walkthrough from zero to a tested, CI-deployed AWS project. E ## AWS — Data +- [Drizzle + Aurora PostgreSQL](https://alchemy.run/aws/data/drizzle-aurora) — Deploy Drizzle with Aurora PostgreSQL and AWS Lambda using IAM authentication, verified TLS, private networking, and controlled schema setup. - [DynamoDB](https://alchemy.run/aws/data/dynamodb) — Add a DynamoDB Table, bind GetItem and PutItem to your Lambda, and serve a typed key/value HTTP API backed by DynamoDB. - [RDS & Aurora](https://alchemy.run/aws/data/rds) — Stand up an Aurora cluster in one call with the Aurora helper, connect from Lambda over the Connect binding with pg, or skip connections entirely with the Data API. - [S3](https://alchemy.run/aws/data/s3) — Add an S3 Bucket to your Stack, bind PutObject and GetObject as runtime capabilities, and let Alchemy mint the IAM policy for you. diff --git a/website/src/content/docs/aws/compute/lambda.mdx b/website/src/content/docs/aws/compute/lambda.mdx index 0086d1e341..bed9391a05 100644 --- a/website/src/content/docs/aws/compute/lambda.mdx +++ b/website/src/content/docs/aws/compute/lambda.mdx @@ -321,6 +321,9 @@ for the model across all runtimes. ## Where next +- [Drizzle + Aurora PostgreSQL](/aws/data/drizzle-aurora) — private database access, + IAM authentication, verified TLS, and deployment-time schema setup. + - [S3](/aws/data/s3) — add a bucket and bind read/write operations into this function; IAM policies generated from the call sites, no policy JSON to hand-write. diff --git a/website/src/content/docs/aws/data/drizzle-aurora.mdx b/website/src/content/docs/aws/data/drizzle-aurora.mdx new file mode 100644 index 0000000000..004a342816 --- /dev/null +++ b/website/src/content/docs/aws/data/drizzle-aurora.mdx @@ -0,0 +1,174 @@ +--- +title: Drizzle + Aurora PostgreSQL +description: Deploy Drizzle with Aurora PostgreSQL and AWS Lambda using IAM authentication, verified TLS, private networking, and controlled schema setup. +--- + +Use **Drizzle + Aurora PostgreSQL from AWS Lambda** when you want Drizzle's PostgreSQL query builder over a private database connection. The [standalone example](https://github.com/alchemy-run/alchemy/tree/main/examples/aws-aurora-drizzle) includes the network, database, schema setup, HTTP handler, and integration test. + +This guide covers Aurora **PostgreSQL**, not Aurora MySQL or [Aurora DSQL](/sql/providers/aws/#aurora-dsql). The RDS Data API is used only for deployment-time setup; runtime Drizzle queries use the PostgreSQL wire protocol. + +## Prerequisites + +Complete [AWS setup](/aws/setup). The example targets `us-west-2`, Aurora PostgreSQL 17.5, and the managed Node.js 22 Lambda runtime. Its two subnets use `us-west-2a` and `us-west-2b`; change both availability zones if you change region, and check engine/Data API availability there. + +:::caution[Cost and retention] +Aurora creation and deletion take several minutes. The example keeps at least 0.5 Aurora capacity units running and incurs charges until destroyed. Its database is disposable; configure backups, deletion protection, and retention before adapting it for production. +::: + +## Provision a private database + +The example's [`src/database.ts`](https://github.com/alchemy-run/alchemy/blob/main/examples/aws-aurora-drizzle/src/database.ts) creates an isolated VPC, two subnets, and separate application/database security groups. The database security group accepts port 5432 only from the application security group. + +```ts +const aurora = yield* AWS.RDS.Aurora("Database", { + databaseName: "app", + engine: "aurora-postgresql", + engineVersion: "17.5", + subnetIds, + securityGroupIds: [dbSecurityGroup.groupId], + secret: { username: "dbadmin" }, + dataApi: true, + cluster: { + enableIAMDatabaseAuthentication: true, + serverlessV2ScalingConfiguration: { MinCapacity: 0.5, MaxCapacity: 1 }, + }, + instance: { dbInstanceClass: "db.serverless", publiclyAccessible: false }, +}); +``` + +The helper creates the admin secret, subnet group, cluster, and writer. See [RDS and Aurora](/aws/data/rds) for provisioning options. The application needs no NAT gateway: it signs IAM database tokens locally using its Lambda credentials and connects inside the VPC. + +## Initialize the schema during deployment + +[`src/bootstrap.ts`](https://github.com/alchemy-run/alchemy/blob/main/examples/aws-aurora-drizzle/src/bootstrap.ts) declares an `Alchemy.Action` that waits for the writer, then submits one atomic PostgreSQL `DO` block through `AWS.RDSData.ExecuteStatement`. + +That initial migration creates `todos` and the `app_iam` login, grants the login `rds_iam`, and grants only CONNECT, schema USAGE, and table SELECT/INSERT/UPDATE/DELETE. The database administrator owns the table. The application cannot create tables or read the admin secret. + +The Action uses the deployment identity over HTTPS, so the deployment machine does **not** need private VPC connectivity. The Lambda's schema-version input depends on the Action output, ordering application deployment after setup. The application has no setup or migration endpoint. + +This is a replay-safe **initial migration**, not automatic Drizzle migration-directory support. Actions rerun when inputs change or deployment is forced. For subsequent schema changes, use reviewed versioned migrations with recorded history; do not replace an already-applied migration with different SQL. A separate socket-based migration runner would need VPC access and a dedicated DDL-capable identity. + +## Bind the application identity + +Inside the Lambda's initialization Effect, bind the cluster as the restricted database login: + +```ts +const connect = yield* AWS.RDS.Connect(cluster, { + auth: "iam", + username: "app_iam", + database: "app", + subnetIds, + securityGroupIds: [lambdaSecurityGroup.groupId], +}); +``` + +`RDS.ConnectHttp` attaches the function to these subnets and grants `rds-db:connect` for this database user. Provide that implementation on the Lambda Effect: + +```ts +Effect.gen(function* () { + // Bind the connection and return the HTTP handler here. +}).pipe(Effect.provide(AWS.RDS.ConnectHttp)); +``` + +IAM permission alone does not create a PostgreSQL login or grant table access. That is why the separate bootstrap step is necessary. + +## Verify the database certificate + +Set the following Lambda environment variable before the Node.js process starts: + +```ts +{ + env: { + NODE_EXTRA_CA_CERTS: "/var/runtime/ca-cert.pem", + }, +} +``` + +The [managed Lambda runtime supplies Amazon CA certificates](https://docs.aws.amazon.com/lambda/latest/dg/lambda-nodejs.html#nodejs-certificate-loading) at that path. Then override the current RDS binding's `no-verify` URL mode when constructing Drizzle: + +```ts +import * as Drizzle from "alchemy/Drizzle/Postgres"; +import * as Effect from "effect/Effect"; +import * as Redacted from "effect/Redacted"; + +const db = yield* Drizzle.Postgres( + connect.pipe( + Effect.map((connection) => { + const url = new URL(Redacted.value(connection.url)); + url.searchParams.set("sslmode", "verify-full"); + return Redacted.make(url.toString()); + }), + ), +); +``` + +This keeps certificate-chain and hostname verification enabled. The CA path belongs to the managed Lambda runtime; a container or other host needs its own trusted RDS certificate bundle. + +## Package the driver + +Install the client dependencies in your application: + +```sh +pnpm add alchemy effect drizzle-orm @effect/sql-pg pg +pnpm add -D @types/pg +``` + +The repository example includes these optional peers and packages `pg` with +`build: { install: ["pg"] }`. Queries run through `@effect/sql-pg`, whose pinned +version implements the PostgreSQL wire protocol directly using Node's socket +and TLS APIs. Use the managed Node.js runtime rather than a browser runtime; +no Cloudflare compatibility flags or Hyperdrive binding are involved. + +## Query with Drizzle + +Define your table with `drizzle-orm/pg-core`, then yield ordinary Drizzle query Effects inside the handler: + +```ts +const rows = yield* db.select().from(todos); +``` + +The connection Effect signs a fresh IAM token when the first query opens a pool in the invocation. Queries share that pool within the invocation; the invocation scope closes it afterward. See [Drizzle Postgres](/sql/drizzle/postgres) for schemas, queries, and transactions, and [connection lifecycle](/sql/effect-sql/lifecycle) for execution scopes. + +## Deploy and invoke + +From `examples/aws-aurora-drizzle` in the repository: + +```sh +pnpm deploy --profile testing --stage aurora-example +``` + +The output includes the function name and an **IAM-authenticated** Function URL. Use a SigV4 client authorized for `lambda:InvokeFunctionUrl` and `lambda:InvokeFunction`, or invoke the HTTP handler through the AWS CLI: + +```sh +aws lambda invoke --region us-west-2 \ + --function-name '' \ + --cli-binary-format raw-in-base64-out \ + --payload file://health-event.json response.json +``` + +The AWS CLI uses its own credential configuration. The supplied event requests `/health`, which reports the database username, TLS status, and SQL privileges without returning credentials. The example README lists the create/read/update/delete routes and how to change the event. Unsigned Function URL requests are rejected. + +## Run the integration test + +```sh +AWS_TEST_SLOW=1 ALCHEMY_PROFILE=testing bun test test/integ.test.ts +``` + +The opt-in suite deploys and redeploys the example, rejects unsigned HTTP requests, +checks the IAM-authenticated Function URL with SigV4-signed HTTP, exercises CRUD +and repeated calls, and destroys the test-stage stack. Aurora setup and teardown have finite 25-minute budgets; query assertions have a two-minute budget. With the flag absent, no test infrastructure is created. + +## Clean up + +```sh +pnpm destroy --profile testing --stage aurora-example +``` + +Wait for deletion to finish. This removes the example database and its data. Use the same profile and stage if cleanup must be resumed. + +## Next steps + +- [SQL on AWS](/sql/providers/aws) — distinguish Aurora PostgreSQL, DSQL, MySQL, and the Data API. +- [Drizzle Postgres](/sql/drizzle/postgres) — portable client and query concepts. +- [RDS and Aurora](/aws/data/rds) — database configuration and connection choices. +- [Lambda](/aws/compute/lambda) — function packaging and runtime behavior. diff --git a/website/src/content/docs/aws/data/rds.mdx b/website/src/content/docs/aws/data/rds.mdx index f5e40fb044..7ca34cfd90 100644 --- a/website/src/content/docs/aws/data/rds.mdx +++ b/website/src/content/docs/aws/data/rds.mdx @@ -308,6 +308,9 @@ and `RollbackTransaction`, each with a matching `*Http` layer. ## Where next +- [Drizzle + Aurora PostgreSQL](/aws/data/drizzle-aurora) — a standalone Lambda + application with IAM database authentication, verified TLS, and controlled schema setup. + - [SQL on AWS](/sql/providers/aws) — Drizzle + Aurora PostgreSQL, Aurora DSQL, client choices, and the distinction between guides, examples, and test fixtures. diff --git a/website/src/content/docs/sql/databases.mdx b/website/src/content/docs/sql/databases.mdx index 9a58e08c41..959e946392 100644 --- a/website/src/content/docs/sql/databases.mdx +++ b/website/src/content/docs/sql/databases.mdx @@ -12,7 +12,7 @@ matter as much as the database's SQL dialect. | Database | Connection and integration guides | | --- | --- | -| Aurora PostgreSQL / RDS Postgres | [SQL on AWS](/sql/providers/aws#aurora-postgresql-and-drizzle), [RDS & Aurora](/aws/data/rds) | +| Aurora PostgreSQL / RDS Postgres | [Drizzle + Aurora PostgreSQL](/aws/data/drizzle-aurora), [SQL on AWS](/sql/providers/aws#aurora-postgresql-and-drizzle), [RDS & Aurora](/aws/data/rds) | | Aurora MySQL / RDS MySQL | [RDS & Aurora](/aws/data/rds), [Effect SQL MySQL](/sql/effect-sql/mysql), [Drizzle MySQL](/sql/drizzle/mysql) — verify the selected driver/auth path; the Aurora example uses Postgres | | Aurora DSQL | [SQL on AWS](/sql/providers/aws#aurora-dsql), [Drizzle full-stack integration](/aws/frontend/full-stack-tanstack-rpc-drizzle) | | Cloudflare D1 | [D1](/cloudflare/data/d1), [Drizzle + D1](/cloudflare/data/d1-drizzle) | diff --git a/website/src/content/docs/sql/index.mdx b/website/src/content/docs/sql/index.mdx index 5c8b39987f..8adef4f367 100644 --- a/website/src/content/docs/sql/index.mdx +++ b/website/src/content/docs/sql/index.mdx @@ -46,7 +46,7 @@ Looking for **Aurora**, **D1**, or another database by name? See | What you want | Where to go | | --- | --- | -| Drizzle + Aurora PostgreSQL + Lambda | [SQL on AWS](/sql/providers/aws#aurora-postgresql-and-drizzle) — current guide, example, and fixture availability | +| Drizzle + Aurora PostgreSQL + Lambda | [AWS Drizzle + Aurora guide](/aws/data/drizzle-aurora) — standalone example with IAM, TLS, and schema setup | | Drizzle + Aurora DSQL + Lambda | [AWS full-stack guide](/aws/frontend/full-stack-tanstack-rpc-drizzle) — DSQL, a separate database from Aurora PostgreSQL | | Drizzle + Neon or PlanetScale + Workers | [Cloudflare Drizzle integration](/cloudflare/data/drizzle) | | Drizzle + D1 + Workers | [Cloudflare D1 integration](/cloudflare/data/d1-drizzle) | diff --git a/website/src/content/docs/sql/providers/aws.mdx b/website/src/content/docs/sql/providers/aws.mdx index f25b144bf8..0f98e48816 100644 --- a/website/src/content/docs/sql/providers/aws.mdx +++ b/website/src/content/docs/sql/providers/aws.mdx @@ -19,13 +19,16 @@ Current material has different scopes: | Material | What it covers | | --- | --- | +| [Drizzle + Aurora PostgreSQL guide](/aws/data/drizzle-aurora) | Lambda, private networking, IAM authentication, verified TLS, and deployment-time schema setup | +| [Runnable aws-aurora-drizzle example](https://github.com/alchemy-run/alchemy/tree/main/examples/aws-aurora-drizzle) | Complete Drizzle + Aurora PostgreSQL application, bootstrap, and opt-in integration test | | [RDS & Aurora guide](/aws/data/rds) | Cluster provisioning, Lambda networking, `pg` connections, and the Data API | | [Runnable aws-rds example](https://github.com/alchemy-run/alchemy/tree/main/examples/aws-rds) | Aurora PostgreSQL + Lambda using `pg`, not Drizzle | | [Drizzle IAM test fixture](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/test/AWS/RDSData/drizzle-iam-handler.ts) | Drizzle over an IAM-authenticated Aurora socket; requires the surrounding test infrastructure and database-user bootstrap | -A standalone Drizzle + Aurora PostgreSQL example is not yet documented here. -The test fixture is implementation evidence, not a complete application to -copy. Use the [Drizzle client guide](/sql/drizzle/postgres) for the query API. +Start with the standalone example for a complete deployment. Its Data API +bootstrap uses deployment credentials; the application receives only a +restricted database login. Use the [Drizzle client guide](/sql/drizzle/postgres) +for the portable query API. ## Aurora DSQL