diff --git a/.drive/projects/emulator-commands/deferred.md b/.drive/projects/emulator-commands/deferred.md new file mode 100644 index 000000000..f78e3206e --- /dev/null +++ b/.drive/projects/emulator-commands/deferred.md @@ -0,0 +1,6 @@ +# Deferred + +- **A start can join a crash-backoff restart that has already exited (prisma/composer#353).** When the fifth fast exit marks an instance `held`, the readiness probe only notices the exit after its current connection attempt. A start request arriving in that gap joins the dying start and gets its old failure, so nothing resumes. Seen once as an intermittent failure of "a held service resumes on start" in `compute-deployment.test.ts`; not reproduced in 10 runs under CPU load. Likely fix: a start must not join an in-flight start whose process has already exited. Belongs in #353, not this project. +- **What a previous app's databases and buckets become when another app runs from the same directory.** Compute instances are deleted (S1a). Databases and buckets hold data; decide in S1b. +- **`[dev]` notices bypass the CLI engine's output.** The emulators hook writes `[dev] compute emulator ready`, the port-change line and the app-switch line with `console.log` to stdout, so a piped `prisma dev` mixes them into the JSON frames. Fixing it needs a channel for the local target's hooks to report through the engine. +- **A deployment that arrives while its instance is being deleted can start a process the emulator no longer tracks.** Same race as #353's app delete. diff --git a/.drive/projects/emulator-commands/design-notes.md b/.drive/projects/emulator-commands/design-notes.md new file mode 100644 index 000000000..3db8fe0c5 --- /dev/null +++ b/.drive/projects/emulator-commands/design-notes.md @@ -0,0 +1,63 @@ +# Design notes — emulator commands + +Decisions taken while shaping, with the reasoning. Will's rulings are marked; the rest are orchestrator decisions he can overturn. + +## Rulings (Will, 2026-10-13) + +- The commands are `prisma dev url
` and `prisma emulator list`. A top-level `prisma url` was rejected as ambiguous (local and deployed running objects). +- `emulator` commands act on the machine's emulators, not on a project. +- The emulators know nothing about Composer. Composer maps (directory, app, address) to emulator instances itself, per directory. Instances carry informative labels only. +- `dev url` resolves a Composer node address only, never an emulator instance id. +- Databases and buckets are in scope alongside services. +- Piped output stays JSON; docs use `jq`. No output-format change to the CLI engine. +- The CLI engine changes so a path can be both a command and a group, keeping `prisma dev `. +- The emulators may move out of Composer later (own project, or prisma-cli). Not part of this work. + +## Facts the design rests on (checked against main, 90a30381) + +- The config does not name the entry or the app. ADR-0003 rejected a config that names the app; `deploy-cli.md` lists entry discovery as out of scope. So `dev url` cannot find the app from `prisma.config.ts`; it needs a record `prisma dev` leaves in the directory. +- Today the emulators are app-aware: Compute routes are `/apps/:app/services/:id`, Postgres server names come from `instanceNameFor(app, db)`, buckets are named `--`. The local identity is the root name alone (`target/src/local-target/container.ts`), so two directories running one app share instances. +- `LocalTargetAttachment.endpoints()` returns `{ address, url }` only. The emulator's `ServiceInfo` has `status: running | backoff | held | stopped`. +- The engine rejects a command path under another command (`packages/cli-engine/src/execution/command-tree.ts:157`). +- When stdout is not a terminal the engine prints JSON frames: `{"kind":"result","envelope":{"ok":true,"result":…}}` (`shared-flags.ts:191`, `settlement.ts:83`). +- `9-public` may import both `framework` and `prisma-cloud` (`architecture.config.json` cross-domain rules). `framework` imports nothing outside itself. + +## Where the code lives + +- **Emulators** (`dev-emulators`): instance-keyed routes and state; every create call accepts a `labels: Record` that is stored and returned verbatim. Each emulator gains a route that lists all its instances. +- **The mapping** is owned by the Prisma Cloud local target (`local-target` / `target/src/local-target`), because that code creates the instances and learns their ids. It lives in `/.prisma-composer/dev/` (the dev state directory, already per working directory), and records the app name plus, per node address, the emulator kind and instance id. +- **`prisma dev url`** goes through the core seam, like the `log` operation: config, then `resolveLocalTargets`, then the target's attachment. The attachment gains a method that resolves one address to `{ address, kind, url, state }`. Core stays target-agnostic; the framework domain imports nothing new. It runs from a Composer project, which `dev url` needs anyway. +- **`prisma emulator list`** needs no project and no config, so it cannot go through the seam. Its handler lives in the Prisma Cloud domain next to the emulator client, and `@prisma/composer-cli` (in `9-public`, which may import `prisma-cloud`) adds it to the command family. No dependency-cruiser exception is needed. +- The emulator client stays the only module that speaks the emulators' HTTP API, so a later move out of Composer replaces one dependency. + +## Ports (Will, 2026-10-13: agents in many worktrees start and stop environments) + +An instance keeps its port from creation until it is deleted, running or stopped. A deployment carries its port inside the service's environment (`COMPOSER_
_PORT`) and in its dependents' URLs, and Alchemy's no-op reconcile on a warm start never redeploys, so a port that could change between sessions would leave services built for the old port. Releasing ports on stop was considered and dropped for that reason (implementer finding, S1a dispatch 1). The cost: instances left behind by deleted worktrees keep their ports until the later remove command deletes them. + +Composer remembers each node's port in the directory's mapping and requests it only when it has to create the instance again, for example after the emulator lost its state. A requested port is a preference: if another instance or process holds it, the emulator gives the lowest free port from 3000 and `prisma dev` prints one line saying the port changed. Failing instead would break an agent's `prisma dev` whenever another worktree happened to take the port. On a first start, ports still go in dependency order (prisma/composer#350). + +## Labels Composer writes + +`composer.directory` (absolute path), `composer.app` (app name), `composer.address` (node address). `emulator list` shows them; nothing reads them back as identity. + +## Output shapes + +- `dev url`, human: the URL alone on stdout's data line, so `--format human` also works in `$(…)`. JSON `result`: `{ address, kind, url, state }`, where `kind` is `service | database | bucket` and `state` is the service status, or `running` for a database or bucket whose emulator is up. +- `emulator list`, JSON `result`: `{ emulators: [{ kind, running, url, instances: [{ id, kind, url, state, labels }] }] }`. Human: one block per emulator, instances listed under it with the address and directory from their labels when present. It doesn't fit the list convention's `{ name, id, status }` items, because two levels are nested. That is a deliberate departure, noted in the slice spec. + +## Error codes (namespace `DEV` for `dev url`, new `EMULATOR` for `emulator list`) + +- `DEV.NOT_STARTED_HERE`: no `prisma dev` has run in this directory (no mapping). Fix: run `prisma dev ` here. +- `DEV.ADDRESS_UNKNOWN`: no node with that address in this directory's app; lists the known addresses. +- `DEV.EMULATOR_UNREACHABLE` (from #353): the node's emulator is not running or not answering. +- `DEV.SERVICE_NOT_RUNNING`: the service exists but is stopped, backing off or held; says which, and points to `prisma dev`. +- `EMULATOR.*`: only if `list` needs one beyond `DEV.EMULATORS_DIR_INVALID` and `DEV.EMULATOR_UNREACHABLE`. Final names are fixed in the slice specs and recorded in ADR-0044. + +## Alternatives considered + +- **`prisma emulator url
`.** Same behaviour, but `url` asks a Composer question about this directory's app, which belongs under `dev`. +- **Rename the run command to `prisma dev start`.** Avoids the engine change, but breaks every `prisma dev module.ts` and reverses ADR-0050's root verb. +- **Name the app on the command line (`url my-app gateway`) or with `--app`.** Addresses are meant relative to the local app, and names collide across directories. +- **Make the local identity "app name + directory hash" and keep app-aware emulators.** Fixes collisions but keeps Composer concepts inside the emulators, which is the opposite of the model. +- **`emulator list` through the core seam.** Requires a project and config for a machine-level command. +- **Print a bare URL even when piped.** An engine output change Will deferred. diff --git a/.drive/projects/emulator-commands/plan.md b/.drive/projects/emulator-commands/plan.md new file mode 100644 index 000000000..81191b0ae --- /dev/null +++ b/.drive/projects/emulator-commands/plan.md @@ -0,0 +1,64 @@ +# Emulator commands — project plan + +**Spec:** `.drive/projects/emulator-commands/spec.md` · **Design notes:** `design-notes.md` + +## Slices + +### S1 — Emulators know nothing about Composer (prisma/composer) + +**Outcome.** The Compute, Postgres and bucket emulators key instances by their own ids, accept and return free-form labels, and each lists all its instances. The Prisma Cloud local target creates instances, labels them, and records the per-directory mapping in `.prisma-composer/dev/`. Every Composer operation on the emulators goes through the mapping: deploy, session start and stop, logs, `--fresh` teardown. `prisma dev` behaves the same from outside, except that pre-upgrade local state is discarded. Two directories running one app get separate instances. Port allocation keeps its current rule on a first start: services reserve ports in dependency order (prisma/composer#350), lowest free from 3000. An instance keeps its port from creation until it is deleted, stopped or not, so a directory's ports never change. The mapping remembers each node's port and requests it when an instance has to be re-created. When `prisma dev` ends it stops all of the directory's instances, Postgres servers included (the Postgres emulator gains a stop that keeps data). + +**Builds on:** prisma/composer#353 (stack on its branch until it merges). +**Hands to S3:** the mapping file and its reader; the emulator client's list-everything calls; ADR amending ADR-0041. + +Likely too large for one review: split at pickup into Compute (with the mapping and remembered ports), then Postgres and buckets (with the Postgres stop). + +### S2 — A command path can also be a group (prisma/prisma-cli) + +**Outcome.** The CLI engine accepts a command mounted at a path that other commands mount under. `prisma dev module.ts` runs the command, `prisma dev url gateway` runs the subcommand, a subcommand name wins over a positional argument with the same spelling, and `prisma dev --help` shows the command's usage and its subcommands. Proven with engine tests using a stand-in command; no Composer change needed. + +**Builds on:** nothing. Runs in parallel with S1. +**Hands to S4:** an engine release that allows the `dev` mount. + +### S3 — `prisma dev url`, `prisma dev stop` and `prisma emulator list` (prisma/composer) + +**Outcome.** The command family exports `dev url`, `dev stop` and `emulator list` with tests first (bun:test; emulator tests set `PRISMA_COMPOSER_EMULATORS_DIR` to a temporary directory). The core local-target attachment gains the resolve-one-address method. Error codes are recorded in ADR-0044. A new ADR amends ADR-0050 (the family is no longer exactly `deploy` and `dev`). `local-dev.md`, the running-locally guide and the core-concepts skill are updated, including that `start` and `stop` are planned. + +**Builds on:** S1. +**Hands to S4:** a Composer release containing both commands. + +### S4 — Mount the commands (prisma/prisma-cli) + +**Outcome.** The `prisma` binary pins the Composer release from S3 and mounts `dev url`, `dev stop` and `emulator list`, with an `emulator` group whose help defines "emulator", `mount-coverage.test.ts` updated, and `EXCLUSIONS` entries in `e2e-coverage.test.ts`. + +**Builds on:** S2, S3 (released). +**Hands to S5:** a `prisma` release with both commands. + +### S5 — Docs stop hard-coding ports (prisma/web) + +**Outcome.** The Composer getting-started and local-development pages use `prisma dev url` instead of `localhost:3000`/`3001`. + +**Builds on:** S4 (released). + +## Sequence + +``` +#353 ─► S1 ─► S3 ─┐ + ├─► S4 ─► S5 +S2 ───────────────┘ +``` + +S1 and S2 run in parallel. S4 and S5 wait on releases, which are currently blocked on an upstream Alchemy issue. + +Five slices is one over the usual ceiling. S4 and S5 are small handoffs into other repos, and each one waits on a release, so they can't fold into a neighbour. + +## Tracking + +No Linear project exists for this work. Slices are tracked in this plan. + +## Close-out + +- [ ] Verify every Project DoD item in `spec.md`. +- [ ] Long-lived content lives in ADRs, `docs/design/10-domains/local-dev.md`, the guide and the skill. +- [ ] Strip repo-wide references to `.drive/projects/emulator-commands/`. +- [ ] Delete `.drive/projects/emulator-commands/`. diff --git a/.drive/projects/emulator-commands/slices/s1a-compute-instances/spec.md b/.drive/projects/emulator-commands/slices/s1a-compute-instances/spec.md new file mode 100644 index 000000000..adc700b86 --- /dev/null +++ b/.drive/projects/emulator-commands/slices/s1a-compute-instances/spec.md @@ -0,0 +1,87 @@ +# S1a — The Compute emulator knows nothing about Composer + +Repo: prisma/composer. Branch `feat/emulator-instances`, stacked on prisma/composer#353 (`fix/dev-ready-after-listening`). PR base: that branch until #353 merges, then `main`. + +## At a glance + +After this slice, the Compute emulator runs service instances keyed by its own ids. It never sees an app name, a node address or a directory, except as label values it stores and returns without reading. Composer keeps, per working directory, a record of which instance backs each service address and which port it last had. + +```text +.prisma-composer/dev/ (per directory, written by Composer) + app: "my-app" + nodes: + quotes → compute instance "svc_k3f9…", port 3000 + gateway → compute instance "svc_81ad…", port 3001 + +Compute emulator (machine-wide, Composer-unaware) + GET /instances → [{ id: "svc_k3f9…", port: 3000, url, status: "running", + labels: { "composer.directory": "/path", "composer.app": "my-app", + "composer.address": "quotes" } }, …] +``` + +Two directories running the same app get separate instances. `prisma dev` otherwise behaves the same from the outside. + +## Chosen design + +**Emulator (`packages/1-prisma-cloud/0-lowering/dev-emulators`).** +- Instances are created by the emulator and get an id it generates. Create accepts `labels: Record` and an optional requested port. +- Every instance operation is by id: deploy, start, stop, delete, logs, get. There is no app grouping. Bulk start and stop take a list of ids. +- `GET` for all instances returns id, port, url, status and labels. +- Ports: an instance gets its port at creation and keeps it until deleted, running or stopped (a deployment's env and its dependents' URLs contain the port, and a warm start does not redeploy). Create takes an optional requested port: used if free, otherwise the lowest free port from 3000; the response reports the port given. No request → lowest free from 3000. +- The deployment body carries no Composer address. Emulator messages name the instance id; Composer adds the service's address when it rethrows, so what `prisma dev` prints is unchanged. +- State written in the old app-keyed format is discarded on load. Local dev state is never migrated across versions (running-locally guide). +- #353's readiness behaviour (a service is `running` only once its port accepts connections), the start and stop concurrency rules, crash supervision and the version-skew restart all carry over unchanged, keyed by instance instead of `(app, id)`. + +**Composer's mapping (Prisma Cloud local target: `0-lowering/local-target`, `1-extensions/target/src/local-target`).** +- One file in `/.prisma-composer/dev/` records the app name and, per service address, the Compute instance id and its last port. One module owns reading and writing it. S3 reads it for `prisma dev url`. +- The emulators hook keeps prisma/composer#350's order: it ensures one instance per service in `graph.nodes` order (dependencies first, ties in declaration order). It reuses the mapped instance when it still exists on the emulator. Otherwise it creates one, requesting the remembered port, and records the result. +- When a requested port was unavailable and the emulator gave a different one, `prisma dev` prints one line naming the service, the old port and the new one. +- Labels on create: `composer.directory` (absolute cwd), `composer.app`, `composer.address`. No Composer code finds an instance by its labels. +- The `App` and `Deployment` providers, attach (endpoints, logs, `startServices`, `stopServices`) and the `--fresh` teardown all reach Compute instances only through the mapping. +- `--fresh` deletes the mapped Compute instances and clears their mapping entries. + +**Docs.** +- A new ADR records the model: the emulators know nothing about Composer, Composer maps its nodes to instances per directory, and labels are informative only. It amends ADR-0041's local identity ("a stable local identity from the app name"). +- `docs/design/10-domains/local-dev.md` is updated for Compute. +- The running-locally guide and the core-concepts skill describe the user-visible change: two directories running one app no longer share services. Follow `.agents/rules/user-facing-surface-changes.mdc`. + +## Coherence rationale + +One reviewer can hold this as "Compute instances become emulator-owned, and Composer's per-directory mapping replaces the app name as the key". The Postgres and bucket emulators stay app-keyed until S1b. Both keys are valid in the meantime: the app name still identifies Postgres and buckets, and the mapping identifies Compute. + +## Scope + +In: the Compute emulator and its client; the mapping module; the emulators hook, the Compute providers, attach and teardown for Compute; tests; the ADR and docs above. + +Deliberately out: +- Postgres and buckets (S1b). +- Stopping Postgres when `dev` ends (S1b). +- `dev url`, `dev stop` and `emulator list` (S3). This slice does add the emulator's list-all route and client call, because the mapping's "does this instance still exist" check needs it. +- Deleting instances whose directory is gone. + +## Pre-investigated edge cases + +| Case | Expected | +| --- | --- | +| Mapping names an instance the emulator no longer has (emulator state lost, or another directory's `--fresh`) | Create a new instance requesting the remembered port; update the mapping | +| Remembered port taken by another directory's running instance | Lowest free port; one-line notice; mapping updated | +| Two directories, same app, started concurrently | Separate instances, no shared ports, neither redeploys over the other | +| Warm restart after Ctrl-C (the #164 warm-restart case) | Same instances, same ports, services started from their last deployment | +| Emulator restarted by a version change | Old app-keyed state discarded; the next `dev` creates instances, requesting remembered ports | + +## Slice-specific done conditions + +- [ ] Two temporary directories run the same fixture app at once. Each has its own Compute instances, and neither directory's services restart when the other deploys. This is proven by a test against a real Compute emulator in a temporary `PRISMA_COMPOSER_EMULATORS_DIR`. +- [ ] A test fails if the Compute emulator's routes, state keys, file names or generated fields carry an app name, address or directory, or if it alters label values or the deployment payload. + +## References + +- Project: `../../spec.md`, `../../design-notes.md`, `../../plan.md`. +- prisma/composer#350 (port order), prisma/composer#353 (readiness; this branch's base). +- `docs/design/10-domains/local-dev.md`, ADR-0041, ADR-0044, ADR-0051. + +## Dispatch plan + +1. **Emulator and client.** The Compute emulator and `computeClient` move to instance ids, labels, requested ports, list-all and bulk start and stop. The emulator's own tests are migrated and extended. *Hands to 2:* a Compute client keyed by instance id. +2. **Composer's mapping.** The mapping module, the emulators hook, the providers, attach and teardown move to the mapping, including the change-of-port notice and the two-directory test. *Hands to 3:* `prisma dev` working end to end on instance ids. +3. **Docs.** The ADR, local-dev.md, the guide and the skill. *Hands to:* the PR. diff --git a/.drive/projects/emulator-commands/slices/s2-command-group/spec.md b/.drive/projects/emulator-commands/slices/s2-command-group/spec.md new file mode 100644 index 000000000..631533c6e --- /dev/null +++ b/.drive/projects/emulator-commands/slices/s2-command-group/spec.md @@ -0,0 +1,59 @@ +# S2 — A command path can also be a group + +Repo: prisma/prisma-cli. Package: `packages/cli-engine` (`@prisma/cli-engine`). Branch `feat/engine-command-group` off `main`. + +## At a glance + +Today the engine refuses to build a CLI where one command is mounted at a path that is also the prefix of other commands (`packages/cli-engine/src/execution/command-tree.ts:157`, "collides with a command mounted at a prefix of it"). After this slice it accepts it: + +```ts +createCli({ + groups: { dev: { brief: "…" } }, + commands: { + dev: devCommand, // prisma dev module.ts + "dev url": devUrlCommand, // prisma dev url gateway + "dev stop": devStopCommand, + }, +}); +``` + +```bash +prisma dev module.ts # runs devCommand with entry "module.ts" +prisma dev url gateway # runs devUrlCommand +prisma dev --help # devCommand's usage, then the subcommands under dev +prisma dev url --help # devUrlCommand's help +``` + +A bare `dev` that also has subcommands is the established convention (`pnpm dev`, `next dev`). The project needs it so `prisma dev ` keeps working while `prisma dev url` and `prisma dev stop` are added. + +## Chosen design + +- A path may carry both a command and child commands. The path still needs a group declaration in `groups`, because it has children; the group's brief is what appears in its parent's listing. +- Routing: when the next argument after the path is the name of a child, the child runs. Otherwise the path's own command runs with the remaining arguments. A child name therefore wins over a positional argument spelled the same. This is documented, and it is harmless for `dev`, whose positional is a file path. +- Help: ` --help` shows the path's own command help and lists the child commands. +- Every existing construction check still applies to both the command and its children. Only the prefix collision check is relaxed. Two commands at the same exact path are still refused. +- Everything else that walks the tree handles a node that is both: shell completion, the `commands` listing, unknown-command suggestions, redirects (a redirect still cannot sit on a mounted path), mount metadata, and anything in `packages/cli` that enumerates mounted paths (`mount-coverage.test.ts`, `e2e-coverage.test.ts`'s key reader). +- If Stricli's route maps support a default command for a route, use that. Otherwise implement the routing in the engine's adapter. The implementer checks which. + +## Coherence rationale + +One engine capability, proven with stand-in commands. No product command changes. Composer's `dev url` and `dev stop` mount later, in S4. + +## Scope + +In: the engine change, its tests, the engine's docs (README and AGENTS.md where they describe the tree), the version bump or changeset that the repo's release process expects for an engine change, and docs in `docs/architecture` or `docs/product` that state the old rule. + +Out: mounting any Composer command; changing `prisma dev`; output formats. + +## Slice-specific done conditions + +- [ ] Engine tests with a stand-in `dev` command and `dev url` child cover: the parent runs with a positional; the child runs; `--help` on each; a child name winning over the same-spelled positional; and two commands at one exact path still being refused. + +## References + +- `docs/architecture/cli-engine-requirements.md` (R12: the shell defines the command tree), `docs/product/command-principles.md`, `docs/product/cli-help-standard.md`. +- Project: `../../spec.md`, `../../design-notes.md`. + +## Dispatch plan + +1. One dispatch: tests first, then the change, then docs. *Hands to:* the PR. diff --git a/.drive/projects/emulator-commands/spec.md b/.drive/projects/emulator-commands/spec.md new file mode 100644 index 000000000..689c299ad --- /dev/null +++ b/.drive/projects/emulator-commands/spec.md @@ -0,0 +1,92 @@ +# Purpose + +Let users, docs and agents ask the `prisma` CLI where a locally running Composer node is, instead of assuming which port it got: + +```bash +npx prisma dev module.ts # terminal 1 +curl "$(npx prisma dev url gateway | jq -r 'select(.kind=="result").envelope.result.url')" # terminal 2 +``` + +and see everything the machine's local emulators are running: + +```bash +npx prisma emulator list +``` + +Today the Composer getting-started page on prisma/web hard-codes `curl localhost:3001` and a POST to `localhost:3000`. Those only work because prisma/composer#350 happens to give the getting-started app those ports. + +# At a glance + +The project rests on one model, settled with Will on 2026-10-13: + +- **The emulators are standalone, machine-local services.** The Compute, Postgres and bucket emulators hold instances: a running service process, a database, a bucket. They know nothing about Composer, apps, directories or node addresses. Each instance carries informative labels that the emulator stores and returns but never interprets, such as "started by Composer in /path/to/project for node `gateway`". Users will be able to start and stop emulators freely (a later project). +- **Composer is one client of the emulators.** `prisma dev` asks them for the instances its topology needs. +- **Composer owns the mapping from its names to emulator instances, per directory.** For each node address of the app run from a directory, Composer records which emulator instance backs it, in `.prisma-composer/dev/`. Every Composer operation on the emulators goes through that mapping: deploy, start, stop, logs, `--fresh`. + +On that model, two commands: + +- **`prisma dev url
`** prints the URL of a node of the app that `prisma dev` runs from the current directory. It covers services (`http://localhost:`), databases (the local connection string) and buckets (the bucket emulator's S3 endpoint and the bucket's name on it). It reads the directory's mapping, then asks the emulator for that instance. `--json` gives the address, URL, kind and state. +- **`prisma emulator list`** prints every emulator on the machine, whether it is running, and each running emulator's instances with their kind, id, URL, state and labels. It needs no Composer project. + +- **`prisma dev stop`** stops every instance this directory's app has on the emulators: service processes and Postgres servers. Data stays. An agent runs it before deleting its worktree, or after a `prisma dev` that was killed without a chance to clean up. + +When `prisma dev` ends (Ctrl-C, SIGTERM, or a failure), it does the same stop. Today it stops only the service processes and leaves Postgres servers running. An instance keeps its port from creation until it is deleted, whether it is running or stopped, so a directory's ports never change. When an instance has to be re-created (its emulator lost its state), Composer requests the port remembered in the directory's mapping. + +The shared emulator daemons themselves keep running; other directories may be using them. + +`prisma dev ` keeps working unchanged. The CLI engine learns to let a path be both a command and a group, so `prisma dev module.ts` runs the app and `prisma dev url gateway` runs the subcommand. A bare `dev` is the established convention (`pnpm dev`, `next dev`). + +# Non-goals + +- **Removing instances and their data.** A worktree that is deleted leaves its stopped instances' records and Postgres data on disk. They hold no process, but they do keep their ports reserved. A remove command comes with the later emulator start/stop project. +- **`prisma emulator start` and `stop`.** Planned; they change what runs on the shared emulators and get their own design. +- **Printing a bare value when stdout is a pipe.** The CLI keeps its rule that a piped stdout gets JSON frames. Docs use `jq`. No output-format change to the CLI engine. +- **`url` for things Composer didn't start.** `prisma dev url` takes a Composer node address only, never an emulator instance id. +- **Moving the emulators out of Composer.** Likely later, into their own project or into prisma-cli. This project keeps the emulator client behind one module boundary so that move stays cheap, but does not make it. +- **Migrating existing local dev state.** Local dev state is never migrated across Composer versions (running-locally guide). After the upgrade, a first `prisma dev` starts from empty instances. Instances the old emulators held are discarded. + +# Place in the larger world + +- **prisma/composer** — the emulators (`packages/1-prisma-cloud/0-lowering/dev-emulators`), the local target that drives them (`0-lowering/local-target`, `1-extensions/target/src/local-target`), the `dev` and `log` operations and the command family (`packages/0-framework/3-tooling/cli`), and `@prisma/composer-cli`, which the `prisma` binary mounts. +- **prisma/prisma-cli** — the CLI engine (`packages/cli-engine`), and the host that decides where each Composer command mounts (`packages/cli/src/cli.ts`, requirement R12 "the shell defines the command tree"). +- **prisma/web** — the Composer getting-started and local-development pages. +- **Depends on prisma/composer#353** (open): a service counts as running only once its port accepts connections, and `withEmulatorRetry` retries only `DEV.EMULATOR_UNREACHABLE`. Without it, a URL `dev url` prints can refuse connections for a second or two after `ready`. +- **Releases.** Composer and CLI releases are currently blocked on an upstream Alchemy issue. The prisma-cli mount and the prisma/web change wait for a Composer release with the commands. + +# Cross-cutting requirements + +- **The emulators never learn a Composer concept.** No emulator route, file or field names an app, address or directory as anything other than an opaque label. A test or lint holds this. +- **Composer's mapping is the only source of truth for "which instance is this node".** Labels are for display. No Composer code finds an instance by reading labels. +- **Architecture boundaries hold.** The framework domain (`0-framework`) still imports nothing from `1-prisma-cloud`. The commands reach the emulators through a boundary dependency-cruiser allows, recorded in an ADR. +- **Every printed URL answers.** `dev url` reports a service as running only when the emulator does, which after #353 means its port accepts connections. +- **Errors follow ADR-0044.** Every failure a user can cause has its own code, raised where it is detected, with why and fix: no `prisma dev` has run in this directory; no node has that address (the known addresses listed); the node's emulator is not running; the service is not running. New codes and any new namespace are recorded in ADR-0044 in the same change. +- **User-facing changes ship with their docs** (`.agents/rules/user-facing-surface-changes.mdc`): `docs/guides/running-locally.md` and the core-concepts skill in Composer, and help text that meets prisma-cli's `docs/product/cli-help-standard.md`, which includes defining "emulator" for a reader who has never used Prisma. +- **`prisma dev` behaves the same from the outside** at every intermediate step, apart from the documented loss of pre-upgrade local state. + +# Transitional-shape constraints + +- The emulator refactor lands one emulator at a time if that keeps PRs reviewable. Each step leaves `prisma dev` fully working on main. +- Until the prisma-cli engine change ships, the Composer family may export the new commands without the host mounting them. The host mounts them only after the engine supports `dev` as both a command and a group. + +# Project DoD + +- [ ] On a machine with nothing running, the getting-started app's `prisma dev module.ts` followed by the documented `curl "$(npx prisma dev url gateway | jq …)"` returns a quote, run from the published `prisma` binary. +- [ ] Two directories running the same app at once get separate instances, and `prisma dev url gateway` in each returns that directory's own service. +- [ ] After Ctrl-C of `prisma dev`, or `prisma dev stop` following a `kill -9`, no process of that directory's app is running; the next `prisma dev` gets the same ports. +- [ ] `prisma emulator list` shows the services, databases and buckets of both, each labelled with the directory and address that started it. +- [ ] No emulator route, state key, file name or emulator-generated field carries an app name, address or directory. Composer data reaches an emulator only as label values and inside the deployment payload, stored verbatim and never interpreted. +- [ ] Composer ADRs record the emulator model (amending ADR-0041) and the new commands (amending ADR-0050). ADR-0044 lists the new codes. +- [ ] `docs/design/10-domains/local-dev.md`, the running-locally guide and the core-concepts skill describe both commands and the model. +- [ ] prisma-cli mounts both commands, with `EXCLUSIONS` entries in the e2e coverage test (local only, no management API). +- [ ] The prisma/web getting-started and local-development pages use `prisma dev url` instead of hard-coded ports. + +# Open questions + +None blocking. Decisions taken by the orchestrator, recorded in `design-notes.md`: the mapping file's location and shape, the label keys Composer writes, the `--json` shapes, and the error code names. + +# References + +- Brief (unvalidated, superseded by this spec where they differ): summarized in the conversation of 2026-10-13. +- `docs/design/10-domains/local-dev.md`, ADR-0041, ADR-0043, ADR-0044, ADR-0050, ADR-0051. +- prisma/composer#350 (merged), prisma/composer#353 (open). +- prisma-cli: `docs/architecture/cli-engine-requirements.md` (R12), `docs/product/cli-help-standard.md`, `docs/product/output-conventions.md`, `packages/cli/tests/e2e-coverage.test.ts`, `packages/cli/tests/mount-coverage.test.ts`. diff --git a/docs/design/10-domains/local-dev.md b/docs/design/10-domains/local-dev.md index 4a4a6b169..dbaaf940a 100644 --- a/docs/design/10-domains/local-dev.md +++ b/docs/design/10-domains/local-dev.md @@ -55,13 +55,16 @@ Dev re-runs [deploy's pipeline](deploy-cli.md#the-pipeline) with these deltas: 3. **Assemble** — identical. Dev consumes the user's built output through the same adapters and produces the same bundles; missing output produces the same "run your build" error. -4. **Containers + emulators** — the `localTarget.container` descriptor resolves a - stable local identity from the app name with no platform calls. Then the +4. **Containers + emulators** — the `localTarget.container` descriptor resolves + the app's local identity, its name, with no platform calls. Then the `localTarget.emulators` hook inspects the loaded graph and ensures one emulator daemon per node kind the topology uses (Compute always; buckets when bucket resources exist; Postgres when `postgres` or `raw-postgres` - resources exist). The Postgres daemon starts no server here: the database - provider asks it for one `@prisma/dev` server per `Database` at provision. + resources exist). Once Compute is up, the hook ensures each service's + Compute instance, in `graph.nodes` order, through the directory's record + (see [Which instance backs a node](#which-instance-backs-a-node)). The + Postgres daemon starts no server here: the database provider asks it for + one `@prisma/dev` server per `Database` at provision. 5. **Lower + converge** — a dev-generated stack file (ADR-0007's pattern, at `.prisma-composer/dev/alchemy.run.ts`), driven with the extension's `localTarget.providers()` layer and `devState()` (Alchemy's built-in file store, exported by `@prisma/composer/local-target`), always @@ -81,16 +84,41 @@ The seam between converge (terminating) and serving (long-running). One emulator daemon per node kind runs machine-scoped and multi-tenant — the Compute emulator is the one the framework builds in full: a small local counterpart of the platform's compute service, with a loopback admin -endpoint. The local `Deployment` provider does not spawn processes; it +endpoint. + +The Compute emulator knows nothing about Composer +([ADR-0052](../90-decisions/ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md)). +It holds **instances**, each a slot for one service process. A client creates +an instance with free-form labels and an optional requested port; the +emulator generates the id (`svc_<16 hex digits>`), stores the labels verbatim +and never reads them. Every later call names the id: put a deployment, start +or stop a list of instances (a stop skips ids it does not have, since they run +nothing; a start fails naming them), delete one, read or follow its log, get +one, or list them all. Creating an instance is the one call the client never +retries, because a retry after a lost response would create a second +instance. The emulator never interprets Composer's data: an app name, a +node address or a directory reaches it only as a label value or inside the +deployment payload (the artifact directory and the env), which it stores +verbatim and hands to the process. No route, state key, file name or field the +emulator generates carries one, and a test fails if one does. + +**Ports.** An instance gets its port when it is created and keeps it until it +is deleted, running or stopped. A requested port is used when no instance +holds it and it can be bound; otherwise the instance gets the lowest such port +from 3000. The port is in the deployment's env (`COMPOSER_
_PORT`) and +in its dependents' URLs, and a warm start does not redeploy, so a port that +changed between sessions would leave services built for the old one. + +The local `Deployment` provider does not spawn processes; it unpacks the artifact into `.prisma-composer/dev/artifacts//`, materializes the complete child env, and puts the deployment at the -emulator, which owns the processes: +service's instance, and the emulator owns the processes: - **New or changed deployment** (artifact hash or env changed) → stop the old child if any, spawn the artifact's `bootstrap.js` under bun with the materialized env. Only the changed service restarts — Alchemy's diff already limited which deployments were re-put. -- **Readiness** → a service declared `compute({ ..., listens: false })` (the cron scheduler, which receives no requests) is `running` as soon as its process is spawned; the descriptor records `listens: false` in the artifact's `compute.manifest.json`, the local `Deployment` provider reads it from there and sends it with the deployment, and the emulator stores it with the service. Every other service is `running` only once its port accepts a TCP connection on loopback (IPv4 or IPv6). The emulator probes the port every 50 ms until it does, so a service that counts connections sees a few extra ones. The compute emulator's `--ready-poll-ms` argument changes that interval; the CLI never passes it, and the emulator's own tests use it to widen the moment between a process exiting and its start noticing. The deployment PUT and the session-resume start return only then, so a converge, and the `[dev] ready:` line after it, finish only when every URL answers. A redeploy of a running service sets it to `stopped` once the old process has exited, and back to `running` when the new one listens. A child that exits before it listens fails the call with its address, exit code, and log path. A child that does not listen within the readiness timeout (30 seconds; `PRISMA_COMPOSER_SERVICE_READY_TIMEOUT_MS`, which the CLI passes to the compute emulator as `--ready-timeout-ms` when it starts the emulator) is stopped, and the call fails with its address, port, log path and the variable's name. A crash-backoff restart that times out ends `held`, like other crash loops. +- **Readiness** → a service declared `compute({ ..., listens: false })` (the cron scheduler, which receives no requests) is `running` as soon as its process is spawned; the descriptor records `listens: false` in the artifact's `compute.manifest.json`, the local `Deployment` provider reads it from there and sends it with the deployment, and the emulator stores it with the instance; it is a property of the deployment, not of an app or address. Every other service is `running` only once its port accepts a TCP connection on loopback (IPv4 or IPv6). The emulator probes the port every 50 ms until it does, so a service that counts connections sees a few extra ones. The compute emulator's `--ready-poll-ms` argument changes that interval; the CLI never passes it, and the emulator's own tests use it to widen the moment between a process exiting and its start noticing. The deployment PUT and the session-resume start return only then, so a converge, and the `ready` line after it, finish only when every URL answers. A redeploy of a running service sets it to `stopped` once the old process has exited, and back to `running` when the new one listens. A child that exits before it listens fails the call with its address, exit code, and log path. A child that does not listen within the readiness timeout (30 seconds; `PRISMA_COMPOSER_SERVICE_READY_TIMEOUT_MS`, which the CLI passes to the compute emulator as `--ready-timeout-ms` when it starts the emulator) is stopped, and the call fails with its address, port, log path and the variable's name. A crash-backoff restart that times out ends `held`, like other crash loops. - **Concurrent starts and stops** → a service has at most one process and at most one start in flight; the rules are in [Concurrent starts and stops](#concurrent-starts-and-stops) below. - **Child exits unexpectedly** → the emulator restarts it with backoff and writes its supervision events into that service's log stream; repeated @@ -104,11 +132,12 @@ emulator, which owns the processes: the `log` operation's `tail` count maps straight onto it. A live follower is reset to the new start when the log is cleared under it (the daemon tracks a per-log clear generation, so a follower never reads stale bytes). -- **Instance deleted** (service removed from the topology, `--fresh`) → stop - and remove. +- **Instance deleted** (service removed from the topology, `--fresh`, or + another app run from the same directory) → stop it, remove it and its log, + and free its port. - **Ctrl-C on the dev command** → the attachment stops the app's service instances and detaches. Every emulator — Compute, Postgres instances, the - bucket emulator — stays up with its data; the next start reprovisions the + bucket emulator — stays up with its data; the next start reuses the same instances (same ports, same data). `--fresh` is what removes instances and data. (A detached mode where services keep serving with no session is a designed extension, not v1.) @@ -121,7 +150,7 @@ emulator, which owns the processes: live status, which Alchemy's state file does not track. So a second `prisma dev` after a plain Ctrl-C can converge with everything reported "noop" and leave every previously-stopped service `stopped` — - the CLI still prints `[dev] ready:` with each service's URL, but nothing + the CLI still prints `ready` with each service's URL, but nothing is listening on them. Confirmed against the open-chat proving port (the warm-restart-noop finding); the existing store proving script's own criterion-6 check doesn't catch it because it asserts port *stability* @@ -147,7 +176,7 @@ Each service has at most one process and at most one start in flight. A stop in | Deployment PUT, identical | `running`, no start or stop in progress | Succeeds at once | | Deployment PUT | Anything else | Stores the deployment and becomes the start in flight, replacing any earlier one: waits for any stop to finish, stops the current process, then spawns | | Stop | Any | Ends the start in flight and any crash-backoff restart, stops the process, and sets `stopped` once it has exited | -| App delete | Any | Marks each service deleted, stops it, and removes it; no start on a deleted service spawns | +| Instance delete | Any | Marks the instance deleted, stops it, and removes it; no start on a deleted instance spawns | | Crash-backoff restart | After an unexpected exit | Becomes the start in flight and spawns at once | A start in flight ends with the first of these that applies: @@ -157,7 +186,7 @@ A start in flight ends with the first of these that applies: | A newer start replaced it | The newer start's result | | `bun` is not on the deployment's `PATH` | Fails with how to install bun; the service is `stopped` | | The service was deployed with `listens: false` | Succeeds as soon as the process is spawned; the service is `running` | -| A stop or an app delete ended it | Fails: "was stopped before it accepted connections" | +| A stop or an instance delete ended it | Fails: "was stopped before it accepted connections" | | Its process exited | Fails with the exit code and the log path; crash supervision handles the exit | | The readiness timeout passed | Fails with the port, the log path and the timeout variable; the process is stopped and the service ends `stopped`, or `held` for a crash-backoff restart | | The port accepted a connection | Succeeds; the service is `running` | @@ -177,6 +206,63 @@ snapshot diffs on bytes — which restart-amplified dependents on every first-after-cold converge. No sanctioned reader consumes sibling rows, so the scoping changes restart behavior only. +The emulator's failure messages name the instance (`instance "svc_…" exited +(code 1) before it accepted connections…`). The local providers and the +attachment replace the id with the service's address before the message +reaches the user, so `prisma dev` says `service "gateway" …`. + +### Which instance backs a node + +Composer records, per working directory, which emulator instance backs each +node of the app run there, in `.prisma-composer/dev/emulator-instances.json`: + +```json +{ + "app": "shop", + "nodes": { + "quotes": { "kind": "compute", "id": "svc_0b6e2f4a91c3d758", "port": 3000 }, + "gateway": { "kind": "compute", "id": "svc_81ad5c07e2f94b31", "port": 3001 } + } +} +``` + +The record is the only place Composer's names are mapped to instances. The +emulators hook, the local `App` and `Deployment` providers, the attachment +(endpoints, logs, start, stop) and `--fresh` all go through it; no Composer +code finds an instance by its labels. The labels Composer writes on create — +`composer.directory` (the absolute directory), `composer.app` and +`composer.address` — are for people reading an emulator listing. The record +can be read from the directory alone, without running the pipeline. + +- **Ensuring an instance.** The emulators hook goes through the services in + `graph.nodes` order (dependencies first, ties in declaration order). A + recorded instance the emulator still has is reused. Otherwise a new one is + created and recorded: on a first start, with no port requested, so ports + follow that order from 3000; after the emulator lost the instance, requesting + the recorded port. A service added by a rebuild during a session gets its + instance from the `App` provider the same way. +- **A remembered port that is taken.** It is a preference: the emulator gives + the lowest free port instead, and `prisma dev` prints + `[dev] service "
" now uses port : its previous port is in use`. +- **Redeploying a re-created instance.** A re-created instance runs nothing, + and a warm start's converge changes nothing, so Alchemy would never call the + providers. The `Deployment` provider's diff asks for an update when its + instance does not run the recorded artifact, and the `App` provider's diff + asks for one when its instance's URL changed, so dependents get the new URL. +- **A removed service.** Deleting its `App` resource deletes its instance and + its entry. +- **Another app in the same directory.** A directory has one dev instance. + When the record names a different app, the emulators hook deletes that app's + Compute instances, starts a new record, and prints + `[dev] removed the local services of "", the app run here before`. +- **`--fresh`** deletes the recorded Compute instances, then removes the dev + state directory with the record in it. + +Two directories running the same app have separate records, so they get +separate instances on separate ports, and neither restarts the other's +services. State the Compute emulator wrote before instances had generated ids +is discarded when it loads; local dev state is never migrated. + ### Process lifetimes 1. **Converge-scoped** — the Alchemy child (ADR-0007), started as @@ -186,12 +272,14 @@ the scoping changes restart behavior only. multi-tenant, holding every long-lived process: the Compute emulator (service children), the Postgres emulator (one `@prisma/dev` server per `Database`, hosted in-process), and the bucket emulator. They survive dev - sessions; `--fresh` removes an app's instances. The registry root is + sessions; `--fresh` removes a directory's instances. The registry root is `~/.prisma-composer/emulators` unless `PRISMA_COMPOSER_EMULATORS_DIR` names another absolute directory, so by default one machine shares one - set. A root isolates the compute and buckets daemons and their registry, - not Postgres: Postgres instance names come from the app and database, so - two checkouts of one app still share a server. A daemon whose version + set. Compute instances belong to a directory, so two checkouts of one app + never share a service. Postgres and buckets are still keyed by the app: + Postgres instance names come from the app and database, and bucket names + from the app and bucket, so two checkouts of one app still share a + database server and bucket names, whatever their roots. A daemon whose version differs from the caller's is replaced, so two Composer versions sharing a root replace each other's daemons back and forth; a separate root per checkout stops that. @@ -215,8 +303,8 @@ semantics): | `Project` | a local identity record; no platform | | `Database` | a database on a local `@prisma/dev` Postgres server, hosted by the Postgres emulator | | `Connection` | the local connection URL | -| `App` | registers the service with the Compute emulator, which allocates its stable port; `appEndpointDomain = http://localhost:` — which makes origin (ADR-0039) work unchanged | -| `Deployment` | unpacks the artifact once per hash, materializes the env, and puts the deployment at the Compute emulator, which (re)starts the child | +| `App` | finds the service's Compute instance through the directory's record, creating it when there is none; the instance holds the service's port; `appEndpointDomain = http://localhost:` — which makes origin (ADR-0039) work unchanged. Deleting it deletes the instance | +| `Deployment` | unpacks the artifact once per hash, materializes the env, and puts the deployment at the service's Compute instance, which (re)starts the child; updated when its instance does not run the recorded artifact | | `EnvironmentVariable` | a key→value row in the dev state store | | `Bucket` | a directory under `.prisma-composer/dev/buckets//`, served by the bucket emulator | | `BucketKey` | credentials registered with the bucket emulator | @@ -335,7 +423,7 @@ Deploy's rule holds: every failure names its fix. (none outstanding from the design phase — see Known limitations below for gaps found during implementation.) -(Settled since the first draft: Postgres runs one named `@prisma/dev` server per `Database` resource; the front door prints every service URL ordered by address depth then name, shallowest first; on a fresh start, services get their ports in dependency order, ties in declaration order — the emulators hook reserves them one at a time from `graph.nodes` before Alchemy applies the `App` resources concurrently — and a warm start keeps each service's saved port; port allocation and the remaining mechanics are described in this document and [ADR-0041](../90-decisions/ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md). Restart latency is measured — see Known limitations.) +(Settled since the first draft: Postgres runs one named `@prisma/dev` server per `Database` resource; the front door prints every service URL ordered by address depth then name, shallowest first; on a fresh start, services get their ports in dependency order, ties in declaration order — the emulators hook creates their Compute instances one at a time from `graph.nodes` before Alchemy applies the `App` resources concurrently — and a warm start keeps each service's port, because an instance holds its port until it is deleted (ADR-0052); port allocation and the remaining mechanics are described in this document and [ADR-0041](../90-decisions/ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md). Restart latency is measured — see Known limitations.) ## Proven against a real app @@ -379,6 +467,9 @@ replaces. - [ADR-0041](../90-decisions/ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md) — the decision this doc details. +- [ADR-0052](../90-decisions/ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md) + — the emulators know nothing about Composer; the per-directory record of + which instance backs each node. - [deploy-cli.md](deploy-cli.md) — the pipeline dev re-runs and the error-surface convention it extends. - [alchemy-lowering.md](../05-prisma-cloud/alchemy-lowering.md) — the resource diff --git a/docs/design/90-decisions/ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md b/docs/design/90-decisions/ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md index 67f42c069..32e5b491a 100644 --- a/docs/design/90-decisions/ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md +++ b/docs/design/90-decisions/ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md @@ -2,6 +2,8 @@ Amended by [ADR-0050](ADR-0050-composer-runs-as-prisma-deploy-and-prisma-dev.md): the command is `prisma dev `, and reading the running app's logs is the `log` operation on `@prisma/composer/control`, not a command. The `prisma-composer` binary is retired. The local pipeline below is unchanged. +Amended by [ADR-0052](ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md): the emulators know nothing about Composer. A dev instance is no longer identified on the emulators by the app name; Composer records, per working directory, which emulator instance backs each node, and the Compute emulator keys its instances by ids it generates. The Postgres and bucket emulators still key instances by app name until they move to the same model. + ## Decision `prisma-composer dev ` boots the whole application locally, with no cloud diff --git a/docs/design/90-decisions/ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md b/docs/design/90-decisions/ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md new file mode 100644 index 000000000..aa9b4c9b3 --- /dev/null +++ b/docs/design/90-decisions/ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md @@ -0,0 +1,60 @@ +# ADR-0052: Composer maps its nodes to emulator instances per directory + +## Decision + +The local emulators know nothing about Composer. An emulator holds instances under ids it generates. Each instance carries labels that the emulator stores and returns but never reads. Composer records, per working directory, which emulator instance backs each node of the app run there. That record is the only place Composer's names are mapped to instances. + +```text +~/work/shop/.prisma-composer/dev/emulator-instances.json (written by Composer) + { "app": "shop", + "nodes": { "quotes": { "kind": "compute", "id": "svc_k3f9…", "port": 3000 }, + "gateway": { "kind": "compute", "id": "svc_81ad…", "port": 3001 } } } + +Compute emulator (one per machine, knows nothing about Composer) + GET /instances → [ { "id": "svc_k3f9…", "port": 3000, "status": "running", + "labels": { "composer.directory": "/Users/me/work/shop", + "composer.app": "shop", + "composer.address": "quotes" } }, … ] +``` + +A second checkout of the same app in `~/work/shop-2` has its own record and its own instances. The two never share a service process or a port. + +This amends [ADR-0041](ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md), whose local container gives a dev instance a stable identity from the app name alone. The Compute emulator is keyed this way. The Postgres and bucket emulators still key their instances by app name and are moved to the same model separately. + +## Reasoning + +`prisma dev` runs an app against local emulators: a Compute emulator that runs service processes, a Postgres emulator and a bucket emulator. Each is a long-lived daemon shared by everything on the machine. ADR-0041 identified an app's instances on those daemons by the app's name, so the emulators' routes and state were shaped around apps: `/apps//services/`. Two directories that run the same app, such as two git worktrees of one repository, therefore resolved to the same instances. Each `prisma dev` redeployed over the other's services. That is a common situation, not an edge case: agents work in many worktrees at once and start and stop environments in each. + +The emulators should not have to know what a directory or an app is to avoid this. So an instance belongs to whoever created it. A client creates an instance, the emulator generates its id, and every later operation names that id: deploy, start, stop, delete, logs. The emulator never interprets Composer's data. An app name, a node address or a directory reaches it only as a label value, or inside the deployment it is handed to run: the artifact directory, which lies under the project, and the environment, whose keys are named after addresses. The emulator stores both verbatim and passes them to the process it spawns. No route, state key, file name or field the emulator generates carries any of them, and a test fails if one does. The emulators can then be used by something other than Composer, and moved out of this repository, without carrying Composer's model along. + +Labels make a listing readable. Composer writes `composer.directory`, `composer.app` and `composer.address` on every instance it creates, so a person looking at the emulator's list of instances can tell what started each one. Nothing reads them back as identity. If Composer found its instances by searching labels, the labels would be a second, unenforced key. + +Composer finds its instances through `emulator-instances.json` in the directory's dev state (`.prisma-composer/dev/`), which is already per directory. It records the app name and, per node address, the emulator kind, the instance id and the instance's port. The emulators hook writes it before converge. It reuses a recorded instance while the emulator still has it, and otherwise creates one and records it. Everything else reaches instances through the record: the local providers during converge, the dev session's endpoints, logs, start and stop, and `--fresh`. The record is readable from the directory alone, so a separate command can answer "where is this node running" without running the pipeline. + +An instance keeps its port from creation until it is deleted, whether it is running or stopped. A port is baked into more than the process: the service's environment carries it (`COMPOSER_
_PORT`), and its dependents' environments carry its URL. A warm start does not redeploy, because nothing in the app changed and Alchemy, the deploy engine, sees nothing to update. A service that received a different port on its next start would run with an environment built for the old one. + +The record also keeps each node's port, for the one case where the port can change: the instance has to be re-created, for example because the emulator lost its state. Composer then asks for the remembered port. A requested port is a preference: when another instance or another process holds it, the emulator gives the lowest free port from 3000, and `prisma dev` prints one line naming the service, the old port and the new one. Failing instead would break `prisma dev` in one worktree whenever another worktree happened to take the port. Because a re-created instance runs nothing, the local Deployment provider asks Alchemy for an update when its instance does not run the recorded artifact, and the local App provider does so when its instance's URL changed, so dependents get the new URL. + +A directory has one dev instance. When `prisma dev` runs a different app from a directory than the one the record names, the previous app's Compute instances are deleted, the record starts over, and `prisma dev` prints which app's local services were removed. Compute instances hold no data, so nothing is lost that a deploy does not recreate. + +## Consequences + +- Two directories running the same app run separate services on separate ports, and neither restarts the other's. +- A directory's ports are stable across `prisma dev` sessions. A port changes only when its instance is re-created and the old port is taken, and `prisma dev` says so. +- A stopped instance holds its port. Instances left behind by a deleted worktree keep their ports until something deletes them. No command does that yet. +- Local dev state written before this change is discarded: the Compute emulator drops app-keyed state when it loads, and the first `prisma dev` creates new instances. Local dev state is never migrated across versions. +- Every emulator kind moves to this model: instances keyed by emulator-generated ids, labels stored but never read, and a list of all instances. Until the Postgres and bucket emulators do, the app name still identifies their instances, so two directories running one app still share its databases and buckets. +- Composer code must not find an instance by its labels. The record is the only mapping. + +## Alternatives considered + +- **App-keyed emulators.** The model this replaces. Simple, but two directories running one app share and overwrite each other's services, and the emulators carry Composer's concepts in their routes and state. +- **The app name plus a hash of the directory as the identity.** Separates directories, but keeps app and directory concepts inside the emulators and makes every emulator route depend on how Composer forms that key. +- **Releasing a port when its instance stops.** Lets stopped worktrees hold no ports, but breaks warm starts: the port is in the deployment's environment and in its dependents' URLs, and a warm start does not redeploy, so a service started on a new port would run with an environment built for the old one. +- **Finding instances by their labels.** Needs no record file, but makes free-form display metadata into an unenforced key that any client could collide with. + +## Related + +- [ADR-0041](ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md) — local dev runs the deploy pipeline against local providers; amended here. +- [ADR-0051](ADR-0051-a-nodes-address-is-its-logical-id-on-the-platform.md) — a node's address, the key of the record. +- [local-dev.md](../10-domains/local-dev.md) — the Compute emulator, the record and port allocation. diff --git a/docs/design/90-decisions/README.md b/docs/design/90-decisions/README.md index 9eeac2420..e2f8d7bf6 100644 --- a/docs/design/90-decisions/README.md +++ b/docs/design/90-decisions/README.md @@ -62,7 +62,7 @@ _Earlier drafts (ADR-0001, ADR-0002) were retired as the high-level design settl - [ADR-0038](ADR-0038-containers-are-an-extension-descriptor.md) — Container lifecycle (ensure/locate/remove) is an optional `container` descriptor on `ExtensionDescriptor`, the same pattern as `preflight`/`teardown`; the resolved instance is opaque to core and crosses the CLI parent→alchemy child boundary as one framework-named environment variable per extension, via the extension's own `serialize()`/`deserialize()`. `StateDescriptor` names its owning extension so core can hand it that extension's resolved container. Deletes the `crossDomainExceptions` entry that let the CLI import `@internal/lowering` directly — `0-framework` imports nothing again. - [ADR-0039](ADR-0039-a-compute-services-own-origin-is-a-target-resolved-property.md) — A compute service's own platform-assigned origin is a target-resolved property, read as the service's `origin()` — never a declared param, never operator config, never in `config()`. It rides ADR-0031's reserved provider-param channel as the first *service-derived* entry (`valueForService(provisioned, address)`, written for every compute service, exposing or not), sourced from the provisioned App's `appEndpointDomain` (ADR-0048) — made trustworthy pre-promote by the upstream PRO-200 fix. `envParam(…)` remains correct for operator-known origins (custom domains); narrows ADR-0032's `appOrigin` example accordingly. - [ADR-0040](ADR-0040-the-orm-binding-carries-the-url-and-a-lazy-client.md) — `postgres(contract)`'s dependency binding is `{ url, client }`: the raw connection string plus the typed client, constructed lazily and memoized on first `client` access — `hydrate` builds nothing. The contract remains the compatibility interface (hash check and deploy-time migration unchanged, ADR-0022); the binding becomes a strict superset of plain `rawPostgres()`'s `{ url }`, so an app that owns its database client still gets framework-run migrations. Contract validation cost and failure move from `load()` (where one bad input poisoned every input, unattributed) to the first `client` access. Cross-kind satisfaction (`'postgres'` satisfying `'postgres'`) rejected in its favor. -- [ADR-0041](ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md) — `prisma-composer dev` runs the **same deploy pipeline** (Load → assemble → lower → Alchemy converge) against local implementations of the same Alchemy resource types, declared on an optional `localTarget` field of `ExtensionDescriptor` (a lazy thunk resolving a `LocalTargetDescriptor`; subpaths `@prisma/composer/local-target` and `@prisma/composer-prisma-cloud/local-target` — "dev" names only the user-facing command/prefix/state dir) (providers, container, preflight, emulators, attach, teardown — **no** `nodes`/`provisions`, so the lowering cannot diverge; no `state` either — dev uses Alchemy's own `localState()` through `LowerOptions.state`). The target runs **emulators per node kind**: Compute and buckets are machine-global, multi-tenant daemons (the Compute emulator owns the service child processes — deployment PUTs, crash supervision, logs; buckets serve the S3 wire over plain files on disk), while Postgres runs one detached ORM `prisma dev` instance per `Database` resource under the ORM CLI's own manager. Providers provision instances by communicating with the emulators during converge, and the dev command is a view through `attach`; `ServiceKey`/`S3Credentials`/`PgWarm`/`OrmMigration` are shared verbatim. Credential-free by requirement. Rejects a local Management API (reimplements another team's server-side semantics, drifts silently) and per-kind dev descriptors (an open-set parallel seam). *(Amended by ADR-0050: the command is `prisma dev `; logs are the `log` operation, not a command.)* +- [ADR-0041](ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md) — `prisma-composer dev` runs the **same deploy pipeline** (Load → assemble → lower → Alchemy converge) against local implementations of the same Alchemy resource types, declared on an optional `localTarget` field of `ExtensionDescriptor` (a lazy thunk resolving a `LocalTargetDescriptor`; subpaths `@prisma/composer/local-target` and `@prisma/composer-prisma-cloud/local-target` — "dev" names only the user-facing command/prefix/state dir) (providers, container, preflight, emulators, attach, teardown — **no** `nodes`/`provisions`, so the lowering cannot diverge; no `state` either — dev uses Alchemy's own `localState()` through `LowerOptions.state`). The target runs **emulators per node kind**: Compute and buckets are machine-global, multi-tenant daemons (the Compute emulator owns the service child processes — deployment PUTs, crash supervision, logs; buckets serve the S3 wire over plain files on disk), while Postgres runs one detached ORM `prisma dev` instance per `Database` resource under the ORM CLI's own manager. Providers provision instances by communicating with the emulators during converge, and the dev command is a view through `attach`; `ServiceKey`/`S3Credentials`/`PgWarm`/`OrmMigration` are shared verbatim. Credential-free by requirement. Rejects a local Management API (reimplements another team's server-side semantics, drifts silently) and per-kind dev descriptors (an open-set parallel seam). *(Amended by ADR-0050: the command is `prisma dev `; logs are the `log` operation, not a command. Amended by ADR-0052: emulator instances are keyed by emulator-generated ids, not the app name, and Composer maps its nodes to them per directory.)* - [ADR-0042](ADR-0042-service-input-is-one-standard-schema.md) — A compute service declares its entire incoming configuration — config and secrets together — as one Standard Schema (`input`), read back through one typed accessor; `params`/`secrets` and `config()`/`secrets()` are replaced. The framework never introspects the schema (validate-only, per the spec): the operator's binding is the traversable structure (sourcing: literals, `envParam`, `envSecret`), the schema is the black-box judge of legality (invoked at deploy over the resolved binding with secrets as opaque `SecretString` boxes, and again at boot), and secretness is a leaf *type* enforced by validation in both directions. The wire format is one self-describing JSON document row per service with `$secret` pointers to platform variables; an env-bound key whose variable is unset resolves to key-omitted and the schema arbitrates absence — subsuming optional secrets and conditional config (`stripeId` only when `stripeEnabled`) without a framework DSL. - [ADR-0043](ADR-0043-the-control-subpath-is-the-programmatic-deploy-surface.md) — `@prisma/composer/control` is the programmatic deploy surface: typed `deploy`/`destroy`/`dev`/`log` operations (structured inputs/results, no argv/console/exit) implemented in `@internal/cli`'s `src/operations/` and re-exported per ADR-0035; the CLI is a thin renderer over them. The entry's static graph stays import-light — each operation lazily imports its executor, so importing the subpath executes nothing, and a tree that cannot load the deploy stack surfaces as a structured `pipeline` failure — and `PRISMA_COMPOSER_DEPLOYMENT_RESULT_FILE` carries the deploy result across the process boundary: the alchemy child's report hook writes a serializable `DeploymentSummary` to the named file, the operation reads it back best-effort (absent/malformed = undefined summary, never a failure). Distinct from an extension's ADR-0017 `/control` entry. *(Amended by ADR-0049: each operation takes a required `config: { value, file }`, and the effect-conflict diagnosis is retired.)* *(Amended by ADR-0050: the `prisma` CLI renders two of the four operations, as `prisma deploy` and `prisma dev`; `destroy` and `log` are called from scripts.)* - [ADR-0044](ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md) — Errors are structural envelopes with dotted `NAMESPACE.SUBCODE` codes (the shared prisma/prisma foundation, duplicated pending extraction): structured at origin with why/fix splits, no catch-all codes, bugs carry no code (exit 1 + report hint), recognition is structural (`CliStructuredError.is()`), operation results ride the shared `Result` `ok` discriminator, expected failures exit 2 — with the alchemy child-status passthrough as the documented exception. *(Amended by ADR-0049: `DEPS.EFFECT_VERSION_CONFLICT` is retired.)* @@ -73,3 +73,4 @@ _Earlier drafts (ADR-0001, ADR-0002) were retired as the high-level design settl - [ADR-0049](ADR-0049-composers-configuration-is-the-composer-section-of-prisma-config.md) — Composer's configuration is the `composer` section of `prisma.config.ts`, loaded by the Prisma CLI engine. The validator checks only the fields that identify extension and state descriptors, passes descriptors through as the file's own objects, rejects unknown keys, and takes the nearest declaring file's section whole so the generated stack imports exactly one file. The old `prisma-composer.config.ts` and the `configPath` pointer are refused with `CONFIG.SECTION_MISSING`, `CONFIG.FIELD_RETIRED` and `CONFIG.FILE_RETIRED`, never migrated. The validator returns the section with its declaring file, and the programmatic operations take the same `ComposerConfigSource` (`{ value, file }`) and run the same checks. The Alchemy child evaluates the whole `prisma.config.ts`. The effect pre-flight and `DEPS.EFFECT_VERSION_CONFLICT` are retired: a broken tree fails evaluation of `prisma.config.ts` with the engine's `CLI.CONFIG_UNREADABLE`. *(Amended: the alternatives section lists the rejected ways to keep a Composer-owned effect check.)* - [ADR-0050](ADR-0050-composer-runs-as-prisma-deploy-and-prisma-dev.md) — Composer has no binary of its own: the `prisma` CLI mounts exactly two Composer commands, `prisma deploy ` and `prisma dev `, mounted at the root as bare verbs on purpose rather than under a noun. `destroy` and `log` are operations on `@prisma/composer/control`, called from a script; a teardown script reads `PRISMA_SERVICE_TOKEN` and `PRISMA_WORKSPACE_ID`. Where teardown and logs belong in the `prisma` command tree is not decided; until it is, agents and users need a script for both. A `destroy` command was rejected: the CLI's destructive verb is `delete`, and no Composer command takes `--production`. Retires the `prisma-composer` binary; a CI lint keeps its name out of user-facing files. - [ADR-0051](ADR-0051-a-nodes-address-is-its-logical-id-on-the-platform.md) — A node's address is its identity on Prisma Cloud: Composer writes it, byte for byte, as the `logicalId` of the node's one platform row (Project, App, Database, Bucket) and as the node's `logicalId` in the application topology, so the platform joins topology nodes to rows, and one node across branches, by string equality. A provision ID defaults to the node's name, so changing a name changes identity. The value is never an Alchemy resource ID, a display name or a platform `id`; App, Database and Bucket rows get it as a prop on the upstream resource that creates them, and the Project from container resolution; no separate resource writes it. Changing an address changes the entity's identity; changing a display name changes nothing. +- [ADR-0052](ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md) — The local emulators know nothing about Composer: an emulator keys its instances by ids it generates and stores labels it never reads. Composer records per working directory, in `.prisma-composer/dev/emulator-instances.json`, which instance backs each node, and nothing else maps Composer's names to instances, so two directories running one app get separate instances. An instance keeps its port until it is deleted, because the port is in the deployment's environment and its dependents' URLs and a warm start does not redeploy; a remembered port is requested, as a preference, only when an instance is re-created. Running a different app from a directory removes the previous app's Compute instances. Amends ADR-0041's app-name local identity. The Postgres and bucket emulators move to this model separately. diff --git a/docs/guides/getting-started.md b/docs/guides/getting-started.md index 14396503e..712f14dbd 100644 --- a/docs/guides/getting-started.md +++ b/docs/guides/getting-started.md @@ -290,9 +290,9 @@ It runs the same pipeline a deploy runs, against local stand-ins for Prisma Cloud, and prints the front door — each service's local URL: ``` -[dev] ready: -[dev] gateway http://localhost:3001 -[dev] quotes http://localhost:3000 +dev: ready +gateway: http://localhost:3001 +quotes: http://localhost:3000 curl localhost:3001 # Make it work, make it right, make it fast. diff --git a/docs/guides/running-locally.md b/docs/guides/running-locally.md index e847d48b4..0b0c0c9a7 100644 --- a/docs/guides/running-locally.md +++ b/docs/guides/running-locally.md @@ -35,10 +35,10 @@ It builds nothing for you (bring your own built output, same as deploy), then stands the app up and prints the **front door** — every service's local URL: ``` -[dev] ready: -[dev] storefront http://localhost:3004 -[dev] catalog.service http://localhost:3000 -[dev] orders.service http://localhost:3003 +dev: ready +storefront: http://localhost:3004 +catalog.service: http://localhost:3000 +orders.service: http://localhost:3003 ``` `dev` prints `ready` only once every service accepts connections on its port, so a request to a printed URL right away connects. After a rebuild, `dev` waits the same way before it prints the URLs again. A service must listen on the port Composer gives it, unless it is declared with `compute({ ..., listens: false })`: such a service, like the cron module's scheduler, is running as soon as its process starts. @@ -67,6 +67,39 @@ instances and data before starting. On a fresh start, each service gets the lowest free port from 3000 up, in dependency order: a service gets its port before the services that call it, and services with no dependency between them go in the order you provisioned them. A warm start keeps the ports services already have. +### Each directory runs its own services + +The services `dev` runs belong to the directory you run it in. Two checkouts +of the same app, such as two git worktrees, run separate service processes on +separate ports, and a rebuild in one never restarts the other's services. +`dev` records which local service backs each of your services in +`.prisma-composer/dev/emulator-instances.json`. + +A service keeps its port in its directory from one `dev` to the next, even +while it is stopped. If the local service is gone, for example because a +Composer upgrade discarded the compute emulator's state, the next `dev` +creates it again and asks for the port it had. When another directory has +taken that port, the service gets the lowest free one and `dev` prints one +line: + +``` +[dev] service "gateway" now uses port 3004: its previous port 3001 is in use +``` + +A directory runs one app at a time. If you run a different app from the same +directory, `dev` removes the previous app's local services first and says so: + +``` +[dev] removed the local services of "shop", the app run here before +``` + +A stopped service keeps its port reserved, so the services of a checkout you +delete keep their ports. There is no command to remove them yet. + +Local databases and buckets are not separated this way yet: they are still +keyed by the app's name, so two checkouts of one app share them (see +[What's local vs. what's real](#whats-local-vs-whats-real)). + `--fresh` is also the fix when a framework upgrade leaves stale rows in this app's local dev state — the symptom is a plan-time error naming an unregistered resource type (for example @@ -142,7 +175,8 @@ token, workspace, or network is involved. Three consequences worth knowing: - **`PRISMA_COMPOSER_EMULATORS_DIR` gives a checkout or a CI job its own compute and buckets emulators.** By default every `dev` on the machine shares one set of emulator daemons, registered under - `~/.prisma-composer/emulators`. Set the variable to an absolute directory + `~/.prisma-composer/emulators`; each directory still gets its own services + on them. Set the variable to an absolute directory and `dev` starts and finds its compute and buckets daemons, their registry and their data there instead. Each picks free ports, so several can run side by side. A relative path is refused with `DEV.EMULATORS_DIR_INVALID`; `~` is diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-deployment.test.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-deployment.test.ts index f3c77b806..45c559607 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-deployment.test.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-deployment.test.ts @@ -6,7 +6,7 @@ * started. */ import { afterEach, beforeEach, describe, expect, test } from 'bun:test'; -import { computeClient, type ServiceInfo } from '../client.ts'; +import { computeClient, type InstanceInfo } from '../client.ts'; import { isPidAlive, stopDaemon } from '../daemon.ts'; import { CRASHING_BOOTSTRAP, @@ -46,30 +46,26 @@ function baseEnv(extra: Record = {}): Record { return { PATH: path, ...extra }; } -async function deployedService(app: string, id: string): Promise { - const client = computeClient({ registryRoot }); - const list = await client.listServices(app); - const svc = list.find((s) => s.id === id); - if (!svc) throw new Error(`service "${id}" not found in app "${app}"`); - return svc; +async function deployedService(id: string): Promise { + const instance = await computeClient({ registryRoot }).getInstance(id); + if (!instance) throw new Error(`instance "${id}" not found`); + return instance; } describe('deployment lifecycle', () => { test('a deployment spawns bun bootstrap.js and the service actually serves HTTP', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-a', 'web'); + const reserved = await client.createInstance({ labels: {} }); - await client.putDeployment('app-a', 'web', { - address: 'app-a.web', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'hash-v1', env: baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv('hello from v1') }), - port: reserved.port, }); await waitFor(async () => { - const svc = await deployedService('app-a', 'web'); + const svc = await deployedService(reserved.id); return svc.status === 'running'; }, 5000); @@ -80,55 +76,47 @@ describe('deployment lifecycle', () => { test('an identical redeploy (same hash and env) does not restart the running child', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-b', 'web'); + const reserved = await client.createInstance({ labels: {} }); const env = baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv('v1') }); - await client.putDeployment('app-b', 'web', { - address: 'app-b.web', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'same-hash', env, - port: reserved.port, }); - await waitFor(async () => (await deployedService('app-b', 'web')).status === 'running', 5000); - const before = await deployedService('app-b', 'web'); + await waitFor(async () => (await deployedService(reserved.id)).status === 'running', 5000); + const before = await deployedService(reserved.id); - await client.putDeployment('app-b', 'web', { - address: 'app-b.web', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'same-hash', env, - port: reserved.port, }); - const after = await deployedService('app-b', 'web'); + const after = await deployedService(reserved.id); expect(after.pid).toBe(before.pid); }); test('a redeploy with a changed artifactHash restarts the child (new pid)', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-c', 'web'); + const reserved = await client.createInstance({ labels: {} }); const env = baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv('v1') }); - await client.putDeployment('app-c', 'web', { - address: 'app-c.web', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'hash-1', env, - port: reserved.port, }); - await waitFor(async () => (await deployedService('app-c', 'web')).status === 'running', 5000); - const before = await deployedService('app-c', 'web'); + await waitFor(async () => (await deployedService(reserved.id)).status === 'running', 5000); + const before = await deployedService(reserved.id); - await client.putDeployment('app-c', 'web', { - address: 'app-c.web', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'hash-2', env, - port: reserved.port, }); await waitFor(async () => { - const svc = await deployedService('app-c', 'web'); + const svc = await deployedService(reserved.id); return svc.status === 'running' && svc.pid !== before.pid; }, 5000); }); @@ -136,27 +124,23 @@ describe('deployment lifecycle', () => { test('a redeploy with a changed env (same hash) also restarts the child', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-d', 'web'); + const reserved = await client.createInstance({ labels: {} }); - await client.putDeployment('app-d', 'web', { - address: 'app-d.web', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'stable-hash', env: baseEnv({ PORT: String(reserved.port), EXTRA: 'a', ...servingBootstrapEnv('v1') }), - port: reserved.port, }); - await waitFor(async () => (await deployedService('app-d', 'web')).status === 'running', 5000); - const before = await deployedService('app-d', 'web'); + await waitFor(async () => (await deployedService(reserved.id)).status === 'running', 5000); + const before = await deployedService(reserved.id); - await client.putDeployment('app-d', 'web', { - address: 'app-d.web', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'stable-hash', env: baseEnv({ PORT: String(reserved.port), EXTRA: 'b', ...servingBootstrapEnv('v1') }), - port: reserved.port, }); await waitFor(async () => { - const svc = await deployedService('app-d', 'web'); + const svc = await deployedService(reserved.id); return svc.status === 'running' && svc.pid !== before.pid; }, 5000); }); @@ -164,24 +148,22 @@ describe('deployment lifecycle', () => { test('a stopped service always starts on redeploy, even with an identical hash and env', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-e', 'web'); + const reserved = await client.createInstance({ labels: {} }); const env = baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv('v1') }); const deployment = { - address: 'app-e.web', artifactDir, artifactHash: 'h', env, - port: reserved.port, }; - await client.putDeployment('app-e', 'web', deployment); - await waitFor(async () => (await deployedService('app-e', 'web')).status === 'running', 5000); + await client.putDeployment(reserved.id, deployment); + await waitFor(async () => (await deployedService(reserved.id)).status === 'running', 5000); - await client.stopApp('app-e'); - await waitFor(async () => (await deployedService('app-e', 'web')).status === 'stopped', 5000); + await client.stopInstances([reserved.id]); + await waitFor(async () => (await deployedService(reserved.id)).status === 'stopped', 5000); - await client.putDeployment('app-e', 'web', deployment); - await waitFor(async () => (await deployedService('app-e', 'web')).status === 'running', 5000); + await client.putDeployment(reserved.id, deployment); + await waitFor(async () => (await deployedService(reserved.id)).status === 'running', 5000); }); }); @@ -189,31 +171,24 @@ describe('crash supervision', () => { test('an unexpected crash goes to backoff and restarts after the pinned 1s·2ⁿ delay', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(CRASHING_BOOTSTRAP); - const reserved = await client.ensureService('app-f', 'crasher'); + const reserved = await client.createInstance({ labels: {} }); await expect( - client.putDeployment('app-f', 'crasher', { - address: 'app-f.crasher', + client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }), ).rejects.toThrow('before it accepted connections'); - await waitFor( - async () => (await deployedService('app-f', 'crasher')).status === 'backoff', - 3000, - ); + await waitFor(async () => (await deployedService(reserved.id)).status === 'backoff', 3000); // The first backoff step is the base case: 1s. Give it a comfortable // margin and confirm it actually restarts (the log shows the pinned // `[emulator] exited ... restarting in 1s` line), not just holds at // `backoff` forever. await waitFor(async () => { - const text = await fetch(`${daemonUrl}/apps/app-f/services/crasher/logs`).then((r) => - r.text(), - ); + const text = await fetch(`${daemonUrl}/instances/${reserved.id}/logs`).then((r) => r.text()); return text.includes('[emulator] exited') && text.includes('restarting in 1s'); }, 3000); }, 10_000); @@ -221,37 +196,26 @@ describe('crash supervision', () => { test('held after 5 consecutive fast crashes, and a redeploy clears held', async () => { const client = computeClient({ registryRoot }); const crashingDir = writeBootstrap(CRASHING_BOOTSTRAP); - const reserved = await client.ensureService('app-g', 'crasher'); + const reserved = await client.createInstance({ labels: {} }); await expect( - client.putDeployment('app-g', 'crasher', { - address: 'app-g.crasher', + client.putDeployment(reserved.id, { artifactDir: crashingDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }), ).rejects.toThrow('before it accepted connections'); - await waitFor( - async () => (await deployedService('app-g', 'crasher')).status === 'held', - 20_000, - 250, - ); + await waitFor(async () => (await deployedService(reserved.id)).status === 'held', 20_000, 250); const stableDir = writeBootstrap(SERVING_BOOTSTRAP); - await client.putDeployment('app-g', 'crasher', { - address: 'app-g.crasher', + await client.putDeployment(reserved.id, { artifactDir: stableDir, artifactHash: 'h2', env: baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv('recovered') }), - port: reserved.port, }); - await waitFor( - async () => (await deployedService('app-g', 'crasher')).status === 'running', - 5000, - ); + await waitFor(async () => (await deployedService(reserved.id)).status === 'running', 5000); const res = await waitForHttp(`http://127.0.0.1:${String(reserved.port)}`, 3000); expect(await res.text()).toBe('recovered'); }, 30_000); @@ -261,20 +225,15 @@ describe('stop truthfulness', () => { test('a child that ignores SIGTERM is only listed stopped once SIGKILL has actually landed', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(`process.on('SIGTERM', () => {});\n${SERVING_BOOTSTRAP}`); - const reserved = await client.ensureService('app-i', 'stubborn'); + const reserved = await client.createInstance({ labels: {} }); - await client.putDeployment('app-i', 'stubborn', { - address: 'app-i.stubborn', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv('stubborn') }), - port: reserved.port, }); - await waitFor( - async () => (await deployedService('app-i', 'stubborn')).status === 'running', - 5000, - ); - const pid = (await deployedService('app-i', 'stubborn')).pid; + await waitFor(async () => (await deployedService(reserved.id)).status === 'running', 5000); + const pid = (await deployedService(reserved.id)).pid; if (pid === undefined) throw new Error('expected the running service to report a pid'); // `running` flips as soon as the child is spawned (spec: spawning IS // the observable action), before the child has actually finished @@ -290,7 +249,7 @@ describe('stop truthfulness', () => { let sawStoppedBeforeDeath = false; const watcher = (async () => { for (;;) { - const svc = await deployedService('app-i', 'stubborn'); + const svc = await deployedService(reserved.id); if (svc.status === 'stopped') { if (isPidAlive(pid)) sawStoppedBeforeDeath = true; return; @@ -300,7 +259,7 @@ describe('stop truthfulness', () => { })(); const stopStart = Date.now(); - await client.stopApp('app-i'); + await client.stopInstances([reserved.id]); const stopElapsedMs = Date.now() - stopStart; await watcher; @@ -313,54 +272,41 @@ describe('stop truthfulness', () => { }, 15_000); }); -describe('session resume (POST /apps//start)', () => { +describe('session resume (POST /instances/start)', () => { test('stop then start brings children back up on the same ports with new pids', async () => { const client = computeClient({ registryRoot }); const oneDir = writeBootstrap(SERVING_BOOTSTRAP); const twoDir = writeBootstrap(SERVING_BOOTSTRAP); - const one = await client.ensureService('app-j', 'web'); - const two = await client.ensureService('app-j', 'worker'); + const one = await client.createInstance({ labels: {} }); + const two = await client.createInstance({ labels: {} }); - await client.putDeployment('app-j', 'web', { - address: 'app-j.web', + await client.putDeployment(one.id, { artifactDir: oneDir, artifactHash: 'h', env: baseEnv({ PORT: String(one.port), ...servingBootstrapEnv('one') }), - port: one.port, }); - await client.putDeployment('app-j', 'worker', { - address: 'app-j.worker', + await client.putDeployment(two.id, { artifactDir: twoDir, artifactHash: 'h', env: baseEnv({ PORT: String(two.port), ...servingBootstrapEnv('two') }), - port: two.port, }); - await waitFor(async () => (await deployedService('app-j', 'web')).status === 'running', 5000); - await waitFor( - async () => (await deployedService('app-j', 'worker')).status === 'running', - 5000, - ); + await waitFor(async () => (await deployedService(one.id)).status === 'running', 5000); + await waitFor(async () => (await deployedService(two.id)).status === 'running', 5000); const pidsBefore = { - web: (await deployedService('app-j', 'web')).pid, - worker: (await deployedService('app-j', 'worker')).pid, + web: (await deployedService(one.id)).pid, + worker: (await deployedService(two.id)).pid, }; - await client.stopApp('app-j'); - await waitFor(async () => (await deployedService('app-j', 'web')).status === 'stopped', 5000); - await waitFor( - async () => (await deployedService('app-j', 'worker')).status === 'stopped', - 5000, - ); + await client.stopInstances([one.id, two.id]); + await waitFor(async () => (await deployedService(one.id)).status === 'stopped', 5000); + await waitFor(async () => (await deployedService(two.id)).status === 'stopped', 5000); - await client.startApp('app-j'); - await waitFor(async () => (await deployedService('app-j', 'web')).status === 'running', 5000); - await waitFor( - async () => (await deployedService('app-j', 'worker')).status === 'running', - 5000, - ); + await client.startInstances([one.id, two.id]); + await waitFor(async () => (await deployedService(one.id)).status === 'running', 5000); + await waitFor(async () => (await deployedService(two.id)).status === 'running', 5000); - const webAfter = await deployedService('app-j', 'web'); - const workerAfter = await deployedService('app-j', 'worker'); + const webAfter = await deployedService(one.id); + const workerAfter = await deployedService(two.id); expect(webAfter.pid).not.toBe(pidsBefore.web); expect(workerAfter.pid).not.toBe(pidsBefore.worker); expect(webAfter.port).toBe(one.port); @@ -375,24 +321,22 @@ describe('session resume (POST /apps//start)', () => { test('a service with no stored deployment is skipped without error', async () => { const client = computeClient({ registryRoot }); const deployedDir = writeBootstrap(SERVING_BOOTSTRAP); - const deployed = await client.ensureService('app-k', 'web'); - await client.ensureService('app-k', 'never-deployed'); + const deployed = await client.createInstance({ labels: {} }); + const neverDeployedInstance = await client.createInstance({ labels: {} }); - await client.putDeployment('app-k', 'web', { - address: 'app-k.web', + await client.putDeployment(deployed.id, { artifactDir: deployedDir, artifactHash: 'h', env: baseEnv({ PORT: String(deployed.port), ...servingBootstrapEnv('deployed') }), - port: deployed.port, }); - await waitFor(async () => (await deployedService('app-k', 'web')).status === 'running', 5000); - await client.stopApp('app-k'); - await waitFor(async () => (await deployedService('app-k', 'web')).status === 'stopped', 5000); + await waitFor(async () => (await deployedService(deployed.id)).status === 'running', 5000); + await client.stopInstances([deployed.id, neverDeployedInstance.id]); + await waitFor(async () => (await deployedService(deployed.id)).status === 'stopped', 5000); - await client.startApp('app-k'); + await client.startInstances([deployed.id, neverDeployedInstance.id]); - await waitFor(async () => (await deployedService('app-k', 'web')).status === 'running', 5000); - const neverDeployed = await deployedService('app-k', 'never-deployed'); + await waitFor(async () => (await deployedService(deployed.id)).status === 'running', 5000); + const neverDeployed = await deployedService(neverDeployedInstance.id); expect(neverDeployed.status).toBe('stopped'); expect(neverDeployed.pid).toBeUndefined(); }, 15_000); @@ -400,24 +344,20 @@ describe('session resume (POST /apps//start)', () => { test('a held service resumes on start', async () => { const client = computeClient({ registryRoot }); const crashingDir = writeBootstrap(CRASHING_BOOTSTRAP); - const reserved = await client.ensureService('app-l', 'crasher'); + const reserved = await client.createInstance({ labels: {} }); await expect( - client.putDeployment('app-l', 'crasher', { - address: 'app-l.crasher', + client.putDeployment(reserved.id, { artifactDir: crashingDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }), ).rejects.toThrow('before it accepted connections'); - await waitFor( - async () => (await deployedService('app-l', 'crasher')).status === 'held', - 20_000, - 250, - ); + await waitFor(async () => (await deployedService(reserved.id)).status === 'held', 20_000, 250); - await expect(client.startApp('app-l')).rejects.toThrow('before it accepted connections'); + await expect(client.startInstances([reserved.id])).rejects.toThrow( + 'before it accepted connections', + ); // The crashing artifact is still what's stored, so the resumed child // crashes again shortly — but it takes 5 fresh fast exits to re-enter @@ -425,31 +365,25 @@ describe('session resume (POST /apps//start)', () => { // steps. Seeing anything other than `held` shortly after `/start` // proves the resume actually cleared it rather than no-op'ing on a // held service. - await waitFor( - async () => (await deployedService('app-l', 'crasher')).status !== 'held', - 2000, - 50, - ); + await waitFor(async () => (await deployedService(reserved.id)).status !== 'held', 2000, 50); }, 30_000); - test('start on an already-running app is a no-op — pids unchanged', async () => { + test('start on running instances is a no-op — pids unchanged', async () => { const client = computeClient({ registryRoot }); const dir = writeBootstrap(SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-m', 'web'); + const reserved = await client.createInstance({ labels: {} }); - await client.putDeployment('app-m', 'web', { - address: 'app-m.web', + await client.putDeployment(reserved.id, { artifactDir: dir, artifactHash: 'h', env: baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv('steady') }), - port: reserved.port, }); - await waitFor(async () => (await deployedService('app-m', 'web')).status === 'running', 5000); - const before = await deployedService('app-m', 'web'); + await waitFor(async () => (await deployedService(reserved.id)).status === 'running', 5000); + const before = await deployedService(reserved.id); - await client.startApp('app-m'); + await client.startInstances([reserved.id]); - const after = await deployedService('app-m', 'web'); + const after = await deployedService(reserved.id); expect(after.status).toBe('running'); expect(after.pid).toBe(before.pid); }, 10_000); @@ -459,23 +393,21 @@ describe('log follow', () => { test('follow yields the child output plus [emulator] supervision lines', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(CRASHING_BOOTSTRAP); - const reserved = await client.ensureService('app-h', 'crasher'); + const reserved = await client.createInstance({ labels: {} }); const chunks: string[] = []; const controller = new AbortController(); const following = (async () => { - for await (const chunk of client.followLogs('app-h', 'crasher', controller.signal)) { + for await (const chunk of client.followLogs(reserved.id, controller.signal)) { chunks.push(chunk); } })().catch(() => undefined); await expect( - client.putDeployment('app-h', 'crasher', { - address: 'app-h.crasher', + client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }), ).rejects.toThrow('before it accepted connections'); @@ -487,22 +419,20 @@ describe('log follow', () => { test('?tail=N replays only the last N history lines before going live', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(LOGGING_BOOTSTRAP); - const reserved = await client.ensureService('app-tail', 'logger'); - await client.putDeployment('app-tail', 'logger', { - address: 'app-tail.logger', + const reserved = await client.createInstance({ labels: {} }); + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 't', env: baseEnv({ PORT: String(reserved.port), LINES: '10' }), - port: reserved.port, }); // All ten lines are on disk before the follow attaches — so what the // follow returns is backlog, not live output racing the loop. await waitFor( async () => - ( - await fetch(`${daemonUrl}/apps/app-tail/services/logger/logs`).then((r) => r.text()) - ).includes('line-10\n'), + (await fetch(`${daemonUrl}/instances/${reserved.id}/logs`).then((r) => r.text())).includes( + 'line-10\n', + ), 5000, 100, ); @@ -510,7 +440,7 @@ describe('log follow', () => { const chunks: string[] = []; const controller = new AbortController(); await (async () => { - for await (const chunk of client.followLogs('app-tail', 'logger', controller.signal, { + for await (const chunk of client.followLogs(reserved.id, controller.signal, { tail: 3, })) { chunks.push(chunk); @@ -528,38 +458,28 @@ describe('log follow', () => { }); describe('log reset on a fresh start', () => { - const readLog = (app: string, id: string): Promise => - fetch(`${daemonUrl}/apps/${app}/services/${id}/logs`).then((r) => r.text()); + const readLog = (id: string): Promise => + fetch(`${daemonUrl}/instances/${id}/logs`).then((r) => r.text()); test('a new deployment starts with an empty log — the prior run is gone', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-reset', 'web'); + const reserved = await client.createInstance({ labels: {} }); const deploy = (hash: string, body: string): Promise => - client.putDeployment('app-reset', 'web', { - address: 'app-reset.web', + client.putDeployment(reserved.id, { artifactDir, artifactHash: hash, env: baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv(body) }), - port: reserved.port, }); await deploy('v1', 'one'); - await waitFor( - async () => (await readLog('app-reset', 'web')).includes('booted: one'), - 5000, - 100, - ); + await waitFor(async () => (await readLog(reserved.id)).includes('booted: one'), 5000, 100); // A changed deployment restarts the service — a fresh start. await deploy('v2', 'two'); - await waitFor( - async () => (await readLog('app-reset', 'web')).includes('booted: two'), - 5000, - 100, - ); + await waitFor(async () => (await readLog(reserved.id)).includes('booted: two'), 5000, 100); - const logText = await readLog('app-reset', 'web'); + const logText = await readLog(reserved.id); expect(logText).toContain('booted: two'); expect(logText).not.toContain('booted: one'); }, 15_000); @@ -567,27 +487,21 @@ describe('log reset on a fresh start', () => { test('a follower survives the clear and picks up the fresh log', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-reset-follow', 'web'); + const reserved = await client.createInstance({ labels: {} }); const deploy = (hash: string, body: string): Promise => - client.putDeployment('app-reset-follow', 'web', { - address: 'app-reset-follow.web', + client.putDeployment(reserved.id, { artifactDir, artifactHash: hash, env: baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv(body) }), - port: reserved.port, }); await deploy('v1', 'first'); - await waitFor( - async () => (await readLog('app-reset-follow', 'web')).includes('booted: first'), - 5000, - 100, - ); + await waitFor(async () => (await readLog(reserved.id)).includes('booted: first'), 5000, 100); const chunks: string[] = []; const controller = new AbortController(); const following = (async () => { - for await (const chunk of client.followLogs('app-reset-follow', 'web', controller.signal, { + for await (const chunk of client.followLogs(reserved.id, controller.signal, { tail: 50, })) { chunks.push(chunk); diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-instances.test.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-instances.test.ts new file mode 100644 index 000000000..3cf63cf98 --- /dev/null +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-instances.test.ts @@ -0,0 +1,357 @@ +/** + * Compute instances: ids the emulator generates, labels it stores without + * reading, ports held from creation until delete, and isolation between + * instances whose labels are identical. + */ +import { afterEach, beforeEach, describe, expect, test } from 'bun:test'; +import * as fs from 'node:fs'; +import * as net from 'node:net'; +import * as path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import getPort, { portNumbers } from 'get-port'; +import { type ComputeClient, computeClient } from '../client.ts'; +import { daemonStateDir, ensureDaemon, stopDaemon } from '../daemon.ts'; +import { + ensureFreshDaemon, + entryFor, + SERVING_BOOTSTRAP, + servingBootstrapEnv, + skipContendedServicePorts, + tempDir, + waitForHttp, + writeBootstrap, +} from './helpers.ts'; + +let registryRoot: string; +let client: ComputeClient; + +beforeEach(async () => { + registryRoot = tempDir('compute-instances-registry'); + await ensureFreshDaemon('compute', registryRoot); + client = computeClient({ registryRoot }); + await skipContendedServicePorts(client); +}); + +afterEach(async () => { + await stopDaemon('compute', { registryRoot }).catch(() => undefined); +}); + +function baseEnv(extra: Record = {}): Record { + const path = process.env['PATH']; + if (!path) throw new Error('PATH is not set in this shell'); + return { PATH: path, ...extra }; +} + +async function deployServing(id: string, port: number, body: string): Promise { + await client.putDeployment(id, { + artifactDir: writeBootstrap(SERVING_BOOTSTRAP), + artifactHash: body, + env: baseEnv({ PORT: String(port), ...servingBootstrapEnv(body) }), + }); +} + +async function statusOf(id: string): Promise { + return (await client.getInstance(id))?.status; +} + +describe('creating and listing instances', () => { + test('the emulator generates distinct ids and returns the labels verbatim', async () => { + const labels = { 'composer.app': 'My App', 'free form': 'any value: /with/slashes' }; + const one = await client.createInstance({ labels }); + const two = await client.createInstance({ labels }); + + expect(one.id).toMatch(/^svc_[0-9a-f]{16}$/); + expect(two.id).not.toBe(one.id); + expect(one.labels).toEqual(labels); + expect(one.status).toBe('stopped'); + expect(one.url).toBe(`http://localhost:${String(one.port)}`); + + const listed = await client.listInstances(); + expect(listed.find((i) => i.id === one.id)?.labels).toEqual(labels); + expect(listed.find((i) => i.id === two.id)?.labels).toEqual(labels); + }); + + test('getting an unknown instance returns nothing', async () => { + expect(await client.getInstance('svc_0000000000000000')).toBeUndefined(); + }); + + test('a start naming an unknown instance fails, names it, and starts nothing', async () => { + const known = await client.createInstance({ labels: {} }); + await deployServing(known.id, known.port, 'known'); + await client.stopInstances([known.id]); + + await expect(client.startInstances([known.id, 'svc_0000000000000000'])).rejects.toThrow( + '"svc_0000000000000000"', + ); + expect(await statusOf(known.id)).toBe('stopped'); + }, 15_000); + + test('a stop skips an unknown instance and stops the others', async () => { + const known = await client.createInstance({ labels: {} }); + await deployServing(known.id, known.port, 'known'); + + await client.stopInstances([known.id, 'svc_0000000000000000']); + + expect(await statusOf(known.id)).toBe('stopped'); + }, 15_000); + + test('a malformed instance id is rejected with 400', async () => { + const res = await fetch(`${client.baseUrl}/instances/web`); + expect(res.status).toBe(400); + expect(await res.text()).toContain('"web"'); + }); + + test('labels that are not all strings are rejected with 400', async () => { + const res = await fetch(`${client.baseUrl}/instances`, { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify({ labels: { count: 1 } }), + }); + expect(res.status).toBe(400); + }); +}); + +describe('ports', () => { + test('two instances get distinct ports, both at or above 3000', async () => { + const one = await client.createInstance({ labels: {} }); + const two = await client.createInstance({ labels: {} }); + + expect(one.port).not.toBe(two.port); + expect(one.port).toBeGreaterThanOrEqual(3000); + expect(two.port).toBeGreaterThanOrEqual(3000); + }); + + test('a requested port that is free is given', async () => { + const requested = await getPort({ port: portNumbers(20_000, 21_000) }); + + const created = await client.createInstance({ labels: {}, port: requested }); + + expect(created.port).toBe(requested); + }); + + test('a requested port another instance holds gives the lowest free port instead', async () => { + const holder = await client.createInstance({ labels: {} }); + + const created = await client.createInstance({ labels: {}, port: holder.port }); + await client.deleteInstance(created.id); + const unrequested = await client.createInstance({ labels: {} }); + + expect(created.port).not.toBe(holder.port); + expect(created.port).toBe(unrequested.port); + }); + + test('a requested port a process outside the emulator has bound gives another port', async () => { + const requested = await getPort({ port: portNumbers(21_000, 22_000) }); + const squatter = net.createServer(); + await new Promise((resolve) => squatter.listen(requested, resolve)); + try { + const created = await client.createInstance({ labels: {}, port: requested }); + expect(created.port).not.toBe(requested); + expect(created.port).toBeGreaterThanOrEqual(3000); + } finally { + squatter.close(); + } + }); + + test('a stopped instance keeps its port, and no new instance is given it', async () => { + const one = await client.createInstance({ labels: {} }); + await deployServing(one.id, one.port, 'one'); + await client.stopInstances([one.id]); + + const created = await client.createInstance({ labels: {}, port: one.port }); + + expect((await client.getInstance(one.id))?.port).toBe(one.port); + expect(created.port).not.toBe(one.port); + }, 15_000); + + test("a deleted instance's port can be requested again", async () => { + const one = await client.createInstance({ labels: {} }); + await client.deleteInstance(one.id); + + const again = await client.createInstance({ labels: {}, port: one.port }); + + expect(again.port).toBe(one.port); + expect(again.id).not.toBe(one.id); + }); +}); + +describe('isolation between instances with the same labels', () => { + const labels = { 'composer.app': 'same-app', 'composer.address': 'web' }; + + test('both serve on their own ports', async () => { + const one = await client.createInstance({ labels }); + const two = await client.createInstance({ labels }); + + await deployServing(one.id, one.port, 'from one'); + await deployServing(two.id, two.port, 'from two'); + + expect(await (await waitForHttp(`http://127.0.0.1:${String(one.port)}`, 3000)).text()).toBe( + 'from one', + ); + expect(await (await waitForHttp(`http://127.0.0.1:${String(two.port)}`, 3000)).text()).toBe( + 'from two', + ); + }, 15_000); + + test('stopping one leaves the other running', async () => { + const one = await client.createInstance({ labels }); + const two = await client.createInstance({ labels }); + await deployServing(one.id, one.port, 'one'); + await deployServing(two.id, two.port, 'two'); + + await client.stopInstances([one.id]); + + expect(await statusOf(one.id)).toBe('stopped'); + expect(await statusOf(two.id)).toBe('running'); + }, 15_000); + + test('deleting one leaves the other', async () => { + const one = await client.createInstance({ labels }); + const two = await client.createInstance({ labels }); + + await client.deleteInstance(one.id); + + expect(await client.getInstance(one.id)).toBeUndefined(); + expect((await client.getInstance(two.id))?.labels).toEqual(labels); + }); +}); + +describe('persisted state', () => { + test('instances, their labels and their ports survive an emulator restart', async () => { + const created = await client.createInstance({ labels: { note: 'kept' } }); + + await stopDaemon('compute', { registryRoot }); + await ensureDaemon('compute', entryFor('compute'), { registryRoot }); + + const reloaded = await computeClient({ registryRoot }).getInstance(created.id); + expect(reloaded?.labels).toEqual({ note: 'kept' }); + expect(reloaded?.port).toBe(created.port); + }); + + test('state written by an earlier version, keyed by app, is discarded on load', async () => { + await stopDaemon('compute', { registryRoot }); + const stateDir = daemonStateDir(registryRoot, 'compute'); + fs.rmSync(path.join(stateDir, 'instances.json'), { force: true }); + const legacyLogDir = path.join(stateDir, 'logs', 'legacy-app'); + fs.mkdirSync(legacyLogDir, { recursive: true }); + fs.writeFileSync(path.join(legacyLogDir, 'web.log'), 'old\n'); + fs.writeFileSync( + path.join(stateDir, 'apps.json'), + JSON.stringify({ + 'legacy-app': { + services: { + web: { id: 'web', address: 'web', port: 3000, status: 'stopped', logPath: '/x' }, + }, + }, + }), + ); + + await ensureDaemon('compute', entryFor('compute'), { registryRoot }); + + expect(await computeClient({ registryRoot }).listInstances()).toEqual([]); + expect(fs.existsSync(path.join(stateDir, 'apps.json'))).toBe(false); + expect(fs.existsSync(legacyLogDir)).toBe(false); + }); +}); + +/** + * The emulator never interprets Composer's data. An app name, a node address + * or a project directory reaches it only as label values and inside the + * opaque deployment payload (the artifact directory and the env), which it + * stores verbatim. These checks fail when the emulator's code names one of + * those concepts, or when one of them shows up anywhere else: a route, a + * state key, a file name or a field the emulator generates. + */ +describe('the emulator never interprets Composer data', () => { + const app = 'zzappmarker'; + const address = 'zzaddressmarker'; + const directory = path.join(tempDir('marker-project'), 'zzdirectorymarker'); + const markers = [app, address, 'zzdirectorymarker']; + const labels = { + 'composer.directory': directory, + 'composer.app': app, + 'composer.address': address, + }; + + /** A deployment shaped like Composer's: its artifact unpacked under the project's dev directory, env keys named after the address. */ + function composerShapedDeployment(port: number) { + const artifactDir = path.join(directory, '.prisma-composer', 'dev', 'artifacts', 'abc123'); + fs.mkdirSync(artifactDir, { recursive: true }); + fs.writeFileSync(path.join(artifactDir, 'bootstrap.js'), SERVING_BOOTSTRAP); + const env = baseEnv({ + PORT: String(port), + [`COMPOSER_${address.toUpperCase()}_PORT`]: String(port), + [`COMPOSER_${address.toUpperCase()}_INPUT`]: '{}', + COMPOSER_GATEWAY_QUOTES_URL: 'http://localhost:3001', + ...servingBootstrapEnv('marker test'), + }); + return { artifactDir, artifactHash: 'abc123', env }; + } + + function without(value: unknown, keys: readonly string[]): unknown { + if (Array.isArray(value)) return value.map((item) => without(item, keys)); + if (typeof value !== 'object' || value === null) return value; + return Object.fromEntries( + Object.entries(value) + .filter(([key]) => !keys.includes(key)) + .map(([key, field]) => [key, without(field, keys)]), + ); + } + + function keysOf(value: unknown): string[] { + if (Array.isArray(value)) return value.flatMap(keysOf); + if (typeof value !== 'object' || value === null) return []; + return Object.entries(value).flatMap(([key, field]) => [key, ...keysOf(field)]); + } + + function filesUnder(dir: string): string[] { + return fs + .readdirSync(dir, { withFileTypes: true, recursive: true }) + .map((entry) => path.join(entry.parentPath, entry.name)); + } + + function mentions(text: string, marker: string): boolean { + return text.toLowerCase().includes(marker); + } + + test('labels and the deployment payload are stored verbatim, and nothing else carries their values', async () => { + const created = await client.createInstance({ labels }); + const deployment = composerShapedDeployment(created.port); + await client.putDeployment(created.id, deployment); + await client.stopInstances([created.id]); + + const stateDir = daemonStateDir(registryRoot, 'compute'); + const state: unknown = JSON.parse( + fs.readFileSync(path.join(stateDir, 'instances.json'), 'utf8'), + ); + const record: unknown = Reflect.get(Reflect.get(state as object, 'instances'), created.id); + expect(record).toMatchObject({ + labels, + artifactDir: deployment.artifactDir, + artifactHash: deployment.artifactHash, + env: deployment.env, + }); + + const opaque = ['labels', 'artifactDir', 'env']; + const stateKeys = keysOf(without(state, ['labels', 'env'])).join('\n'); + const listed: unknown = await (await fetch(`${client.baseUrl}/instances`)).json(); + const own: unknown = await (await fetch(`${client.baseUrl}/instances/${created.id}`)).json(); + for (const marker of markers) { + expect(mentions(JSON.stringify(without(state, opaque)), marker)).toBe(false); + expect(mentions(stateKeys, marker)).toBe(false); + expect(mentions(JSON.stringify(without(listed, ['labels'])), marker)).toBe(false); + expect(mentions(JSON.stringify(without(own, ['labels'])), marker)).toBe(false); + for (const file of filesUnder(stateDir)) expect(mentions(file, marker)).toBe(false); + } + }, 15_000); + + test("the emulator's code names no app, address, directory or Composer concept", () => { + const source = fs.readFileSync( + fileURLToPath(new URL('../compute-main.ts', import.meta.url)), + 'utf8', + ); + const code = source.replace(/\/\*[\s\S]*?\*\//g, '').replace(/\/\/.*$/gm, ''); + + expect(code).not.toMatch(/\b(apps?|appName|address(es)?|directory|composer)\b/i); + }); +}); diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-multi-app.test.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-multi-app.test.ts deleted file mode 100644 index 871f5510f..000000000 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-multi-app.test.ts +++ /dev/null @@ -1,157 +0,0 @@ -/** - * Multi-app isolation on the Compute emulator (local-dev spec § 2): two - * different apps with the SAME service name must get distinct, non-colliding - * ports, and each other's admin operations must not cross-affect. Also - * covers the shared path-segment hygiene rule. - */ -import { afterEach, beforeEach, describe, expect, test } from 'bun:test'; -import { computeClient } from '../client.ts'; -import { ensureDaemon, stopDaemon } from '../daemon.ts'; -import { - ensureFreshDaemon, - entryFor, - SERVING_BOOTSTRAP, - servingBootstrapEnv, - skipContendedServicePorts, - tempDir, - waitFor, - waitForHttp, - writeBootstrap, -} from './helpers.ts'; - -let registryRoot: string; - -beforeEach(async () => { - registryRoot = tempDir('compute-multi-app-registry'); - await ensureFreshDaemon('compute', registryRoot); - // The "serving" tests actually bind their reserved port (Bun.serve) — - // steer past any low service port a process outside this test's control - // already holds, the same class of shared-machine contention the - // daemon-port retry feature addresses one layer down. - await skipContendedServicePorts(computeClient({ registryRoot })); -}); - -afterEach(async () => { - await stopDaemon('compute', { registryRoot }).catch(() => undefined); -}); - -function baseEnv(extra: Record = {}): Record { - const path = process.env['PATH']; - if (!path) throw new Error('PATH is not set in this shell'); - return { PATH: path, ...extra }; -} - -test('two apps with the same service id get distinct ports, both ≥ 3000', async () => { - const client = computeClient({ registryRoot }); - const one = await client.ensureService('tenant-one', 'web'); - const two = await client.ensureService('tenant-two', 'web'); - - expect(one.port).not.toBe(two.port); - expect(one.port).toBeGreaterThanOrEqual(3000); - expect(two.port).toBeGreaterThanOrEqual(3000); -}); - -test('deploying and serving the same service id under two apps never collides', async () => { - const client = computeClient({ registryRoot }); - const oneDir = writeBootstrap(SERVING_BOOTSTRAP); - const twoDir = writeBootstrap(SERVING_BOOTSTRAP); - - const one = await client.ensureService('tenant-one', 'web'); - const two = await client.ensureService('tenant-two', 'web'); - - await client.putDeployment('tenant-one', 'web', { - address: 'tenant-one.web', - artifactDir: oneDir, - artifactHash: 'h1', - env: baseEnv({ PORT: String(one.port), ...servingBootstrapEnv('from tenant one') }), - port: one.port, - }); - await client.putDeployment('tenant-two', 'web', { - address: 'tenant-two.web', - artifactDir: twoDir, - artifactHash: 'h1', - env: baseEnv({ PORT: String(two.port), ...servingBootstrapEnv('from tenant two') }), - port: two.port, - }); - - const resOne = await waitForHttp(`http://127.0.0.1:${String(one.port)}`, 3000); - const resTwo = await waitForHttp(`http://127.0.0.1:${String(two.port)}`, 3000); - expect(await resOne.text()).toBe('from tenant one'); - expect(await resTwo.text()).toBe('from tenant two'); -}); - -test("stopping one app's services leaves the other app's services running", async () => { - const client = computeClient({ registryRoot }); - const oneDir = writeBootstrap(SERVING_BOOTSTRAP); - const twoDir = writeBootstrap(SERVING_BOOTSTRAP); - const one = await client.ensureService('tenant-one', 'web'); - const two = await client.ensureService('tenant-two', 'web'); - - await client.putDeployment('tenant-one', 'web', { - address: 'tenant-one.web', - artifactDir: oneDir, - artifactHash: 'h', - env: baseEnv({ PORT: String(one.port), ...servingBootstrapEnv('one') }), - port: one.port, - }); - await client.putDeployment('tenant-two', 'web', { - address: 'tenant-two.web', - artifactDir: twoDir, - artifactHash: 'h', - env: baseEnv({ PORT: String(two.port), ...servingBootstrapEnv('two') }), - port: two.port, - }); - await waitFor(async () => { - const listOne = await client.listServices('tenant-one'); - const listTwo = await client.listServices('tenant-two'); - return listOne[0]?.status === 'running' && listTwo[0]?.status === 'running'; - }, 5000); - - await client.stopApp('tenant-one'); - - await waitFor( - async () => (await client.listServices('tenant-one'))[0]?.status === 'stopped', - 5000, - ); - const twoList = await client.listServices('tenant-two'); - expect(twoList[0]?.status).toBe('running'); -}); - -test('deleting one app leaves the other app registered', async () => { - const client = computeClient({ registryRoot }); - await client.ensureService('tenant-one', 'web'); - await client.ensureService('tenant-two', 'web'); - - await client.deleteApp('tenant-one'); - - expect(await client.listServices('tenant-one')).toEqual([]); - const twoList = await client.listServices('tenant-two'); - expect(twoList).toHaveLength(1); -}); - -test('an app deleted and reserved again gets back the ports it freed', async () => { - const client = computeClient({ registryRoot }); - const quotes = await client.ensureService('tenant-one', 'quotes'); - const gateway = await client.ensureService('tenant-one', 'gateway'); - - await client.deleteApp('tenant-one'); - - expect((await client.ensureService('tenant-one', 'quotes')).port).toBe(quotes.port); - expect((await client.ensureService('tenant-one', 'gateway')).port).toBe(gateway.port); -}); - -describe('path-segment hygiene', () => { - test('an app name with an uppercase letter is rejected with 400', async () => { - const { url } = await ensureDaemon('compute', entryFor('compute'), { registryRoot }); - const res = await fetch(`${url}/apps/BadApp/services/web`, { method: 'PUT' }); - expect(res.status).toBe(400); - expect(await res.text()).toContain('BadApp'); - }); - - test('a service id over 63 characters is rejected with 400', async () => { - const { url } = await ensureDaemon('compute', entryFor('compute'), { registryRoot }); - const longId = 'a'.repeat(64); - const res = await fetch(`${url}/apps/tenant-one/services/${longId}`, { method: 'PUT' }); - expect(res.status).toBe(400); - }); -}); diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts index a9f7e8325..a3b22929b 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts @@ -11,7 +11,7 @@ import * as net from 'node:net'; import * as path from 'node:path'; import { fileURLToPath } from 'node:url'; import getPort, { portNumbers } from 'get-port'; -import { type ComputeClient, computeClient } from '../client.ts'; +import { type ComputeClient, computeClient, type InstanceInfo } from '../client.ts'; import { ensureDaemon, isPidAlive, SERVICE_READY_TIMEOUT_ENV, stopDaemon } from '../daemon.ts'; import { CRASH_ONCE_THEN_SILENT_BOOTSTRAP, @@ -60,33 +60,30 @@ async function startDaemon(entry?: string): Promise { return computeClient({ registryRoot }); } -const OWN_PORT_MIN = 3400; -const OWN_PORT_MAX = 3599; - /** - * Reserves the service and picks its port, which the test then deploys it - * on. Other test files run their own Compute emulators at the same time and - * give ports from 3000, and several tests here assert that a stopped - * service's port refuses connections: a port another file's service holds - * would make that fail. So each service here takes a currently free port - * from a range only this file uses. + * This file's ports. Other test files run their own Compute emulators at the + * same time and give ports from 3000, and several tests here assert that a + * stopped instance's port refuses connections: a port another file's service + * holds would make that fail. So each instance here asks for a currently free + * port from a range only this file uses. */ -async function ensureServiceOnOwnPort( - client: ComputeClient, - app: string, - id: string, -): Promise<{ port: number }> { - await client.ensureService(app, id); - const port = await getPort({ port: portNumbers(OWN_PORT_MIN, OWN_PORT_MAX) }); - expect(port).toBeGreaterThanOrEqual(OWN_PORT_MIN); - expect(port).toBeLessThanOrEqual(OWN_PORT_MAX); - return { port }; +const OWN_PORTS = portNumbers(3400, 3599); + +async function createInstanceOnOwnPort(client: ComputeClient): Promise { + const port = await getPort({ port: OWN_PORTS }); + const instance = await client.createInstance({ labels: {}, port }); + expect(instance.port).toBe(port); + return instance; } -async function statusOf(client: ComputeClient, app: string, id: string): Promise { - const svc = (await client.listServices(app)).find((s) => s.id === id); - if (!svc) throw new Error(`service "${id}" not found in app "${app}"`); - return svc.status; +async function instanceOf(client: ComputeClient, id: string): Promise { + const instance = await client.getInstance(id); + if (!instance) throw new Error(`instance "${id}" not found`); + return instance; +} + +async function statusOf(client: ComputeClient, id: string): Promise { + return (await instanceOf(client, id)).status; } describe('with the default readiness timeout', () => { @@ -98,10 +95,9 @@ describe('with the default readiness timeout', () => { test('a deployment returns only once a slow-booting service accepts connections', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await ensureServiceOnOwnPort(client, 'app-slow', 'web'); + const reserved = await createInstanceOnOwnPort(client); - const deployed = client.putDeployment('app-slow', 'web', { - address: 'slow.web', + const deployed = client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv({ @@ -109,24 +105,22 @@ describe('with the default readiness timeout', () => { LISTEN_DELAY_MS: String(LISTEN_DELAY_MS), ...servingBootstrapEnv('slow but ready'), }), - port: reserved.port, }); await Bun.sleep(LISTEN_DELAY_MS / 2); - expect(await statusOf(client, 'app-slow', 'web')).not.toBe('running'); + expect(await statusOf(client, reserved.id)).not.toBe('running'); await deployed; expect(await acceptsConnection(reserved.port)).toBe(true); - expect(await statusOf(client, 'app-slow', 'web')).toBe('running'); + expect(await statusOf(client, reserved.id)).toBe('running'); const res = await fetch(`http://localhost:${String(reserved.port)}`); expect(await res.text()).toBe('slow but ready'); }, 15_000); test('a session-resume start returns only once a slow-booting service accepts connections', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await ensureServiceOnOwnPort(client, 'app-resume', 'web'); - await client.putDeployment('app-resume', 'web', { - address: 'resume.web', + const reserved = await createInstanceOnOwnPort(client); + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv({ @@ -134,50 +128,47 @@ describe('with the default readiness timeout', () => { LISTEN_DELAY_MS: String(LISTEN_DELAY_MS), ...servingBootstrapEnv('resumed'), }), - port: reserved.port, }); - await client.stopApp('app-resume'); + await client.stopInstances([reserved.id]); expect(await acceptsConnection(reserved.port)).toBe(false); - await client.startApp('app-resume'); + await client.startInstances([reserved.id]); expect(await acceptsConnection(reserved.port)).toBe(true); - expect(await statusOf(client, 'app-resume', 'web')).toBe('running'); + expect(await statusOf(client, reserved.id)).toBe('running'); }, 20_000); test('a service that exits before it listens fails the deployment, naming the service and its log', async () => { const artifactDir = writeBootstrap(CRASHING_BOOTSTRAP); - const reserved = await ensureServiceOnOwnPort(client, 'app-crash', 'web'); + const reserved = await createInstanceOnOwnPort(client); - const deployed = client.putDeployment('app-crash', 'web', { - address: 'crash.web', + const deployed = client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }); await expect(deployed).rejects.toThrow( - /service "crash\.web" exited \(code 1\) before it accepted connections on its port \d+\. Its log: .*web\.log/, + new RegExp( + `instance "${reserved.id}" exited \\(code 1\\) before it accepted connections on its port ${String(reserved.port)}\\. Its log: .*${reserved.id}\\.log`, + ), ); }, 15_000); test('a session-resume start of a service that exits before it listens fails, naming the service', async () => { const artifactDir = writeBootstrap(CRASHING_BOOTSTRAP); - const reserved = await ensureServiceOnOwnPort(client, 'app-crash-resume', 'web'); + const reserved = await createInstanceOnOwnPort(client); await client - .putDeployment('app-crash-resume', 'web', { - address: 'crash-resume.web', + .putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }) .catch(() => undefined); - await client.stopApp('app-crash-resume'); + await client.stopInstances([reserved.id]); - await expect(client.startApp('app-crash-resume')).rejects.toThrow( - /service "crash-resume\.web" exited \(code 1\) before it accepted connections/, + await expect(client.startInstances([reserved.id])).rejects.toThrow( + `instance "${reserved.id}" exited (code 1) before it accepted connections`, ); }, 15_000); }); @@ -191,24 +182,22 @@ describe('a service that does not listen on its port', () => { test('is running once spawned, without waiting for its port', async () => { const artifactDir = writeBootstrap(NEVER_LISTENING_BOOTSTRAP); - const reserved = await ensureServiceOnOwnPort(client, 'app-clock', 'scheduler'); + const reserved = await createInstanceOnOwnPort(client); const started = Date.now(); - await client.putDeployment('app-clock', 'scheduler', { - address: 'cron.scheduler', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', - env: baseEnv(), - port: reserved.port, + env: baseEnv({ PORT: String(reserved.port) }), listens: false, }); expect(Date.now() - started).toBeLessThan(5000); - expect(await statusOf(client, 'app-clock', 'scheduler')).toBe('running'); + expect(await statusOf(client, reserved.id)).toBe('running'); - await client.stopApp('app-clock'); - await client.startApp('app-clock'); - expect(await statusOf(client, 'app-clock', 'scheduler')).toBe('running'); + await client.stopInstances([reserved.id]); + await client.startInstances([reserved.id]); + expect(await statusOf(client, reserved.id)).toBe('running'); }, 20_000); }); @@ -221,7 +210,6 @@ describe('a start that is replaced, stopped or joined', () => { function delayedDeployment(port: number, hash: string, body: string, artifactDir: string) { return { - address: 'slow.web', artifactDir, artifactHash: hash, env: baseEnv({ @@ -229,52 +217,46 @@ describe('a start that is replaced, stopped or joined', () => { LISTEN_DELAY_MS: String(LISTEN_DELAY_MS), ...servingBootstrapEnv(body), }), - port, }; } test('a redeploy of a running service is not running until the new process listens', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await ensureServiceOnOwnPort(client, 'app-redeploy', 'web'); + const reserved = await createInstanceOnOwnPort(client); await client.putDeployment( - 'app-redeploy', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v1', 'first', artifactDir), ); const redeployed = client.putDeployment( - 'app-redeploy', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v2', 'second', artifactDir), ); await Bun.sleep(LISTEN_DELAY_MS / 2); - expect(await statusOf(client, 'app-redeploy', 'web')).not.toBe('running'); + expect(await statusOf(client, reserved.id)).not.toBe('running'); await redeployed; - expect(await statusOf(client, 'app-redeploy', 'web')).toBe('running'); + expect(await statusOf(client, reserved.id)).toBe('running'); const res = await fetch(`http://localhost:${String(reserved.port)}`); expect(await res.text()).toBe('second'); }, 20_000); test('a repeated identical deployment during a redeploy waits for the new process', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await ensureServiceOnOwnPort(client, 'app-repeat', 'web'); + const reserved = await createInstanceOnOwnPort(client); await client.putDeployment( - 'app-repeat', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v1', 'first', artifactDir), ); const redeployed = client.putDeployment( - 'app-repeat', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v2', 'second', artifactDir), ); await Bun.sleep(300); await client.putDeployment( - 'app-repeat', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v2', 'second', artifactDir), ); @@ -286,20 +268,18 @@ describe('a start that is replaced, stopped or joined', () => { test('a session-resume start during a redeploy waits for the new process', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await ensureServiceOnOwnPort(client, 'app-resume-redeploy', 'web'); + const reserved = await createInstanceOnOwnPort(client); await client.putDeployment( - 'app-resume-redeploy', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v1', 'first', artifactDir), ); const redeployed = client.putDeployment( - 'app-resume-redeploy', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v2', 'second', artifactDir), ); await Bun.sleep(300); - await client.startApp('app-resume-redeploy'); + await client.startInstances([reserved.id]); const res = await fetch(`http://localhost:${String(reserved.port)}`); expect(await res.text()).toBe('second'); @@ -308,36 +288,33 @@ describe('a start that is replaced, stopped or joined', () => { test('two concurrent session-resume starts share one process', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await ensureServiceOnOwnPort(client, 'app-twice', 'web'); + const reserved = await createInstanceOnOwnPort(client); await client.putDeployment( - 'app-twice', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v1', 'once', artifactDir), ); - await client.stopApp('app-twice'); + await client.stopInstances([reserved.id]); - await Promise.all([client.startApp('app-twice'), client.startApp('app-twice')]); + await Promise.all([client.startInstances([reserved.id]), client.startInstances([reserved.id])]); - const log = await (await fetch(`${client.baseUrl}/apps/app-twice/services/web/logs`)).text(); + const log = await (await fetch(`${client.baseUrl}/instances/${reserved.id}/logs`)).text(); expect(log.match(/booted pid/g)?.length).toBe(1); - const [svc] = await client.listServices('app-twice'); + const svc = await instanceOf(client, reserved.id); expect(svc?.status).toBe('running'); expect(log).toContain(`booted pid ${String(svc?.pid)}`); }, 20_000); test('a deployment replaced during its start waits for the replacement instead of failing', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await ensureServiceOnOwnPort(client, 'app-replaced', 'web'); + const reserved = await createInstanceOnOwnPort(client); const first = client.putDeployment( - 'app-replaced', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v1', 'first', artifactDir), ); await Bun.sleep(300); const second = client.putDeployment( - 'app-replaced', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v2', 'second', artifactDir), ); @@ -350,33 +327,27 @@ describe('a start that is replaced, stopped or joined', () => { const artifactDir = writeBootstrap( `process.on('SIGTERM', () => {});\n${DELAYED_SERVING_BOOTSTRAP}`, ); - const reserved = await ensureServiceOnOwnPort(client, 'app-stop-start', 'web'); + const reserved = await createInstanceOnOwnPort(client); const deployed = client - .putDeployment( - 'app-stop-start', - 'web', - delayedDeployment(reserved.port, 'v1', 'first', artifactDir), - ) + .putDeployment(reserved.id, delayedDeployment(reserved.port, 'v1', 'first', artifactDir)) .catch(() => undefined); await Bun.sleep(300); - const [first] = await client.listServices('app-stop-start'); + const first = await instanceOf(client, reserved.id); const firstPid = first?.pid; expect(firstPid).toBeDefined(); - const stopped = client.stopApp('app-stop-start'); + const stopped = client.stopInstances([reserved.id]); await Bun.sleep(200); - await client.startApp('app-stop-start'); + await client.startInstances([reserved.id]); await stopped; await deployed; - const [svc] = await client.listServices('app-stop-start'); + const svc = await instanceOf(client, reserved.id); expect(svc?.status).toBe('running'); expect(svc?.pid).not.toBe(firstPid); expect(isPidAlive(firstPid ?? 0)).toBe(false); - const log = await ( - await fetch(`${client.baseUrl}/apps/app-stop-start/services/web/logs`) - ).text(); + const log = await (await fetch(`${client.baseUrl}/instances/${reserved.id}/logs`)).text(); expect(log).not.toContain('[emulator] exited'); expect(log.match(/booted pid/g)?.length).toBe(1); @@ -388,55 +359,53 @@ describe('a start that is replaced, stopped or joined', () => { function quickDeployment(port: number, artifactDir: string) { return { - address: 'quick.web', artifactDir, artifactHash: 'v1', env: baseEnv({ PORT: String(port), LISTEN_DELAY_MS: '300', ...servingBootstrapEnv('x') }), - port, }; } test('stop, start, stop: a stop during a waiting start cancels it', async () => { const artifactDir = writeBootstrap(IGNORES_SIGTERM); - const reserved = await ensureServiceOnOwnPort(client, 'app-sss', 'web'); - await client.putDeployment('app-sss', 'web', quickDeployment(reserved.port, artifactDir)); + const reserved = await createInstanceOnOwnPort(client); + await client.putDeployment(reserved.id, quickDeployment(reserved.port, artifactDir)); - const firstStop = client.stopApp('app-sss'); + const firstStop = client.stopInstances([reserved.id]); await Bun.sleep(200); - const start = client.startApp('app-sss').catch((e: unknown) => String(e)); + const start = client.startInstances([reserved.id]).catch((e: unknown) => String(e)); await Bun.sleep(200); - const secondStop = client.stopApp('app-sss'); + const secondStop = client.stopInstances([reserved.id]); await Promise.all([firstStop, start, secondStop]); - expect(await statusOf(client, 'app-sss', 'web')).toBe('stopped'); + expect(await statusOf(client, reserved.id)).toBe('stopped'); expect(await start).toContain('was stopped before it accepted connections'); expect(await acceptsConnection(reserved.port)).toBe(false); }, 30_000); test('an identical deployment during a stop waits for the stop and starts the service again', async () => { const artifactDir = writeBootstrap(IGNORES_SIGTERM); - const reserved = await ensureServiceOnOwnPort(client, 'app-same-put', 'web'); - await client.putDeployment('app-same-put', 'web', quickDeployment(reserved.port, artifactDir)); + const reserved = await createInstanceOnOwnPort(client); + await client.putDeployment(reserved.id, quickDeployment(reserved.port, artifactDir)); - const stopped = client.stopApp('app-same-put'); + const stopped = client.stopInstances([reserved.id]); await Bun.sleep(200); - await client.putDeployment('app-same-put', 'web', quickDeployment(reserved.port, artifactDir)); + await client.putDeployment(reserved.id, quickDeployment(reserved.port, artifactDir)); await stopped; - expect(await statusOf(client, 'app-same-put', 'web')).toBe('running'); + expect(await statusOf(client, reserved.id)).toBe('running'); expect(await acceptsConnection(reserved.port)).toBe(true); - const [svc] = await client.listServices('app-same-put'); + const svc = await instanceOf(client, reserved.id); if (svc?.pid !== undefined) process.kill(svc.pid, 'SIGKILL'); }, 30_000); - test('a start during a delete of the app leaves no process behind', async () => { + test('a start during an instance delete leaves no process behind', async () => { const artifactDir = writeBootstrap(IGNORES_SIGTERM); - const reserved = await ensureServiceOnOwnPort(client, 'app-del-start', 'web'); - await client.putDeployment('app-del-start', 'web', quickDeployment(reserved.port, artifactDir)); + const reserved = await createInstanceOnOwnPort(client); + await client.putDeployment(reserved.id, quickDeployment(reserved.port, artifactDir)); - const deleted = client.deleteApp('app-del-start'); + const deleted = client.deleteInstance(reserved.id); await Bun.sleep(200); - const start = await client.startApp('app-del-start').then( + const start = await client.startInstances([reserved.id]).then( () => 'ok', (e: unknown) => String(e), ); @@ -455,25 +424,24 @@ describe('a start that is replaced, stopped or joined', () => { for (const pid of leakedPids.split('\n').filter(Boolean)) process.kill(Number(pid), 'SIGKILL'); expect(await leaked).toBe(false); expect(start).toContain('was stopped before it accepted connections'); - expect(await client.listServices('app-del-start')).toEqual([]); + expect(await client.getInstance(reserved.id)).toBeUndefined(); }, 30_000); test('a deployment stopped during its start fails saying it was stopped', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await ensureServiceOnOwnPort(client, 'app-stopped', 'web'); + const reserved = await createInstanceOnOwnPort(client); const deployed = client.putDeployment( - 'app-stopped', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v1', 'first', artifactDir), ); await Bun.sleep(300); - await client.stopApp('app-stopped'); + await client.stopInstances([reserved.id]); await expect(deployed).rejects.toThrow( - 'service "slow.web" was stopped before it accepted connections', + `instance "${reserved.id}" was stopped before it accepted connections`, ); - expect(await statusOf(client, 'app-stopped', 'web')).toBe('stopped'); + expect(await statusOf(client, reserved.id)).toBe('stopped'); }, 20_000); }); @@ -483,23 +451,23 @@ describe('with a 1-second readiness timeout', () => { fileURLToPath(new URL('./fixtures/short-ready-timeout.ts', import.meta.url)), ); const artifactDir = writeBootstrap(NEVER_LISTENING_BOOTSTRAP); - const reserved = await ensureServiceOnOwnPort(client, 'app-silent', 'web'); + const reserved = await createInstanceOnOwnPort(client); - const deployed = client.putDeployment('app-silent', 'web', { - address: 'silent.web', + const deployed = client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }); await expect(deployed).rejects.toThrow( - /service "silent\.web" did not accept connections on its port \d+ within 1s\. Its log: .*web\.log/, + new RegExp( + `instance "${reserved.id}" did not accept connections on its port ${String(reserved.port)} within 1s\\. Its log: .*${reserved.id}\\.log`, + ), ); - const [svc] = await client.listServices('app-silent'); + const svc = await instanceOf(client, reserved.id); expect(svc?.status).toBe('stopped'); expect(svc?.pid).toBeUndefined(); - const log = await (await fetch(`${client.baseUrl}/apps/app-silent/services/web/logs`)).text(); + const log = await (await fetch(`${client.baseUrl}/instances/${reserved.id}/logs`)).text(); expect(log).toContain('[emulator] did not accept connections on port'); }, 15_000); @@ -508,28 +476,20 @@ describe('with a 1-second readiness timeout', () => { fileURLToPath(new URL('./fixtures/short-ready-timeout.ts', import.meta.url)), ); const artifactDir = writeBootstrap(CRASH_ONCE_THEN_SILENT_BOOTSTRAP); - const reserved = await ensureServiceOnOwnPort(client, 'app-restart-silent', 'web'); + const reserved = await createInstanceOnOwnPort(client); await expect( - client.putDeployment('app-restart-silent', 'web', { - address: 'restart-silent.web', + client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }), ).rejects.toThrow('exited (code 1)'); - await waitFor( - async () => (await statusOf(client, 'app-restart-silent', 'web')) === 'held', - 8000, - 100, - ); - const [svc] = await client.listServices('app-restart-silent'); + await waitFor(async () => (await statusOf(client, reserved.id)) === 'held', 8000, 100); + const svc = await instanceOf(client, reserved.id); expect(svc?.pid).toBeUndefined(); - const log = await ( - await fetch(`${client.baseUrl}/apps/app-restart-silent/services/web/logs`) - ).text(); + const log = await (await fetch(`${client.baseUrl}/instances/${reserved.id}/logs`)).text(); expect(log).toContain('did not accept connections on port'); expect(log).toContain('held'); }, 20_000); @@ -542,13 +502,7 @@ describe('with a 3-second readiness poll', () => { const slowPoll = fileURLToPath(new URL('./fixtures/slow-ready-poll.ts', import.meta.url)); function crashingDeployment(artifactDir: string, port: number) { - return { - address: 'crash-loop.web', - artifactDir, - artifactHash: 'h', - env: baseEnv({ PORT: String(port) }), - port, - }; + return { artifactDir, artifactHash: 'h', env: baseEnv({ PORT: String(port) }) }; } function allowServing(artifactDir: string): void { @@ -558,50 +512,42 @@ describe('with a 3-second readiness poll', () => { test('a session-resume start just after a crash loop is held starts the service again', async () => { const client = await startDaemon(slowPoll); const artifactDir = writeBootstrap(CRASH_UNTIL_ALLOWED_BOOTSTRAP); - const reserved = await ensureServiceOnOwnPort(client, 'app-held-resume', 'web'); - - await expect( - client.putDeployment( - 'app-held-resume', - 'web', - crashingDeployment(artifactDir, reserved.port), - ), - ).rejects.toThrow('exited (code 1)'); - await waitFor( - async () => (await statusOf(client, 'app-held-resume', 'web')) === 'held', - 25_000, - 20, - ); + const reserved = await createInstanceOnOwnPort(client); + + // Not awaited: it settles only when the start in flight at `held` notices + // its process exited, which is the moment the resume below must not wait for. + const first = client + .putDeployment(reserved.id, crashingDeployment(artifactDir, reserved.port)) + .then( + () => 'succeeded', + (error: unknown) => String(error), + ); + await waitFor(async () => (await statusOf(client, reserved.id)) === 'held', 25_000, 20); allowServing(artifactDir); - await client.startApp('app-held-resume'); + await client.startInstances([reserved.id]); - expect(await statusOf(client, 'app-held-resume', 'web')).toBe('running'); + expect(await statusOf(client, reserved.id)).toBe('running'); expect(await acceptsConnection(reserved.port)).toBe(true); + expect(await first).toContain('exited (code 1)'); }, 45_000); test('an identical deployment just after the deployed process exits starts it again', async () => { const client = await startDaemon(slowPoll); const artifactDir = writeBootstrap(CRASH_UNTIL_ALLOWED_BOOTSTRAP); - const reserved = await ensureServiceOnOwnPort(client, 'app-crash-redeploy', 'web'); + const reserved = await createInstanceOnOwnPort(client); const deployment = crashingDeployment(artifactDir, reserved.port); - const first = client.putDeployment('app-crash-redeploy', 'web', deployment); - await waitFor( - async () => (await statusOf(client, 'app-crash-redeploy', 'web')) === 'backoff', - 5000, - 20, - ); + const first = client.putDeployment(reserved.id, deployment); + await waitFor(async () => (await statusOf(client, reserved.id)) === 'backoff', 5000, 20); allowServing(artifactDir); - await client.putDeployment('app-crash-redeploy', 'web', deployment); + await client.putDeployment(reserved.id, deployment); - expect(await statusOf(client, 'app-crash-redeploy', 'web')).toBe('running'); + expect(await statusOf(client, reserved.id)).toBe('running'); // A start that waited for the first deployment would leave the crash // trail in the log; a fresh start begins a clean one. - const log = await ( - await fetch(`${client.baseUrl}/apps/app-crash-redeploy/services/web/logs`) - ).text(); + const log = await (await fetch(`${client.baseUrl}/instances/${reserved.id}/logs`)).text(); expect(log).not.toContain('restarting in'); await first; }, 20_000); @@ -619,24 +565,20 @@ describe(`with ${SERVICE_READY_TIMEOUT_ENV}`, () => { process.env[SERVICE_READY_TIMEOUT_ENV] = '1000'; const client = await startDaemon(); const artifactDir = writeBootstrap(NEVER_LISTENING_BOOTSTRAP); - const reserved = await ensureServiceOnOwnPort(client, 'app-env-timeout', 'web'); + const reserved = await createInstanceOnOwnPort(client); - const deployed = client.putDeployment('app-env-timeout', 'web', { - address: 'env-timeout.web', + const deployed = client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }); await expect(deployed).rejects.toThrow('within 1s. Its log: '); await expect( - client.putDeployment('app-env-timeout', 'web', { - address: 'env-timeout.web', + client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h2', env: baseEnv(), - port: reserved.port, }), ).rejects.toThrow(SERVICE_READY_TIMEOUT_ENV); }, 15_000); diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/helpers.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/helpers.ts index 6cd80cc0f..dc9c2e4d0 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/helpers.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/helpers.ts @@ -160,20 +160,19 @@ export async function skipContendedDaemonPorts(registryRoot: string): Promise, + client: Pick, minPort: number = DEFAULT_MIN_SERVICE_PORT, ): Promise { const freePort = await getPort({ port: portNumbers(minPort, minPort + 200) }); for (let i = 0; i < freePort - minPort; i++) { - await client.ensureService('port-skip', `dummy-${String(i)}`); + await client.createInstance({ labels: { purpose: 'port-skip' } }); } } diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/client.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/client.ts index 692f98c51..74b57470d 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/client.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/client.ts @@ -73,20 +73,27 @@ const ADMIN_FETCH_RETRY_DELAY_MS = 150; * `fetch` for the loopback admin calls, retried on transient socket errors: * a keep-alive reuse race (the runtime re-uses a pooled connection the * daemon's HTTP server just closed idle — surfaces as "fetch failed" / - * "other side closed") or a briefly overloaded daemon. Every admin call is - * idempotent by design, so a short blind retry is safe. A caller-driven - * abort is never retried. When every attempt fails to connect, it throws - * `DEV.EMULATOR_UNREACHABLE`; callers retry that code and nothing else. + * "other side closed") or a briefly overloaded daemon. A short blind retry is + * safe only for an idempotent call; a call that is not, such as creating an + * instance, passes `retry: false`, because the emulator may have handled the + * first attempt. A caller-driven abort is never retried. When every attempt + * fails to connect, it throws `DEV.EMULATOR_UNREACHABLE`; callers retry that + * code and nothing else. */ -async function adminFetch(url: string, init?: RequestInit): Promise { +async function adminFetch( + url: string, + init?: RequestInit, + opts: { readonly retry?: boolean } = {}, +): Promise { + const attempts = opts.retry === false ? 1 : ADMIN_FETCH_ATTEMPTS; let lastError: unknown; - for (let attempt = 1; attempt <= ADMIN_FETCH_ATTEMPTS; attempt += 1) { + for (let attempt = 1; attempt <= attempts; attempt += 1) { try { return await fetch(url, init); } catch (error) { if (init?.signal?.aborted === true) throw error; lastError = error; - if (attempt < ADMIN_FETCH_ATTEMPTS) { + if (attempt < attempts) { await new Promise((resolve) => setTimeout(resolve, ADMIN_FETCH_RETRY_DELAY_MS)); } } @@ -111,52 +118,67 @@ async function expectOk(res: Response): Promise { return res; } -export interface ServiceInfo { +export type InstanceStatus = 'running' | 'backoff' | 'held' | 'stopped'; + +export interface InstanceInfo { readonly id: string; - readonly address: string; + /** Stored and returned verbatim; the emulator never reads them. */ + readonly labels: Readonly>; readonly port: number; readonly url: string; - readonly status: 'running' | 'backoff' | 'held' | 'stopped'; + readonly status: InstanceStatus; readonly pid?: number; readonly lastExitCode?: number; readonly artifactHash?: string; readonly logPath: string; } +export interface CreateInstanceRequest { + readonly labels: Readonly>; + /** Used if no instance holds it and it is bindable; otherwise the emulator gives the lowest free port from 3000. */ + readonly port?: number; +} + export interface DeploymentRequest { - readonly address: string; readonly artifactDir: string; readonly artifactHash: string; readonly env: Readonly>; - readonly port: number; /** `false` when the service does not listen on its port: it is `running` once spawned. Defaults to `true`. */ readonly listens?: boolean; } -const SERVICE_STATUSES = new Set(['running', 'backoff', 'held', 'stopped']); +const INSTANCE_STATUSES = new Set(['running', 'backoff', 'held', 'stopped']); -function isServiceInfo(value: unknown): value is ServiceInfo { +function isStringRecord(value: unknown): value is Record { + return ( + typeof value === 'object' && + value !== null && + Object.values(value).every((v) => typeof v === 'string') + ); +} + +function isInstanceInfo(value: unknown): value is InstanceInfo { return ( typeof value === 'object' && value !== null && 'id' in value && typeof value.id === 'string' && - 'address' in value && - typeof value.address === 'string' && + 'labels' in value && + isStringRecord(value.labels) && 'port' in value && typeof value.port === 'number' && 'url' in value && typeof value.url === 'string' && 'status' in value && typeof value.status === 'string' && - SERVICE_STATUSES.has(value.status) && + INSTANCE_STATUSES.has(value.status) && 'logPath' in value && typeof value.logPath === 'string' ); } -function isServiceInfoArray(value: unknown): value is ServiceInfo[] { - return Array.isArray(value) && value.every(isServiceInfo); +function isInstanceInfoArray(value: unknown): value is InstanceInfo[] { + return Array.isArray(value) && value.every(isInstanceInfo); } interface HealthBody { @@ -172,44 +194,42 @@ function isHealthBody(value: unknown): value is HealthBody { ); } -interface ServiceReservation { - readonly port: number; - readonly url: string; -} - -function isServiceReservation(value: unknown): value is ServiceReservation { - return ( - typeof value === 'object' && - value !== null && - 'port' in value && - typeof value.port === 'number' && - 'url' in value && - typeof value.url === 'string' - ); -} - +/** Each compute instance is keyed by an id the emulator generates when it creates the instance. */ export interface ComputeClient { readonly baseUrl: string; health(): Promise<{ version: string }>; - /** `PUT /apps//services/` — reserves (or returns) the service's stable port. Idempotent. */ - ensureService(app: string, id: string): Promise<{ port: number; url: string }>; - /** `PUT /apps//services//deployment`. */ - putDeployment(app: string, id: string, deployment: DeploymentRequest): Promise; - /** `GET /apps//services`. */ - listServices(app: string): Promise; - /** `GET /apps//services//logs?follow=1` — yields decoded text chunks; ends when `signal` aborts or the daemon closes the stream. */ + /** `POST /instances` — creates a stopped instance holding a port until it is deleted. Never retried: a retry could create a second instance. */ + createInstance(request: CreateInstanceRequest): Promise; + /** `GET /instances`. */ + listInstances(): Promise; + /** `GET /instances/`; undefined when the emulator has no such instance. */ + getInstance(id: string): Promise; + /** `PUT /instances//deployment` — returns once the instance accepts connections on its port. */ + putDeployment(id: string, deployment: DeploymentRequest): Promise; + /** `GET /instances//logs?follow=1` — yields decoded text chunks; ends when `signal` aborts or the daemon closes the stream. */ followLogs( - app: string, id: string, signal?: AbortSignal, opts?: { readonly tail?: number }, ): AsyncIterable; - /** `POST /apps//stop`. */ - stopApp(app: string): Promise; - /** `POST /apps//start` — the session-resume signal; starts every service with a stored deployment that isn't already running. */ - startApp(app: string): Promise; - /** `DELETE /apps/`. */ - deleteApp(app: string): Promise; + /** `POST /instances/start` — the session-resume signal; starts each listed instance with a stored deployment that isn't already running. */ + startInstances(ids: readonly string[]): Promise; + /** `POST /instances/stop`. The instances keep their ports; an unknown id is skipped. */ + stopInstances(ids: readonly string[]): Promise; + /** `DELETE /instances/` — stops the instance and frees its port. Deleting an unknown id succeeds. */ + deleteInstance(id: string): Promise; +} + +function instancePath(baseUrl: string, id: string): string { + return `${baseUrl}/instances/${encodeURIComponent(id)}`; +} + +function postJson(body: unknown): RequestInit { + return { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify(body), + }; } export function computeClient(opts: DaemonRootOptions = {}): ComputeClient { @@ -227,20 +247,39 @@ export function computeClient(opts: DaemonRootOptions = {}): ComputeClient { return body; }, - async ensureService(app, id) { - const url = `${baseUrl}/apps/${encodeSegment(app)}/services/${encodeSegment(id)}`; - const res = await expectOk(await adminFetch(url, { method: 'PUT' })); + async createInstance(request) { + const res = await expectOk( + await adminFetch(`${baseUrl}/instances`, postJson(request), { retry: false }), + ); const body: unknown = await res.json(); - if (!isServiceReservation(body)) { - throw new Error('malformed service-reservation response from the compute emulator'); + if (!isInstanceInfo(body)) { + throw new Error('malformed instance-create response from the compute emulator'); } return body; }, - async putDeployment(app, id, deployment) { - const url = `${baseUrl}/apps/${encodeSegment(app)}/services/${encodeSegment(id)}/deployment`; + async listInstances() { + const res = await expectOk(await adminFetch(`${baseUrl}/instances`)); + const body: unknown = await res.json(); + if (!isInstanceInfoArray(body)) { + throw new Error('malformed instances listing from the compute emulator'); + } + return body; + }, + + async getInstance(id) { + const res = await adminFetch(instancePath(baseUrl, id)); + if (res.status === 404) return undefined; + const body: unknown = await (await expectOk(res)).json(); + if (!isInstanceInfo(body)) { + throw new Error('malformed instance response from the compute emulator'); + } + return body; + }, + + async putDeployment(id, deployment) { await expectOk( - await adminFetch(url, { + await adminFetch(`${instancePath(baseUrl, id)}/deployment`, { method: 'PUT', headers: { 'content-type': 'application/json' }, body: JSON.stringify(deployment), @@ -248,20 +287,10 @@ export function computeClient(opts: DaemonRootOptions = {}): ComputeClient { ); }, - async listServices(app) { - const url = `${baseUrl}/apps/${encodeSegment(app)}/services`; - const res = await expectOk(await adminFetch(url)); - const body: unknown = await res.json(); - if (!isServiceInfoArray(body)) { - throw new Error('malformed services listing from the compute emulator'); - } - return body; - }, - - async *followLogs(app, id, signal, opts) { + async *followLogs(id, signal, opts) { const tail = opts?.tail !== undefined ? `&tail=${String(Math.max(0, Math.trunc(opts.tail)))}` : ''; - const url = `${baseUrl}/apps/${encodeSegment(app)}/services/${encodeSegment(id)}/logs?follow=1${tail}`; + const url = `${instancePath(baseUrl, id)}/logs?follow=1${tail}`; const res = await expectOk(await adminFetch(url, signal ? { signal } : undefined)); const body = res.body; if (!body) return; @@ -278,19 +307,16 @@ export function computeClient(opts: DaemonRootOptions = {}): ComputeClient { } }, - async stopApp(app) { - const url = `${baseUrl}/apps/${encodeSegment(app)}/stop`; - await expectOk(await adminFetch(url, { method: 'POST' })); + async startInstances(ids) { + await expectOk(await adminFetch(`${baseUrl}/instances/start`, postJson({ ids }))); }, - async startApp(app) { - const url = `${baseUrl}/apps/${encodeSegment(app)}/start`; - await expectOk(await adminFetch(url, { method: 'POST' })); + async stopInstances(ids) { + await expectOk(await adminFetch(`${baseUrl}/instances/stop`, postJson({ ids }))); }, - async deleteApp(app) { - const url = `${baseUrl}/apps/${encodeSegment(app)}`; - await expectOk(await adminFetch(url, { method: 'DELETE' })); + async deleteInstance(id) { + await expectOk(await adminFetch(instancePath(baseUrl, id), { method: 'DELETE' })); }, }; } diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/compute-main.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/compute-main.ts index 498a808d2..29e68c8e1 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/compute-main.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/compute-main.ts @@ -1,9 +1,12 @@ /** * The Compute emulator daemon (local-dev spec § 2 `compute-main.ts`): a small - * local counterpart of the platform's compute service. Owns service child - * processes (spawned `bun bootstrap.js` from a real packaged artifact), - * supervises them (crash backoff, held after repeated fast crashes), and - * serves a loopback JSON admin API plus per-service log streaming. + * local counterpart of the platform's compute service. Holds service + * instances under ids it generates, owns their child processes (spawned + * `bun bootstrap.js` from a real packaged artifact), supervises them (crash + * backoff, held after repeated fast crashes), and serves a loopback JSON + * admin API plus per-instance log streaming. An instance's labels are its + * creator's to read: they are stored and returned verbatim, never + * interpreted. * * Runs as its own OS process, started by `daemon.ts`'s `ensureDaemon` via * `process.execPath --port --state-dir `. Everything it @@ -11,13 +14,13 @@ * environment variable of its own. */ import { type ChildProcess, spawn } from 'node:child_process'; +import { randomBytes } from 'node:crypto'; import * as fs from 'node:fs'; import * as http from 'node:http'; import * as net from 'node:net'; import * as path from 'node:path'; import getPort, { clearLockedPorts, portNumbers } from 'get-port'; import { readOwnVersion, SERVICE_READY_TIMEOUT_ENV } from './daemon.ts'; -import { isValidSegment } from './segments.ts'; import { readJsonFile, StateFile } from './state-file.ts'; const MIN_SERVICE_PORT = 3000; @@ -31,18 +34,19 @@ const LOG_POLL_INTERVAL_MS = 100; const DEFAULT_READY_TIMEOUT_MS = 30_000; const DEFAULT_READY_POLL_MS = 50; const READY_PROBE_TIMEOUT_MS = 1000; -const APPS_STATE_MODE = 0o600; +const STATE_MODE = 0o600; const BUN_NOT_FOUND_MESSAGE = 'local dev runs services under bun — the Prisma Compute runtime — and `bun` was not found on PATH. Install it: https://bun.sh.'; -type ServiceStatus = 'running' | 'backoff' | 'held' | 'stopped'; +type InstanceStatus = 'running' | 'backoff' | 'held' | 'stopped'; -interface ServiceRecord { - id: string; - address: string; - port: number; - status: ServiceStatus; +interface InstanceRecord { + readonly id: string; + readonly labels: Record; + /** Held from creation until the instance is deleted, running or stopped: a deployment's env carries it. */ + readonly port: number; + status: InstanceStatus; pid?: number; lastExitCode?: number; artifactHash?: string; @@ -53,13 +57,17 @@ interface ServiceRecord { readonly logPath: string; } -interface AppRecord { - services: Record; +interface InstancesState { + instances: Record; } -type AppsState = Record; +const INSTANCE_ID_RE = /^svc_[0-9a-f]{16}$/; + +function newInstanceId(): string { + return `svc_${randomBytes(8).toString('hex')}`; +} -function isServiceStatus(value: unknown): value is ServiceStatus { +function isInstanceStatus(value: unknown): value is InstanceStatus { return value === 'running' || value === 'backoff' || value === 'held' || value === 'stopped'; } @@ -67,67 +75,68 @@ function isStringRecord(value: unknown): value is Record { return ( typeof value === 'object' && value !== null && + !Array.isArray(value) && Object.values(value).every((v) => typeof v === 'string') ); } -function isServiceRecord(value: unknown): value is ServiceRecord { +function isPort(value: unknown): value is number { + return ( + typeof value === 'number' && Number.isInteger(value) && value >= 1 && value <= MAX_SERVICE_PORT + ); +} + +function isOptional(value: object, key: string, check: (field: unknown) => boolean): boolean { + return !(key in value) || Reflect.get(value, key) === undefined || check(Reflect.get(value, key)); +} + +function isInstanceRecord(value: unknown): value is InstanceRecord { if (typeof value !== 'object' || value === null) return false; if (!('id' in value) || typeof value.id !== 'string') return false; - if (!('address' in value) || typeof value.address !== 'string') return false; - if (!('port' in value) || typeof value.port !== 'number') return false; - if (!('status' in value) || !isServiceStatus(value.status)) return false; + if (!('labels' in value) || !isStringRecord(value.labels)) return false; + if (!('port' in value) || !isPort(value.port)) return false; + if (!('status' in value) || !isInstanceStatus(value.status)) return false; if (!('logPath' in value) || typeof value.logPath !== 'string') return false; - if ('pid' in value && value.pid !== undefined && typeof value.pid !== 'number') return false; - if ( - 'lastExitCode' in value && - value.lastExitCode !== undefined && - typeof value.lastExitCode !== 'number' - ) { - return false; - } - if ( - 'artifactHash' in value && - value.artifactHash !== undefined && - typeof value.artifactHash !== 'string' - ) { - return false; - } - if ( - 'artifactDir' in value && - value.artifactDir !== undefined && - typeof value.artifactDir !== 'string' - ) { - return false; - } - if ('env' in value && value.env !== undefined && !isStringRecord(value.env)) return false; - if ('listens' in value && value.listens !== undefined && typeof value.listens !== 'boolean') { - return false; - } - return true; + return ( + isOptional(value, 'pid', (v) => typeof v === 'number') && + isOptional(value, 'lastExitCode', (v) => typeof v === 'number') && + isOptional(value, 'artifactHash', (v) => typeof v === 'string') && + isOptional(value, 'artifactDir', (v) => typeof v === 'string') && + isOptional(value, 'env', isStringRecord) && + isOptional(value, 'listens', (v) => typeof v === 'boolean') + ); } -function isAppRecord(value: unknown): value is AppRecord { +function isInstancesState(value: unknown): value is InstancesState { return ( typeof value === 'object' && value !== null && - 'services' in value && - typeof value.services === 'object' && - value.services !== null && - Object.values(value.services).every(isServiceRecord) + 'instances' in value && + typeof value.instances === 'object' && + value.instances !== null && + Object.values(value.instances).every(isInstanceRecord) ); } -function isAppsState(value: unknown): value is AppsState { - return typeof value === 'object' && value !== null && Object.values(value).every(isAppRecord); +interface CreateBody { + readonly labels: Record; + readonly port?: number; +} + +function isCreateBody(value: unknown): value is CreateBody { + return ( + typeof value === 'object' && + value !== null && + 'labels' in value && + isStringRecord(value.labels) && + isOptional(value, 'port', isPort) + ); } interface DeploymentBody { - readonly address: string; readonly artifactDir: string; readonly artifactHash: string; readonly env: Record; - readonly port: number; readonly listens?: boolean; } @@ -135,21 +144,31 @@ function isDeploymentBody(value: unknown): value is DeploymentBody { return ( typeof value === 'object' && value !== null && - 'address' in value && - typeof value.address === 'string' && 'artifactDir' in value && typeof value.artifactDir === 'string' && 'artifactHash' in value && typeof value.artifactHash === 'string' && 'env' in value && isStringRecord(value.env) && - 'port' in value && - typeof value.port === 'number' && (!('listens' in value) || value.listens === undefined || typeof value.listens === 'boolean') ); } -/** Live, non-persisted bookkeeping for one service — the child handle and its supervision counters. */ +interface IdsBody { + readonly ids: readonly string[]; +} + +function isIdsBody(value: unknown): value is IdsBody { + return ( + typeof value === 'object' && + value !== null && + 'ids' in value && + Array.isArray(value.ids) && + value.ids.every((id) => typeof id === 'string') + ); +} + +/** Live, non-persisted bookkeeping for one instance — the child handle and its supervision counters. */ interface RuntimeInfo { child: ChildProcess | undefined; consecutiveFastExits: number; @@ -157,12 +176,12 @@ interface RuntimeInfo { stableTimer: NodeJS.Timeout | undefined; backoffTimer: NodeJS.Timeout | undefined; expectedExit: boolean; - log: ServiceLog | undefined; + log: InstanceLog | undefined; /** The start in flight, until its child accepts connections or fails to. A newer start or a stop replaces or clears it. */ start: InFlightStart | undefined; /** A stop still waiting for its child to exit. A start waits for it, so it never spawns beside a dying child. */ stopping: Promise | undefined; - /** Set when the app is being deleted: no start may spawn on this runtime again. */ + /** Set when the instance is being deleted: no start may spawn on this runtime again. */ deleted: boolean; } @@ -191,8 +210,8 @@ function sleep(ms: number): Promise { return new Promise((resolve) => setTimeout(resolve, ms)); } -/** Append-only per-service log file — child stdout/stderr plus `[emulator]` supervision lines. No rotation in v1. */ -class ServiceLog { +/** Append-only per-instance log file — child stdout/stderr plus `[emulator]` supervision lines. No rotation in v1. */ +class InstanceLog { private fd: number | undefined; constructor(private readonly filePath: string) {} @@ -213,7 +232,7 @@ class ServiceLog { fs.writeSync(this.ensureOpen(), `${line}\n`); } - /** Empties the file — a fresh service start begins a clean log; past sessions' and past deployments' output is not kept. */ + /** Empties the file — a fresh start begins a clean log; past sessions' and past deployments' output is not kept. */ clear(): void { fs.ftruncateSync(this.ensureOpen(), 0); } @@ -320,24 +339,20 @@ function resolveBunOnPath(env: Readonly>): string | undef function main(): void { const { port, stateDir, readyTimeoutMs, readyPollMs } = parseArgs(process.argv.slice(2)); const ownVersion = readOwnVersion(); - const appsJsonPath = path.join(stateDir, 'apps.json'); - const stateFile = new StateFile(appsJsonPath, APPS_STATE_MODE); + const statePath = path.join(stateDir, 'instances.json'); + const stateFile = new StateFile(statePath, STATE_MODE); + const logsDir = path.join(stateDir, 'logs'); - let state: AppsState = {}; + let state: InstancesState = { instances: {} }; const runtimes = new Map(); - // Bumped every time a log file is cleared (a fresh service start). A live + // Bumped every time a log file is cleared (a fresh start). A live // follower watches this rather than the file size, so it resets to the new // start reliably — even when the clear and the first new write both land // between two of its polls, which a size comparison alone would miss. const logGeneration = new Map(); - function runtimeKey(app: string, id: string): string { - return `${app}/${id}`; - } - - function getRuntime(app: string, id: string): RuntimeInfo { - const key = runtimeKey(app, id); - let rt = runtimes.get(key); + function getRuntime(id: string): RuntimeInfo { + let rt = runtimes.get(id); if (!rt) { rt = { child: undefined, @@ -351,18 +366,14 @@ function main(): void { stopping: undefined, deleted: false, }; - runtimes.set(key, rt); + runtimes.set(id, rt); } return rt; } - function serviceLogPath(app: string, id: string): string { - return path.join(stateDir, 'logs', app, `${id}.log`); - } - - function getLog(app: string, id: string, logPath: string): ServiceLog { - const rt = getRuntime(app, id); - if (!rt.log) rt.log = new ServiceLog(logPath); + function getLog(inst: InstanceRecord): InstanceLog { + const rt = getRuntime(inst.id); + if (!rt.log) rt.log = new InstanceLog(inst.logPath); return rt.log; } @@ -380,45 +391,37 @@ function main(): void { rt.backoffTimer = undefined; } - function usedServicePorts(): Set { - const ports = new Set(); - for (const app of Object.values(state)) { - for (const svc of Object.values(app.services)) ports.add(svc.port); - } - return ports; + function heldPorts(): Set { + return new Set(Object.values(state.instances).map((inst) => inst.port)); } /** - * The smallest genuinely free port at or above `MIN_SERVICE_PORT`, - * excluding this daemon's own persisted allocations — persistence and - * the range policy stay ours; whether a candidate is actually bindable - * is `get-port`'s (spec § 2's dependency razor). + * The requested port when no instance holds it and it is bindable, + * otherwise the smallest such port at or above `MIN_SERVICE_PORT`. Which + * ports instances hold is ours; whether a candidate is actually bindable is + * `get-port`'s (spec § 2's dependency razor). */ - async function smallestUnusedServicePort(): Promise { - return getPort({ - port: portNumbers(MIN_SERVICE_PORT, MAX_SERVICE_PORT), - exclude: usedServicePorts(), - }); + async function allocatePort(requested: number | undefined): Promise { + const exclude = heldPorts(); + if (requested !== undefined && !exclude.has(requested)) { + const given = await getPort({ port: requested, exclude }); + if (given === requested) return given; + } + return getPort({ port: portNumbers(MIN_SERVICE_PORT, MAX_SERVICE_PORT), exclude }); } - async function getOrCreateService(app: string, id: string): Promise { - let appRec = state[app]; - if (!appRec) { - appRec = { services: {} }; - state[app] = appRec; - } - let svc = appRec.services[id]; - if (svc) return svc; - svc = { + async function createInstance(body: CreateBody): Promise { + const id = newInstanceId(); + const inst: InstanceRecord = { id, - address: id, - port: await smallestUnusedServicePort(), + labels: { ...body.labels }, + port: await allocatePort(body.port), status: 'stopped', - logPath: serviceLogPath(app, id), + logPath: path.join(logsDir, `${id}.log`), }; - appRec.services[id] = svc; + state.instances[id] = inst; schedulePersist(); - return svc; + return inst; } function waitForExit(child: ChildProcess): Promise { @@ -446,9 +449,7 @@ function main(): void { } function handleChildExit( - app: string, - id: string, - svc: ServiceRecord, + inst: InstanceRecord, rt: RuntimeInfo, child: ChildProcess, code: number | null, @@ -456,8 +457,8 @@ function main(): void { if (child !== rt.child) return; clearStableTimer(rt); rt.child = undefined; - delete svc.pid; - if (code !== null) svc.lastExitCode = code; + delete inst.pid; + if (code !== null) inst.lastExitCode = code; if (rt.expectedExit) { rt.expectedExit = false; @@ -471,28 +472,28 @@ function main(): void { if (uptime < STABLE_UPTIME_MS) rt.consecutiveFastExits += 1; if (rt.consecutiveFastExits >= MAX_CONSECUTIVE_FAST_EXITS) { - svc.status = 'held'; + inst.status = 'held'; schedulePersist(); - getLog(app, id, svc.logPath).writeLine( + getLog(inst).writeLine( `[emulator] held after ${String(MAX_CONSECUTIVE_FAST_EXITS)} consecutive fast exits — send a new deployment to resume`, ); return; } const delayMs = Math.min(MAX_BACKOFF_MS, BASE_BACKOFF_MS * 2 ** attemptNumber); - svc.status = 'backoff'; + inst.status = 'backoff'; schedulePersist(); - getLog(app, id, svc.logPath).writeLine( + getLog(inst).writeLine( `[emulator] exited (code ${code === null ? 'null' : String(code)}) — restarting in ${String(delayMs / 1000)}s`, ); rt.backoffTimer = setTimeout(() => { rt.backoffTimer = undefined; // A crash-backoff respawn — keep the crash trail this run has written. - void spawnService(app, id, svc, rt, false); + void spawnInstance(inst, rt, false); }, delayMs); } - /** Records `run` as the service's start in flight, replacing any earlier one. */ + /** Records `run` as the instance's start in flight, replacing any earlier one. */ function beginStart( rt: RuntimeInfo, fresh: boolean, @@ -511,9 +512,9 @@ function main(): void { return rt.start?.token.processExited === false ? rt.start : undefined; } - /** Undefined while `token` is still the service's start. Otherwise what that start reports: a newer start's result, or "stopped" after a stop or an app delete. */ + /** Undefined while `token` is still the instance's start. Otherwise what that start reports: a newer start's result, or "stopped" after a stop or a delete. */ function supersededOutcome( - svc: ServiceRecord, + inst: InstanceRecord, rt: RuntimeInfo, token: StartToken, ): Promise | undefined { @@ -521,51 +522,45 @@ function main(): void { if (!rt.deleted && rt.start) return rt.start.result; return Promise.resolve({ ok: false, - message: `service "${svc.address}" was stopped before it accepted connections`, + message: `instance "${inst.id}" was stopped before it accepted connections`, }); } /** - * Spawns the service's child and resolves once it accepts connections on + * Spawns the instance's child and resolves once it accepts connections on * its port — only then is it `running`. A child that exits first, or never * listens within the readiness timeout, fails the start with a message - * naming the service and its log. + * naming the instance and its log. */ - function spawnService( - app: string, - id: string, - svc: ServiceRecord, + function spawnInstance( + inst: InstanceRecord, rt: RuntimeInfo, /** A fresh start (a deployment or a session resume) begins a clean log; a crash-backoff respawn keeps the run's crash trail. */ fresh: boolean, ): Promise { - return beginStart(rt, fresh, (token) => - spawnAndWaitUntilListening(app, id, svc, rt, fresh, token), - ); + return beginStart(rt, fresh, (token) => spawnAndWaitUntilListening(inst, rt, fresh, token)); } async function spawnAndWaitUntilListening( - app: string, - id: string, - svc: ServiceRecord, + inst: InstanceRecord, rt: RuntimeInfo, fresh: boolean, token: StartToken, ): Promise { - const env = svc.env ?? {}; - const artifactDir = svc.artifactDir ?? '.'; + const env = inst.env ?? {}; + const artifactDir = inst.artifactDir ?? '.'; const bunPath = resolveBunOnPath(env); if (!bunPath) { - svc.status = 'stopped'; - delete svc.pid; + inst.status = 'stopped'; + delete inst.pid; schedulePersist(); return { ok: false, message: BUN_NOT_FOUND_MESSAGE }; } - const log = getLog(app, id, svc.logPath); + const log = getLog(inst); if (fresh) { log.clear(); - logGeneration.set(svc.logPath, (logGeneration.get(svc.logPath) ?? 0) + 1); + logGeneration.set(inst.logPath, (logGeneration.get(inst.logPath) ?? 0) + 1); } const child = spawn(bunPath, ['bootstrap.js'], { cwd: artifactDir, @@ -582,54 +577,54 @@ function main(): void { child.stdout?.on('data', (chunk: Buffer) => log.writeRaw(chunk)); child.stderr?.on('data', (chunk: Buffer) => log.writeRaw(chunk)); - child.on('exit', (code) => handleChildExit(app, id, svc, rt, child, code)); + child.on('exit', (code) => handleChildExit(inst, rt, child, code)); child.once('exit', () => { token.processExited = true; }); - if (child.pid !== undefined) svc.pid = child.pid; - if (svc.listens === false) { - svc.status = 'running'; + if (child.pid !== undefined) inst.pid = child.pid; + if (inst.listens === false) { + inst.status = 'running'; schedulePersist(); return { ok: true }; } schedulePersist(); - const port = String(svc.port); - const outcome = await waitUntilListening(child, svc.port, readyTimeoutMs, readyPollMs); - const superseded = supersededOutcome(svc, rt, token); + const port = String(inst.port); + const outcome = await waitUntilListening(child, inst.port, readyTimeoutMs, readyPollMs); + const superseded = supersededOutcome(inst, rt, token); if (superseded) return superseded; - const service = `service "${svc.address}"`; + const instance = `instance "${inst.id}"`; if (outcome.kind === 'exited') { const how = outcome.code !== null ? `code ${String(outcome.code)}` : (outcome.signal ?? 'no exit code'); return { ok: false, - message: `${service} exited (${how}) before it accepted connections on its port ${port}. Its log: ${svc.logPath}`, + message: `${instance} exited (${how}) before it accepted connections on its port ${port}. Its log: ${inst.logPath}`, }; } if (outcome.kind === 'timed-out') { const seconds = String(readyTimeoutMs / 1000); - const message = `${service} did not accept connections on its port ${port} within ${seconds}s. Its log: ${svc.logPath}. If it needs longer to start, set ${SERVICE_READY_TIMEOUT_ENV} (milliseconds) and restart the compute emulator. If it never listens on a port, declare it with compute({ listens: false }).`; + const message = `${instance} did not accept connections on its port ${port} within ${seconds}s. Its log: ${inst.logPath}. If it needs longer to start, set ${SERVICE_READY_TIMEOUT_ENV} (milliseconds) and restart the compute emulator. If it never listens on its port, deploy it with listens: false.`; if (fresh) { log.writeLine( `[emulator] did not accept connections on port ${port} within ${seconds}s — stopped`, ); - await stopService(svc, rt); + await stopInstance(inst, rt); } else { log.writeLine( `[emulator] restart did not accept connections on port ${port} within ${seconds}s — held; send a new deployment to resume`, ); - await stopService(svc, rt, 'held'); + await stopInstance(inst, rt, 'held'); } return { ok: false, message }; } - svc.status = 'running'; + inst.status = 'running'; schedulePersist(); return { ok: true }; } - async function stopService( - svc: ServiceRecord, + async function stopInstance( + inst: InstanceRecord, rt: RuntimeInfo, finalStatus: 'stopped' | 'held' = 'stopped', ): Promise { @@ -637,8 +632,8 @@ function main(): void { clearBackoffTimer(rt); clearStableTimer(rt); const stopping = killChild(rt, TERMINATE_GRACE_MS).then(() => { - svc.status = finalStatus; - delete svc.pid; + inst.status = finalStatus; + delete inst.pid; schedulePersist(); }); rt.stopping = stopping; @@ -655,30 +650,45 @@ function main(): void { /** * The session-resume signal: Alchemy's no-op reconcile never fires a - * deployment PUT, so nothing else restarts a service a previous session + * deployment PUT, so nothing else restarts an instance a previous session * stopped. Reuses the exact deployment-PUT start rules — same spawn * path, same supervision reset, and an explicit resume clears `held` - * the same way a deployment PUT does — but from the service's already - * STORED deployment spec, since no new one arrives with a start. + * the same way a deployment PUT does — but from the instance's already + * STORED deployment, since no new one arrives with a start. */ - async function startService(app: string, id: string, svc: ServiceRecord): Promise { - if (svc.artifactHash === undefined) return { ok: true }; // never deployed — nothing to start - const rt = getRuntime(app, id); + async function startInstance(inst: InstanceRecord): Promise { + if (inst.artifactHash === undefined) return { ok: true }; // never deployed — nothing to start + const rt = getRuntime(inst.id); const inFlight = joinableStart(rt); if (inFlight) return inFlight.result; - if (svc.status === 'running' && rt.stopping === undefined) return { ok: true }; // already up + if (inst.status === 'running' && rt.stopping === undefined) return { ok: true }; // already up // A session resume is a fresh start — begin a clean log. return beginStart(rt, true, async (token) => { clearBackoffTimer(rt); await stopsFinished(rt); - const superseded = supersededOutcome(svc, rt, token); + const superseded = supersededOutcome(inst, rt, token); if (superseded) return superseded; clearStableTimer(rt); rt.consecutiveFastExits = 0; - return spawnAndWaitUntilListening(app, id, svc, rt, true, token); + return spawnAndWaitUntilListening(inst, rt, true, token); }); } + async function deleteInstance(inst: InstanceRecord): Promise { + const rt = getRuntime(inst.id); + rt.deleted = true; + await stopInstance(inst, rt); + rt.log?.close(); + runtimes.delete(inst.id); + delete state.instances[inst.id]; + schedulePersist(); + // get-port holds every port it returns for up to 30 seconds; the freed + // port must be free again for an instance created right after. + // `heldPorts()` already keeps other instances' ports out. + clearLockedPorts(); + await fs.promises.rm(inst.logPath, { force: true }); + } + // ——— HTTP plumbing ——— function json(res: http.ServerResponse, status: number, body: unknown): void { @@ -692,12 +702,9 @@ function main(): void { res.end(body); } - function badSegment(res: http.ServerResponse, segment: string): void { - text( - res, - 400, - `invalid path segment "${segment}": must match /^[a-z0-9][a-z0-9-]*$/ and be at most 63 characters`, - ); + function noContent(res: http.ServerResponse): void { + res.writeHead(204); + res.end(); } function readBody(req: http.IncomingMessage): Promise { @@ -709,17 +716,26 @@ function main(): void { }); } - function serviceView(svc: ServiceRecord): unknown { + async function readJsonBody(req: http.IncomingMessage): Promise { + const raw = await readBody(req); + try { + return JSON.parse(raw.toString('utf8')); + } catch { + return undefined; + } + } + + function instanceView(inst: InstanceRecord): unknown { return { - id: svc.id, - address: svc.address, - port: svc.port, - url: `http://localhost:${String(svc.port)}`, - status: svc.status, - ...(svc.pid !== undefined ? { pid: svc.pid } : {}), - ...(svc.lastExitCode !== undefined ? { lastExitCode: svc.lastExitCode } : {}), - ...(svc.artifactHash !== undefined ? { artifactHash: svc.artifactHash } : {}), - logPath: svc.logPath, + id: inst.id, + labels: inst.labels, + port: inst.port, + url: `http://localhost:${String(inst.port)}`, + status: inst.status, + ...(inst.pid !== undefined ? { pid: inst.pid } : {}), + ...(inst.lastExitCode !== undefined ? { lastExitCode: inst.lastExitCode } : {}), + ...(inst.artifactHash !== undefined ? { artifactHash: inst.artifactHash } : {}), + logPath: inst.logPath, }; } @@ -734,26 +750,17 @@ function main(): void { async function handleDeployment( req: http.IncomingMessage, res: http.ServerResponse, - app: string, - id: string, + inst: InstanceRecord, ): Promise { - const raw = await readBody(req); - let parsed: unknown; - try { - parsed = JSON.parse(raw.toString('utf8')); - } catch { - return text(res, 400, 'malformed JSON body'); - } - if (!isDeploymentBody(parsed)) return text(res, 400, 'malformed deployment body'); + const body = await readJsonBody(req); + if (!isDeploymentBody(body)) return text(res, 400, 'malformed deployment body'); - const svc = await getOrCreateService(app, id); - const rt = getRuntime(app, id); - - const listens = parsed.listens ?? true; + const rt = getRuntime(inst.id); + const listens = body.listens ?? true; const changed = - svc.artifactHash !== parsed.artifactHash || - !envEquals(svc.env, parsed.env) || - (svc.listens ?? true) !== listens; + inst.artifactHash !== body.artifactHash || + !envEquals(inst.env, body.env) || + (inst.listens ?? true) !== listens; let result: SpawnResult; const inFlight = joinableStart(rt); @@ -764,39 +771,36 @@ function main(): void { !changed && rt.start === undefined && rt.stopping === undefined && - svc.status === 'running' + inst.status === 'running' ) { - // A true no-op: nothing about this service's persisted record + // A true no-op: nothing about this instance's persisted record // changes, so nothing is mutated here either — nothing to persist. result = { ok: true }; } else { - svc.address = parsed.address; - svc.port = parsed.port; - svc.artifactHash = parsed.artifactHash; - svc.env = parsed.env; - svc.artifactDir = parsed.artifactDir; - svc.listens = listens; + inst.artifactHash = body.artifactHash; + inst.env = body.env; + inst.artifactDir = body.artifactDir; + inst.listens = listens; clearBackoffTimer(rt); schedulePersist(); - // A new or changed deployment, or a service that is not up: a fresh + // A new or changed deployment, or an instance that is not up: a fresh // start with a clean log, once the old process (if any) has exited. result = await beginStart(rt, true, async (token) => { await stopsFinished(rt); await killChild(rt, TERMINATE_GRACE_MS); - const superseded = supersededOutcome(svc, rt, token); + const superseded = supersededOutcome(inst, rt, token); if (superseded) return superseded; - svc.status = 'stopped'; - delete svc.pid; + inst.status = 'stopped'; + delete inst.pid; clearStableTimer(rt); rt.consecutiveFastExits = 0; - return spawnAndWaitUntilListening(app, id, svc, rt, true, token); + return spawnAndWaitUntilListening(inst, rt, true, token); }); } if (!result.ok) { - return text(res, 500, result.message ?? 'failed to start the service'); + return text(res, 500, result.message ?? 'failed to start the instance'); } - res.writeHead(204); - res.end(); + noContent(res); } /** Byte offset a follow should start at to show the last `tailLines` complete lines; `<= 0` (or an empty file) starts at the current end, no backlog. */ @@ -843,10 +847,10 @@ function main(): void { const tick = (): void => { if (stopped) return; - // A fresh service start cleared the file — resume from its new - // beginning. Watching the clear generation (not just a size decrease) - // catches it even when the clear and the first new write both land - // between two ticks. + // A fresh start cleared the file — resume from its new beginning. + // Watching the clear generation (not just a size decrease) catches it + // even when the clear and the first new write both land between two + // ticks. const currentGeneration = logGeneration.get(logPath) ?? 0; if (currentGeneration !== generation) { generation = currentGeneration; @@ -878,6 +882,52 @@ function main(): void { res.on('close', stop); } + function handleLogs(url: URL, res: http.ServerResponse, inst: InstanceRecord): void { + if (url.searchParams.get('follow') === '1') { + const tailParam = url.searchParams.get('tail'); + const tailLines = tailParam === null ? 0 : Math.max(0, Number.parseInt(tailParam, 10) || 0); + handleFollowLogs(res, inst.logPath, tailLines); + return; + } + const content = fs.existsSync(inst.logPath) ? fs.readFileSync(inst.logPath) : Buffer.alloc(0); + res.writeHead(200, { 'content-type': 'text/plain; charset=utf-8' }); + res.end(content); + } + + /** The instances `ids` name, or the ids no instance has. */ + function instancesByIds( + ids: readonly string[], + ): { readonly found: InstanceRecord[] } | { readonly unknown: string[] } { + const unknown = ids.filter((id) => state.instances[id] === undefined); + if (unknown.length > 0) return { unknown }; + return { found: ids.flatMap((id) => state.instances[id] ?? []) }; + } + + function unknownInstances(res: http.ServerResponse, ids: readonly string[]): void { + text(res, 404, `no such instance: ${ids.map((id) => `"${id}"`).join(', ')}`); + } + + async function handleBulk( + req: http.IncomingMessage, + res: http.ServerResponse, + action: 'start' | 'stop', + ): Promise { + const body = await readJsonBody(req); + if (!isIdsBody(body)) return text(res, 400, 'malformed body: expected { "ids": string[] }'); + if (action === 'stop') { + // An unknown instance runs nothing, so stopping it is already done. + const known = body.ids.flatMap((id) => state.instances[id] ?? []); + await Promise.all(known.map((inst) => stopInstance(inst, getRuntime(inst.id)))); + return noContent(res); + } + const selected = instancesByIds(body.ids); + if ('unknown' in selected) return unknownInstances(res, selected.unknown); + const results = await Promise.all(selected.found.map(startInstance)); + const failures = results.flatMap((r) => (r.ok ? [] : [r.message ?? 'failed to start'])); + if (failures.length > 0) return text(res, 500, failures.join('\n')); + noContent(res); + } + async function handleRequest(req: http.IncomingMessage, res: http.ServerResponse): Promise { const url = new URL(req.url ?? '/', `http://127.0.0.1:${String(port)}`); const method = req.method ?? 'GET'; @@ -890,111 +940,52 @@ function main(): void { return json(res, 200, { version: ownVersion }); } - if (segments[0] === 'apps' && segments.length >= 2) { - const app = segments[1]; - if (app === undefined || !isValidSegment(app)) return badSegment(res, app ?? ''); - - if (method === 'PUT' && segments.length === 4 && segments[2] === 'services') { - const id = segments[3]; - if (id === undefined || !isValidSegment(id)) return badSegment(res, id ?? ''); - const svc = await getOrCreateService(app, id); - return json(res, 200, { port: svc.port, url: `http://localhost:${String(svc.port)}` }); - } - - if ( - method === 'PUT' && - segments.length === 5 && - segments[2] === 'services' && - segments[4] === 'deployment' - ) { - const id = segments[3]; - if (id === undefined || !isValidSegment(id)) return badSegment(res, id ?? ''); - return handleDeployment(req, res, app, id); - } - - if (method === 'GET' && segments.length === 3 && segments[2] === 'services') { - const appRec = state[app]; - const list = appRec ? Object.values(appRec.services).map(serviceView) : []; - return json(res, 200, list); - } - - if ( - method === 'GET' && - segments.length === 5 && - segments[2] === 'services' && - segments[4] === 'logs' - ) { - const id = segments[3]; - if (id === undefined || !isValidSegment(id)) return badSegment(res, id ?? ''); - const svc = state[app]?.services[id]; - if (!svc) return text(res, 404, `no such service "${id}" in app "${app}"`); - if (url.searchParams.get('follow') === '1') { - const tailParam = url.searchParams.get('tail'); - const tailLines = - tailParam === null ? 0 : Math.max(0, Number.parseInt(tailParam, 10) || 0); - handleFollowLogs(res, svc.logPath, tailLines); - return; - } - const content = fs.existsSync(svc.logPath) ? fs.readFileSync(svc.logPath) : Buffer.alloc(0); - res.writeHead(200, { 'content-type': 'text/plain; charset=utf-8' }); - res.end(content); - return; - } + if (segments[0] !== 'instances') { + res.writeHead(404); + res.end(); + return; + } - if (method === 'POST' && segments.length === 3 && segments[2] === 'stop') { - const appRec = state[app]; - if (appRec) { - await Promise.all( - Object.entries(appRec.services).map(([id, svc]) => - stopService(svc, getRuntime(app, id)), - ), + if (segments.length === 1) { + if (method === 'GET') return json(res, 200, Object.values(state.instances).map(instanceView)); + if (method === 'POST') { + const body = await readJsonBody(req); + if (!isCreateBody(body)) { + return text( + res, + 400, + 'malformed body: expected { "labels": Record, "port"?: number }', ); } - res.writeHead(204); - res.end(); - return; + return json(res, 201, instanceView(await createInstance(body))); } + } - if (method === 'POST' && segments.length === 3 && segments[2] === 'start') { - const appRec = state[app]; - const results = appRec - ? await Promise.all( - Object.entries(appRec.services).map(([id, svc]) => startService(app, id, svc)), - ) - : []; - const failures = results.flatMap((r) => (r.ok ? [] : [r.message ?? 'failed to start'])); - if (failures.length > 0) return text(res, 500, failures.join('\n')); - res.writeHead(204); - res.end(); - return; - } + if (method === 'POST' && segments.length === 2 && segments[1] === 'start') { + return handleBulk(req, res, 'start'); + } + if (method === 'POST' && segments.length === 2 && segments[1] === 'stop') { + return handleBulk(req, res, 'stop'); + } - if (method === 'DELETE' && segments.length === 2) { - const appRec = state[app]; - if (appRec) { - for (const id of Object.keys(appRec.services)) getRuntime(app, id).deleted = true; - await Promise.all( - Object.entries(appRec.services).map(([id, svc]) => - stopService(svc, getRuntime(app, id)), - ), - ); - for (const id of Object.keys(appRec.services)) { - const key = runtimeKey(app, id); - runtimes.get(key)?.log?.close(); - runtimes.delete(key); - } - delete state[app]; - schedulePersist(); - // get-port holds every port it returns for up to 30 seconds; the - // freed ports must be free again for a `--fresh` restart. - // `usedServicePorts()` already keeps other apps' ports out. - clearLockedPorts(); - } - await fs.promises.rm(path.join(stateDir, 'logs', app), { recursive: true, force: true }); - res.writeHead(204); - res.end(); - return; - } + const id = segments[1]; + if (id !== undefined && !INSTANCE_ID_RE.test(id)) { + return text(res, 400, `invalid instance id "${id}"`); + } + const inst = id === undefined ? undefined : state.instances[id]; + + if (method === 'DELETE' && segments.length === 2) { + if (inst) await deleteInstance(inst); + return noContent(res); + } + if (!inst) return unknownInstances(res, id === undefined ? [] : [id]); + + if (method === 'GET' && segments.length === 2) return json(res, 200, instanceView(inst)); + if (method === 'PUT' && segments.length === 3 && segments[2] === 'deployment') { + return handleDeployment(req, res, inst); + } + if (method === 'GET' && segments.length === 3 && segments[2] === 'logs') { + return handleLogs(url, res, inst); } res.writeHead(404); @@ -1029,19 +1020,18 @@ function main(): void { process.on('SIGTERM', () => void shutdown()); process.on('SIGINT', () => void shutdown()); - readJsonFile(appsJsonPath, isAppsState) + discardStateFromEarlierVersions(stateDir, statePath, logsDir) + .then(() => readJsonFile(statePath, isInstancesState)) .then((loaded) => { if (loaded) { // A fresh process has no live children to supervise. `held` is an // operator-visible signal and survives; `running`/`backoff` assumed a // child this process never spawned, so they reset to `stopped` — the // next deployment PUT always starts fresh (spec's "ALWAYS starts"). - for (const appRec of Object.values(loaded)) { - for (const svc of Object.values(appRec.services)) { - if (svc.status === 'running' || svc.status === 'backoff') { - svc.status = 'stopped'; - delete svc.pid; - } + for (const inst of Object.values(loaded.instances)) { + if (inst.status === 'running' || inst.status === 'backoff') { + inst.status = 'stopped'; + delete inst.pid; } } state = loaded; @@ -1061,4 +1051,30 @@ function main(): void { }); } +/** + * Removes state files this version does not write, and log directories: an + * earlier version kept its state in another JSON file and grouped its logs in + * directories. Local dev state is never migrated across versions; those + * instances are simply gone. + */ +async function discardStateFromEarlierVersions( + stateDir: string, + statePath: string, + logsDir: string, +): Promise { + const stateFiles = await fs.promises.readdir(stateDir).catch(() => []); + await Promise.all( + stateFiles + .map((name) => path.join(stateDir, name)) + .filter((file) => file.endsWith('.json') && file !== statePath) + .map((file) => fs.promises.rm(file, { force: true })), + ); + const logEntries = await fs.promises.readdir(logsDir, { withFileTypes: true }).catch(() => []); + await Promise.all( + logEntries + .filter((entry) => entry.isDirectory()) + .map((entry) => fs.promises.rm(path.join(logsDir, entry.name), { recursive: true })), + ); +} + main(); diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/exports/index.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/exports/index.ts index 55af18be1..46a39ae16 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/exports/index.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/exports/index.ts @@ -8,10 +8,12 @@ export type { BucketsClient, ComputeClient, + CreateInstanceRequest, DatabaseInfo, DeploymentRequest, + InstanceInfo, + InstanceStatus, PostgresClient, - ServiceInfo, } from '../client.ts'; export { bucketsClient, computeClient, postgresClient } from '../client.ts'; export type { DaemonName, DaemonRootOptions, RegistryEntry } from '../daemon.ts'; diff --git a/packages/1-prisma-cloud/0-lowering/local-target/package.json b/packages/1-prisma-cloud/0-lowering/local-target/package.json index 787124ace..f742bea68 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/package.json +++ b/packages/1-prisma-cloud/0-lowering/local-target/package.json @@ -24,6 +24,7 @@ "tar": "^7.5.21" }, "devDependencies": { + "@internal/foundation": "workspace:0.30.0", "@internal/tsdown-config": "workspace:0.30.0", "@types/bun": "^1.3.13", "@types/node": "^26.0.1", diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-port-order.test.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-port-order.test.ts index ecdbc46b9..acd214585 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-port-order.test.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-port-order.test.ts @@ -4,21 +4,19 @@ import * as os from 'node:os'; import * as path from 'node:path'; import { fileURLToPath } from 'node:url'; import type { ContainerInstance } from '@internal/core/config'; -import { - computeClient, - EMULATORS_DIR_ENV, - ensureDaemon, - stopDaemon, -} from '@internal/dev-emulators'; +import { EMULATORS_DIR_ENV, ensureDaemon, stopDaemon } from '@internal/dev-emulators'; import { App } from 'alchemy/Prisma'; import * as Effect from 'effect/Effect'; -import { LocalAppProvider, reserveServicePorts } from '../compute.ts'; +import { ensureServiceInstances, LocalAppProvider } from '../compute.ts'; +import { computeNodeInstances, ensureComputeInstance } from '../emulator-instances.ts'; const savedEmulatorsDir = process.env[EMULATORS_DIR_ENV]; let registryRoot: string; +let devDir: string; beforeEach(async () => { registryRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'local-target-port-order-')); + devDir = path.join(registryRoot, 'project', '.prisma-composer', 'dev'); process.env[EMULATORS_DIR_ENV] = registryRoot; const entry = fileURLToPath(import.meta.resolve('@internal/dev-emulators/compute-main')); await ensureDaemon('compute', entry, { registryRoot }); @@ -49,7 +47,7 @@ const reconcileInput = (id: string, news: Record) => async function reconcileApp(container: ContainerInstance, address: string): Promise { const provider = await Effect.runPromise( - App.Provider.pipe(Effect.provide(LocalAppProvider({ container, devDir: '/dev/null/unused' }))), + App.Provider.pipe(Effect.provide(LocalAppProvider({ container, devDir }))), ); const attributes: App['Attributes'] = await Effect.runPromise( provider.reconcile( @@ -59,16 +57,16 @@ async function reconcileApp(container: ContainerInstance, address: string): Prom return Number(new URL(attributes.appEndpointDomain ?? '').port); } -async function reservedPorts(appName: string): Promise> { - const services = await computeClient().listServices(appName); - return Object.fromEntries(services.map((svc) => [svc.id, svc.port])); +async function reservedPorts(app: string): Promise> { + const nodes = await computeNodeInstances({ devDir, app }); + return Object.fromEntries(nodes.map(({ address, instance }) => [address, instance.port])); } test('each App provider returns the port reserved for its service, even when the providers race', async () => { for (let run = 0; run < 10; run++) { const appName = `port-order-${String(run)}`; const container = fakeContainer(appName); - await reserveServicePorts(container, ['quotes', 'gateway']); + await ensureServiceInstances(container, devDir, ['quotes', 'gateway']); const reserved = await reservedPorts(appName); const [gateway, quotes] = await Promise.all([ @@ -84,28 +82,32 @@ test('each App provider returns the port reserved for its service, even when the }); test('a failed reservation names the service', async () => { - await expect(reserveServicePorts(fakeContainer('Bad-App'), ['quotes'])).rejects.toThrow( - 'service "quotes"', - ); + await stopDaemon('compute', { registryRoot }); + + await expect( + ensureServiceInstances(fakeContainer('port-order-down'), devDir, ['quotes']), + ).rejects.toThrow('service "quotes"'); }); test('reserving the services of an app that already has ports keeps those ports', async () => { const container = fakeContainer('port-order-warm'); - const client = computeClient(); - const gateway = await client.ensureService('port-order-warm', 'gateway'); - const quotes = await client.ensureService('port-order-warm', 'quotes'); + const scope = { devDir, app: 'port-order-warm' }; + const { instance: gateway } = await ensureComputeInstance(scope, 'gateway'); + const { instance: quotes } = await ensureComputeInstance(scope, 'quotes'); - await reserveServicePorts(container, ['quotes', 'gateway']); + await ensureServiceInstances(container, devDir, ['quotes', 'gateway']); expect(await reconcileApp(container, 'quotes')).toBe(quotes.port); expect(await reconcileApp(container, 'gateway')).toBe(gateway.port); }); -test('a dotted service address reserves the same emulator service the App provider uses', async () => { +test('a dotted service address reserves the same emulator instance the App provider uses', async () => { const container = fakeContainer('port-order-nested'); - await reserveServicePorts(container, ['shop.quotes']); + await ensureServiceInstances(container, devDir, ['shop.quotes']); - const listed = await computeClient().listServices('port-order-nested'); + const listed = (await computeNodeInstances({ devDir, app: 'port-order-nested' })).map( + (node) => node.instance, + ); expect(await reconcileApp(container, 'shop.quotes')).toBe(listed[0]?.port ?? -1); expect(listed).toHaveLength(1); }); diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts new file mode 100644 index 000000000..51ee8d033 --- /dev/null +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts @@ -0,0 +1,435 @@ +/** + * The per-directory record of which Compute instance backs each service, + * against a real Compute emulator in a temporary emulators directory. + */ +import { afterEach, beforeEach, describe, expect, test } from 'bun:test'; +import * as fs from 'node:fs'; +import * as os from 'node:os'; +import * as path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import type { ContainerInstance } from '@internal/core/config'; +import { + computeClient, + EMULATORS_DIR_ENV, + ensureDaemon, + stopDaemon, +} from '@internal/dev-emulators'; +import { CliStructuredError } from '@internal/foundation/errors'; +import { packageComputeArtifact } from '@internal/lowering/compute'; +import { App, Deployment } from 'alchemy/Prisma'; +import * as Effect from 'effect/Effect'; +import { ensureServiceInstances, LocalAppProvider, LocalDeploymentProvider } from '../compute.ts'; +import { + deleteRecordedInstances, + EMULATOR_INSTANCES_FILE, + type EmulatorInstances, + readEmulatorInstances, + retirePreviousApp, + withServiceAddresses, +} from '../emulator-instances.ts'; + +const APP = 'shared-app'; +const ADDRESSES = ['quotes', 'gateway']; +const savedEmulatorsDir = process.env[EMULATORS_DIR_ENV]; +let root: string; +let registryRoot: string; + +const container: ContainerInstance = { + input: { appName: APP, stage: undefined }, + serialize: () => 'x', +}; + +function projectDir(name: string): { readonly directory: string; readonly devDir: string } { + const directory = path.join(root, name); + return { directory, devDir: path.join(directory, '.prisma-composer', 'dev') }; +} + +/** A packaged artifact whose service answers with its address, its working directory and `version`. */ +function artifactFor(address: string, version: string): string { + const bundleDir = fs.mkdtempSync(path.join(root, `bundle-${address}-`)); + fs.writeFileSync( + path.join(bundleDir, 'main.js'), + `export default { run: async (address) => { + const port = Number(process.env['COMPOSER_' + address.toUpperCase() + '_PORT']); + Bun.serve({ port, fetch: () => new Response(address + ' ' + process.cwd() + ' ${version}') }); +} }; +`, + ); + fs.writeFileSync(path.join(bundleDir, 'server.js'), ''); + return packageComputeArtifact({ id: address, bundleDir, appEntry: 'server.js', address }).path; +} + +/** A packaged artifact whose service exits before it listens. */ +function crashingArtifactFor(address: string): string { + const bundleDir = fs.mkdtempSync(path.join(root, `bundle-${address}-`)); + fs.writeFileSync( + path.join(bundleDir, 'main.js'), + 'export default { run: async () => { process.exit(1); } };\n', + ); + fs.writeFileSync(path.join(bundleDir, 'server.js'), ''); + return packageComputeArtifact({ id: address, bundleDir, appEntry: 'server.js', address }).path; +} + +const reconcileInput = (id: string, news: Record) => + ({ + id, + fqn: id, + instanceId: id, + news, + olds: undefined, + output: undefined, + session: undefined as never, + bindings: [], + }) as never; + +async function deploy( + devDir: string, + address: string, + version: string, + artifactPath: string = artifactFor(address, version), +): Promise { + const provider = await Effect.runPromise( + Deployment.Provider.pipe(Effect.provide(LocalDeploymentProvider({ container, devDir }))), + ); + return Effect.runPromise( + provider.reconcile( + reconcileInput(`${address}-deployment`, { + app: address, + artifactPath, + }), + ), + ); +} + +async function deploymentDiff(devDir: string, output: Deployment['Attributes']): Promise { + const provider = await Effect.runPromise( + Deployment.Provider.pipe(Effect.provide(LocalDeploymentProvider({ container, devDir }))), + ); + const diff = provider.diff; + if (!diff) throw new Error('the local Deployment provider has no diff'); + return Effect.runPromise(diff({ output } as never)); +} + +async function appProvider(devDir: string) { + return Effect.runPromise( + App.Provider.pipe(Effect.provide(LocalAppProvider({ container, devDir }))), + ); +} + +function mappingOf(directory: string): Promise { + return readEmulatorInstances(directory); +} + +async function nodeOf(directory: string, address: string) { + const node = (await mappingOf(directory))?.nodes[address]; + if (!node) throw new Error(`no instance recorded for "${address}" in ${directory}`); + return node; +} + +async function pidOf(id: string): Promise { + return (await computeClient().getInstance(id))?.pid; +} + +describe('errors name services by address', () => { + const nodes = [{ address: 'shop.quotes', instance: { id: 'svc_0123456789abcdef' } }]; + + test('an instance id in a plain error becomes the service address, keeping the original as cause', () => { + const original = new Error( + 'request failed (500): instance "svc_0123456789abcdef" exited (code 1) before it accepted connections', + ); + + const rewritten = withServiceAddresses(original, nodes); + + expect(rewritten).toBeInstanceOf(Error); + expect((rewritten as Error).message).toBe( + 'request failed (500): service "shop.quotes" exited (code 1) before it accepted connections', + ); + expect((rewritten as Error).cause).toBe(original); + }); + + test("the emulator's listens hint becomes the compute() option, alongside the address", () => { + const original = new Error( + 'request failed (500): instance "svc_0123456789abcdef" did not accept connections on its port 3001 within 30s. If it never listens on its port, deploy it with listens: false.', + ); + + const rewritten = withServiceAddresses(original, nodes); + + expect((rewritten as Error).message).toBe( + 'request failed (500): service "shop.quotes" did not accept connections on its port 3001 within 30s. If it never listens on its port, declare it with compute({ listens: false }).', + ); + }); + + test('a structured error is left untouched', () => { + const structured = new CliStructuredError( + 'DEV.EMULATOR_UNREACHABLE', + 'instance "svc_0123456789abcdef" unreachable', + ); + + expect(withServiceAddresses(structured, nodes)).toBe(structured); + }); +}); + +// Local dev does not support Windows, so neither do the tests that start a real Compute emulator. +describe.skipIf(process.platform === 'win32')('against a real Compute emulator', () => { + beforeEach(async () => { + root = fs.mkdtempSync(path.join(os.tmpdir(), 'local-target-emulator-instances-')); + registryRoot = path.join(root, 'emulators'); + process.env[EMULATORS_DIR_ENV] = registryRoot; + const entry = fileURLToPath(import.meta.resolve('@internal/dev-emulators/compute-main')); + await ensureDaemon('compute', entry, { registryRoot }); + }); + + afterEach(async () => { + await stopDaemon('compute', { registryRoot }).catch(() => undefined); + fs.rmSync(root, { recursive: true, force: true }); + if (savedEmulatorsDir === undefined) delete process.env[EMULATORS_DIR_ENV]; + else process.env[EMULATORS_DIR_ENV] = savedEmulatorsDir; + }); + + describe('two directories running the same app', () => { + test('each gets its own instances, and a deploy in one restarts nothing in the other', async () => { + const a = projectDir('a'); + const b = projectDir('b'); + await Promise.all([ + ensureServiceInstances(container, a.devDir, ADDRESSES), + ensureServiceInstances(container, b.devDir, ADDRESSES), + ]); + await Promise.all( + [a, b].flatMap(({ devDir }) => ADDRESSES.map((address) => deploy(devDir, address, 'v1'))), + ); + + const nodesA = (await mappingOf(a.directory))?.nodes ?? {}; + const nodesB = (await mappingOf(b.directory))?.nodes ?? {}; + const ids = [...Object.values(nodesA), ...Object.values(nodesB)].map((n) => n.id); + const ports = [...Object.values(nodesA), ...Object.values(nodesB)].map((n) => n.port); + expect(new Set(ids).size).toBe(4); + expect(new Set(ports).size).toBe(4); + for (const [{ directory }, nodes] of [ + [a, nodesA], + [b, nodesB], + ] as const) { + for (const [address, node] of Object.entries(nodes)) { + const answer = await (await fetch(`http://localhost:${String(node.port)}`)).text(); + expect(answer).toStartWith(`${address} ${fs.realpathSync(directory)}`); + } + } + + const pidsA = await Promise.all(Object.values(nodesA).map((n) => pidOf(n.id))); + await Promise.all(ADDRESSES.map((address) => deploy(b.devDir, address, 'v2'))); + + expect(await Promise.all(Object.values(nodesA).map((n) => pidOf(n.id)))).toEqual(pidsA); + const quotesB = await nodeOf(b.directory, 'quotes'); + expect(await (await fetch(`http://localhost:${String(quotesB.port)}`)).text()).toEndWith( + ' v2', + ); + }, 60_000); + }); + + describe('reusing the recorded instances', () => { + test('a recorded instance the emulator no longer has is re-created on its remembered port', async () => { + const { directory, devDir } = projectDir('lost'); + await ensureServiceInstances(container, devDir, ['quotes']); + const before = await nodeOf(directory, 'quotes'); + await computeClient().deleteInstance(before.id); + + const changes = await ensureServiceInstances(container, devDir, ['quotes']); + + const after = await nodeOf(directory, 'quotes'); + expect(after.id).not.toBe(before.id); + expect(after.port).toBe(before.port); + expect(changes).toEqual([]); + }); + + test('a remembered port another instance now holds gives a new port, reported as a change', async () => { + const { directory, devDir } = projectDir('moved'); + await ensureServiceInstances(container, devDir, ['quotes']); + const before = await nodeOf(directory, 'quotes'); + await computeClient().deleteInstance(before.id); + await computeClient().createInstance({ labels: {}, port: before.port }); + + const changes = await ensureServiceInstances(container, devDir, ['quotes']); + + const after = await nodeOf(directory, 'quotes'); + expect(after.port).not.toBe(before.port); + expect(changes).toEqual([{ address: 'quotes', previous: before.port, given: after.port }]); + }); + + test('a warm restart after a stop reuses the same instances and ports', async () => { + const { directory, devDir } = projectDir('warm'); + await ensureServiceInstances(container, devDir, ADDRESSES); + for (const address of ADDRESSES) await deploy(devDir, address, 'v1'); + const before = (await mappingOf(directory))?.nodes; + const ids = Object.values(before ?? {}).map((n) => n.id); + await computeClient().stopInstances(ids); + + const changes = await ensureServiceInstances(container, devDir, ADDRESSES); + await computeClient().startInstances(ids); + + expect(changes).toEqual([]); + expect((await mappingOf(directory))?.nodes).toEqual(before ?? {}); + for (const node of Object.values(before ?? {})) { + expect((await computeClient().getInstance(node.id))?.status).toBe('running'); + } + }, 30_000); + + test('a deployment whose instance was re-created is deployed again even when nothing changed', async () => { + const { directory, devDir } = projectDir('redeploy'); + await ensureServiceInstances(container, devDir, ['quotes']); + const deployed = await deploy(devDir, 'quotes', 'v1'); + expect(await deploymentDiff(devDir, deployed)).toBeUndefined(); + + await computeClient().deleteInstance((await nodeOf(directory, 'quotes')).id); + await ensureServiceInstances(container, devDir, ['quotes']); + + expect(await deploymentDiff(devDir, deployed)).toEqual({ action: 'update' }); + }, 30_000); + + test("deleting a removed service's App deletes its instance and forgets it", async () => { + const { directory, devDir } = projectDir('removed'); + await ensureServiceInstances(container, devDir, ADDRESSES); + const quotes = await nodeOf(directory, 'quotes'); + const provider = await appProvider(devDir); + + await Effect.runPromise(provider.delete({ output: { name: 'quotes' } } as never)); + + expect(await computeClient().getInstance(quotes.id)).toBeUndefined(); + expect(Object.keys((await mappingOf(directory))?.nodes ?? {})).toEqual(['gateway']); + }); + }); + + test('the record is one JSON file in the dev directory, readable from the directory alone', async () => { + const { directory, devDir } = projectDir('file'); + await ensureServiceInstances(container, devDir, ['quotes']); + + const onDisk: unknown = JSON.parse( + fs.readFileSync(path.join(devDir, EMULATOR_INSTANCES_FILE), 'utf8'), + ); + const node = await nodeOf(directory, 'quotes'); + expect(onDisk).toEqual({ app: APP, nodes: { quotes: node } }); + expect((await computeClient().getInstance(node.id))?.labels).toEqual({ + 'composer.directory': directory, + 'composer.app': APP, + 'composer.address': 'quotes', + }); + expect(await readEmulatorInstances(path.join(root, 'never-ran'))).toBeUndefined(); + }); + + describe('switching apps in a directory', () => { + test("running another app deletes the previous app's compute instances and starts a new record", async () => { + const { directory, devDir } = projectDir('switch'); + await ensureServiceInstances(container, devDir, ADDRESSES); + const previous = Object.values((await mappingOf(directory))?.nodes ?? {}); + + const retired = await retirePreviousApp({ devDir, app: 'another-app' }); + + expect(retired).toBe(APP); + for (const node of previous) + expect(await computeClient().getInstance(node.id)).toBeUndefined(); + expect(await mappingOf(directory)).toEqual({ app: 'another-app', nodes: {} }); + }); + + test('running the same app again retires nothing', async () => { + const { directory, devDir } = projectDir('same'); + await ensureServiceInstances(container, devDir, ['quotes']); + const quotes = await nodeOf(directory, 'quotes'); + + expect(await retirePreviousApp({ devDir, app: APP })).toBeUndefined(); + expect(await computeClient().getInstance(quotes.id)).toBeDefined(); + }); + }); + + describe('--fresh', () => { + const otherApp: ContainerInstance = { + input: { appName: 'other-app', stage: undefined }, + serialize: () => 'x', + }; + + async function liveIds(ids: readonly string[]): Promise { + const live = await Promise.all(ids.map((id) => computeClient().getInstance(id))); + return live.flatMap((instance) => (instance ? [instance.id] : [])); + } + + test('deletes the recorded instances even when the record names the app run here before', async () => { + const { directory, devDir } = projectDir('fresh-previous'); + await ensureServiceInstances(container, devDir, ADDRESSES); + const ids = Object.values((await mappingOf(directory))?.nodes ?? {}).map((n) => n.id); + + await deleteRecordedInstances(devDir); + + expect(await liveIds(ids)).toEqual([]); + expect((await mappingOf(directory))?.nodes).toEqual({}); + }); + + test('after switching apps, leaves no instance of either app', async () => { + const { directory, devDir } = projectDir('fresh-switch'); + await ensureServiceInstances(container, devDir, ADDRESSES); + const first = Object.values((await mappingOf(directory))?.nodes ?? {}).map((n) => n.id); + await retirePreviousApp({ devDir, app: 'other-app' }); + await ensureServiceInstances(otherApp, devDir, ['web']); + const second = Object.values((await mappingOf(directory))?.nodes ?? {}).map((n) => n.id); + + await deleteRecordedInstances(devDir); + + expect(await liveIds([...first, ...second])).toEqual([]); + }); + }); + + describe("the App provider's diff", () => { + async function reconcileApp(devDir: string, address: string): Promise { + const provider = await appProvider(devDir); + return Effect.runPromise( + provider.reconcile( + reconcileInput(`${address}-svc`, { project: 'local', displayName: address }), + ), + ); + } + + async function appDiff(devDir: string, output: App['Attributes']): Promise { + const diff = (await appProvider(devDir)).diff; + if (!diff) throw new Error('the local App provider has no diff'); + return Effect.runPromise(diff({ output } as never)); + } + + test('asks for an update when the re-created instance got another port', async () => { + const { directory, devDir } = projectDir('app-moved'); + await ensureServiceInstances(container, devDir, ['quotes']); + const before = await reconcileApp(devDir, 'quotes'); + const recorded = await nodeOf(directory, 'quotes'); + await computeClient().deleteInstance(recorded.id); + await computeClient().createInstance({ labels: {}, port: recorded.port }); + await ensureServiceInstances(container, devDir, ['quotes']); + + expect(await appDiff(devDir, before)).toEqual({ action: 'update' }); + const after = await reconcileApp(devDir, 'quotes'); + expect(after.appEndpointDomain).toBe( + `http://localhost:${String((await nodeOf(directory, 'quotes')).port)}`, + ); + expect(after.appEndpointDomain).not.toBe(before.appEndpointDomain); + }); + + test('asks for nothing when the re-created instance kept its port', async () => { + const { directory, devDir } = projectDir('app-kept'); + await ensureServiceInstances(container, devDir, ['quotes']); + const before = await reconcileApp(devDir, 'quotes'); + await computeClient().deleteInstance((await nodeOf(directory, 'quotes')).id); + await ensureServiceInstances(container, devDir, ['quotes']); + + expect(await appDiff(devDir, before)).toBeUndefined(); + }); + }); + + describe('errors name services by address', () => { + test('a deployed service that exits before it listens fails naming its address', async () => { + const { devDir } = projectDir('crash'); + await ensureServiceInstances(container, devDir, ['quotes']); + + const failure = await deploy(devDir, 'quotes', 'v1', crashingArtifactFor('quotes')).then( + () => undefined, + (error: unknown) => (error instanceof Error ? error.message : String(error)), + ); + + expect(failure).toContain('service "quotes" exited (code 1) before it accepted connections'); + expect(failure).not.toContain('instance "svc_'); + }, 30_000); + }); +}); diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts index d64a9f0fa..aa8c0cf8d 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts @@ -21,6 +21,14 @@ import * as Redacted from 'effect/Redacted'; import { appNameOf } from './app-name.ts'; import { extractComputeArtifact } from './artifact-extract.ts'; import { envStore, secretsStore } from './dev-store.ts'; +import { + type AppScope, + deleteComputeInstance, + ensureComputeInstance, + liveComputeInstance, + type PortChange, + withServiceAddresses, +} from './emulator-instances.ts'; import { DEV_TIMESTAMP, projectIdOfInput } from './upstream-attributes.ts'; /** Reads an app id from upstream's `app` input: a plain string or a resolved `Prisma.App` attributes record. */ @@ -154,20 +162,6 @@ export function readComputeManifest(artifactDir: string): { return { address: parsed.address, listens: parsed.listens !== false }; } -/** - * A service address is dot-separated (`"orders.service"`) but the emulator's - * `` segment must match `/^[a-z0-9][a-z0-9-]*$/`: disallowed characters - * become dashes, runs collapse. The real address still rides the deployment - * body's `address` field untouched, so listings show it verbatim. - */ -function slugServiceId(address: string): string { - const slug = address - .toLowerCase() - .replace(/[^a-z0-9]+/g, '-') - .replace(/^-+|-+$/g, ''); - return slug.length > 0 ? slug : 'svc'; -} - async function materializeEnv( devDir: string, address: string, @@ -182,12 +176,20 @@ async function materializeEnv( return env; } +function appScopeOf(input: LocalTargetProvidersInput): AppScope { + return { devDir: input.devDir, app: appNameOf(input.container) }; +} + +function localUrl(port: number): string { + return `http://localhost:${String(port)}`; +} + /** - * `Prisma.App` → the Compute emulator: reserves (or returns) the service's - * stable port. The app id here IS the service's own address, which - * `Deployment` slugs back into the emulator's id. `delete` is a no-op — - * instance removal belongs to `teardown` (`DELETE /apps/`), not - * per-resource Alchemy deletes. + * `Prisma.App` → the Compute emulator: finds or creates the instance that + * runs the service, which holds the service's port. The app id here IS the + * service's own address. An App whose instance was re-created on another + * port is updated, so its dependents get the new URL. Deleting an App (its + * service was removed) deletes its instance. */ export function LocalAppProvider( input: LocalTargetProvidersInput, @@ -197,9 +199,9 @@ export function LocalAppProvider( reconcile: ({ id, news }) => Effect.tryPromise({ try: async () => { - const appName = appNameOf(input.container); const name = news.displayName ?? id; - const { url } = await computeClient().ensureService(appName, slugServiceId(name)); + const { instance } = await ensureComputeInstance(appScopeOf(input), name); + const url = localUrl(instance.port); return { appId: name, name, @@ -214,34 +216,53 @@ export function LocalAppProvider( }, catch: (cause) => cause, }), - delete: () => Effect.void, + diff: ({ output }) => + Effect.tryPromise({ + try: async () => { + if (output === undefined) return undefined; + const live = await liveComputeInstance(appScopeOf(input), output.name); + if (live?.url === output.appEndpointDomain) return undefined; + return { action: 'update' as const }; + }, + catch: (cause) => cause, + }), + delete: ({ output }) => + Effect.tryPromise({ + try: () => deleteComputeInstance(appScopeOf(input), output.name), + catch: (cause) => cause, + }), read: ({ output }) => Effect.succeed(output), }; return Provider.effect(App, Effect.succeed(service)); } /** - * Reserves each service's emulator port one at a time, in the order given, - * before Alchemy applies the `App` resources concurrently. A new service gets + * Ensures each service's emulator instance one at a time, in the order given, + * before Alchemy applies the `App` resources concurrently. A new instance gets * the smallest free port, so on a fresh start the ports follow this order; a - * service that already has a port keeps it. `serviceAppNames` are the - * `App` resources' `displayName`s. + * service that already has an instance keeps its port. `serviceAppNames` are + * the `App` resources' `displayName`s. Returns the services whose instance had + * to be re-created and could not get its remembered port back. */ -export async function reserveServicePorts( +export async function ensureServiceInstances( container: ContainerInstance | undefined, + devDir: string, serviceAppNames: readonly string[], -): Promise { - const appName = appNameOf(container); +): Promise { + const scope = { devDir, app: appNameOf(container) }; + const changes: PortChange[] = []; for (const service of serviceAppNames) { try { - await computeClient().ensureService(appName, slugServiceId(service)); + const { portChange } = await ensureComputeInstance(scope, service); + if (portChange) changes.push(portChange); } catch (cause) { const reason = cause instanceof Error ? cause.message : String(cause); - throw new Error(`reserving a local port for service "${service}" failed: ${reason}`, { + throw new Error(`ensuring the local service for service "${service}" failed: ${reason}`, { cause, }); } } + return changes; } /** `Prisma.EnvironmentVariable` → a key/value row in `/env.json`. Upstream's value is `Redacted`; env.json holds the plain string the child process is given. */ @@ -288,8 +309,8 @@ export function LocalEnvironmentVariableProvider( } /** - * `Prisma.Deployment` → unpacks the artifact once per hash, fetches the - * emulator's assigned port, materializes the child's full env (env store + + * `Prisma.Deployment` → unpacks the artifact once per hash, finds the + * service's emulator instance and its port, materializes the child's full env (env store + * secrets + the port override + `PATH`/`HOME`), and puts the deployment — the * emulator (re)starts the child only when the hash or env actually changed. * `portMapping` is ignored on purpose: the emulator owns port allocation, and @@ -304,9 +325,7 @@ export function LocalDeploymentProvider( reconcile: ({ news }) => Effect.tryPromise({ try: async (): Promise => { - const app = appNameOf(input.container); const appId = appIdOfInput(news.app); - const emulatorId = slugServiceId(appId); if (news.artifactPath === undefined) { throw new Error('local Deployment requires an artifactPath — nothing to run.'); } @@ -318,18 +337,20 @@ export function LocalDeploymentProvider( } const { address, listens } = readComputeManifest(artifactDir); - const { port } = await computeClient().ensureService(app, emulatorId); - const env = await materializeEnv(input.devDir, address, port); - await computeClient().putDeployment(app, emulatorId, { - address, - artifactDir, - artifactHash, - env, - port, - listens, - }); + const { instance } = await ensureComputeInstance(appScopeOf(input), appId); + const env = await materializeEnv(input.devDir, address, instance.port); + try { + await computeClient().putDeployment(instance.id, { + artifactDir, + artifactHash, + env, + listens, + }); + } catch (error) { + throw withServiceAddresses(error, [{ address, instance }]); + } - const url = `http://localhost:${port}`; + const url = localUrl(instance.port); return { deploymentId: artifactHash, appId, @@ -346,8 +367,21 @@ export function LocalDeploymentProvider( }, catch: (cause) => cause, }), + // An instance that no longer runs this deployment — re-created after the + // emulator lost it, or never deployed — needs the deployment again, even + // when nothing in the app changed. + diff: ({ output }) => + Effect.tryPromise({ + try: async () => { + if (output === undefined) return undefined; + const live = await liveComputeInstance(appScopeOf(input), output.appId); + if (live?.artifactHash === output.artifactHash) return undefined; + return { action: 'update' as const }; + }, + catch: (cause) => cause, + }), // Content-addressed, cheap to leave unpacked; `--fresh` removes the whole - // dev dir and `teardown` removes the emulator's instance. + // dev dir, and deleting the App deletes the emulator's instance. delete: () => Effect.void, }; return Provider.effect(Deployment, Effect.succeed(service)); diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/dev-store.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/dev-store.ts index cc05e0504..6cb297d1e 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/dev-store.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/dev-store.ts @@ -80,7 +80,7 @@ export interface DevStore { update(mutate: (current: T) => T): Promise; } -function createStore( +export function createStore( filePath: string, isValid: (value: unknown) => value is T, empty: T, diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/emulator-instances.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/emulator-instances.ts new file mode 100644 index 000000000..f16d8bb66 --- /dev/null +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/emulator-instances.ts @@ -0,0 +1,213 @@ +/** + * Which emulator instance backs each node of the app run from a directory. + * `prisma dev` records it in `/.prisma-composer/dev/`, and every + * Composer operation on an emulator instance goes through it. The emulators + * know nothing about Composer: the labels written on create are for people + * reading an emulator listing, and nothing here reads them back. + */ +import * as path from 'node:path'; +import { DEV_DIR } from '@internal/core/config'; +import { computeClient, type InstanceInfo } from '@internal/dev-emulators'; +import { createStore, type DevStore } from './dev-store.ts'; + +export const EMULATOR_INSTANCES_FILE = 'emulator-instances.json'; + +export interface ComputeNodeInstance { + readonly kind: 'compute'; + readonly id: string; + /** The port the instance was given, requested again if the instance has to be re-created. */ + readonly port: number; +} + +export type NodeInstance = ComputeNodeInstance; + +export interface EmulatorInstances { + readonly app: string; + /** Keyed by node address. */ + readonly nodes: Readonly>; +} + +/** The app run from the directory that holds `devDir` (`/.prisma-composer/dev`). */ +export interface AppScope { + readonly devDir: string; + readonly app: string; +} + +/** A service whose remembered port another instance or process now holds. */ +export interface PortChange { + readonly address: string; + readonly previous: number; + readonly given: number; +} + +function isNodeInstance(value: unknown): value is NodeInstance { + return ( + typeof value === 'object' && + value !== null && + 'kind' in value && + value.kind === 'compute' && + 'id' in value && + typeof value.id === 'string' && + 'port' in value && + typeof value.port === 'number' + ); +} + +function isEmulatorInstances(value: unknown): value is EmulatorInstances { + return ( + typeof value === 'object' && + value !== null && + 'app' in value && + typeof value.app === 'string' && + 'nodes' in value && + typeof value.nodes === 'object' && + value.nodes !== null && + Object.values(value.nodes).every(isNodeInstance) + ); +} + +function directoryOf(devDir: string): string { + const normalized = path.normalize(devDir); + return normalized.endsWith(DEV_DIR) ? normalized.slice(0, -DEV_DIR.length - 1) : normalized; +} + +function store(devDir: string): DevStore { + return createStore(path.join(devDir, EMULATOR_INSTANCES_FILE), isEmulatorInstances, undefined); +} + +/** The record `prisma dev` left in `directory`; undefined when it has not run there. */ +export function readEmulatorInstances(directory: string): Promise { + return store(path.join(directory, DEV_DIR)).read(); +} + +/** This app's nodes. A record left by another app run from the same directory counts as empty. */ +async function appNodes(scope: AppScope): Promise>> { + const current = await store(scope.devDir).read(); + return current?.app === scope.app ? current.nodes : {}; +} + +async function updateNodes( + scope: AppScope, + mutate: (nodes: Record) => void, +): Promise { + await store(scope.devDir).update((current) => { + const nodes = { ...(current?.app === scope.app ? current.nodes : {}) }; + mutate(nodes); + return { app: scope.app, nodes }; + }); +} + +/** + * A directory has one dev instance. When the record names another app, that + * app's compute instances are deleted and the record starts over for this + * one. Returns the previous app's name when it was retired. + */ +export async function retirePreviousApp(scope: AppScope): Promise { + const current = await store(scope.devDir).read(); + if (current === undefined || current.app === scope.app) return undefined; + const client = computeClient(); + for (const instance of Object.values(current.nodes)) await client.deleteInstance(instance.id); + await store(scope.devDir).update(() => ({ app: scope.app, nodes: {} })); + return current.app; +} + +/** Each compute node of the app, with the instance that backs it. */ +export async function computeNodeInstances( + scope: AppScope, +): Promise<{ readonly address: string; readonly instance: ComputeNodeInstance }[]> { + return Object.entries(await appNodes(scope)).map(([address, instance]) => ({ + address, + instance, + })); +} + +/** The compute instance recorded for `address`; undefined when there is none. */ +export async function mappedComputeInstance( + scope: AppScope, + address: string, +): Promise { + return (await appNodes(scope))[address]; +} + +/** + * The instance that runs the service at `address`. The recorded instance is + * reused while the Compute emulator still has it. Otherwise a new one is + * created, requesting the recorded port, and recorded. + */ +export async function ensureComputeInstance( + scope: AppScope, + address: string, +): Promise<{ readonly instance: ComputeNodeInstance; readonly portChange?: PortChange }> { + const client = computeClient(); + const recorded = await mappedComputeInstance(scope, address); + if (recorded && (await client.getInstance(recorded.id))) return { instance: recorded }; + + const created = await client.createInstance({ + labels: { + 'composer.directory': directoryOf(scope.devDir), + 'composer.app': scope.app, + 'composer.address': address, + }, + ...(recorded ? { port: recorded.port } : {}), + }); + const instance: ComputeNodeInstance = { kind: 'compute', id: created.id, port: created.port }; + await updateNodes(scope, (nodes) => { + nodes[address] = instance; + }); + if (recorded && recorded.port !== created.port) { + return { instance, portChange: { address, previous: recorded.port, given: created.port } }; + } + return { instance }; +} + +/** The emulator's view of the instance recorded for `address`; undefined when either is missing. */ +export async function liveComputeInstance( + scope: AppScope, + address: string, +): Promise { + const recorded = await mappedComputeInstance(scope, address); + return recorded ? computeClient().getInstance(recorded.id) : undefined; +} + +/** Deletes the instance recorded for `address` and forgets it. */ +export async function deleteComputeInstance(scope: AppScope, address: string): Promise { + const recorded = await mappedComputeInstance(scope, address); + if (!recorded) return; + await computeClient().deleteInstance(recorded.id); + await updateNodes(scope, (nodes) => { + delete nodes[address]; + }); +} + +/** + * `--fresh`: deletes every instance the directory's record names, whichever + * app wrote it, then empties the record. Teardown runs before the emulators + * hook, so the record may still name the app run here before. + */ +export async function deleteRecordedInstances(devDir: string): Promise { + const current = await store(devDir).read(); + if (current === undefined) return; + const client = computeClient(); + for (const instance of Object.values(current.nodes)) await client.deleteInstance(instance.id); + await store(devDir).update(() => ({ app: current.app, nodes: {} })); +} + +/** + * The Compute emulator's failure messages name instances by id; `prisma dev` + * names services by address. Rewrites a plain `Error`'s message and leaves + * any other error, such as a structured one, untouched. + */ +export function withServiceAddresses( + error: unknown, + nodes: readonly { readonly address: string; readonly instance: { readonly id: string } }[], +): unknown { + if (!(error instanceof Error) || error.constructor !== Error) return error; + let message = error.message.replace( + 'deploy it with listens: false', + 'declare it with compute({ listens: false })', + ); + for (const { address, instance } of nodes) { + message = message.replaceAll(`instance "${instance.id}"`, `service "${address}"`); + } + return message === error.message ? error : new Error(message, { cause: error }); +} diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/exports/index.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/exports/index.ts index 621f20fa5..5d7f97127 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/exports/index.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/exports/index.ts @@ -3,6 +3,7 @@ export * from '../app-name.ts'; export * from '../bucket.ts'; export * from '../compute.ts'; export * from '../dev-store.ts'; +export * from '../emulator-instances.ts'; export * from '../postgres.ts'; export * from '../providers.ts'; export * from '../resolve-package-entry.ts'; diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts index 184fedcc1..a29124b94 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts @@ -1,4 +1,4 @@ -import { beforeEach, expect, mock, test } from 'bun:test'; +import { afterEach, beforeEach, expect, mock, spyOn, test } from 'bun:test'; import type { Contract } from '@internal/core'; import { dependency, Load, module, string } from '@internal/core'; import type { ContainerInstance } from '@internal/core/config'; @@ -7,6 +7,8 @@ import * as RealLocalTarget from '@internal/local-target'; import { compute, rawPostgres } from '../exports/index.ts'; const events: string[] = []; +let portChanges: { address: string; previous: number; given: number }[] = []; +let retiredApp: string | undefined; mock.module('@internal/dev-emulators', () => ({ ...RealDevEmulators, @@ -19,11 +21,14 @@ mock.module('@internal/dev-emulators', () => ({ mock.module('@internal/local-target', () => ({ ...RealLocalTarget, resolvePackageEntry: (specifier: string) => specifier, - reserveServicePorts: async ( + retirePreviousApp: async () => retiredApp, + ensureServiceInstances: async ( container: ContainerInstance | undefined, + _devDir: string, addresses: readonly string[], ) => { - events.push(`reserve ${container?.input.appName ?? '?'}: ${addresses.join(', ')}`); + events.push(`ensure ${container?.input.appName ?? '?'}: ${addresses.join(', ')}`); + return portChanges; }, })); @@ -53,11 +58,23 @@ const container: ContainerInstance = { serialize: () => 'x', }; +const logged: string[] = []; +const consoleLog = spyOn(console, 'log').mockImplementation((line: unknown) => { + logged.push(String(line)); +}); + beforeEach(() => { events.length = 0; + logged.length = 0; + portChanges = []; + retiredApp = undefined; +}); + +afterEach(() => { + consoleLog.mockClear(); }); -test('reserves the getting-started services in dependency order once the compute emulator is up', async () => { +test('ensures the getting-started services in dependency order once the compute emulator is up', async () => { const graph = Load( module('my-app', ({ provision }) => { const quotes = provision( @@ -71,10 +88,10 @@ test('reserves the getting-started services in dependency order once the compute await runDevEmulators({ graph, container, devDir: '/dev/null/unused' }); - expect(events).toEqual(['daemon compute', 'reserve my-app: quotes, gateway']); + expect(events).toEqual(['daemon compute', 'ensure my-app: quotes, gateway']); }); -test('reserves only services, in declaration order when nothing orders them', async () => { +test('ensures only services, in declaration order when nothing orders them', async () => { const graph = Load( module('my-app', ({ provision }) => { provision(compute({ name: 'web', deps: {}, build })); @@ -85,5 +102,35 @@ test('reserves only services, in declaration order when nothing orders them', as await runDevEmulators({ graph, container, devDir: '/dev/null/unused' }); - expect(events).toEqual(['daemon compute', 'reserve my-app: web, admin', 'daemon postgres']); + expect(events).toEqual(['daemon compute', 'ensure my-app: web, admin', 'daemon postgres']); +}); + +test('prints one line for each service that could not get its remembered port back', async () => { + portChanges = [{ address: 'gateway', previous: 3001, given: 3004 }]; + const graph = Load( + module('my-app', ({ provision }) => { + provision(compute({ name: 'gateway', deps: {}, build })); + }), + ); + + await runDevEmulators({ graph, container, devDir: '/dev/null/unused' }); + + expect(logged.filter((line) => line.includes('port'))).toEqual([ + '[dev] service "gateway" now uses port 3004: its previous port 3001 is in use', + ]); +}); + +test("prints one line when the directory's previous app's local services were removed", async () => { + retiredApp = 'old-app'; + const graph = Load( + module('my-app', ({ provision }) => { + provision(compute({ name: 'web', deps: {}, build })); + }), + ); + + await runDevEmulators({ graph, container, devDir: '/dev/null/unused' }); + + expect(logged.filter((line) => line.includes('removed'))).toEqual([ + '[dev] removed the local services of "old-app", the app run here before', + ]); }); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-teardown.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-teardown.test.ts index 179ba52da..92987a370 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-teardown.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-teardown.test.ts @@ -18,12 +18,6 @@ mock.module('@internal/dev-emulators', () => ({ events.push(`daemon ${name}`); return { url: `http://127.0.0.1/${name}` }; }, - computeClient: () => ({ - deleteApp: async (app: string) => { - events.push(`delete compute ${app}`); - if (computeDeleteFails) throw new Error('compute delete failed'); - }, - }), postgresClient: unreachable('postgres'), bucketsClient: unreachable('buckets'), })); @@ -31,6 +25,10 @@ mock.module('@internal/dev-emulators', () => ({ mock.module('@internal/local-target', () => ({ ...RealLocalTarget, resolvePackageEntry: (specifier: string) => specifier, + deleteRecordedInstances: async (devDir: string) => { + events.push(`delete compute ${devDir.endsWith('/.prisma-composer/dev') ? 'recorded' : devDir}`); + if (computeDeleteFails) throw new Error('compute delete failed'); + }, removeLocalPaths: () => { events.push('remove local paths'); }, @@ -51,10 +49,10 @@ beforeEach(() => { computeDeleteFails = false; }); -test('starts the compute emulator before deleting the app from it, so a stopped emulator still forgets the old ports', async () => { +test('starts the compute emulator before deleting the recorded instances, so a stopped emulator still forgets the recorded instances', async () => { await runDevTeardown({ container, stage: undefined }); - expect(events).toEqual(['daemon compute', 'delete compute my-app', 'remove local paths']); + expect(events).toEqual(['daemon compute', 'delete compute recorded', 'remove local paths']); }); test('a failed compute delete still removes the local state, then fails the teardown', async () => { @@ -63,5 +61,5 @@ test('a failed compute delete still removes the local state, then fails the tear await expect(runDevTeardown({ container, stage: undefined })).rejects.toThrow( 'compute delete failed', ); - expect(events).toEqual(['daemon compute', 'delete compute my-app', 'remove local paths']); + expect(events).toEqual(['daemon compute', 'delete compute recorded', 'remove local paths']); }); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/local-target/attach.ts b/packages/1-prisma-cloud/1-extensions/target/src/local-target/attach.ts index 7d29e4552..5e2e81df6 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/local-target/attach.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/local-target/attach.ts @@ -6,6 +6,7 @@ */ import type { LocalTargetAttachInput, LocalTargetAttachment } from '@internal/core/config'; import { computeClient } from '@internal/dev-emulators'; +import { type AppScope, computeNodeInstances, withServiceAddresses } from '@internal/local-target'; import { prismaCloudContainerOf } from '../container.ts'; const RELIST_INTERVAL_MS = 2000; @@ -22,7 +23,11 @@ interface LogLine { * dropped (an emulator restart shows a gap in that service's stream, never a * dead session). */ -async function* mergedLogs(app: string, signal: AbortSignal, tail: number): AsyncIterable { +async function* mergedLogs( + scope: AppScope, + signal: AbortSignal, + tail: number, +): AsyncIterable { const client = computeClient(); const followed = new Set(); // Backlog is emitted once per service, on its FIRST follow — a follower @@ -46,7 +51,7 @@ async function* mergedLogs(app: string, signal: AbortSignal, tail: number): Asyn void (async () => { try { let buffer = ''; - for await (const chunk of client.followLogs(app, id, signal, { tail: followTail })) { + for await (const chunk of client.followLogs(id, signal, { tail: followTail })) { buffer += chunk; let newlineAt = buffer.indexOf('\n'); while (newlineAt !== -1) { @@ -64,14 +69,12 @@ async function* mergedLogs(app: string, signal: AbortSignal, tail: number): Asyn }; const relist = async (): Promise => { - const services = await client.listServices(app); - // The emulator's `id` is the (path-segment-safe) key `followLogs` needs; - // the log line is labelled with the real, dotted address — the same - // identity the front door prints, so a service's endpoint line and its - // log lines name it identically (a nested module's service address - // contains dots the emulator's own id path segment cannot carry — see - // compute.ts's `slugServiceId`). - for (const svc of services) follow(svc.id, svc.address); + // Log lines are labelled with the service's address — the identity the + // front door prints, so a service's endpoint line and its log lines name + // it identically. + for (const { address, instance } of await computeNodeInstances(scope)) { + follow(instance.id, address); + } }; await relist(); @@ -99,16 +102,32 @@ async function* mergedLogs(app: string, signal: AbortSignal, tail: number): Asyn } export async function devAttach(input: LocalTargetAttachInput): Promise { - const app = prismaCloudContainerOf(input.container).input.appName; + const scope = { + devDir: input.devDir, + app: prismaCloudContainerOf(input.container).input.appName, + }; const client = computeClient(); return { - startServices: () => client.startApp(app), + startServices: async () => { + const nodes = await computeNodeInstances(scope); + try { + await client.startInstances(nodes.map(({ instance }) => instance.id)); + } catch (error) { + throw withServiceAddresses(error, nodes); + } + }, endpoints: async () => { - const services = await client.listServices(app); - return services.map((svc) => ({ address: svc.address, url: svc.url })); + const nodes = await computeNodeInstances(scope); + return nodes.map(({ address, instance }) => ({ + address, + url: `http://localhost:${String(instance.port)}`, + })); + }, + logs: (signal, opts) => mergedLogs(scope, signal, opts?.tail ?? 0), + stopServices: async () => { + const nodes = await computeNodeInstances(scope); + await client.stopInstances(nodes.map(({ instance }) => instance.id)); }, - logs: (signal, opts) => mergedLogs(app, signal, opts?.tail ?? 0), - stopServices: () => client.stopApp(app), }; } diff --git a/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts b/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts index 236d8cdbb..21df6d008 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts @@ -6,7 +6,7 @@ * `postgres` resource kind (REVISED — Postgres is a first-class daemon * since the programmatic `@prisma/dev` adoption, operator review of #162). * - * Once Compute is up, every service's port is reserved in graph order, so a + * Once Compute is up, every service's instance is ensured in graph order, so a * fresh start numbers ports dependencies first; Alchemy's concurrent `App` * applies would otherwise pick them in arrival order. * @@ -24,7 +24,7 @@ */ import type { LocalTargetEmulatorsInput } from '@internal/core/config'; import { ensureDaemon } from '@internal/dev-emulators'; -import { reserveServicePorts } from '@internal/local-target'; +import { appNameOf, ensureServiceInstances, retirePreviousApp } from '@internal/local-target'; import { PRISMA_CLOUD_EXTENSION_ID } from '../container.ts'; import { serviceAppName } from '../service-app.ts'; import { daemonEntry } from './daemon-entry.ts'; @@ -50,7 +50,23 @@ function serviceAppNames(input: LocalTargetEmulatorsInput): string[] { export async function runDevEmulators(input: LocalTargetEmulatorsInput): Promise { const { url: computeUrl } = await ensureDaemon('compute', daemonEntry('compute')); console.log(`[dev] compute emulator ready at ${computeUrl}`); - await reserveServicePorts(input.container, serviceAppNames(input)); + const retired = await retirePreviousApp({ + devDir: input.devDir, + app: appNameOf(input.container), + }); + if (retired !== undefined) { + console.log(`[dev] removed the local services of "${retired}", the app run here before`); + } + const portChanges = await ensureServiceInstances( + input.container, + input.devDir, + serviceAppNames(input), + ); + for (const { address, previous, given } of portChanges) { + console.log( + `[dev] service "${address}" now uses port ${String(given)}: its previous port ${String(previous)} is in use`, + ); + } if (usesBuckets(input)) { const { url: bucketsUrl } = await ensureDaemon('buckets', daemonEntry('buckets')); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/local-target/teardown.ts b/packages/1-prisma-cloud/1-extensions/target/src/local-target/teardown.ts index 0b9befc7b..bc59ef4af 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/local-target/teardown.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/local-target/teardown.ts @@ -18,13 +18,8 @@ import type { TeardownInput } from '@internal/core/config'; import { DEV_DIR } from '@internal/core/config'; -import { - bucketsClient, - computeClient, - ensureDaemon, - postgresClient, -} from '@internal/dev-emulators'; -import { removeLocalPaths } from '@internal/local-target'; +import { bucketsClient, ensureDaemon, postgresClient } from '@internal/dev-emulators'; +import { deleteRecordedInstances, removeLocalPaths } from '@internal/local-target'; import { prismaCloudContainerOf } from '../container.ts'; import { daemonEntry } from './daemon-entry.ts'; @@ -41,15 +36,16 @@ async function tolerateUnreachable(action: () => Promise): Promise { export async function runDevTeardown(input: TeardownInput): Promise { const app = prismaCloudContainerOf(input.container).input.appName; const cwd = process.cwd(); + const devDir = `${cwd}/${DEV_DIR}`; await tolerateUnreachable(() => postgresClient().deleteApp(app)); - // Compute persists port allocations across restarts, so a stopped daemon - // would hand the old ports back; start it so the delete always lands. + // Compute persists instances and their ports across restarts, so a stopped + // daemon would hand the old ports back; start it so the delete always lands. try { await ensureDaemon('compute', daemonEntry('compute')); - await computeClient().deleteApp(app); + await deleteRecordedInstances(devDir); } finally { await tolerateUnreachable(() => bucketsClient().deleteApp(app)); - removeLocalPaths([`${cwd}/${DEV_DIR}`, `${cwd}/.alchemy/state/${app}/dev`]); + removeLocalPaths([devDir, `${cwd}/.alchemy/state/${app}/dev`]); } } diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 266fc290a..093dbd8bc 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -886,6 +886,9 @@ importers: specifier: ^7.5.21 version: 7.5.21 devDependencies: + '@internal/foundation': + specifier: workspace:0.30.0 + version: link:../../../0-framework/0-foundation/foundation '@internal/tsdown-config': specifier: workspace:0.30.0 version: link:../../../0-framework/0-foundation/tsdown-config diff --git a/skills/prisma-composer-core-concepts/SKILL.md b/skills/prisma-composer-core-concepts/SKILL.md index f4ae50211..d70772c08 100644 --- a/skills/prisma-composer-core-concepts/SKILL.md +++ b/skills/prisma-composer-core-concepts/SKILL.md @@ -443,6 +443,7 @@ that surprise: changes. `ready` prints only once every service accepts connections on its port, except a service declared `compute({ ..., listens: false })` (one that never listens, like the cron scheduler), which counts as running once its process starts. A service that exits before it listens, or does not listen within 30 seconds, fails `dev` (and stops the app's other services) with a message naming the service, its port, and its log file: on a first start it is in the deploy engine's output, before the error that the deploy engine exited (`DEV.CONVERGE_FAILED` from the `dev` operation); on a warm start it is the `DEV.SERVICE_START_FAILED` message. `PRISMA_COMPOSER_SERVICE_READY_TIMEOUT_MS` raises the 30-second limit (milliseconds); the compute emulator reads it when it starts, so a running emulator keeps its old value until its process ends. An emulator that does not accept connections fails `dev` and `log` with `DEV.EMULATOR_UNREACHABLE` after a few seconds of retries; nothing else is retried. 2. Ctrl-C stops the app's processes but leaves local databases, buckets, and their data up: the next `dev` is a warm start. Starting clean, wiping this app's local instances and data first, is an explicit opt-in flag. On a fresh start, each service gets the lowest free port from 3000 up, in dependency order: a service before the services that call it, otherwise provision order. A warm start keeps the ports services already have. + Services belong to the directory `dev` runs in: two checkouts of one app (two git worktrees) run separate processes on separate ports and never restart each other's. `.prisma-composer/dev/emulator-instances.json` records which local service backs each service. A service keeps its port in its directory across runs, even while stopped. If its local service is gone (a Composer upgrade discarded the compute emulator's state), the next `dev` re-creates it and asks for the old port; when another directory has taken it, the service gets the lowest free port and `dev` prints `[dev] service "
" now uses port : its previous port is in use`. A directory runs one app at a time: running a different app there removes the previous app's local services and prints `[dev] removed the local services of "", the app run here before`. A stopped service keeps its port reserved, so a deleted checkout's services keep theirs; there is no command to remove them yet. Local databases and buckets are still keyed by the app's name, so two checkouts of one app share them. 3. `dev` does not print service logs. The `log` operation follows the already-running app's merged logs; it never builds, provisions, starts, or stops anything: diff --git a/test/integration/test/local-dev-store.integration.ts b/test/integration/test/local-dev-store.integration.ts index f71517e01..d121e5028 100644 --- a/test/integration/test/local-dev-store.integration.ts +++ b/test/integration/test/local-dev-store.integration.ts @@ -240,13 +240,22 @@ interface ServiceInfo { readonly pid?: number; } -/** The documented Compute-emulator wire protocol (spec § 2 compute-main.ts) — plain fetch, never a typed client. */ +/** The services `prisma dev` recorded in storeDir's `.prisma-composer/dev/emulator-instances.json`, as the Compute emulator reports them (plain fetch, never the typed client). */ async function listComputeServices(app: string): Promise { const entry = readEmulatorEntry('compute'); if (entry === undefined) throw new Error('compute emulator registry entry not found'); - const res = await fetch(`http://127.0.0.1:${entry.port}/apps/${app}/services`); - if (!res.ok) throw new Error(`compute emulator listServices failed: ${res.status}`); - return (await res.json()) as ServiceInfo[]; + const mapping = readJson( + path.join(storeDir, '.prisma-composer', 'dev', 'emulator-instances.json'), + ) as { app: string; nodes: Record } | undefined; + if (mapping === undefined || mapping.app !== app) return []; + const services: ServiceInfo[] = []; + for (const [address, node] of Object.entries(mapping.nodes)) { + const res = await fetch(`http://127.0.0.1:${entry.port}/instances/${node.id}`); + if (res.status === 404) continue; + if (!res.ok) throw new Error(`compute emulator getInstance failed: ${res.status}`); + services.push({ ...((await res.json()) as ServiceInfo), address }); + } + return services; } interface PostgresDatabaseInfo { diff --git a/test/integration/test/local-dev.integration.ts b/test/integration/test/local-dev.integration.ts index 85373e50f..2bdd2b929 100644 --- a/test/integration/test/local-dev.integration.ts +++ b/test/integration/test/local-dev.integration.ts @@ -299,13 +299,22 @@ interface ServiceInfo { readonly pid?: number; } -/** The documented Compute-emulator wire protocol (spec § 2 compute-main.ts) — plain fetch, never the typed client. */ +/** The services `prisma dev` recorded in fixtureDir's `.prisma-composer/dev/emulator-instances.json`, as the Compute emulator reports them (plain fetch, never the typed client). */ async function listComputeServices(): Promise { const entry = readEmulatorEntry('compute'); if (entry === undefined) throw new Error('compute emulator registry entry not found'); - const res = await fetch(`http://127.0.0.1:${entry.port}/apps/${APP_NAME}/services`); - if (!res.ok) throw new Error(`compute emulator listServices failed: ${res.status}`); - return (await res.json()) as ServiceInfo[]; + const mapping = readJson( + path.join(fixtureDir, '.prisma-composer', 'dev', 'emulator-instances.json'), + ) as { app: string; nodes: Record } | undefined; + if (mapping === undefined || mapping.app !== APP_NAME) return []; + const services: ServiceInfo[] = []; + for (const [address, node] of Object.entries(mapping.nodes)) { + const res = await fetch(`http://127.0.0.1:${entry.port}/instances/${node.id}`); + if (res.status === 404) continue; + if (!res.ok) throw new Error(`compute emulator getInstance failed: ${res.status}`); + services.push({ ...((await res.json()) as ServiceInfo), address }); + } + return services; } /**