From 7bc6d7e8b64c358487c1849615ef0f238997a31e Mon Sep 17 00:00:00 2001 From: willbot Date: Tue, 13 Oct 2026 16:06:25 +0200 Subject: [PATCH 01/17] docs(drive): emulator commands project spec, plan and first slice specs Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- .../emulator-commands/design-notes.md | 61 ++++++++++++ .drive/projects/emulator-commands/plan.md | 64 +++++++++++++ .../slices/s1a-compute-instances/spec.md | 86 +++++++++++++++++ .../slices/s2-command-group/spec.md | 59 ++++++++++++ .drive/projects/emulator-commands/spec.md | 92 +++++++++++++++++++ 5 files changed, 362 insertions(+) create mode 100644 .drive/projects/emulator-commands/design-notes.md create mode 100644 .drive/projects/emulator-commands/plan.md create mode 100644 .drive/projects/emulator-commands/slices/s1a-compute-instances/spec.md create mode 100644 .drive/projects/emulator-commands/slices/s2-command-group/spec.md create mode 100644 .drive/projects/emulator-commands/spec.md diff --git a/.drive/projects/emulator-commands/design-notes.md b/.drive/projects/emulator-commands/design-notes.md new file mode 100644 index 000000000..92cbdee99 --- /dev/null +++ b/.drive/projects/emulator-commands/design-notes.md @@ -0,0 +1,61 @@ +# Design notes — emulator commands + +Decisions taken while shaping, with the reasoning. Will's rulings are marked; the rest are orchestrator decisions he can overturn. + +## Rulings (Will, 2026-10-13) + +- The commands are `prisma dev url
` and `prisma emulator list`. A top-level `prisma url` was rejected as ambiguous (local and deployed running objects). +- `emulator` commands act on the machine's emulators, not on a project. +- The emulators know nothing about Composer. Composer maps (directory, app, address) to emulator instances itself, per directory. Instances carry informative labels only. +- `dev url` resolves a Composer node address only, never an emulator instance id. +- Databases and buckets are in scope alongside services. +- Piped output stays JSON; docs use `jq`. No output-format change to the CLI engine. +- The CLI engine changes so a path can be both a command and a group, keeping `prisma dev `. +- The emulators may move out of Composer later (own project, or prisma-cli). Not part of this work. + +## Facts the design rests on (checked against main, 90a30381) + +- The config does not name the entry or the app. ADR-0003 rejected a config that names the app; `deploy-cli.md` lists entry discovery as out of scope. So `dev url` cannot find the app from `prisma.config.ts`; it needs a record `prisma dev` leaves in the directory. +- Today the emulators are app-aware: Compute routes are `/apps/:app/services/:id`, Postgres server names come from `instanceNameFor(app, db)`, buckets are named `--`. The local identity is the root name alone (`target/src/local-target/container.ts`), so two directories running one app share instances. +- `LocalTargetAttachment.endpoints()` returns `{ address, url }` only. The emulator's `ServiceInfo` has `status: running | backoff | held | stopped`. +- The engine rejects a command path under another command (`packages/cli-engine/src/execution/command-tree.ts:157`). +- When stdout is not a terminal the engine prints JSON frames: `{"kind":"result","envelope":{"ok":true,"result":…}}` (`shared-flags.ts:191`, `settlement.ts:83`). +- `9-public` may import both `framework` and `prisma-cloud` (`architecture.config.json` cross-domain rules). `framework` imports nothing outside itself. + +## Where the code lives + +- **Emulators** (`dev-emulators`): instance-keyed routes and state; every create call accepts a `labels: Record` that is stored and returned verbatim. Each emulator gains a route that lists all its instances. +- **The mapping** is owned by the Prisma Cloud local target (`local-target` / `target/src/local-target`), because that code creates the instances and learns their ids. It lives in `/.prisma-composer/dev/` (the dev state directory, already per working directory), and records the app name plus, per node address, the emulator kind and instance id. +- **`prisma dev url`** goes through the core seam, like the `log` operation: config, then `resolveLocalTargets`, then the target's attachment. The attachment gains a method that resolves one address to `{ address, kind, url, state }`. Core stays target-agnostic; the framework domain imports nothing new. It runs from a Composer project, which `dev url` needs anyway. +- **`prisma emulator list`** needs no project and no config, so it cannot go through the seam. Its handler lives in the Prisma Cloud domain next to the emulator client, and `@prisma/composer-cli` (in `9-public`, which may import `prisma-cloud`) adds it to the command family. No dependency-cruiser exception is needed. +- The emulator client stays the only module that speaks the emulators' HTTP API, so a later move out of Composer replaces one dependency. + +## Ports (Will, 2026-10-13: agents in many worktrees start and stop environments) + +A stopped instance holds no port. Composer remembers each node's last port in the directory's mapping and requests it on the next start. A requested port is a preference: if another instance or process holds it, the emulator gives the lowest free port from 3000 and `prisma dev` prints one line saying the port changed. Failing instead would break an agent's `prisma dev` whenever another worktree happened to take the port. On a first start, ports still go in dependency order (prisma/composer#350). + +## Labels Composer writes + +`composer.directory` (absolute path), `composer.app` (app name), `composer.address` (node address). `emulator list` shows them; nothing reads them back as identity. + +## Output shapes + +- `dev url`, human: the URL alone on stdout's data line, so `--format human` also works in `$(…)`. JSON `result`: `{ address, kind, url, state }`, where `kind` is `service | database | bucket` and `state` is the service status, or `running` for a database or bucket whose emulator is up. +- `emulator list`, JSON `result`: `{ emulators: [{ kind, running, url, instances: [{ id, kind, url, state, labels }] }] }`. Human: one block per emulator, instances listed under it with the address and directory from their labels when present. It doesn't fit the list convention's `{ name, id, status }` items, because two levels are nested. That is a deliberate departure, noted in the slice spec. + +## Error codes (namespace `DEV` for `dev url`, new `EMULATOR` for `emulator list`) + +- `DEV.NOT_STARTED_HERE`: no `prisma dev` has run in this directory (no mapping). Fix: run `prisma dev ` here. +- `DEV.ADDRESS_UNKNOWN`: no node with that address in this directory's app; lists the known addresses. +- `DEV.EMULATOR_UNREACHABLE` (from #353): the node's emulator is not running or not answering. +- `DEV.SERVICE_NOT_RUNNING`: the service exists but is stopped, backing off or held; says which, and points to `prisma dev`. +- `EMULATOR.*`: only if `list` needs one beyond `DEV.EMULATORS_DIR_INVALID` and `DEV.EMULATOR_UNREACHABLE`. Final names are fixed in the slice specs and recorded in ADR-0044. + +## Alternatives considered + +- **`prisma emulator url
`.** Same behaviour, but `url` asks a Composer question about this directory's app, which belongs under `dev`. +- **Rename the run command to `prisma dev start`.** Avoids the engine change, but breaks every `prisma dev module.ts` and reverses ADR-0050's root verb. +- **Name the app on the command line (`url my-app gateway`) or with `--app`.** Addresses are meant relative to the local app, and names collide across directories. +- **Make the local identity "app name + directory hash" and keep app-aware emulators.** Fixes collisions but keeps Composer concepts inside the emulators, which is the opposite of the model. +- **`emulator list` through the core seam.** Requires a project and config for a machine-level command. +- **Print a bare URL even when piped.** An engine output change Will deferred. diff --git a/.drive/projects/emulator-commands/plan.md b/.drive/projects/emulator-commands/plan.md new file mode 100644 index 000000000..5d8d1d2bb --- /dev/null +++ b/.drive/projects/emulator-commands/plan.md @@ -0,0 +1,64 @@ +# Emulator commands — project plan + +**Spec:** `.drive/projects/emulator-commands/spec.md` · **Design notes:** `design-notes.md` + +## Slices + +### S1 — Emulators know nothing about Composer (prisma/composer) + +**Outcome.** The Compute, Postgres and bucket emulators key instances by their own ids, accept and return free-form labels, and each lists all its instances. The Prisma Cloud local target creates instances, labels them, and records the per-directory mapping in `.prisma-composer/dev/`. Every Composer operation on the emulators goes through the mapping: deploy, session start and stop, logs, `--fresh` teardown. `prisma dev` behaves the same from outside, except that pre-upgrade local state is discarded. Two directories running one app get separate instances. Port allocation keeps its current rule on a first start: services reserve ports in dependency order (prisma/composer#350), lowest free from 3000. The mapping remembers each node's port and asks for it on every later start. Stopped instances hold no port. When `prisma dev` ends it stops all of the directory's instances, Postgres servers included (the Postgres emulator gains a stop that keeps data). + +**Builds on:** prisma/composer#353 (stack on its branch until it merges). +**Hands to S3:** the mapping file and its reader; the emulator client's list-everything calls; ADR amending ADR-0041. + +Likely too large for one review: split at pickup into Compute (with the mapping and remembered ports), then Postgres and buckets (with the Postgres stop). + +### S2 — A command path can also be a group (prisma/prisma-cli) + +**Outcome.** The CLI engine accepts a command mounted at a path that other commands mount under. `prisma dev module.ts` runs the command, `prisma dev url gateway` runs the subcommand, a subcommand name wins over a positional argument with the same spelling, and `prisma dev --help` shows the command's usage and its subcommands. Proven with engine tests using a stand-in command; no Composer change needed. + +**Builds on:** nothing. Runs in parallel with S1. +**Hands to S4:** an engine release that allows the `dev` mount. + +### S3 — `prisma dev url`, `prisma dev stop` and `prisma emulator list` (prisma/composer) + +**Outcome.** The command family exports `dev url`, `dev stop` and `emulator list` with tests first (bun:test; emulator tests set `PRISMA_COMPOSER_EMULATORS_DIR` to a temporary directory). The core local-target attachment gains the resolve-one-address method. Error codes are recorded in ADR-0044. A new ADR amends ADR-0050 (the family is no longer exactly `deploy` and `dev`). `local-dev.md`, the running-locally guide and the core-concepts skill are updated, including that `start` and `stop` are planned. + +**Builds on:** S1. +**Hands to S4:** a Composer release containing both commands. + +### S4 — Mount the commands (prisma/prisma-cli) + +**Outcome.** The `prisma` binary pins the Composer release from S3 and mounts `dev url`, `dev stop` and `emulator list`, with an `emulator` group whose help defines "emulator", `mount-coverage.test.ts` updated, and `EXCLUSIONS` entries in `e2e-coverage.test.ts`. + +**Builds on:** S2, S3 (released). +**Hands to S5:** a `prisma` release with both commands. + +### S5 — Docs stop hard-coding ports (prisma/web) + +**Outcome.** The Composer getting-started and local-development pages use `prisma dev url` instead of `localhost:3000`/`3001`. + +**Builds on:** S4 (released). + +## Sequence + +``` +#353 ─► S1 ─► S3 ─┐ + ├─► S4 ─► S5 +S2 ───────────────┘ +``` + +S1 and S2 run in parallel. S4 and S5 wait on releases, which are currently blocked on an upstream Alchemy issue. + +Five slices is one over the usual ceiling. S4 and S5 are small handoffs into other repos, and each one waits on a release, so they can't fold into a neighbour. + +## Tracking + +No Linear project exists for this work. Slices are tracked in this plan. + +## Close-out + +- [ ] Verify every Project DoD item in `spec.md`. +- [ ] Long-lived content lives in ADRs, `docs/design/10-domains/local-dev.md`, the guide and the skill. +- [ ] Strip repo-wide references to `.drive/projects/emulator-commands/`. +- [ ] Delete `.drive/projects/emulator-commands/`. diff --git a/.drive/projects/emulator-commands/slices/s1a-compute-instances/spec.md b/.drive/projects/emulator-commands/slices/s1a-compute-instances/spec.md new file mode 100644 index 000000000..30343c476 --- /dev/null +++ b/.drive/projects/emulator-commands/slices/s1a-compute-instances/spec.md @@ -0,0 +1,86 @@ +# S1a — The Compute emulator knows nothing about Composer + +Repo: prisma/composer. Branch `feat/emulator-instances`, stacked on prisma/composer#353 (`fix/dev-ready-after-listening`). PR base: that branch until #353 merges, then `main`. + +## At a glance + +After this slice, the Compute emulator runs service instances keyed by its own ids. It never sees an app name, a node address or a directory, except as label values it stores and returns without reading. Composer keeps, per working directory, a record of which instance backs each service address and which port it last had. + +```text +.prisma-composer/dev/ (per directory, written by Composer) + app: "my-app" + nodes: + quotes → compute instance "svc_k3f9…", port 3000 + gateway → compute instance "svc_81ad…", port 3001 + +Compute emulator (machine-wide, Composer-unaware) + GET /instances → [{ id: "svc_k3f9…", port: 3000, url, status: "running", + labels: { "composer.directory": "/path", "composer.app": "my-app", + "composer.address": "quotes" } }, …] +``` + +Two directories running the same app get separate instances. `prisma dev` otherwise behaves the same from the outside. + +## Chosen design + +**Emulator (`packages/1-prisma-cloud/0-lowering/dev-emulators`).** +- Instances are created by the emulator and get an id it generates. Create accepts `labels: Record` and an optional requested port. +- Every instance operation is by id: deploy, start, stop, delete, logs, get. There is no app grouping. Bulk start and stop take a list of ids. +- `GET` for all instances returns id, port, url, status and labels. +- Ports: a stopped instance holds no port. A requested port is used if it is free; otherwise the emulator gives the lowest free port from 3000 and reports the port it chose. With no request, it gives the lowest free port from 3000. +- State written in the old app-keyed format is discarded on load. Local dev state is never migrated across versions (running-locally guide). +- #353's readiness behaviour (a service is `running` only once its port accepts connections), the start and stop concurrency rules, crash supervision and the version-skew restart all carry over unchanged, keyed by instance instead of `(app, id)`. + +**Composer's mapping (Prisma Cloud local target: `0-lowering/local-target`, `1-extensions/target/src/local-target`).** +- One file in `/.prisma-composer/dev/` records the app name and, per service address, the Compute instance id and its last port. One module owns reading and writing it. S3 reads it for `prisma dev url`. +- The emulators hook keeps prisma/composer#350's order: it ensures one instance per service in `graph.nodes` order (dependencies first, ties in declaration order). It reuses the mapped instance when it still exists on the emulator. Otherwise it creates one, requesting the remembered port, and records the result. +- When a requested port was unavailable and the emulator gave a different one, `prisma dev` prints one line naming the service, the old port and the new one. +- Labels on create: `composer.directory` (absolute cwd), `composer.app`, `composer.address`. No Composer code finds an instance by its labels. +- The `App` and `Deployment` providers, attach (endpoints, logs, `startServices`, `stopServices`) and the `--fresh` teardown all reach Compute instances only through the mapping. +- `--fresh` deletes the mapped Compute instances and clears their mapping entries. + +**Docs.** +- A new ADR records the model: the emulators know nothing about Composer, Composer maps its nodes to instances per directory, and labels are informative only. It amends ADR-0041's local identity ("a stable local identity from the app name"). +- `docs/design/10-domains/local-dev.md` is updated for Compute. +- The running-locally guide and the core-concepts skill describe the user-visible change: two directories running one app no longer share services. Follow `.agents/rules/user-facing-surface-changes.mdc`. + +## Coherence rationale + +One reviewer can hold this as "Compute instances become emulator-owned, and Composer's per-directory mapping replaces the app name as the key". The Postgres and bucket emulators stay app-keyed until S1b. Both keys are valid in the meantime: the app name still identifies Postgres and buckets, and the mapping identifies Compute. + +## Scope + +In: the Compute emulator and its client; the mapping module; the emulators hook, the Compute providers, attach and teardown for Compute; tests; the ADR and docs above. + +Deliberately out: +- Postgres and buckets (S1b). +- Stopping Postgres when `dev` ends (S1b). +- `dev url`, `dev stop` and `emulator list` (S3). This slice does add the emulator's list-all route and client call, because the mapping's "does this instance still exist" check needs it. +- Deleting instances whose directory is gone. + +## Pre-investigated edge cases + +| Case | Expected | +| --- | --- | +| Mapping names an instance the emulator no longer has (emulator state lost, or another directory's `--fresh`) | Create a new instance requesting the remembered port; update the mapping | +| Remembered port taken by another directory's running instance | Lowest free port; one-line notice; mapping updated | +| Two directories, same app, started concurrently | Separate instances, no shared ports, neither redeploys over the other | +| Warm restart after Ctrl-C (the #164 warm-restart case) | Same instances, same ports, services started from their last deployment | +| Emulator restarted by a version change | Old app-keyed state discarded; the next `dev` creates instances, requesting remembered ports | + +## Slice-specific done conditions + +- [ ] Two temporary directories run the same fixture app at once. Each has its own Compute instances, and neither directory's services restart when the other deploys. This is proven by a test against a real Compute emulator in a temporary `PRISMA_COMPOSER_EMULATORS_DIR`. +- [ ] A test fails if the Compute emulator's source or state stores an app name, address or directory anywhere except label values. + +## References + +- Project: `../../spec.md`, `../../design-notes.md`, `../../plan.md`. +- prisma/composer#350 (port order), prisma/composer#353 (readiness; this branch's base). +- `docs/design/10-domains/local-dev.md`, ADR-0041, ADR-0044, ADR-0051. + +## Dispatch plan + +1. **Emulator and client.** The Compute emulator and `computeClient` move to instance ids, labels, requested ports, list-all and bulk start and stop, with ports released on stop. The emulator's own tests are migrated and extended. *Hands to 2:* a Compute client keyed by instance id. +2. **Composer's mapping.** The mapping module, the emulators hook, the providers, attach and teardown move to the mapping, including the change-of-port notice and the two-directory test. *Hands to 3:* `prisma dev` working end to end on instance ids. +3. **Docs.** The ADR, local-dev.md, the guide and the skill. *Hands to:* the PR. diff --git a/.drive/projects/emulator-commands/slices/s2-command-group/spec.md b/.drive/projects/emulator-commands/slices/s2-command-group/spec.md new file mode 100644 index 000000000..631533c6e --- /dev/null +++ b/.drive/projects/emulator-commands/slices/s2-command-group/spec.md @@ -0,0 +1,59 @@ +# S2 — A command path can also be a group + +Repo: prisma/prisma-cli. Package: `packages/cli-engine` (`@prisma/cli-engine`). Branch `feat/engine-command-group` off `main`. + +## At a glance + +Today the engine refuses to build a CLI where one command is mounted at a path that is also the prefix of other commands (`packages/cli-engine/src/execution/command-tree.ts:157`, "collides with a command mounted at a prefix of it"). After this slice it accepts it: + +```ts +createCli({ + groups: { dev: { brief: "…" } }, + commands: { + dev: devCommand, // prisma dev module.ts + "dev url": devUrlCommand, // prisma dev url gateway + "dev stop": devStopCommand, + }, +}); +``` + +```bash +prisma dev module.ts # runs devCommand with entry "module.ts" +prisma dev url gateway # runs devUrlCommand +prisma dev --help # devCommand's usage, then the subcommands under dev +prisma dev url --help # devUrlCommand's help +``` + +A bare `dev` that also has subcommands is the established convention (`pnpm dev`, `next dev`). The project needs it so `prisma dev ` keeps working while `prisma dev url` and `prisma dev stop` are added. + +## Chosen design + +- A path may carry both a command and child commands. The path still needs a group declaration in `groups`, because it has children; the group's brief is what appears in its parent's listing. +- Routing: when the next argument after the path is the name of a child, the child runs. Otherwise the path's own command runs with the remaining arguments. A child name therefore wins over a positional argument spelled the same. This is documented, and it is harmless for `dev`, whose positional is a file path. +- Help: ` --help` shows the path's own command help and lists the child commands. +- Every existing construction check still applies to both the command and its children. Only the prefix collision check is relaxed. Two commands at the same exact path are still refused. +- Everything else that walks the tree handles a node that is both: shell completion, the `commands` listing, unknown-command suggestions, redirects (a redirect still cannot sit on a mounted path), mount metadata, and anything in `packages/cli` that enumerates mounted paths (`mount-coverage.test.ts`, `e2e-coverage.test.ts`'s key reader). +- If Stricli's route maps support a default command for a route, use that. Otherwise implement the routing in the engine's adapter. The implementer checks which. + +## Coherence rationale + +One engine capability, proven with stand-in commands. No product command changes. Composer's `dev url` and `dev stop` mount later, in S4. + +## Scope + +In: the engine change, its tests, the engine's docs (README and AGENTS.md where they describe the tree), the version bump or changeset that the repo's release process expects for an engine change, and docs in `docs/architecture` or `docs/product` that state the old rule. + +Out: mounting any Composer command; changing `prisma dev`; output formats. + +## Slice-specific done conditions + +- [ ] Engine tests with a stand-in `dev` command and `dev url` child cover: the parent runs with a positional; the child runs; `--help` on each; a child name winning over the same-spelled positional; and two commands at one exact path still being refused. + +## References + +- `docs/architecture/cli-engine-requirements.md` (R12: the shell defines the command tree), `docs/product/command-principles.md`, `docs/product/cli-help-standard.md`. +- Project: `../../spec.md`, `../../design-notes.md`. + +## Dispatch plan + +1. One dispatch: tests first, then the change, then docs. *Hands to:* the PR. diff --git a/.drive/projects/emulator-commands/spec.md b/.drive/projects/emulator-commands/spec.md new file mode 100644 index 000000000..30ae72ecf --- /dev/null +++ b/.drive/projects/emulator-commands/spec.md @@ -0,0 +1,92 @@ +# Purpose + +Let users, docs and agents ask the `prisma` CLI where a locally running Composer node is, instead of assuming which port it got: + +```bash +npx prisma dev module.ts # terminal 1 +curl "$(npx prisma dev url gateway | jq -r 'select(.kind=="result").envelope.result.url')" # terminal 2 +``` + +and see everything the machine's local emulators are running: + +```bash +npx prisma emulator list +``` + +Today the Composer getting-started page on prisma/web hard-codes `curl localhost:3001` and a POST to `localhost:3000`. Those only work because prisma/composer#350 happens to give the getting-started app those ports. + +# At a glance + +The project rests on one model, settled with Will on 2026-10-13: + +- **The emulators are standalone, machine-local services.** The Compute, Postgres and bucket emulators hold instances: a running service process, a database, a bucket. They know nothing about Composer, apps, directories or node addresses. Each instance carries informative labels that the emulator stores and returns but never interprets, such as "started by Composer in /path/to/project for node `gateway`". Users will be able to start and stop emulators freely (a later project). +- **Composer is one client of the emulators.** `prisma dev` asks them for the instances its topology needs. +- **Composer owns the mapping from its names to emulator instances, per directory.** For each node address of the app run from a directory, Composer records which emulator instance backs it, in `.prisma-composer/dev/`. Every Composer operation on the emulators goes through that mapping: deploy, start, stop, logs, `--fresh`. + +On that model, two commands: + +- **`prisma dev url
`** prints the URL of a node of the app that `prisma dev` runs from the current directory. It covers services (`http://localhost:`), databases (the local connection string) and buckets (the bucket emulator's S3 endpoint and the bucket's name on it). It reads the directory's mapping, then asks the emulator for that instance. `--json` gives the address, URL, kind and state. +- **`prisma emulator list`** prints every emulator on the machine, whether it is running, and each running emulator's instances with their kind, id, URL, state and labels. It needs no Composer project. + +- **`prisma dev stop`** stops every instance this directory's app has on the emulators: service processes and Postgres servers. Data stays. An agent runs it before deleting its worktree, or after a `prisma dev` that was killed without a chance to clean up. + +When `prisma dev` ends (Ctrl-C, SIGTERM, or a failure), it does the same stop. Today it stops only the service processes and leaves Postgres servers running. A stopped instance holds no port. Composer remembers each node's port in the directory's mapping and asks the emulator for that port on the next start, so ports stay stable within a directory without the emulator reserving them. + +The shared emulator daemons themselves keep running; other directories may be using them. + +`prisma dev ` keeps working unchanged. The CLI engine learns to let a path be both a command and a group, so `prisma dev module.ts` runs the app and `prisma dev url gateway` runs the subcommand. A bare `dev` is the established convention (`pnpm dev`, `next dev`). + +# Non-goals + +- **Removing instances and their data.** A worktree that is deleted leaves its stopped instances' records and Postgres data on disk. They hold no process and no port. A remove command comes with the later emulator start/stop project. +- **`prisma emulator start` and `stop`.** Planned; they change what runs on the shared emulators and get their own design. +- **Printing a bare value when stdout is a pipe.** The CLI keeps its rule that a piped stdout gets JSON frames. Docs use `jq`. No output-format change to the CLI engine. +- **`url` for things Composer didn't start.** `prisma dev url` takes a Composer node address only, never an emulator instance id. +- **Moving the emulators out of Composer.** Likely later, into their own project or into prisma-cli. This project keeps the emulator client behind one module boundary so that move stays cheap, but does not make it. +- **Migrating existing local dev state.** Local dev state is never migrated across Composer versions (running-locally guide). After the upgrade, a first `prisma dev` starts from empty instances. Instances the old emulators held are discarded. + +# Place in the larger world + +- **prisma/composer** — the emulators (`packages/1-prisma-cloud/0-lowering/dev-emulators`), the local target that drives them (`0-lowering/local-target`, `1-extensions/target/src/local-target`), the `dev` and `log` operations and the command family (`packages/0-framework/3-tooling/cli`), and `@prisma/composer-cli`, which the `prisma` binary mounts. +- **prisma/prisma-cli** — the CLI engine (`packages/cli-engine`), and the host that decides where each Composer command mounts (`packages/cli/src/cli.ts`, requirement R12 "the shell defines the command tree"). +- **prisma/web** — the Composer getting-started and local-development pages. +- **Depends on prisma/composer#353** (open): a service counts as running only once its port accepts connections, and `withEmulatorRetry` retries only `DEV.EMULATOR_UNREACHABLE`. Without it, a URL `dev url` prints can refuse connections for a second or two after `ready`. +- **Releases.** Composer and CLI releases are currently blocked on an upstream Alchemy issue. The prisma-cli mount and the prisma/web change wait for a Composer release with the commands. + +# Cross-cutting requirements + +- **The emulators never learn a Composer concept.** No emulator route, file or field names an app, address or directory as anything other than an opaque label. A test or lint holds this. +- **Composer's mapping is the only source of truth for "which instance is this node".** Labels are for display. No Composer code finds an instance by reading labels. +- **Architecture boundaries hold.** The framework domain (`0-framework`) still imports nothing from `1-prisma-cloud`. The commands reach the emulators through a boundary dependency-cruiser allows, recorded in an ADR. +- **Every printed URL answers.** `dev url` reports a service as running only when the emulator does, which after #353 means its port accepts connections. +- **Errors follow ADR-0044.** Every failure a user can cause has its own code, raised where it is detected, with why and fix: no `prisma dev` has run in this directory; no node has that address (the known addresses listed); the node's emulator is not running; the service is not running. New codes and any new namespace are recorded in ADR-0044 in the same change. +- **User-facing changes ship with their docs** (`.agents/rules/user-facing-surface-changes.mdc`): `docs/guides/running-locally.md` and the core-concepts skill in Composer, and help text that meets prisma-cli's `docs/product/cli-help-standard.md`, which includes defining "emulator" for a reader who has never used Prisma. +- **`prisma dev` behaves the same from the outside** at every intermediate step, apart from the documented loss of pre-upgrade local state. + +# Transitional-shape constraints + +- The emulator refactor lands one emulator at a time if that keeps PRs reviewable. Each step leaves `prisma dev` fully working on main. +- Until the prisma-cli engine change ships, the Composer family may export the new commands without the host mounting them. The host mounts them only after the engine supports `dev` as both a command and a group. + +# Project DoD + +- [ ] On a machine with nothing running, the getting-started app's `prisma dev module.ts` followed by the documented `curl "$(npx prisma dev url gateway | jq …)"` returns a quote, run from the published `prisma` binary. +- [ ] Two directories running the same app at once get separate instances, and `prisma dev url gateway` in each returns that directory's own service. +- [ ] After Ctrl-C of `prisma dev`, or `prisma dev stop` following a `kill -9`, no process of that directory's app is running and none of its ports is held; the next `prisma dev` gets the same ports. +- [ ] `prisma emulator list` shows the services, databases and buckets of both, each labelled with the directory and address that started it. +- [ ] No emulator source or state file contains an app name, address or directory outside a label value. +- [ ] Composer ADRs record the emulator model (amending ADR-0041) and the new commands (amending ADR-0050). ADR-0044 lists the new codes. +- [ ] `docs/design/10-domains/local-dev.md`, the running-locally guide and the core-concepts skill describe both commands and the model. +- [ ] prisma-cli mounts both commands, with `EXCLUSIONS` entries in the e2e coverage test (local only, no management API). +- [ ] The prisma/web getting-started and local-development pages use `prisma dev url` instead of hard-coded ports. + +# Open questions + +None blocking. Decisions taken by the orchestrator, recorded in `design-notes.md`: the mapping file's location and shape, the label keys Composer writes, the `--json` shapes, and the error code names. + +# References + +- Brief (unvalidated, superseded by this spec where they differ): summarized in the conversation of 2026-10-13. +- `docs/design/10-domains/local-dev.md`, ADR-0041, ADR-0043, ADR-0044, ADR-0050, ADR-0051. +- prisma/composer#350 (merged), prisma/composer#353 (open). +- prisma-cli: `docs/architecture/cli-engine-requirements.md` (R12), `docs/product/cli-help-standard.md`, `docs/product/output-conventions.md`, `packages/cli/tests/e2e-coverage.test.ts`, `packages/cli/tests/mount-coverage.test.ts`. From 77ff3e9980eea82077e95c65099d523bac9c76cb Mon Sep 17 00:00:00 2001 From: willbot Date: Tue, 13 Oct 2026 16:09:46 +0200 Subject: [PATCH 02/17] docs(drive): an emulator instance keeps its port until deleted Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- .drive/projects/emulator-commands/design-notes.md | 4 +++- .drive/projects/emulator-commands/plan.md | 2 +- .../emulator-commands/slices/s1a-compute-instances/spec.md | 5 +++-- .drive/projects/emulator-commands/spec.md | 6 +++--- 4 files changed, 10 insertions(+), 7 deletions(-) diff --git a/.drive/projects/emulator-commands/design-notes.md b/.drive/projects/emulator-commands/design-notes.md index 92cbdee99..3db8fe0c5 100644 --- a/.drive/projects/emulator-commands/design-notes.md +++ b/.drive/projects/emulator-commands/design-notes.md @@ -32,7 +32,9 @@ Decisions taken while shaping, with the reasoning. Will's rulings are marked; th ## Ports (Will, 2026-10-13: agents in many worktrees start and stop environments) -A stopped instance holds no port. Composer remembers each node's last port in the directory's mapping and requests it on the next start. A requested port is a preference: if another instance or process holds it, the emulator gives the lowest free port from 3000 and `prisma dev` prints one line saying the port changed. Failing instead would break an agent's `prisma dev` whenever another worktree happened to take the port. On a first start, ports still go in dependency order (prisma/composer#350). +An instance keeps its port from creation until it is deleted, running or stopped. A deployment carries its port inside the service's environment (`COMPOSER_
_PORT`) and in its dependents' URLs, and Alchemy's no-op reconcile on a warm start never redeploys, so a port that could change between sessions would leave services built for the old port. Releasing ports on stop was considered and dropped for that reason (implementer finding, S1a dispatch 1). The cost: instances left behind by deleted worktrees keep their ports until the later remove command deletes them. + +Composer remembers each node's port in the directory's mapping and requests it only when it has to create the instance again, for example after the emulator lost its state. A requested port is a preference: if another instance or process holds it, the emulator gives the lowest free port from 3000 and `prisma dev` prints one line saying the port changed. Failing instead would break an agent's `prisma dev` whenever another worktree happened to take the port. On a first start, ports still go in dependency order (prisma/composer#350). ## Labels Composer writes diff --git a/.drive/projects/emulator-commands/plan.md b/.drive/projects/emulator-commands/plan.md index 5d8d1d2bb..81191b0ae 100644 --- a/.drive/projects/emulator-commands/plan.md +++ b/.drive/projects/emulator-commands/plan.md @@ -6,7 +6,7 @@ ### S1 — Emulators know nothing about Composer (prisma/composer) -**Outcome.** The Compute, Postgres and bucket emulators key instances by their own ids, accept and return free-form labels, and each lists all its instances. The Prisma Cloud local target creates instances, labels them, and records the per-directory mapping in `.prisma-composer/dev/`. Every Composer operation on the emulators goes through the mapping: deploy, session start and stop, logs, `--fresh` teardown. `prisma dev` behaves the same from outside, except that pre-upgrade local state is discarded. Two directories running one app get separate instances. Port allocation keeps its current rule on a first start: services reserve ports in dependency order (prisma/composer#350), lowest free from 3000. The mapping remembers each node's port and asks for it on every later start. Stopped instances hold no port. When `prisma dev` ends it stops all of the directory's instances, Postgres servers included (the Postgres emulator gains a stop that keeps data). +**Outcome.** The Compute, Postgres and bucket emulators key instances by their own ids, accept and return free-form labels, and each lists all its instances. The Prisma Cloud local target creates instances, labels them, and records the per-directory mapping in `.prisma-composer/dev/`. Every Composer operation on the emulators goes through the mapping: deploy, session start and stop, logs, `--fresh` teardown. `prisma dev` behaves the same from outside, except that pre-upgrade local state is discarded. Two directories running one app get separate instances. Port allocation keeps its current rule on a first start: services reserve ports in dependency order (prisma/composer#350), lowest free from 3000. An instance keeps its port from creation until it is deleted, stopped or not, so a directory's ports never change. The mapping remembers each node's port and requests it when an instance has to be re-created. When `prisma dev` ends it stops all of the directory's instances, Postgres servers included (the Postgres emulator gains a stop that keeps data). **Builds on:** prisma/composer#353 (stack on its branch until it merges). **Hands to S3:** the mapping file and its reader; the emulator client's list-everything calls; ADR amending ADR-0041. diff --git a/.drive/projects/emulator-commands/slices/s1a-compute-instances/spec.md b/.drive/projects/emulator-commands/slices/s1a-compute-instances/spec.md index 30343c476..bdbf6dca3 100644 --- a/.drive/projects/emulator-commands/slices/s1a-compute-instances/spec.md +++ b/.drive/projects/emulator-commands/slices/s1a-compute-instances/spec.md @@ -27,7 +27,8 @@ Two directories running the same app get separate instances. `prisma dev` otherw - Instances are created by the emulator and get an id it generates. Create accepts `labels: Record` and an optional requested port. - Every instance operation is by id: deploy, start, stop, delete, logs, get. There is no app grouping. Bulk start and stop take a list of ids. - `GET` for all instances returns id, port, url, status and labels. -- Ports: a stopped instance holds no port. A requested port is used if it is free; otherwise the emulator gives the lowest free port from 3000 and reports the port it chose. With no request, it gives the lowest free port from 3000. +- Ports: an instance gets its port at creation and keeps it until deleted, running or stopped (a deployment's env and its dependents' URLs contain the port, and a warm start does not redeploy). Create takes an optional requested port: used if free, otherwise the lowest free port from 3000; the response reports the port given. No request → lowest free from 3000. +- The deployment body carries no Composer address. Emulator messages name the instance id; Composer adds the service's address when it rethrows, so what `prisma dev` prints is unchanged. - State written in the old app-keyed format is discarded on load. Local dev state is never migrated across versions (running-locally guide). - #353's readiness behaviour (a service is `running` only once its port accepts connections), the start and stop concurrency rules, crash supervision and the version-skew restart all carry over unchanged, keyed by instance instead of `(app, id)`. @@ -81,6 +82,6 @@ Deliberately out: ## Dispatch plan -1. **Emulator and client.** The Compute emulator and `computeClient` move to instance ids, labels, requested ports, list-all and bulk start and stop, with ports released on stop. The emulator's own tests are migrated and extended. *Hands to 2:* a Compute client keyed by instance id. +1. **Emulator and client.** The Compute emulator and `computeClient` move to instance ids, labels, requested ports, list-all and bulk start and stop. The emulator's own tests are migrated and extended. *Hands to 2:* a Compute client keyed by instance id. 2. **Composer's mapping.** The mapping module, the emulators hook, the providers, attach and teardown move to the mapping, including the change-of-port notice and the two-directory test. *Hands to 3:* `prisma dev` working end to end on instance ids. 3. **Docs.** The ADR, local-dev.md, the guide and the skill. *Hands to:* the PR. diff --git a/.drive/projects/emulator-commands/spec.md b/.drive/projects/emulator-commands/spec.md index 30ae72ecf..ce5fe4cac 100644 --- a/.drive/projects/emulator-commands/spec.md +++ b/.drive/projects/emulator-commands/spec.md @@ -30,7 +30,7 @@ On that model, two commands: - **`prisma dev stop`** stops every instance this directory's app has on the emulators: service processes and Postgres servers. Data stays. An agent runs it before deleting its worktree, or after a `prisma dev` that was killed without a chance to clean up. -When `prisma dev` ends (Ctrl-C, SIGTERM, or a failure), it does the same stop. Today it stops only the service processes and leaves Postgres servers running. A stopped instance holds no port. Composer remembers each node's port in the directory's mapping and asks the emulator for that port on the next start, so ports stay stable within a directory without the emulator reserving them. +When `prisma dev` ends (Ctrl-C, SIGTERM, or a failure), it does the same stop. Today it stops only the service processes and leaves Postgres servers running. An instance keeps its port from creation until it is deleted, whether it is running or stopped, so a directory's ports never change. When an instance has to be re-created (its emulator lost its state), Composer requests the port remembered in the directory's mapping. The shared emulator daemons themselves keep running; other directories may be using them. @@ -38,7 +38,7 @@ The shared emulator daemons themselves keep running; other directories may be us # Non-goals -- **Removing instances and their data.** A worktree that is deleted leaves its stopped instances' records and Postgres data on disk. They hold no process and no port. A remove command comes with the later emulator start/stop project. +- **Removing instances and their data.** A worktree that is deleted leaves its stopped instances' records and Postgres data on disk. They hold no process, but they do keep their ports reserved. A remove command comes with the later emulator start/stop project. - **`prisma emulator start` and `stop`.** Planned; they change what runs on the shared emulators and get their own design. - **Printing a bare value when stdout is a pipe.** The CLI keeps its rule that a piped stdout gets JSON frames. Docs use `jq`. No output-format change to the CLI engine. - **`url` for things Composer didn't start.** `prisma dev url` takes a Composer node address only, never an emulator instance id. @@ -72,7 +72,7 @@ The shared emulator daemons themselves keep running; other directories may be us - [ ] On a machine with nothing running, the getting-started app's `prisma dev module.ts` followed by the documented `curl "$(npx prisma dev url gateway | jq …)"` returns a quote, run from the published `prisma` binary. - [ ] Two directories running the same app at once get separate instances, and `prisma dev url gateway` in each returns that directory's own service. -- [ ] After Ctrl-C of `prisma dev`, or `prisma dev stop` following a `kill -9`, no process of that directory's app is running and none of its ports is held; the next `prisma dev` gets the same ports. +- [ ] After Ctrl-C of `prisma dev`, or `prisma dev stop` following a `kill -9`, no process of that directory's app is running; the next `prisma dev` gets the same ports. - [ ] `prisma emulator list` shows the services, databases and buckets of both, each labelled with the directory and address that started it. - [ ] No emulator source or state file contains an app name, address or directory outside a label value. - [ ] Composer ADRs record the emulator model (amending ADR-0041) and the new commands (amending ADR-0050). ADR-0044 lists the new codes. From 540e7482a3da05450f4c3be7674d3f152afbc8c8 Mon Sep 17 00:00:00 2001 From: willbot Date: Tue, 13 Oct 2026 16:26:35 +0200 Subject: [PATCH 03/17] feat(dev-emulators): key compute instances by ids the emulator generates The Compute emulator no longer knows apps or service addresses. A client creates an instance with free-form labels and an optional requested port; the emulator generates the id, stores the labels verbatim and never reads them. Every operation is by instance id, start and stop take a list of ids, and GET /instances lists every instance. An instance holds its port from creation until it is deleted, running or stopped, because a deployment's env and its dependents' URLs contain the port. A requested port is used when no instance holds it and it is bindable; otherwise the lowest free port from 3000. The deployment body no longer carries an address, so failure messages name the instance id. State from the app-keyed version is discarded on load. A test fails if the emulator's code names an app, address, directory or Composer, or if a label value appears in its state, files or listings outside labels. Callers in local-target and the prisma-cloud extension move in the next commit; `pnpm typecheck` fails for them at this commit. Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- .../src/__tests__/compute-deployment.test.ts | 300 +++----- .../src/__tests__/compute-instances.test.ts | 303 ++++++++ .../src/__tests__/compute-multi-app.test.ts | 157 ----- .../src/__tests__/compute-readiness.test.ts | 223 +++--- .../dev-emulators/src/__tests__/helpers.ts | 17 +- .../0-lowering/dev-emulators/src/client.ts | 159 +++-- .../dev-emulators/src/compute-main.ts | 662 +++++++++--------- .../dev-emulators/src/exports/index.ts | 4 +- 8 files changed, 942 insertions(+), 883 deletions(-) create mode 100644 packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-instances.test.ts delete mode 100644 packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-multi-app.test.ts diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-deployment.test.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-deployment.test.ts index f3c77b806..79364bd73 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-deployment.test.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-deployment.test.ts @@ -6,7 +6,7 @@ * started. */ import { afterEach, beforeEach, describe, expect, test } from 'bun:test'; -import { computeClient, type ServiceInfo } from '../client.ts'; +import { computeClient, type InstanceInfo } from '../client.ts'; import { isPidAlive, stopDaemon } from '../daemon.ts'; import { CRASHING_BOOTSTRAP, @@ -46,30 +46,26 @@ function baseEnv(extra: Record = {}): Record { return { PATH: path, ...extra }; } -async function deployedService(app: string, id: string): Promise { - const client = computeClient({ registryRoot }); - const list = await client.listServices(app); - const svc = list.find((s) => s.id === id); - if (!svc) throw new Error(`service "${id}" not found in app "${app}"`); - return svc; +async function deployedService(id: string): Promise { + const instance = await computeClient({ registryRoot }).getInstance(id); + if (!instance) throw new Error(`instance "${id}" not found`); + return instance; } describe('deployment lifecycle', () => { test('a deployment spawns bun bootstrap.js and the service actually serves HTTP', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-a', 'web'); + const reserved = await client.createInstance({ labels: {} }); - await client.putDeployment('app-a', 'web', { - address: 'app-a.web', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'hash-v1', env: baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv('hello from v1') }), - port: reserved.port, }); await waitFor(async () => { - const svc = await deployedService('app-a', 'web'); + const svc = await deployedService(reserved.id); return svc.status === 'running'; }, 5000); @@ -80,55 +76,47 @@ describe('deployment lifecycle', () => { test('an identical redeploy (same hash and env) does not restart the running child', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-b', 'web'); + const reserved = await client.createInstance({ labels: {} }); const env = baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv('v1') }); - await client.putDeployment('app-b', 'web', { - address: 'app-b.web', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'same-hash', env, - port: reserved.port, }); - await waitFor(async () => (await deployedService('app-b', 'web')).status === 'running', 5000); - const before = await deployedService('app-b', 'web'); + await waitFor(async () => (await deployedService(reserved.id)).status === 'running', 5000); + const before = await deployedService(reserved.id); - await client.putDeployment('app-b', 'web', { - address: 'app-b.web', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'same-hash', env, - port: reserved.port, }); - const after = await deployedService('app-b', 'web'); + const after = await deployedService(reserved.id); expect(after.pid).toBe(before.pid); }); test('a redeploy with a changed artifactHash restarts the child (new pid)', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-c', 'web'); + const reserved = await client.createInstance({ labels: {} }); const env = baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv('v1') }); - await client.putDeployment('app-c', 'web', { - address: 'app-c.web', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'hash-1', env, - port: reserved.port, }); - await waitFor(async () => (await deployedService('app-c', 'web')).status === 'running', 5000); - const before = await deployedService('app-c', 'web'); + await waitFor(async () => (await deployedService(reserved.id)).status === 'running', 5000); + const before = await deployedService(reserved.id); - await client.putDeployment('app-c', 'web', { - address: 'app-c.web', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'hash-2', env, - port: reserved.port, }); await waitFor(async () => { - const svc = await deployedService('app-c', 'web'); + const svc = await deployedService(reserved.id); return svc.status === 'running' && svc.pid !== before.pid; }, 5000); }); @@ -136,27 +124,23 @@ describe('deployment lifecycle', () => { test('a redeploy with a changed env (same hash) also restarts the child', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-d', 'web'); + const reserved = await client.createInstance({ labels: {} }); - await client.putDeployment('app-d', 'web', { - address: 'app-d.web', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'stable-hash', env: baseEnv({ PORT: String(reserved.port), EXTRA: 'a', ...servingBootstrapEnv('v1') }), - port: reserved.port, }); - await waitFor(async () => (await deployedService('app-d', 'web')).status === 'running', 5000); - const before = await deployedService('app-d', 'web'); + await waitFor(async () => (await deployedService(reserved.id)).status === 'running', 5000); + const before = await deployedService(reserved.id); - await client.putDeployment('app-d', 'web', { - address: 'app-d.web', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'stable-hash', env: baseEnv({ PORT: String(reserved.port), EXTRA: 'b', ...servingBootstrapEnv('v1') }), - port: reserved.port, }); await waitFor(async () => { - const svc = await deployedService('app-d', 'web'); + const svc = await deployedService(reserved.id); return svc.status === 'running' && svc.pid !== before.pid; }, 5000); }); @@ -164,24 +148,22 @@ describe('deployment lifecycle', () => { test('a stopped service always starts on redeploy, even with an identical hash and env', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-e', 'web'); + const reserved = await client.createInstance({ labels: {} }); const env = baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv('v1') }); const deployment = { - address: 'app-e.web', artifactDir, artifactHash: 'h', env, - port: reserved.port, }; - await client.putDeployment('app-e', 'web', deployment); - await waitFor(async () => (await deployedService('app-e', 'web')).status === 'running', 5000); + await client.putDeployment(reserved.id, deployment); + await waitFor(async () => (await deployedService(reserved.id)).status === 'running', 5000); - await client.stopApp('app-e'); - await waitFor(async () => (await deployedService('app-e', 'web')).status === 'stopped', 5000); + await client.stopInstances([reserved.id]); + await waitFor(async () => (await deployedService(reserved.id)).status === 'stopped', 5000); - await client.putDeployment('app-e', 'web', deployment); - await waitFor(async () => (await deployedService('app-e', 'web')).status === 'running', 5000); + await client.putDeployment(reserved.id, deployment); + await waitFor(async () => (await deployedService(reserved.id)).status === 'running', 5000); }); }); @@ -189,31 +171,24 @@ describe('crash supervision', () => { test('an unexpected crash goes to backoff and restarts after the pinned 1s·2ⁿ delay', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(CRASHING_BOOTSTRAP); - const reserved = await client.ensureService('app-f', 'crasher'); + const reserved = await client.createInstance({ labels: {} }); await expect( - client.putDeployment('app-f', 'crasher', { - address: 'app-f.crasher', + client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }), ).rejects.toThrow('before it accepted connections'); - await waitFor( - async () => (await deployedService('app-f', 'crasher')).status === 'backoff', - 3000, - ); + await waitFor(async () => (await deployedService(reserved.id)).status === 'backoff', 3000); // The first backoff step is the base case: 1s. Give it a comfortable // margin and confirm it actually restarts (the log shows the pinned // `[emulator] exited ... restarting in 1s` line), not just holds at // `backoff` forever. await waitFor(async () => { - const text = await fetch(`${daemonUrl}/apps/app-f/services/crasher/logs`).then((r) => - r.text(), - ); + const text = await fetch(`${daemonUrl}/instances/${reserved.id}/logs`).then((r) => r.text()); return text.includes('[emulator] exited') && text.includes('restarting in 1s'); }, 3000); }, 10_000); @@ -221,37 +196,26 @@ describe('crash supervision', () => { test('held after 5 consecutive fast crashes, and a redeploy clears held', async () => { const client = computeClient({ registryRoot }); const crashingDir = writeBootstrap(CRASHING_BOOTSTRAP); - const reserved = await client.ensureService('app-g', 'crasher'); + const reserved = await client.createInstance({ labels: {} }); await expect( - client.putDeployment('app-g', 'crasher', { - address: 'app-g.crasher', + client.putDeployment(reserved.id, { artifactDir: crashingDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }), ).rejects.toThrow('before it accepted connections'); - await waitFor( - async () => (await deployedService('app-g', 'crasher')).status === 'held', - 20_000, - 250, - ); + await waitFor(async () => (await deployedService(reserved.id)).status === 'held', 20_000, 250); const stableDir = writeBootstrap(SERVING_BOOTSTRAP); - await client.putDeployment('app-g', 'crasher', { - address: 'app-g.crasher', + await client.putDeployment(reserved.id, { artifactDir: stableDir, artifactHash: 'h2', env: baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv('recovered') }), - port: reserved.port, }); - await waitFor( - async () => (await deployedService('app-g', 'crasher')).status === 'running', - 5000, - ); + await waitFor(async () => (await deployedService(reserved.id)).status === 'running', 5000); const res = await waitForHttp(`http://127.0.0.1:${String(reserved.port)}`, 3000); expect(await res.text()).toBe('recovered'); }, 30_000); @@ -261,20 +225,15 @@ describe('stop truthfulness', () => { test('a child that ignores SIGTERM is only listed stopped once SIGKILL has actually landed', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(`process.on('SIGTERM', () => {});\n${SERVING_BOOTSTRAP}`); - const reserved = await client.ensureService('app-i', 'stubborn'); + const reserved = await client.createInstance({ labels: {} }); - await client.putDeployment('app-i', 'stubborn', { - address: 'app-i.stubborn', + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv('stubborn') }), - port: reserved.port, }); - await waitFor( - async () => (await deployedService('app-i', 'stubborn')).status === 'running', - 5000, - ); - const pid = (await deployedService('app-i', 'stubborn')).pid; + await waitFor(async () => (await deployedService(reserved.id)).status === 'running', 5000); + const pid = (await deployedService(reserved.id)).pid; if (pid === undefined) throw new Error('expected the running service to report a pid'); // `running` flips as soon as the child is spawned (spec: spawning IS // the observable action), before the child has actually finished @@ -290,7 +249,7 @@ describe('stop truthfulness', () => { let sawStoppedBeforeDeath = false; const watcher = (async () => { for (;;) { - const svc = await deployedService('app-i', 'stubborn'); + const svc = await deployedService(reserved.id); if (svc.status === 'stopped') { if (isPidAlive(pid)) sawStoppedBeforeDeath = true; return; @@ -300,7 +259,7 @@ describe('stop truthfulness', () => { })(); const stopStart = Date.now(); - await client.stopApp('app-i'); + await client.stopInstances([reserved.id]); const stopElapsedMs = Date.now() - stopStart; await watcher; @@ -313,54 +272,41 @@ describe('stop truthfulness', () => { }, 15_000); }); -describe('session resume (POST /apps//start)', () => { +describe('session resume (POST /instances/start)', () => { test('stop then start brings children back up on the same ports with new pids', async () => { const client = computeClient({ registryRoot }); const oneDir = writeBootstrap(SERVING_BOOTSTRAP); const twoDir = writeBootstrap(SERVING_BOOTSTRAP); - const one = await client.ensureService('app-j', 'web'); - const two = await client.ensureService('app-j', 'worker'); + const one = await client.createInstance({ labels: {} }); + const two = await client.createInstance({ labels: {} }); - await client.putDeployment('app-j', 'web', { - address: 'app-j.web', + await client.putDeployment(one.id, { artifactDir: oneDir, artifactHash: 'h', env: baseEnv({ PORT: String(one.port), ...servingBootstrapEnv('one') }), - port: one.port, }); - await client.putDeployment('app-j', 'worker', { - address: 'app-j.worker', + await client.putDeployment(two.id, { artifactDir: twoDir, artifactHash: 'h', env: baseEnv({ PORT: String(two.port), ...servingBootstrapEnv('two') }), - port: two.port, }); - await waitFor(async () => (await deployedService('app-j', 'web')).status === 'running', 5000); - await waitFor( - async () => (await deployedService('app-j', 'worker')).status === 'running', - 5000, - ); + await waitFor(async () => (await deployedService(one.id)).status === 'running', 5000); + await waitFor(async () => (await deployedService(two.id)).status === 'running', 5000); const pidsBefore = { - web: (await deployedService('app-j', 'web')).pid, - worker: (await deployedService('app-j', 'worker')).pid, + web: (await deployedService(one.id)).pid, + worker: (await deployedService(two.id)).pid, }; - await client.stopApp('app-j'); - await waitFor(async () => (await deployedService('app-j', 'web')).status === 'stopped', 5000); - await waitFor( - async () => (await deployedService('app-j', 'worker')).status === 'stopped', - 5000, - ); + await client.stopInstances([one.id, two.id]); + await waitFor(async () => (await deployedService(one.id)).status === 'stopped', 5000); + await waitFor(async () => (await deployedService(two.id)).status === 'stopped', 5000); - await client.startApp('app-j'); - await waitFor(async () => (await deployedService('app-j', 'web')).status === 'running', 5000); - await waitFor( - async () => (await deployedService('app-j', 'worker')).status === 'running', - 5000, - ); + await client.startInstances([one.id, two.id]); + await waitFor(async () => (await deployedService(one.id)).status === 'running', 5000); + await waitFor(async () => (await deployedService(two.id)).status === 'running', 5000); - const webAfter = await deployedService('app-j', 'web'); - const workerAfter = await deployedService('app-j', 'worker'); + const webAfter = await deployedService(one.id); + const workerAfter = await deployedService(two.id); expect(webAfter.pid).not.toBe(pidsBefore.web); expect(workerAfter.pid).not.toBe(pidsBefore.worker); expect(webAfter.port).toBe(one.port); @@ -375,24 +321,22 @@ describe('session resume (POST /apps//start)', () => { test('a service with no stored deployment is skipped without error', async () => { const client = computeClient({ registryRoot }); const deployedDir = writeBootstrap(SERVING_BOOTSTRAP); - const deployed = await client.ensureService('app-k', 'web'); - await client.ensureService('app-k', 'never-deployed'); + const deployed = await client.createInstance({ labels: {} }); + const neverDeployedInstance = await client.createInstance({ labels: {} }); - await client.putDeployment('app-k', 'web', { - address: 'app-k.web', + await client.putDeployment(deployed.id, { artifactDir: deployedDir, artifactHash: 'h', env: baseEnv({ PORT: String(deployed.port), ...servingBootstrapEnv('deployed') }), - port: deployed.port, }); - await waitFor(async () => (await deployedService('app-k', 'web')).status === 'running', 5000); - await client.stopApp('app-k'); - await waitFor(async () => (await deployedService('app-k', 'web')).status === 'stopped', 5000); + await waitFor(async () => (await deployedService(deployed.id)).status === 'running', 5000); + await client.stopInstances([deployed.id, neverDeployedInstance.id]); + await waitFor(async () => (await deployedService(deployed.id)).status === 'stopped', 5000); - await client.startApp('app-k'); + await client.startInstances([deployed.id, neverDeployedInstance.id]); - await waitFor(async () => (await deployedService('app-k', 'web')).status === 'running', 5000); - const neverDeployed = await deployedService('app-k', 'never-deployed'); + await waitFor(async () => (await deployedService(deployed.id)).status === 'running', 5000); + const neverDeployed = await deployedService(neverDeployedInstance.id); expect(neverDeployed.status).toBe('stopped'); expect(neverDeployed.pid).toBeUndefined(); }, 15_000); @@ -400,24 +344,20 @@ describe('session resume (POST /apps//start)', () => { test('a held service resumes on start', async () => { const client = computeClient({ registryRoot }); const crashingDir = writeBootstrap(CRASHING_BOOTSTRAP); - const reserved = await client.ensureService('app-l', 'crasher'); + const reserved = await client.createInstance({ labels: {} }); await expect( - client.putDeployment('app-l', 'crasher', { - address: 'app-l.crasher', + client.putDeployment(reserved.id, { artifactDir: crashingDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }), ).rejects.toThrow('before it accepted connections'); - await waitFor( - async () => (await deployedService('app-l', 'crasher')).status === 'held', - 20_000, - 250, - ); + await waitFor(async () => (await deployedService(reserved.id)).status === 'held', 20_000, 250); - await expect(client.startApp('app-l')).rejects.toThrow('before it accepted connections'); + await expect(client.startInstances([reserved.id])).rejects.toThrow( + 'before it accepted connections', + ); // The crashing artifact is still what's stored, so the resumed child // crashes again shortly — but it takes 5 fresh fast exits to re-enter @@ -425,31 +365,25 @@ describe('session resume (POST /apps//start)', () => { // steps. Seeing anything other than `held` shortly after `/start` // proves the resume actually cleared it rather than no-op'ing on a // held service. - await waitFor( - async () => (await deployedService('app-l', 'crasher')).status !== 'held', - 2000, - 50, - ); + await waitFor(async () => (await deployedService(reserved.id)).status !== 'held', 2000, 50); }, 30_000); test('start on an already-running app is a no-op — pids unchanged', async () => { const client = computeClient({ registryRoot }); const dir = writeBootstrap(SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-m', 'web'); + const reserved = await client.createInstance({ labels: {} }); - await client.putDeployment('app-m', 'web', { - address: 'app-m.web', + await client.putDeployment(reserved.id, { artifactDir: dir, artifactHash: 'h', env: baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv('steady') }), - port: reserved.port, }); - await waitFor(async () => (await deployedService('app-m', 'web')).status === 'running', 5000); - const before = await deployedService('app-m', 'web'); + await waitFor(async () => (await deployedService(reserved.id)).status === 'running', 5000); + const before = await deployedService(reserved.id); - await client.startApp('app-m'); + await client.startInstances([reserved.id]); - const after = await deployedService('app-m', 'web'); + const after = await deployedService(reserved.id); expect(after.status).toBe('running'); expect(after.pid).toBe(before.pid); }, 10_000); @@ -459,23 +393,21 @@ describe('log follow', () => { test('follow yields the child output plus [emulator] supervision lines', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(CRASHING_BOOTSTRAP); - const reserved = await client.ensureService('app-h', 'crasher'); + const reserved = await client.createInstance({ labels: {} }); const chunks: string[] = []; const controller = new AbortController(); const following = (async () => { - for await (const chunk of client.followLogs('app-h', 'crasher', controller.signal)) { + for await (const chunk of client.followLogs(reserved.id, controller.signal)) { chunks.push(chunk); } })().catch(() => undefined); await expect( - client.putDeployment('app-h', 'crasher', { - address: 'app-h.crasher', + client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }), ).rejects.toThrow('before it accepted connections'); @@ -487,22 +419,20 @@ describe('log follow', () => { test('?tail=N replays only the last N history lines before going live', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(LOGGING_BOOTSTRAP); - const reserved = await client.ensureService('app-tail', 'logger'); - await client.putDeployment('app-tail', 'logger', { - address: 'app-tail.logger', + const reserved = await client.createInstance({ labels: {} }); + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 't', env: baseEnv({ PORT: String(reserved.port), LINES: '10' }), - port: reserved.port, }); // All ten lines are on disk before the follow attaches — so what the // follow returns is backlog, not live output racing the loop. await waitFor( async () => - ( - await fetch(`${daemonUrl}/apps/app-tail/services/logger/logs`).then((r) => r.text()) - ).includes('line-10\n'), + (await fetch(`${daemonUrl}/instances/${reserved.id}/logs`).then((r) => r.text())).includes( + 'line-10\n', + ), 5000, 100, ); @@ -510,7 +440,7 @@ describe('log follow', () => { const chunks: string[] = []; const controller = new AbortController(); await (async () => { - for await (const chunk of client.followLogs('app-tail', 'logger', controller.signal, { + for await (const chunk of client.followLogs(reserved.id, controller.signal, { tail: 3, })) { chunks.push(chunk); @@ -528,38 +458,28 @@ describe('log follow', () => { }); describe('log reset on a fresh start', () => { - const readLog = (app: string, id: string): Promise => - fetch(`${daemonUrl}/apps/${app}/services/${id}/logs`).then((r) => r.text()); + const readLog = (id: string): Promise => + fetch(`${daemonUrl}/instances/${id}/logs`).then((r) => r.text()); test('a new deployment starts with an empty log — the prior run is gone', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-reset', 'web'); + const reserved = await client.createInstance({ labels: {} }); const deploy = (hash: string, body: string): Promise => - client.putDeployment('app-reset', 'web', { - address: 'app-reset.web', + client.putDeployment(reserved.id, { artifactDir, artifactHash: hash, env: baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv(body) }), - port: reserved.port, }); await deploy('v1', 'one'); - await waitFor( - async () => (await readLog('app-reset', 'web')).includes('booted: one'), - 5000, - 100, - ); + await waitFor(async () => (await readLog(reserved.id)).includes('booted: one'), 5000, 100); // A changed deployment restarts the service — a fresh start. await deploy('v2', 'two'); - await waitFor( - async () => (await readLog('app-reset', 'web')).includes('booted: two'), - 5000, - 100, - ); + await waitFor(async () => (await readLog(reserved.id)).includes('booted: two'), 5000, 100); - const logText = await readLog('app-reset', 'web'); + const logText = await readLog(reserved.id); expect(logText).toContain('booted: two'); expect(logText).not.toContain('booted: one'); }, 15_000); @@ -567,27 +487,21 @@ describe('log reset on a fresh start', () => { test('a follower survives the clear and picks up the fresh log', async () => { const client = computeClient({ registryRoot }); const artifactDir = writeBootstrap(SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-reset-follow', 'web'); + const reserved = await client.createInstance({ labels: {} }); const deploy = (hash: string, body: string): Promise => - client.putDeployment('app-reset-follow', 'web', { - address: 'app-reset-follow.web', + client.putDeployment(reserved.id, { artifactDir, artifactHash: hash, env: baseEnv({ PORT: String(reserved.port), ...servingBootstrapEnv(body) }), - port: reserved.port, }); await deploy('v1', 'first'); - await waitFor( - async () => (await readLog('app-reset-follow', 'web')).includes('booted: first'), - 5000, - 100, - ); + await waitFor(async () => (await readLog(reserved.id)).includes('booted: first'), 5000, 100); const chunks: string[] = []; const controller = new AbortController(); const following = (async () => { - for await (const chunk of client.followLogs('app-reset-follow', 'web', controller.signal, { + for await (const chunk of client.followLogs(reserved.id, controller.signal, { tail: 50, })) { chunks.push(chunk); diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-instances.test.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-instances.test.ts new file mode 100644 index 000000000..8186f5690 --- /dev/null +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-instances.test.ts @@ -0,0 +1,303 @@ +/** + * Compute instances: ids the emulator generates, labels it stores without + * reading, ports held from creation until delete, and isolation between + * instances whose labels are identical. + */ +import { afterEach, beforeEach, describe, expect, test } from 'bun:test'; +import * as fs from 'node:fs'; +import * as net from 'node:net'; +import * as path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import getPort, { portNumbers } from 'get-port'; +import { type ComputeClient, computeClient } from '../client.ts'; +import { daemonStateDir, ensureDaemon, stopDaemon } from '../daemon.ts'; +import { + ensureFreshDaemon, + entryFor, + SERVING_BOOTSTRAP, + servingBootstrapEnv, + skipContendedServicePorts, + tempDir, + waitForHttp, + writeBootstrap, +} from './helpers.ts'; + +let registryRoot: string; +let client: ComputeClient; + +beforeEach(async () => { + registryRoot = tempDir('compute-instances-registry'); + await ensureFreshDaemon('compute', registryRoot); + client = computeClient({ registryRoot }); + await skipContendedServicePorts(client); +}); + +afterEach(async () => { + await stopDaemon('compute', { registryRoot }).catch(() => undefined); +}); + +function baseEnv(extra: Record = {}): Record { + const path = process.env['PATH']; + if (!path) throw new Error('PATH is not set in this shell'); + return { PATH: path, ...extra }; +} + +async function deployServing(id: string, port: number, body: string): Promise { + await client.putDeployment(id, { + artifactDir: writeBootstrap(SERVING_BOOTSTRAP), + artifactHash: body, + env: baseEnv({ PORT: String(port), ...servingBootstrapEnv(body) }), + }); +} + +async function statusOf(id: string): Promise { + return (await client.getInstance(id))?.status; +} + +describe('creating and listing instances', () => { + test('the emulator generates distinct ids and returns the labels verbatim', async () => { + const labels = { 'composer.app': 'My App', 'free form': 'any value: /with/slashes' }; + const one = await client.createInstance({ labels }); + const two = await client.createInstance({ labels }); + + expect(one.id).toMatch(/^svc_[0-9a-f]{16}$/); + expect(two.id).not.toBe(one.id); + expect(one.labels).toEqual(labels); + expect(one.status).toBe('stopped'); + expect(one.url).toBe(`http://localhost:${String(one.port)}`); + + const listed = await client.listInstances(); + expect(listed.find((i) => i.id === one.id)?.labels).toEqual(labels); + expect(listed.find((i) => i.id === two.id)?.labels).toEqual(labels); + }); + + test('getting an unknown instance returns nothing', async () => { + expect(await client.getInstance('svc_0000000000000000')).toBeUndefined(); + }); + + test('a start or stop naming an unknown instance fails and names it', async () => { + const known = await client.createInstance({ labels: {} }); + await expect(client.startInstances([known.id, 'svc_0000000000000000'])).rejects.toThrow( + '"svc_0000000000000000"', + ); + await expect(client.stopInstances(['svc_0000000000000000'])).rejects.toThrow('(404)'); + }); + + test('a malformed instance id is rejected with 400', async () => { + const res = await fetch(`${client.baseUrl}/instances/web`); + expect(res.status).toBe(400); + expect(await res.text()).toContain('"web"'); + }); + + test('labels that are not all strings are rejected with 400', async () => { + const res = await fetch(`${client.baseUrl}/instances`, { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify({ labels: { count: 1 } }), + }); + expect(res.status).toBe(400); + }); +}); + +describe('ports', () => { + test('two instances get distinct ports, both at or above 3000', async () => { + const one = await client.createInstance({ labels: {} }); + const two = await client.createInstance({ labels: {} }); + + expect(one.port).not.toBe(two.port); + expect(one.port).toBeGreaterThanOrEqual(3000); + expect(two.port).toBeGreaterThanOrEqual(3000); + }); + + test('a requested port that is free is given', async () => { + const requested = await getPort({ port: portNumbers(20_000, 21_000) }); + + const created = await client.createInstance({ labels: {}, port: requested }); + + expect(created.port).toBe(requested); + }); + + test('a requested port another instance holds gives the lowest free port instead', async () => { + const holder = await client.createInstance({ labels: {} }); + + const created = await client.createInstance({ labels: {}, port: holder.port }); + await client.deleteInstance(created.id); + const unrequested = await client.createInstance({ labels: {} }); + + expect(created.port).not.toBe(holder.port); + expect(created.port).toBe(unrequested.port); + }); + + test('a requested port a process outside the emulator has bound gives another port', async () => { + const requested = await getPort({ port: portNumbers(21_000, 22_000) }); + const squatter = net.createServer(); + await new Promise((resolve) => squatter.listen(requested, resolve)); + try { + const created = await client.createInstance({ labels: {}, port: requested }); + expect(created.port).not.toBe(requested); + expect(created.port).toBeGreaterThanOrEqual(3000); + } finally { + squatter.close(); + } + }); + + test('a stopped instance keeps its port, and no new instance is given it', async () => { + const one = await client.createInstance({ labels: {} }); + await deployServing(one.id, one.port, 'one'); + await client.stopInstances([one.id]); + + const created = await client.createInstance({ labels: {}, port: one.port }); + + expect((await client.getInstance(one.id))?.port).toBe(one.port); + expect(created.port).not.toBe(one.port); + }, 15_000); + + test("a deleted instance's port can be requested again", async () => { + const one = await client.createInstance({ labels: {} }); + await client.deleteInstance(one.id); + + const again = await client.createInstance({ labels: {}, port: one.port }); + + expect(again.port).toBe(one.port); + expect(again.id).not.toBe(one.id); + }); +}); + +describe('isolation between instances with the same labels', () => { + const labels = { 'composer.app': 'same-app', 'composer.address': 'web' }; + + test('both serve on their own ports', async () => { + const one = await client.createInstance({ labels }); + const two = await client.createInstance({ labels }); + + await deployServing(one.id, one.port, 'from one'); + await deployServing(two.id, two.port, 'from two'); + + expect(await (await waitForHttp(`http://127.0.0.1:${String(one.port)}`, 3000)).text()).toBe( + 'from one', + ); + expect(await (await waitForHttp(`http://127.0.0.1:${String(two.port)}`, 3000)).text()).toBe( + 'from two', + ); + }, 15_000); + + test('stopping one leaves the other running', async () => { + const one = await client.createInstance({ labels }); + const two = await client.createInstance({ labels }); + await deployServing(one.id, one.port, 'one'); + await deployServing(two.id, two.port, 'two'); + + await client.stopInstances([one.id]); + + expect(await statusOf(one.id)).toBe('stopped'); + expect(await statusOf(two.id)).toBe('running'); + }, 15_000); + + test('deleting one leaves the other', async () => { + const one = await client.createInstance({ labels }); + const two = await client.createInstance({ labels }); + + await client.deleteInstance(one.id); + + expect(await client.getInstance(one.id)).toBeUndefined(); + expect((await client.getInstance(two.id))?.labels).toEqual(labels); + }); +}); + +describe('persisted state', () => { + test('instances, their labels and their ports survive an emulator restart', async () => { + const created = await client.createInstance({ labels: { note: 'kept' } }); + + await stopDaemon('compute', { registryRoot }); + await ensureDaemon('compute', entryFor('compute'), { registryRoot }); + + const reloaded = await computeClient({ registryRoot }).getInstance(created.id); + expect(reloaded?.labels).toEqual({ note: 'kept' }); + expect(reloaded?.port).toBe(created.port); + }); + + test('state written by an earlier version, keyed by app, is discarded on load', async () => { + await stopDaemon('compute', { registryRoot }); + const stateDir = daemonStateDir(registryRoot, 'compute'); + fs.rmSync(path.join(stateDir, 'instances.json'), { force: true }); + const legacyLogDir = path.join(stateDir, 'logs', 'legacy-app'); + fs.mkdirSync(legacyLogDir, { recursive: true }); + fs.writeFileSync(path.join(legacyLogDir, 'web.log'), 'old\n'); + fs.writeFileSync( + path.join(stateDir, 'apps.json'), + JSON.stringify({ + 'legacy-app': { + services: { + web: { id: 'web', address: 'web', port: 3000, status: 'stopped', logPath: '/x' }, + }, + }, + }), + ); + + await ensureDaemon('compute', entryFor('compute'), { registryRoot }); + + expect(await computeClient({ registryRoot }).listInstances()).toEqual([]); + expect(fs.existsSync(path.join(stateDir, 'apps.json'))).toBe(false); + expect(fs.existsSync(legacyLogDir)).toBe(false); + }); +}); + +/** + * The emulator knows nothing about Composer: an app name, a node address or + * a project directory may reach it only as label values. These checks fail + * when the emulator's code names one of those concepts, or when one of them + * ends up in its state or files anywhere other than inside labels. + */ +describe('the emulator stores Composer names only in labels', () => { + const markers = { + 'composer.app': 'zz-app-marker', + 'composer.address': 'zz-address-marker', + 'composer.directory': '/zz-directory-marker/project', + }; + const markerValues = Object.values(markers); + + function withoutLabels(value: unknown): unknown { + if (Array.isArray(value)) return value.map(withoutLabels); + if (typeof value !== 'object' || value === null) return value; + return Object.fromEntries( + Object.entries(value) + .filter(([key]) => key !== 'labels') + .map(([key, field]) => [key, withoutLabels(field)]), + ); + } + + function filesUnder(dir: string): string[] { + return fs + .readdirSync(dir, { withFileTypes: true, recursive: true }) + .map((entry) => path.join(entry.parentPath, entry.name)); + } + + test('state, file names and listings hold the label values only inside labels', async () => { + const created = await client.createInstance({ labels: markers }); + await deployServing(created.id, created.port, 'marker test'); + await client.stopInstances([created.id]); + + const stateDir = daemonStateDir(registryRoot, 'compute'); + const state: unknown = JSON.parse( + fs.readFileSync(path.join(stateDir, 'instances.json'), 'utf8'), + ); + const listed: unknown = await (await fetch(`${client.baseUrl}/instances`)).json(); + + for (const marker of markerValues) { + expect(JSON.stringify(state)).toContain(marker); + expect(JSON.stringify(withoutLabels(state))).not.toContain(marker); + expect(JSON.stringify(withoutLabels(listed))).not.toContain(marker); + for (const file of filesUnder(stateDir)) expect(file).not.toContain(marker); + } + }, 15_000); + + test("the emulator's code names no app, address, directory or Composer concept", () => { + const source = fs.readFileSync( + fileURLToPath(new URL('../compute-main.ts', import.meta.url)), + 'utf8', + ); + const code = source.replace(/\/\*[\s\S]*?\*\//g, '').replace(/\/\/.*$/gm, ''); + + expect(code).not.toMatch(/\b(apps?|appName|address(es)?|directory|composer)\b/i); + }); +}); diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-multi-app.test.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-multi-app.test.ts deleted file mode 100644 index 871f5510f..000000000 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-multi-app.test.ts +++ /dev/null @@ -1,157 +0,0 @@ -/** - * Multi-app isolation on the Compute emulator (local-dev spec § 2): two - * different apps with the SAME service name must get distinct, non-colliding - * ports, and each other's admin operations must not cross-affect. Also - * covers the shared path-segment hygiene rule. - */ -import { afterEach, beforeEach, describe, expect, test } from 'bun:test'; -import { computeClient } from '../client.ts'; -import { ensureDaemon, stopDaemon } from '../daemon.ts'; -import { - ensureFreshDaemon, - entryFor, - SERVING_BOOTSTRAP, - servingBootstrapEnv, - skipContendedServicePorts, - tempDir, - waitFor, - waitForHttp, - writeBootstrap, -} from './helpers.ts'; - -let registryRoot: string; - -beforeEach(async () => { - registryRoot = tempDir('compute-multi-app-registry'); - await ensureFreshDaemon('compute', registryRoot); - // The "serving" tests actually bind their reserved port (Bun.serve) — - // steer past any low service port a process outside this test's control - // already holds, the same class of shared-machine contention the - // daemon-port retry feature addresses one layer down. - await skipContendedServicePorts(computeClient({ registryRoot })); -}); - -afterEach(async () => { - await stopDaemon('compute', { registryRoot }).catch(() => undefined); -}); - -function baseEnv(extra: Record = {}): Record { - const path = process.env['PATH']; - if (!path) throw new Error('PATH is not set in this shell'); - return { PATH: path, ...extra }; -} - -test('two apps with the same service id get distinct ports, both ≥ 3000', async () => { - const client = computeClient({ registryRoot }); - const one = await client.ensureService('tenant-one', 'web'); - const two = await client.ensureService('tenant-two', 'web'); - - expect(one.port).not.toBe(two.port); - expect(one.port).toBeGreaterThanOrEqual(3000); - expect(two.port).toBeGreaterThanOrEqual(3000); -}); - -test('deploying and serving the same service id under two apps never collides', async () => { - const client = computeClient({ registryRoot }); - const oneDir = writeBootstrap(SERVING_BOOTSTRAP); - const twoDir = writeBootstrap(SERVING_BOOTSTRAP); - - const one = await client.ensureService('tenant-one', 'web'); - const two = await client.ensureService('tenant-two', 'web'); - - await client.putDeployment('tenant-one', 'web', { - address: 'tenant-one.web', - artifactDir: oneDir, - artifactHash: 'h1', - env: baseEnv({ PORT: String(one.port), ...servingBootstrapEnv('from tenant one') }), - port: one.port, - }); - await client.putDeployment('tenant-two', 'web', { - address: 'tenant-two.web', - artifactDir: twoDir, - artifactHash: 'h1', - env: baseEnv({ PORT: String(two.port), ...servingBootstrapEnv('from tenant two') }), - port: two.port, - }); - - const resOne = await waitForHttp(`http://127.0.0.1:${String(one.port)}`, 3000); - const resTwo = await waitForHttp(`http://127.0.0.1:${String(two.port)}`, 3000); - expect(await resOne.text()).toBe('from tenant one'); - expect(await resTwo.text()).toBe('from tenant two'); -}); - -test("stopping one app's services leaves the other app's services running", async () => { - const client = computeClient({ registryRoot }); - const oneDir = writeBootstrap(SERVING_BOOTSTRAP); - const twoDir = writeBootstrap(SERVING_BOOTSTRAP); - const one = await client.ensureService('tenant-one', 'web'); - const two = await client.ensureService('tenant-two', 'web'); - - await client.putDeployment('tenant-one', 'web', { - address: 'tenant-one.web', - artifactDir: oneDir, - artifactHash: 'h', - env: baseEnv({ PORT: String(one.port), ...servingBootstrapEnv('one') }), - port: one.port, - }); - await client.putDeployment('tenant-two', 'web', { - address: 'tenant-two.web', - artifactDir: twoDir, - artifactHash: 'h', - env: baseEnv({ PORT: String(two.port), ...servingBootstrapEnv('two') }), - port: two.port, - }); - await waitFor(async () => { - const listOne = await client.listServices('tenant-one'); - const listTwo = await client.listServices('tenant-two'); - return listOne[0]?.status === 'running' && listTwo[0]?.status === 'running'; - }, 5000); - - await client.stopApp('tenant-one'); - - await waitFor( - async () => (await client.listServices('tenant-one'))[0]?.status === 'stopped', - 5000, - ); - const twoList = await client.listServices('tenant-two'); - expect(twoList[0]?.status).toBe('running'); -}); - -test('deleting one app leaves the other app registered', async () => { - const client = computeClient({ registryRoot }); - await client.ensureService('tenant-one', 'web'); - await client.ensureService('tenant-two', 'web'); - - await client.deleteApp('tenant-one'); - - expect(await client.listServices('tenant-one')).toEqual([]); - const twoList = await client.listServices('tenant-two'); - expect(twoList).toHaveLength(1); -}); - -test('an app deleted and reserved again gets back the ports it freed', async () => { - const client = computeClient({ registryRoot }); - const quotes = await client.ensureService('tenant-one', 'quotes'); - const gateway = await client.ensureService('tenant-one', 'gateway'); - - await client.deleteApp('tenant-one'); - - expect((await client.ensureService('tenant-one', 'quotes')).port).toBe(quotes.port); - expect((await client.ensureService('tenant-one', 'gateway')).port).toBe(gateway.port); -}); - -describe('path-segment hygiene', () => { - test('an app name with an uppercase letter is rejected with 400', async () => { - const { url } = await ensureDaemon('compute', entryFor('compute'), { registryRoot }); - const res = await fetch(`${url}/apps/BadApp/services/web`, { method: 'PUT' }); - expect(res.status).toBe(400); - expect(await res.text()).toContain('BadApp'); - }); - - test('a service id over 63 characters is rejected with 400', async () => { - const { url } = await ensureDaemon('compute', entryFor('compute'), { registryRoot }); - const longId = 'a'.repeat(64); - const res = await fetch(`${url}/apps/tenant-one/services/${longId}`, { method: 'PUT' }); - expect(res.status).toBe(400); - }); -}); diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts index dc8ae9d67..e3038ee52 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts @@ -8,7 +8,7 @@ import { afterEach, beforeEach, describe, expect, test } from 'bun:test'; import * as net from 'node:net'; import { fileURLToPath } from 'node:url'; -import { type ComputeClient, computeClient } from '../client.ts'; +import { type ComputeClient, computeClient, type InstanceInfo } from '../client.ts'; import { ensureDaemon, isPidAlive, SERVICE_READY_TIMEOUT_ENV, stopDaemon } from '../daemon.ts'; import { CRASH_ONCE_THEN_SILENT_BOOTSTRAP, @@ -59,10 +59,14 @@ async function startDaemon(entry?: string): Promise { return client; } -async function statusOf(client: ComputeClient, app: string, id: string): Promise { - const svc = (await client.listServices(app)).find((s) => s.id === id); - if (!svc) throw new Error(`service "${id}" not found in app "${app}"`); - return svc.status; +async function instanceOf(client: ComputeClient, id: string): Promise { + const instance = await client.getInstance(id); + if (!instance) throw new Error(`instance "${id}" not found`); + return instance; +} + +async function statusOf(client: ComputeClient, id: string): Promise { + return (await instanceOf(client, id)).status; } describe('with the default readiness timeout', () => { @@ -74,10 +78,9 @@ describe('with the default readiness timeout', () => { test('a deployment returns only once a slow-booting service accepts connections', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-slow', 'web'); + const reserved = await client.createInstance({ labels: {} }); - const deployed = client.putDeployment('app-slow', 'web', { - address: 'slow.web', + const deployed = client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv({ @@ -85,24 +88,22 @@ describe('with the default readiness timeout', () => { LISTEN_DELAY_MS: String(LISTEN_DELAY_MS), ...servingBootstrapEnv('slow but ready'), }), - port: reserved.port, }); await Bun.sleep(LISTEN_DELAY_MS / 2); - expect(await statusOf(client, 'app-slow', 'web')).not.toBe('running'); + expect(await statusOf(client, reserved.id)).not.toBe('running'); await deployed; expect(await acceptsConnection(reserved.port)).toBe(true); - expect(await statusOf(client, 'app-slow', 'web')).toBe('running'); + expect(await statusOf(client, reserved.id)).toBe('running'); const res = await fetch(`http://localhost:${String(reserved.port)}`); expect(await res.text()).toBe('slow but ready'); }, 15_000); test('a session-resume start returns only once a slow-booting service accepts connections', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-resume', 'web'); - await client.putDeployment('app-resume', 'web', { - address: 'resume.web', + const reserved = await client.createInstance({ labels: {} }); + await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv({ @@ -110,50 +111,47 @@ describe('with the default readiness timeout', () => { LISTEN_DELAY_MS: String(LISTEN_DELAY_MS), ...servingBootstrapEnv('resumed'), }), - port: reserved.port, }); - await client.stopApp('app-resume'); + await client.stopInstances([reserved.id]); expect(await acceptsConnection(reserved.port)).toBe(false); - await client.startApp('app-resume'); + await client.startInstances([reserved.id]); expect(await acceptsConnection(reserved.port)).toBe(true); - expect(await statusOf(client, 'app-resume', 'web')).toBe('running'); + expect(await statusOf(client, reserved.id)).toBe('running'); }, 20_000); test('a service that exits before it listens fails the deployment, naming the service and its log', async () => { const artifactDir = writeBootstrap(CRASHING_BOOTSTRAP); - const reserved = await client.ensureService('app-crash', 'web'); + const reserved = await client.createInstance({ labels: {} }); - const deployed = client.putDeployment('app-crash', 'web', { - address: 'crash.web', + const deployed = client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }); await expect(deployed).rejects.toThrow( - /service "crash\.web" exited \(code 1\) before it accepted connections on its port \d+\. Its log: .*web\.log/, + new RegExp( + `instance "${reserved.id}" exited \\(code 1\\) before it accepted connections on its port ${String(reserved.port)}\\. Its log: .*${reserved.id}\\.log`, + ), ); }, 15_000); test('a session-resume start of a service that exits before it listens fails, naming the service', async () => { const artifactDir = writeBootstrap(CRASHING_BOOTSTRAP); - const reserved = await client.ensureService('app-crash-resume', 'web'); + const reserved = await client.createInstance({ labels: {} }); await client - .putDeployment('app-crash-resume', 'web', { - address: 'crash-resume.web', + .putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }) .catch(() => undefined); - await client.stopApp('app-crash-resume'); + await client.stopInstances([reserved.id]); - await expect(client.startApp('app-crash-resume')).rejects.toThrow( - /service "crash-resume\.web" exited \(code 1\) before it accepted connections/, + await expect(client.startInstances([reserved.id])).rejects.toThrow( + `instance "${reserved.id}" exited (code 1) before it accepted connections`, ); }, 15_000); }); @@ -167,7 +165,6 @@ describe('a start that is replaced, stopped or joined', () => { function delayedDeployment(port: number, hash: string, body: string, artifactDir: string) { return { - address: 'slow.web', artifactDir, artifactHash: hash, env: baseEnv({ @@ -175,52 +172,46 @@ describe('a start that is replaced, stopped or joined', () => { LISTEN_DELAY_MS: String(LISTEN_DELAY_MS), ...servingBootstrapEnv(body), }), - port, }; } test('a redeploy of a running service is not running until the new process listens', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-redeploy', 'web'); + const reserved = await client.createInstance({ labels: {} }); await client.putDeployment( - 'app-redeploy', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v1', 'first', artifactDir), ); const redeployed = client.putDeployment( - 'app-redeploy', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v2', 'second', artifactDir), ); await Bun.sleep(LISTEN_DELAY_MS / 2); - expect(await statusOf(client, 'app-redeploy', 'web')).not.toBe('running'); + expect(await statusOf(client, reserved.id)).not.toBe('running'); await redeployed; - expect(await statusOf(client, 'app-redeploy', 'web')).toBe('running'); + expect(await statusOf(client, reserved.id)).toBe('running'); const res = await fetch(`http://localhost:${String(reserved.port)}`); expect(await res.text()).toBe('second'); }, 20_000); test('a repeated identical deployment during a redeploy waits for the new process', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-repeat', 'web'); + const reserved = await client.createInstance({ labels: {} }); await client.putDeployment( - 'app-repeat', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v1', 'first', artifactDir), ); const redeployed = client.putDeployment( - 'app-repeat', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v2', 'second', artifactDir), ); await Bun.sleep(300); await client.putDeployment( - 'app-repeat', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v2', 'second', artifactDir), ); @@ -232,20 +223,18 @@ describe('a start that is replaced, stopped or joined', () => { test('a session-resume start during a redeploy waits for the new process', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-resume-redeploy', 'web'); + const reserved = await client.createInstance({ labels: {} }); await client.putDeployment( - 'app-resume-redeploy', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v1', 'first', artifactDir), ); const redeployed = client.putDeployment( - 'app-resume-redeploy', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v2', 'second', artifactDir), ); await Bun.sleep(300); - await client.startApp('app-resume-redeploy'); + await client.startInstances([reserved.id]); const res = await fetch(`http://localhost:${String(reserved.port)}`); expect(await res.text()).toBe('second'); @@ -254,36 +243,33 @@ describe('a start that is replaced, stopped or joined', () => { test('two concurrent session-resume starts share one process', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-twice', 'web'); + const reserved = await client.createInstance({ labels: {} }); await client.putDeployment( - 'app-twice', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v1', 'once', artifactDir), ); - await client.stopApp('app-twice'); + await client.stopInstances([reserved.id]); - await Promise.all([client.startApp('app-twice'), client.startApp('app-twice')]); + await Promise.all([client.startInstances([reserved.id]), client.startInstances([reserved.id])]); - const log = await (await fetch(`${client.baseUrl}/apps/app-twice/services/web/logs`)).text(); + const log = await (await fetch(`${client.baseUrl}/instances/${reserved.id}/logs`)).text(); expect(log.match(/booted pid/g)?.length).toBe(1); - const [svc] = await client.listServices('app-twice'); + const svc = await instanceOf(client, reserved.id); expect(svc?.status).toBe('running'); expect(log).toContain(`booted pid ${String(svc?.pid)}`); }, 20_000); test('a deployment replaced during its start waits for the replacement instead of failing', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-replaced', 'web'); + const reserved = await client.createInstance({ labels: {} }); const first = client.putDeployment( - 'app-replaced', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v1', 'first', artifactDir), ); await Bun.sleep(300); const second = client.putDeployment( - 'app-replaced', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v2', 'second', artifactDir), ); @@ -296,33 +282,27 @@ describe('a start that is replaced, stopped or joined', () => { const artifactDir = writeBootstrap( `process.on('SIGTERM', () => {});\n${DELAYED_SERVING_BOOTSTRAP}`, ); - const reserved = await client.ensureService('app-stop-start', 'web'); + const reserved = await client.createInstance({ labels: {} }); const deployed = client - .putDeployment( - 'app-stop-start', - 'web', - delayedDeployment(reserved.port, 'v1', 'first', artifactDir), - ) + .putDeployment(reserved.id, delayedDeployment(reserved.port, 'v1', 'first', artifactDir)) .catch(() => undefined); await Bun.sleep(300); - const [first] = await client.listServices('app-stop-start'); + const first = await instanceOf(client, reserved.id); const firstPid = first?.pid; expect(firstPid).toBeDefined(); - const stopped = client.stopApp('app-stop-start'); + const stopped = client.stopInstances([reserved.id]); await Bun.sleep(200); - await client.startApp('app-stop-start'); + await client.startInstances([reserved.id]); await stopped; await deployed; - const [svc] = await client.listServices('app-stop-start'); + const svc = await instanceOf(client, reserved.id); expect(svc?.status).toBe('running'); expect(svc?.pid).not.toBe(firstPid); expect(isPidAlive(firstPid ?? 0)).toBe(false); - const log = await ( - await fetch(`${client.baseUrl}/apps/app-stop-start/services/web/logs`) - ).text(); + const log = await (await fetch(`${client.baseUrl}/instances/${reserved.id}/logs`)).text(); expect(log).not.toContain('[emulator] exited'); expect(log.match(/booted pid/g)?.length).toBe(1); @@ -334,55 +314,53 @@ describe('a start that is replaced, stopped or joined', () => { function quickDeployment(port: number, artifactDir: string) { return { - address: 'quick.web', artifactDir, artifactHash: 'v1', env: baseEnv({ PORT: String(port), LISTEN_DELAY_MS: '300', ...servingBootstrapEnv('x') }), - port, }; } test('stop, start, stop: a stop during a waiting start cancels it', async () => { const artifactDir = writeBootstrap(IGNORES_SIGTERM); - const reserved = await client.ensureService('app-sss', 'web'); - await client.putDeployment('app-sss', 'web', quickDeployment(reserved.port, artifactDir)); + const reserved = await client.createInstance({ labels: {} }); + await client.putDeployment(reserved.id, quickDeployment(reserved.port, artifactDir)); - const firstStop = client.stopApp('app-sss'); + const firstStop = client.stopInstances([reserved.id]); await Bun.sleep(200); - const start = client.startApp('app-sss').catch((e: unknown) => String(e)); + const start = client.startInstances([reserved.id]).catch((e: unknown) => String(e)); await Bun.sleep(200); - const secondStop = client.stopApp('app-sss'); + const secondStop = client.stopInstances([reserved.id]); await Promise.all([firstStop, start, secondStop]); - expect(await statusOf(client, 'app-sss', 'web')).toBe('stopped'); + expect(await statusOf(client, reserved.id)).toBe('stopped'); expect(await start).toContain('was stopped before it accepted connections'); expect(await acceptsConnection(reserved.port)).toBe(false); }, 30_000); test('an identical deployment during a stop waits for the stop and starts the service again', async () => { const artifactDir = writeBootstrap(IGNORES_SIGTERM); - const reserved = await client.ensureService('app-same-put', 'web'); - await client.putDeployment('app-same-put', 'web', quickDeployment(reserved.port, artifactDir)); + const reserved = await client.createInstance({ labels: {} }); + await client.putDeployment(reserved.id, quickDeployment(reserved.port, artifactDir)); - const stopped = client.stopApp('app-same-put'); + const stopped = client.stopInstances([reserved.id]); await Bun.sleep(200); - await client.putDeployment('app-same-put', 'web', quickDeployment(reserved.port, artifactDir)); + await client.putDeployment(reserved.id, quickDeployment(reserved.port, artifactDir)); await stopped; - expect(await statusOf(client, 'app-same-put', 'web')).toBe('running'); + expect(await statusOf(client, reserved.id)).toBe('running'); expect(await acceptsConnection(reserved.port)).toBe(true); - const [svc] = await client.listServices('app-same-put'); + const svc = await instanceOf(client, reserved.id); if (svc?.pid !== undefined) process.kill(svc.pid, 'SIGKILL'); }, 30_000); test('a start during a delete of the app leaves no process behind', async () => { const artifactDir = writeBootstrap(IGNORES_SIGTERM); - const reserved = await client.ensureService('app-del-start', 'web'); - await client.putDeployment('app-del-start', 'web', quickDeployment(reserved.port, artifactDir)); + const reserved = await client.createInstance({ labels: {} }); + await client.putDeployment(reserved.id, quickDeployment(reserved.port, artifactDir)); - const deleted = client.deleteApp('app-del-start'); + const deleted = client.deleteInstance(reserved.id); await Bun.sleep(200); - const start = await client.startApp('app-del-start').then( + const start = await client.startInstances([reserved.id]).then( () => 'ok', (e: unknown) => String(e), ); @@ -401,25 +379,24 @@ describe('a start that is replaced, stopped or joined', () => { for (const pid of leakedPids.split('\n').filter(Boolean)) process.kill(Number(pid), 'SIGKILL'); expect(await leaked).toBe(false); expect(start).toContain('was stopped before it accepted connections'); - expect(await client.listServices('app-del-start')).toEqual([]); + expect(await client.getInstance(reserved.id)).toBeUndefined(); }, 30_000); test('a deployment stopped during its start fails saying it was stopped', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await client.ensureService('app-stopped', 'web'); + const reserved = await client.createInstance({ labels: {} }); const deployed = client.putDeployment( - 'app-stopped', - 'web', + reserved.id, delayedDeployment(reserved.port, 'v1', 'first', artifactDir), ); await Bun.sleep(300); - await client.stopApp('app-stopped'); + await client.stopInstances([reserved.id]); await expect(deployed).rejects.toThrow( - 'service "slow.web" was stopped before it accepted connections', + `instance "${reserved.id}" was stopped before it accepted connections`, ); - expect(await statusOf(client, 'app-stopped', 'web')).toBe('stopped'); + expect(await statusOf(client, reserved.id)).toBe('stopped'); }, 20_000); }); @@ -429,23 +406,23 @@ describe('with a 1-second readiness timeout', () => { fileURLToPath(new URL('./fixtures/short-ready-timeout.ts', import.meta.url)), ); const artifactDir = writeBootstrap(NEVER_LISTENING_BOOTSTRAP); - const reserved = await client.ensureService('app-silent', 'web'); + const reserved = await client.createInstance({ labels: {} }); - const deployed = client.putDeployment('app-silent', 'web', { - address: 'silent.web', + const deployed = client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }); await expect(deployed).rejects.toThrow( - /service "silent\.web" did not accept connections on its port \d+ within 1s\. Its log: .*web\.log/, + new RegExp( + `instance "${reserved.id}" did not accept connections on its port ${String(reserved.port)} within 1s\\. Its log: .*${reserved.id}\\.log`, + ), ); - const [svc] = await client.listServices('app-silent'); + const svc = await instanceOf(client, reserved.id); expect(svc?.status).toBe('stopped'); expect(svc?.pid).toBeUndefined(); - const log = await (await fetch(`${client.baseUrl}/apps/app-silent/services/web/logs`)).text(); + const log = await (await fetch(`${client.baseUrl}/instances/${reserved.id}/logs`)).text(); expect(log).toContain('[emulator] did not accept connections on port'); }, 15_000); @@ -454,28 +431,20 @@ describe('with a 1-second readiness timeout', () => { fileURLToPath(new URL('./fixtures/short-ready-timeout.ts', import.meta.url)), ); const artifactDir = writeBootstrap(CRASH_ONCE_THEN_SILENT_BOOTSTRAP); - const reserved = await client.ensureService('app-restart-silent', 'web'); + const reserved = await client.createInstance({ labels: {} }); await expect( - client.putDeployment('app-restart-silent', 'web', { - address: 'restart-silent.web', + client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }), ).rejects.toThrow('exited (code 1)'); - await waitFor( - async () => (await statusOf(client, 'app-restart-silent', 'web')) === 'held', - 8000, - 100, - ); - const [svc] = await client.listServices('app-restart-silent'); + await waitFor(async () => (await statusOf(client, reserved.id)) === 'held', 8000, 100); + const svc = await instanceOf(client, reserved.id); expect(svc?.pid).toBeUndefined(); - const log = await ( - await fetch(`${client.baseUrl}/apps/app-restart-silent/services/web/logs`) - ).text(); + const log = await (await fetch(`${client.baseUrl}/instances/${reserved.id}/logs`)).text(); expect(log).toContain('did not accept connections on port'); expect(log).toContain('held'); }, 20_000); @@ -493,24 +462,20 @@ describe(`with ${SERVICE_READY_TIMEOUT_ENV}`, () => { process.env[SERVICE_READY_TIMEOUT_ENV] = '1000'; const client = await startDaemon(); const artifactDir = writeBootstrap(NEVER_LISTENING_BOOTSTRAP); - const reserved = await client.ensureService('app-env-timeout', 'web'); + const reserved = await client.createInstance({ labels: {} }); - const deployed = client.putDeployment('app-env-timeout', 'web', { - address: 'env-timeout.web', + const deployed = client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', env: baseEnv(), - port: reserved.port, }); await expect(deployed).rejects.toThrow('within 1s. Its log: '); await expect( - client.putDeployment('app-env-timeout', 'web', { - address: 'env-timeout.web', + client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h2', env: baseEnv(), - port: reserved.port, }), ).rejects.toThrow(SERVICE_READY_TIMEOUT_ENV); }, 15_000); diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/helpers.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/helpers.ts index 9c7b52a1c..950224cef 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/helpers.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/helpers.ts @@ -154,20 +154,19 @@ export async function skipContendedDaemonPorts(registryRoot: string): Promise, + client: Pick, minPort: number = DEFAULT_MIN_SERVICE_PORT, ): Promise { const freePort = await getPort({ port: portNumbers(minPort, minPort + 200) }); for (let i = 0; i < freePort - minPort; i++) { - await client.ensureService('port-skip', `dummy-${String(i)}`); + await client.createInstance({ labels: { purpose: 'port-skip' } }); } } diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/client.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/client.ts index 891bc9a97..5b5ec17e1 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/client.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/client.ts @@ -111,50 +111,65 @@ async function expectOk(res: Response): Promise { return res; } -export interface ServiceInfo { +export type InstanceStatus = 'running' | 'backoff' | 'held' | 'stopped'; + +export interface InstanceInfo { readonly id: string; - readonly address: string; + /** Stored and returned verbatim; the emulator never reads them. */ + readonly labels: Readonly>; readonly port: number; readonly url: string; - readonly status: 'running' | 'backoff' | 'held' | 'stopped'; + readonly status: InstanceStatus; readonly pid?: number; readonly lastExitCode?: number; readonly artifactHash?: string; readonly logPath: string; } +export interface CreateInstanceRequest { + readonly labels: Readonly>; + /** Used if no instance holds it and it is bindable; otherwise the emulator gives the lowest free port from 3000. */ + readonly port?: number; +} + export interface DeploymentRequest { - readonly address: string; readonly artifactDir: string; readonly artifactHash: string; readonly env: Readonly>; - readonly port: number; } -const SERVICE_STATUSES = new Set(['running', 'backoff', 'held', 'stopped']); +const INSTANCE_STATUSES = new Set(['running', 'backoff', 'held', 'stopped']); -function isServiceInfo(value: unknown): value is ServiceInfo { +function isStringRecord(value: unknown): value is Record { + return ( + typeof value === 'object' && + value !== null && + Object.values(value).every((v) => typeof v === 'string') + ); +} + +function isInstanceInfo(value: unknown): value is InstanceInfo { return ( typeof value === 'object' && value !== null && 'id' in value && typeof value.id === 'string' && - 'address' in value && - typeof value.address === 'string' && + 'labels' in value && + isStringRecord(value.labels) && 'port' in value && typeof value.port === 'number' && 'url' in value && typeof value.url === 'string' && 'status' in value && typeof value.status === 'string' && - SERVICE_STATUSES.has(value.status) && + INSTANCE_STATUSES.has(value.status) && 'logPath' in value && typeof value.logPath === 'string' ); } -function isServiceInfoArray(value: unknown): value is ServiceInfo[] { - return Array.isArray(value) && value.every(isServiceInfo); +function isInstanceInfoArray(value: unknown): value is InstanceInfo[] { + return Array.isArray(value) && value.every(isInstanceInfo); } interface HealthBody { @@ -170,44 +185,42 @@ function isHealthBody(value: unknown): value is HealthBody { ); } -interface ServiceReservation { - readonly port: number; - readonly url: string; -} - -function isServiceReservation(value: unknown): value is ServiceReservation { - return ( - typeof value === 'object' && - value !== null && - 'port' in value && - typeof value.port === 'number' && - 'url' in value && - typeof value.url === 'string' - ); -} - +/** Each compute instance is keyed by an id the emulator generates when it creates the instance. */ export interface ComputeClient { readonly baseUrl: string; health(): Promise<{ version: string }>; - /** `PUT /apps//services/` — reserves (or returns) the service's stable port. Idempotent. */ - ensureService(app: string, id: string): Promise<{ port: number; url: string }>; - /** `PUT /apps//services//deployment`. */ - putDeployment(app: string, id: string, deployment: DeploymentRequest): Promise; - /** `GET /apps//services`. */ - listServices(app: string): Promise; - /** `GET /apps//services//logs?follow=1` — yields decoded text chunks; ends when `signal` aborts or the daemon closes the stream. */ + /** `POST /instances` — creates a stopped instance holding a port until it is deleted. */ + createInstance(request: CreateInstanceRequest): Promise; + /** `GET /instances`. */ + listInstances(): Promise; + /** `GET /instances/`; undefined when the emulator has no such instance. */ + getInstance(id: string): Promise; + /** `PUT /instances//deployment` — returns once the instance accepts connections on its port. */ + putDeployment(id: string, deployment: DeploymentRequest): Promise; + /** `GET /instances//logs?follow=1` — yields decoded text chunks; ends when `signal` aborts or the daemon closes the stream. */ followLogs( - app: string, id: string, signal?: AbortSignal, opts?: { readonly tail?: number }, ): AsyncIterable; - /** `POST /apps//stop`. */ - stopApp(app: string): Promise; - /** `POST /apps//start` — the session-resume signal; starts every service with a stored deployment that isn't already running. */ - startApp(app: string): Promise; - /** `DELETE /apps/`. */ - deleteApp(app: string): Promise; + /** `POST /instances/start` — the session-resume signal; starts each listed instance with a stored deployment that isn't already running. */ + startInstances(ids: readonly string[]): Promise; + /** `POST /instances/stop`. The instances keep their ports. */ + stopInstances(ids: readonly string[]): Promise; + /** `DELETE /instances/` — stops the instance and frees its port. Deleting an unknown id succeeds. */ + deleteInstance(id: string): Promise; +} + +function instancePath(baseUrl: string, id: string): string { + return `${baseUrl}/instances/${encodeURIComponent(id)}`; +} + +function postJson(body: unknown): RequestInit { + return { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify(body), + }; } export function computeClient(opts: DaemonRootOptions = {}): ComputeClient { @@ -225,20 +238,37 @@ export function computeClient(opts: DaemonRootOptions = {}): ComputeClient { return body; }, - async ensureService(app, id) { - const url = `${baseUrl}/apps/${encodeSegment(app)}/services/${encodeSegment(id)}`; - const res = await expectOk(await adminFetch(url, { method: 'PUT' })); + async createInstance(request) { + const res = await expectOk(await adminFetch(`${baseUrl}/instances`, postJson(request))); + const body: unknown = await res.json(); + if (!isInstanceInfo(body)) { + throw new Error('malformed instance-create response from the compute emulator'); + } + return body; + }, + + async listInstances() { + const res = await expectOk(await adminFetch(`${baseUrl}/instances`)); const body: unknown = await res.json(); - if (!isServiceReservation(body)) { - throw new Error('malformed service-reservation response from the compute emulator'); + if (!isInstanceInfoArray(body)) { + throw new Error('malformed instances listing from the compute emulator'); + } + return body; + }, + + async getInstance(id) { + const res = await adminFetch(instancePath(baseUrl, id)); + if (res.status === 404) return undefined; + const body: unknown = await (await expectOk(res)).json(); + if (!isInstanceInfo(body)) { + throw new Error('malformed instance response from the compute emulator'); } return body; }, - async putDeployment(app, id, deployment) { - const url = `${baseUrl}/apps/${encodeSegment(app)}/services/${encodeSegment(id)}/deployment`; + async putDeployment(id, deployment) { await expectOk( - await adminFetch(url, { + await adminFetch(`${instancePath(baseUrl, id)}/deployment`, { method: 'PUT', headers: { 'content-type': 'application/json' }, body: JSON.stringify(deployment), @@ -246,20 +276,10 @@ export function computeClient(opts: DaemonRootOptions = {}): ComputeClient { ); }, - async listServices(app) { - const url = `${baseUrl}/apps/${encodeSegment(app)}/services`; - const res = await expectOk(await adminFetch(url)); - const body: unknown = await res.json(); - if (!isServiceInfoArray(body)) { - throw new Error('malformed services listing from the compute emulator'); - } - return body; - }, - - async *followLogs(app, id, signal, opts) { + async *followLogs(id, signal, opts) { const tail = opts?.tail !== undefined ? `&tail=${String(Math.max(0, Math.trunc(opts.tail)))}` : ''; - const url = `${baseUrl}/apps/${encodeSegment(app)}/services/${encodeSegment(id)}/logs?follow=1${tail}`; + const url = `${instancePath(baseUrl, id)}/logs?follow=1${tail}`; const res = await expectOk(await adminFetch(url, signal ? { signal } : undefined)); const body = res.body; if (!body) return; @@ -276,19 +296,16 @@ export function computeClient(opts: DaemonRootOptions = {}): ComputeClient { } }, - async stopApp(app) { - const url = `${baseUrl}/apps/${encodeSegment(app)}/stop`; - await expectOk(await adminFetch(url, { method: 'POST' })); + async startInstances(ids) { + await expectOk(await adminFetch(`${baseUrl}/instances/start`, postJson({ ids }))); }, - async startApp(app) { - const url = `${baseUrl}/apps/${encodeSegment(app)}/start`; - await expectOk(await adminFetch(url, { method: 'POST' })); + async stopInstances(ids) { + await expectOk(await adminFetch(`${baseUrl}/instances/stop`, postJson({ ids }))); }, - async deleteApp(app) { - const url = `${baseUrl}/apps/${encodeSegment(app)}`; - await expectOk(await adminFetch(url, { method: 'DELETE' })); + async deleteInstance(id) { + await expectOk(await adminFetch(instancePath(baseUrl, id), { method: 'DELETE' })); }, }; } diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/compute-main.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/compute-main.ts index aa2e9f0e1..566e2056e 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/compute-main.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/compute-main.ts @@ -1,9 +1,12 @@ /** * The Compute emulator daemon (local-dev spec § 2 `compute-main.ts`): a small - * local counterpart of the platform's compute service. Owns service child - * processes (spawned `bun bootstrap.js` from a real packaged artifact), - * supervises them (crash backoff, held after repeated fast crashes), and - * serves a loopback JSON admin API plus per-service log streaming. + * local counterpart of the platform's compute service. Holds service + * instances under ids it generates, owns their child processes (spawned + * `bun bootstrap.js` from a real packaged artifact), supervises them (crash + * backoff, held after repeated fast crashes), and serves a loopback JSON + * admin API plus per-instance log streaming. An instance's labels are its + * creator's to read: they are stored and returned verbatim, never + * interpreted. * * Runs as its own OS process, started by `daemon.ts`'s `ensureDaemon` via * `process.execPath --port --state-dir `. Everything it @@ -11,13 +14,13 @@ * environment variable of its own. */ import { type ChildProcess, spawn } from 'node:child_process'; +import { randomBytes } from 'node:crypto'; import * as fs from 'node:fs'; import * as http from 'node:http'; import * as net from 'node:net'; import * as path from 'node:path'; import getPort, { clearLockedPorts, portNumbers } from 'get-port'; import { readOwnVersion, SERVICE_READY_TIMEOUT_ENV } from './daemon.ts'; -import { isValidSegment } from './segments.ts'; import { readJsonFile, StateFile } from './state-file.ts'; const MIN_SERVICE_PORT = 3000; @@ -31,18 +34,19 @@ const LOG_POLL_INTERVAL_MS = 100; const DEFAULT_READY_TIMEOUT_MS = 30_000; const READY_POLL_INTERVAL_MS = 50; const READY_PROBE_TIMEOUT_MS = 1000; -const APPS_STATE_MODE = 0o600; +const STATE_MODE = 0o600; const BUN_NOT_FOUND_MESSAGE = 'local dev runs services under bun — the Prisma Compute runtime — and `bun` was not found on PATH. Install it: https://bun.sh.'; -type ServiceStatus = 'running' | 'backoff' | 'held' | 'stopped'; +type InstanceStatus = 'running' | 'backoff' | 'held' | 'stopped'; -interface ServiceRecord { - id: string; - address: string; - port: number; - status: ServiceStatus; +interface InstanceRecord { + readonly id: string; + readonly labels: Record; + /** Held from creation until the instance is deleted, running or stopped: a deployment's env carries it. */ + readonly port: number; + status: InstanceStatus; pid?: number; lastExitCode?: number; artifactHash?: string; @@ -51,13 +55,17 @@ interface ServiceRecord { readonly logPath: string; } -interface AppRecord { - services: Record; +interface InstancesState { + instances: Record; } -type AppsState = Record; +const INSTANCE_ID_RE = /^svc_[0-9a-f]{16}$/; + +function newInstanceId(): string { + return `svc_${randomBytes(8).toString('hex')}`; +} -function isServiceStatus(value: unknown): value is ServiceStatus { +function isInstanceStatus(value: unknown): value is InstanceStatus { return value === 'running' || value === 'backoff' || value === 'held' || value === 'stopped'; } @@ -65,84 +73,97 @@ function isStringRecord(value: unknown): value is Record { return ( typeof value === 'object' && value !== null && + !Array.isArray(value) && Object.values(value).every((v) => typeof v === 'string') ); } -function isServiceRecord(value: unknown): value is ServiceRecord { +function isPort(value: unknown): value is number { + return ( + typeof value === 'number' && Number.isInteger(value) && value >= 1 && value <= MAX_SERVICE_PORT + ); +} + +function isOptional(value: object, key: string, check: (field: unknown) => boolean): boolean { + return !(key in value) || Reflect.get(value, key) === undefined || check(Reflect.get(value, key)); +} + +function isInstanceRecord(value: unknown): value is InstanceRecord { if (typeof value !== 'object' || value === null) return false; if (!('id' in value) || typeof value.id !== 'string') return false; - if (!('address' in value) || typeof value.address !== 'string') return false; - if (!('port' in value) || typeof value.port !== 'number') return false; - if (!('status' in value) || !isServiceStatus(value.status)) return false; + if (!('labels' in value) || !isStringRecord(value.labels)) return false; + if (!('port' in value) || !isPort(value.port)) return false; + if (!('status' in value) || !isInstanceStatus(value.status)) return false; if (!('logPath' in value) || typeof value.logPath !== 'string') return false; - if ('pid' in value && value.pid !== undefined && typeof value.pid !== 'number') return false; - if ( - 'lastExitCode' in value && - value.lastExitCode !== undefined && - typeof value.lastExitCode !== 'number' - ) { - return false; - } - if ( - 'artifactHash' in value && - value.artifactHash !== undefined && - typeof value.artifactHash !== 'string' - ) { - return false; - } - if ( - 'artifactDir' in value && - value.artifactDir !== undefined && - typeof value.artifactDir !== 'string' - ) { - return false; - } - if ('env' in value && value.env !== undefined && !isStringRecord(value.env)) return false; - return true; + return ( + isOptional(value, 'pid', (v) => typeof v === 'number') && + isOptional(value, 'lastExitCode', (v) => typeof v === 'number') && + isOptional(value, 'artifactHash', (v) => typeof v === 'string') && + isOptional(value, 'artifactDir', (v) => typeof v === 'string') && + isOptional(value, 'env', isStringRecord) + ); } -function isAppRecord(value: unknown): value is AppRecord { +function isInstancesState(value: unknown): value is InstancesState { return ( typeof value === 'object' && value !== null && - 'services' in value && - typeof value.services === 'object' && - value.services !== null && - Object.values(value.services).every(isServiceRecord) + 'instances' in value && + typeof value.instances === 'object' && + value.instances !== null && + Object.values(value.instances).every(isInstanceRecord) ); } -function isAppsState(value: unknown): value is AppsState { - return typeof value === 'object' && value !== null && Object.values(value).every(isAppRecord); +interface CreateBody { + readonly labels: Record; + readonly port?: number; +} + +function isCreateBody(value: unknown): value is CreateBody { + return ( + typeof value === 'object' && + value !== null && + 'labels' in value && + isStringRecord(value.labels) && + isOptional(value, 'port', isPort) + ); } interface DeploymentBody { - readonly address: string; readonly artifactDir: string; readonly artifactHash: string; readonly env: Record; - readonly port: number; } function isDeploymentBody(value: unknown): value is DeploymentBody { return ( typeof value === 'object' && value !== null && - 'address' in value && - typeof value.address === 'string' && 'artifactDir' in value && typeof value.artifactDir === 'string' && 'artifactHash' in value && typeof value.artifactHash === 'string' && 'env' in value && - isStringRecord(value.env) && - 'port' in value && - typeof value.port === 'number' + isStringRecord(value.env) + ); +} + +interface IdsBody { + readonly ids: readonly string[]; +} + +function isIdsBody(value: unknown): value is IdsBody { + return ( + typeof value === 'object' && + value !== null && + 'ids' in value && + Array.isArray(value.ids) && + value.ids.every((id) => typeof id === 'string') ); } -/** Live, non-persisted bookkeeping for one service — the child handle and its supervision counters. */ +/** Live, non-persisted bookkeeping for one instance — the child handle and its supervision counters. */ interface RuntimeInfo { child: ChildProcess | undefined; consecutiveFastExits: number; @@ -150,12 +171,12 @@ interface RuntimeInfo { stableTimer: NodeJS.Timeout | undefined; backoffTimer: NodeJS.Timeout | undefined; expectedExit: boolean; - log: ServiceLog | undefined; + log: InstanceLog | undefined; /** The start in flight, until its child accepts connections or fails to. A newer start or a stop replaces or clears it. */ start: InFlightStart | undefined; /** A stop still waiting for its child to exit. A start waits for it, so it never spawns beside a dying child. */ stopping: Promise | undefined; - /** Set when the app is being deleted: no start may spawn on this runtime again. */ + /** Set when the instance is being deleted: no start may spawn on this runtime again. */ deleted: boolean; } @@ -179,8 +200,8 @@ function sleep(ms: number): Promise { return new Promise((resolve) => setTimeout(resolve, ms)); } -/** Append-only per-service log file — child stdout/stderr plus `[emulator]` supervision lines. No rotation in v1. */ -class ServiceLog { +/** Append-only per-instance log file — child stdout/stderr plus `[emulator]` supervision lines. No rotation in v1. */ +class InstanceLog { private fd: number | undefined; constructor(private readonly filePath: string) {} @@ -201,7 +222,7 @@ class ServiceLog { fs.writeSync(this.ensureOpen(), `${line}\n`); } - /** Empties the file — a fresh service start begins a clean log; past sessions' and past deployments' output is not kept. */ + /** Empties the file — a fresh start begins a clean log; past sessions' and past deployments' output is not kept. */ clear(): void { fs.ftruncateSync(this.ensureOpen(), 0); } @@ -301,24 +322,20 @@ function resolveBunOnPath(env: Readonly>): string | undef function main(): void { const { port, stateDir, readyTimeoutMs } = parseArgs(process.argv.slice(2)); const ownVersion = readOwnVersion(); - const appsJsonPath = path.join(stateDir, 'apps.json'); - const stateFile = new StateFile(appsJsonPath, APPS_STATE_MODE); + const statePath = path.join(stateDir, 'instances.json'); + const stateFile = new StateFile(statePath, STATE_MODE); + const logsDir = path.join(stateDir, 'logs'); - let state: AppsState = {}; + let state: InstancesState = { instances: {} }; const runtimes = new Map(); - // Bumped every time a log file is cleared (a fresh service start). A live + // Bumped every time a log file is cleared (a fresh start). A live // follower watches this rather than the file size, so it resets to the new // start reliably — even when the clear and the first new write both land // between two of its polls, which a size comparison alone would miss. const logGeneration = new Map(); - function runtimeKey(app: string, id: string): string { - return `${app}/${id}`; - } - - function getRuntime(app: string, id: string): RuntimeInfo { - const key = runtimeKey(app, id); - let rt = runtimes.get(key); + function getRuntime(id: string): RuntimeInfo { + let rt = runtimes.get(id); if (!rt) { rt = { child: undefined, @@ -332,18 +349,14 @@ function main(): void { stopping: undefined, deleted: false, }; - runtimes.set(key, rt); + runtimes.set(id, rt); } return rt; } - function serviceLogPath(app: string, id: string): string { - return path.join(stateDir, 'logs', app, `${id}.log`); - } - - function getLog(app: string, id: string, logPath: string): ServiceLog { - const rt = getRuntime(app, id); - if (!rt.log) rt.log = new ServiceLog(logPath); + function getLog(inst: InstanceRecord): InstanceLog { + const rt = getRuntime(inst.id); + if (!rt.log) rt.log = new InstanceLog(inst.logPath); return rt.log; } @@ -361,45 +374,37 @@ function main(): void { rt.backoffTimer = undefined; } - function usedServicePorts(): Set { - const ports = new Set(); - for (const app of Object.values(state)) { - for (const svc of Object.values(app.services)) ports.add(svc.port); - } - return ports; + function heldPorts(): Set { + return new Set(Object.values(state.instances).map((inst) => inst.port)); } /** - * The smallest genuinely free port at or above `MIN_SERVICE_PORT`, - * excluding this daemon's own persisted allocations — persistence and - * the range policy stay ours; whether a candidate is actually bindable - * is `get-port`'s (spec § 2's dependency razor). + * The requested port when no instance holds it and it is bindable, + * otherwise the smallest such port at or above `MIN_SERVICE_PORT`. Which + * ports instances hold is ours; whether a candidate is actually bindable is + * `get-port`'s (spec § 2's dependency razor). */ - async function smallestUnusedServicePort(): Promise { - return getPort({ - port: portNumbers(MIN_SERVICE_PORT, MAX_SERVICE_PORT), - exclude: usedServicePorts(), - }); + async function allocatePort(requested: number | undefined): Promise { + const exclude = heldPorts(); + if (requested !== undefined && !exclude.has(requested)) { + const given = await getPort({ port: requested, exclude }); + if (given === requested) return given; + } + return getPort({ port: portNumbers(MIN_SERVICE_PORT, MAX_SERVICE_PORT), exclude }); } - async function getOrCreateService(app: string, id: string): Promise { - let appRec = state[app]; - if (!appRec) { - appRec = { services: {} }; - state[app] = appRec; - } - let svc = appRec.services[id]; - if (svc) return svc; - svc = { + async function createInstance(body: CreateBody): Promise { + const id = newInstanceId(); + const inst: InstanceRecord = { id, - address: id, - port: await smallestUnusedServicePort(), + labels: { ...body.labels }, + port: await allocatePort(body.port), status: 'stopped', - logPath: serviceLogPath(app, id), + logPath: path.join(logsDir, `${id}.log`), }; - appRec.services[id] = svc; + state.instances[id] = inst; schedulePersist(); - return svc; + return inst; } function waitForExit(child: ChildProcess): Promise { @@ -427,9 +432,7 @@ function main(): void { } function handleChildExit( - app: string, - id: string, - svc: ServiceRecord, + inst: InstanceRecord, rt: RuntimeInfo, child: ChildProcess, code: number | null, @@ -437,8 +440,8 @@ function main(): void { if (child !== rt.child) return; clearStableTimer(rt); rt.child = undefined; - delete svc.pid; - if (code !== null) svc.lastExitCode = code; + delete inst.pid; + if (code !== null) inst.lastExitCode = code; if (rt.expectedExit) { rt.expectedExit = false; @@ -452,28 +455,28 @@ function main(): void { if (uptime < STABLE_UPTIME_MS) rt.consecutiveFastExits += 1; if (rt.consecutiveFastExits >= MAX_CONSECUTIVE_FAST_EXITS) { - svc.status = 'held'; + inst.status = 'held'; schedulePersist(); - getLog(app, id, svc.logPath).writeLine( + getLog(inst).writeLine( `[emulator] held after ${String(MAX_CONSECUTIVE_FAST_EXITS)} consecutive fast exits — send a new deployment to resume`, ); return; } const delayMs = Math.min(MAX_BACKOFF_MS, BASE_BACKOFF_MS * 2 ** attemptNumber); - svc.status = 'backoff'; + inst.status = 'backoff'; schedulePersist(); - getLog(app, id, svc.logPath).writeLine( + getLog(inst).writeLine( `[emulator] exited (code ${code === null ? 'null' : String(code)}) — restarting in ${String(delayMs / 1000)}s`, ); rt.backoffTimer = setTimeout(() => { rt.backoffTimer = undefined; // A crash-backoff respawn — keep the crash trail this run has written. - void spawnService(app, id, svc, rt, false); + void spawnInstance(inst, rt, false); }, delayMs); } - /** Records `run` as the service's start in flight, replacing any earlier one. */ + /** Records `run` as the instance's start in flight, replacing any earlier one. */ function beginStart( rt: RuntimeInfo, fresh: boolean, @@ -487,9 +490,9 @@ function main(): void { return result; } - /** Undefined while `token` is still the service's start. Otherwise what that start reports: a newer start's result, or "stopped" after a stop or an app delete. */ + /** Undefined while `token` is still the instance's start. Otherwise what that start reports: a newer start's result, or "stopped" after a stop or a delete. */ function supersededOutcome( - svc: ServiceRecord, + inst: InstanceRecord, rt: RuntimeInfo, token: object, ): Promise | undefined { @@ -497,51 +500,45 @@ function main(): void { if (!rt.deleted && rt.start) return rt.start.result; return Promise.resolve({ ok: false, - message: `service "${svc.address}" was stopped before it accepted connections`, + message: `instance "${inst.id}" was stopped before it accepted connections`, }); } /** - * Spawns the service's child and resolves once it accepts connections on + * Spawns the instance's child and resolves once it accepts connections on * its port — only then is it `running`. A child that exits first, or never * listens within the readiness timeout, fails the start with a message - * naming the service and its log. + * naming the instance and its log. */ - function spawnService( - app: string, - id: string, - svc: ServiceRecord, + function spawnInstance( + inst: InstanceRecord, rt: RuntimeInfo, /** A fresh start (a deployment or a session resume) begins a clean log; a crash-backoff respawn keeps the run's crash trail. */ fresh: boolean, ): Promise { - return beginStart(rt, fresh, (token) => - spawnAndWaitUntilListening(app, id, svc, rt, fresh, token), - ); + return beginStart(rt, fresh, (token) => spawnAndWaitUntilListening(inst, rt, fresh, token)); } async function spawnAndWaitUntilListening( - app: string, - id: string, - svc: ServiceRecord, + inst: InstanceRecord, rt: RuntimeInfo, fresh: boolean, token: object, ): Promise { - const env = svc.env ?? {}; - const artifactDir = svc.artifactDir ?? '.'; + const env = inst.env ?? {}; + const artifactDir = inst.artifactDir ?? '.'; const bunPath = resolveBunOnPath(env); if (!bunPath) { - svc.status = 'stopped'; - delete svc.pid; + inst.status = 'stopped'; + delete inst.pid; schedulePersist(); return { ok: false, message: BUN_NOT_FOUND_MESSAGE }; } - const log = getLog(app, id, svc.logPath); + const log = getLog(inst); if (fresh) { log.clear(); - logGeneration.set(svc.logPath, (logGeneration.get(svc.logPath) ?? 0) + 1); + logGeneration.set(inst.logPath, (logGeneration.get(inst.logPath) ?? 0) + 1); } const child = spawn(bunPath, ['bootstrap.js'], { cwd: artifactDir, @@ -558,46 +555,46 @@ function main(): void { child.stdout?.on('data', (chunk: Buffer) => log.writeRaw(chunk)); child.stderr?.on('data', (chunk: Buffer) => log.writeRaw(chunk)); - child.on('exit', (code) => handleChildExit(app, id, svc, rt, child, code)); - if (child.pid !== undefined) svc.pid = child.pid; + child.on('exit', (code) => handleChildExit(inst, rt, child, code)); + if (child.pid !== undefined) inst.pid = child.pid; schedulePersist(); - const port = String(svc.port); - const outcome = await waitUntilListening(child, svc.port, readyTimeoutMs); - const superseded = supersededOutcome(svc, rt, token); + const port = String(inst.port); + const outcome = await waitUntilListening(child, inst.port, readyTimeoutMs); + const superseded = supersededOutcome(inst, rt, token); if (superseded) return superseded; - const service = `service "${svc.address}"`; + const instance = `instance "${inst.id}"`; if (outcome.kind === 'exited') { const how = outcome.code !== null ? `code ${String(outcome.code)}` : (outcome.signal ?? 'no exit code'); return { ok: false, - message: `${service} exited (${how}) before it accepted connections on its port ${port}. Its log: ${svc.logPath}`, + message: `${instance} exited (${how}) before it accepted connections on its port ${port}. Its log: ${inst.logPath}`, }; } if (outcome.kind === 'timed-out') { const seconds = String(readyTimeoutMs / 1000); - const message = `${service} did not accept connections on its port ${port} within ${seconds}s. Its log: ${svc.logPath}. If it needs longer to start, set ${SERVICE_READY_TIMEOUT_ENV} (milliseconds) and restart the compute emulator.`; + const message = `${instance} did not accept connections on its port ${port} within ${seconds}s. Its log: ${inst.logPath}. If it needs longer to start, set ${SERVICE_READY_TIMEOUT_ENV} (milliseconds) and restart the compute emulator.`; if (fresh) { log.writeLine( `[emulator] did not accept connections on port ${port} within ${seconds}s — stopped`, ); - await stopService(svc, rt); + await stopInstance(inst, rt); } else { log.writeLine( `[emulator] restart did not accept connections on port ${port} within ${seconds}s — held; send a new deployment to resume`, ); - await stopService(svc, rt, 'held'); + await stopInstance(inst, rt, 'held'); } return { ok: false, message }; } - svc.status = 'running'; + inst.status = 'running'; schedulePersist(); return { ok: true }; } - async function stopService( - svc: ServiceRecord, + async function stopInstance( + inst: InstanceRecord, rt: RuntimeInfo, finalStatus: 'stopped' | 'held' = 'stopped', ): Promise { @@ -605,8 +602,8 @@ function main(): void { clearBackoffTimer(rt); clearStableTimer(rt); const stopping = killChild(rt, TERMINATE_GRACE_MS).then(() => { - svc.status = finalStatus; - delete svc.pid; + inst.status = finalStatus; + delete inst.pid; schedulePersist(); }); rt.stopping = stopping; @@ -623,29 +620,44 @@ function main(): void { /** * The session-resume signal: Alchemy's no-op reconcile never fires a - * deployment PUT, so nothing else restarts a service a previous session + * deployment PUT, so nothing else restarts an instance a previous session * stopped. Reuses the exact deployment-PUT start rules — same spawn * path, same supervision reset, and an explicit resume clears `held` - * the same way a deployment PUT does — but from the service's already - * STORED deployment spec, since no new one arrives with a start. + * the same way a deployment PUT does — but from the instance's already + * STORED deployment, since no new one arrives with a start. */ - async function startService(app: string, id: string, svc: ServiceRecord): Promise { - if (svc.artifactHash === undefined) return { ok: true }; // never deployed — nothing to start - const rt = getRuntime(app, id); + async function startInstance(inst: InstanceRecord): Promise { + if (inst.artifactHash === undefined) return { ok: true }; // never deployed — nothing to start + const rt = getRuntime(inst.id); if (rt.start) return rt.start.result; - if (svc.status === 'running' && rt.stopping === undefined) return { ok: true }; // already up + if (inst.status === 'running' && rt.stopping === undefined) return { ok: true }; // already up // A session resume is a fresh start — begin a clean log. return beginStart(rt, true, async (token) => { clearBackoffTimer(rt); await stopsFinished(rt); - const superseded = supersededOutcome(svc, rt, token); + const superseded = supersededOutcome(inst, rt, token); if (superseded) return superseded; clearStableTimer(rt); rt.consecutiveFastExits = 0; - return spawnAndWaitUntilListening(app, id, svc, rt, true, token); + return spawnAndWaitUntilListening(inst, rt, true, token); }); } + async function deleteInstance(inst: InstanceRecord): Promise { + const rt = getRuntime(inst.id); + rt.deleted = true; + await stopInstance(inst, rt); + rt.log?.close(); + runtimes.delete(inst.id); + delete state.instances[inst.id]; + schedulePersist(); + // get-port holds every port it returns for up to 30 seconds; the freed + // port must be free again for an instance created right after. + // `heldPorts()` already keeps other instances' ports out. + clearLockedPorts(); + await fs.promises.rm(inst.logPath, { force: true }); + } + // ——— HTTP plumbing ——— function json(res: http.ServerResponse, status: number, body: unknown): void { @@ -659,12 +671,9 @@ function main(): void { res.end(body); } - function badSegment(res: http.ServerResponse, segment: string): void { - text( - res, - 400, - `invalid path segment "${segment}": must match /^[a-z0-9][a-z0-9-]*$/ and be at most 63 characters`, - ); + function noContent(res: http.ServerResponse): void { + res.writeHead(204); + res.end(); } function readBody(req: http.IncomingMessage): Promise { @@ -676,17 +685,26 @@ function main(): void { }); } - function serviceView(svc: ServiceRecord): unknown { + async function readJsonBody(req: http.IncomingMessage): Promise { + const raw = await readBody(req); + try { + return JSON.parse(raw.toString('utf8')); + } catch { + return undefined; + } + } + + function instanceView(inst: InstanceRecord): unknown { return { - id: svc.id, - address: svc.address, - port: svc.port, - url: `http://localhost:${String(svc.port)}`, - status: svc.status, - ...(svc.pid !== undefined ? { pid: svc.pid } : {}), - ...(svc.lastExitCode !== undefined ? { lastExitCode: svc.lastExitCode } : {}), - ...(svc.artifactHash !== undefined ? { artifactHash: svc.artifactHash } : {}), - logPath: svc.logPath, + id: inst.id, + labels: inst.labels, + port: inst.port, + url: `http://localhost:${String(inst.port)}`, + status: inst.status, + ...(inst.pid !== undefined ? { pid: inst.pid } : {}), + ...(inst.lastExitCode !== undefined ? { lastExitCode: inst.lastExitCode } : {}), + ...(inst.artifactHash !== undefined ? { artifactHash: inst.artifactHash } : {}), + logPath: inst.logPath, }; } @@ -701,22 +719,13 @@ function main(): void { async function handleDeployment( req: http.IncomingMessage, res: http.ServerResponse, - app: string, - id: string, + inst: InstanceRecord, ): Promise { - const raw = await readBody(req); - let parsed: unknown; - try { - parsed = JSON.parse(raw.toString('utf8')); - } catch { - return text(res, 400, 'malformed JSON body'); - } - if (!isDeploymentBody(parsed)) return text(res, 400, 'malformed deployment body'); + const body = await readJsonBody(req); + if (!isDeploymentBody(body)) return text(res, 400, 'malformed deployment body'); - const svc = await getOrCreateService(app, id); - const rt = getRuntime(app, id); - - const changed = svc.artifactHash !== parsed.artifactHash || !envEquals(svc.env, parsed.env); + const rt = getRuntime(inst.id); + const changed = inst.artifactHash !== body.artifactHash || !envEquals(inst.env, body.env); let result: SpawnResult; if (!changed && rt.start?.fresh === true) { @@ -726,38 +735,35 @@ function main(): void { !changed && rt.start === undefined && rt.stopping === undefined && - svc.status === 'running' + inst.status === 'running' ) { - // A true no-op: nothing about this service's persisted record + // A true no-op: nothing about this instance's persisted record // changes, so nothing is mutated here either — nothing to persist. result = { ok: true }; } else { - svc.address = parsed.address; - svc.port = parsed.port; - svc.artifactHash = parsed.artifactHash; - svc.env = parsed.env; - svc.artifactDir = parsed.artifactDir; + inst.artifactHash = body.artifactHash; + inst.env = body.env; + inst.artifactDir = body.artifactDir; clearBackoffTimer(rt); schedulePersist(); - // A new or changed deployment, or a service that is not up: a fresh + // A new or changed deployment, or an instance that is not up: a fresh // start with a clean log, once the old process (if any) has exited. result = await beginStart(rt, true, async (token) => { await stopsFinished(rt); await killChild(rt, TERMINATE_GRACE_MS); - const superseded = supersededOutcome(svc, rt, token); + const superseded = supersededOutcome(inst, rt, token); if (superseded) return superseded; - svc.status = 'stopped'; - delete svc.pid; + inst.status = 'stopped'; + delete inst.pid; clearStableTimer(rt); rt.consecutiveFastExits = 0; - return spawnAndWaitUntilListening(app, id, svc, rt, true, token); + return spawnAndWaitUntilListening(inst, rt, true, token); }); } if (!result.ok) { - return text(res, 500, result.message ?? 'failed to start the service'); + return text(res, 500, result.message ?? 'failed to start the instance'); } - res.writeHead(204); - res.end(); + noContent(res); } /** Byte offset a follow should start at to show the last `tailLines` complete lines; `<= 0` (or an empty file) starts at the current end, no backlog. */ @@ -804,10 +810,10 @@ function main(): void { const tick = (): void => { if (stopped) return; - // A fresh service start cleared the file — resume from its new - // beginning. Watching the clear generation (not just a size decrease) - // catches it even when the clear and the first new write both land - // between two ticks. + // A fresh start cleared the file — resume from its new beginning. + // Watching the clear generation (not just a size decrease) catches it + // even when the clear and the first new write both land between two + // ticks. const currentGeneration = logGeneration.get(logPath) ?? 0; if (currentGeneration !== generation) { generation = currentGeneration; @@ -839,6 +845,50 @@ function main(): void { res.on('close', stop); } + function handleLogs(url: URL, res: http.ServerResponse, inst: InstanceRecord): void { + if (url.searchParams.get('follow') === '1') { + const tailParam = url.searchParams.get('tail'); + const tailLines = tailParam === null ? 0 : Math.max(0, Number.parseInt(tailParam, 10) || 0); + handleFollowLogs(res, inst.logPath, tailLines); + return; + } + const content = fs.existsSync(inst.logPath) ? fs.readFileSync(inst.logPath) : Buffer.alloc(0); + res.writeHead(200, { 'content-type': 'text/plain; charset=utf-8' }); + res.end(content); + } + + /** The instances `ids` name, or the ids no instance has. */ + function instancesByIds( + ids: readonly string[], + ): { readonly found: InstanceRecord[] } | { readonly unknown: string[] } { + const unknown = ids.filter((id) => state.instances[id] === undefined); + if (unknown.length > 0) return { unknown }; + return { found: ids.flatMap((id) => state.instances[id] ?? []) }; + } + + function unknownInstances(res: http.ServerResponse, ids: readonly string[]): void { + text(res, 404, `no such instance: ${ids.map((id) => `"${id}"`).join(', ')}`); + } + + async function handleBulk( + req: http.IncomingMessage, + res: http.ServerResponse, + action: 'start' | 'stop', + ): Promise { + const body = await readJsonBody(req); + if (!isIdsBody(body)) return text(res, 400, 'malformed body: expected { "ids": string[] }'); + const selected = instancesByIds(body.ids); + if ('unknown' in selected) return unknownInstances(res, selected.unknown); + if (action === 'stop') { + await Promise.all(selected.found.map((inst) => stopInstance(inst, getRuntime(inst.id)))); + return noContent(res); + } + const results = await Promise.all(selected.found.map(startInstance)); + const failures = results.flatMap((r) => (r.ok ? [] : [r.message ?? 'failed to start'])); + if (failures.length > 0) return text(res, 500, failures.join('\n')); + noContent(res); + } + async function handleRequest(req: http.IncomingMessage, res: http.ServerResponse): Promise { const url = new URL(req.url ?? '/', `http://127.0.0.1:${String(port)}`); const method = req.method ?? 'GET'; @@ -851,111 +901,52 @@ function main(): void { return json(res, 200, { version: ownVersion }); } - if (segments[0] === 'apps' && segments.length >= 2) { - const app = segments[1]; - if (app === undefined || !isValidSegment(app)) return badSegment(res, app ?? ''); - - if (method === 'PUT' && segments.length === 4 && segments[2] === 'services') { - const id = segments[3]; - if (id === undefined || !isValidSegment(id)) return badSegment(res, id ?? ''); - const svc = await getOrCreateService(app, id); - return json(res, 200, { port: svc.port, url: `http://localhost:${String(svc.port)}` }); - } - - if ( - method === 'PUT' && - segments.length === 5 && - segments[2] === 'services' && - segments[4] === 'deployment' - ) { - const id = segments[3]; - if (id === undefined || !isValidSegment(id)) return badSegment(res, id ?? ''); - return handleDeployment(req, res, app, id); - } - - if (method === 'GET' && segments.length === 3 && segments[2] === 'services') { - const appRec = state[app]; - const list = appRec ? Object.values(appRec.services).map(serviceView) : []; - return json(res, 200, list); - } - - if ( - method === 'GET' && - segments.length === 5 && - segments[2] === 'services' && - segments[4] === 'logs' - ) { - const id = segments[3]; - if (id === undefined || !isValidSegment(id)) return badSegment(res, id ?? ''); - const svc = state[app]?.services[id]; - if (!svc) return text(res, 404, `no such service "${id}" in app "${app}"`); - if (url.searchParams.get('follow') === '1') { - const tailParam = url.searchParams.get('tail'); - const tailLines = - tailParam === null ? 0 : Math.max(0, Number.parseInt(tailParam, 10) || 0); - handleFollowLogs(res, svc.logPath, tailLines); - return; - } - const content = fs.existsSync(svc.logPath) ? fs.readFileSync(svc.logPath) : Buffer.alloc(0); - res.writeHead(200, { 'content-type': 'text/plain; charset=utf-8' }); - res.end(content); - return; - } + if (segments[0] !== 'instances') { + res.writeHead(404); + res.end(); + return; + } - if (method === 'POST' && segments.length === 3 && segments[2] === 'stop') { - const appRec = state[app]; - if (appRec) { - await Promise.all( - Object.entries(appRec.services).map(([id, svc]) => - stopService(svc, getRuntime(app, id)), - ), + if (segments.length === 1) { + if (method === 'GET') return json(res, 200, Object.values(state.instances).map(instanceView)); + if (method === 'POST') { + const body = await readJsonBody(req); + if (!isCreateBody(body)) { + return text( + res, + 400, + 'malformed body: expected { "labels": Record, "port"?: number }', ); } - res.writeHead(204); - res.end(); - return; + return json(res, 201, instanceView(await createInstance(body))); } + } - if (method === 'POST' && segments.length === 3 && segments[2] === 'start') { - const appRec = state[app]; - const results = appRec - ? await Promise.all( - Object.entries(appRec.services).map(([id, svc]) => startService(app, id, svc)), - ) - : []; - const failures = results.flatMap((r) => (r.ok ? [] : [r.message ?? 'failed to start'])); - if (failures.length > 0) return text(res, 500, failures.join('\n')); - res.writeHead(204); - res.end(); - return; - } + if (method === 'POST' && segments.length === 2 && segments[1] === 'start') { + return handleBulk(req, res, 'start'); + } + if (method === 'POST' && segments.length === 2 && segments[1] === 'stop') { + return handleBulk(req, res, 'stop'); + } - if (method === 'DELETE' && segments.length === 2) { - const appRec = state[app]; - if (appRec) { - for (const id of Object.keys(appRec.services)) getRuntime(app, id).deleted = true; - await Promise.all( - Object.entries(appRec.services).map(([id, svc]) => - stopService(svc, getRuntime(app, id)), - ), - ); - for (const id of Object.keys(appRec.services)) { - const key = runtimeKey(app, id); - runtimes.get(key)?.log?.close(); - runtimes.delete(key); - } - delete state[app]; - schedulePersist(); - // get-port holds every port it returns for up to 30 seconds; the - // freed ports must be free again for a `--fresh` restart. - // `usedServicePorts()` already keeps other apps' ports out. - clearLockedPorts(); - } - await fs.promises.rm(path.join(stateDir, 'logs', app), { recursive: true, force: true }); - res.writeHead(204); - res.end(); - return; - } + const id = segments[1]; + if (id !== undefined && !INSTANCE_ID_RE.test(id)) { + return text(res, 400, `invalid instance id "${id}"`); + } + const inst = id === undefined ? undefined : state.instances[id]; + + if (method === 'DELETE' && segments.length === 2) { + if (inst) await deleteInstance(inst); + return noContent(res); + } + if (!inst) return unknownInstances(res, id === undefined ? [] : [id]); + + if (method === 'GET' && segments.length === 2) return json(res, 200, instanceView(inst)); + if (method === 'PUT' && segments.length === 3 && segments[2] === 'deployment') { + return handleDeployment(req, res, inst); + } + if (method === 'GET' && segments.length === 3 && segments[2] === 'logs') { + return handleLogs(url, res, inst); } res.writeHead(404); @@ -990,19 +981,18 @@ function main(): void { process.on('SIGTERM', () => void shutdown()); process.on('SIGINT', () => void shutdown()); - readJsonFile(appsJsonPath, isAppsState) + discardStateFromEarlierVersions(stateDir, statePath, logsDir) + .then(() => readJsonFile(statePath, isInstancesState)) .then((loaded) => { if (loaded) { // A fresh process has no live children to supervise. `held` is an // operator-visible signal and survives; `running`/`backoff` assumed a // child this process never spawned, so they reset to `stopped` — the // next deployment PUT always starts fresh (spec's "ALWAYS starts"). - for (const appRec of Object.values(loaded)) { - for (const svc of Object.values(appRec.services)) { - if (svc.status === 'running' || svc.status === 'backoff') { - svc.status = 'stopped'; - delete svc.pid; - } + for (const inst of Object.values(loaded.instances)) { + if (inst.status === 'running' || inst.status === 'backoff') { + inst.status = 'stopped'; + delete inst.pid; } } state = loaded; @@ -1022,4 +1012,30 @@ function main(): void { }); } +/** + * Removes state files this version does not write, and log directories: an + * earlier version kept its state in another JSON file and grouped its logs in + * directories. Local dev state is never migrated across versions; those + * instances are simply gone. + */ +async function discardStateFromEarlierVersions( + stateDir: string, + statePath: string, + logsDir: string, +): Promise { + const stateFiles = await fs.promises.readdir(stateDir).catch(() => []); + await Promise.all( + stateFiles + .map((name) => path.join(stateDir, name)) + .filter((file) => file.endsWith('.json') && file !== statePath) + .map((file) => fs.promises.rm(file, { force: true })), + ); + const logEntries = await fs.promises.readdir(logsDir, { withFileTypes: true }).catch(() => []); + await Promise.all( + logEntries + .filter((entry) => entry.isDirectory()) + .map((entry) => fs.promises.rm(path.join(logsDir, entry.name), { recursive: true })), + ); +} + main(); diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/exports/index.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/exports/index.ts index 55af18be1..46a39ae16 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/exports/index.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/exports/index.ts @@ -8,10 +8,12 @@ export type { BucketsClient, ComputeClient, + CreateInstanceRequest, DatabaseInfo, DeploymentRequest, + InstanceInfo, + InstanceStatus, PostgresClient, - ServiceInfo, } from '../client.ts'; export { bucketsClient, computeClient, postgresClient } from '../client.ts'; export type { DaemonName, DaemonRootOptions, RegistryEntry } from '../daemon.ts'; From c34554bcdaad5b466eb8def5cf97f44e598ec978 Mon Sep 17 00:00:00 2001 From: willbot Date: Tue, 13 Oct 2026 16:26:40 +0200 Subject: [PATCH 04/17] refactor(local-target): find compute instances by their labels until the mapping lands Temporary adaptation so `prisma dev` keeps working on instance ids. The App and Deployment providers, the emulators hook, attach and the --fresh teardown find a service's instance by the labels Composer gives it on create (composer.directory, composer.app, composer.address), creating it when none exists. The per-directory mapping file replaces this lookup. Failures that name an instance id are rewritten to name the service's address, so what `prisma dev` prints is unchanged. The integration tests read the new listing route. Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- .../src/__tests__/compute-port-order.test.ts | 48 ++++++----- .../local-target/src/compute-instances.ts | 80 +++++++++++++++++++ .../0-lowering/local-target/src/compute.ts | 63 ++++++--------- .../local-target/src/exports/index.ts | 1 + .../__tests__/local-target-emulators.test.ts | 1 + .../__tests__/local-target-teardown.test.ts | 10 +-- .../target/src/local-target/attach.ts | 56 +++++++++---- .../target/src/local-target/emulators.ts | 4 +- .../target/src/local-target/teardown.ts | 18 ++--- .../test/local-dev-store.integration.ts | 11 ++- .../integration/test/local-dev.integration.ts | 11 ++- 11 files changed, 200 insertions(+), 103 deletions(-) create mode 100644 packages/1-prisma-cloud/0-lowering/local-target/src/compute-instances.ts diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-port-order.test.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-port-order.test.ts index ecdbc46b9..08eef9cd9 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-port-order.test.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-port-order.test.ts @@ -4,21 +4,23 @@ import * as os from 'node:os'; import * as path from 'node:path'; import { fileURLToPath } from 'node:url'; import type { ContainerInstance } from '@internal/core/config'; -import { - computeClient, - EMULATORS_DIR_ENV, - ensureDaemon, - stopDaemon, -} from '@internal/dev-emulators'; +import { EMULATORS_DIR_ENV, ensureDaemon, stopDaemon } from '@internal/dev-emulators'; import { App } from 'alchemy/Prisma'; import * as Effect from 'effect/Effect'; import { LocalAppProvider, reserveServicePorts } from '../compute.ts'; +import { + appServiceInstances, + ensureServiceInstance, + serviceAddressOf, +} from '../compute-instances.ts'; const savedEmulatorsDir = process.env[EMULATORS_DIR_ENV]; let registryRoot: string; +let devDir: string; beforeEach(async () => { registryRoot = fs.mkdtempSync(path.join(os.tmpdir(), 'local-target-port-order-')); + devDir = path.join(registryRoot, 'project', '.prisma-composer', 'dev'); process.env[EMULATORS_DIR_ENV] = registryRoot; const entry = fileURLToPath(import.meta.resolve('@internal/dev-emulators/compute-main')); await ensureDaemon('compute', entry, { registryRoot }); @@ -49,7 +51,7 @@ const reconcileInput = (id: string, news: Record) => async function reconcileApp(container: ContainerInstance, address: string): Promise { const provider = await Effect.runPromise( - App.Provider.pipe(Effect.provide(LocalAppProvider({ container, devDir: '/dev/null/unused' }))), + App.Provider.pipe(Effect.provide(LocalAppProvider({ container, devDir }))), ); const attributes: App['Attributes'] = await Effect.runPromise( provider.reconcile( @@ -59,16 +61,18 @@ async function reconcileApp(container: ContainerInstance, address: string): Prom return Number(new URL(attributes.appEndpointDomain ?? '').port); } -async function reservedPorts(appName: string): Promise> { - const services = await computeClient().listServices(appName); - return Object.fromEntries(services.map((svc) => [svc.id, svc.port])); +async function reservedPorts(app: string): Promise> { + const instances = await appServiceInstances({ devDir, app }); + return Object.fromEntries( + instances.map((instance) => [serviceAddressOf(instance), instance.port]), + ); } test('each App provider returns the port reserved for its service, even when the providers race', async () => { for (let run = 0; run < 10; run++) { const appName = `port-order-${String(run)}`; const container = fakeContainer(appName); - await reserveServicePorts(container, ['quotes', 'gateway']); + await reserveServicePorts(container, devDir, ['quotes', 'gateway']); const reserved = await reservedPorts(appName); const [gateway, quotes] = await Promise.all([ @@ -84,28 +88,30 @@ test('each App provider returns the port reserved for its service, even when the }); test('a failed reservation names the service', async () => { - await expect(reserveServicePorts(fakeContainer('Bad-App'), ['quotes'])).rejects.toThrow( - 'service "quotes"', - ); + await stopDaemon('compute', { registryRoot }); + + await expect( + reserveServicePorts(fakeContainer('port-order-down'), devDir, ['quotes']), + ).rejects.toThrow('service "quotes"'); }); test('reserving the services of an app that already has ports keeps those ports', async () => { const container = fakeContainer('port-order-warm'); - const client = computeClient(); - const gateway = await client.ensureService('port-order-warm', 'gateway'); - const quotes = await client.ensureService('port-order-warm', 'quotes'); + const scope = { devDir, app: 'port-order-warm' }; + const gateway = await ensureServiceInstance(scope, 'gateway'); + const quotes = await ensureServiceInstance(scope, 'quotes'); - await reserveServicePorts(container, ['quotes', 'gateway']); + await reserveServicePorts(container, devDir, ['quotes', 'gateway']); expect(await reconcileApp(container, 'quotes')).toBe(quotes.port); expect(await reconcileApp(container, 'gateway')).toBe(gateway.port); }); -test('a dotted service address reserves the same emulator service the App provider uses', async () => { +test('a dotted service address reserves the same emulator instance the App provider uses', async () => { const container = fakeContainer('port-order-nested'); - await reserveServicePorts(container, ['shop.quotes']); + await reserveServicePorts(container, devDir, ['shop.quotes']); - const listed = await computeClient().listServices('port-order-nested'); + const listed = await appServiceInstances({ devDir, app: 'port-order-nested' }); expect(await reconcileApp(container, 'shop.quotes')).toBe(listed[0]?.port ?? -1); expect(listed).toHaveLength(1); }); diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/compute-instances.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/compute-instances.ts new file mode 100644 index 000000000..4799ed9aa --- /dev/null +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/compute-instances.ts @@ -0,0 +1,80 @@ +/** + * Temporary: finds the Compute instance that runs a service of the app run + * from a directory by reading the labels Composer gives the instance when it + * creates it. The per-directory mapping file replaces this module; Composer + * must not identify instances by their labels. + */ +import * as path from 'node:path'; +import { computeClient, type InstanceInfo } from '@internal/dev-emulators'; + +/** The app run from the directory that holds `devDir` (`/.prisma-composer/dev`). */ +export interface ServiceInstanceScope { + readonly devDir: string; + readonly app: string; +} + +const DIRECTORY_LABEL = 'composer.directory'; +const APP_LABEL = 'composer.app'; +const ADDRESS_LABEL = 'composer.address'; + +function directoryOf(devDir: string): string { + return path.resolve(devDir, '..', '..'); +} + +function belongsTo(instance: InstanceInfo, scope: ServiceInstanceScope): boolean { + return ( + instance.labels[DIRECTORY_LABEL] === directoryOf(scope.devDir) && + instance.labels[APP_LABEL] === scope.app + ); +} + +export function serviceAddressOf(instance: InstanceInfo): string { + return instance.labels[ADDRESS_LABEL] ?? instance.id; +} + +export async function appServiceInstances(scope: ServiceInstanceScope): Promise { + const instances = await computeClient().listInstances(); + return instances.filter((instance) => belongsTo(instance, scope)); +} + +/** The instance that runs `address`, created when there is none. */ +export async function ensureServiceInstance( + scope: ServiceInstanceScope, + address: string, +): Promise { + const existing = (await appServiceInstances(scope)).find( + (instance) => serviceAddressOf(instance) === address, + ); + if (existing) return existing; + return computeClient().createInstance({ + labels: { + [DIRECTORY_LABEL]: directoryOf(scope.devDir), + [APP_LABEL]: scope.app, + [ADDRESS_LABEL]: address, + }, + }); +} + +export async function deleteAppServiceInstances(scope: ServiceInstanceScope): Promise { + const client = computeClient(); + for (const instance of await appServiceInstances(scope)) { + await client.deleteInstance(instance.id); + } +} + +/** + * The emulator's failure messages name instances by id; `prisma dev` names + * services by address. Rewrites a plain `Error`'s message and leaves any other + * error, such as a structured one, untouched. + */ +export function withServiceAddresses(error: unknown, instances: readonly InstanceInfo[]): unknown { + if (!(error instanceof Error) || error.constructor !== Error) return error; + let message = error.message; + for (const instance of instances) { + message = message.replaceAll( + `instance "${instance.id}"`, + `service "${serviceAddressOf(instance)}"`, + ); + } + return message === error.message ? error : new Error(message, { cause: error }); +} diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts index d8fe3754c..446fcb2c8 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts @@ -20,6 +20,7 @@ import * as Predicate from 'effect/Predicate'; import * as Redacted from 'effect/Redacted'; import { appNameOf } from './app-name.ts'; import { extractComputeArtifact } from './artifact-extract.ts'; +import { ensureServiceInstance, withServiceAddresses } from './compute-instances.ts'; import { envStore, secretsStore } from './dev-store.ts'; import { DEV_TIMESTAMP, projectIdOfInput } from './upstream-attributes.ts'; @@ -148,20 +149,6 @@ function readManifestAddress(artifactDir: string): string { return parsed.address; } -/** - * A service address is dot-separated (`"orders.service"`) but the emulator's - * `` segment must match `/^[a-z0-9][a-z0-9-]*$/`: disallowed characters - * become dashes, runs collapse. The real address still rides the deployment - * body's `address` field untouched, so listings show it verbatim. - */ -function slugServiceId(address: string): string { - const slug = address - .toLowerCase() - .replace(/[^a-z0-9]+/g, '-') - .replace(/^-+|-+$/g, ''); - return slug.length > 0 ? slug : 'svc'; -} - async function materializeEnv( devDir: string, address: string, @@ -177,11 +164,10 @@ async function materializeEnv( } /** - * `Prisma.App` → the Compute emulator: reserves (or returns) the service's - * stable port. The app id here IS the service's own address, which - * `Deployment` slugs back into the emulator's id. `delete` is a no-op — - * instance removal belongs to `teardown` (`DELETE /apps/`), not - * per-resource Alchemy deletes. + * `Prisma.App` → the Compute emulator: finds or creates the instance that + * runs the service, which holds the service's port. The app id here IS the + * service's own address. `delete` is a no-op — instance removal belongs to + * `teardown`, not per-resource Alchemy deletes. */ export function LocalAppProvider( input: LocalTargetProvidersInput, @@ -191,9 +177,9 @@ export function LocalAppProvider( reconcile: ({ id, news }) => Effect.tryPromise({ try: async () => { - const appName = appNameOf(input.container); + const scope = { devDir: input.devDir, app: appNameOf(input.container) }; const name = news.displayName ?? id; - const { url } = await computeClient().ensureService(appName, slugServiceId(name)); + const { url } = await ensureServiceInstance(scope, name); return { appId: name, name, @@ -214,20 +200,21 @@ export function LocalAppProvider( } /** - * Reserves each service's emulator port one at a time, in the order given, - * before Alchemy applies the `App` resources concurrently. A new service gets + * Ensures each service's emulator instance one at a time, in the order given, + * before Alchemy applies the `App` resources concurrently. A new instance gets * the smallest free port, so on a fresh start the ports follow this order; a - * service that already has a port keeps it. `serviceAppNames` are the - * `App` resources' `displayName`s. + * service that already has an instance keeps its port. `serviceAppNames` are + * the `App` resources' `displayName`s. */ export async function reserveServicePorts( container: ContainerInstance | undefined, + devDir: string, serviceAppNames: readonly string[], ): Promise { - const appName = appNameOf(container); + const scope = { devDir, app: appNameOf(container) }; for (const service of serviceAppNames) { try { - await computeClient().ensureService(appName, slugServiceId(service)); + await ensureServiceInstance(scope, service); } catch (cause) { const reason = cause instanceof Error ? cause.message : String(cause); throw new Error(`reserving a local port for service "${service}" failed: ${reason}`, { @@ -281,8 +268,8 @@ export function LocalEnvironmentVariableProvider( } /** - * `Prisma.Deployment` → unpacks the artifact once per hash, fetches the - * emulator's assigned port, materializes the child's full env (env store + + * `Prisma.Deployment` → unpacks the artifact once per hash, finds the + * service's emulator instance and its port, materializes the child's full env (env store + * secrets + the port override + `PATH`/`HOME`), and puts the deployment — the * emulator (re)starts the child only when the hash or env actually changed. * `portMapping` is ignored on purpose: the emulator owns port allocation, and @@ -297,9 +284,8 @@ export function LocalDeploymentProvider( reconcile: ({ news }) => Effect.tryPromise({ try: async (): Promise => { - const app = appNameOf(input.container); + const scope = { devDir: input.devDir, app: appNameOf(input.container) }; const appId = appIdOfInput(news.app); - const emulatorId = slugServiceId(appId); if (news.artifactPath === undefined) { throw new Error('local Deployment requires an artifactPath — nothing to run.'); } @@ -311,15 +297,14 @@ export function LocalDeploymentProvider( } const address = readManifestAddress(artifactDir); - const { port } = await computeClient().ensureService(app, emulatorId); + const instance = await ensureServiceInstance(scope, appId); + const { port } = instance; const env = await materializeEnv(input.devDir, address, port); - await computeClient().putDeployment(app, emulatorId, { - address, - artifactDir, - artifactHash, - env, - port, - }); + try { + await computeClient().putDeployment(instance.id, { artifactDir, artifactHash, env }); + } catch (error) { + throw withServiceAddresses(error, [instance]); + } const url = `http://localhost:${port}`; return { diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/exports/index.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/exports/index.ts index 621f20fa5..d03e76ab1 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/exports/index.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/exports/index.ts @@ -2,6 +2,7 @@ export * from '../app-name.ts'; export * from '../bucket.ts'; export * from '../compute.ts'; +export * from '../compute-instances.ts'; export * from '../dev-store.ts'; export * from '../postgres.ts'; export * from '../providers.ts'; diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts index 184fedcc1..916e4ee93 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts @@ -21,6 +21,7 @@ mock.module('@internal/local-target', () => ({ resolvePackageEntry: (specifier: string) => specifier, reserveServicePorts: async ( container: ContainerInstance | undefined, + _devDir: string, addresses: readonly string[], ) => { events.push(`reserve ${container?.input.appName ?? '?'}: ${addresses.join(', ')}`); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-teardown.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-teardown.test.ts index 179ba52da..1e7924d16 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-teardown.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-teardown.test.ts @@ -18,12 +18,6 @@ mock.module('@internal/dev-emulators', () => ({ events.push(`daemon ${name}`); return { url: `http://127.0.0.1/${name}` }; }, - computeClient: () => ({ - deleteApp: async (app: string) => { - events.push(`delete compute ${app}`); - if (computeDeleteFails) throw new Error('compute delete failed'); - }, - }), postgresClient: unreachable('postgres'), bucketsClient: unreachable('buckets'), })); @@ -31,6 +25,10 @@ mock.module('@internal/dev-emulators', () => ({ mock.module('@internal/local-target', () => ({ ...RealLocalTarget, resolvePackageEntry: (specifier: string) => specifier, + deleteAppServiceInstances: async ({ app }: { app: string }) => { + events.push(`delete compute ${app}`); + if (computeDeleteFails) throw new Error('compute delete failed'); + }, removeLocalPaths: () => { events.push('remove local paths'); }, diff --git a/packages/1-prisma-cloud/1-extensions/target/src/local-target/attach.ts b/packages/1-prisma-cloud/1-extensions/target/src/local-target/attach.ts index 7d29e4552..3c60d12bc 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/local-target/attach.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/local-target/attach.ts @@ -6,6 +6,12 @@ */ import type { LocalTargetAttachInput, LocalTargetAttachment } from '@internal/core/config'; import { computeClient } from '@internal/dev-emulators'; +import { + appServiceInstances, + type ServiceInstanceScope, + serviceAddressOf, + withServiceAddresses, +} from '@internal/local-target'; import { prismaCloudContainerOf } from '../container.ts'; const RELIST_INTERVAL_MS = 2000; @@ -22,7 +28,11 @@ interface LogLine { * dropped (an emulator restart shows a gap in that service's stream, never a * dead session). */ -async function* mergedLogs(app: string, signal: AbortSignal, tail: number): AsyncIterable { +async function* mergedLogs( + scope: ServiceInstanceScope, + signal: AbortSignal, + tail: number, +): AsyncIterable { const client = computeClient(); const followed = new Set(); // Backlog is emitted once per service, on its FIRST follow — a follower @@ -46,7 +56,7 @@ async function* mergedLogs(app: string, signal: AbortSignal, tail: number): Asyn void (async () => { try { let buffer = ''; - for await (const chunk of client.followLogs(app, id, signal, { tail: followTail })) { + for await (const chunk of client.followLogs(id, signal, { tail: followTail })) { buffer += chunk; let newlineAt = buffer.indexOf('\n'); while (newlineAt !== -1) { @@ -64,14 +74,12 @@ async function* mergedLogs(app: string, signal: AbortSignal, tail: number): Asyn }; const relist = async (): Promise => { - const services = await client.listServices(app); - // The emulator's `id` is the (path-segment-safe) key `followLogs` needs; - // the log line is labelled with the real, dotted address — the same - // identity the front door prints, so a service's endpoint line and its - // log lines name it identically (a nested module's service address - // contains dots the emulator's own id path segment cannot carry — see - // compute.ts's `slugServiceId`). - for (const svc of services) follow(svc.id, svc.address); + // Log lines are labelled with the service's address — the identity the + // front door prints, so a service's endpoint line and its log lines name + // it identically. + for (const instance of await appServiceInstances(scope)) { + follow(instance.id, serviceAddressOf(instance)); + } }; await relist(); @@ -99,16 +107,32 @@ async function* mergedLogs(app: string, signal: AbortSignal, tail: number): Asyn } export async function devAttach(input: LocalTargetAttachInput): Promise { - const app = prismaCloudContainerOf(input.container).input.appName; + const scope = { + devDir: input.devDir, + app: prismaCloudContainerOf(input.container).input.appName, + }; const client = computeClient(); return { - startServices: () => client.startApp(app), + startServices: async () => { + const instances = await appServiceInstances(scope); + try { + await client.startInstances(instances.map((instance) => instance.id)); + } catch (error) { + throw withServiceAddresses(error, instances); + } + }, endpoints: async () => { - const services = await client.listServices(app); - return services.map((svc) => ({ address: svc.address, url: svc.url })); + const instances = await appServiceInstances(scope); + return instances.map((instance) => ({ + address: serviceAddressOf(instance), + url: instance.url, + })); + }, + logs: (signal, opts) => mergedLogs(scope, signal, opts?.tail ?? 0), + stopServices: async () => { + const instances = await appServiceInstances(scope); + await client.stopInstances(instances.map((instance) => instance.id)); }, - logs: (signal, opts) => mergedLogs(app, signal, opts?.tail ?? 0), - stopServices: () => client.stopApp(app), }; } diff --git a/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts b/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts index 236d8cdbb..8b3fbdc19 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts @@ -6,7 +6,7 @@ * `postgres` resource kind (REVISED — Postgres is a first-class daemon * since the programmatic `@prisma/dev` adoption, operator review of #162). * - * Once Compute is up, every service's port is reserved in graph order, so a + * Once Compute is up, every service's instance is ensured in graph order, so a * fresh start numbers ports dependencies first; Alchemy's concurrent `App` * applies would otherwise pick them in arrival order. * @@ -50,7 +50,7 @@ function serviceAppNames(input: LocalTargetEmulatorsInput): string[] { export async function runDevEmulators(input: LocalTargetEmulatorsInput): Promise { const { url: computeUrl } = await ensureDaemon('compute', daemonEntry('compute')); console.log(`[dev] compute emulator ready at ${computeUrl}`); - await reserveServicePorts(input.container, serviceAppNames(input)); + await reserveServicePorts(input.container, input.devDir, serviceAppNames(input)); if (usesBuckets(input)) { const { url: bucketsUrl } = await ensureDaemon('buckets', daemonEntry('buckets')); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/local-target/teardown.ts b/packages/1-prisma-cloud/1-extensions/target/src/local-target/teardown.ts index 0b9befc7b..e24c9c492 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/local-target/teardown.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/local-target/teardown.ts @@ -18,13 +18,8 @@ import type { TeardownInput } from '@internal/core/config'; import { DEV_DIR } from '@internal/core/config'; -import { - bucketsClient, - computeClient, - ensureDaemon, - postgresClient, -} from '@internal/dev-emulators'; -import { removeLocalPaths } from '@internal/local-target'; +import { bucketsClient, ensureDaemon, postgresClient } from '@internal/dev-emulators'; +import { deleteAppServiceInstances, removeLocalPaths } from '@internal/local-target'; import { prismaCloudContainerOf } from '../container.ts'; import { daemonEntry } from './daemon-entry.ts'; @@ -41,15 +36,16 @@ async function tolerateUnreachable(action: () => Promise): Promise { export async function runDevTeardown(input: TeardownInput): Promise { const app = prismaCloudContainerOf(input.container).input.appName; const cwd = process.cwd(); + const devDir = `${cwd}/${DEV_DIR}`; await tolerateUnreachable(() => postgresClient().deleteApp(app)); - // Compute persists port allocations across restarts, so a stopped daemon - // would hand the old ports back; start it so the delete always lands. + // Compute persists instances and their ports across restarts, so a stopped + // daemon would hand the old ports back; start it so the delete always lands. try { await ensureDaemon('compute', daemonEntry('compute')); - await computeClient().deleteApp(app); + await deleteAppServiceInstances({ devDir, app }); } finally { await tolerateUnreachable(() => bucketsClient().deleteApp(app)); - removeLocalPaths([`${cwd}/${DEV_DIR}`, `${cwd}/.alchemy/state/${app}/dev`]); + removeLocalPaths([devDir, `${cwd}/.alchemy/state/${app}/dev`]); } } diff --git a/test/integration/test/local-dev-store.integration.ts b/test/integration/test/local-dev-store.integration.ts index f71517e01..5193fa510 100644 --- a/test/integration/test/local-dev-store.integration.ts +++ b/test/integration/test/local-dev-store.integration.ts @@ -240,13 +240,16 @@ interface ServiceInfo { readonly pid?: number; } -/** The documented Compute-emulator wire protocol (spec § 2 compute-main.ts) — plain fetch, never a typed client. */ +/** The documented Compute-emulator wire protocol (spec § 2 compute-main.ts) — plain fetch, never the typed client. Finds the app's instances by the labels Composer gives them. */ async function listComputeServices(app: string): Promise { const entry = readEmulatorEntry('compute'); if (entry === undefined) throw new Error('compute emulator registry entry not found'); - const res = await fetch(`http://127.0.0.1:${entry.port}/apps/${app}/services`); - if (!res.ok) throw new Error(`compute emulator listServices failed: ${res.status}`); - return (await res.json()) as ServiceInfo[]; + const res = await fetch(`http://127.0.0.1:${entry.port}/instances`); + if (!res.ok) throw new Error(`compute emulator listInstances failed: ${res.status}`); + const instances = (await res.json()) as (ServiceInfo & { labels: Record })[]; + return instances + .filter((instance) => instance.labels['composer.app'] === app) + .map((instance) => ({ ...instance, address: instance.labels['composer.address'] ?? '' })); } interface PostgresDatabaseInfo { diff --git a/test/integration/test/local-dev.integration.ts b/test/integration/test/local-dev.integration.ts index 85373e50f..98331bcdc 100644 --- a/test/integration/test/local-dev.integration.ts +++ b/test/integration/test/local-dev.integration.ts @@ -299,13 +299,16 @@ interface ServiceInfo { readonly pid?: number; } -/** The documented Compute-emulator wire protocol (spec § 2 compute-main.ts) — plain fetch, never the typed client. */ +/** The documented Compute-emulator wire protocol (spec § 2 compute-main.ts) — plain fetch, never the typed client. Finds the app's instances by the labels Composer gives them. */ async function listComputeServices(): Promise { const entry = readEmulatorEntry('compute'); if (entry === undefined) throw new Error('compute emulator registry entry not found'); - const res = await fetch(`http://127.0.0.1:${entry.port}/apps/${APP_NAME}/services`); - if (!res.ok) throw new Error(`compute emulator listServices failed: ${res.status}`); - return (await res.json()) as ServiceInfo[]; + const res = await fetch(`http://127.0.0.1:${entry.port}/instances`); + if (!res.ok) throw new Error(`compute emulator listInstances failed: ${res.status}`); + const instances = (await res.json()) as (ServiceInfo & { labels: Record })[]; + return instances + .filter((instance) => instance.labels['composer.app'] === APP_NAME) + .map((instance) => ({ ...instance, address: instance.labels['composer.address'] ?? '' })); } /** From b20468ac92cd827c4ec215f6ef66ba4e62ca3937 Mon Sep 17 00:00:00 2001 From: willbot Date: Tue, 13 Oct 2026 16:34:00 +0200 Subject: [PATCH 05/17] feat(local-target): record each directory's compute instances in emulator-instances.json `prisma dev` now records, in `.prisma-composer/dev/emulator-instances.json`, the app name and, per service address, the Compute instance that runs it and that instance's port. Every Composer operation on a Compute instance goes through this record: the emulators hook, the App and Deployment providers, attach (endpoints, logs, start and stop) and the --fresh teardown. Labels are written on create and never read back, so two directories running the same app get separate instances. The emulators hook reuses a recorded instance while the emulator still has it. Otherwise it creates one, requesting the remembered port, and prints one line when the emulator had to give another port. A re-created instance runs no deployment, and a warm start's converge is a no-op, so the Deployment provider's diff asks for an update when its instance does not run the recorded artifact, and the App provider's diff asks for one when its instance's URL changed. Deleting a removed service's App deletes its instance and its entry. `readEmulatorInstances(directory)` reads the record from a fresh process. Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- .../src/__tests__/compute-port-order.test.ts | 20 +- .../src/__tests__/emulator-instances.test.ts | 251 ++++++++++++++++++ .../local-target/src/compute-instances.ts | 80 ------ .../0-lowering/local-target/src/compute.ts | 77 ++++-- .../0-lowering/local-target/src/dev-store.ts | 2 +- .../local-target/src/emulator-instances.ts | 190 +++++++++++++ .../local-target/src/exports/index.ts | 2 +- .../__tests__/local-target-emulators.test.ts | 30 ++- .../__tests__/local-target-teardown.test.ts | 2 +- .../target/src/local-target/attach.ts | 31 +-- .../target/src/local-target/emulators.ts | 11 +- .../target/src/local-target/teardown.ts | 4 +- .../test/local-dev-store.integration.ts | 20 +- .../integration/test/local-dev.integration.ts | 20 +- 14 files changed, 593 insertions(+), 147 deletions(-) create mode 100644 packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts delete mode 100644 packages/1-prisma-cloud/0-lowering/local-target/src/compute-instances.ts create mode 100644 packages/1-prisma-cloud/0-lowering/local-target/src/emulator-instances.ts diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-port-order.test.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-port-order.test.ts index 08eef9cd9..2c50b1c7e 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-port-order.test.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-port-order.test.ts @@ -8,11 +8,7 @@ import { EMULATORS_DIR_ENV, ensureDaemon, stopDaemon } from '@internal/dev-emula import { App } from 'alchemy/Prisma'; import * as Effect from 'effect/Effect'; import { LocalAppProvider, reserveServicePorts } from '../compute.ts'; -import { - appServiceInstances, - ensureServiceInstance, - serviceAddressOf, -} from '../compute-instances.ts'; +import { computeNodeInstances, ensureComputeInstance } from '../emulator-instances.ts'; const savedEmulatorsDir = process.env[EMULATORS_DIR_ENV]; let registryRoot: string; @@ -62,10 +58,8 @@ async function reconcileApp(container: ContainerInstance, address: string): Prom } async function reservedPorts(app: string): Promise> { - const instances = await appServiceInstances({ devDir, app }); - return Object.fromEntries( - instances.map((instance) => [serviceAddressOf(instance), instance.port]), - ); + const nodes = await computeNodeInstances({ devDir, app }); + return Object.fromEntries(nodes.map(({ address, instance }) => [address, instance.port])); } test('each App provider returns the port reserved for its service, even when the providers race', async () => { @@ -98,8 +92,8 @@ test('a failed reservation names the service', async () => { test('reserving the services of an app that already has ports keeps those ports', async () => { const container = fakeContainer('port-order-warm'); const scope = { devDir, app: 'port-order-warm' }; - const gateway = await ensureServiceInstance(scope, 'gateway'); - const quotes = await ensureServiceInstance(scope, 'quotes'); + const { instance: gateway } = await ensureComputeInstance(scope, 'gateway'); + const { instance: quotes } = await ensureComputeInstance(scope, 'quotes'); await reserveServicePorts(container, devDir, ['quotes', 'gateway']); @@ -111,7 +105,9 @@ test('a dotted service address reserves the same emulator instance the App provi const container = fakeContainer('port-order-nested'); await reserveServicePorts(container, devDir, ['shop.quotes']); - const listed = await appServiceInstances({ devDir, app: 'port-order-nested' }); + const listed = (await computeNodeInstances({ devDir, app: 'port-order-nested' })).map( + (node) => node.instance, + ); expect(await reconcileApp(container, 'shop.quotes')).toBe(listed[0]?.port ?? -1); expect(listed).toHaveLength(1); }); diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts new file mode 100644 index 000000000..aef280287 --- /dev/null +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts @@ -0,0 +1,251 @@ +/** + * The per-directory record of which Compute instance backs each service, + * against a real Compute emulator in a temporary emulators directory. + */ +import { afterEach, beforeEach, describe, expect, test } from 'bun:test'; +import * as fs from 'node:fs'; +import * as os from 'node:os'; +import * as path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import type { ContainerInstance } from '@internal/core/config'; +import { + computeClient, + EMULATORS_DIR_ENV, + ensureDaemon, + stopDaemon, +} from '@internal/dev-emulators'; +import { packageComputeArtifact } from '@internal/lowering/compute'; +import { App, Deployment } from 'alchemy/Prisma'; +import * as Effect from 'effect/Effect'; +import { LocalAppProvider, LocalDeploymentProvider, reserveServicePorts } from '../compute.ts'; +import { + EMULATOR_INSTANCES_FILE, + type EmulatorInstances, + readEmulatorInstances, +} from '../emulator-instances.ts'; + +const APP = 'shared-app'; +const ADDRESSES = ['quotes', 'gateway']; +const savedEmulatorsDir = process.env[EMULATORS_DIR_ENV]; +let root: string; +let registryRoot: string; + +beforeEach(async () => { + root = fs.mkdtempSync(path.join(os.tmpdir(), 'local-target-emulator-instances-')); + registryRoot = path.join(root, 'emulators'); + process.env[EMULATORS_DIR_ENV] = registryRoot; + const entry = fileURLToPath(import.meta.resolve('@internal/dev-emulators/compute-main')); + await ensureDaemon('compute', entry, { registryRoot }); +}); + +afterEach(async () => { + await stopDaemon('compute', { registryRoot }).catch(() => undefined); + fs.rmSync(root, { recursive: true, force: true }); + if (savedEmulatorsDir === undefined) delete process.env[EMULATORS_DIR_ENV]; + else process.env[EMULATORS_DIR_ENV] = savedEmulatorsDir; +}); + +const container: ContainerInstance = { + input: { appName: APP, stage: undefined }, + serialize: () => 'x', +}; + +function projectDir(name: string): { readonly directory: string; readonly devDir: string } { + const directory = path.join(root, name); + return { directory, devDir: path.join(directory, '.prisma-composer', 'dev') }; +} + +/** A packaged artifact whose service answers with its address, its working directory and `version`. */ +function artifactFor(address: string, version: string): string { + const bundleDir = fs.mkdtempSync(path.join(root, `bundle-${address}-`)); + fs.writeFileSync( + path.join(bundleDir, 'main.js'), + `export default { run: async (address) => { + const port = Number(process.env['COMPOSER_' + address.toUpperCase() + '_PORT']); + Bun.serve({ port, fetch: () => new Response(address + ' ' + process.cwd() + ' ${version}') }); +} }; +`, + ); + fs.writeFileSync(path.join(bundleDir, 'server.js'), ''); + return packageComputeArtifact({ id: address, bundleDir, appEntry: 'server.js', address }).path; +} + +const reconcileInput = (id: string, news: Record) => + ({ + id, + fqn: id, + instanceId: id, + news, + olds: undefined, + output: undefined, + session: undefined as never, + bindings: [], + }) as never; + +async function deploy( + devDir: string, + address: string, + version: string, +): Promise { + const provider = await Effect.runPromise( + Deployment.Provider.pipe(Effect.provide(LocalDeploymentProvider({ container, devDir }))), + ); + return Effect.runPromise( + provider.reconcile( + reconcileInput(`${address}-deployment`, { + app: address, + artifactPath: artifactFor(address, version), + }), + ), + ); +} + +async function deploymentDiff(devDir: string, output: Deployment['Attributes']): Promise { + const provider = await Effect.runPromise( + Deployment.Provider.pipe(Effect.provide(LocalDeploymentProvider({ container, devDir }))), + ); + const diff = provider.diff; + if (!diff) throw new Error('the local Deployment provider has no diff'); + return Effect.runPromise(diff({ output } as never)); +} + +async function appProvider(devDir: string) { + return Effect.runPromise( + App.Provider.pipe(Effect.provide(LocalAppProvider({ container, devDir }))), + ); +} + +function mappingOf(directory: string): Promise { + return readEmulatorInstances(directory); +} + +async function nodeOf(directory: string, address: string) { + const node = (await mappingOf(directory))?.nodes[address]; + if (!node) throw new Error(`no instance recorded for "${address}" in ${directory}`); + return node; +} + +async function pidOf(id: string): Promise { + return (await computeClient().getInstance(id))?.pid; +} + +describe('two directories running the same app', () => { + test('each gets its own instances, and a deploy in one restarts nothing in the other', async () => { + const a = projectDir('a'); + const b = projectDir('b'); + await Promise.all([ + reserveServicePorts(container, a.devDir, ADDRESSES), + reserveServicePorts(container, b.devDir, ADDRESSES), + ]); + await Promise.all( + [a, b].flatMap(({ devDir }) => ADDRESSES.map((address) => deploy(devDir, address, 'v1'))), + ); + + const nodesA = (await mappingOf(a.directory))?.nodes ?? {}; + const nodesB = (await mappingOf(b.directory))?.nodes ?? {}; + const ids = [...Object.values(nodesA), ...Object.values(nodesB)].map((n) => n.id); + const ports = [...Object.values(nodesA), ...Object.values(nodesB)].map((n) => n.port); + expect(new Set(ids).size).toBe(4); + expect(new Set(ports).size).toBe(4); + for (const [{ directory }, nodes] of [ + [a, nodesA], + [b, nodesB], + ] as const) { + for (const [address, node] of Object.entries(nodes)) { + const answer = await (await fetch(`http://localhost:${String(node.port)}`)).text(); + expect(answer).toStartWith(`${address} ${fs.realpathSync(directory)}`); + } + } + + const pidsA = await Promise.all(Object.values(nodesA).map((n) => pidOf(n.id))); + await Promise.all(ADDRESSES.map((address) => deploy(b.devDir, address, 'v2'))); + + expect(await Promise.all(Object.values(nodesA).map((n) => pidOf(n.id)))).toEqual(pidsA); + const quotesB = await nodeOf(b.directory, 'quotes'); + expect(await (await fetch(`http://localhost:${String(quotesB.port)}`)).text()).toEndWith(' v2'); + }, 60_000); +}); + +describe('reusing the recorded instances', () => { + test('a recorded instance the emulator no longer has is re-created on its remembered port', async () => { + const { directory, devDir } = projectDir('lost'); + await reserveServicePorts(container, devDir, ['quotes']); + const before = await nodeOf(directory, 'quotes'); + await computeClient().deleteInstance(before.id); + + const changes = await reserveServicePorts(container, devDir, ['quotes']); + + const after = await nodeOf(directory, 'quotes'); + expect(after.id).not.toBe(before.id); + expect(after.port).toBe(before.port); + expect(changes).toEqual([]); + }); + + test('a remembered port another instance now holds gives a new port, reported as a change', async () => { + const { directory, devDir } = projectDir('moved'); + await reserveServicePorts(container, devDir, ['quotes']); + const before = await nodeOf(directory, 'quotes'); + await computeClient().deleteInstance(before.id); + await computeClient().createInstance({ labels: {}, port: before.port }); + + const changes = await reserveServicePorts(container, devDir, ['quotes']); + + const after = await nodeOf(directory, 'quotes'); + expect(after.port).not.toBe(before.port); + expect(changes).toEqual([{ address: 'quotes', previous: before.port, given: after.port }]); + }); + + test('a warm restart after a stop reuses the same instances and ports', async () => { + const { directory, devDir } = projectDir('warm'); + await reserveServicePorts(container, devDir, ADDRESSES); + for (const address of ADDRESSES) await deploy(devDir, address, 'v1'); + const before = (await mappingOf(directory))?.nodes; + const ids = Object.values(before ?? {}).map((n) => n.id); + await computeClient().stopInstances(ids); + + const changes = await reserveServicePorts(container, devDir, ADDRESSES); + await computeClient().startInstances(ids); + + expect(changes).toEqual([]); + expect((await mappingOf(directory))?.nodes).toEqual(before ?? {}); + for (const node of Object.values(before ?? {})) { + expect((await computeClient().getInstance(node.id))?.status).toBe('running'); + } + }, 30_000); + + test('a deployment whose instance was re-created is deployed again even when nothing changed', async () => { + const { directory, devDir } = projectDir('redeploy'); + await reserveServicePorts(container, devDir, ['quotes']); + const deployed = await deploy(devDir, 'quotes', 'v1'); + expect(await deploymentDiff(devDir, deployed)).toBeUndefined(); + + await computeClient().deleteInstance((await nodeOf(directory, 'quotes')).id); + await reserveServicePorts(container, devDir, ['quotes']); + + expect(await deploymentDiff(devDir, deployed)).toEqual({ action: 'update' }); + }, 30_000); + + test("deleting a removed service's App deletes its instance and forgets it", async () => { + const { directory, devDir } = projectDir('removed'); + await reserveServicePorts(container, devDir, ADDRESSES); + const quotes = await nodeOf(directory, 'quotes'); + const provider = await appProvider(devDir); + + await Effect.runPromise(provider.delete({ output: { name: 'quotes' } } as never)); + + expect(await computeClient().getInstance(quotes.id)).toBeUndefined(); + expect(Object.keys((await mappingOf(directory))?.nodes ?? {})).toEqual(['gateway']); + }); +}); + +test('the record is one JSON file in the dev directory, readable from the directory alone', async () => { + const { directory, devDir } = projectDir('file'); + await reserveServicePorts(container, devDir, ['quotes']); + + const onDisk: unknown = JSON.parse( + fs.readFileSync(path.join(devDir, EMULATOR_INSTANCES_FILE), 'utf8'), + ); + const node = await nodeOf(directory, 'quotes'); + expect(onDisk).toEqual({ app: APP, nodes: { quotes: node } }); + expect(await readEmulatorInstances(path.join(root, 'never-ran'))).toBeUndefined(); +}); diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/compute-instances.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/compute-instances.ts deleted file mode 100644 index 4799ed9aa..000000000 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/compute-instances.ts +++ /dev/null @@ -1,80 +0,0 @@ -/** - * Temporary: finds the Compute instance that runs a service of the app run - * from a directory by reading the labels Composer gives the instance when it - * creates it. The per-directory mapping file replaces this module; Composer - * must not identify instances by their labels. - */ -import * as path from 'node:path'; -import { computeClient, type InstanceInfo } from '@internal/dev-emulators'; - -/** The app run from the directory that holds `devDir` (`/.prisma-composer/dev`). */ -export interface ServiceInstanceScope { - readonly devDir: string; - readonly app: string; -} - -const DIRECTORY_LABEL = 'composer.directory'; -const APP_LABEL = 'composer.app'; -const ADDRESS_LABEL = 'composer.address'; - -function directoryOf(devDir: string): string { - return path.resolve(devDir, '..', '..'); -} - -function belongsTo(instance: InstanceInfo, scope: ServiceInstanceScope): boolean { - return ( - instance.labels[DIRECTORY_LABEL] === directoryOf(scope.devDir) && - instance.labels[APP_LABEL] === scope.app - ); -} - -export function serviceAddressOf(instance: InstanceInfo): string { - return instance.labels[ADDRESS_LABEL] ?? instance.id; -} - -export async function appServiceInstances(scope: ServiceInstanceScope): Promise { - const instances = await computeClient().listInstances(); - return instances.filter((instance) => belongsTo(instance, scope)); -} - -/** The instance that runs `address`, created when there is none. */ -export async function ensureServiceInstance( - scope: ServiceInstanceScope, - address: string, -): Promise { - const existing = (await appServiceInstances(scope)).find( - (instance) => serviceAddressOf(instance) === address, - ); - if (existing) return existing; - return computeClient().createInstance({ - labels: { - [DIRECTORY_LABEL]: directoryOf(scope.devDir), - [APP_LABEL]: scope.app, - [ADDRESS_LABEL]: address, - }, - }); -} - -export async function deleteAppServiceInstances(scope: ServiceInstanceScope): Promise { - const client = computeClient(); - for (const instance of await appServiceInstances(scope)) { - await client.deleteInstance(instance.id); - } -} - -/** - * The emulator's failure messages name instances by id; `prisma dev` names - * services by address. Rewrites a plain `Error`'s message and leaves any other - * error, such as a structured one, untouched. - */ -export function withServiceAddresses(error: unknown, instances: readonly InstanceInfo[]): unknown { - if (!(error instanceof Error) || error.constructor !== Error) return error; - let message = error.message; - for (const instance of instances) { - message = message.replaceAll( - `instance "${instance.id}"`, - `service "${serviceAddressOf(instance)}"`, - ); - } - return message === error.message ? error : new Error(message, { cause: error }); -} diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts index 446fcb2c8..209fe5b8e 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts @@ -20,8 +20,15 @@ import * as Predicate from 'effect/Predicate'; import * as Redacted from 'effect/Redacted'; import { appNameOf } from './app-name.ts'; import { extractComputeArtifact } from './artifact-extract.ts'; -import { ensureServiceInstance, withServiceAddresses } from './compute-instances.ts'; import { envStore, secretsStore } from './dev-store.ts'; +import { + type AppScope, + deleteComputeInstance, + ensureComputeInstance, + liveComputeInstance, + type PortChange, + withServiceAddresses, +} from './emulator-instances.ts'; import { DEV_TIMESTAMP, projectIdOfInput } from './upstream-attributes.ts'; /** Reads an app id from upstream's `app` input: a plain string or a resolved `Prisma.App` attributes record. */ @@ -163,11 +170,20 @@ async function materializeEnv( return env; } +function appScopeOf(input: LocalTargetProvidersInput): AppScope { + return { devDir: input.devDir, app: appNameOf(input.container) }; +} + +function localUrl(port: number): string { + return `http://localhost:${String(port)}`; +} + /** * `Prisma.App` → the Compute emulator: finds or creates the instance that * runs the service, which holds the service's port. The app id here IS the - * service's own address. `delete` is a no-op — instance removal belongs to - * `teardown`, not per-resource Alchemy deletes. + * service's own address. An App whose instance was re-created on another + * port is updated, so its dependents get the new URL. Deleting an App (its + * service was removed) deletes its instance. */ export function LocalAppProvider( input: LocalTargetProvidersInput, @@ -177,9 +193,9 @@ export function LocalAppProvider( reconcile: ({ id, news }) => Effect.tryPromise({ try: async () => { - const scope = { devDir: input.devDir, app: appNameOf(input.container) }; const name = news.displayName ?? id; - const { url } = await ensureServiceInstance(scope, name); + const { instance } = await ensureComputeInstance(appScopeOf(input), name); + const url = localUrl(instance.port); return { appId: name, name, @@ -193,7 +209,21 @@ export function LocalAppProvider( }, catch: (cause) => cause, }), - delete: () => Effect.void, + diff: ({ output }) => + Effect.tryPromise({ + try: async () => { + if (output === undefined) return undefined; + const live = await liveComputeInstance(appScopeOf(input), output.name); + if (live?.url === output.appEndpointDomain) return undefined; + return { action: 'update' as const }; + }, + catch: (cause) => cause, + }), + delete: ({ output }) => + Effect.tryPromise({ + try: () => deleteComputeInstance(appScopeOf(input), output.name), + catch: (cause) => cause, + }), read: ({ output }) => Effect.succeed(output), }; return Provider.effect(App, Effect.succeed(service)); @@ -204,17 +234,20 @@ export function LocalAppProvider( * before Alchemy applies the `App` resources concurrently. A new instance gets * the smallest free port, so on a fresh start the ports follow this order; a * service that already has an instance keeps its port. `serviceAppNames` are - * the `App` resources' `displayName`s. + * the `App` resources' `displayName`s. Returns the services whose instance had + * to be re-created and could not get its remembered port back. */ export async function reserveServicePorts( container: ContainerInstance | undefined, devDir: string, serviceAppNames: readonly string[], -): Promise { +): Promise { const scope = { devDir, app: appNameOf(container) }; + const changes: PortChange[] = []; for (const service of serviceAppNames) { try { - await ensureServiceInstance(scope, service); + const { portChange } = await ensureComputeInstance(scope, service); + if (portChange) changes.push(portChange); } catch (cause) { const reason = cause instanceof Error ? cause.message : String(cause); throw new Error(`reserving a local port for service "${service}" failed: ${reason}`, { @@ -222,6 +255,7 @@ export async function reserveServicePorts( }); } } + return changes; } /** `Prisma.EnvironmentVariable` → a key/value row in `/env.json`. Upstream's value is `Redacted`; env.json holds the plain string the child process is given. */ @@ -284,7 +318,6 @@ export function LocalDeploymentProvider( reconcile: ({ news }) => Effect.tryPromise({ try: async (): Promise => { - const scope = { devDir: input.devDir, app: appNameOf(input.container) }; const appId = appIdOfInput(news.app); if (news.artifactPath === undefined) { throw new Error('local Deployment requires an artifactPath — nothing to run.'); @@ -297,16 +330,15 @@ export function LocalDeploymentProvider( } const address = readManifestAddress(artifactDir); - const instance = await ensureServiceInstance(scope, appId); - const { port } = instance; - const env = await materializeEnv(input.devDir, address, port); + const { instance } = await ensureComputeInstance(appScopeOf(input), appId); + const env = await materializeEnv(input.devDir, address, instance.port); try { await computeClient().putDeployment(instance.id, { artifactDir, artifactHash, env }); } catch (error) { - throw withServiceAddresses(error, [instance]); + throw withServiceAddresses(error, [{ address, instance }]); } - const url = `http://localhost:${port}`; + const url = localUrl(instance.port); return { deploymentId: artifactHash, appId, @@ -323,8 +355,21 @@ export function LocalDeploymentProvider( }, catch: (cause) => cause, }), + // An instance that no longer runs this deployment — re-created after the + // emulator lost it, or never deployed — needs the deployment again, even + // when nothing in the app changed. + diff: ({ output }) => + Effect.tryPromise({ + try: async () => { + if (output === undefined) return undefined; + const live = await liveComputeInstance(appScopeOf(input), output.appId); + if (live?.artifactHash === output.artifactHash) return undefined; + return { action: 'update' as const }; + }, + catch: (cause) => cause, + }), // Content-addressed, cheap to leave unpacked; `--fresh` removes the whole - // dev dir and `teardown` removes the emulator's instance. + // dev dir, and deleting the App deletes the emulator's instance. delete: () => Effect.void, }; return Provider.effect(Deployment, Effect.succeed(service)); diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/dev-store.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/dev-store.ts index cc05e0504..6cb297d1e 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/dev-store.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/dev-store.ts @@ -80,7 +80,7 @@ export interface DevStore { update(mutate: (current: T) => T): Promise; } -function createStore( +export function createStore( filePath: string, isValid: (value: unknown) => value is T, empty: T, diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/emulator-instances.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/emulator-instances.ts new file mode 100644 index 000000000..b35a4c204 --- /dev/null +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/emulator-instances.ts @@ -0,0 +1,190 @@ +/** + * Which emulator instance backs each node of the app run from a directory. + * `prisma dev` records it in `/.prisma-composer/dev/`, and every + * Composer operation on an emulator instance goes through it. The emulators + * know nothing about Composer: the labels written on create are for people + * reading an emulator listing, and nothing here reads them back. + */ +import * as path from 'node:path'; +import { DEV_DIR } from '@internal/core/config'; +import { computeClient, type InstanceInfo } from '@internal/dev-emulators'; +import { createStore, type DevStore } from './dev-store.ts'; + +export const EMULATOR_INSTANCES_FILE = 'emulator-instances.json'; + +export interface ComputeNodeInstance { + readonly kind: 'compute'; + readonly id: string; + /** The port the instance was given, requested again if the instance has to be re-created. */ + readonly port: number; +} + +export type NodeInstance = ComputeNodeInstance; + +export interface EmulatorInstances { + readonly app: string; + /** Keyed by node address. */ + readonly nodes: Readonly>; +} + +/** The app run from the directory that holds `devDir` (`/.prisma-composer/dev`). */ +export interface AppScope { + readonly devDir: string; + readonly app: string; +} + +/** A service whose remembered port another instance or process now holds. */ +export interface PortChange { + readonly address: string; + readonly previous: number; + readonly given: number; +} + +function isNodeInstance(value: unknown): value is NodeInstance { + return ( + typeof value === 'object' && + value !== null && + 'kind' in value && + value.kind === 'compute' && + 'id' in value && + typeof value.id === 'string' && + 'port' in value && + typeof value.port === 'number' + ); +} + +function isEmulatorInstances(value: unknown): value is EmulatorInstances { + return ( + typeof value === 'object' && + value !== null && + 'app' in value && + typeof value.app === 'string' && + 'nodes' in value && + typeof value.nodes === 'object' && + value.nodes !== null && + Object.values(value.nodes).every(isNodeInstance) + ); +} + +function directoryOf(devDir: string): string { + const normalized = path.normalize(devDir); + return normalized.endsWith(DEV_DIR) ? normalized.slice(0, -DEV_DIR.length - 1) : normalized; +} + +function store(devDir: string): DevStore { + return createStore(path.join(devDir, EMULATOR_INSTANCES_FILE), isEmulatorInstances, undefined); +} + +/** The record `prisma dev` left in `directory`; undefined when it has not run there. */ +export function readEmulatorInstances(directory: string): Promise { + return store(path.join(directory, DEV_DIR)).read(); +} + +/** This app's nodes. A record left by another app run from the same directory counts as empty. */ +async function appNodes(scope: AppScope): Promise>> { + const current = await store(scope.devDir).read(); + return current?.app === scope.app ? current.nodes : {}; +} + +async function updateNodes( + scope: AppScope, + mutate: (nodes: Record) => void, +): Promise { + await store(scope.devDir).update((current) => { + const nodes = { ...(current?.app === scope.app ? current.nodes : {}) }; + mutate(nodes); + return { app: scope.app, nodes }; + }); +} + +/** Each compute node of the app, with the instance that backs it. */ +export async function computeNodeInstances( + scope: AppScope, +): Promise<{ readonly address: string; readonly instance: ComputeNodeInstance }[]> { + return Object.entries(await appNodes(scope)).map(([address, instance]) => ({ + address, + instance, + })); +} + +/** The compute instance recorded for `address`; undefined when there is none. */ +export async function mappedComputeInstance( + scope: AppScope, + address: string, +): Promise { + return (await appNodes(scope))[address]; +} + +/** + * The instance that runs the service at `address`. The recorded instance is + * reused while the Compute emulator still has it. Otherwise a new one is + * created, requesting the recorded port, and recorded. + */ +export async function ensureComputeInstance( + scope: AppScope, + address: string, +): Promise<{ readonly instance: ComputeNodeInstance; readonly portChange?: PortChange }> { + const client = computeClient(); + const recorded = await mappedComputeInstance(scope, address); + if (recorded && (await client.getInstance(recorded.id))) return { instance: recorded }; + + const created = await client.createInstance({ + labels: { + 'composer.directory': directoryOf(scope.devDir), + 'composer.app': scope.app, + 'composer.address': address, + }, + ...(recorded ? { port: recorded.port } : {}), + }); + const instance: ComputeNodeInstance = { kind: 'compute', id: created.id, port: created.port }; + await updateNodes(scope, (nodes) => { + nodes[address] = instance; + }); + if (recorded && recorded.port !== created.port) { + return { instance, portChange: { address, previous: recorded.port, given: created.port } }; + } + return { instance }; +} + +/** The emulator's view of the instance recorded for `address`; undefined when either is missing. */ +export async function liveComputeInstance( + scope: AppScope, + address: string, +): Promise { + const recorded = await mappedComputeInstance(scope, address); + return recorded ? computeClient().getInstance(recorded.id) : undefined; +} + +/** Deletes the instance recorded for `address` and forgets it. */ +export async function deleteComputeInstance(scope: AppScope, address: string): Promise { + const recorded = await mappedComputeInstance(scope, address); + if (!recorded) return; + await computeClient().deleteInstance(recorded.id); + await updateNodes(scope, (nodes) => { + delete nodes[address]; + }); +} + +/** Deletes every compute instance recorded for the app and forgets them. */ +export async function deleteComputeInstances(scope: AppScope): Promise { + for (const { address } of await computeNodeInstances(scope)) { + await deleteComputeInstance(scope, address); + } +} + +/** + * The Compute emulator's failure messages name instances by id; `prisma dev` + * names services by address. Rewrites a plain `Error`'s message and leaves + * any other error, such as a structured one, untouched. + */ +export function withServiceAddresses( + error: unknown, + nodes: readonly { readonly address: string; readonly instance: { readonly id: string } }[], +): unknown { + if (!(error instanceof Error) || error.constructor !== Error) return error; + let message = error.message; + for (const { address, instance } of nodes) { + message = message.replaceAll(`instance "${instance.id}"`, `service "${address}"`); + } + return message === error.message ? error : new Error(message, { cause: error }); +} diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/exports/index.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/exports/index.ts index d03e76ab1..5d7f97127 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/exports/index.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/exports/index.ts @@ -2,8 +2,8 @@ export * from '../app-name.ts'; export * from '../bucket.ts'; export * from '../compute.ts'; -export * from '../compute-instances.ts'; export * from '../dev-store.ts'; +export * from '../emulator-instances.ts'; export * from '../postgres.ts'; export * from '../providers.ts'; export * from '../resolve-package-entry.ts'; diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts index 916e4ee93..5f3ce21cc 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts @@ -1,4 +1,4 @@ -import { beforeEach, expect, mock, test } from 'bun:test'; +import { afterEach, beforeEach, expect, mock, spyOn, test } from 'bun:test'; import type { Contract } from '@internal/core'; import { dependency, Load, module, string } from '@internal/core'; import type { ContainerInstance } from '@internal/core/config'; @@ -7,6 +7,7 @@ import * as RealLocalTarget from '@internal/local-target'; import { compute, rawPostgres } from '../exports/index.ts'; const events: string[] = []; +let portChanges: { address: string; previous: number; given: number }[] = []; mock.module('@internal/dev-emulators', () => ({ ...RealDevEmulators, @@ -25,6 +26,7 @@ mock.module('@internal/local-target', () => ({ addresses: readonly string[], ) => { events.push(`reserve ${container?.input.appName ?? '?'}: ${addresses.join(', ')}`); + return portChanges; }, })); @@ -54,8 +56,19 @@ const container: ContainerInstance = { serialize: () => 'x', }; +const logged: string[] = []; +const consoleLog = spyOn(console, 'log').mockImplementation((line: unknown) => { + logged.push(String(line)); +}); + beforeEach(() => { events.length = 0; + logged.length = 0; + portChanges = []; +}); + +afterEach(() => { + consoleLog.mockClear(); }); test('reserves the getting-started services in dependency order once the compute emulator is up', async () => { @@ -88,3 +101,18 @@ test('reserves only services, in declaration order when nothing orders them', as expect(events).toEqual(['daemon compute', 'reserve my-app: web, admin', 'daemon postgres']); }); + +test('prints one line for each service that could not get its remembered port back', async () => { + portChanges = [{ address: 'gateway', previous: 3001, given: 3004 }]; + const graph = Load( + module('my-app', ({ provision }) => { + provision(compute({ name: 'gateway', deps: {}, build })); + }), + ); + + await runDevEmulators({ graph, container, devDir: '/dev/null/unused' }); + + expect(logged.filter((line) => line.includes('port'))).toEqual([ + '[dev] service "gateway" now uses port 3004: its previous port 3001 is in use', + ]); +}); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-teardown.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-teardown.test.ts index 1e7924d16..5beaba25a 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-teardown.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-teardown.test.ts @@ -25,7 +25,7 @@ mock.module('@internal/dev-emulators', () => ({ mock.module('@internal/local-target', () => ({ ...RealLocalTarget, resolvePackageEntry: (specifier: string) => specifier, - deleteAppServiceInstances: async ({ app }: { app: string }) => { + deleteComputeInstances: async ({ app }: { app: string }) => { events.push(`delete compute ${app}`); if (computeDeleteFails) throw new Error('compute delete failed'); }, diff --git a/packages/1-prisma-cloud/1-extensions/target/src/local-target/attach.ts b/packages/1-prisma-cloud/1-extensions/target/src/local-target/attach.ts index 3c60d12bc..5e2e81df6 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/local-target/attach.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/local-target/attach.ts @@ -6,12 +6,7 @@ */ import type { LocalTargetAttachInput, LocalTargetAttachment } from '@internal/core/config'; import { computeClient } from '@internal/dev-emulators'; -import { - appServiceInstances, - type ServiceInstanceScope, - serviceAddressOf, - withServiceAddresses, -} from '@internal/local-target'; +import { type AppScope, computeNodeInstances, withServiceAddresses } from '@internal/local-target'; import { prismaCloudContainerOf } from '../container.ts'; const RELIST_INTERVAL_MS = 2000; @@ -29,7 +24,7 @@ interface LogLine { * dead session). */ async function* mergedLogs( - scope: ServiceInstanceScope, + scope: AppScope, signal: AbortSignal, tail: number, ): AsyncIterable { @@ -77,8 +72,8 @@ async function* mergedLogs( // Log lines are labelled with the service's address — the identity the // front door prints, so a service's endpoint line and its log lines name // it identically. - for (const instance of await appServiceInstances(scope)) { - follow(instance.id, serviceAddressOf(instance)); + for (const { address, instance } of await computeNodeInstances(scope)) { + follow(instance.id, address); } }; @@ -115,24 +110,24 @@ export async function devAttach(input: LocalTargetAttachInput): Promise { - const instances = await appServiceInstances(scope); + const nodes = await computeNodeInstances(scope); try { - await client.startInstances(instances.map((instance) => instance.id)); + await client.startInstances(nodes.map(({ instance }) => instance.id)); } catch (error) { - throw withServiceAddresses(error, instances); + throw withServiceAddresses(error, nodes); } }, endpoints: async () => { - const instances = await appServiceInstances(scope); - return instances.map((instance) => ({ - address: serviceAddressOf(instance), - url: instance.url, + const nodes = await computeNodeInstances(scope); + return nodes.map(({ address, instance }) => ({ + address, + url: `http://localhost:${String(instance.port)}`, })); }, logs: (signal, opts) => mergedLogs(scope, signal, opts?.tail ?? 0), stopServices: async () => { - const instances = await appServiceInstances(scope); - await client.stopInstances(instances.map((instance) => instance.id)); + const nodes = await computeNodeInstances(scope); + await client.stopInstances(nodes.map(({ instance }) => instance.id)); }, }; } diff --git a/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts b/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts index 8b3fbdc19..4d09082b0 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts @@ -50,7 +50,16 @@ function serviceAppNames(input: LocalTargetEmulatorsInput): string[] { export async function runDevEmulators(input: LocalTargetEmulatorsInput): Promise { const { url: computeUrl } = await ensureDaemon('compute', daemonEntry('compute')); console.log(`[dev] compute emulator ready at ${computeUrl}`); - await reserveServicePorts(input.container, input.devDir, serviceAppNames(input)); + const portChanges = await reserveServicePorts( + input.container, + input.devDir, + serviceAppNames(input), + ); + for (const { address, previous, given } of portChanges) { + console.log( + `[dev] service "${address}" now uses port ${String(given)}: its previous port ${String(previous)} is in use`, + ); + } if (usesBuckets(input)) { const { url: bucketsUrl } = await ensureDaemon('buckets', daemonEntry('buckets')); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/local-target/teardown.ts b/packages/1-prisma-cloud/1-extensions/target/src/local-target/teardown.ts index e24c9c492..e90d26b66 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/local-target/teardown.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/local-target/teardown.ts @@ -19,7 +19,7 @@ import type { TeardownInput } from '@internal/core/config'; import { DEV_DIR } from '@internal/core/config'; import { bucketsClient, ensureDaemon, postgresClient } from '@internal/dev-emulators'; -import { deleteAppServiceInstances, removeLocalPaths } from '@internal/local-target'; +import { deleteComputeInstances, removeLocalPaths } from '@internal/local-target'; import { prismaCloudContainerOf } from '../container.ts'; import { daemonEntry } from './daemon-entry.ts'; @@ -43,7 +43,7 @@ export async function runDevTeardown(input: TeardownInput): Promise { // daemon would hand the old ports back; start it so the delete always lands. try { await ensureDaemon('compute', daemonEntry('compute')); - await deleteAppServiceInstances({ devDir, app }); + await deleteComputeInstances({ devDir, app }); } finally { await tolerateUnreachable(() => bucketsClient().deleteApp(app)); removeLocalPaths([devDir, `${cwd}/.alchemy/state/${app}/dev`]); diff --git a/test/integration/test/local-dev-store.integration.ts b/test/integration/test/local-dev-store.integration.ts index 5193fa510..d121e5028 100644 --- a/test/integration/test/local-dev-store.integration.ts +++ b/test/integration/test/local-dev-store.integration.ts @@ -240,16 +240,22 @@ interface ServiceInfo { readonly pid?: number; } -/** The documented Compute-emulator wire protocol (spec § 2 compute-main.ts) — plain fetch, never the typed client. Finds the app's instances by the labels Composer gives them. */ +/** The services `prisma dev` recorded in storeDir's `.prisma-composer/dev/emulator-instances.json`, as the Compute emulator reports them (plain fetch, never the typed client). */ async function listComputeServices(app: string): Promise { const entry = readEmulatorEntry('compute'); if (entry === undefined) throw new Error('compute emulator registry entry not found'); - const res = await fetch(`http://127.0.0.1:${entry.port}/instances`); - if (!res.ok) throw new Error(`compute emulator listInstances failed: ${res.status}`); - const instances = (await res.json()) as (ServiceInfo & { labels: Record })[]; - return instances - .filter((instance) => instance.labels['composer.app'] === app) - .map((instance) => ({ ...instance, address: instance.labels['composer.address'] ?? '' })); + const mapping = readJson( + path.join(storeDir, '.prisma-composer', 'dev', 'emulator-instances.json'), + ) as { app: string; nodes: Record } | undefined; + if (mapping === undefined || mapping.app !== app) return []; + const services: ServiceInfo[] = []; + for (const [address, node] of Object.entries(mapping.nodes)) { + const res = await fetch(`http://127.0.0.1:${entry.port}/instances/${node.id}`); + if (res.status === 404) continue; + if (!res.ok) throw new Error(`compute emulator getInstance failed: ${res.status}`); + services.push({ ...((await res.json()) as ServiceInfo), address }); + } + return services; } interface PostgresDatabaseInfo { diff --git a/test/integration/test/local-dev.integration.ts b/test/integration/test/local-dev.integration.ts index 98331bcdc..2bdd2b929 100644 --- a/test/integration/test/local-dev.integration.ts +++ b/test/integration/test/local-dev.integration.ts @@ -299,16 +299,22 @@ interface ServiceInfo { readonly pid?: number; } -/** The documented Compute-emulator wire protocol (spec § 2 compute-main.ts) — plain fetch, never the typed client. Finds the app's instances by the labels Composer gives them. */ +/** The services `prisma dev` recorded in fixtureDir's `.prisma-composer/dev/emulator-instances.json`, as the Compute emulator reports them (plain fetch, never the typed client). */ async function listComputeServices(): Promise { const entry = readEmulatorEntry('compute'); if (entry === undefined) throw new Error('compute emulator registry entry not found'); - const res = await fetch(`http://127.0.0.1:${entry.port}/instances`); - if (!res.ok) throw new Error(`compute emulator listInstances failed: ${res.status}`); - const instances = (await res.json()) as (ServiceInfo & { labels: Record })[]; - return instances - .filter((instance) => instance.labels['composer.app'] === APP_NAME) - .map((instance) => ({ ...instance, address: instance.labels['composer.address'] ?? '' })); + const mapping = readJson( + path.join(fixtureDir, '.prisma-composer', 'dev', 'emulator-instances.json'), + ) as { app: string; nodes: Record } | undefined; + if (mapping === undefined || mapping.app !== APP_NAME) return []; + const services: ServiceInfo[] = []; + for (const [address, node] of Object.entries(mapping.nodes)) { + const res = await fetch(`http://127.0.0.1:${entry.port}/instances/${node.id}`); + if (res.status === 404) continue; + if (!res.ok) throw new Error(`compute emulator getInstance failed: ${res.status}`); + services.push({ ...((await res.json()) as ServiceInfo), address }); + } + return services; } /** From 2c1aaa01b6bdb0c1b4322176ef669ebe232a85a2 Mon Sep 17 00:00:00 2001 From: willbot Date: Tue, 13 Oct 2026 16:48:09 +0200 Subject: [PATCH 06/17] feat(local-target): switching apps in a directory removes the previous app's services A directory has one dev instance. When `prisma dev` runs a different app from a directory than the one its emulator-instances.json names, the emulators hook deletes the previous app's Compute instances, which hold no data, starts a new record, and prints one line naming the app whose local services were removed. Without this, every switch left the previous app's instances holding their ports. Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- .../src/__tests__/emulator-instances.test.ts | 24 +++++++++++++++++++ .../local-target/src/emulator-instances.ts | 14 +++++++++++ .../__tests__/local-target-emulators.test.ts | 18 ++++++++++++++ .../target/src/local-target/emulators.ts | 9 ++++++- 4 files changed, 64 insertions(+), 1 deletion(-) diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts index aef280287..aa556dfd2 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts @@ -22,6 +22,7 @@ import { EMULATOR_INSTANCES_FILE, type EmulatorInstances, readEmulatorInstances, + retirePreviousApp, } from '../emulator-instances.ts'; const APP = 'shared-app'; @@ -249,3 +250,26 @@ test('the record is one JSON file in the dev directory, readable from the direct expect(onDisk).toEqual({ app: APP, nodes: { quotes: node } }); expect(await readEmulatorInstances(path.join(root, 'never-ran'))).toBeUndefined(); }); + +describe('switching apps in a directory', () => { + test("running another app deletes the previous app's compute instances and starts a new record", async () => { + const { directory, devDir } = projectDir('switch'); + await reserveServicePorts(container, devDir, ADDRESSES); + const previous = Object.values((await mappingOf(directory))?.nodes ?? {}); + + const retired = await retirePreviousApp({ devDir, app: 'another-app' }); + + expect(retired).toBe(APP); + for (const node of previous) expect(await computeClient().getInstance(node.id)).toBeUndefined(); + expect(await mappingOf(directory)).toEqual({ app: 'another-app', nodes: {} }); + }); + + test('running the same app again retires nothing', async () => { + const { directory, devDir } = projectDir('same'); + await reserveServicePorts(container, devDir, ['quotes']); + const quotes = await nodeOf(directory, 'quotes'); + + expect(await retirePreviousApp({ devDir, app: APP })).toBeUndefined(); + expect(await computeClient().getInstance(quotes.id)).toBeDefined(); + }); +}); diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/emulator-instances.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/emulator-instances.ts index b35a4c204..f716da4a1 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/emulator-instances.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/emulator-instances.ts @@ -97,6 +97,20 @@ async function updateNodes( }); } +/** + * A directory has one dev instance. When the record names another app, that + * app's compute instances are deleted and the record starts over for this + * one. Returns the previous app's name when it was retired. + */ +export async function retirePreviousApp(scope: AppScope): Promise { + const current = await store(scope.devDir).read(); + if (current === undefined || current.app === scope.app) return undefined; + const client = computeClient(); + for (const instance of Object.values(current.nodes)) await client.deleteInstance(instance.id); + await store(scope.devDir).update(() => ({ app: scope.app, nodes: {} })); + return current.app; +} + /** Each compute node of the app, with the instance that backs it. */ export async function computeNodeInstances( scope: AppScope, diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts index 5f3ce21cc..9c3162be7 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts @@ -8,6 +8,7 @@ import { compute, rawPostgres } from '../exports/index.ts'; const events: string[] = []; let portChanges: { address: string; previous: number; given: number }[] = []; +let retiredApp: string | undefined; mock.module('@internal/dev-emulators', () => ({ ...RealDevEmulators, @@ -20,6 +21,7 @@ mock.module('@internal/dev-emulators', () => ({ mock.module('@internal/local-target', () => ({ ...RealLocalTarget, resolvePackageEntry: (specifier: string) => specifier, + retirePreviousApp: async () => retiredApp, reserveServicePorts: async ( container: ContainerInstance | undefined, _devDir: string, @@ -65,6 +67,7 @@ beforeEach(() => { events.length = 0; logged.length = 0; portChanges = []; + retiredApp = undefined; }); afterEach(() => { @@ -116,3 +119,18 @@ test('prints one line for each service that could not get its remembered port ba '[dev] service "gateway" now uses port 3004: its previous port 3001 is in use', ]); }); + +test("prints one line when the directory's previous app's local services were removed", async () => { + retiredApp = 'old-app'; + const graph = Load( + module('my-app', ({ provision }) => { + provision(compute({ name: 'web', deps: {}, build })); + }), + ); + + await runDevEmulators({ graph, container, devDir: '/dev/null/unused' }); + + expect(logged.filter((line) => line.includes('removed'))).toEqual([ + '[dev] removed the local services of "old-app", the app run here before', + ]); +}); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts b/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts index 4d09082b0..ed279c283 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts @@ -24,7 +24,7 @@ */ import type { LocalTargetEmulatorsInput } from '@internal/core/config'; import { ensureDaemon } from '@internal/dev-emulators'; -import { reserveServicePorts } from '@internal/local-target'; +import { appNameOf, reserveServicePorts, retirePreviousApp } from '@internal/local-target'; import { PRISMA_CLOUD_EXTENSION_ID } from '../container.ts'; import { serviceAppName } from '../service-app.ts'; import { daemonEntry } from './daemon-entry.ts'; @@ -50,6 +50,13 @@ function serviceAppNames(input: LocalTargetEmulatorsInput): string[] { export async function runDevEmulators(input: LocalTargetEmulatorsInput): Promise { const { url: computeUrl } = await ensureDaemon('compute', daemonEntry('compute')); console.log(`[dev] compute emulator ready at ${computeUrl}`); + const retired = await retirePreviousApp({ + devDir: input.devDir, + app: appNameOf(input.container), + }); + if (retired !== undefined) { + console.log(`[dev] removed the local services of "${retired}", the app run here before`); + } const portChanges = await reserveServicePorts( input.container, input.devDir, From 4e405a73b5b2c2d34fe66c8a90705d58e2baea52 Mon Sep 17 00:00:00 2001 From: willbot Date: Tue, 13 Oct 2026 16:51:52 +0200 Subject: [PATCH 07/17] docs(design): record that Composer maps its nodes to emulator instances per directory ADR-0052 records the model: the emulators know nothing about Composer, instances are keyed by emulator-generated ids and carry labels the emulator never reads, Composer's per-directory emulator-instances.json is the only mapping from its names to instances, an instance keeps its port until deleted, a remembered port is a preference used when an instance is re-created, and switching apps in a directory removes the previous app's services. It amends ADR-0041's app-name local identity. local-dev.md describes the Compute emulator's instances and ports, the record and how each part of dev uses it, and notes that Postgres and buckets are still keyed by the app. Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- docs/design/10-domains/local-dev.md | 125 +++++++++++++++--- ...deploy-pipeline-against-local-providers.md | 2 + ...des-to-emulator-instances-per-directory.md | 60 +++++++++ docs/design/90-decisions/README.md | 3 +- 4 files changed, 169 insertions(+), 21 deletions(-) create mode 100644 docs/design/90-decisions/ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md diff --git a/docs/design/10-domains/local-dev.md b/docs/design/10-domains/local-dev.md index 3a573eaaa..090a51730 100644 --- a/docs/design/10-domains/local-dev.md +++ b/docs/design/10-domains/local-dev.md @@ -55,13 +55,16 @@ Dev re-runs [deploy's pipeline](deploy-cli.md#the-pipeline) with these deltas: 3. **Assemble** — identical. Dev consumes the user's built output through the same adapters and produces the same bundles; missing output produces the same "run your build" error. -4. **Containers + emulators** — the `localTarget.container` descriptor resolves a - stable local identity from the app name with no platform calls. Then the +4. **Containers + emulators** — the `localTarget.container` descriptor resolves + the app's local identity, its name, with no platform calls. Then the `localTarget.emulators` hook inspects the loaded graph and ensures one emulator daemon per node kind the topology uses (Compute always; buckets when bucket resources exist; Postgres when `postgres` or `raw-postgres` - resources exist). The Postgres daemon starts no server here: the database - provider asks it for one `@prisma/dev` server per `Database` at provision. + resources exist). Once Compute is up, the hook ensures each service's + Compute instance, in `graph.nodes` order, through the directory's record + (see [Which instance backs a node](#which-instance-backs-a-node)). The + Postgres daemon starts no server here: the database provider asks it for + one `@prisma/dev` server per `Database` at provision. 5. **Lower + converge** — a dev-generated stack file (ADR-0007's pattern, at `.prisma-composer/dev/alchemy.run.ts`), driven with the extension's `localTarget.providers()` layer and `devState()` (Alchemy's built-in file store, exported by `@prisma/composer/local-target`), always @@ -81,16 +84,35 @@ The seam between converge (terminating) and serving (long-running). One emulator daemon per node kind runs machine-scoped and multi-tenant — the Compute emulator is the one the framework builds in full: a small local counterpart of the platform's compute service, with a loopback admin -endpoint. The local `Deployment` provider does not spawn processes; it +endpoint. + +The Compute emulator knows nothing about Composer +([ADR-0052](../90-decisions/ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md)). +It holds **instances**, each a slot for one service process. A client creates +an instance with free-form labels and an optional requested port; the +emulator generates the id (`svc_<16 hex digits>`), stores the labels verbatim +and never reads them. Every later call names the id: put a deployment, start +or stop a list of instances, delete one, read or follow its log, get one, or +list them all. The emulator's routes and state hold no app name, node address +or directory outside a label, and a test fails if one appears. + +**Ports.** An instance gets its port when it is created and keeps it until it +is deleted, running or stopped. A requested port is used when no instance +holds it and it can be bound; otherwise the instance gets the lowest such port +from 3000. The port is in the deployment's env (`COMPOSER_
_PORT`) and +in its dependents' URLs, and a warm start does not redeploy, so a port that +changed between sessions would leave services built for the old one. + +The local `Deployment` provider does not spawn processes; it unpacks the artifact into `.prisma-composer/dev/artifacts//`, materializes the complete child env, and puts the deployment at the -emulator, which owns the processes: +service's instance, and the emulator owns the processes: - **New or changed deployment** (artifact hash or env changed) → stop the old child if any, spawn the artifact's `bootstrap.js` under bun with the materialized env. Only the changed service restarts — Alchemy's diff already limited which deployments were re-put. -- **Readiness** → a service is `running` only once its port accepts a TCP connection on loopback (IPv4 or IPv6). The emulator probes the port every 50 ms until it does, so a service that counts connections sees a few extra ones. The deployment PUT and the session-resume start return only then, so a converge, and the `[dev] ready:` line after it, finish only when every URL answers. A redeploy of a running service sets it to `stopped` once the old process has exited, and back to `running` when the new one listens. A child that exits before it listens fails the call with its address, exit code, and log path. A child that does not listen within the readiness timeout (30 seconds; `PRISMA_COMPOSER_SERVICE_READY_TIMEOUT_MS`, which the CLI passes to the compute emulator as `--ready-timeout-ms` when it starts the emulator) is stopped, and the call fails with its address, port, log path and the variable's name. A crash-backoff restart that times out ends `held`, like other crash loops. +- **Readiness** → a service is `running` only once its port accepts a TCP connection on loopback (IPv4 or IPv6). The emulator probes the port every 50 ms until it does, so a service that counts connections sees a few extra ones. The deployment PUT and the session-resume start return only then, so a converge, and the `ready` line after it, finish only when every URL answers. A redeploy of a running service sets it to `stopped` once the old process has exited, and back to `running` when the new one listens. A child that exits before it listens fails the call with its address, exit code, and log path. A child that does not listen within the readiness timeout (30 seconds; `PRISMA_COMPOSER_SERVICE_READY_TIMEOUT_MS`, which the CLI passes to the compute emulator as `--ready-timeout-ms` when it starts the emulator) is stopped, and the call fails with its address, port, log path and the variable's name. A crash-backoff restart that times out ends `held`, like other crash loops. - **Concurrent starts and stops** → a service has at most one process and at most one start in flight; the rules are in [Concurrent starts and stops](#concurrent-starts-and-stops) below. - **Child exits unexpectedly** → the emulator restarts it with backoff and writes its supervision events into that service's log stream; repeated @@ -104,11 +126,12 @@ emulator, which owns the processes: the `log` operation's `tail` count maps straight onto it. A live follower is reset to the new start when the log is cleared under it (the daemon tracks a per-log clear generation, so a follower never reads stale bytes). -- **Instance deleted** (service removed from the topology, `--fresh`) → stop - and remove. +- **Instance deleted** (service removed from the topology, `--fresh`, or + another app run from the same directory) → stop it, remove it and its log, + and free its port. - **Ctrl-C on the dev command** → the attachment stops the app's service instances and detaches. Every emulator — Compute, Postgres instances, the - bucket emulator — stays up with its data; the next start reprovisions the + bucket emulator — stays up with its data; the next start reuses the same instances (same ports, same data). `--fresh` is what removes instances and data. (A detached mode where services keep serving with no session is a designed extension, not v1.) @@ -121,7 +144,7 @@ emulator, which owns the processes: live status, which Alchemy's state file does not track. So a second `prisma dev` after a plain Ctrl-C can converge with everything reported "noop" and leave every previously-stopped service `stopped` — - the CLI still prints `[dev] ready:` with each service's URL, but nothing + the CLI still prints `ready` with each service's URL, but nothing is listening on them. Confirmed against the open-chat proving port (the warm-restart-noop finding); the existing store proving script's own criterion-6 check doesn't catch it because it asserts port *stability* @@ -147,7 +170,7 @@ Each service has at most one process and at most one start in flight. A stop in | Deployment PUT, identical | `running`, no start or stop in progress | Succeeds at once | | Deployment PUT | Anything else | Stores the deployment and becomes the start in flight, replacing any earlier one: waits for any stop to finish, stops the current process, then spawns | | Stop | Any | Ends the start in flight and any crash-backoff restart, stops the process, and sets `stopped` once it has exited | -| App delete | Any | Marks each service deleted, stops it, and removes it; no start on a deleted service spawns | +| Instance delete | Any | Marks the instance deleted, stops it, and removes it; no start on a deleted instance spawns | | Crash-backoff restart | After an unexpected exit | Becomes the start in flight and spawns at once | A start in flight ends with the first of these that applies: @@ -156,7 +179,7 @@ A start in flight ends with the first of these that applies: | --- | --- | | A newer start replaced it | The newer start's result | | `bun` is not on the deployment's `PATH` | Fails with how to install bun; the service is `stopped` | -| A stop or an app delete ended it | Fails: "was stopped before it accepted connections" | +| A stop or an instance delete ended it | Fails: "was stopped before it accepted connections" | | Its process exited | Fails with the exit code and the log path; crash supervision handles the exit | | The readiness timeout passed | Fails with the port, the log path and the timeout variable; the process is stopped and the service ends `stopped`, or `held` for a crash-backoff restart | | The port accepted a connection | Succeeds; the service is `running` | @@ -176,6 +199,63 @@ snapshot diffs on bytes — which restart-amplified dependents on every first-after-cold converge. No sanctioned reader consumes sibling rows, so the scoping changes restart behavior only. +The emulator's failure messages name the instance (`instance "svc_…" exited +(code 1) before it accepted connections…`). The local providers and the +attachment replace the id with the service's address before the message +reaches the user, so `prisma dev` says `service "gateway" …`. + +### Which instance backs a node + +Composer records, per working directory, which emulator instance backs each +node of the app run there, in `.prisma-composer/dev/emulator-instances.json`: + +```json +{ + "app": "shop", + "nodes": { + "quotes": { "kind": "compute", "id": "svc_0b6e2f4a91c3d758", "port": 3000 }, + "gateway": { "kind": "compute", "id": "svc_81ad5c07e2f94b31", "port": 3001 } + } +} +``` + +The record is the only place Composer's names are mapped to instances. The +emulators hook, the local `App` and `Deployment` providers, the attachment +(endpoints, logs, start, stop) and `--fresh` all go through it; no Composer +code finds an instance by its labels. The labels Composer writes on create — +`composer.directory` (the absolute directory), `composer.app` and +`composer.address` — are for people reading an emulator listing. The record +can be read from the directory alone, without running the pipeline. + +- **Ensuring an instance.** The emulators hook goes through the services in + `graph.nodes` order (dependencies first, ties in declaration order). A + recorded instance the emulator still has is reused. Otherwise a new one is + created and recorded: on a first start, with no port requested, so ports + follow that order from 3000; after the emulator lost the instance, requesting + the recorded port. A service added by a rebuild during a session gets its + instance from the `App` provider the same way. +- **A remembered port that is taken.** It is a preference: the emulator gives + the lowest free port instead, and `prisma dev` prints + `[dev] service "
" now uses port : its previous port is in use`. +- **Redeploying a re-created instance.** A re-created instance runs nothing, + and a warm start's converge changes nothing, so Alchemy would never call the + providers. The `Deployment` provider's diff asks for an update when its + instance does not run the recorded artifact, and the `App` provider's diff + asks for one when its instance's URL changed, so dependents get the new URL. +- **A removed service.** Deleting its `App` resource deletes its instance and + its entry. +- **Another app in the same directory.** A directory has one dev instance. + When the record names a different app, the emulators hook deletes that app's + Compute instances, starts a new record, and prints + `[dev] removed the local services of "", the app run here before`. +- **`--fresh`** deletes the recorded Compute instances, then removes the dev + state directory with the record in it. + +Two directories running the same app have separate records, so they get +separate instances on separate ports, and neither restarts the other's +services. State the Compute emulator wrote before instances had generated ids +is discarded when it loads; local dev state is never migrated. + ### Process lifetimes 1. **Converge-scoped** — the Alchemy child (ADR-0007), started as @@ -185,12 +265,14 @@ the scoping changes restart behavior only. multi-tenant, holding every long-lived process: the Compute emulator (service children), the Postgres emulator (one `@prisma/dev` server per `Database`, hosted in-process), and the bucket emulator. They survive dev - sessions; `--fresh` removes an app's instances. The registry root is + sessions; `--fresh` removes a directory's instances. The registry root is `~/.prisma-composer/emulators` unless `PRISMA_COMPOSER_EMULATORS_DIR` names another absolute directory, so by default one machine shares one - set. A root isolates the compute and buckets daemons and their registry, - not Postgres: Postgres instance names come from the app and database, so - two checkouts of one app still share a server. A daemon whose version + set. Compute instances belong to a directory, so two checkouts of one app + never share a service. Postgres and buckets are still keyed by the app: + Postgres instance names come from the app and database, and bucket names + from the app and bucket, so two checkouts of one app still share a + database server and bucket names, whatever their roots. A daemon whose version differs from the caller's is replaced, so two Composer versions sharing a root replace each other's daemons back and forth; a separate root per checkout stops that. @@ -214,8 +296,8 @@ semantics): | `Project` | a local identity record; no platform | | `Database` | a database on a local `@prisma/dev` Postgres server, hosted by the Postgres emulator | | `Connection` | the local connection URL | -| `App` | registers the service with the Compute emulator, which allocates its stable port; `appEndpointDomain = http://localhost:` — which makes origin (ADR-0039) work unchanged | -| `Deployment` | unpacks the artifact once per hash, materializes the env, and puts the deployment at the Compute emulator, which (re)starts the child | +| `App` | finds the service's Compute instance through the directory's record, creating it when there is none; the instance holds the service's port; `appEndpointDomain = http://localhost:` — which makes origin (ADR-0039) work unchanged. Deleting it deletes the instance | +| `Deployment` | unpacks the artifact once per hash, materializes the env, and puts the deployment at the service's Compute instance, which (re)starts the child; updated when its instance does not run the recorded artifact | | `EnvironmentVariable` | a key→value row in the dev state store | | `Bucket` | a directory under `.prisma-composer/dev/buckets//`, served by the bucket emulator | | `BucketKey` | credentials registered with the bucket emulator | @@ -334,7 +416,7 @@ Deploy's rule holds: every failure names its fix. (none outstanding from the design phase — see Known limitations below for gaps found during implementation.) -(Settled since the first draft: Postgres runs one named `@prisma/dev` server per `Database` resource; the front door prints every service URL ordered by address depth then name, shallowest first; on a fresh start, services get their ports in dependency order, ties in declaration order — the emulators hook reserves them one at a time from `graph.nodes` before Alchemy applies the `App` resources concurrently — and a warm start keeps each service's saved port; port allocation and the remaining mechanics are described in this document and [ADR-0041](../90-decisions/ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md). Restart latency is measured — see Known limitations.) +(Settled since the first draft: Postgres runs one named `@prisma/dev` server per `Database` resource; the front door prints every service URL ordered by address depth then name, shallowest first; on a fresh start, services get their ports in dependency order, ties in declaration order — the emulators hook creates their Compute instances one at a time from `graph.nodes` before Alchemy applies the `App` resources concurrently — and a warm start keeps each service's port, because an instance holds its port until it is deleted (ADR-0052); port allocation and the remaining mechanics are described in this document and [ADR-0041](../90-decisions/ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md). Restart latency is measured — see Known limitations.) ## Proven against a real app @@ -378,6 +460,9 @@ replaces. - [ADR-0041](../90-decisions/ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md) — the decision this doc details. +- [ADR-0052](../90-decisions/ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md) + — the emulators know nothing about Composer; the per-directory record of + which instance backs each node. - [deploy-cli.md](deploy-cli.md) — the pipeline dev re-runs and the error-surface convention it extends. - [alchemy-lowering.md](../05-prisma-cloud/alchemy-lowering.md) — the resource diff --git a/docs/design/90-decisions/ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md b/docs/design/90-decisions/ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md index 67f42c069..32e5b491a 100644 --- a/docs/design/90-decisions/ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md +++ b/docs/design/90-decisions/ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md @@ -2,6 +2,8 @@ Amended by [ADR-0050](ADR-0050-composer-runs-as-prisma-deploy-and-prisma-dev.md): the command is `prisma dev `, and reading the running app's logs is the `log` operation on `@prisma/composer/control`, not a command. The `prisma-composer` binary is retired. The local pipeline below is unchanged. +Amended by [ADR-0052](ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md): the emulators know nothing about Composer. A dev instance is no longer identified on the emulators by the app name; Composer records, per working directory, which emulator instance backs each node, and the Compute emulator keys its instances by ids it generates. The Postgres and bucket emulators still key instances by app name until they move to the same model. + ## Decision `prisma-composer dev ` boots the whole application locally, with no cloud diff --git a/docs/design/90-decisions/ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md b/docs/design/90-decisions/ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md new file mode 100644 index 000000000..65be0e0a1 --- /dev/null +++ b/docs/design/90-decisions/ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md @@ -0,0 +1,60 @@ +# ADR-0052: Composer maps its nodes to emulator instances per directory + +## Decision + +The local emulators know nothing about Composer. An emulator holds instances under ids it generates. Each instance carries labels that the emulator stores and returns but never reads. Composer records, per working directory, which emulator instance backs each node of the app run there. That record is the only place Composer's names are mapped to instances. + +```text +~/work/shop/.prisma-composer/dev/emulator-instances.json (written by Composer) + { "app": "shop", + "nodes": { "quotes": { "kind": "compute", "id": "svc_k3f9…", "port": 3000 }, + "gateway": { "kind": "compute", "id": "svc_81ad…", "port": 3001 } } } + +Compute emulator (one per machine, knows nothing about Composer) + GET /instances → [ { "id": "svc_k3f9…", "port": 3000, "status": "running", + "labels": { "composer.directory": "/Users/me/work/shop", + "composer.app": "shop", + "composer.address": "quotes" } }, … ] +``` + +A second checkout of the same app in `~/work/shop-2` has its own record and its own instances. The two never share a service process or a port. + +This amends [ADR-0041](ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md), whose local container gives a dev instance a stable identity from the app name alone. The Compute emulator is keyed this way. The Postgres and bucket emulators still key their instances by app name and are moved to the same model separately. + +## Reasoning + +`prisma dev` runs an app against local emulators: a Compute emulator that runs service processes, a Postgres emulator and a bucket emulator. Each is a long-lived daemon shared by everything on the machine. ADR-0041 identified an app's instances on those daemons by the app's name, so the emulators' routes and state were shaped around apps: `/apps//services/`. Two directories that run the same app, such as two git worktrees of one repository, therefore resolved to the same instances. Each `prisma dev` redeployed over the other's services. That is a common situation, not an edge case: agents work in many worktrees at once and start and stop environments in each. + +The emulators should not have to know what a directory or an app is to avoid this. So an instance belongs to whoever created it. A client creates an instance, the emulator generates its id, and every later operation names that id: deploy, start, stop, delete, logs. The emulator's API and its state contain no app name, node address or directory, and a test fails if one appears there outside a label. The emulators can then be used by something other than Composer, and moved out of this repository, without carrying Composer's model along. + +Labels make a listing readable. Composer writes `composer.directory`, `composer.app` and `composer.address` on every instance it creates, so a person looking at the emulator's list of instances can tell what started each one. Nothing reads them back as identity. If Composer found its instances by searching labels, the labels would be a second, unenforced key. + +Composer finds its instances through `emulator-instances.json` in the directory's dev state (`.prisma-composer/dev/`), which is already per directory. It records the app name and, per node address, the emulator kind, the instance id and the instance's port. The emulators hook writes it before converge. It reuses a recorded instance while the emulator still has it, and otherwise creates one and records it. Everything else reaches instances through the record: the local providers during converge, the dev session's endpoints, logs, start and stop, and `--fresh`. The record is readable from the directory alone, so a separate command can answer "where is this node running" without running the pipeline. + +An instance keeps its port from creation until it is deleted, whether it is running or stopped. A port is baked into more than the process: the service's environment carries it (`COMPOSER_
_PORT`), and its dependents' environments carry its URL. A warm start does not redeploy, because nothing in the app changed and Alchemy, the deploy engine, sees nothing to update. A service that received a different port on its next start would run with an environment built for the old one. + +The record also keeps each node's port, for the one case where the port can change: the instance has to be re-created, for example because the emulator lost its state. Composer then asks for the remembered port. A requested port is a preference: when another instance or another process holds it, the emulator gives the lowest free port from 3000, and `prisma dev` prints one line naming the service, the old port and the new one. Failing instead would break `prisma dev` in one worktree whenever another worktree happened to take the port. Because a re-created instance runs nothing, the local Deployment provider asks Alchemy for an update when its instance does not run the recorded artifact, and the local App provider does so when its instance's URL changed, so dependents get the new URL. + +A directory has one dev instance. When `prisma dev` runs a different app from a directory than the one the record names, the previous app's Compute instances are deleted, the record starts over, and `prisma dev` prints which app's local services were removed. Compute instances hold no data, so nothing is lost that a deploy does not recreate. + +## Consequences + +- Two directories running the same app run separate services on separate ports, and neither restarts the other's. +- A directory's ports are stable across `prisma dev` sessions. A port changes only when its instance is re-created and the old port is taken, and `prisma dev` says so. +- A stopped instance holds its port. Instances left behind by a deleted worktree keep their ports until something deletes them. No command does that yet. +- Local dev state written before this change is discarded: the Compute emulator drops app-keyed state when it loads, and the first `prisma dev` creates new instances. Local dev state is never migrated across versions. +- Every emulator kind moves to this model: instances keyed by emulator-generated ids, labels stored but never read, and a list of all instances. Until the Postgres and bucket emulators do, the app name still identifies their instances, so two directories running one app still share its databases and buckets. +- Composer code must not find an instance by its labels. The record is the only mapping. + +## Alternatives considered + +- **App-keyed emulators.** The model this replaces. Simple, but two directories running one app share and overwrite each other's services, and the emulators carry Composer's concepts in their routes and state. +- **The app name plus a hash of the directory as the identity.** Separates directories, but keeps app and directory concepts inside the emulators and makes every emulator route depend on how Composer forms that key. +- **Releasing a port when its instance stops.** Lets stopped worktrees hold no ports, but breaks warm starts: the port is in the deployment's environment and in its dependents' URLs, and a warm start does not redeploy, so a service started on a new port would run with an environment built for the old one. +- **Finding instances by their labels.** Needs no record file, but makes free-form display metadata into an unenforced key that any client could collide with. + +## Related + +- [ADR-0041](ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md) — local dev runs the deploy pipeline against local providers; amended here. +- [ADR-0051](ADR-0051-a-nodes-address-is-its-logical-id-on-the-platform.md) — a node's address, the key of the record. +- [local-dev.md](../10-domains/local-dev.md) — the Compute emulator, the record and port allocation. diff --git a/docs/design/90-decisions/README.md b/docs/design/90-decisions/README.md index 9eeac2420..e2f8d7bf6 100644 --- a/docs/design/90-decisions/README.md +++ b/docs/design/90-decisions/README.md @@ -62,7 +62,7 @@ _Earlier drafts (ADR-0001, ADR-0002) were retired as the high-level design settl - [ADR-0038](ADR-0038-containers-are-an-extension-descriptor.md) — Container lifecycle (ensure/locate/remove) is an optional `container` descriptor on `ExtensionDescriptor`, the same pattern as `preflight`/`teardown`; the resolved instance is opaque to core and crosses the CLI parent→alchemy child boundary as one framework-named environment variable per extension, via the extension's own `serialize()`/`deserialize()`. `StateDescriptor` names its owning extension so core can hand it that extension's resolved container. Deletes the `crossDomainExceptions` entry that let the CLI import `@internal/lowering` directly — `0-framework` imports nothing again. - [ADR-0039](ADR-0039-a-compute-services-own-origin-is-a-target-resolved-property.md) — A compute service's own platform-assigned origin is a target-resolved property, read as the service's `origin()` — never a declared param, never operator config, never in `config()`. It rides ADR-0031's reserved provider-param channel as the first *service-derived* entry (`valueForService(provisioned, address)`, written for every compute service, exposing or not), sourced from the provisioned App's `appEndpointDomain` (ADR-0048) — made trustworthy pre-promote by the upstream PRO-200 fix. `envParam(…)` remains correct for operator-known origins (custom domains); narrows ADR-0032's `appOrigin` example accordingly. - [ADR-0040](ADR-0040-the-orm-binding-carries-the-url-and-a-lazy-client.md) — `postgres(contract)`'s dependency binding is `{ url, client }`: the raw connection string plus the typed client, constructed lazily and memoized on first `client` access — `hydrate` builds nothing. The contract remains the compatibility interface (hash check and deploy-time migration unchanged, ADR-0022); the binding becomes a strict superset of plain `rawPostgres()`'s `{ url }`, so an app that owns its database client still gets framework-run migrations. Contract validation cost and failure move from `load()` (where one bad input poisoned every input, unattributed) to the first `client` access. Cross-kind satisfaction (`'postgres'` satisfying `'postgres'`) rejected in its favor. -- [ADR-0041](ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md) — `prisma-composer dev` runs the **same deploy pipeline** (Load → assemble → lower → Alchemy converge) against local implementations of the same Alchemy resource types, declared on an optional `localTarget` field of `ExtensionDescriptor` (a lazy thunk resolving a `LocalTargetDescriptor`; subpaths `@prisma/composer/local-target` and `@prisma/composer-prisma-cloud/local-target` — "dev" names only the user-facing command/prefix/state dir) (providers, container, preflight, emulators, attach, teardown — **no** `nodes`/`provisions`, so the lowering cannot diverge; no `state` either — dev uses Alchemy's own `localState()` through `LowerOptions.state`). The target runs **emulators per node kind**: Compute and buckets are machine-global, multi-tenant daemons (the Compute emulator owns the service child processes — deployment PUTs, crash supervision, logs; buckets serve the S3 wire over plain files on disk), while Postgres runs one detached ORM `prisma dev` instance per `Database` resource under the ORM CLI's own manager. Providers provision instances by communicating with the emulators during converge, and the dev command is a view through `attach`; `ServiceKey`/`S3Credentials`/`PgWarm`/`OrmMigration` are shared verbatim. Credential-free by requirement. Rejects a local Management API (reimplements another team's server-side semantics, drifts silently) and per-kind dev descriptors (an open-set parallel seam). *(Amended by ADR-0050: the command is `prisma dev `; logs are the `log` operation, not a command.)* +- [ADR-0041](ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md) — `prisma-composer dev` runs the **same deploy pipeline** (Load → assemble → lower → Alchemy converge) against local implementations of the same Alchemy resource types, declared on an optional `localTarget` field of `ExtensionDescriptor` (a lazy thunk resolving a `LocalTargetDescriptor`; subpaths `@prisma/composer/local-target` and `@prisma/composer-prisma-cloud/local-target` — "dev" names only the user-facing command/prefix/state dir) (providers, container, preflight, emulators, attach, teardown — **no** `nodes`/`provisions`, so the lowering cannot diverge; no `state` either — dev uses Alchemy's own `localState()` through `LowerOptions.state`). The target runs **emulators per node kind**: Compute and buckets are machine-global, multi-tenant daemons (the Compute emulator owns the service child processes — deployment PUTs, crash supervision, logs; buckets serve the S3 wire over plain files on disk), while Postgres runs one detached ORM `prisma dev` instance per `Database` resource under the ORM CLI's own manager. Providers provision instances by communicating with the emulators during converge, and the dev command is a view through `attach`; `ServiceKey`/`S3Credentials`/`PgWarm`/`OrmMigration` are shared verbatim. Credential-free by requirement. Rejects a local Management API (reimplements another team's server-side semantics, drifts silently) and per-kind dev descriptors (an open-set parallel seam). *(Amended by ADR-0050: the command is `prisma dev `; logs are the `log` operation, not a command. Amended by ADR-0052: emulator instances are keyed by emulator-generated ids, not the app name, and Composer maps its nodes to them per directory.)* - [ADR-0042](ADR-0042-service-input-is-one-standard-schema.md) — A compute service declares its entire incoming configuration — config and secrets together — as one Standard Schema (`input`), read back through one typed accessor; `params`/`secrets` and `config()`/`secrets()` are replaced. The framework never introspects the schema (validate-only, per the spec): the operator's binding is the traversable structure (sourcing: literals, `envParam`, `envSecret`), the schema is the black-box judge of legality (invoked at deploy over the resolved binding with secrets as opaque `SecretString` boxes, and again at boot), and secretness is a leaf *type* enforced by validation in both directions. The wire format is one self-describing JSON document row per service with `$secret` pointers to platform variables; an env-bound key whose variable is unset resolves to key-omitted and the schema arbitrates absence — subsuming optional secrets and conditional config (`stripeId` only when `stripeEnabled`) without a framework DSL. - [ADR-0043](ADR-0043-the-control-subpath-is-the-programmatic-deploy-surface.md) — `@prisma/composer/control` is the programmatic deploy surface: typed `deploy`/`destroy`/`dev`/`log` operations (structured inputs/results, no argv/console/exit) implemented in `@internal/cli`'s `src/operations/` and re-exported per ADR-0035; the CLI is a thin renderer over them. The entry's static graph stays import-light — each operation lazily imports its executor, so importing the subpath executes nothing, and a tree that cannot load the deploy stack surfaces as a structured `pipeline` failure — and `PRISMA_COMPOSER_DEPLOYMENT_RESULT_FILE` carries the deploy result across the process boundary: the alchemy child's report hook writes a serializable `DeploymentSummary` to the named file, the operation reads it back best-effort (absent/malformed = undefined summary, never a failure). Distinct from an extension's ADR-0017 `/control` entry. *(Amended by ADR-0049: each operation takes a required `config: { value, file }`, and the effect-conflict diagnosis is retired.)* *(Amended by ADR-0050: the `prisma` CLI renders two of the four operations, as `prisma deploy` and `prisma dev`; `destroy` and `log` are called from scripts.)* - [ADR-0044](ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md) — Errors are structural envelopes with dotted `NAMESPACE.SUBCODE` codes (the shared prisma/prisma foundation, duplicated pending extraction): structured at origin with why/fix splits, no catch-all codes, bugs carry no code (exit 1 + report hint), recognition is structural (`CliStructuredError.is()`), operation results ride the shared `Result` `ok` discriminator, expected failures exit 2 — with the alchemy child-status passthrough as the documented exception. *(Amended by ADR-0049: `DEPS.EFFECT_VERSION_CONFLICT` is retired.)* @@ -73,3 +73,4 @@ _Earlier drafts (ADR-0001, ADR-0002) were retired as the high-level design settl - [ADR-0049](ADR-0049-composers-configuration-is-the-composer-section-of-prisma-config.md) — Composer's configuration is the `composer` section of `prisma.config.ts`, loaded by the Prisma CLI engine. The validator checks only the fields that identify extension and state descriptors, passes descriptors through as the file's own objects, rejects unknown keys, and takes the nearest declaring file's section whole so the generated stack imports exactly one file. The old `prisma-composer.config.ts` and the `configPath` pointer are refused with `CONFIG.SECTION_MISSING`, `CONFIG.FIELD_RETIRED` and `CONFIG.FILE_RETIRED`, never migrated. The validator returns the section with its declaring file, and the programmatic operations take the same `ComposerConfigSource` (`{ value, file }`) and run the same checks. The Alchemy child evaluates the whole `prisma.config.ts`. The effect pre-flight and `DEPS.EFFECT_VERSION_CONFLICT` are retired: a broken tree fails evaluation of `prisma.config.ts` with the engine's `CLI.CONFIG_UNREADABLE`. *(Amended: the alternatives section lists the rejected ways to keep a Composer-owned effect check.)* - [ADR-0050](ADR-0050-composer-runs-as-prisma-deploy-and-prisma-dev.md) — Composer has no binary of its own: the `prisma` CLI mounts exactly two Composer commands, `prisma deploy ` and `prisma dev `, mounted at the root as bare verbs on purpose rather than under a noun. `destroy` and `log` are operations on `@prisma/composer/control`, called from a script; a teardown script reads `PRISMA_SERVICE_TOKEN` and `PRISMA_WORKSPACE_ID`. Where teardown and logs belong in the `prisma` command tree is not decided; until it is, agents and users need a script for both. A `destroy` command was rejected: the CLI's destructive verb is `delete`, and no Composer command takes `--production`. Retires the `prisma-composer` binary; a CI lint keeps its name out of user-facing files. - [ADR-0051](ADR-0051-a-nodes-address-is-its-logical-id-on-the-platform.md) — A node's address is its identity on Prisma Cloud: Composer writes it, byte for byte, as the `logicalId` of the node's one platform row (Project, App, Database, Bucket) and as the node's `logicalId` in the application topology, so the platform joins topology nodes to rows, and one node across branches, by string equality. A provision ID defaults to the node's name, so changing a name changes identity. The value is never an Alchemy resource ID, a display name or a platform `id`; App, Database and Bucket rows get it as a prop on the upstream resource that creates them, and the Project from container resolution; no separate resource writes it. Changing an address changes the entity's identity; changing a display name changes nothing. +- [ADR-0052](ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md) — The local emulators know nothing about Composer: an emulator keys its instances by ids it generates and stores labels it never reads. Composer records per working directory, in `.prisma-composer/dev/emulator-instances.json`, which instance backs each node, and nothing else maps Composer's names to instances, so two directories running one app get separate instances. An instance keeps its port until it is deleted, because the port is in the deployment's environment and its dependents' URLs and a warm start does not redeploy; a remembered port is requested, as a preference, only when an instance is re-created. Running a different app from a directory removes the previous app's Compute instances. Amends ADR-0041's app-name local identity. The Postgres and bucket emulators move to this model separately. From 809974665ec67dc3d5d81718a71d4efe966c510d Mon Sep 17 00:00:00 2001 From: willbot Date: Tue, 13 Oct 2026 16:51:52 +0200 Subject: [PATCH 08/17] docs(guides): each directory runs its own local services The running-locally guide and the core-concepts skill say that two checkouts of one app no longer share services, that a service keeps its port in its directory, what dev prints when a remembered port is taken or when a different app runs from the same directory, and that local databases and buckets are still shared by app name. The front-door examples in the running-locally and getting-started guides now match what `prisma dev` prints (`dev: ready`, then `
: `). Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- docs/guides/getting-started.md | 6 +-- docs/guides/running-locally.md | 44 ++++++++++++++++--- skills/prisma-composer-core-concepts/SKILL.md | 1 + 3 files changed, 43 insertions(+), 8 deletions(-) diff --git a/docs/guides/getting-started.md b/docs/guides/getting-started.md index 14396503e..712f14dbd 100644 --- a/docs/guides/getting-started.md +++ b/docs/guides/getting-started.md @@ -290,9 +290,9 @@ It runs the same pipeline a deploy runs, against local stand-ins for Prisma Cloud, and prints the front door — each service's local URL: ``` -[dev] ready: -[dev] gateway http://localhost:3001 -[dev] quotes http://localhost:3000 +dev: ready +gateway: http://localhost:3001 +quotes: http://localhost:3000 curl localhost:3001 # Make it work, make it right, make it fast. diff --git a/docs/guides/running-locally.md b/docs/guides/running-locally.md index 9454fc548..ff871a0ef 100644 --- a/docs/guides/running-locally.md +++ b/docs/guides/running-locally.md @@ -35,10 +35,10 @@ It builds nothing for you (bring your own built output, same as deploy), then stands the app up and prints the **front door** — every service's local URL: ``` -[dev] ready: -[dev] storefront http://localhost:3004 -[dev] catalog.service http://localhost:3000 -[dev] orders.service http://localhost:3003 +dev: ready +storefront: http://localhost:3004 +catalog.service: http://localhost:3000 +orders.service: http://localhost:3003 ``` `dev` prints `ready` only once every service accepts connections on its port, so a request to a printed URL right away connects. After a rebuild, `dev` waits the same way before it prints the URLs again. A service must listen on the port Composer gives it. @@ -67,6 +67,39 @@ instances and data before starting. On a fresh start, each service gets the lowest free port from 3000 up, in dependency order: a service gets its port before the services that call it, and services with no dependency between them go in the order you provisioned them. A warm start keeps the ports services already have. +### Each directory runs its own services + +The services `dev` runs belong to the directory you run it in. Two checkouts +of the same app, such as two git worktrees, run separate service processes on +separate ports, and a rebuild in one never restarts the other's services. +`dev` records which local service backs each of your services in +`.prisma-composer/dev/emulator-instances.json`. + +A service keeps its port in its directory from one `dev` to the next, even +while it is stopped. If the local service is gone, for example because a +Composer upgrade discarded the compute emulator's state, the next `dev` +creates it again and asks for the port it had. When another directory has +taken that port, the service gets the lowest free one and `dev` prints one +line: + +``` +[dev] service "gateway" now uses port 3004: its previous port 3001 is in use +``` + +A directory runs one app at a time. If you run a different app from the same +directory, `dev` removes the previous app's local services first and says so: + +``` +[dev] removed the local services of "shop", the app run here before +``` + +A stopped service keeps its port reserved, so the services of a checkout you +delete keep their ports. There is no command to remove them yet. + +Local databases and buckets are not separated this way yet: they are still +keyed by the app's name, so two checkouts of one app share them (see +[What's local vs. what's real](#whats-local-vs-whats-real)). + `--fresh` is also the fix when a framework upgrade leaves stale rows in this app's local dev state — the symptom is a plan-time error naming an unregistered resource type (for example @@ -142,7 +175,8 @@ token, workspace, or network is involved. Three consequences worth knowing: - **`PRISMA_COMPOSER_EMULATORS_DIR` gives a checkout or a CI job its own compute and buckets emulators.** By default every `dev` on the machine shares one set of emulator daemons, registered under - `~/.prisma-composer/emulators`. Set the variable to an absolute directory + `~/.prisma-composer/emulators`; each directory still gets its own services + on them. Set the variable to an absolute directory and `dev` starts and finds its compute and buckets daemons, their registry and their data there instead. Each picks free ports, so several can run side by side. A relative path is refused with `DEV.EMULATORS_DIR_INVALID`; `~` is diff --git a/skills/prisma-composer-core-concepts/SKILL.md b/skills/prisma-composer-core-concepts/SKILL.md index 611a9bf35..11ed68bd1 100644 --- a/skills/prisma-composer-core-concepts/SKILL.md +++ b/skills/prisma-composer-core-concepts/SKILL.md @@ -435,6 +435,7 @@ that surprise: changes. `ready` prints only once every service accepts connections on its port. A service that exits before it listens, or does not listen within 30 seconds, fails `dev` (and stops the app's other services) with a message naming the service, its port, and its log file: on a first start it is in the deploy engine's output, before the error that the deploy engine exited (`DEV.CONVERGE_FAILED` from the `dev` operation); on a warm start it is the `DEV.SERVICE_START_FAILED` message. `PRISMA_COMPOSER_SERVICE_READY_TIMEOUT_MS` raises the 30-second limit (milliseconds); the compute emulator reads it when it starts, so a running emulator keeps its old value until its process ends. An emulator that does not accept connections fails `dev` and `log` with `DEV.EMULATOR_UNREACHABLE` after a few seconds of retries; nothing else is retried. 2. Ctrl-C stops the app's processes but leaves local databases, buckets, and their data up: the next `dev` is a warm start. Starting clean, wiping this app's local instances and data first, is an explicit opt-in flag. On a fresh start, each service gets the lowest free port from 3000 up, in dependency order: a service before the services that call it, otherwise provision order. A warm start keeps the ports services already have. + Services belong to the directory `dev` runs in: two checkouts of one app (two git worktrees) run separate processes on separate ports and never restart each other's. `.prisma-composer/dev/emulator-instances.json` records which local service backs each service. A service keeps its port in its directory across runs, even while stopped. If its local service is gone (a Composer upgrade discarded the compute emulator's state), the next `dev` re-creates it and asks for the old port; when another directory has taken it, the service gets the lowest free port and `dev` prints `[dev] service "
" now uses port : its previous port is in use`. A directory runs one app at a time: running a different app there removes the previous app's local services and prints `[dev] removed the local services of "", the app run here before`. A stopped service keeps its port reserved, so a deleted checkout's services keep theirs; there is no command to remove them yet. Local databases and buckets are still keyed by the app's name, so two checkouts of one app share them. 3. `dev` does not print service logs. The `log` operation follows the already-running app's merged logs; it never builds, provisions, starts, or stops anything: From a5d3eabcaf0f3e5505cf26d29fe8de64f6123918 Mon Sep 17 00:00:00 2001 From: willbot Date: Tue, 13 Oct 2026 16:52:12 +0200 Subject: [PATCH 09/17] docs(drive): record deferred items for emulator commands Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- .drive/projects/emulator-commands/deferred.md | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 .drive/projects/emulator-commands/deferred.md diff --git a/.drive/projects/emulator-commands/deferred.md b/.drive/projects/emulator-commands/deferred.md new file mode 100644 index 000000000..57dc36212 --- /dev/null +++ b/.drive/projects/emulator-commands/deferred.md @@ -0,0 +1,4 @@ +# Deferred + +- **A start can join a crash-backoff restart that has already exited (prisma/composer#353).** When the fifth fast exit marks an instance `held`, the readiness probe only notices the exit after its current connection attempt. A start request arriving in that gap joins the dying start and gets its old failure, so nothing resumes. Seen once as an intermittent failure of "a held service resumes on start" in `compute-deployment.test.ts`; not reproduced in 10 runs under CPU load. Likely fix: a start must not join an in-flight start whose process has already exited. Belongs in #353, not this project. +- **What a previous app's databases and buckets become when another app runs from the same directory.** Compute instances are deleted (S1a). Databases and buckets hold data; decide in S1b. From 7139aa053e296e744f10569c7935c260aea5f5a3 Mon Sep 17 00:00:00 2001 From: willbot Date: Tue, 13 Oct 2026 17:08:22 +0200 Subject: [PATCH 10/17] fix(dev-emulators): never retry instance creation, and let a stop skip unknown instances Creating an instance is not idempotent: if the emulator handled the first attempt and the connection then failed, a retry created a second instance that held its port with nothing pointing at it. The client no longer retries POST /instances; the other admin calls are idempotent and keep their retry. A bulk stop now skips ids the emulator does not have and stops the rest, so Ctrl-C stops a directory's services even if one recorded instance has vanished. A bulk start still fails on an unknown id. The state test now deploys a payload shaped like Composer's (an artifact directory under the project's dev directory, env keys named after the address) and asserts what holds: labels and payload are stored verbatim, and no route, state key, file name or emulator-generated field carries their values. Two test names that still said "app" now say instances. Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- .../src/__tests__/compute-deployment.test.ts | 2 +- .../src/__tests__/compute-instances.test.ts | 106 +++++++++++++----- .../src/__tests__/compute-readiness.test.ts | 2 +- .../0-lowering/dev-emulators/src/client.ts | 29 +++-- .../dev-emulators/src/compute-main.ts | 8 +- 5 files changed, 106 insertions(+), 41 deletions(-) diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-deployment.test.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-deployment.test.ts index 79364bd73..45c559607 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-deployment.test.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-deployment.test.ts @@ -368,7 +368,7 @@ describe('session resume (POST /instances/start)', () => { await waitFor(async () => (await deployedService(reserved.id)).status !== 'held', 2000, 50); }, 30_000); - test('start on an already-running app is a no-op — pids unchanged', async () => { + test('start on running instances is a no-op — pids unchanged', async () => { const client = computeClient({ registryRoot }); const dir = writeBootstrap(SERVING_BOOTSTRAP); const reserved = await client.createInstance({ labels: {} }); diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-instances.test.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-instances.test.ts index 8186f5690..3cf63cf98 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-instances.test.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-instances.test.ts @@ -75,13 +75,25 @@ describe('creating and listing instances', () => { expect(await client.getInstance('svc_0000000000000000')).toBeUndefined(); }); - test('a start or stop naming an unknown instance fails and names it', async () => { + test('a start naming an unknown instance fails, names it, and starts nothing', async () => { const known = await client.createInstance({ labels: {} }); + await deployServing(known.id, known.port, 'known'); + await client.stopInstances([known.id]); + await expect(client.startInstances([known.id, 'svc_0000000000000000'])).rejects.toThrow( '"svc_0000000000000000"', ); - await expect(client.stopInstances(['svc_0000000000000000'])).rejects.toThrow('(404)'); - }); + expect(await statusOf(known.id)).toBe('stopped'); + }, 15_000); + + test('a stop skips an unknown instance and stops the others', async () => { + const known = await client.createInstance({ labels: {} }); + await deployServing(known.id, known.port, 'known'); + + await client.stopInstances([known.id, 'svc_0000000000000000']); + + expect(await statusOf(known.id)).toBe('stopped'); + }, 15_000); test('a malformed instance id is rejected with 400', async () => { const res = await fetch(`${client.baseUrl}/instances/web`); @@ -243,51 +255,93 @@ describe('persisted state', () => { }); /** - * The emulator knows nothing about Composer: an app name, a node address or - * a project directory may reach it only as label values. These checks fail - * when the emulator's code names one of those concepts, or when one of them - * ends up in its state or files anywhere other than inside labels. + * The emulator never interprets Composer's data. An app name, a node address + * or a project directory reaches it only as label values and inside the + * opaque deployment payload (the artifact directory and the env), which it + * stores verbatim. These checks fail when the emulator's code names one of + * those concepts, or when one of them shows up anywhere else: a route, a + * state key, a file name or a field the emulator generates. */ -describe('the emulator stores Composer names only in labels', () => { - const markers = { - 'composer.app': 'zz-app-marker', - 'composer.address': 'zz-address-marker', - 'composer.directory': '/zz-directory-marker/project', +describe('the emulator never interprets Composer data', () => { + const app = 'zzappmarker'; + const address = 'zzaddressmarker'; + const directory = path.join(tempDir('marker-project'), 'zzdirectorymarker'); + const markers = [app, address, 'zzdirectorymarker']; + const labels = { + 'composer.directory': directory, + 'composer.app': app, + 'composer.address': address, }; - const markerValues = Object.values(markers); - function withoutLabels(value: unknown): unknown { - if (Array.isArray(value)) return value.map(withoutLabels); + /** A deployment shaped like Composer's: its artifact unpacked under the project's dev directory, env keys named after the address. */ + function composerShapedDeployment(port: number) { + const artifactDir = path.join(directory, '.prisma-composer', 'dev', 'artifacts', 'abc123'); + fs.mkdirSync(artifactDir, { recursive: true }); + fs.writeFileSync(path.join(artifactDir, 'bootstrap.js'), SERVING_BOOTSTRAP); + const env = baseEnv({ + PORT: String(port), + [`COMPOSER_${address.toUpperCase()}_PORT`]: String(port), + [`COMPOSER_${address.toUpperCase()}_INPUT`]: '{}', + COMPOSER_GATEWAY_QUOTES_URL: 'http://localhost:3001', + ...servingBootstrapEnv('marker test'), + }); + return { artifactDir, artifactHash: 'abc123', env }; + } + + function without(value: unknown, keys: readonly string[]): unknown { + if (Array.isArray(value)) return value.map((item) => without(item, keys)); if (typeof value !== 'object' || value === null) return value; return Object.fromEntries( Object.entries(value) - .filter(([key]) => key !== 'labels') - .map(([key, field]) => [key, withoutLabels(field)]), + .filter(([key]) => !keys.includes(key)) + .map(([key, field]) => [key, without(field, keys)]), ); } + function keysOf(value: unknown): string[] { + if (Array.isArray(value)) return value.flatMap(keysOf); + if (typeof value !== 'object' || value === null) return []; + return Object.entries(value).flatMap(([key, field]) => [key, ...keysOf(field)]); + } + function filesUnder(dir: string): string[] { return fs .readdirSync(dir, { withFileTypes: true, recursive: true }) .map((entry) => path.join(entry.parentPath, entry.name)); } - test('state, file names and listings hold the label values only inside labels', async () => { - const created = await client.createInstance({ labels: markers }); - await deployServing(created.id, created.port, 'marker test'); + function mentions(text: string, marker: string): boolean { + return text.toLowerCase().includes(marker); + } + + test('labels and the deployment payload are stored verbatim, and nothing else carries their values', async () => { + const created = await client.createInstance({ labels }); + const deployment = composerShapedDeployment(created.port); + await client.putDeployment(created.id, deployment); await client.stopInstances([created.id]); const stateDir = daemonStateDir(registryRoot, 'compute'); const state: unknown = JSON.parse( fs.readFileSync(path.join(stateDir, 'instances.json'), 'utf8'), ); - const listed: unknown = await (await fetch(`${client.baseUrl}/instances`)).json(); + const record: unknown = Reflect.get(Reflect.get(state as object, 'instances'), created.id); + expect(record).toMatchObject({ + labels, + artifactDir: deployment.artifactDir, + artifactHash: deployment.artifactHash, + env: deployment.env, + }); - for (const marker of markerValues) { - expect(JSON.stringify(state)).toContain(marker); - expect(JSON.stringify(withoutLabels(state))).not.toContain(marker); - expect(JSON.stringify(withoutLabels(listed))).not.toContain(marker); - for (const file of filesUnder(stateDir)) expect(file).not.toContain(marker); + const opaque = ['labels', 'artifactDir', 'env']; + const stateKeys = keysOf(without(state, ['labels', 'env'])).join('\n'); + const listed: unknown = await (await fetch(`${client.baseUrl}/instances`)).json(); + const own: unknown = await (await fetch(`${client.baseUrl}/instances/${created.id}`)).json(); + for (const marker of markers) { + expect(mentions(JSON.stringify(without(state, opaque)), marker)).toBe(false); + expect(mentions(stateKeys, marker)).toBe(false); + expect(mentions(JSON.stringify(without(listed, ['labels'])), marker)).toBe(false); + expect(mentions(JSON.stringify(without(own, ['labels'])), marker)).toBe(false); + for (const file of filesUnder(stateDir)) expect(mentions(file, marker)).toBe(false); } }, 15_000); diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts index e3038ee52..0630da0e9 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts @@ -353,7 +353,7 @@ describe('a start that is replaced, stopped or joined', () => { if (svc?.pid !== undefined) process.kill(svc.pid, 'SIGKILL'); }, 30_000); - test('a start during a delete of the app leaves no process behind', async () => { + test('a start during an instance delete leaves no process behind', async () => { const artifactDir = writeBootstrap(IGNORES_SIGTERM); const reserved = await client.createInstance({ labels: {} }); await client.putDeployment(reserved.id, quickDeployment(reserved.port, artifactDir)); diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/client.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/client.ts index 5b5ec17e1..52b562def 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/client.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/client.ts @@ -73,20 +73,27 @@ const ADMIN_FETCH_RETRY_DELAY_MS = 150; * `fetch` for the loopback admin calls, retried on transient socket errors: * a keep-alive reuse race (the runtime re-uses a pooled connection the * daemon's HTTP server just closed idle — surfaces as "fetch failed" / - * "other side closed") or a briefly overloaded daemon. Every admin call is - * idempotent by design, so a short blind retry is safe. A caller-driven - * abort is never retried. When every attempt fails to connect, it throws - * `DEV.EMULATOR_UNREACHABLE`; callers retry that code and nothing else. + * "other side closed") or a briefly overloaded daemon. A short blind retry is + * safe only for an idempotent call; a call that is not, such as creating an + * instance, passes `retry: false`, because the emulator may have handled the + * first attempt. A caller-driven abort is never retried. When every attempt + * fails to connect, it throws `DEV.EMULATOR_UNREACHABLE`; callers retry that + * code and nothing else. */ -async function adminFetch(url: string, init?: RequestInit): Promise { +async function adminFetch( + url: string, + init?: RequestInit, + opts: { readonly retry?: boolean } = {}, +): Promise { + const attempts = opts.retry === false ? 1 : ADMIN_FETCH_ATTEMPTS; let lastError: unknown; - for (let attempt = 1; attempt <= ADMIN_FETCH_ATTEMPTS; attempt += 1) { + for (let attempt = 1; attempt <= attempts; attempt += 1) { try { return await fetch(url, init); } catch (error) { if (init?.signal?.aborted === true) throw error; lastError = error; - if (attempt < ADMIN_FETCH_ATTEMPTS) { + if (attempt < attempts) { await new Promise((resolve) => setTimeout(resolve, ADMIN_FETCH_RETRY_DELAY_MS)); } } @@ -189,7 +196,7 @@ function isHealthBody(value: unknown): value is HealthBody { export interface ComputeClient { readonly baseUrl: string; health(): Promise<{ version: string }>; - /** `POST /instances` — creates a stopped instance holding a port until it is deleted. */ + /** `POST /instances` — creates a stopped instance holding a port until it is deleted. Never retried: a retry could create a second instance. */ createInstance(request: CreateInstanceRequest): Promise; /** `GET /instances`. */ listInstances(): Promise; @@ -205,7 +212,7 @@ export interface ComputeClient { ): AsyncIterable; /** `POST /instances/start` — the session-resume signal; starts each listed instance with a stored deployment that isn't already running. */ startInstances(ids: readonly string[]): Promise; - /** `POST /instances/stop`. The instances keep their ports. */ + /** `POST /instances/stop`. The instances keep their ports; an unknown id is skipped. */ stopInstances(ids: readonly string[]): Promise; /** `DELETE /instances/` — stops the instance and frees its port. Deleting an unknown id succeeds. */ deleteInstance(id: string): Promise; @@ -239,7 +246,9 @@ export function computeClient(opts: DaemonRootOptions = {}): ComputeClient { }, async createInstance(request) { - const res = await expectOk(await adminFetch(`${baseUrl}/instances`, postJson(request))); + const res = await expectOk( + await adminFetch(`${baseUrl}/instances`, postJson(request), { retry: false }), + ); const body: unknown = await res.json(); if (!isInstanceInfo(body)) { throw new Error('malformed instance-create response from the compute emulator'); diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/compute-main.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/compute-main.ts index 566e2056e..cc0388af7 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/compute-main.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/compute-main.ts @@ -877,12 +877,14 @@ function main(): void { ): Promise { const body = await readJsonBody(req); if (!isIdsBody(body)) return text(res, 400, 'malformed body: expected { "ids": string[] }'); - const selected = instancesByIds(body.ids); - if ('unknown' in selected) return unknownInstances(res, selected.unknown); if (action === 'stop') { - await Promise.all(selected.found.map((inst) => stopInstance(inst, getRuntime(inst.id)))); + // An unknown instance runs nothing, so stopping it is already done. + const known = body.ids.flatMap((id) => state.instances[id] ?? []); + await Promise.all(known.map((inst) => stopInstance(inst, getRuntime(inst.id)))); return noContent(res); } + const selected = instancesByIds(body.ids); + if ('unknown' in selected) return unknownInstances(res, selected.unknown); const results = await Promise.all(selected.found.map(startInstance)); const failures = results.flatMap((r) => (r.ok ? [] : [r.message ?? 'failed to start'])); if (failures.length > 0) return text(res, 500, failures.join('\n')); From b927591138ee28e78fac60885b747c6be05c6007 Mon Sep 17 00:00:00 2001 From: willbot Date: Tue, 13 Oct 2026 17:10:41 +0200 Subject: [PATCH 11/17] fix(local-target): --fresh deletes every recorded instance, whichever app the record names Teardown runs before the emulators hook, so after switching apps the record still names the app run here before. Teardown deleted only the current app's instances, then removed the record, leaving the previous app's instances holding their ports with nothing pointing at them. It now deletes every instance the record names, then empties it. `reserveServicePorts` is renamed `ensureServiceInstances`, since it ensures one instance per service rather than reserving ports. New tests against a real emulator: --fresh after switching apps leaves no instance of either app; the App provider's diff asks for an update when a re-created instance got another port and for nothing when it kept its port; a deployed service that exits before it listens fails naming its address; the record-file test checks the three labels written on create. A unit test covers the instance-id-to-address rewrite. Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- .../0-lowering/local-target/package.json | 1 + .../src/__tests__/compute-port-order.test.ts | 10 +- .../src/__tests__/emulator-instances.test.ts | 172 ++++++++++++++++-- .../0-lowering/local-target/src/compute.ts | 4 +- .../local-target/src/emulator-instances.ts | 16 +- .../__tests__/local-target-emulators.test.ts | 12 +- .../__tests__/local-target-teardown.test.ts | 10 +- .../target/src/local-target/emulators.ts | 4 +- .../target/src/local-target/teardown.ts | 4 +- pnpm-lock.yaml | 3 + 10 files changed, 193 insertions(+), 43 deletions(-) diff --git a/packages/1-prisma-cloud/0-lowering/local-target/package.json b/packages/1-prisma-cloud/0-lowering/local-target/package.json index 43f914ec0..7871be5da 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/package.json +++ b/packages/1-prisma-cloud/0-lowering/local-target/package.json @@ -24,6 +24,7 @@ "tar": "^7.5.21" }, "devDependencies": { + "@internal/foundation": "workspace:0.29.1", "@internal/tsdown-config": "workspace:0.29.1", "@types/bun": "^1.3.13", "@types/node": "^26.0.1", diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-port-order.test.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-port-order.test.ts index 2c50b1c7e..acd214585 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-port-order.test.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/compute-port-order.test.ts @@ -7,7 +7,7 @@ import type { ContainerInstance } from '@internal/core/config'; import { EMULATORS_DIR_ENV, ensureDaemon, stopDaemon } from '@internal/dev-emulators'; import { App } from 'alchemy/Prisma'; import * as Effect from 'effect/Effect'; -import { LocalAppProvider, reserveServicePorts } from '../compute.ts'; +import { ensureServiceInstances, LocalAppProvider } from '../compute.ts'; import { computeNodeInstances, ensureComputeInstance } from '../emulator-instances.ts'; const savedEmulatorsDir = process.env[EMULATORS_DIR_ENV]; @@ -66,7 +66,7 @@ test('each App provider returns the port reserved for its service, even when the for (let run = 0; run < 10; run++) { const appName = `port-order-${String(run)}`; const container = fakeContainer(appName); - await reserveServicePorts(container, devDir, ['quotes', 'gateway']); + await ensureServiceInstances(container, devDir, ['quotes', 'gateway']); const reserved = await reservedPorts(appName); const [gateway, quotes] = await Promise.all([ @@ -85,7 +85,7 @@ test('a failed reservation names the service', async () => { await stopDaemon('compute', { registryRoot }); await expect( - reserveServicePorts(fakeContainer('port-order-down'), devDir, ['quotes']), + ensureServiceInstances(fakeContainer('port-order-down'), devDir, ['quotes']), ).rejects.toThrow('service "quotes"'); }); @@ -95,7 +95,7 @@ test('reserving the services of an app that already has ports keeps those ports' const { instance: gateway } = await ensureComputeInstance(scope, 'gateway'); const { instance: quotes } = await ensureComputeInstance(scope, 'quotes'); - await reserveServicePorts(container, devDir, ['quotes', 'gateway']); + await ensureServiceInstances(container, devDir, ['quotes', 'gateway']); expect(await reconcileApp(container, 'quotes')).toBe(quotes.port); expect(await reconcileApp(container, 'gateway')).toBe(gateway.port); @@ -103,7 +103,7 @@ test('reserving the services of an app that already has ports keeps those ports' test('a dotted service address reserves the same emulator instance the App provider uses', async () => { const container = fakeContainer('port-order-nested'); - await reserveServicePorts(container, devDir, ['shop.quotes']); + await ensureServiceInstances(container, devDir, ['shop.quotes']); const listed = (await computeNodeInstances({ devDir, app: 'port-order-nested' })).map( (node) => node.instance, diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts index aa556dfd2..fcf6588a5 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts @@ -14,15 +14,18 @@ import { ensureDaemon, stopDaemon, } from '@internal/dev-emulators'; +import { CliStructuredError } from '@internal/foundation/errors'; import { packageComputeArtifact } from '@internal/lowering/compute'; import { App, Deployment } from 'alchemy/Prisma'; import * as Effect from 'effect/Effect'; -import { LocalAppProvider, LocalDeploymentProvider, reserveServicePorts } from '../compute.ts'; +import { ensureServiceInstances, LocalAppProvider, LocalDeploymentProvider } from '../compute.ts'; import { + deleteRecordedInstances, EMULATOR_INSTANCES_FILE, type EmulatorInstances, readEmulatorInstances, retirePreviousApp, + withServiceAddresses, } from '../emulator-instances.ts'; const APP = 'shared-app'; @@ -71,6 +74,17 @@ function artifactFor(address: string, version: string): string { return packageComputeArtifact({ id: address, bundleDir, appEntry: 'server.js', address }).path; } +/** A packaged artifact whose service exits before it listens. */ +function crashingArtifactFor(address: string): string { + const bundleDir = fs.mkdtempSync(path.join(root, `bundle-${address}-`)); + fs.writeFileSync( + path.join(bundleDir, 'main.js'), + 'export default { run: async () => { process.exit(1); } };\n', + ); + fs.writeFileSync(path.join(bundleDir, 'server.js'), ''); + return packageComputeArtifact({ id: address, bundleDir, appEntry: 'server.js', address }).path; +} + const reconcileInput = (id: string, news: Record) => ({ id, @@ -87,6 +101,7 @@ async function deploy( devDir: string, address: string, version: string, + artifactPath: string = artifactFor(address, version), ): Promise { const provider = await Effect.runPromise( Deployment.Provider.pipe(Effect.provide(LocalDeploymentProvider({ container, devDir }))), @@ -95,7 +110,7 @@ async function deploy( provider.reconcile( reconcileInput(`${address}-deployment`, { app: address, - artifactPath: artifactFor(address, version), + artifactPath, }), ), ); @@ -135,8 +150,8 @@ describe('two directories running the same app', () => { const a = projectDir('a'); const b = projectDir('b'); await Promise.all([ - reserveServicePorts(container, a.devDir, ADDRESSES), - reserveServicePorts(container, b.devDir, ADDRESSES), + ensureServiceInstances(container, a.devDir, ADDRESSES), + ensureServiceInstances(container, b.devDir, ADDRESSES), ]); await Promise.all( [a, b].flatMap(({ devDir }) => ADDRESSES.map((address) => deploy(devDir, address, 'v1'))), @@ -170,11 +185,11 @@ describe('two directories running the same app', () => { describe('reusing the recorded instances', () => { test('a recorded instance the emulator no longer has is re-created on its remembered port', async () => { const { directory, devDir } = projectDir('lost'); - await reserveServicePorts(container, devDir, ['quotes']); + await ensureServiceInstances(container, devDir, ['quotes']); const before = await nodeOf(directory, 'quotes'); await computeClient().deleteInstance(before.id); - const changes = await reserveServicePorts(container, devDir, ['quotes']); + const changes = await ensureServiceInstances(container, devDir, ['quotes']); const after = await nodeOf(directory, 'quotes'); expect(after.id).not.toBe(before.id); @@ -184,12 +199,12 @@ describe('reusing the recorded instances', () => { test('a remembered port another instance now holds gives a new port, reported as a change', async () => { const { directory, devDir } = projectDir('moved'); - await reserveServicePorts(container, devDir, ['quotes']); + await ensureServiceInstances(container, devDir, ['quotes']); const before = await nodeOf(directory, 'quotes'); await computeClient().deleteInstance(before.id); await computeClient().createInstance({ labels: {}, port: before.port }); - const changes = await reserveServicePorts(container, devDir, ['quotes']); + const changes = await ensureServiceInstances(container, devDir, ['quotes']); const after = await nodeOf(directory, 'quotes'); expect(after.port).not.toBe(before.port); @@ -198,13 +213,13 @@ describe('reusing the recorded instances', () => { test('a warm restart after a stop reuses the same instances and ports', async () => { const { directory, devDir } = projectDir('warm'); - await reserveServicePorts(container, devDir, ADDRESSES); + await ensureServiceInstances(container, devDir, ADDRESSES); for (const address of ADDRESSES) await deploy(devDir, address, 'v1'); const before = (await mappingOf(directory))?.nodes; const ids = Object.values(before ?? {}).map((n) => n.id); await computeClient().stopInstances(ids); - const changes = await reserveServicePorts(container, devDir, ADDRESSES); + const changes = await ensureServiceInstances(container, devDir, ADDRESSES); await computeClient().startInstances(ids); expect(changes).toEqual([]); @@ -216,19 +231,19 @@ describe('reusing the recorded instances', () => { test('a deployment whose instance was re-created is deployed again even when nothing changed', async () => { const { directory, devDir } = projectDir('redeploy'); - await reserveServicePorts(container, devDir, ['quotes']); + await ensureServiceInstances(container, devDir, ['quotes']); const deployed = await deploy(devDir, 'quotes', 'v1'); expect(await deploymentDiff(devDir, deployed)).toBeUndefined(); await computeClient().deleteInstance((await nodeOf(directory, 'quotes')).id); - await reserveServicePorts(container, devDir, ['quotes']); + await ensureServiceInstances(container, devDir, ['quotes']); expect(await deploymentDiff(devDir, deployed)).toEqual({ action: 'update' }); }, 30_000); test("deleting a removed service's App deletes its instance and forgets it", async () => { const { directory, devDir } = projectDir('removed'); - await reserveServicePorts(container, devDir, ADDRESSES); + await ensureServiceInstances(container, devDir, ADDRESSES); const quotes = await nodeOf(directory, 'quotes'); const provider = await appProvider(devDir); @@ -241,20 +256,25 @@ describe('reusing the recorded instances', () => { test('the record is one JSON file in the dev directory, readable from the directory alone', async () => { const { directory, devDir } = projectDir('file'); - await reserveServicePorts(container, devDir, ['quotes']); + await ensureServiceInstances(container, devDir, ['quotes']); const onDisk: unknown = JSON.parse( fs.readFileSync(path.join(devDir, EMULATOR_INSTANCES_FILE), 'utf8'), ); const node = await nodeOf(directory, 'quotes'); expect(onDisk).toEqual({ app: APP, nodes: { quotes: node } }); + expect((await computeClient().getInstance(node.id))?.labels).toEqual({ + 'composer.directory': directory, + 'composer.app': APP, + 'composer.address': 'quotes', + }); expect(await readEmulatorInstances(path.join(root, 'never-ran'))).toBeUndefined(); }); describe('switching apps in a directory', () => { test("running another app deletes the previous app's compute instances and starts a new record", async () => { const { directory, devDir } = projectDir('switch'); - await reserveServicePorts(container, devDir, ADDRESSES); + await ensureServiceInstances(container, devDir, ADDRESSES); const previous = Object.values((await mappingOf(directory))?.nodes ?? {}); const retired = await retirePreviousApp({ devDir, app: 'another-app' }); @@ -266,10 +286,130 @@ describe('switching apps in a directory', () => { test('running the same app again retires nothing', async () => { const { directory, devDir } = projectDir('same'); - await reserveServicePorts(container, devDir, ['quotes']); + await ensureServiceInstances(container, devDir, ['quotes']); const quotes = await nodeOf(directory, 'quotes'); expect(await retirePreviousApp({ devDir, app: APP })).toBeUndefined(); expect(await computeClient().getInstance(quotes.id)).toBeDefined(); }); }); + +describe('--fresh', () => { + const otherApp: ContainerInstance = { + input: { appName: 'other-app', stage: undefined }, + serialize: () => 'x', + }; + + async function liveIds(ids: readonly string[]): Promise { + const live = await Promise.all(ids.map((id) => computeClient().getInstance(id))); + return live.flatMap((instance) => (instance ? [instance.id] : [])); + } + + test('deletes the recorded instances even when the record names the app run here before', async () => { + const { directory, devDir } = projectDir('fresh-previous'); + await ensureServiceInstances(container, devDir, ADDRESSES); + const ids = Object.values((await mappingOf(directory))?.nodes ?? {}).map((n) => n.id); + + await deleteRecordedInstances(devDir); + + expect(await liveIds(ids)).toEqual([]); + expect((await mappingOf(directory))?.nodes).toEqual({}); + }); + + test('after switching apps, leaves no instance of either app', async () => { + const { directory, devDir } = projectDir('fresh-switch'); + await ensureServiceInstances(container, devDir, ADDRESSES); + const first = Object.values((await mappingOf(directory))?.nodes ?? {}).map((n) => n.id); + await retirePreviousApp({ devDir, app: 'other-app' }); + await ensureServiceInstances(otherApp, devDir, ['web']); + const second = Object.values((await mappingOf(directory))?.nodes ?? {}).map((n) => n.id); + + await deleteRecordedInstances(devDir); + + expect(await liveIds([...first, ...second])).toEqual([]); + }); +}); + +describe("the App provider's diff", () => { + async function reconcileApp(devDir: string, address: string): Promise { + const provider = await appProvider(devDir); + return Effect.runPromise( + provider.reconcile( + reconcileInput(`${address}-svc`, { project: 'local', displayName: address }), + ), + ); + } + + async function appDiff(devDir: string, output: App['Attributes']): Promise { + const diff = (await appProvider(devDir)).diff; + if (!diff) throw new Error('the local App provider has no diff'); + return Effect.runPromise(diff({ output } as never)); + } + + test('asks for an update when the re-created instance got another port', async () => { + const { directory, devDir } = projectDir('app-moved'); + await ensureServiceInstances(container, devDir, ['quotes']); + const before = await reconcileApp(devDir, 'quotes'); + const recorded = await nodeOf(directory, 'quotes'); + await computeClient().deleteInstance(recorded.id); + await computeClient().createInstance({ labels: {}, port: recorded.port }); + await ensureServiceInstances(container, devDir, ['quotes']); + + expect(await appDiff(devDir, before)).toEqual({ action: 'update' }); + const after = await reconcileApp(devDir, 'quotes'); + expect(after.appEndpointDomain).toBe( + `http://localhost:${String((await nodeOf(directory, 'quotes')).port)}`, + ); + expect(after.appEndpointDomain).not.toBe(before.appEndpointDomain); + }); + + test('asks for nothing when the re-created instance kept its port', async () => { + const { directory, devDir } = projectDir('app-kept'); + await ensureServiceInstances(container, devDir, ['quotes']); + const before = await reconcileApp(devDir, 'quotes'); + await computeClient().deleteInstance((await nodeOf(directory, 'quotes')).id); + await ensureServiceInstances(container, devDir, ['quotes']); + + expect(await appDiff(devDir, before)).toBeUndefined(); + }); +}); + +describe('errors name services by address', () => { + const nodes = [{ address: 'shop.quotes', instance: { id: 'svc_0123456789abcdef' } }]; + + test('an instance id in a plain error becomes the service address, keeping the original as cause', () => { + const original = new Error( + 'request failed (500): instance "svc_0123456789abcdef" exited (code 1) before it accepted connections', + ); + + const rewritten = withServiceAddresses(original, nodes); + + expect(rewritten).toBeInstanceOf(Error); + expect((rewritten as Error).message).toBe( + 'request failed (500): service "shop.quotes" exited (code 1) before it accepted connections', + ); + expect((rewritten as Error).cause).toBe(original); + }); + + test('a structured error is left untouched', () => { + const structured = new CliStructuredError( + 'DEV.EMULATOR_UNREACHABLE', + 'instance "svc_0123456789abcdef" unreachable', + ); + + expect(withServiceAddresses(structured, nodes)).toBe(structured); + }); + + test('a deployed service that exits before it listens fails naming its address', async () => { + const { devDir } = projectDir('crash'); + await ensureServiceInstances(container, devDir, ['quotes']); + + const failure = await deploy(devDir, 'quotes', 'v1', crashingArtifactFor('quotes')).then( + () => undefined, + (error: unknown) => (error instanceof Error ? error.message : String(error)), + ); + + expect(failure).toContain('service "quotes" exited (code 1) before it accepted connections'); + expect(failure).not.toContain('instance "svc_'); + }, 30_000); +}); diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts index 209fe5b8e..a24d7fe4e 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/compute.ts @@ -237,7 +237,7 @@ export function LocalAppProvider( * the `App` resources' `displayName`s. Returns the services whose instance had * to be re-created and could not get its remembered port back. */ -export async function reserveServicePorts( +export async function ensureServiceInstances( container: ContainerInstance | undefined, devDir: string, serviceAppNames: readonly string[], @@ -250,7 +250,7 @@ export async function reserveServicePorts( if (portChange) changes.push(portChange); } catch (cause) { const reason = cause instanceof Error ? cause.message : String(cause); - throw new Error(`reserving a local port for service "${service}" failed: ${reason}`, { + throw new Error(`ensuring the local service for service "${service}" failed: ${reason}`, { cause, }); } diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/emulator-instances.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/emulator-instances.ts index f716da4a1..50c6d0c0a 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/emulator-instances.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/emulator-instances.ts @@ -179,11 +179,17 @@ export async function deleteComputeInstance(scope: AppScope, address: string): P }); } -/** Deletes every compute instance recorded for the app and forgets them. */ -export async function deleteComputeInstances(scope: AppScope): Promise { - for (const { address } of await computeNodeInstances(scope)) { - await deleteComputeInstance(scope, address); - } +/** + * `--fresh`: deletes every instance the directory's record names, whichever + * app wrote it, then empties the record. Teardown runs before the emulators + * hook, so the record may still name the app run here before. + */ +export async function deleteRecordedInstances(devDir: string): Promise { + const current = await store(devDir).read(); + if (current === undefined) return; + const client = computeClient(); + for (const instance of Object.values(current.nodes)) await client.deleteInstance(instance.id); + await store(devDir).update(() => ({ app: current.app, nodes: {} })); } /** diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts index 9c3162be7..a29124b94 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-emulators.test.ts @@ -22,12 +22,12 @@ mock.module('@internal/local-target', () => ({ ...RealLocalTarget, resolvePackageEntry: (specifier: string) => specifier, retirePreviousApp: async () => retiredApp, - reserveServicePorts: async ( + ensureServiceInstances: async ( container: ContainerInstance | undefined, _devDir: string, addresses: readonly string[], ) => { - events.push(`reserve ${container?.input.appName ?? '?'}: ${addresses.join(', ')}`); + events.push(`ensure ${container?.input.appName ?? '?'}: ${addresses.join(', ')}`); return portChanges; }, })); @@ -74,7 +74,7 @@ afterEach(() => { consoleLog.mockClear(); }); -test('reserves the getting-started services in dependency order once the compute emulator is up', async () => { +test('ensures the getting-started services in dependency order once the compute emulator is up', async () => { const graph = Load( module('my-app', ({ provision }) => { const quotes = provision( @@ -88,10 +88,10 @@ test('reserves the getting-started services in dependency order once the compute await runDevEmulators({ graph, container, devDir: '/dev/null/unused' }); - expect(events).toEqual(['daemon compute', 'reserve my-app: quotes, gateway']); + expect(events).toEqual(['daemon compute', 'ensure my-app: quotes, gateway']); }); -test('reserves only services, in declaration order when nothing orders them', async () => { +test('ensures only services, in declaration order when nothing orders them', async () => { const graph = Load( module('my-app', ({ provision }) => { provision(compute({ name: 'web', deps: {}, build })); @@ -102,7 +102,7 @@ test('reserves only services, in declaration order when nothing orders them', as await runDevEmulators({ graph, container, devDir: '/dev/null/unused' }); - expect(events).toEqual(['daemon compute', 'reserve my-app: web, admin', 'daemon postgres']); + expect(events).toEqual(['daemon compute', 'ensure my-app: web, admin', 'daemon postgres']); }); test('prints one line for each service that could not get its remembered port back', async () => { diff --git a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-teardown.test.ts b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-teardown.test.ts index 5beaba25a..92987a370 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-teardown.test.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/__tests__/local-target-teardown.test.ts @@ -25,8 +25,8 @@ mock.module('@internal/dev-emulators', () => ({ mock.module('@internal/local-target', () => ({ ...RealLocalTarget, resolvePackageEntry: (specifier: string) => specifier, - deleteComputeInstances: async ({ app }: { app: string }) => { - events.push(`delete compute ${app}`); + deleteRecordedInstances: async (devDir: string) => { + events.push(`delete compute ${devDir.endsWith('/.prisma-composer/dev') ? 'recorded' : devDir}`); if (computeDeleteFails) throw new Error('compute delete failed'); }, removeLocalPaths: () => { @@ -49,10 +49,10 @@ beforeEach(() => { computeDeleteFails = false; }); -test('starts the compute emulator before deleting the app from it, so a stopped emulator still forgets the old ports', async () => { +test('starts the compute emulator before deleting the recorded instances, so a stopped emulator still forgets the recorded instances', async () => { await runDevTeardown({ container, stage: undefined }); - expect(events).toEqual(['daemon compute', 'delete compute my-app', 'remove local paths']); + expect(events).toEqual(['daemon compute', 'delete compute recorded', 'remove local paths']); }); test('a failed compute delete still removes the local state, then fails the teardown', async () => { @@ -61,5 +61,5 @@ test('a failed compute delete still removes the local state, then fails the tear await expect(runDevTeardown({ container, stage: undefined })).rejects.toThrow( 'compute delete failed', ); - expect(events).toEqual(['daemon compute', 'delete compute my-app', 'remove local paths']); + expect(events).toEqual(['daemon compute', 'delete compute recorded', 'remove local paths']); }); diff --git a/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts b/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts index ed279c283..21df6d008 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/local-target/emulators.ts @@ -24,7 +24,7 @@ */ import type { LocalTargetEmulatorsInput } from '@internal/core/config'; import { ensureDaemon } from '@internal/dev-emulators'; -import { appNameOf, reserveServicePorts, retirePreviousApp } from '@internal/local-target'; +import { appNameOf, ensureServiceInstances, retirePreviousApp } from '@internal/local-target'; import { PRISMA_CLOUD_EXTENSION_ID } from '../container.ts'; import { serviceAppName } from '../service-app.ts'; import { daemonEntry } from './daemon-entry.ts'; @@ -57,7 +57,7 @@ export async function runDevEmulators(input: LocalTargetEmulatorsInput): Promise if (retired !== undefined) { console.log(`[dev] removed the local services of "${retired}", the app run here before`); } - const portChanges = await reserveServicePorts( + const portChanges = await ensureServiceInstances( input.container, input.devDir, serviceAppNames(input), diff --git a/packages/1-prisma-cloud/1-extensions/target/src/local-target/teardown.ts b/packages/1-prisma-cloud/1-extensions/target/src/local-target/teardown.ts index e90d26b66..bc59ef4af 100644 --- a/packages/1-prisma-cloud/1-extensions/target/src/local-target/teardown.ts +++ b/packages/1-prisma-cloud/1-extensions/target/src/local-target/teardown.ts @@ -19,7 +19,7 @@ import type { TeardownInput } from '@internal/core/config'; import { DEV_DIR } from '@internal/core/config'; import { bucketsClient, ensureDaemon, postgresClient } from '@internal/dev-emulators'; -import { deleteComputeInstances, removeLocalPaths } from '@internal/local-target'; +import { deleteRecordedInstances, removeLocalPaths } from '@internal/local-target'; import { prismaCloudContainerOf } from '../container.ts'; import { daemonEntry } from './daemon-entry.ts'; @@ -43,7 +43,7 @@ export async function runDevTeardown(input: TeardownInput): Promise { // daemon would hand the old ports back; start it so the delete always lands. try { await ensureDaemon('compute', daemonEntry('compute')); - await deleteComputeInstances({ devDir, app }); + await deleteRecordedInstances(devDir); } finally { await tolerateUnreachable(() => bucketsClient().deleteApp(app)); removeLocalPaths([devDir, `${cwd}/.alchemy/state/${app}/dev`]); diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index d62ec416f..42a58f69f 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -886,6 +886,9 @@ importers: specifier: ^7.5.21 version: 7.5.21 devDependencies: + '@internal/foundation': + specifier: workspace:0.29.1 + version: link:../../../0-framework/0-foundation/foundation '@internal/tsdown-config': specifier: workspace:0.29.1 version: link:../../../0-framework/0-foundation/tsdown-config From 4ce59e6e8d9359ce65d2b91aeea9249a1a6aa4ef Mon Sep 17 00:00:00 2001 From: willbot Date: Tue, 13 Oct 2026 17:11:07 +0200 Subject: [PATCH 12/17] docs(design): state what the emulator actually receives from Composer The claim that the emulator's state holds no app, address or directory outside labels was false for every real deployment: the artifact directory lies under the project and env keys are named after addresses. ADR-0052 and local-dev.md now say what holds: the emulator never interprets Composer's data, which reaches it only as label values and inside the deployment payload it stores verbatim, and no route, state key, file name or emulator-generated field carries it. local-dev.md also notes that a bulk stop skips unknown ids and that creating an instance is never retried. Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- docs/design/10-domains/local-dev.md | 12 +++++++++--- ...-its-nodes-to-emulator-instances-per-directory.md | 2 +- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/docs/design/10-domains/local-dev.md b/docs/design/10-domains/local-dev.md index 090a51730..31ca0d494 100644 --- a/docs/design/10-domains/local-dev.md +++ b/docs/design/10-domains/local-dev.md @@ -92,9 +92,15 @@ It holds **instances**, each a slot for one service process. A client creates an instance with free-form labels and an optional requested port; the emulator generates the id (`svc_<16 hex digits>`), stores the labels verbatim and never reads them. Every later call names the id: put a deployment, start -or stop a list of instances, delete one, read or follow its log, get one, or -list them all. The emulator's routes and state hold no app name, node address -or directory outside a label, and a test fails if one appears. +or stop a list of instances (a stop skips ids it does not have, since they run +nothing; a start fails naming them), delete one, read or follow its log, get +one, or list them all. Creating an instance is the one call the client never +retries, because a retry after a lost response would create a second +instance. The emulator never interprets Composer's data: an app name, a +node address or a directory reaches it only as a label value or inside the +deployment payload (the artifact directory and the env), which it stores +verbatim and hands to the process. No route, state key, file name or field the +emulator generates carries one, and a test fails if one does. **Ports.** An instance gets its port when it is created and keeps it until it is deleted, running or stopped. A requested port is used when no instance diff --git a/docs/design/90-decisions/ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md b/docs/design/90-decisions/ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md index 65be0e0a1..aa9b4c9b3 100644 --- a/docs/design/90-decisions/ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md +++ b/docs/design/90-decisions/ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md @@ -25,7 +25,7 @@ This amends [ADR-0041](ADR-0041-local-dev-runs-the-deploy-pipeline-against-local `prisma dev` runs an app against local emulators: a Compute emulator that runs service processes, a Postgres emulator and a bucket emulator. Each is a long-lived daemon shared by everything on the machine. ADR-0041 identified an app's instances on those daemons by the app's name, so the emulators' routes and state were shaped around apps: `/apps//services/`. Two directories that run the same app, such as two git worktrees of one repository, therefore resolved to the same instances. Each `prisma dev` redeployed over the other's services. That is a common situation, not an edge case: agents work in many worktrees at once and start and stop environments in each. -The emulators should not have to know what a directory or an app is to avoid this. So an instance belongs to whoever created it. A client creates an instance, the emulator generates its id, and every later operation names that id: deploy, start, stop, delete, logs. The emulator's API and its state contain no app name, node address or directory, and a test fails if one appears there outside a label. The emulators can then be used by something other than Composer, and moved out of this repository, without carrying Composer's model along. +The emulators should not have to know what a directory or an app is to avoid this. So an instance belongs to whoever created it. A client creates an instance, the emulator generates its id, and every later operation names that id: deploy, start, stop, delete, logs. The emulator never interprets Composer's data. An app name, a node address or a directory reaches it only as a label value, or inside the deployment it is handed to run: the artifact directory, which lies under the project, and the environment, whose keys are named after addresses. The emulator stores both verbatim and passes them to the process it spawns. No route, state key, file name or field the emulator generates carries any of them, and a test fails if one does. The emulators can then be used by something other than Composer, and moved out of this repository, without carrying Composer's model along. Labels make a listing readable. Composer writes `composer.directory`, `composer.app` and `composer.address` on every instance it creates, so a person looking at the emulator's list of instances can tell what started each one. Nothing reads them back as identity. If Composer found its instances by searching labels, the labels would be a second, unenforced key. From dea7fbbb49ad90ed2b45acac7855b69a1635c495 Mon Sep 17 00:00:00 2001 From: willbot Date: Tue, 13 Oct 2026 17:11:33 +0200 Subject: [PATCH 13/17] docs(drive): state what the emulator receives from Composer; record deferred items Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- .drive/projects/emulator-commands/deferred.md | 2 ++ .../emulator-commands/slices/s1a-compute-instances/spec.md | 2 +- .drive/projects/emulator-commands/spec.md | 2 +- 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/.drive/projects/emulator-commands/deferred.md b/.drive/projects/emulator-commands/deferred.md index 57dc36212..f78e3206e 100644 --- a/.drive/projects/emulator-commands/deferred.md +++ b/.drive/projects/emulator-commands/deferred.md @@ -2,3 +2,5 @@ - **A start can join a crash-backoff restart that has already exited (prisma/composer#353).** When the fifth fast exit marks an instance `held`, the readiness probe only notices the exit after its current connection attempt. A start request arriving in that gap joins the dying start and gets its old failure, so nothing resumes. Seen once as an intermittent failure of "a held service resumes on start" in `compute-deployment.test.ts`; not reproduced in 10 runs under CPU load. Likely fix: a start must not join an in-flight start whose process has already exited. Belongs in #353, not this project. - **What a previous app's databases and buckets become when another app runs from the same directory.** Compute instances are deleted (S1a). Databases and buckets hold data; decide in S1b. +- **`[dev]` notices bypass the CLI engine's output.** The emulators hook writes `[dev] compute emulator ready`, the port-change line and the app-switch line with `console.log` to stdout, so a piped `prisma dev` mixes them into the JSON frames. Fixing it needs a channel for the local target's hooks to report through the engine. +- **A deployment that arrives while its instance is being deleted can start a process the emulator no longer tracks.** Same race as #353's app delete. diff --git a/.drive/projects/emulator-commands/slices/s1a-compute-instances/spec.md b/.drive/projects/emulator-commands/slices/s1a-compute-instances/spec.md index bdbf6dca3..adc700b86 100644 --- a/.drive/projects/emulator-commands/slices/s1a-compute-instances/spec.md +++ b/.drive/projects/emulator-commands/slices/s1a-compute-instances/spec.md @@ -72,7 +72,7 @@ Deliberately out: ## Slice-specific done conditions - [ ] Two temporary directories run the same fixture app at once. Each has its own Compute instances, and neither directory's services restart when the other deploys. This is proven by a test against a real Compute emulator in a temporary `PRISMA_COMPOSER_EMULATORS_DIR`. -- [ ] A test fails if the Compute emulator's source or state stores an app name, address or directory anywhere except label values. +- [ ] A test fails if the Compute emulator's routes, state keys, file names or generated fields carry an app name, address or directory, or if it alters label values or the deployment payload. ## References diff --git a/.drive/projects/emulator-commands/spec.md b/.drive/projects/emulator-commands/spec.md index ce5fe4cac..689c299ad 100644 --- a/.drive/projects/emulator-commands/spec.md +++ b/.drive/projects/emulator-commands/spec.md @@ -74,7 +74,7 @@ The shared emulator daemons themselves keep running; other directories may be us - [ ] Two directories running the same app at once get separate instances, and `prisma dev url gateway` in each returns that directory's own service. - [ ] After Ctrl-C of `prisma dev`, or `prisma dev stop` following a `kill -9`, no process of that directory's app is running; the next `prisma dev` gets the same ports. - [ ] `prisma emulator list` shows the services, databases and buckets of both, each labelled with the directory and address that started it. -- [ ] No emulator source or state file contains an app name, address or directory outside a label value. +- [ ] No emulator route, state key, file name or emulator-generated field carries an app name, address or directory. Composer data reaches an emulator only as label values and inside the deployment payload, stored verbatim and never interpreted. - [ ] Composer ADRs record the emulator model (amending ADR-0041) and the new commands (amending ADR-0050). ADR-0044 lists the new codes. - [ ] `docs/design/10-domains/local-dev.md`, the running-locally guide and the core-concepts skill describe both commands and the model. - [ ] prisma-cli mounts both commands, with `EXCLUSIONS` entries in the e2e coverage test (local only, no management API). From a99555f9b13a77c8056f83adbc91ee8a242b7071 Mon Sep 17 00:00:00 2001 From: willbot Date: Tue, 13 Oct 2026 17:26:06 +0200 Subject: [PATCH 14/17] test(local-target): skip the real-emulator instance tests on Windows The tests in emulator-instances.test.ts start a real Compute emulator, and local dev does not support Windows: on windows-latest the emulator fails to clear a service log that is still open (EPERM, ftruncate). Run them under describe.skipIf(process.platform === 'win32'), as the CLI's dev() and log() tests already do. The two error-rewriting tests that start no emulator stay outside it and still run on Windows. Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- .../src/__tests__/emulator-instances.test.ts | 480 +++++++++--------- 1 file changed, 244 insertions(+), 236 deletions(-) diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts index fcf6588a5..17f1a1e5e 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts @@ -34,21 +34,6 @@ const savedEmulatorsDir = process.env[EMULATORS_DIR_ENV]; let root: string; let registryRoot: string; -beforeEach(async () => { - root = fs.mkdtempSync(path.join(os.tmpdir(), 'local-target-emulator-instances-')); - registryRoot = path.join(root, 'emulators'); - process.env[EMULATORS_DIR_ENV] = registryRoot; - const entry = fileURLToPath(import.meta.resolve('@internal/dev-emulators/compute-main')); - await ensureDaemon('compute', entry, { registryRoot }); -}); - -afterEach(async () => { - await stopDaemon('compute', { registryRoot }).catch(() => undefined); - fs.rmSync(root, { recursive: true, force: true }); - if (savedEmulatorsDir === undefined) delete process.env[EMULATORS_DIR_ENV]; - else process.env[EMULATORS_DIR_ENV] = savedEmulatorsDir; -}); - const container: ContainerInstance = { input: { appName: APP, stage: undefined }, serialize: () => 'x', @@ -145,271 +130,294 @@ async function pidOf(id: string): Promise { return (await computeClient().getInstance(id))?.pid; } -describe('two directories running the same app', () => { - test('each gets its own instances, and a deploy in one restarts nothing in the other', async () => { - const a = projectDir('a'); - const b = projectDir('b'); - await Promise.all([ - ensureServiceInstances(container, a.devDir, ADDRESSES), - ensureServiceInstances(container, b.devDir, ADDRESSES), - ]); - await Promise.all( - [a, b].flatMap(({ devDir }) => ADDRESSES.map((address) => deploy(devDir, address, 'v1'))), +describe('errors name services by address', () => { + const nodes = [{ address: 'shop.quotes', instance: { id: 'svc_0123456789abcdef' } }]; + + test('an instance id in a plain error becomes the service address, keeping the original as cause', () => { + const original = new Error( + 'request failed (500): instance "svc_0123456789abcdef" exited (code 1) before it accepted connections', ); - const nodesA = (await mappingOf(a.directory))?.nodes ?? {}; - const nodesB = (await mappingOf(b.directory))?.nodes ?? {}; - const ids = [...Object.values(nodesA), ...Object.values(nodesB)].map((n) => n.id); - const ports = [...Object.values(nodesA), ...Object.values(nodesB)].map((n) => n.port); - expect(new Set(ids).size).toBe(4); - expect(new Set(ports).size).toBe(4); - for (const [{ directory }, nodes] of [ - [a, nodesA], - [b, nodesB], - ] as const) { - for (const [address, node] of Object.entries(nodes)) { - const answer = await (await fetch(`http://localhost:${String(node.port)}`)).text(); - expect(answer).toStartWith(`${address} ${fs.realpathSync(directory)}`); - } - } + const rewritten = withServiceAddresses(original, nodes); - const pidsA = await Promise.all(Object.values(nodesA).map((n) => pidOf(n.id))); - await Promise.all(ADDRESSES.map((address) => deploy(b.devDir, address, 'v2'))); + expect(rewritten).toBeInstanceOf(Error); + expect((rewritten as Error).message).toBe( + 'request failed (500): service "shop.quotes" exited (code 1) before it accepted connections', + ); + expect((rewritten as Error).cause).toBe(original); + }); - expect(await Promise.all(Object.values(nodesA).map((n) => pidOf(n.id)))).toEqual(pidsA); - const quotesB = await nodeOf(b.directory, 'quotes'); - expect(await (await fetch(`http://localhost:${String(quotesB.port)}`)).text()).toEndWith(' v2'); - }, 60_000); -}); + test('a structured error is left untouched', () => { + const structured = new CliStructuredError( + 'DEV.EMULATOR_UNREACHABLE', + 'instance "svc_0123456789abcdef" unreachable', + ); -describe('reusing the recorded instances', () => { - test('a recorded instance the emulator no longer has is re-created on its remembered port', async () => { - const { directory, devDir } = projectDir('lost'); - await ensureServiceInstances(container, devDir, ['quotes']); - const before = await nodeOf(directory, 'quotes'); - await computeClient().deleteInstance(before.id); + expect(withServiceAddresses(structured, nodes)).toBe(structured); + }); +}); - const changes = await ensureServiceInstances(container, devDir, ['quotes']); +// Local dev does not support Windows, so neither do the tests that start a real Compute emulator. +describe.skipIf(process.platform === 'win32')('against a real Compute emulator', () => { + beforeEach(async () => { + root = fs.mkdtempSync(path.join(os.tmpdir(), 'local-target-emulator-instances-')); + registryRoot = path.join(root, 'emulators'); + process.env[EMULATORS_DIR_ENV] = registryRoot; + const entry = fileURLToPath(import.meta.resolve('@internal/dev-emulators/compute-main')); + await ensureDaemon('compute', entry, { registryRoot }); + }); - const after = await nodeOf(directory, 'quotes'); - expect(after.id).not.toBe(before.id); - expect(after.port).toBe(before.port); - expect(changes).toEqual([]); + afterEach(async () => { + await stopDaemon('compute', { registryRoot }).catch(() => undefined); + fs.rmSync(root, { recursive: true, force: true }); + if (savedEmulatorsDir === undefined) delete process.env[EMULATORS_DIR_ENV]; + else process.env[EMULATORS_DIR_ENV] = savedEmulatorsDir; }); - test('a remembered port another instance now holds gives a new port, reported as a change', async () => { - const { directory, devDir } = projectDir('moved'); - await ensureServiceInstances(container, devDir, ['quotes']); - const before = await nodeOf(directory, 'quotes'); - await computeClient().deleteInstance(before.id); - await computeClient().createInstance({ labels: {}, port: before.port }); + describe('two directories running the same app', () => { + test('each gets its own instances, and a deploy in one restarts nothing in the other', async () => { + const a = projectDir('a'); + const b = projectDir('b'); + await Promise.all([ + ensureServiceInstances(container, a.devDir, ADDRESSES), + ensureServiceInstances(container, b.devDir, ADDRESSES), + ]); + await Promise.all( + [a, b].flatMap(({ devDir }) => ADDRESSES.map((address) => deploy(devDir, address, 'v1'))), + ); + + const nodesA = (await mappingOf(a.directory))?.nodes ?? {}; + const nodesB = (await mappingOf(b.directory))?.nodes ?? {}; + const ids = [...Object.values(nodesA), ...Object.values(nodesB)].map((n) => n.id); + const ports = [...Object.values(nodesA), ...Object.values(nodesB)].map((n) => n.port); + expect(new Set(ids).size).toBe(4); + expect(new Set(ports).size).toBe(4); + for (const [{ directory }, nodes] of [ + [a, nodesA], + [b, nodesB], + ] as const) { + for (const [address, node] of Object.entries(nodes)) { + const answer = await (await fetch(`http://localhost:${String(node.port)}`)).text(); + expect(answer).toStartWith(`${address} ${fs.realpathSync(directory)}`); + } + } - const changes = await ensureServiceInstances(container, devDir, ['quotes']); + const pidsA = await Promise.all(Object.values(nodesA).map((n) => pidOf(n.id))); + await Promise.all(ADDRESSES.map((address) => deploy(b.devDir, address, 'v2'))); - const after = await nodeOf(directory, 'quotes'); - expect(after.port).not.toBe(before.port); - expect(changes).toEqual([{ address: 'quotes', previous: before.port, given: after.port }]); + expect(await Promise.all(Object.values(nodesA).map((n) => pidOf(n.id)))).toEqual(pidsA); + const quotesB = await nodeOf(b.directory, 'quotes'); + expect(await (await fetch(`http://localhost:${String(quotesB.port)}`)).text()).toEndWith( + ' v2', + ); + }, 60_000); }); - test('a warm restart after a stop reuses the same instances and ports', async () => { - const { directory, devDir } = projectDir('warm'); - await ensureServiceInstances(container, devDir, ADDRESSES); - for (const address of ADDRESSES) await deploy(devDir, address, 'v1'); - const before = (await mappingOf(directory))?.nodes; - const ids = Object.values(before ?? {}).map((n) => n.id); - await computeClient().stopInstances(ids); - - const changes = await ensureServiceInstances(container, devDir, ADDRESSES); - await computeClient().startInstances(ids); - - expect(changes).toEqual([]); - expect((await mappingOf(directory))?.nodes).toEqual(before ?? {}); - for (const node of Object.values(before ?? {})) { - expect((await computeClient().getInstance(node.id))?.status).toBe('running'); - } - }, 30_000); - - test('a deployment whose instance was re-created is deployed again even when nothing changed', async () => { - const { directory, devDir } = projectDir('redeploy'); - await ensureServiceInstances(container, devDir, ['quotes']); - const deployed = await deploy(devDir, 'quotes', 'v1'); - expect(await deploymentDiff(devDir, deployed)).toBeUndefined(); + describe('reusing the recorded instances', () => { + test('a recorded instance the emulator no longer has is re-created on its remembered port', async () => { + const { directory, devDir } = projectDir('lost'); + await ensureServiceInstances(container, devDir, ['quotes']); + const before = await nodeOf(directory, 'quotes'); + await computeClient().deleteInstance(before.id); + + const changes = await ensureServiceInstances(container, devDir, ['quotes']); + + const after = await nodeOf(directory, 'quotes'); + expect(after.id).not.toBe(before.id); + expect(after.port).toBe(before.port); + expect(changes).toEqual([]); + }); + + test('a remembered port another instance now holds gives a new port, reported as a change', async () => { + const { directory, devDir } = projectDir('moved'); + await ensureServiceInstances(container, devDir, ['quotes']); + const before = await nodeOf(directory, 'quotes'); + await computeClient().deleteInstance(before.id); + await computeClient().createInstance({ labels: {}, port: before.port }); + + const changes = await ensureServiceInstances(container, devDir, ['quotes']); + + const after = await nodeOf(directory, 'quotes'); + expect(after.port).not.toBe(before.port); + expect(changes).toEqual([{ address: 'quotes', previous: before.port, given: after.port }]); + }); + + test('a warm restart after a stop reuses the same instances and ports', async () => { + const { directory, devDir } = projectDir('warm'); + await ensureServiceInstances(container, devDir, ADDRESSES); + for (const address of ADDRESSES) await deploy(devDir, address, 'v1'); + const before = (await mappingOf(directory))?.nodes; + const ids = Object.values(before ?? {}).map((n) => n.id); + await computeClient().stopInstances(ids); + + const changes = await ensureServiceInstances(container, devDir, ADDRESSES); + await computeClient().startInstances(ids); + + expect(changes).toEqual([]); + expect((await mappingOf(directory))?.nodes).toEqual(before ?? {}); + for (const node of Object.values(before ?? {})) { + expect((await computeClient().getInstance(node.id))?.status).toBe('running'); + } + }, 30_000); - await computeClient().deleteInstance((await nodeOf(directory, 'quotes')).id); - await ensureServiceInstances(container, devDir, ['quotes']); + test('a deployment whose instance was re-created is deployed again even when nothing changed', async () => { + const { directory, devDir } = projectDir('redeploy'); + await ensureServiceInstances(container, devDir, ['quotes']); + const deployed = await deploy(devDir, 'quotes', 'v1'); + expect(await deploymentDiff(devDir, deployed)).toBeUndefined(); - expect(await deploymentDiff(devDir, deployed)).toEqual({ action: 'update' }); - }, 30_000); + await computeClient().deleteInstance((await nodeOf(directory, 'quotes')).id); + await ensureServiceInstances(container, devDir, ['quotes']); - test("deleting a removed service's App deletes its instance and forgets it", async () => { - const { directory, devDir } = projectDir('removed'); - await ensureServiceInstances(container, devDir, ADDRESSES); - const quotes = await nodeOf(directory, 'quotes'); - const provider = await appProvider(devDir); + expect(await deploymentDiff(devDir, deployed)).toEqual({ action: 'update' }); + }, 30_000); - await Effect.runPromise(provider.delete({ output: { name: 'quotes' } } as never)); + test("deleting a removed service's App deletes its instance and forgets it", async () => { + const { directory, devDir } = projectDir('removed'); + await ensureServiceInstances(container, devDir, ADDRESSES); + const quotes = await nodeOf(directory, 'quotes'); + const provider = await appProvider(devDir); - expect(await computeClient().getInstance(quotes.id)).toBeUndefined(); - expect(Object.keys((await mappingOf(directory))?.nodes ?? {})).toEqual(['gateway']); - }); -}); + await Effect.runPromise(provider.delete({ output: { name: 'quotes' } } as never)); -test('the record is one JSON file in the dev directory, readable from the directory alone', async () => { - const { directory, devDir } = projectDir('file'); - await ensureServiceInstances(container, devDir, ['quotes']); - - const onDisk: unknown = JSON.parse( - fs.readFileSync(path.join(devDir, EMULATOR_INSTANCES_FILE), 'utf8'), - ); - const node = await nodeOf(directory, 'quotes'); - expect(onDisk).toEqual({ app: APP, nodes: { quotes: node } }); - expect((await computeClient().getInstance(node.id))?.labels).toEqual({ - 'composer.directory': directory, - 'composer.app': APP, - 'composer.address': 'quotes', + expect(await computeClient().getInstance(quotes.id)).toBeUndefined(); + expect(Object.keys((await mappingOf(directory))?.nodes ?? {})).toEqual(['gateway']); + }); }); - expect(await readEmulatorInstances(path.join(root, 'never-ran'))).toBeUndefined(); -}); - -describe('switching apps in a directory', () => { - test("running another app deletes the previous app's compute instances and starts a new record", async () => { - const { directory, devDir } = projectDir('switch'); - await ensureServiceInstances(container, devDir, ADDRESSES); - const previous = Object.values((await mappingOf(directory))?.nodes ?? {}); - - const retired = await retirePreviousApp({ devDir, app: 'another-app' }); - expect(retired).toBe(APP); - for (const node of previous) expect(await computeClient().getInstance(node.id)).toBeUndefined(); - expect(await mappingOf(directory)).toEqual({ app: 'another-app', nodes: {} }); - }); - - test('running the same app again retires nothing', async () => { - const { directory, devDir } = projectDir('same'); + test('the record is one JSON file in the dev directory, readable from the directory alone', async () => { + const { directory, devDir } = projectDir('file'); await ensureServiceInstances(container, devDir, ['quotes']); - const quotes = await nodeOf(directory, 'quotes'); - expect(await retirePreviousApp({ devDir, app: APP })).toBeUndefined(); - expect(await computeClient().getInstance(quotes.id)).toBeDefined(); + const onDisk: unknown = JSON.parse( + fs.readFileSync(path.join(devDir, EMULATOR_INSTANCES_FILE), 'utf8'), + ); + const node = await nodeOf(directory, 'quotes'); + expect(onDisk).toEqual({ app: APP, nodes: { quotes: node } }); + expect((await computeClient().getInstance(node.id))?.labels).toEqual({ + 'composer.directory': directory, + 'composer.app': APP, + 'composer.address': 'quotes', + }); + expect(await readEmulatorInstances(path.join(root, 'never-ran'))).toBeUndefined(); }); -}); -describe('--fresh', () => { - const otherApp: ContainerInstance = { - input: { appName: 'other-app', stage: undefined }, - serialize: () => 'x', - }; + describe('switching apps in a directory', () => { + test("running another app deletes the previous app's compute instances and starts a new record", async () => { + const { directory, devDir } = projectDir('switch'); + await ensureServiceInstances(container, devDir, ADDRESSES); + const previous = Object.values((await mappingOf(directory))?.nodes ?? {}); - async function liveIds(ids: readonly string[]): Promise { - const live = await Promise.all(ids.map((id) => computeClient().getInstance(id))); - return live.flatMap((instance) => (instance ? [instance.id] : [])); - } + const retired = await retirePreviousApp({ devDir, app: 'another-app' }); - test('deletes the recorded instances even when the record names the app run here before', async () => { - const { directory, devDir } = projectDir('fresh-previous'); - await ensureServiceInstances(container, devDir, ADDRESSES); - const ids = Object.values((await mappingOf(directory))?.nodes ?? {}).map((n) => n.id); + expect(retired).toBe(APP); + for (const node of previous) + expect(await computeClient().getInstance(node.id)).toBeUndefined(); + expect(await mappingOf(directory)).toEqual({ app: 'another-app', nodes: {} }); + }); - await deleteRecordedInstances(devDir); + test('running the same app again retires nothing', async () => { + const { directory, devDir } = projectDir('same'); + await ensureServiceInstances(container, devDir, ['quotes']); + const quotes = await nodeOf(directory, 'quotes'); - expect(await liveIds(ids)).toEqual([]); - expect((await mappingOf(directory))?.nodes).toEqual({}); + expect(await retirePreviousApp({ devDir, app: APP })).toBeUndefined(); + expect(await computeClient().getInstance(quotes.id)).toBeDefined(); + }); }); - test('after switching apps, leaves no instance of either app', async () => { - const { directory, devDir } = projectDir('fresh-switch'); - await ensureServiceInstances(container, devDir, ADDRESSES); - const first = Object.values((await mappingOf(directory))?.nodes ?? {}).map((n) => n.id); - await retirePreviousApp({ devDir, app: 'other-app' }); - await ensureServiceInstances(otherApp, devDir, ['web']); - const second = Object.values((await mappingOf(directory))?.nodes ?? {}).map((n) => n.id); - - await deleteRecordedInstances(devDir); + describe('--fresh', () => { + const otherApp: ContainerInstance = { + input: { appName: 'other-app', stage: undefined }, + serialize: () => 'x', + }; - expect(await liveIds([...first, ...second])).toEqual([]); - }); -}); + async function liveIds(ids: readonly string[]): Promise { + const live = await Promise.all(ids.map((id) => computeClient().getInstance(id))); + return live.flatMap((instance) => (instance ? [instance.id] : [])); + } -describe("the App provider's diff", () => { - async function reconcileApp(devDir: string, address: string): Promise { - const provider = await appProvider(devDir); - return Effect.runPromise( - provider.reconcile( - reconcileInput(`${address}-svc`, { project: 'local', displayName: address }), - ), - ); - } + test('deletes the recorded instances even when the record names the app run here before', async () => { + const { directory, devDir } = projectDir('fresh-previous'); + await ensureServiceInstances(container, devDir, ADDRESSES); + const ids = Object.values((await mappingOf(directory))?.nodes ?? {}).map((n) => n.id); - async function appDiff(devDir: string, output: App['Attributes']): Promise { - const diff = (await appProvider(devDir)).diff; - if (!diff) throw new Error('the local App provider has no diff'); - return Effect.runPromise(diff({ output } as never)); - } + await deleteRecordedInstances(devDir); - test('asks for an update when the re-created instance got another port', async () => { - const { directory, devDir } = projectDir('app-moved'); - await ensureServiceInstances(container, devDir, ['quotes']); - const before = await reconcileApp(devDir, 'quotes'); - const recorded = await nodeOf(directory, 'quotes'); - await computeClient().deleteInstance(recorded.id); - await computeClient().createInstance({ labels: {}, port: recorded.port }); - await ensureServiceInstances(container, devDir, ['quotes']); + expect(await liveIds(ids)).toEqual([]); + expect((await mappingOf(directory))?.nodes).toEqual({}); + }); - expect(await appDiff(devDir, before)).toEqual({ action: 'update' }); - const after = await reconcileApp(devDir, 'quotes'); - expect(after.appEndpointDomain).toBe( - `http://localhost:${String((await nodeOf(directory, 'quotes')).port)}`, - ); - expect(after.appEndpointDomain).not.toBe(before.appEndpointDomain); - }); + test('after switching apps, leaves no instance of either app', async () => { + const { directory, devDir } = projectDir('fresh-switch'); + await ensureServiceInstances(container, devDir, ADDRESSES); + const first = Object.values((await mappingOf(directory))?.nodes ?? {}).map((n) => n.id); + await retirePreviousApp({ devDir, app: 'other-app' }); + await ensureServiceInstances(otherApp, devDir, ['web']); + const second = Object.values((await mappingOf(directory))?.nodes ?? {}).map((n) => n.id); - test('asks for nothing when the re-created instance kept its port', async () => { - const { directory, devDir } = projectDir('app-kept'); - await ensureServiceInstances(container, devDir, ['quotes']); - const before = await reconcileApp(devDir, 'quotes'); - await computeClient().deleteInstance((await nodeOf(directory, 'quotes')).id); - await ensureServiceInstances(container, devDir, ['quotes']); + await deleteRecordedInstances(devDir); - expect(await appDiff(devDir, before)).toBeUndefined(); + expect(await liveIds([...first, ...second])).toEqual([]); + }); }); -}); -describe('errors name services by address', () => { - const nodes = [{ address: 'shop.quotes', instance: { id: 'svc_0123456789abcdef' } }]; - - test('an instance id in a plain error becomes the service address, keeping the original as cause', () => { - const original = new Error( - 'request failed (500): instance "svc_0123456789abcdef" exited (code 1) before it accepted connections', - ); - - const rewritten = withServiceAddresses(original, nodes); - - expect(rewritten).toBeInstanceOf(Error); - expect((rewritten as Error).message).toBe( - 'request failed (500): service "shop.quotes" exited (code 1) before it accepted connections', - ); - expect((rewritten as Error).cause).toBe(original); - }); + describe("the App provider's diff", () => { + async function reconcileApp(devDir: string, address: string): Promise { + const provider = await appProvider(devDir); + return Effect.runPromise( + provider.reconcile( + reconcileInput(`${address}-svc`, { project: 'local', displayName: address }), + ), + ); + } - test('a structured error is left untouched', () => { - const structured = new CliStructuredError( - 'DEV.EMULATOR_UNREACHABLE', - 'instance "svc_0123456789abcdef" unreachable', - ); + async function appDiff(devDir: string, output: App['Attributes']): Promise { + const diff = (await appProvider(devDir)).diff; + if (!diff) throw new Error('the local App provider has no diff'); + return Effect.runPromise(diff({ output } as never)); + } - expect(withServiceAddresses(structured, nodes)).toBe(structured); + test('asks for an update when the re-created instance got another port', async () => { + const { directory, devDir } = projectDir('app-moved'); + await ensureServiceInstances(container, devDir, ['quotes']); + const before = await reconcileApp(devDir, 'quotes'); + const recorded = await nodeOf(directory, 'quotes'); + await computeClient().deleteInstance(recorded.id); + await computeClient().createInstance({ labels: {}, port: recorded.port }); + await ensureServiceInstances(container, devDir, ['quotes']); + + expect(await appDiff(devDir, before)).toEqual({ action: 'update' }); + const after = await reconcileApp(devDir, 'quotes'); + expect(after.appEndpointDomain).toBe( + `http://localhost:${String((await nodeOf(directory, 'quotes')).port)}`, + ); + expect(after.appEndpointDomain).not.toBe(before.appEndpointDomain); + }); + + test('asks for nothing when the re-created instance kept its port', async () => { + const { directory, devDir } = projectDir('app-kept'); + await ensureServiceInstances(container, devDir, ['quotes']); + const before = await reconcileApp(devDir, 'quotes'); + await computeClient().deleteInstance((await nodeOf(directory, 'quotes')).id); + await ensureServiceInstances(container, devDir, ['quotes']); + + expect(await appDiff(devDir, before)).toBeUndefined(); + }); }); - test('a deployed service that exits before it listens fails naming its address', async () => { - const { devDir } = projectDir('crash'); - await ensureServiceInstances(container, devDir, ['quotes']); + describe('errors name services by address', () => { + test('a deployed service that exits before it listens fails naming its address', async () => { + const { devDir } = projectDir('crash'); + await ensureServiceInstances(container, devDir, ['quotes']); - const failure = await deploy(devDir, 'quotes', 'v1', crashingArtifactFor('quotes')).then( - () => undefined, - (error: unknown) => (error instanceof Error ? error.message : String(error)), - ); + const failure = await deploy(devDir, 'quotes', 'v1', crashingArtifactFor('quotes')).then( + () => undefined, + (error: unknown) => (error instanceof Error ? error.message : String(error)), + ); - expect(failure).toContain('service "quotes" exited (code 1) before it accepted connections'); - expect(failure).not.toContain('instance "svc_'); - }, 30_000); + expect(failure).toContain('service "quotes" exited (code 1) before it accepted connections'); + expect(failure).not.toContain('instance "svc_'); + }, 30_000); + }); }); From f309007218d6170656f98d4871f3e5a7008313ea Mon Sep 17 00:00:00 2001 From: willbot Date: Wed, 14 Oct 2026 08:37:01 +0200 Subject: [PATCH 15/17] test(local-target): the emulator's listens hint is rewritten to the compute() option A plain emulator error that names an instance and suggests deploying it with `listens: false` comes back naming the service by address and suggesting `compute({ listens: false })`. Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- .../src/__tests__/emulator-instances.test.ts | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts index 17f1a1e5e..51ee8d033 100644 --- a/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts +++ b/packages/1-prisma-cloud/0-lowering/local-target/src/__tests__/emulator-instances.test.ts @@ -147,6 +147,18 @@ describe('errors name services by address', () => { expect((rewritten as Error).cause).toBe(original); }); + test("the emulator's listens hint becomes the compute() option, alongside the address", () => { + const original = new Error( + 'request failed (500): instance "svc_0123456789abcdef" did not accept connections on its port 3001 within 30s. If it never listens on its port, deploy it with listens: false.', + ); + + const rewritten = withServiceAddresses(original, nodes); + + expect((rewritten as Error).message).toBe( + 'request failed (500): service "shop.quotes" did not accept connections on its port 3001 within 30s. If it never listens on its port, declare it with compute({ listens: false }).', + ); + }); + test('a structured error is left untouched', () => { const structured = new CliStructuredError( 'DEV.EMULATOR_UNREACHABLE', From 2bc73cfc74d07a4d175c577d73eb5553bb015dc2 Mon Sep 17 00:00:00 2001 From: willbot Date: Thu, 15 Oct 2026 17:13:37 +0200 Subject: [PATCH 16/17] test(dev-emulators): readiness tests take ports no other test file uses Two readiness tests assert that a stopped instance's port refuses connections. Other test files' Compute emulators give ports from 3000 at the same time, and on Linux CI another file's service answered on the same port. Each instance in this file now asks for a currently free port from 3400 to 3599, a range only this file uses, and checks it got it. Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- .../src/__tests__/compute-readiness.test.ts | 58 ++++++++++++------- 1 file changed, 36 insertions(+), 22 deletions(-) diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts index a7be99088..29114185d 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts @@ -8,6 +8,7 @@ import { afterEach, beforeEach, describe, expect, test } from 'bun:test'; import * as net from 'node:net'; import { fileURLToPath } from 'node:url'; +import getPort, { portNumbers } from 'get-port'; import { type ComputeClient, computeClient, type InstanceInfo } from '../client.ts'; import { ensureDaemon, isPidAlive, SERVICE_READY_TIMEOUT_ENV, stopDaemon } from '../daemon.ts'; import { @@ -17,7 +18,6 @@ import { ensureFreshDaemon, NEVER_LISTENING_BOOTSTRAP, servingBootstrapEnv, - skipContendedServicePorts, tempDir, waitFor, writeBootstrap, @@ -54,9 +54,23 @@ function acceptsConnection(port: number): Promise { async function startDaemon(entry?: string): Promise { registryRoot = tempDir('compute-readiness-registry'); await ensureFreshDaemon('compute', registryRoot, entry); - const client = computeClient({ registryRoot }); - await skipContendedServicePorts(client); - return client; + return computeClient({ registryRoot }); +} + +/** + * This file's ports. Other test files run their own Compute emulators at the + * same time and give ports from 3000, and several tests here assert that a + * stopped instance's port refuses connections: a port another file's service + * holds would make that fail. So each instance here asks for a currently free + * port from a range only this file uses. + */ +const OWN_PORTS = portNumbers(3400, 3599); + +async function createInstanceOnOwnPort(client: ComputeClient): Promise { + const port = await getPort({ port: OWN_PORTS }); + const instance = await client.createInstance({ labels: {}, port }); + expect(instance.port).toBe(port); + return instance; } async function instanceOf(client: ComputeClient, id: string): Promise { @@ -78,7 +92,7 @@ describe('with the default readiness timeout', () => { test('a deployment returns only once a slow-booting service accepts connections', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); const deployed = client.putDeployment(reserved.id, { artifactDir, @@ -102,7 +116,7 @@ describe('with the default readiness timeout', () => { test('a session-resume start returns only once a slow-booting service accepts connections', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); await client.putDeployment(reserved.id, { artifactDir, artifactHash: 'h', @@ -123,7 +137,7 @@ describe('with the default readiness timeout', () => { test('a service that exits before it listens fails the deployment, naming the service and its log', async () => { const artifactDir = writeBootstrap(CRASHING_BOOTSTRAP); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); const deployed = client.putDeployment(reserved.id, { artifactDir, @@ -140,7 +154,7 @@ describe('with the default readiness timeout', () => { test('a session-resume start of a service that exits before it listens fails, naming the service', async () => { const artifactDir = writeBootstrap(CRASHING_BOOTSTRAP); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); await client .putDeployment(reserved.id, { artifactDir, @@ -165,7 +179,7 @@ describe('a service that does not listen on its port', () => { test('is running once spawned, without waiting for its port', async () => { const artifactDir = writeBootstrap(NEVER_LISTENING_BOOTSTRAP); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); const started = Date.now(); await client.putDeployment(reserved.id, { @@ -205,7 +219,7 @@ describe('a start that is replaced, stopped or joined', () => { test('a redeploy of a running service is not running until the new process listens', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); await client.putDeployment( reserved.id, delayedDeployment(reserved.port, 'v1', 'first', artifactDir), @@ -227,7 +241,7 @@ describe('a start that is replaced, stopped or joined', () => { test('a repeated identical deployment during a redeploy waits for the new process', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); await client.putDeployment( reserved.id, delayedDeployment(reserved.port, 'v1', 'first', artifactDir), @@ -251,7 +265,7 @@ describe('a start that is replaced, stopped or joined', () => { test('a session-resume start during a redeploy waits for the new process', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); await client.putDeployment( reserved.id, delayedDeployment(reserved.port, 'v1', 'first', artifactDir), @@ -271,7 +285,7 @@ describe('a start that is replaced, stopped or joined', () => { test('two concurrent session-resume starts share one process', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); await client.putDeployment( reserved.id, delayedDeployment(reserved.port, 'v1', 'once', artifactDir), @@ -289,7 +303,7 @@ describe('a start that is replaced, stopped or joined', () => { test('a deployment replaced during its start waits for the replacement instead of failing', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); const first = client.putDeployment( reserved.id, @@ -310,7 +324,7 @@ describe('a start that is replaced, stopped or joined', () => { const artifactDir = writeBootstrap( `process.on('SIGTERM', () => {});\n${DELAYED_SERVING_BOOTSTRAP}`, ); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); const deployed = client .putDeployment(reserved.id, delayedDeployment(reserved.port, 'v1', 'first', artifactDir)) @@ -350,7 +364,7 @@ describe('a start that is replaced, stopped or joined', () => { test('stop, start, stop: a stop during a waiting start cancels it', async () => { const artifactDir = writeBootstrap(IGNORES_SIGTERM); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); await client.putDeployment(reserved.id, quickDeployment(reserved.port, artifactDir)); const firstStop = client.stopInstances([reserved.id]); @@ -367,7 +381,7 @@ describe('a start that is replaced, stopped or joined', () => { test('an identical deployment during a stop waits for the stop and starts the service again', async () => { const artifactDir = writeBootstrap(IGNORES_SIGTERM); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); await client.putDeployment(reserved.id, quickDeployment(reserved.port, artifactDir)); const stopped = client.stopInstances([reserved.id]); @@ -383,7 +397,7 @@ describe('a start that is replaced, stopped or joined', () => { test('a start during an instance delete leaves no process behind', async () => { const artifactDir = writeBootstrap(IGNORES_SIGTERM); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); await client.putDeployment(reserved.id, quickDeployment(reserved.port, artifactDir)); const deleted = client.deleteInstance(reserved.id); @@ -412,7 +426,7 @@ describe('a start that is replaced, stopped or joined', () => { test('a deployment stopped during its start fails saying it was stopped', async () => { const artifactDir = writeBootstrap(DELAYED_SERVING_BOOTSTRAP); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); const deployed = client.putDeployment( reserved.id, @@ -434,7 +448,7 @@ describe('with a 1-second readiness timeout', () => { fileURLToPath(new URL('./fixtures/short-ready-timeout.ts', import.meta.url)), ); const artifactDir = writeBootstrap(NEVER_LISTENING_BOOTSTRAP); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); const deployed = client.putDeployment(reserved.id, { artifactDir, @@ -459,7 +473,7 @@ describe('with a 1-second readiness timeout', () => { fileURLToPath(new URL('./fixtures/short-ready-timeout.ts', import.meta.url)), ); const artifactDir = writeBootstrap(CRASH_ONCE_THEN_SILENT_BOOTSTRAP); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); await expect( client.putDeployment(reserved.id, { @@ -490,7 +504,7 @@ describe(`with ${SERVICE_READY_TIMEOUT_ENV}`, () => { process.env[SERVICE_READY_TIMEOUT_ENV] = '1000'; const client = await startDaemon(); const artifactDir = writeBootstrap(NEVER_LISTENING_BOOTSTRAP); - const reserved = await client.createInstance({ labels: {} }); + const reserved = await createInstanceOnOwnPort(client); const deployed = client.putDeployment(reserved.id, { artifactDir, From 21c630868e6fd4de6f4eff1cf8cb1020ec429061 Mon Sep 17 00:00:00 2001 From: willbot Date: Thu, 15 Oct 2026 17:41:49 +0200 Subject: [PATCH 17/17] test(dev-emulators): drop a repeated assertion in the session-resume race test The test checked twice that the first deployment failed with "exited (code 1)"; the second copy was left by porting the test. Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- .../dev-emulators/src/__tests__/compute-readiness.test.ts | 1 - 1 file changed, 1 deletion(-) diff --git a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts index 9681ddab5..a3b22929b 100644 --- a/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts +++ b/packages/1-prisma-cloud/0-lowering/dev-emulators/src/__tests__/compute-readiness.test.ts @@ -530,7 +530,6 @@ describe('with a 3-second readiness poll', () => { expect(await statusOf(client, reserved.id)).toBe('running'); expect(await acceptsConnection(reserved.port)).toBe(true); expect(await first).toContain('exited (code 1)'); - expect(await first).toContain('exited (code 1)'); }, 45_000); test('an identical deployment just after the deployed process exits starts it again', async () => {