diff --git a/apps/lsp-playground/package.json b/apps/lsp-playground/package.json index 5db2a18eda73..4900131eccf6 100644 --- a/apps/lsp-playground/package.json +++ b/apps/lsp-playground/package.json @@ -35,7 +35,7 @@ "ws": "^8.21.2" }, "devDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@types/node": "catalog:", "@types/ws": "^8.5.13", diff --git a/apps/telemetry-backend/package.json b/apps/telemetry-backend/package.json index 5f6aca9e5fd2..a623b66493a6 100644 --- a/apps/telemetry-backend/package.json +++ b/apps/telemetry-backend/package.json @@ -27,7 +27,7 @@ "evlog": "^2.24.0" }, "devDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@prisma/compute-sdk": "^0.43.0", "@prisma/management-api-sdk": "^1.76.0", "@repo/test-utils": "workspace:8.0.0-rc.16", diff --git a/docs/CLI Style Guide.md b/docs/CLI Style Guide.md index e4309b07cf04..702f81796a6e 100644 --- a/docs/CLI Style Guide.md +++ b/docs/CLI Style Guide.md @@ -117,23 +117,29 @@ The CLI checks `process.stdout.isTTY` once at startup to determine the output mo ### Destructive operation confirmation -Destructive operations (drops, type changes, overwriting generated files, overwriting an existing signature marker, …) require **explicit consent**: the user types the name of the thing being changed, or passes that name as `--confirm `. Consent is separate from the `-y` "accept prompt defaults" mechanism, and no command invents a flag that skips it. +Operations that lose data (dropping a table or a column, a type change that can change values), operations that widen who can read or write rows, overwriting generated files, and overwriting an existing signature marker require **explicit consent**. Dropping an object that holds no data, such as an index or a constraint, is `widening` and asks nothing. Consent takes one of two forms: + +- **Statements**, for a plan that would lose data or widen access. The command asks one question per model, field or storage name at stake, and each is answered with a statement naming it: `--delete `, `--rename :` or `--allow `. `migration plan` and `db update` use this form. +- **A consent token**, for a command with one thing at stake. The user types the name of the thing being changed, or passes that name as `--confirm `. `init` uses this form. + +Consent is separate from the `-y` "accept prompt defaults" mechanism, and no command invents a flag that skips it. This is a deliberate divergence from clig.dev §Arguments §Confirmation. AI agents and CI scripts routinely pass `-y` to suppress prompts on long-running pipelines; conflating "skip prompts" with "consent to destruction" is a footgun the project has decided to avoid. Typing a name also rules out the other accident a yes/no prompt allows — consenting to the right operation against the wrong database. #### Rules -- A command that performs a destructive action MUST ask for it with `ctx.prompt.consent(question, { token })`. The token is the natural noun of what is at stake: the database name for `db update`, the working directory's basename for an `init` re-scaffold. Interactively the user types the token; non-interactively `--confirm ` grants it. There is no `--force`. +- A command whose plan can lose data or widen access MUST ask about each subject with `ctx.prompt.statements(questions)`, one question per subject, before it writes or applies anything. A statement that answers no question fails the run with the engine's `CLI.CONSENT_UNUSED`; non-interactively, unanswered questions fail it with `CLI.CONSENT_REQUIRED`, listing each question and the flags that answer it. `--confirm` answers none of these questions. +- A command with one thing at stake MUST ask for it with `ctx.prompt.consent(question, { token })`. The token is the natural noun of what is at stake: the working directory's basename for an `init` re-scaffold. Interactively the user types the token; non-interactively `--confirm ` grants it. There is no `--force`. - The question MUST list the destructive operations (or describe them concretely, e.g. "this will overwrite all generated files") so the user can decline knowing what's at stake. A command that cannot fill the list, or cannot derive a token, MUST fail with a structured error rather than ask a question that says nothing or accepts anything. - The token MUST identify the thing being changed as precisely as the invocation allows. Falling back to something a whole class of runs shares — a target id, a product name — makes one `--confirm` value grant data loss in every project that shares it. - Non-interactively (closed stdin, CI, `--no-interactive`) without `--confirm`: no prompt is shown; the command fails with the engine's `CLI.CONSENT_REQUIRED` (exit `2`), whose `meta.consentToken` and `nextActions` name the token to pass. A cancelled prompt is `CLI.PROMPT_CANCELLED` (exit `3`); a mistyped token is `CLI.PROMPT_INVALID` (exit `2`). - `--confirm` is read only when the run is non-interactive or `--yes` is set. A script that runs from a terminal must pass `--no-interactive --confirm `, or it will stop at the prompt. - Each `--confirm` value grants at most one consent; a command that asks twice needs two. -- The internal control API retains a programmatic equivalent (e.g. `acceptDataLoss: boolean`) for consumers that drive the planner directly; `--confirm` is the user-facing CLI form. Note that consent authorises the operations the user was shown, and the plan is recomputed on the call that carries it — a command that cannot bind the two SHOULD report what it applied beyond what was consented to. +- The control API takes the same consent programmatically: `migration plan` and `db update` take an answer callback for their questions, and `db update`'s `acceptDataLoss: true` answers every data-loss question and `acceptAccessWidening: true` every access question. #### Examples -- `db update`: when the plan includes destructive ops, asks the user to type the database name; `--no-interactive --confirm ` applies without a prompt. The name is the `database` a driver connection object carries, or the connection URL's first path segment, else its host, falling back to the target id. +- `db update`: asks one question per model, field or storage name an operation would lose data from, answered by `--delete` or `--rename`, and one per model whose rows an operation would open up, answered by `--allow`; `--no-interactive --delete Legacy --allow User` applies without a prompt. - `init`: re-running `init` in a directory with a generated `prisma.config.ts` asks the user to type the directory's basename; `-y` alone is not sufficient to authorise overwriting generated files. (The commander-era `--force` retired with the commander shell in the S5 cutover; the engine-hosted `init` uses the consent form above.) - `db sign`: overwriting a marker that holds a different hash happens without consent (the previous hash is reported). This is an intentional exception to the rule above: `db sign` verifies the live schema against the contract before writing, so the overwrite only ever records a contract the database already satisfies. If that ever grows a switch, it takes the consent form above — not a `--force`. diff --git a/docs/architecture docs/adrs/ADR 195 - Planner IR with two renderers.md b/docs/architecture docs/adrs/ADR 195 - Planner IR with two renderers.md index 32a622307a9c..60c424f11d09 100644 --- a/docs/architecture docs/adrs/ADR 195 - Planner IR with two renderers.md +++ b/docs/architecture docs/adrs/ADR 195 - Planner IR with two renderers.md @@ -1,5 +1,7 @@ # ADR 195 — Planner IR with two renderers (OpFactoryCall pattern) +> **Note (2026-10-07).** A MongoDB validator change and a change-stream image setting are now classed `widening` whatever they change, because neither removes a document; the planner no longer runs `classifyValidatorUpdate`. The planner still chooses the class where it depends on the origin: a Postgres type change is `widening` when it keeps every value and `destructive` otherwise, so `AlterColumnTypeCall` carries a computed `operationClass` too. + ## At a glance The planner diffs two contracts and determines that a unique ascending index on `users.email` needs to be created. Rather than directly constructing the full operation ([ADR 188](ADR%20188%20-%20MongoDB%20migration%20operation%20model.md)), it produces an IR node: diff --git a/docs/architecture docs/subsystems/7. Migration System.md b/docs/architecture docs/subsystems/7. Migration System.md index 03b17c642469..231d9e9dbffd 100644 --- a/docs/architecture docs/subsystems/7. Migration System.md +++ b/docs/architecture docs/subsystems/7. Migration System.md @@ -124,7 +124,7 @@ Two renderers consume the IR: A scaffolded SQL `migration.ts` writes SQL text as strings, and prettier formats the file with `singleQuote: true`, which picks whichever quote the text does not contain. A text that holds both `'` and `"` on one line, such as `"kind" IN ('admin', 'user')`, would still need escaped quotes, so the TypeScript renderer writes it as an untagged template literal instead (`tsQuotedTextSource` in `@internal/ts-render`). This covers every place that [Opaque SQL in DDL](#opaque-sql-in-ddl) lists, except the `USING` conversion of `alterColumnType`, which the planner never writes. The column defaults include the function default that the Postgres `setDefault` and the SQLite `addColumn` and `recreateTable` write; a literal default is a JSON value and is written as JSON. It also covers SQL text that the planner builds whole: the postcheck SQL of the SQLite `recreateTable`. A text with a line break, a control character or a lone surrogate stays a string literal. The SQL is the same either way, so `ops.json` does not change. -Planner-derived semantics (`operationClass`, `label`) ride on the IR because only the planner has the origin context needed for classification (e.g., whether a `collMod` is `widening` vs `destructive`). The renderers read these values; they don't re-derive them. +Planner-derived semantics (`operationClass`, `label`) ride on the IR because only the planner has the origin context needed for classification (e.g., whether a Postgres type change keeps every value, which makes it `widening`, or not, which makes it `destructive`). The renderers read these values; they don't re-derive them. ### Offline planning via contract-to-schema @@ -140,7 +140,7 @@ The canonical contract IR stays planner-agnostic and hash-stable; it only encode A table rename is a core operation the author states: `this.renameTable({ table, to })` in a hand-written migration returns the table rename plus a rename of every object whose name the planner derives from the table name, and the planner refuses a drop-and-create pair that is the signature of the older storage-naming rule with `MIGRATION.TABLE_NAME_CASE_CHANGED`. See [ADR 264 — A model names its storage verbatim, and a rename is an operation](../adrs/ADR%20264%20-%20A%20model%20names%20its%20storage%20verbatim,%20and%20a%20rename%20is%20an%20operation.md). -Additive structure is covered by core operations: create table, add nullable column, add index/unique, add foreign key, set column default (SQL); create collection, create index, set validation (Mongo). A rename needs a statement (see [Statements](#statements)), and a destructive change needs a policy that allows it; the planner fails fast without them. See [ADR 028 — Migration Structure & Operations](../adrs/ADR%20028%20-%20Migration%20Structure%20&%20Operations.md) and [ADR 161 — Explicit FK constraint and index configuration](../adrs/ADR%20161%20-%20Explicit%20foreign%20key%20constraint%20and%20index%20configuration.md). +Core operations cover additive structure: create table, add nullable column, add index/unique, add foreign key, set column default (SQL); create collection, create index (Mongo). Setting a Mongo validation is a core operation too, classed `widening` (see [Operation Model](#operation-model)). A rename needs a statement (see [Statements](#statements)), and a destructive change needs a policy that allows it; the planner fails fast without them. See [ADR 028 — Migration Structure & Operations](../adrs/ADR%20028%20-%20Migration%20Structure%20&%20Operations.md) and [ADR 161 — Explicit FK constraint and index configuration](../adrs/ADR%20161%20-%20Explicit%20foreign%20key%20constraint%20and%20index%20configuration.md). ### `migration plan` @@ -170,19 +170,31 @@ When `--to` is omitted, `to_contract` is the emitted contract; when `--to` is su An auto-baseline dates the delta package at the current time and the baseline package one minute earlier, so the baseline sorts first. Folder names have minute resolution, so packages planned within the same minute sort by name. +**Data loss.** Before it writes either package, including the baseline of an auto-baseline, `migration plan` asks what each operation that would lose data means (see [The questions](#statements)). It does not ask about access: a written package is reviewed before it is applied. + The diff never compares constraint names, so the planner compares names itself: a primary key, unique constraint or foreign key that the start tree and the end contract both have, otherwise unchanged, is renamed with `RENAME CONSTRAINT` when its start name and end name differ, each the stated name or the one the planner derives. The start tree comes from the start contract for `migration plan` and from the database for `db update`, and `renameTable` applies the same rule across a table rename. The runner skips a rename when the database already has the new name ([`constraint-name-renames.ts`](../../../packages/3-targets/3-targets/postgres/src/core/migrations/constraint-name-renames.ts)). Plan-time refuse diagnostics name the resolved hash, list refs pointing at graph nodes, and suggest `--from `. Snapshot-missing diagnostics suggest `db update --advance-ref ` to repopulate or `migration ref delete ` to clear an orphan pointer. ### Statements -The origin and destination contracts cannot say that a model or field was renamed rather than dropped and replaced. The user states it on the command line, and the planner takes the statements as its third input, beside the destination contract and the origin schema. Statements are never part of `contract.json` or its hashes. +The origin and destination contracts cannot say whether a model or field that is gone was renamed or meant to be deleted, nor whether an operation that opens up a model's rows is meant. The user says so on the command line with statements, each a repeatable flag: + +- `--rename :` keeps a model's or field's data under a new name. Renames are the only statements that reach the planner, which takes them as its third input, beside the destination contract and the origin schema. +- `--delete ` lets the plan lose a subject's data. +- `--allow ` lets `db update` change who can read or write a model's rows, one operation at a time. + +Statements are never part of `contract.json` or its hashes. ```bash -prisma migration plan --name tidy-users --rename Profile:User --rename User.name:User.fullName -prisma db update --rename Profile:User --rename User.name:User.fullName +prisma migration plan --name tidy-users --rename Profile:User --rename User.name:User.fullName --delete Legacy +prisma db update --rename Profile:User --rename User.name:User.fullName --delete Legacy --allow User ``` +**The questions.** The planner's success result lists what the plan would lose (`dataLoss`) and whose access it would widen (`accessWidening`), each entry an operation's position and its subject (a `MigrationSubject`: a model, a field, or a storage name no model of the origin contract stores). Before `migration plan` writes anything, and before `db update` applies anything, the command asks one question per data-loss subject, answered by `delete` or, for a model or field the destination no longer has, `rename`. Where the target's migrations capability sets `renameStatements: { refused: true, keepDataByHand }`, as MongoDB's does, the question offers no rename and ends with the target's text on keeping the data by hand (renaming the collection in `mongosh` on each database before a plan that drops it is applied there; `db update` then drops nothing, and a migration written by `migration plan` still drops it, so remove that operation or do not apply it there). `db update` also asks one question per operation that changes who can read or write a model's rows, answered by `allow`: the question says the operation widens access when it does, such as disabling row-level security, and that it changes access otherwise, such as dropping a policy, since a permissive policy grants access. Two such operations on one model are two questions, each answered by its own `--allow`. The questions go through the CLI engine's statement prompt (`ctx.prompt.statements(questions, { last: true })`), which answers them from the flags first. In a terminal it asks the rest in turn, and the user types `delete`, `allow` or `rename :`; a typed rename is planned again with the other renames, and a plan that still has the operation asked about fails with `MIGRATION.STATEMENT_DID_NOT_RESOLVE_LOSS`; a loss the new plan has on the same subject, such as a type change on the renamed field, is asked next. Where nobody can answer (not interactive, or `--yes`), the command fails with `CLI.CONSENT_REQUIRED`, listing every unanswered question with the flags that answer it, and a `--delete` or `--allow` that answers no question fails with `CLI.CONSENT_UNUSED`; both happen before anything is written or applied. A `db update --dry-run` asks nothing; it checks the same flags against the subjects an apply would ask about and fails an unmatched one with `MIGRATION.STATEMENT_ANSWERS_NO_QUESTION`, so a script that checks the code handles both. `--confirm` answers none of these questions. The `--rename` values are taken from the run first (`ctx.statements.take('rename')`) and planned from the start, so the drop a rename replaces is never asked about. The shared question loop is `askPlanQuestions` in [`plan-questions.ts`](../../../packages/1-framework/3-tooling/cli/src/control-api/statements/plan-questions.ts); the control API takes the answers through a callback, so it never depends on the engine's prompt. + +#### Rename statements + **Grammar.** `--rename :` can be given more than once. Each side is one of `Model`, `namespace.Model`, `Model.field` or `namespace.Model.field`, and both sides name the same kind of thing. A field is named through its model as the destination contract names it, so the field statement above names `User`, not `Profile`. A field cannot move to another model. The CLI parses each statement in [`parse-rename.ts`](../../../packages/1-framework/3-tooling/cli/src/control-api/statements/parse-rename.ts); a malformed statement fails with `MIGRATION.STATEMENT_INVALID`, and so does a field statement that names the model by its old name, with the corrected statement in the message. **Resolution.** The framework resolves each statement against the origin and destination contracts of the application contract space, in [`resolve-statements.ts`](../../../packages/1-framework/3-tooling/cli/src/control-api/statements/resolve-statements.ts). The old name must be in the origin and not in the destination; the new name must be in the destination and not in the origin. A resolved statement is a `ResolvedMigrationStatement` in domain coordinates (namespace, model and field), with no table or column names, so the framework stays family-blind. A statement that does not resolve fails with `MIGRATION.STATEMENT_UNRESOLVED`. The framework passes the resolved statements to the family planner as `statements` in the `plan` options; the aggregate planner passes them to the application space only, through `PlannerInput.appSpace`. @@ -193,22 +205,30 @@ prisma db update --rename Profile:User --rename User.name:User.fullName - A field statement whose column name changes renames the column, and renames the constraints and indexes whose names derive from it. These are the operations `this.renameColumn({ table, column, to })` returns. SQLite cannot rename an index, so there a renamed index is dropped and created. On Postgres, a check, a row-level-security policy, or an expression or partial index whose SQL text names the column is dropped and created again: Postgres rewrites that text when it renames the column, but the working copy keeps the old text, so the diff sees a changed object. Adding the check back reads every row of the table while holding an `ACCESS EXCLUSIVE` lock, and an index is rebuilt, so on a large table such a column rename is not instant. - A statement whose storage name does not change, because `@@map` or `@map` keeps it, or because the field is a relation, produces no operations. -Rename operations are `widening`, so neither command asks for consent. The `migration.ts` that `migration plan` writes contains `...this.renameTable(...)` and `...this.renameColumn(...)` calls, as a user would write them by hand, and running it again writes the same `ops.json` and `migration.json`. +Rename operations are `widening`, so neither command asks about them. The `migration.ts` that `migration plan` writes contains `...this.renameTable(...)` and `...this.renameColumn(...)` calls, as a user would write them by hand, and running it again writes the same `ops.json` and `migration.json`. **Result.** The planner's success result carries `appliedStatements` (`AppliedMigrationStatement`): one entry per statement, in order, with `operationIndexes`: the positions, in the plan's operations, of the operations the statement accounts for, empty when its storage did not change. Statements point at operations by position because operation ids are not unique within a plan (on Postgres, same-named tables in two schemas share them), and ids stay as they are so that re-emitting an existing migration does not change its hash. The framework's `describeMigrationStatement` writes the statement in domain names, such as `rename model "Profile" to "User"`, for every family's messages and for the CLI. `migration plan` and `db update` print each statement with its description and number of operations, and their JSON output carries the description and the operation positions, which index the output's `operations`. Messages name a model of the unbound namespace by its name alone, and JSON output leaves `namespaceId` out of a coordinate in the unbound namespace (`migrationStatementJson`), because that namespace's id is internal. **Refusals.** When the planner cannot carry out a resolved statement, planning fails with `MIGRATION.PLANNING_FAILED` and a conflict of kind `statementRefused` that carries the statement in its `refusedStatement` field. The [error reference](../../reference/error-reference.md#migrationplanning_failed) lists the cases. -**The origin of `db update`.** `migration plan` already has an origin contract: the one `--from` resolves to. `db update` introspects the database instead, so it reads an origin contract only when statements are given. It reads the contract for the marker's hash from the [snapshot store](#contract-resolution-through-the-snapshot-store). Without statements it does not read the snapshot store, and with or without them the plan it hands the runner keeps `origin: null` (see [`db update`](#db-update-live-reconciliation)). `db update` stores a snapshot only when it advances a ref: the `db` ref by default, or with `--db ` only the ref named by `--advance-ref `. So a database last updated with `--db ` and no `--advance-ref` has no snapshot, and statements fail there with `MIGRATION.STATEMENT_ORIGIN_UNKNOWN`, which names the hash and the directory it looked in. Running `db update` again with the same statements fails with `MIGRATION.STATEMENT_UNRESOLVED`, because the origin is then a contract without the old names. - -**Limits in this release.** +**Rename limits in this release.** - A model move to another namespace resolves, but the planner refuses it with `statementRefused`. - A model that keeps its name but changes its table cannot be renamed by a statement, because the model has the same name on both sides. Rename the table by hand with `this.renameTable(...)` in its own `migration.ts`. A field statement on such a model is refused with the same advice. - Only models and fields can be renamed. Value objects, their fields, enum values and namespaces cannot. - The MongoDB planner refuses every statement with `statementRefused`, so `migration plan` and `db update` with `--rename` fail with `MIGRATION.PLANNING_FAILED` and plan nothing, rather than dropping and creating the collection. -The rest of the migration-statements work adds `--delete` and refuses plans that lose data by default; adds `--convert`, `--backfill`, the remaining renames and model moves across namespaces; and brings statements to MongoDB. +#### Delete and allow statements + +**Delete statements.** `--delete ` consents to losing the data of a subject the planner's `dataLoss` names: a model or field of the origin contract written as a statement names it (`Legacy`, `User.name`, with the namespace when the origin contract has several), or the storage name of data no model stores. A field of a model the plan renames is written through the model's new name, as a field rename is written (`--delete User.nickname` after `--rename Profile:User`); the CLI matches answers by subject, not by its text. A delete never reaches a planner: the CLI matches it, by equality, against the subjects of the plan's `dataLoss` entries. It needs no origin contract, so with no snapshot a rename fails with `MIGRATION.STATEMENT_ORIGIN_UNKNOWN` and a delete still works. The CLI writes a delete as its question named the subject (`consentDescription` in [`plan-questions.ts`](../../../packages/1-framework/3-tooling/cli/src/control-api/statements/plan-questions.ts)): `delete model "Legacy"`, `delete field "User.nickname"` after a model rename, or `delete storage "audit_log"`, and `migration plan` and `db update` list it under `Statements applied` with the positions of the operations it consented to; in JSON each `appliedStatements` entry carries its `verb`. + +**Allow statements.** `--allow ` consents to one operation the planner's `accessWidening` names (a `MigrationAccessChange`, whose `widens` says whether it widens access or only changes it): disabling row-level security, or dropping a policy that no other operation of the plan creates again, on Postgres. A policy whose generated name changes with its body is created again when the plan creates a policy with the same name prefix on the same table, so editing its body asks nothing, as editing a policy with a stated name does. Only `db update` asks about access: `migration plan` writes a package, and the package is reviewed before it is applied. Like a delete, an allow never reaches a planner and is matched by subject; the CLI writes it the same way, as `allow model "User"` or `allow storage "user"`. + +#### The origin of `db update` + +`migration plan` already has an origin contract: the one `--from` resolves to. `db update` introspects the database instead, and on every run reads the contract for the marker's hash from the [snapshot store](#contract-resolution-through-the-snapshot-store). The planner gets it as `fromContract`, and names each subject of `dataLoss` and `accessWidening` through it. A missing or unreadable snapshot fails only a run with `--rename`; otherwise each subject is named by its storage name. The plan it hands the runner keeps `origin: null` either way (see [`db update`](#db-update-live-reconciliation)). The planner's operations do not depend on whether the snapshot was read: every strategy that emits a data transform needs the `data` operation class, which `db update` does not allow, and codec field-event hooks get the prior contract only when the plan has an origin, which a `db update` plan never has. `db update` stores a snapshot only when it advances a ref: the `db` ref by default, or with `--db ` only the ref named by `--advance-ref `. So a database last updated with `--db ` and no `--advance-ref` has no snapshot, and renames fail there with `MIGRATION.STATEMENT_ORIGIN_UNKNOWN`, which names the hash and the directory it looked in. Running `db update` again with the same statements fails with `MIGRATION.STATEMENT_UNRESOLVED`, because the origin is then a contract without the old names. + +The rest of the migration-statements work adds `--convert`, `--backfill`, the remaining renames and model moves across namespaces; and brings statements to MongoDB. ## Authoring Surface @@ -422,7 +442,14 @@ See [ADR 199 — Storage-only migration identity](../adrs/ADR%20199%20-%20Storag ## Operation Model -Every operation has a class that says what it can do to the data (`MigrationOperationClass`). An operation is `destructive` when it loses data, such as dropping a table or a column, and only then. A rename, a relaxed constraint or a wider type, and a drop of an object that holds no data (an index; a unique, foreign-key or check constraint; a default; a native enum type; a row-level-security policy) or disabling row-level security are `widening`, so `db update` applies them without asking for consent. Dropping a policy or disabling row-level security widens who can read and write rows. Adding structure is `additive`, and transforming data is `data`. MongoDB still classes dropping an index as `destructive`. +Every operation has a class that says what it does to the data (`MigrationOperationClass`): + +- `additive` adds structure and leaves existing structure and data as they are. +- `widening` changes existing structure without losing data. An operation that cannot keep every value fails instead of losing it, so tightening a constraint is `widening`: `SET NOT NULL` fails on a NULL (Postgres checks first; a SQLite table rebuild fails on the copy), and a MongoDB validator, new or stricter, applies to later writes only. A rename, a relaxed constraint, a type change that keeps every value (`int2` to `int4` to `int8`, `float4` to `float8` on Postgres), a drop of an object that holds no data (an index; a unique, foreign-key or check constraint; a default; a native enum type; a row-level-security policy), disabling row-level security and changing MongoDB's change-stream image setting are `widening` too. Dropping a policy or disabling row-level security widens who can read and write rows. +- `destructive` can lose rows or values: dropping a table, a column or a collection, and any other type change, including a SQLite table rebuild that changes a column type. +- `data` reads and writes rows, such as a backfill or a type conversion. It runs what its author wrote, which may lose data by design; only the planner's own classification promises that an operation loses data exactly when it is `destructive`. + +`db update` applies `additive` operations, and `widening` operations that do not widen access, without asking. Before an apply it asks about each operation that would lose data and each that would widen who can read and write rows (see [`db update`](#db-update-live-reconciliation)). Each migration operation is a data envelope with three phases — `precheck[]`, `execute[]`, `postcheck[]`. The phases are composed from existing AST primitives (DDL commands, inspection commands, filter expressions for Mongo; SQL statements for Postgres) that serialize naturally to JSON. The envelope carries no behavior of its own; the semantic richness lives in the commands and expressions inside it. @@ -794,7 +821,9 @@ Greenfield, brownfield-conservative, and brownfield-incremental paths are suppor When run against the default dev database URL, `db update` implicitly advances the `db` ref — this is the primary mechanism for recording "the contract hash this dev database has been brought up to" on disk for offline planning. Same `--advance-ref` override and explicit-`--db` opt-out as `db init`. See [ADR 218](../adrs/ADR%20218%20-%20Refs%20with%20paired%20contract%20snapshots%20and%20universal%20graph-node%20invariant.md) (superseded in part — see its Status note) and [ADR 240](../adrs/ADR%20240%20-%20Contract%20snapshots%20live%20in%20a%20content-addressed%20store.md). -Unlike `db migrate`, which replays serialized edges, `db update` always works from a live introspection. When statements are given, it also reads the origin contract they resolve against from the snapshot store entry for the marker's hash; see [Statements](#statements). Plans produced by the planner carry `origin: null` (no prior-state assertion). The runner treats this as "trust the planner": operations are always applied regardless of marker state. This ensures `db update` can recover from schema drift — for example, when a DBA manually drops a column, the planner detects the missing column and re-adds it even though the contract marker already matches the destination hash. +Unlike `db migrate`, which replays serialized edges, `db update` always works from a live introspection. It also reads the origin contract from the snapshot store entry for the marker's hash, to name what an operation would lose and resolve rename statements; see [The origin of `db update`](#the-origin-of-db-update). Plans produced by the planner carry `origin: null` (no prior-state assertion). The runner treats this as "trust the planner": operations are always applied regardless of marker state. This ensures `db update` can recover from schema drift — for example, when a DBA manually drops a column, the planner detects the missing column and re-adds it even though the contract marker already matches the destination hash. + +**Questions before an apply.** An apply asks, through the CLI engine's statement prompt, one question per subject of the plan's `dataLoss`, answered by `--delete` or `--rename`, and one per subject of its `accessWidening`, answered by `--allow` (see [Statements](#statements)). It asks before it applies anything. Extension spaces that apply recorded migrations contribute a data-loss question for each destructive operation, named by its storage name, and no access question. A run where nobody can answer fails with `CLI.CONSENT_REQUIRED`; a `--delete` or `--allow` that answers no question fails with `CLI.CONSENT_UNUSED`. A dry run asks nothing and lists the questions as `dataLoss` and `accessWidening`, each with the subject's `text` as the question writes it. It takes the `--delete` and `--allow` values and hands them to the control API as statements, which refuses one that names no subject an apply would ask about with `MIGRATION.STATEMENT_ANSWERS_NO_QUESTION`. The control API's `dbUpdate` takes the same questions through a required `answerQuestions` callback, called at least once per apply, with an empty list when nothing is in question. A callback that answers too few questions, or with a verb or text its question rejects, is a bug in the caller and throws `InternalError`. `delete` and `allow` statements answer their questions without asking, and one that answers no question fails with `MIGRATION.STATEMENT_ANSWERS_NO_QUESTION`, in plan mode as in apply mode, before anything is applied; `acceptDataLoss: true` answers every data-loss question and `acceptAccessWidening: true` every access question. By contrast, `db migrate` constructs plans with `origin: { storageHash }` from the migration chain. When the marker matches the destination, the runner skips operations (the migration was already applied). This provides safe idempotent replays. diff --git a/docs/commands/SUMMARY.md b/docs/commands/SUMMARY.md index 0d02e7827743..1880946c6ef6 100644 --- a/docs/commands/SUMMARY.md +++ b/docs/commands/SUMMARY.md @@ -74,8 +74,8 @@ For SQL targets, plan mode also prints a DDL preview derived from planned operat The planner supports three operation classes: - **Additive**: Create tables, add columns, add indexes/constraints -- **Widening**: Relax nullability (NOT NULL → nullable) -- **Destructive**: Drop tables, drop columns, alter column types, tighten nullability +- **Widening**: Change existing structure without losing data: relax or tighten nullability, widen a column type, rename, drop an index or a constraint +- **Destructive**: Lose rows or values: drop tables, drop columns, change a column type in a way that can change values ### Scenario 3: Local contract divergent from remote database (conflicts) @@ -121,13 +121,14 @@ Both commands share the same flag surface: | `--db ` | Database connection string | | `--config ` | Path to `prisma.config.ts` | | `--dry-run` | Preview planned operations without applying | -| `-y, --yes` | Accept the declared default of every prompt. It cannot grant consent, so it never authorises `db update`'s destructive operations | -| `--confirm ` | Grant `db update`'s destructive consent without being asked. Read only when the run is non-interactive, so a script running from a terminal needs `--no-interactive --confirm ` | +| `-y, --yes` | Accept the declared default of every prompt. It answers none of `db update`'s questions | | `--json [format]` | Output as JSON (`object` format only) | | `-q, --quiet` | Quiet mode: errors only | | `-v, --verbose` | Verbose output: debug info, timings | | `--no-color` | Disable color output | +`db update` also takes statements, each repeatable, which answer the questions it asks before an apply: `--rename :` keeps a model's or field's data under a new name, `--delete ` lets the update lose a model's, field's or storage name's data, and `--allow ` lets it change who can read or write a model's rows, one operation per flag. Without them, an apply that would lose data or change access asks in a terminal, and fails with `CLI.CONSENT_REQUIRED` elsewhere. `--confirm` answers none of them. See the CLI README's `db update` section. + ## Programmatic API Both commands are available via the control client: @@ -144,16 +145,22 @@ const client = createControlClient({ // db init const initResult = await client.dbInit({ - contractIR: contractJson, + contract: contractJson, mode: 'apply', + migrationsDir: 'migrations', connection: databaseUrl, }); // db update const updateResult = await client.dbUpdate({ - contractIR: contractJson, - mode: 'plan', // or 'apply' + contract: contractJson, + mode: 'apply', // or 'plan' + migrationsDir: 'migrations', connection: databaseUrl, + answerQuestions: async (questions) => { + if (questions.length > 0) throw new Error('db update would lose data or widen access'); + return []; + }, }); await client.close(); diff --git a/docs/reference/error-reference.md b/docs/reference/error-reference.md index 7ea56b5f5e55..77952c36bbb8 100644 --- a/docs/reference/error-reference.md +++ b/docs/reference/error-reference.md @@ -79,13 +79,13 @@ The config module evaluated, but its default export was not created by the curre The migration-file CLI (`prisma migration`) received `--config` without a path argument, either a bare trailing `--config`, or `--config` immediately followed by another flag (e.g. `--config --dry-run`). The CLI fails fast instead of consuming the next flag as the config path or silently falling back to default config discovery. Payload: `nextToken` (present only when another flag followed `--config`). -### CLI.CONSENT_OPERATIONS_MISSING +### CLI.CONSENT_REQUIRED -`db update` was told the plan is destructive but was given no operations to name, so the consent prompt would have asked you to authorise a list of nothing. The command refuses instead of prompting. This is an inconsistency between the CLI and the control API rather than something your project can be wrong about; run `prisma db update --dry-run` to see the plan, and report the run. Payload: none. +Raised by the CLI engine, not by Prisma ORM's own code; the engine's entry is the reference: [CLI.CONSENT_REQUIRED](https://docs.prisma.io/docs/cli/error-reference#CLI.CONSENT_REQUIRED). `migration plan` and `db update` raise it when the plan would lose data, or for `db update` widen who can read or write rows, and the run has nobody to answer: it is not interactive, or `--yes` is set. The command asks one question per model, field or storage name whose data an operation would lose, and `db update` also one per model whose rows an operation would open up, such as disabling row-level security. The error lists every question still unanswered, with the flags that answer each: `--delete ` lets the plan lose that data, for a model or field `--rename :` keeps it under a new name, and `--allow ` lets `db update` widen access to that model's rows. Data no model of the origin contract stores can only be deleted. A field of a model the plan renames is named through the model's new name: after `--rename Profile:User`, the question about the dropped `nickname` field is answered with `--delete User.nickname` or `--rename User.nickname:User.handle`. Nothing is written or applied. `--confirm` does not answer these questions. Payload: `subject`, `verbs`, `unanswered` (each with its `subject` and `verbs`). -### CLI.CONSENT_TOKEN_UNRESOLVED +### CLI.CONSENT_UNUSED -`db update` could not derive a name for the database it is about to change, so there is nothing for the consent prompt to ask you to type, and an empty token would let a bare Enter (or `--confirm ""`) authorise data loss. The name comes from the `database` a driver connection object carries, or from the connection URL (its first path segment, else its host), falling back to the target id. Name the database in `db.connection` or pass `--db `. Payload: none. +Raised by the CLI engine; the engine's entry is the reference: [CLI.CONSENT_UNUSED](https://docs.prisma.io/docs/cli/error-reference#CLI.CONSENT_UNUSED). `migration plan` and `db update` raise it when a `--delete` or `--allow` value answers no question the plan asks, for example a mistyped name or a model the plan does not drop, before they write or apply anything. A `db update --dry-run` asks no questions; it checks its `--delete` and `--allow` values against the subjects an apply would ask about and refuses an unmatched one with `MIGRATION.STATEMENT_ANSWERS_NO_QUESTION` instead. ### CLI.CONTRACT_ARG_CONFLICT @@ -1374,10 +1374,6 @@ A `migration check` finding, carried as an `error` diagnostic on a completed run `this.renameColumn({ table, column, to })` in a hand-written migration does not match the migration's contracts: the migration has no start contract, the table does not exist at that point of the migration (or, with no `schema`, is declared in more than one namespace), the table has no column `column`, the table already has a column `to`, or the end contract has no column `to` on that table. Raised when the migration's operations are built, so `migration.ts` writes no `ops.json`. Order the rename calls in the sequence the renames happen and check the table, column and namespace names. Payload: `table`, `from`, `to`. -### MIGRATION.CONSENT_PLAN_MISMATCH - -An apply carrying consent was refused because the plan recomputed for it is not the plan that was consented to. `db update` recomputes the plan at apply time and compares its hash against the one the consent was given for; a mismatch means the schema, the contract, or the database moved in between, so applying would carry out operations nobody agreed to. Re-run the command and review the freshly planned operations before consenting again. Payload: `consentedPlanHash`, `planHash`. - ### MIGRATION.CONTRACT_DESERIALIZATION_FAILED A contract JSON on disk failed to deserialize into a valid contract: either a snapshot-store entry read while migration tooling resolved a contract at a ref or hash, or the emitted `contract.json` read as the fallback source by `db sign` / `db update --to` (invalid JSON, or a value that is not a JSON object). Re-emit the owning migration package (or re-run `prisma contract emit` for the emitted contract), or restore the file from version control. Payload: `filePath`, `message`. Also raised by `migration new` when the emitted `contract.json` fails to deserialize; that site has no meta and attaches the deserialization failure as `cause`. @@ -1422,10 +1418,6 @@ An extension descriptor publishes a `contractSpace` whose `headRef.hash` does no Runner-level failure during apply (`db init`, `db update`, `db migrate`): the plan's destination storage hash (or profile hash) does not match the destination contract handed to the runner alongside it. Indicates the plan and contract came from different emits. Payload: `planStorageHash`/`contractStorageHash` (or `planProfileHash`/`contractProfileHash`). -### MIGRATION.DESTRUCTIVE_CHANGES - -The planned operations include destructive changes (e.g. DROP) and the command was run without explicit consent. `db update` asks for that consent instead of failing: interactively it asks you to type the name of the database it is about to change, and outside an interactive terminal it is granted by `--confirm ` (`--yes` accepts declared prompt defaults and never grants consent; `--confirm` is read only when the run is non-interactive or `--yes` is set, so a script run from a terminal needs `--no-interactive --confirm `). The name is the `database` a driver connection object carries, or the connection URL's first path segment, else its host, falling back to the target id. A run with nobody to ask and no `--confirm` settles as `CLI.CONSENT_REQUIRED` at exit 2; a run whose prompt is cancelled settles as `CLI.PROMPT_CANCELLED` at exit 3. `--dry-run` never asks; it settles as this error instead. Use it to preview the operations first. `migration plan` raises the same refusal before writing an auto-baseline package (planned on an empty migrations directory from the `db` ref) whose operations would remove data when applied; there the consent token is the project directory name, so a non-interactive run passes `--no-interactive --confirm `, and a consented re-run that no longer plans the consented baseline settles as `MIGRATION.CONSENT_PLAN_MISMATCH`. Payload at the `migration plan` site: `destructiveOperations`, `planHash`. - ### MIGRATION.DIR_EXISTS `migration new`/`migration plan` refused to scaffold because the target migration directory already exists: each migration needs a unique directory. Pick a different `--name` or delete the existing directory. Payload: `dir`. @@ -1568,7 +1560,7 @@ An apply command (`db migrate`/`db update`) cannot find a path through the on-di ### MIGRATION.PLANNING_FAILED -Migration planning (typically during `db init`/`db update`) failed because of conflicts, e.g. the live database already contains objects that clash with the plan. The envelope aggregates each conflict's summary and suggested fix. Payload: `conflicts`; a conflict's `location` and `refusedStatement` leave out `namespaceId` for the unbound namespace, whose id is internal. A conflict of kind `statementRefused` refuses a `--rename` statement the planner cannot carry out, and carries that statement (`refusedStatement`, in namespace, model and field names): moving a model to another namespace (not supported in this release); renaming a table or column whose control policy is not `managed`; a model with no table; a table or column the starting schema does not have, or already has under the new name; a field that has a column on one side of the statement only; a field rename on a model whose table changes when no statement renames the table (the message gives the steps: change the contract so only the table name changes and emit it, create a migration with `prisma migration new --from `, add `...this.renameTable(...)` to its operations and run its `migration.ts`, then emit the final contract and plan the field rename with `--from` that migration; with `db update`: change the contract so only the table name changes and emit it, rename the table in the database yourself with the SQL the message gives, check that `db update --dry-run` plans no drop of a table or column and run `db update`, then emit the final contract and run `db update` with the field statement); a rename that produces an operation the command's operation policy does not allow; or any statement on MongoDB, whose planner refuses statements in this release and plans nothing. `db init` applies only additive operations, so when a conflict is an operation its policy refused (the conflict carries `refusedOperationClass`), such as adding a validator to a Mongo collection that already holds documents, the error's next action depends on the refused classes. When `db update` allows all of them (`widening` and `destructive`), the next action is `db update`, which applies them, asking you to confirm destructive ones. When one is `data`, which `db update` does not apply, the next action is `migration plan`, to plan a migration that `db migrate` then applies. +Migration planning (typically during `db init`/`db update`) failed because of conflicts, e.g. the live database already contains objects that clash with the plan. The envelope aggregates each conflict's summary and suggested fix. Payload: `conflicts`; a conflict's `location` and `refusedStatement` leave out `namespaceId` for the unbound namespace, whose id is internal. A conflict of kind `statementRefused` refuses a `--rename` statement the planner cannot carry out, and carries that statement (`refusedStatement`, in namespace, model and field names): moving a model to another namespace (not supported in this release); renaming a table or column whose control policy is not `managed`; a model with no table; a table or column the starting schema does not have, or already has under the new name; a field that has a column on one side of the statement only; a field rename on a model whose table changes when no statement renames the table (the message gives the steps: change the contract so only the table name changes and emit it, create a migration with `prisma migration new --from `, add `...this.renameTable(...)` to its operations and run its `migration.ts`, then emit the final contract and plan the field rename with `--from` that migration; with `db update`: change the contract so only the table name changes and emit it, rename the table in the database yourself with the SQL the message gives, check that `db update --dry-run` plans no drop of a table or column and run `db update`, then emit the final contract and run `db update` with the field statement); a rename that produces an operation the command's operation policy does not allow; or any statement on MongoDB, whose planner refuses statements in this release and plans nothing. `db init` applies only additive operations, so when a conflict is an operation its policy refused (the conflict carries `refusedOperationClass`), such as adding a validator to a Mongo collection that already holds documents, the error's next action depends on the refused classes. When `db update` allows all of them (`widening` and `destructive`), the next action is `db update`, which applies them, asking what each operation that would lose data means. When one is `data`, which `db update` does not apply, the next action is `migration plan`, to plan a migration that `db migrate` then applies. `migration plan` raises it when the planner produces no operations. The conflict's kind names the case, and its `why` is the next action. With an earlier contract (`noDatabaseChange`), the contract changed in a way the database does not see, for example after a Prisma upgrade that changes how `contract.json` records values. If you deploy with migrations, run `prisma migration new --from `, with the earlier contract's hash from the message, to write a migration with no operations, then `prisma db migrate`. If you manage the database with `db init` or `db update`, run `prisma db sign` on each database instead. With no earlier contract (`nothingToCreate`), the contract describes nothing to create; if the database needs something the contract does not describe, run `prisma migration new` and write the operations in `migration.ts` by hand. `migration new` writes a migration only to the emitted contract, so with `--to` the message says to emit the target first, and for a database managed with `db update` it names `prisma db sign `. When the migrations directory is empty and the `db` ref points at a contract that describes nothing to create (`nothingToBaseline`), there is no baseline migration to plan, and no command writes one with no operations yet; keep using `db update`. @@ -1656,6 +1648,14 @@ A `--from` reference cannot produce a contract: either a ref name has no pointer SQLite twin of `MIGRATION.POSTGRES_CONTROL_STACK_MISSING`: a `SqliteMigration` operation needing the control adapter was invoked on an instance constructed without a control stack (only introspection is valid in that form). Payload: `operation`. +### MIGRATION.STATEMENT_ANSWERS_NO_QUESTION + +It is the control API's form of `CLI.CONSENT_UNUSED`: the same condition, met where the engine does not check the flags. A program called the control API's `dbUpdate` or `executeMigrationPlanCommand` with `delete` or `allow` statements in `statements`, and some of them name no subject the plan asks about: no operation would lose that subject's data or, for `db update`, widen access to its rows. `migration plan` asks no access questions, so an `allow` given to it always fails. Such a statement would consent to nothing, so the call fails before anything is written or applied, in plan mode as in apply mode. `db update --dry-run` raises it too, because a dry run asks nothing and hands its `--delete` and `--allow` values to the control API to check; an apply's unused flag is the engine's `CLI.CONSENT_UNUSED`. Leave the statements out, or name a subject the plan asks about. Payload: `statements` (each `{ verb, text }` that answered nothing), `subjects` (the subjects the plan asks about). + +### MIGRATION.STATEMENT_DID_NOT_RESOLVE_LOSS + +A `rename` typed at a data-loss question of `migration plan` or `db update` was planned with the plan's other renames, and the plan still loses the data of the subject the question was about: the rename resolves, but it does not keep that data, for example because it renames a different model or field. Nothing is written or applied. Answer the question with `--delete ` if the data may be lost, or with a rename whose old name stores that data. Payload: `statement` (the rename's text), `verb` (`rename`), `subject`. + ### MIGRATION.STATEMENT_INVALID A `--rename :` statement is malformed or pairs two things that cannot be renamed into each other: it has no `:` or more than one, a side has an empty name or more than three names, it names a model on one side and a field on the other, it names fields of two different models (a field cannot move between models), or an earlier statement in the same command already renames the same model or field, or renames another one to the same new name. It also refuses a statement that renames a model or field to itself (if only `@map` or `@@map` changed, a plan without the statement drops and creates the table or column with its data: with `migration plan`, edit the planned `migration.ts` to rename it instead, with `...this.renameTable(...)` or `...this.renameColumn(...)` on SQL; with `db update`, rename it in the database yourself first, then run `db update`, which then finds nothing that loses data), and two statements that swap names, which one plan cannot do: statements resolve against contracts, so the message gives the three plans that make the swap: the first name to a temporary name, the second name to the first, then the temporary name to the second. Every message's next step is the statement to type: the accepted forms with examples, or for a model on one side and a field on the other, both statements the user may have meant. Each side is one of `Model`, `namespace.Model`, `Model.field` or `namespace.Model.field`. A field's model is named as the destination contract names it: when the old side names the model by its old name, as in `--rename Profile:User --rename Profile.name:User.fullName`, the message gives the corrected statement, `--rename User.name:User.fullName`, and says when the model also needs its own `--rename`. Payload: `statement` (its text) and `verb` (`rename`). diff --git a/examples/bundle-size/package.json b/examples/bundle-size/package.json index 7634038d20bc..620b06c44421 100644 --- a/examples/bundle-size/package.json +++ b/examples/bundle-size/package.json @@ -28,7 +28,7 @@ "pg": "catalog:" }, "devDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@prisma/dev": "catalog:", "@repo/test-utils": "workspace:8.0.0-rc.16", "@repo/tsconfig": "workspace:8.0.0-rc.16", diff --git a/examples/mongo-blog-leaderboard/package.json b/examples/mongo-blog-leaderboard/package.json index 205cae62f4a2..34fb85883efc 100644 --- a/examples/mongo-blog-leaderboard/package.json +++ b/examples/mongo-blog-leaderboard/package.json @@ -19,7 +19,7 @@ "mongodb": "catalog:" }, "devDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@types/node": "catalog:", "mongodb-memory-server": "catalog:", diff --git a/examples/mongo-demo/package.json b/examples/mongo-demo/package.json index ebe605c214a0..6434b5ccc4f2 100644 --- a/examples/mongo-demo/package.json +++ b/examples/mongo-demo/package.json @@ -24,7 +24,7 @@ "react-dom": "^19.2.8" }, "devDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/test-utils": "workspace:8.0.0-rc.16", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@types/node": "catalog:", diff --git a/examples/multi-extension-monorepo/package.json b/examples/multi-extension-monorepo/package.json index fc19f687a336..f9d23f8491e7 100644 --- a/examples/multi-extension-monorepo/package.json +++ b/examples/multi-extension-monorepo/package.json @@ -18,7 +18,7 @@ "@prisma/orm-postgres": "workspace:8.0.0-rc.16" }, "devDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/test-utils": "workspace:8.0.0-rc.16", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@types/node": "catalog:", diff --git a/examples/paradedb-demo/package.json b/examples/paradedb-demo/package.json index b504789a30e9..50e2f94a35e4 100644 --- a/examples/paradedb-demo/package.json +++ b/examples/paradedb-demo/package.json @@ -25,7 +25,7 @@ "pg": "catalog:" }, "devDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/test-utils": "workspace:8.0.0-rc.16", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@types/node": "catalog:", diff --git a/examples/prisma-8-cloudflare-worker/package.json b/examples/prisma-8-cloudflare-worker/package.json index c98fa8bb5ffb..db010dfde7ca 100644 --- a/examples/prisma-8-cloudflare-worker/package.json +++ b/examples/prisma-8-cloudflare-worker/package.json @@ -29,7 +29,7 @@ "devDependencies": { "@cloudflare/vitest-pool-workers": "0.20.3", "@cloudflare/workers-types": "5.20260804.1", - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@types/node": "catalog:", "@types/pg": "catalog:", diff --git a/examples/prisma-8-demo-sqlite/package.json b/examples/prisma-8-demo-sqlite/package.json index b8521d002375..7d364e5c9a45 100644 --- a/examples/prisma-8-demo-sqlite/package.json +++ b/examples/prisma-8-demo-sqlite/package.json @@ -20,7 +20,7 @@ "dotenv": "^17.4.2" }, "devDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@types/node": "catalog:", "tsx": "catalog:", diff --git a/examples/prisma-8-demo/package.json b/examples/prisma-8-demo/package.json index 45779b5b7ff1..9b3b81552ad4 100644 --- a/examples/prisma-8-demo/package.json +++ b/examples/prisma-8-demo/package.json @@ -32,7 +32,7 @@ "react-dom": "^19.2.8" }, "devDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@prisma/dev": "catalog:", "@repo/test-utils": "workspace:8.0.0-rc.16", "@repo/tsconfig": "workspace:8.0.0-rc.16", diff --git a/examples/prisma-8-postgis-demo/package.json b/examples/prisma-8-postgis-demo/package.json index da2b43340000..8eb1f7a50287 100644 --- a/examples/prisma-8-postgis-demo/package.json +++ b/examples/prisma-8-postgis-demo/package.json @@ -33,7 +33,7 @@ "react-leaflet": "^5.0.0" }, "devDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/test-utils": "workspace:8.0.0-rc.16", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@types/leaflet": "^1.9.22", diff --git a/examples/prisma7-adoption/package.json b/examples/prisma7-adoption/package.json index b3b08030868a..6f24b4ec21d1 100644 --- a/examples/prisma7-adoption/package.json +++ b/examples/prisma7-adoption/package.json @@ -28,7 +28,7 @@ "pg": "catalog:" }, "devDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@prisma/dev": "catalog:", "@prisma/prisma7": "7.10.0", "@repo/test-utils": "workspace:8.0.0-rc.16", diff --git a/examples/prisma7-adoption/test/handover.test.ts b/examples/prisma7-adoption/test/handover.test.ts index 3b326fd2d688..285d5e7c29a5 100644 --- a/examples/prisma7-adoption/test/handover.test.ts +++ b/examples/prisma7-adoption/test/handover.test.ts @@ -221,7 +221,30 @@ describe('Prisma 8 taking over migrations from the Prisma 7 schema', () => { editSchema('edit-2.prisma'); await v8('contract', 'emit'); const destructiveHash = storageHash(dir); - const destructivePlan = await v8('migration', 'plan', '--name', 'drop-bio'); + const refused = await runAllowingFailure(dir, connectionString, 'prisma', [ + 'migration', + 'plan', + '--name', + 'drop-bio', + ]); + expect(refused.status, refused.output).toBe(2); + expect(resultEnvelope(refused.output)).toMatchObject({ + ok: false, + error: { + code: 'CLI.CONSENT_REQUIRED', + nextActions: expect.arrayContaining([ + expect.objectContaining({ label: 'Run the command again with --delete User.bio' }), + ]), + }, + }); + const destructivePlan = await v8( + 'migration', + 'plan', + '--name', + 'drop-bio', + '--delete', + 'User.bio', + ); expect(destructivePlan).toMatchObject({ from: additiveHash, to: destructiveHash }); expect(destructivePlan['baselineDir']).toBeUndefined(); const destructive = readBundle(dir, String(destructivePlan['dir'])); diff --git a/examples/react-router-demo/package.json b/examples/react-router-demo/package.json index f42084b1077a..42a82906aa25 100644 --- a/examples/react-router-demo/package.json +++ b/examples/react-router-demo/package.json @@ -30,7 +30,7 @@ "react-router": "^8.1.0" }, "devDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@prisma/dev": "catalog:", "@react-router/dev": "^8.1.0", "@repo/test-utils": "workspace:8.0.0-rc.16", diff --git a/examples/retail-store/migrations/app/20260513T0508_backfill_product_status/migration.json b/examples/retail-store/migrations/app/20260513T0508_backfill_product_status/migration.json index 214965199787..f882b213f3f2 100644 --- a/examples/retail-store/migrations/app/20260513T0508_backfill_product_status/migration.json +++ b/examples/retail-store/migrations/app/20260513T0508_backfill_product_status/migration.json @@ -3,5 +3,5 @@ "to": "9af6d90aaa4cd03bf7eb2040a1c1b5c1225a2b33883ea3d0552f312f4684f8dc", "providedInvariants": [], "createdAt": "2026-05-13T05:08:23.957Z", - "migrationHash": "5553cd547b573ea9b36d62510050a58a4d3ef213ae0290e2b9bb1cb58b4fc2fe" + "migrationHash": "2bce4ded744ff835ac51e0c815c7567047ccc1acc25069e0bebadaf72fd5f7d7" } diff --git a/examples/retail-store/migrations/app/20260513T0508_backfill_product_status/ops.json b/examples/retail-store/migrations/app/20260513T0508_backfill_product_status/ops.json index 66250d7d5107..52e7218845ac 100644 --- a/examples/retail-store/migrations/app/20260513T0508_backfill_product_status/ops.json +++ b/examples/retail-store/migrations/app/20260513T0508_backfill_product_status/ops.json @@ -2,7 +2,7 @@ { "id": "collection.products.setValidation", "label": "Set validation on products", - "operationClass": "destructive", + "operationClass": "widening", "precheck": [], "execute": [ { diff --git a/examples/retail-store/package.json b/examples/retail-store/package.json index feefc92d12c0..9c483c11ecbd 100644 --- a/examples/retail-store/package.json +++ b/examples/retail-store/package.json @@ -39,7 +39,7 @@ "tailwindcss": "^4.3.3" }, "devDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/test-utils": "workspace:8.0.0-rc.16", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@types/node": "catalog:", diff --git a/examples/supabase/package.json b/examples/supabase/package.json index 1a92abbdfd15..2e4cf6c8c588 100644 --- a/examples/supabase/package.json +++ b/examples/supabase/package.json @@ -18,7 +18,7 @@ "pg": "catalog:" }, "devDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/test-utils": "workspace:8.0.0-rc.16", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@types/node": "catalog:", diff --git a/packages/1-framework/1-core/errors/README.md b/packages/1-framework/1-core/errors/README.md index 567dde8c652b..f56b005b3d3b 100644 --- a/packages/1-framework/1-core/errors/README.md +++ b/packages/1-framework/1-core/errors/README.md @@ -36,4 +36,4 @@ if (CliStructuredError.is(caught)) { | Entry point | Domain | Code range | Contents | |---|---|---|---| | `./control` | CLI | `PN-CLI-4xxx` | `CliStructuredError` class, `CliErrorEnvelope`/`CliErrorConflict` types, 14 CLI factory functions, `errorUnexpected` | -| `./execution` | RUN | `PN-RUN-3xxx` | 8 runtime factory functions, `ERROR_CODE_DESTRUCTIVE_CHANGES` constant | +| `./execution` | RUN | `PN-RUN-3xxx` | Runtime factory functions | diff --git a/packages/1-framework/1-core/errors/src/execution.ts b/packages/1-framework/1-core/errors/src/execution.ts index e70f6eeb30b1..1750be0791ff 100644 --- a/packages/1-framework/1-core/errors/src/execution.ts +++ b/packages/1-framework/1-core/errors/src/execution.ts @@ -281,54 +281,6 @@ export function errorRunnerFailed( }); } -/** Error code for destructive changes that require explicit confirmation. */ -export const ERROR_CODE_DESTRUCTIVE_CHANGES = 'MIGRATION.DESTRUCTIVE_CHANGES'; - -/** - * Destructive operations require explicit confirmation via -y/--yes. - */ -export function errorDestructiveChanges( - summary: string, - options?: { - readonly why?: string; - readonly fix?: string; - readonly meta?: Record; - }, -): CliStructuredError { - return new CliStructuredError(ERROR_CODE_DESTRUCTIVE_CHANGES, summary, { - why: options?.why ?? 'Planned operations include destructive changes that require confirmation', - fix: options?.fix ?? 'Re-run with `-y` to apply, or use `--dry-run` to preview first', - ...(options?.meta ? { meta: options.meta } : {}), - }); -} - -export const ERROR_CODE_CONSENT_PLAN_MISMATCH = 'MIGRATION.CONSENT_PLAN_MISMATCH'; - -/** - * An apply carrying consent was refused because the plan recomputed for it is - * not the plan that was consented to. - */ -export function errorConsentPlanMismatch(options: { - readonly consentedPlanHash: string; - readonly planHash: string; - readonly why?: string; -}): CliStructuredError { - return new CliStructuredError( - ERROR_CODE_CONSENT_PLAN_MISMATCH, - 'The plan changed between consent and apply', - { - why: - options.why ?? - 'The plan recomputed for the consented apply is not the plan that was consented to, so applying it could destroy something nobody agreed to.', - fix: 'Re-run the command and review the freshly planned operations before consenting again', - meta: { - consentedPlanHash: options.consentedPlanHash, - planHash: options.planHash, - }, - }, - ); -} - /** * Generic runtime error carrying a caller-provided dotted code. */ diff --git a/packages/1-framework/1-core/errors/src/exports/execution.ts b/packages/1-framework/1-core/errors/src/exports/execution.ts index b7e46b86b547..dd9bcac52400 100644 --- a/packages/1-framework/1-core/errors/src/exports/execution.ts +++ b/packages/1-framework/1-core/errors/src/exports/execution.ts @@ -1,8 +1,4 @@ export { - ERROR_CODE_CONSENT_PLAN_MISMATCH, - ERROR_CODE_DESTRUCTIVE_CHANGES, - errorConsentPlanMismatch, - errorDestructiveChanges, errorHashMismatch, errorMarkerMissing, errorMarkerReadFailed, diff --git a/packages/1-framework/1-core/errors/test/execution.test.ts b/packages/1-framework/1-core/errors/test/execution.test.ts index 955e3d93d520..3fc19130582e 100644 --- a/packages/1-framework/1-core/errors/test/execution.test.ts +++ b/packages/1-framework/1-core/errors/test/execution.test.ts @@ -2,7 +2,6 @@ import { structuredError } from '@internal/utils/structured-error'; import { describe, expect, it } from 'vitest'; import { CliStructuredError } from '../src/control'; import { - errorDestructiveChanges, errorHashMismatch, errorMarkerMissing, errorMarkerReadFailed, @@ -107,23 +106,6 @@ describe('Runtime Errors', () => { expect(error.cause).toBe(cause); }); - it('errorDestructiveChanges creates correct error', () => { - const error = errorDestructiveChanges('Destructive changes detected'); - expect(error.code).toBe('MIGRATION.DESTRUCTIVE_CHANGES'); - expect(error.message).toBe('Destructive changes detected'); - }); - - it('errorDestructiveChanges with all options', () => { - const error = errorDestructiveChanges('Destructive changes detected', { - why: 'Custom why', - fix: 'Custom fix', - meta: { key: 'value' }, - }); - expect(error.why).toBe('Custom why'); - expect(error.fix).toBe('Custom fix'); - expect(error.meta).toEqual({ key: 'value' }); - }); - it('errorRuntime carries the caller-provided code', () => { const error = errorRuntime('MIGRATION.SPACE_NOT_FOUND', 'Something failed'); expect(error.code).toBe('MIGRATION.SPACE_NOT_FOUND'); diff --git a/packages/1-framework/1-core/framework-components/src/control/control-instances.ts b/packages/1-framework/1-core/framework-components/src/control/control-instances.ts index 0d5ec50ee43c..3b3f2778afa8 100644 --- a/packages/1-framework/1-core/framework-components/src/control/control-instances.ts +++ b/packages/1-framework/1-core/framework-components/src/control/control-instances.ts @@ -7,6 +7,7 @@ import type { TargetBoundComponentDescriptor, TargetInstance, } from '../shared/framework-components'; +import type { MigrationPlanOperation } from './control-migration-types'; import type { SpaceSignature, SpaceToSign, @@ -26,6 +27,9 @@ export interface ControlFamilyInstance */ deserializeContract(contractJson: unknown): Contract; + /** The name the database knows the object an operation acts on by. */ + storageNameOf(operation: MigrationPlanOperation): string; + verify(options: { readonly driver: ControlDriverInstance; readonly contract: unknown; diff --git a/packages/1-framework/1-core/framework-components/src/control/control-migration-types.ts b/packages/1-framework/1-core/framework-components/src/control/control-migration-types.ts index 23077db5a28c..8d1f0b85e152 100644 --- a/packages/1-framework/1-core/framework-components/src/control/control-migration-types.ts +++ b/packages/1-framework/1-core/framework-components/src/control/control-migration-types.ts @@ -9,7 +9,7 @@ * with additional fields for execution (precheck SQL, execute SQL, etc.). */ -import type { Contract } from '@internal/contract/types'; +import type { Contract, ContractWithDomain } from '@internal/contract/types'; import type { ImportRequirement } from '@internal/ts-render'; import type { Result } from '@internal/utils/result'; import type { TargetBoundComponentDescriptor } from '../shared/framework-components'; @@ -20,7 +20,12 @@ import type { ControlFamilyInstance, } from './control-instances'; import type { OperationContext } from './control-operation-results'; -import type { AppliedMigrationStatement, ResolvedMigrationStatement } from './migration-statements'; +import type { + AppliedMigrationStatement, + MigrationPlanSubjects, + MigrationSubject, + ResolvedMigrationStatement, +} from './migration-statements'; // ============================================================================ // Migration Package Metadata @@ -68,16 +73,18 @@ export interface MigrationMetadata { // ============================================================================ /** - * What an operation can do to the data. `destructive` means the operation loses data; no other - * class loses a row or a value. - * - 'additive': adds structure without changing what exists. - * - 'widening': changes existing structure without losing data: a rename, a relaxed constraint or - * a wider type, and a drop of an object that holds no data (an index; a unique, foreign-key or - * check constraint; a default; a native enum type; a row-level-security policy) or disabling - * row-level security. Dropping a policy or disabling row-level security widens who can read and - * write rows. - * - 'destructive': loses data, such as dropping a table or a column. - * - 'data': transforms data, such as a backfill or a type conversion. + * What an operation does to the data. Of the classes a planner chooses, only `destructive` loses + * data by itself; a `data` operation runs what its author wrote, which may lose data by design. + * - 'additive': adds structure and leaves existing structure and data as they are. + * - 'widening': changes existing structure without losing data. An operation that cannot keep + * every value fails instead, so a tightened constraint is widening: `SET NOT NULL` fails on a + * NULL, and a MongoDB validator applies to later writes only. So are a rename, a relaxed + * constraint, a type change that keeps every value, and a drop of an object that holds no data, + * such as an index, a constraint, a default or a row-level-security policy. Dropping a policy or + * disabling row-level security widens who can read and write rows. + * - 'destructive': can lose rows or values: dropping a table, a column or a collection, or a type + * change that can change values. + * - 'data': reads and writes rows, such as a backfill or a type conversion, as its author wrote. */ export type MigrationOperationClass = 'additive' | 'widening' | 'destructive' | 'data'; @@ -307,7 +314,7 @@ export interface MigrationPlannerConflict { * The plan is typed as `MigrationPlanWithAuthoringSurface` so the CLI can * uniformly ask any plan to render itself to TypeScript. */ -export interface MigrationPlannerSuccessResult { +export interface MigrationPlannerSuccessResult extends MigrationPlanSubjects { readonly kind: 'success'; readonly plan: MigrationPlanWithAuthoringSurface; readonly warnings?: readonly MigrationPlannerConflict[]; @@ -646,6 +653,18 @@ export interface TargetMigrationsCapability< adapter: ControlAdapterInstance, ): MigrationPlanner; createRunner(family: TFamilyInstance): MigrationRunner; + /** + * Set when the target's planner carries out no rename statement: a data-loss question then + * offers no rename, and says instead how to keep the data by hand. + */ + readonly renameStatements?: { + readonly refused: true; + /** How to keep the data of `subject` by hand, so that the plan made afterwards loses nothing. */ + readonly keepDataByHand: ( + subject: MigrationSubject, + fromContract: ContractWithDomain, + ) => string; + }; /** * Synthesizes a family-specific schema IR from a contract for offline planning. * The returned schema can be passed to `planner.plan({ schema })` as the "from" state. diff --git a/packages/1-framework/1-core/framework-components/src/control/migration-statements.ts b/packages/1-framework/1-core/framework-components/src/control/migration-statements.ts index 9ff9b08a94af..d646cfd14a32 100644 --- a/packages/1-framework/1-core/framework-components/src/control/migration-statements.ts +++ b/packages/1-framework/1-core/framework-components/src/control/migration-statements.ts @@ -39,6 +39,55 @@ export type ResolvedMigrationStatement = | ResolvedModelRenameStatement | ResolvedFieldRenameStatement; +/** + * What an operation of a plan is about, in the contract's terms where the family can say: a model + * or a field of the origin contract, or, when no model of the origin contract stores it, the name + * the database knows it by. Told apart by `kind`, not by a statement's `entity`, because a storage + * name is not an entity of the contract. + */ +export type MigrationSubject = + | ({ readonly kind: 'model' } & ModelCoordinate) + | ({ readonly kind: 'field' } & FieldCoordinate) + | { readonly kind: 'storage'; readonly name: string }; + +/** A key that is equal for two subjects exactly when they name the same thing. */ +export function migrationSubjectKey(subject: MigrationSubject): string { + switch (subject.kind) { + case 'model': + return JSON.stringify(['model', subject.namespaceId, subject.model]); + case 'field': + return JSON.stringify(['field', subject.namespaceId, subject.model, subject.field]); + case 'storage': + return JSON.stringify(['storage', subject.name]); + } +} + +/** An operation of a plan, by its position in the plan's `operations`, and its subject. */ +export interface MigrationOperationSubject { + readonly operationIndex: number; + readonly subject: MigrationSubject; +} + +/** An operation that changes who can read or write the rows of its subject. */ +export interface MigrationAccessChange extends MigrationOperationSubject { + /** + * True when it lets more people read or write, such as disabling row-level security; false when + * the change can go either way, such as dropping a policy. + */ + readonly widens: boolean; +} + +/** What the operations of a plan lose and whose access they change, each by its position in the plan. */ +export interface MigrationPlanSubjects< + TEntry extends MigrationOperationSubject = MigrationOperationSubject, + TAccess extends MigrationAccessChange = MigrationAccessChange, +> { + /** Each operation that loses data, in plan order, with what it loses. */ + readonly dataLoss: readonly TEntry[]; + /** Each operation that changes who can read or write data, in plan order, with what it is about. */ + readonly accessWidening: readonly TAccess[]; +} + /** * A statement as a plan applied it: `operationIndexes` are the positions, in the plan's * `operations`, of the operations the statement accounts for, in plan order, and empty when the @@ -92,6 +141,12 @@ export type MigrationStatementJson = readonly to: FieldCoordinateJson; }; +/** A subject in JSON output: `namespaceId` is left out for the unbound namespace. */ +export type MigrationSubjectJson = + | ({ readonly kind: 'model' } & ModelCoordinateJson) + | ({ readonly kind: 'field' } & FieldCoordinateJson) + | { readonly kind: 'storage'; readonly name: string }; + function coordinateJson( coordinate: T, ): Omit & ModelCoordinateJson { @@ -109,10 +164,32 @@ export function migrationStatementJson( : { ...statement, from: coordinateJson(statement.from), to: coordinateJson(statement.to) }; } +/** The subject as JSON output writes it. */ +export function migrationSubjectJson(subject: MigrationSubject): MigrationSubjectJson { + switch (subject.kind) { + case 'model': + return { + kind: 'model', + ...coordinateJson({ namespaceId: subject.namespaceId, model: subject.model }), + }; + case 'field': + return { + kind: 'field', + ...coordinateJson({ + namespaceId: subject.namespaceId, + model: subject.model, + field: subject.field, + }), + }; + case 'storage': + return subject; + } +} + /** * The text that reports a statement, in domain names: a model or field is named with its - * namespace only when its contract has more than one. A field is named through its model as the - * destination contract names it, as the statement itself is written. + * namespace only when its contract has more than one. A renamed field is named through its model as + * the destination contract names it, as the statement itself is written. */ export function describeMigrationStatement( statement: ResolvedMigrationStatement, diff --git a/packages/1-framework/1-core/framework-components/src/exports/control.ts b/packages/1-framework/1-core/framework-components/src/exports/control.ts index 191aa00a2672..e2aa10f9de94 100644 --- a/packages/1-framework/1-core/framework-components/src/exports/control.ts +++ b/packages/1-framework/1-core/framework-components/src/exports/control.ts @@ -126,7 +126,12 @@ export type { AppliedMigrationStatement, FieldCoordinate, FieldCoordinateJson, + MigrationAccessChange, + MigrationOperationSubject, + MigrationPlanSubjects, MigrationStatementJson, + MigrationSubject, + MigrationSubjectJson, ModelCoordinate, ModelCoordinateJson, ResolvedFieldRenameStatement, @@ -136,6 +141,8 @@ export type { export { describeMigrationStatement, migrationStatementJson, + migrationSubjectJson, + migrationSubjectKey, modelDisplayName, } from '../control/migration-statements'; export { orderIssuesByDependencies } from '../control/order-issues-by-dependencies'; diff --git a/packages/1-framework/1-core/framework-components/test/control-migration-types.test.ts b/packages/1-framework/1-core/framework-components/test/control-migration-types.test.ts index 8082b67c94f3..e9942c3dfa2d 100644 --- a/packages/1-framework/1-core/framework-components/test/control-migration-types.test.ts +++ b/packages/1-framework/1-core/framework-components/test/control-migration-types.test.ts @@ -12,11 +12,15 @@ describe('MigrationPlannerSuccessResult', () => { kind: 'success', plan: stubPlan, appliedStatements: [], + dataLoss: [], + accessWidening: [], }; expect(result).toEqual({ kind: 'success', plan: stubPlan, appliedStatements: [], + dataLoss: [], + accessWidening: [], }); expect('warnings' in result).toBe(false); }); @@ -30,6 +34,8 @@ describe('MigrationPlannerSuccessResult', () => { kind: 'success', plan: stubPlan, appliedStatements: [], + dataLoss: [], + accessWidening: [], warnings: Object.freeze([Object.freeze(warning)]), }; expect(result.warnings).toEqual([warning]); diff --git a/packages/1-framework/1-core/framework-components/test/migration-statements.test.ts b/packages/1-framework/1-core/framework-components/test/migration-statements.test.ts index 7b82cba57e81..971518ecd23b 100644 --- a/packages/1-framework/1-core/framework-components/test/migration-statements.test.ts +++ b/packages/1-framework/1-core/framework-components/test/migration-statements.test.ts @@ -3,6 +3,8 @@ import { describe, expect, it } from 'vitest'; import { describeMigrationStatement, migrationStatementJson, + migrationSubjectJson, + migrationSubjectKey, modelDisplayName, type ResolvedMigrationStatement, } from '../src/control/migration-statements'; @@ -98,6 +100,26 @@ describe('describeMigrationStatement', () => { }); }); +describe('migrationSubjectJson', () => { + it('leaves namespaceId out for the unbound namespace and keeps it otherwise', () => { + const unbound = asNamespaceId(UNBOUND_NAMESPACE_ID); + expect([ + migrationSubjectJson({ + kind: 'field', + namespaceId: unbound, + model: 'User', + field: 'name', + }), + migrationSubjectJson({ kind: 'model', namespaceId: app, model: 'Legacy' }), + migrationSubjectJson({ kind: 'storage', name: 'audit_log' }), + ]).toEqual([ + { kind: 'field', model: 'User', field: 'name' }, + { kind: 'model', namespaceId: 'app', model: 'Legacy' }, + { kind: 'storage', name: 'audit_log' }, + ]); + }); +}); + describe('modelDisplayName', () => { it('qualifies a model with its namespace, except in the unbound namespace', () => { expect([ @@ -134,3 +156,22 @@ describe('migrationStatementJson', () => { ]); }); }); + +describe('migrationSubjectKey', () => { + it('is equal for subjects that name the same thing, whatever their key order', () => { + const app = asNamespaceId('app'); + expect( + migrationSubjectKey({ kind: 'field', namespaceId: app, model: 'User', field: 'name' }), + ).toBe(migrationSubjectKey({ field: 'name', model: 'User', namespaceId: app, kind: 'field' })); + }); + + it('differs between a model, a field and a storage name that share a name', () => { + const app = asNamespaceId('app'); + const keys = new Set([ + migrationSubjectKey({ kind: 'model', namespaceId: app, model: 'User' }), + migrationSubjectKey({ kind: 'field', namespaceId: app, model: 'User', field: 'User' }), + migrationSubjectKey({ kind: 'storage', name: 'User' }), + ]); + expect(keys.size).toBe(3); + }); +}); diff --git a/packages/1-framework/3-tooling/cli/README.md b/packages/1-framework/3-tooling/cli/README.md index d242debc1acc..c6ef0fd78a92 100644 --- a/packages/1-framework/3-tooling/cli/README.md +++ b/packages/1-framework/3-tooling/cli/README.md @@ -943,18 +943,27 @@ Update your database schema to match the currently emitted contract. - Allows `additive`, `widening`, and `destructive` operation classes where supported by planner/runner - Disables per-operation runner execution checks by default (precheck/postcheck/idempotency) - In `--dry-run` mode for SQL targets, prints a DDL preview derived from planned operations -- In interactive mode, destructive plans require confirmation before apply -- In non-interactive mode, destructive plans fail unless `--confirm ` is provided +- Before an apply, asks what each operation that would lose data means and whether each that would widen access may run (see **Questions before an apply** below) **Command:** ```bash -prisma db update [--db ] [--to ] [--advance-ref ] [--config ] [--dry-run] [--rename ]... [--confirm ] [--interactive|--no-interactive] [--json] [-v] [-q] [--color/--no-color] +prisma db update [--db ] [--to ] [--advance-ref ] [--config ] [--dry-run] [--rename ]... [--delete ]... [--allow ]... [--interactive|--no-interactive] [--json] [-v] [-q] [--color/--no-color] ``` -**Rename statements (`--rename :`, repeatable):** tell `db update` that a model or field was renamed, so it renames the table or column instead of dropping and creating it. The statements work exactly as they do for `migration plan` (see below), and produce the same operations. The origin contract they resolve against is the contract the database's marker names, read from the local snapshot store (`migrations/snapshots//`). The snapshot is read only when statements are given; without them `db update` plans and applies exactly as it does without this flag. Either way the plan applies from whatever state the database is in, as `db update` always has. `db update` keeps a snapshot of the contract it applies whenever it advances a ref: the `db` ref by default, or with `--db ` only the ref you name with `--advance-ref `. So to use renames against a database you update with `--db `, give that earlier run `--advance-ref `. Without a readable snapshot, a run with statements fails with `MIGRATION.STATEMENT_ORIGIN_UNKNOWN`, naming the hash and the directory it looked in, and gives the steps that store the missing snapshot: emit the contract the database is at, run `db update --advance-ref --dry-run` against it and check that it plans no operations, then run it without `--dry-run`, which changes nothing in the database and stores the snapshot, then emit the new contract and run the rename. A database with no marker has nothing to rename, and fails the same way. Running the same statements a second time fails with `MIGRATION.STATEMENT_UNRESOLVED`, because the database is now at a contract that no longer has the old names. The statements the plan applied are listed under `Statements applied` after the operations (dry run and apply), and as `appliedStatements` in `--json` output, each with its `description` and `operationIndexes`, the positions in `operations` of the operations it accounts for. +**Rename statements (`--rename :`, repeatable):** tell `db update` that a model or field was renamed, so it renames the table or column instead of dropping and creating it. The statements work exactly as they do for `migration plan` (see below), and produce the same operations. The origin contract they resolve against is the contract the database's marker names, read from the local snapshot store (`migrations/snapshots//`). The snapshot is read on every run, so the planner sees the prior contract whenever one is stored; a missing or unreadable snapshot fails only a run with `--rename`. Either way the plan applies from whatever state the database is in, as `db update` always has. `db update` keeps a snapshot of the contract it applies whenever it advances a ref: the `db` ref by default, or with `--db ` only the ref you name with `--advance-ref `. So to use renames against a database you update with `--db `, give that earlier run `--advance-ref `. Without a readable snapshot, a run with statements fails with `MIGRATION.STATEMENT_ORIGIN_UNKNOWN`, naming the hash and the directory it looked in, and gives the steps that store the missing snapshot: emit the contract the database is at, run `db update --advance-ref --dry-run` against it and check that it plans no operations, then run it without `--dry-run`, which changes nothing in the database and stores the snapshot, then emit the new contract and run the rename. A database with no marker has nothing to rename, and fails the same way. Running the same statements a second time fails with `MIGRATION.STATEMENT_UNRESOLVED`, because the database is now at a contract that no longer has the old names. The statements the plan applied are listed under `Statements applied` after the operations (dry run and apply), and as `appliedStatements` in `--json` output, each with its `description` and `operationIndexes`, the positions in `operations` of the operations it accounts for. + +**Questions before an apply:** an apply asks one question per model, field or storage name whose data an operation would lose (dropping a table or a column, or a type change that can change values), and one per model whose rows an operation would let more people read or write (disabling row-level security, for example). It asks before it applies anything. Subjects are named through the origin contract read from the snapshot store, as `migration plan` names them; without a readable snapshot each subject is its storage name, which can only be deleted or allowed, and the question says the origin contract is unknown. Recorded migrations of extension spaces are asked about for the data they would lose, never for widened access. Each question is answered with a statement: + +- `--delete ` lets the update lose that data, for example `--delete Legacy` or `--delete User.nickname`. A field of a model the update renames is named through the model's new name. +- `--rename :` keeps the data of a model or field under a new name instead. A field's new name keeps its model: `--rename User.nickname:User.handle`, which the question writes as `User.nickname:User.`. +- `--allow ` lets the update change who can read or write that model's rows, for example `--allow User`. Each such operation is its own question: dropping a policy and disabling row-level security on one model take two `--allow User` flags. The question says disabling row-level security widens access, and that dropping a policy changes it, since a permissive policy grants access. + +Where nobody can answer (the run is not interactive, or `--yes` is set), the command fails with `CLI.CONSENT_REQUIRED` and lists every unanswered question with the flags that answer it. In a terminal it asks each question in turn; a typed rename is planned again, and when the plan still loses the subject's data the command fails with `MIGRATION.STATEMENT_DID_NOT_RESOLVE_LOSS`. A `--delete` or `--allow` that answers no question fails with `CLI.CONSENT_UNUSED` before anything is applied. `--confirm` no longer consents to anything here. A dry run asks nothing: it lists the questions an apply would ask under `An apply asks about`, each with the flag that answers it, and as `dataLoss` and `accessWidening` in `--json` output, each entry's `text` being the subject as `--delete` or `--allow` takes it. A dry run checks its `--delete` and `--allow` values against those subjects and refuses one that matches none with `MIGRATION.STATEMENT_ANSWERS_NO_QUESTION`, so the exact command an apply will run can be previewed first. A question the dry run's flags answer is marked `(answered)` (`answered: true` in JSON), and the apply it suggests repeats the statement flags it was given, `--to` and `--advance-ref`, and `--db ` as a placeholder when the dry run named its database (the URL is never printed). The answers are listed under `Statements applied` after the renames, for example `delete field "User.nickname" (1 operation)`, and as `appliedStatements` entries with `verb` `delete` or `allow`. Programmatic callers of `dbUpdate` pass `answerQuestions`, which answers every question in order or throws to refuse. It is called at least once per apply, with an empty list when nothing is in question, even under `acceptDataLoss: true`; return `[]` then. `delete` and `allow` statements in `statements` answer their questions without asking (an `allow` answers one operation, so two operations on one model take two `allow` statements, as on the command line), and one that answers no question fails with `MIGRATION.STATEMENT_ANSWERS_NO_QUESTION`, in plan mode as in apply mode, before anything is applied. `acceptDataLoss: true` answers every data-loss question, and `acceptAccessWidening: true` every access-widening question. **Error codes (additional to shared CLI/runtime codes):** - `RUNNER_FAILED`: runner rejected apply (origin mismatch, failed checks, policy failures, or execution errors) +- `CLI.CONSENT_REQUIRED`, `CLI.CONSENT_UNUSED`: a question nobody answered, or a `--delete` or `--allow` that answered no question +- `MIGRATION.STATEMENT_DID_NOT_RESOLVE_LOSS`: a rename typed at the prompt did not stop the loss it answered - `MIGRATION.STATEMENT_INVALID`, `MIGRATION.STATEMENT_UNRESOLVED`, `MIGRATION.STATEMENT_ORIGIN_UNKNOWN`: a `--rename` statement is malformed, does not resolve in the contracts, or has no origin contract to resolve against **Config File (`prisma.config.ts`):** @@ -1003,7 +1012,7 @@ The `contract.output` field specifies the path to `contract.json`. This is the c Plan a migration from contract changes. Compares a starting contract against a destination contract and produces a new migration package with the required operations. No database connection is needed — fully offline. ```bash -prisma migration plan [--config ] [--name ] [--from ] [--to ] [--rename ]... [--confirm ] [--interactive|--no-interactive] [--json] [-v] [-q] [--color/--no-color] +prisma migration plan [--config ] [--name ] [--from ] [--to ] [--rename ]... [--delete ]... [--interactive|--no-interactive] [--json] [-v] [-q] [--color/--no-color] ``` **Options:** @@ -1012,6 +1021,7 @@ prisma migration plan [--config ] [--name ] [--from ] [--t - `--from `: Starting contract reference (hash, prefix, ref name, migration directory, `^`, or `@empty`). `@empty` names the empty-database origin deliberately. Defaults to the `db` ref; when the ref is absent, greenfield only on an empty graph — over existing migrations the command refuses (`MIGRATION.PLAN_ORIGIN_UNKNOWN`) unless `--from @empty` is passed. - `--to `: Destination contract reference (hash, prefix, ref name, migration directory, or `^`). Defaults to the emitted `contract.json`. Use `--to ^` to plan a rollback toward a predecessor state. - `--rename :`: Rename a model or a field instead of dropping and creating it. Repeat the flag for several statements; they apply in the order given. See **Rename statements** below. +- `--delete `: Let the plan lose the data of a model, a field, or a storage name the refusal lists. Repeat the flag for several. See **Data loss** below. - `--json`: Output as JSON object - `-q, --quiet`: Quiet mode (errors only) - `-v, --verbose`: Verbose output (debug info, timings) @@ -1032,6 +1042,13 @@ prisma migration plan [--config ] [--name ] [--from ] [--t **Rename statements:** each `--rename` names the old and the new name in contract vocabulary, never a table or a column. Each side is one of `Model`, `namespace.Model`, `Model.field` or `namespace.Model.field`; both sides name a model, or both name a field of the same model. Names match exactly, including case. A name without a namespace resolves when exactly one namespace declares the model. A field's model is named as the destination contract names it, so a model rename followed by a field rename on it is `--rename Profile:User --rename User.name:User.fullName`. The old name must exist in the starting contract and not in the destination; the new name must exist in the destination and not in the starting contract. Statements resolve before anything is written; a statement that is malformed fails with `MIGRATION.STATEMENT_INVALID`, and one that does not resolve fails with `MIGRATION.STATEMENT_UNRESOLVED`, naming what it searched and what it found. Each error's next step is the statement to type instead: the accepted forms, the statement written the right way round, the statements a model-and-field mix may have meant, or the order that works. A statement that has already been applied (the origin already has the new name and not the old one) says to leave it out. Statements cannot swap two names in one plan, and the error gives the three plans that do, through a temporary name. A plan from an empty database (`--from @empty`, or greenfield) has no models to rename, so every statement is unresolved. Moving a model to another namespace and renaming value objects are not supported in this release. On MongoDB the planner refuses every statement in this release: planning fails with `MIGRATION.PLANNING_FAILED`, carrying the statement, and nothing is planned. The planner renames the table or column, and the constraints and indexes named after it, ahead of the rest of the plan. On Postgres, a check, a row-level-security policy, or an expression or partial index whose SQL names a renamed column is dropped and created again, and adding a check back reads every row of the table under an exclusive lock, so on a large table such a rename takes time; `migration.ts` contains the same `...this.renameTable(...)` and `...this.renameColumn(...)` calls you could have written by hand. The statements are listed under `Statements applied` after the operations, each with its number of operations, and as `appliedStatements` in `--json` output, each with its `statement`, `description` and `operationIndexes`, the positions in `operations` of the operations it accounts for. A coordinate in a contract with no namespaces has no `namespaceId` in the JSON. A statement whose storage does not change, such as a model whose table name is kept with `@@map`, is listed with no operations. When the storage did not change at all and no statement needs an operation, no migration package is written and the output says so; when the storage changed but the planner planned nothing, planning fails with `MIGRATION.PLANNING_FAILED`, statements or not. A statement the planner cannot carry out, for example on a table whose control policy is not `managed`, fails planning with `MIGRATION.PLANNING_FAILED`, and the refused statement is carried in the error's conflicts. +**Data loss:** a plan that would lose data is written only once you say what each such operation means. The command asks one question per model, field or storage name whose data an operation would lose (dropping a table or a column, or a type change that can change values), before it writes anything, including the baseline package of an auto-baseline. Each question is answered with a statement: + +- `--delete ` lets the plan lose that data, for example `--delete Legacy` for a model or `--delete User.nickname` for a field. Data that no model of the starting contract stores is named by its storage name, as the question gives it, and can only be deleted. A field of a model the plan renames is named through the model's new name, as a field rename is written: after `--rename Profile:User`, a dropped `nickname` field is answered with `--delete User.nickname` or `--rename User.nickname:User.handle`. +- `--rename :` keeps the data of a model or field under a new name; a field's new name keeps its model (`--rename User.nickname:User.handle`). A rename on the command line is planned from the start, so the drop it replaces is never asked about. The question offers it only for a model or field the destination no longer has, so a field whose type changes is answered with `--delete`, and not on MongoDB, whose planner refuses renames in this release. There the question says how to keep the documents instead: rename the collection in `mongosh` on each database before a plan that drops it is applied; `db update` then drops nothing, and a migration written by `migration plan` still drops it. + +Where nobody can answer (the run is not interactive, or `--yes` is set), the command fails with `CLI.CONSENT_REQUIRED` and lists every unanswered question with the flags that answer it. In a terminal it asks each question in turn: type `delete` to let the data go, or `rename :`; a rejected answer is asked again. A typed rename is planned again with the other renames, and when the plan still loses the subject's data the command fails with `MIGRATION.STATEMENT_DID_NOT_RESOLVE_LOSS`. A `--delete` that answers no question fails with `CLI.CONSENT_UNUSED` before anything is written. `--confirm` no longer consents to anything here. The deletes are listed under `Statements applied` after the renames, for example `delete model "Legacy" (1 operation)`, and in `--json` output each `appliedStatements` entry carries its `verb`, `rename` or `delete`; a delete's `statement` is `{ kind: 'delete', subject }`. The `⚠` marker stays on each destructive operation. + **Branching with `--from` and `--to`:** Use `--from` to create a migration edge from a specific contract hash instead of the default starting point. Use `--to` to plan toward any resolved contract — including a rollback via `^` — instead of the emitted contract. This enables branched migration graphs and arbitrary-target (including reverse) edges without editing contract source. ### `prisma migration show` @@ -1437,8 +1454,18 @@ try { // Run operations const verifyResult = await client.verify({ contract }); - const initResult = await client.dbInit({ contract, mode: 'apply' }); - const updateResult = await client.dbUpdate({ contract, mode: 'apply' }); + const initResult = await client.dbInit({ contract, mode: 'apply', migrationsDir: 'migrations' }); + const updateResult = await client.dbUpdate({ + contract, + mode: 'apply', + migrationsDir: 'migrations', + // Asked before an apply about each operation that would lose data or widen access. + // Answer each question in order, as `{ verb, text }`, or throw to refuse. + answerQuestions: async (questions) => { + if (questions.length > 0) throw new Error('db update would lose data or widen access'); + return []; + }, + }); const introspectResult = await client.introspect(); } finally { // Clean up diff --git a/packages/1-framework/3-tooling/cli/package.json b/packages/1-framework/3-tooling/cli/package.json index ced7154e8ced..a7da364da94a 100644 --- a/packages/1-framework/3-tooling/cli/package.json +++ b/packages/1-framework/3-tooling/cli/package.json @@ -52,7 +52,7 @@ "@internal/sql-contract-ts": "workspace:8.0.0-rc.16", "@internal/sql-operations": "workspace:8.0.0-rc.16", "@internal/sql-runtime": "workspace:8.0.0-rc.16", - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/test-utils": "workspace:8.0.0-rc.16", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@repo/tsdown": "workspace:8.0.0-rc.16", @@ -62,7 +62,7 @@ "vitest": "catalog:" }, "peerDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "typescript": ">=5.9" }, "peerDependenciesMeta": { diff --git a/packages/1-framework/3-tooling/cli/src/control-api/client.ts b/packages/1-framework/3-tooling/cli/src/control-api/client.ts index 1f2ec11223c5..533d8cd4fc6b 100644 --- a/packages/1-framework/3-tooling/cli/src/control-api/client.ts +++ b/packages/1-framework/3-tooling/cli/src/control-api/client.ts @@ -419,8 +419,9 @@ class ControlClientImpl implements ControlClient { targetId: this.options.target.targetId, extensions: this.options.extensions ?? [], ...ifDefined('acceptDataLoss', options.acceptDataLoss), - ...ifDefined('consent', options.consent), + ...ifDefined('acceptAccessWidening', options.acceptAccessWidening), ...ifDefined('statements', options.statements), + answerQuestions: options.answerQuestions, ...ifDefined('verifySnapshotContent', this.snapshotVerifier), ...ifDefined('onProgress', onProgress), }); diff --git a/packages/1-framework/3-tooling/cli/src/control-api/operations/contract-space-aggregate-loader.ts b/packages/1-framework/3-tooling/cli/src/control-api/operations/contract-space-aggregate-loader.ts index cf16e3418585..06c55fe3bc5f 100644 --- a/packages/1-framework/3-tooling/cli/src/control-api/operations/contract-space-aggregate-loader.ts +++ b/packages/1-framework/3-tooling/cli/src/control-api/operations/contract-space-aggregate-loader.ts @@ -1,7 +1,10 @@ import { readFile } from 'node:fs/promises'; import type { PrismaNextConfig } from '@internal/config/config-types'; import type { Contract } from '@internal/contract/types'; -import type { ControlExtensionDescriptor } from '@internal/framework-components/control'; +import type { + ControlExtensionDescriptor, + MigrationPlanOperation, +} from '@internal/framework-components/control'; import { createControlStack } from '@internal/framework-components/control'; import type { ContractSpaceAggregate, @@ -361,7 +364,11 @@ export async function buildReadAggregate( }, ): Promise< Result< - { readonly aggregate: ContractSpaceAggregate; readonly contractHash: string }, + { + readonly aggregate: ContractSpaceAggregate; + readonly contractHash: string; + readonly storageNameOf: (operation: MigrationPlanOperation) => string; + }, CliStructuredError > > { @@ -406,7 +413,11 @@ export async function buildReadAggregate( if (!loaded.ok) { return loaded; } - return ok({ aggregate: loaded.value, contractHash }); + return ok({ + aggregate: loaded.value, + contractHash, + storageNameOf: (operation: MigrationPlanOperation) => familyInstance.storageNameOf(operation), + }); } catch (error) { return notOk( errorFromCaught(error, (message) => `Failed to read migrations directory: ${message}`), diff --git a/packages/1-framework/3-tooling/cli/src/control-api/operations/db-run.ts b/packages/1-framework/3-tooling/cli/src/control-api/operations/db-run.ts index 02781e5e2b37..c099b09e038c 100644 --- a/packages/1-framework/3-tooling/cli/src/control-api/operations/db-run.ts +++ b/packages/1-framework/3-tooling/cli/src/control-api/operations/db-run.ts @@ -10,8 +10,10 @@ import type { ControlExtensionDescriptor, ControlFamilyInstance, MigrationOperationPolicy, + MigrationOperationSubject, MigrationPlannerConflict, MigrationPlanOperation, + MigrationPlanSubjects, OperationPreview, ResolvedMigrationStatement, TargetMigrationsCapability, @@ -20,9 +22,11 @@ import { CONTRACT_SNAPSHOTS_DIRNAME, hasOperationPreview, isStorageHashHex, + migrationSubjectKey, } from '@internal/framework-components/control'; import type { ContractMarkerRecordLike } from '@internal/migration-tools/aggregate'; import { + type PlannerSuccess as AggregatePlan, type ContractSpaceAggregate, collectAggregateNamespaces, type PlannerError, @@ -37,16 +41,28 @@ import { MigrationToolsError } from '@internal/migration-tools/errors'; import { blindCast } from '@internal/utils/casts'; import { ifDefined } from '@internal/utils/defined'; import { InternalError } from '@internal/utils/internal-error'; -import { notOk, ok } from '@internal/utils/result'; +import { notOk, ok, type Result } from '@internal/utils/result'; import { join } from 'pathe'; import { CliStructuredError } from '../../utils/cli-errors'; +import { + type AnswerPlanQuestions, + askPlanQuestions, + type ConsentedSubject, + keepDataByHandFor, + type PlannedQuestions, + refuseUnusedConsents, + subjectText, +} from '../statements/plan-questions'; import { type AppliedStatementReport, reportAppliedStatements, + reportConsentStatement, } from '../statements/report-applied-statements'; import { resolveStatements, type StatementOrigin } from '../statements/resolve-statements'; import type { StatementText } from '../statements/statement-text'; import type { + AskedAccessChange, + AskedSubject, DbInitFailure, DbInitResult, DbInitSuccess, @@ -61,7 +77,6 @@ import { buildContractSpaceAggregate, } from './contract-space-aggregate-loader'; import { stripOperations } from './migration-helpers'; -import { computePlanHash } from './plan-identity'; import { buildPerSpaceBreakdown, collectOrdered, @@ -104,26 +119,30 @@ export interface ExecuteRunSharedOptions>; readonly targetId: TTargetId; readonly policy: MigrationOperationPolicy; - /** - * Identity of the plan the caller consented to (`db update` only). When - * set, the apply refuses with `CONSENT_PLAN_MISMATCH` if the freshly - * computed plan differs — consent binds to one plan, not to data loss in - * general. - */ - readonly consentedPlanHash?: string; /** Content check for contract snapshots the aggregate loader resolves. */ readonly verifySnapshotContent?: SnapshotContentVerifier; readonly onProgress?: OnControlProgress; } -/** `db init` takes no statements; `db update` takes the statements as the user wrote them, in order. */ +/** + * `db init` takes no statements. `db update` takes the statements as the user wrote them, in + * order, and the answer to each question its plan raises: what an operation that would lose data + * means and, when it applies, whether an operation may widen access. `acceptDataLoss` answers + * every data-loss question with consent, and `acceptAccessWidening` every access question. + */ export type ExecuteRunOptions< TFamilyId extends string, TTargetId extends string, > = ExecuteRunSharedOptions & ( | { readonly action: 'dbInit' } - | { readonly action: 'dbUpdate'; readonly statements: readonly StatementText[] } + | { + readonly action: 'dbUpdate'; + readonly statements: readonly StatementText[]; + readonly answerQuestions: AnswerPlanQuestions; + readonly acceptDataLoss: boolean; + readonly acceptAccessWidening: boolean; + } ); /** @@ -183,30 +202,36 @@ export async function executeRun familyInstance.deserializeContract(json), + ...ifDefined('verifySnapshotContent', options.verifySnapshotContent), + }) + : undefined; + const fromContract = appOrigin?.contract ?? null; const statementTexts = options.action === 'dbUpdate' ? options.statements : []; - if (statementTexts.length > 0) { - const appOrigin = await readAppOrigin({ - marker: markerRows.get(aggregate.app.spaceId) ?? null, - migrationsDir, - deserializeContract: (json) => familyInstance.deserializeContract(json), - ...ifDefined('verifySnapshotContent', options.verifySnapshotContent), - }); + const resolveRenames = ( + texts: readonly StatementText[], + ): readonly ResolvedMigrationStatement[] => { + if (texts.length === 0 || appOrigin === undefined) return []; const resolved = resolveStatements({ - statements: statementTexts, + statements: texts, origin: appOrigin.origin, destination: contract, }); if (!resolved.ok) { throw resolved.failure; } - statements = resolved.value; - fromContract = appOrigin.contract; - } + return resolved.value; + }; + const statements = resolveRenames(statementTexts); // 2a. Orphan-marker pre-flight: refuse to *apply* when a marker row // exists for a space that is not declared in the aggregate. Plan mode @@ -237,29 +262,100 @@ export async function executeRun({ - aggregate, - currentDBState: { markersBySpaceId: markerRows, schemaIntrospection: schemaIR }, - adapter, - migrations, - frameworkComponents, - callerPolicy: { ignoreGraphFor: new Set([aggregate.app.spaceId]) }, - operationPolicy: policy, - appSpace: { fromContract, statements }, - }); - if (!planResult.ok) { - onProgress?.({ action, kind: 'spanEnd', spanId: SPAN_IDS.plan, outcome: 'error' }); - return mapPlannerError(planResult.failure); - } - onProgress?.({ action, kind: 'spanEnd', spanId: SPAN_IDS.plan, outcome: 'ok' }); + const plan = async ( + resolved: readonly ResolvedMigrationStatement[], + ): Promise> => { + onProgress?.({ + action, + kind: 'spanStart', + spanId: SPAN_IDS.plan, + label: 'Planning migration', + }); + const planned = await planMigration({ + aggregate, + currentDBState: { markersBySpaceId: markerRows, schemaIntrospection: schemaIR }, + adapter, + migrations, + frameworkComponents, + callerPolicy: { ignoreGraphFor: new Set([aggregate.app.spaceId]) }, + operationPolicy: policy, + appSpace: { fromContract, statements: resolved }, + storageNameOf: (operation) => familyInstance.storageNameOf(operation), + }); + if (!planned.ok) { + onProgress?.({ action, kind: 'spanEnd', spanId: SPAN_IDS.plan, outcome: 'error' }); + return notOk(mapPlannerError(planned.failure)); + } + onProgress?.({ action, kind: 'spanEnd', spanId: SPAN_IDS.plan, outcome: 'ok' }); + const orderedResolutions = collectOrdered(planned.value.applyOrder, planned.value.perSpace); + const subjects: MigrationPlanSubjects = + action === 'dbUpdate' + ? subjectsAcrossSpaces(orderedResolutions) + : { dataLoss: [], accessWidening: [] }; + const operations = orderedResolutions.flatMap((r) => r.entry.displayOps); + const labelled = (entries: readonly TEntry[]) => + entries.map((entry) => ({ + ...entry, + label: operations[entry.operationIndex]?.label ?? JSON.stringify(entry.subject), + })); + return ok({ + planned: planned.value, + orderedResolutions, + subjects, + dataLoss: labelled(subjects.dataLoss), + accessWidening: labelled(subjects.accessWidening), + }); + }; - const orderedResolutions = collectOrdered(planResult.value.applyOrder, planResult.value.perSpace); + const first = await plan(statements); + if (!first.ok) { + return first.failure; + } + let run = first.value; + let renameTexts = statementTexts.filter((statement) => statement.verb === 'rename'); + let consented: readonly ConsentedSubject[] = []; + // 3a. Questions: an apply of `db update` asks what each operation that would lose data means, + // and whether each that would widen access may run, before it applies anything. A dry run asks + // nothing and lists them. + if (options.action === 'dbUpdate' && mode === 'apply') { + const answered = await askPlanQuestions({ + plan: run, + askAccess: true, + renames: renameTexts, + preAnswers: statementTexts.filter((statement) => statement.verb !== 'rename'), + consentAll: { delete: options.acceptDataLoss, allow: options.acceptAccessWidening }, + origin: fromContract ?? EMPTY_ORIGIN, + originKnown: fromContract !== null, + keepDataByHand: keepDataByHandFor(migrations, fromContract ?? EMPTY_ORIGIN), + destination: contract, + answer: options.answerQuestions, + replan: (renames) => plan(resolveRenames(renames)), + }); + if (!answered.ok) { + if (CliStructuredError.is(answered.failure)) { + throw answered.failure; + } + return answered.failure; + } + run = answered.value.plan; + renameTexts = [...answered.value.renames]; + consented = answered.value.consented; + } + if (options.action === 'dbUpdate' && mode === 'plan') { + const refused = refuseUnusedConsents({ + plan: run, + statements: statementTexts, + contracts: { + origin: fromContract ?? EMPTY_ORIGIN, + destination: contract, + renames: renameTexts, + }, + }); + if (!refused.ok) { + throw refused.failure; + } + } + const { planned, orderedResolutions } = run; const plannerWarnings = aggregatePlannerWarnings(orderedResolutions); // The destination's structural shape comes from the app's plan — its @@ -273,12 +369,41 @@ export async function executeRun + reportConsentStatement( + entry, + entry.verb === 'allow' && entry.operationIndex !== undefined + ? [entry.operationIndex] + : run.subjects.dataLoss + .filter( + ({ subject }) => + migrationSubjectKey(subject) === migrationSubjectKey(entry.subject), + ) + .map(({ operationIndex }) => operationIndex), + ), + ), + ] + : undefined; + const contractsAsked = { + origin: fromContract ?? EMPTY_ORIGIN, + destination: contract, + renames: renameTexts, + }; + const asked = (entries: readonly TEntry[]) => + entries.map((entry) => ({ ...entry, text: subjectText(entry.subject, contractsAsked) })); + const subjects = + action === 'dbUpdate' + ? { + dataLoss: asked(run.subjects.dataLoss), + accessWidening: asked(run.subjects.accessWidening), + } : undefined; // 4. Plan-mode: surface aggregate operations without applying. @@ -298,32 +423,11 @@ export async function executeRun r.entry.displayOps)), - destination: appPlan.destination, - }); - if (planHash !== options.consentedPlanHash) { - const failure: DbUpdateFailure = { - code: 'CONSENT_PLAN_MISMATCH', - summary: 'The plan changed between consent and apply', - why: 'The plan recomputed for the consented apply is not the plan that was consented to, so applying it could destroy something nobody agreed to.', - conflicts: undefined, - meta: undefined, - consentPlanMismatch: { consentedPlanHash: options.consentedPlanHash, planHash }, - ...ifDefined('warnings', plannerWarnings), - }; - return notOk(failure); - } - } - // 5. Run mode: hand off to the shared `runMigration` primitive. // The runner-driving tail is identical for `db init` / `db update` / // `migrate` — only how each caller produces `perSpacePlans` @@ -332,8 +436,8 @@ export async function executeRun count + resolution.entry.displayOps.length, 0); } +/** + * What the planned operations lose and whose access they widen, across every space, each at its + * position in the operations the result reports. + */ +function subjectsAcrossSpaces( + orderedResolutions: readonly OrderedResolution[], +): MigrationPlanSubjects { + const offset = ( + resolution: OrderedResolution, + entries: readonly TEntry[], + ) => + entries.map((entry) => ({ + ...entry, + operationIndex: + entry.operationIndex + operationsBefore(orderedResolutions, resolution.spaceId), + })); + return { + dataLoss: orderedResolutions.flatMap((resolution) => + offset(resolution, resolution.entry.dataLoss), + ), + accessWidening: orderedResolutions.flatMap((resolution) => + offset(resolution, resolution.entry.accessWidening), + ), + }; +} + function aggregatePlannerWarnings( orderedResolutions: readonly OrderedResolution[], ): readonly MigrationPlannerConflict[] | undefined { @@ -530,11 +671,14 @@ function wrapPlanResult(args: { readonly summary: string; /** `undefined` for `db init`, which reports no statements. */ readonly appliedStatements: readonly AppliedStatementReport[] | undefined; + /** `undefined` for `db init`, which reports no data loss. */ + readonly subjects: MigrationPlanSubjects | undefined; readonly warnings?: readonly MigrationPlannerConflict[]; }): DbInitResult | DbUpdateResult { const success: DbInitSuccess | DbUpdateSuccess = { mode: 'plan', ...ifDefined('appliedStatements', args.appliedStatements), + ...args.subjects, plan: { operations: stripOperations(args.operations), ...ifDefined('preview', args.preview), @@ -559,11 +703,14 @@ function wrapApplyResult(args: { readonly summary: string; /** `undefined` for `db init`, which reports no statements. */ readonly appliedStatements: readonly AppliedStatementReport[] | undefined; + /** `undefined` for `db init`, which reports no data loss. */ + readonly subjects: MigrationPlanSubjects | undefined; readonly warnings?: readonly MigrationPlannerConflict[]; }): DbInitResult | DbUpdateResult { const success: DbInitSuccess | DbUpdateSuccess = { mode: 'apply', ...ifDefined('appliedStatements', args.appliedStatements), + ...args.subjects, plan: { operations: stripOperations(args.operations) }, destination: { storageHash: args.destination.storageHash, diff --git a/packages/1-framework/3-tooling/cli/src/control-api/operations/db-update.ts b/packages/1-framework/3-tooling/cli/src/control-api/operations/db-update.ts index df3ac1e68b2e..9c09cb1bc11b 100644 --- a/packages/1-framework/3-tooling/cli/src/control-api/operations/db-update.ts +++ b/packages/1-framework/3-tooling/cli/src/control-api/operations/db-update.ts @@ -9,11 +9,10 @@ import type { } from '@internal/framework-components/control'; import type { SnapshotContentVerifier } from '@internal/migration-tools/contract-snapshot-store'; import { ifDefined } from '@internal/utils/defined'; -import { notOk } from '@internal/utils/result'; +import type { AnswerPlanQuestions } from '../statements/plan-questions'; import type { StatementText } from '../statements/statement-text'; import type { DbUpdateResult, OnControlProgress } from '../types'; -import { type ExecuteRunSharedOptions, executeRun } from './db-run'; -import { computePlanHash } from './plan-identity'; +import { executeRun } from './db-run'; export const DB_UPDATE_POLICY = { allowedOperationClasses: ['additive', 'widening', 'destructive'] as const, @@ -23,10 +22,9 @@ export const DB_UPDATE_POLICY = { * Options for the `db update` operation. * * Same loader → planner → runner pipeline as `db init`, but with the - * widened operation policy (additive + widening + destructive). The - * destructive-change confirmation gate runs at this layer: when - * `mode === 'apply'` and `acceptDataLoss` is `false`, the operation - * pre-plans, surfaces destructive ops to the caller, and aborts. + * widened operation policy (additive + widening + destructive). An apply + * asks `answerQuestions` what each operation that would lose data means and + * whether each that would widen access may run, before it applies anything. */ export interface ExecuteDbUpdateOptions { readonly driver: ControlDriverInstance; @@ -40,39 +38,30 @@ export interface ExecuteDbUpdateOptions >; readonly frameworkComponents: ReadonlyArray>; + /** Consents to every operation that would lose data, without asking. */ readonly acceptDataLoss?: boolean; - /** - * Consent to the plan a prior `DESTRUCTIVE_CHANGES` refusal named by its - * `planHash`. The apply recomputes its plan and refuses with - * `CONSENT_PLAN_MISMATCH` when the fresh plan differs. - */ - readonly consent?: { readonly planHash: string }; + /** Consents to every operation that would widen who can read or write rows, without asking. */ + readonly acceptAccessWidening?: boolean; readonly migrationsDir: string; readonly targetId: TTargetId; readonly extensions?: ReadonlyArray>; /** Content check for contract snapshots the aggregate loader resolves. */ readonly verifySnapshotContent?: SnapshotContentVerifier; /** - * The statements as the user wrote them, in the order given. Both the plan - * and the destructive pre-plan resolve and apply them. + * The statements as the user wrote them, in the order given. A `rename` is planned; a `delete` + * or `allow` answers the question about the subject it names. */ readonly statements?: readonly StatementText[]; + /** Asks the questions no statement answered; see {@link AnswerPlanQuestions}. */ + readonly answerQuestions: AnswerPlanQuestions; readonly onProgress?: OnControlProgress; } -/** - * Execute `db update` against the configured contract. - * - * Routes through the loader → planner → runner pipeline. Destructive - * operations require either `acceptDataLoss: true` or a prior - * `mode: 'plan'` invocation that surfaces the destructive ops; the - * confirmation gate is implemented here so the lower-level applier - * remains policy-agnostic. - */ +/** Execute `db update` against the configured contract. */ export async function executeDbUpdate( options: ExecuteDbUpdateOptions, ): Promise { - const sharedInputs = { + return (await executeRun({ driver: options.driver, adapter: options.adapter, familyInstance: options.familyInstance, @@ -83,59 +72,13 @@ export async function executeDbUpdate(sharedInputs); - if (refusal !== null) return refusal; - } - return (await executeRun({ - ...sharedInputs, - mode: options.mode, - ...ifDefined('consentedPlanHash', options.consent?.planHash), - })) as DbUpdateResult; -} - -/** - * Pre-plan once when running `db update apply` without `acceptDataLoss`. - * Surfaces destructive operations across every space; if any are - * planned, returns a `DESTRUCTIVE_CHANGES` failure that the CLI shows - * as a confirmation prompt. Returns `null` when the apply is safe to - * run. - */ -async function guardDestructiveChanges( - sharedInputs: Omit, 'mode'> & { - readonly action: 'dbUpdate'; - readonly statements: readonly StatementText[]; - }, -): Promise { - const planResult = (await executeRun({ - ...sharedInputs, - mode: 'plan', })) as DbUpdateResult; - if (!planResult.ok) return planResult; - const destructiveOps = planResult.value.plan.operations - .filter((op) => op.operationClass === 'destructive') - .map((op) => ({ id: op.id, label: op.label })); - if (destructiveOps.length === 0) return null; - const databaseName = await sharedInputs.driver.databaseName?.(); - return notOk({ - code: 'DESTRUCTIVE_CHANGES', - summary: `Planned ${destructiveOps.length} destructive operation(s) that require confirmation`, - why: 'Destructive operations require explicit consent, which the caller has not given', - conflicts: undefined, - meta: undefined, - destructiveChanges: { - destructiveOperations: destructiveOps, - databaseName, - planHash: computePlanHash({ - operations: planResult.value.plan.operations, - destination: planResult.value.destination, - }), - }, - ...ifDefined('warnings', planResult.value.warnings), - }); } diff --git a/packages/1-framework/3-tooling/cli/src/control-api/operations/migrate-show.ts b/packages/1-framework/3-tooling/cli/src/control-api/operations/migrate-show.ts index 50605066fc80..9729f785770c 100644 --- a/packages/1-framework/3-tooling/cli/src/control-api/operations/migrate-show.ts +++ b/packages/1-framework/3-tooling/cli/src/control-api/operations/migrate-show.ts @@ -128,7 +128,7 @@ export async function executeMigrateShowPlan( if (!loaded.ok) { return notOk(loaded.failure); } - const { aggregate, contractHash } = loaded.value; + const { aggregate, contractHash, storageNameOf } = loaded.value; const appGraph = aggregate.app.graph(); // Resolve the --to target (defaults to the on-disk contract, same as migrate). @@ -245,6 +245,7 @@ export async function executeMigrateShowPlan( targetHash: spaceTargetHash, refInvariants: spaceRefInvariants, liveMarker, + storageNameOf, }); if (outcome.kind === 'at-head') { diff --git a/packages/1-framework/3-tooling/cli/src/control-api/operations/migrate.ts b/packages/1-framework/3-tooling/cli/src/control-api/operations/migrate.ts index b1cba6bf26f5..546fba544bce 100644 --- a/packages/1-framework/3-tooling/cli/src/control-api/operations/migrate.ts +++ b/packages/1-framework/3-tooling/cli/src/control-api/operations/migrate.ts @@ -8,6 +8,7 @@ import type { ControlDriverInstance, ControlExtensionDescriptor, ControlFamilyInstance, + MigrationPlanOperation, TargetMigrationsCapability, } from '@internal/framework-components/control'; import { @@ -177,6 +178,7 @@ export async function executeMigrate familyInstance.storageNameOf(operation), }); if (outcome.kind === 'at-head') { @@ -387,6 +389,7 @@ export function planSpacePath({ refInvariants, liveMarker, refName, + storageNameOf, }: { readonly space: AggregateContractSpace; readonly aggregate: Pick; @@ -394,6 +397,7 @@ export function planSpacePath({ readonly refInvariants: readonly string[] | undefined; readonly liveMarker: ContractMarkerRecordLike | null; readonly refName?: string; + readonly storageNameOf: (operation: MigrationPlanOperation) => string; }): SpacePathOutcome { const isAppSpace = space.spaceId === aggregate.app.spaceId; const headRef = requireHeadRef(space); @@ -459,6 +463,7 @@ export function planSpacePath({ space: targetSpace, currentMarker: liveMarker, ...(isAppSpace && refName !== undefined ? { refName } : {}), + storageNameOf, }); if (walked.kind === 'unreachable') { @@ -509,6 +514,8 @@ function buildAtHeadResolution(args: { destinationContract: space.contract(), strategy: 'declared-state', appliedStatements: [], + dataLoss: [], + accessWidening: [], migrationEdges: [ buildFabricatedMigrationEdge({ currentMarkerStorageHash: liveMarker?.storageHash, diff --git a/packages/1-framework/3-tooling/cli/src/control-api/operations/migration-plan.ts b/packages/1-framework/3-tooling/cli/src/control-api/operations/migration-plan.ts index 93d1310f6c96..b58f25d10613 100644 --- a/packages/1-framework/3-tooling/cli/src/control-api/operations/migration-plan.ts +++ b/packages/1-framework/3-tooling/cli/src/control-api/operations/migration-plan.ts @@ -9,6 +9,8 @@ import { createControlStack, hasOperationPreview, type MigrationPlanOperation, + type MigrationSubject, + migrationSubjectKey, type OperationPreview, planOriginOf, type ResolvedMigrationStatement, @@ -31,9 +33,7 @@ import { join, relative } from 'pathe'; import { type CliErrorConflict, CliStructuredError, - errorConsentPlanMismatch, errorContractValidationFailed, - errorDestructiveChanges, errorFileNotFound, errorMigrationPlanningFailed, errorPlanProducedNoOperations, @@ -50,13 +50,21 @@ import { toExtensionInputs } from '../../utils/extension-pack-inputs'; import { assertFrameworkComponentsCompatible } from '../../utils/framework-components'; import { createProjectSpecifierResolver } from '../../utils/project-import-root'; import { snapshotVerifierFor } from '../../utils/snapshot-content-verification'; +import { + type AnswerPlanQuestions, + askPlanQuestions, + keepDataByHandFor, + type PlannedSubject, + subjectText, +} from '../statements/plan-questions'; import { type AppliedStatementReport, reportAppliedStatements, + reportConsentStatement, } from '../statements/report-applied-statements'; import { resolveStatements } from '../statements/resolve-statements'; import type { StatementText } from '../statements/statement-text'; -import type { ControlClient, DestructivePlanOperation } from '../types'; +import type { ControlClient } from '../types'; import { errorFromCaught } from './caught-errors'; import { buildContractSpaceAggregate, @@ -66,7 +74,6 @@ import { type ContractSpaceSeedPhaseRecord, runContractSpaceSeedPhase, } from './contract-space-seed-phase'; -import { computePlanHash } from './plan-identity'; import { resolveFromForPlan, resolveToForPlan } from './plan-resolution'; import { renderSnapshotDeclarations } from './snapshot-declarations'; @@ -84,36 +91,21 @@ export interface MigrationPlanOptions { readonly from?: string; readonly to?: string; /** - * The statements as the user wrote them, in the order given. They resolve - * against the origin and destination contracts before anything is written. + * The statements as the user wrote them, in the order given. A `rename` + * resolves against the origin and destination contracts before anything is + * written; a `delete` answers the question about the subject it names, and + * one that answers no question, or an `allow`, which `migration plan` never + * asks for, fails with `MIGRATION.STATEMENT_ANSWERS_NO_QUESTION`. */ readonly statements?: readonly StatementText[]; - /** Renders the declarations of the destination snapshot from its `contract.json`. */ - readonly client: Pick; /** - * Consent to the auto-baseline plan a prior `MIGRATION.DESTRUCTIVE_CHANGES` - * refusal named by its `planHash`. The consented run recomputes the baseline - * plan and refuses with `MIGRATION.CONSENT_PLAN_MISMATCH` when it differs. + * Asks what each operation that would lose data means, before anything is + * written. A delete answer consents to the loss; a rename answer is planned + * again with the other renames and must remove the loss. */ - readonly consent?: { readonly planHash: string }; - /** - * Extension-space migration packages a refused first run of this same - * invocation already materialised. The consented re-run finds them on disk - * (its seed phase reports `unchanged`), so the caller threads them back in - * to keep `emittedExtensionDirs` and the summary describing the whole - * invocation, not just the second run. - */ - readonly carryEmittedExtensionDirs?: readonly { - readonly spaceId: string; - readonly dirName: string; - }[]; -} - -/** The verdict a `MIGRATION.DESTRUCTIVE_CHANGES` plan refusal carries in its meta. */ -export interface DestructiveBaselineVerdict { - readonly destructiveOperations: ReadonlyArray; - /** Content hash of the refused baseline plan; consent is granted against it. */ - readonly planHash: string; + readonly answerQuestions: AnswerPlanQuestions; + /** Renders the declarations of the destination snapshot from its `contract.json`. */ + readonly client: Pick; } type PlannerSuccess = { @@ -121,6 +113,8 @@ type PlannerSuccess = { readonly migrationTsContent: string; readonly hasPlaceholders: boolean; readonly appliedStatements: readonly AppliedStatementReport[]; + /** The operations that would lose data, positioned in the plan's operations. */ + readonly dataLoss: readonly PlannedSubject[]; }; /** The origin of a plan from an empty database: no models, so no statement resolves. */ @@ -188,11 +182,12 @@ async function runPlannerLeg( }); } - let plannedOps: readonly MigrationPlanOperation[] = []; + // The operation at each position of the plan, or `undefined` where it is an unfilled placeholder. + let resolved: readonly (MigrationPlanOperation | undefined)[] = []; let hasPlaceholders = false; try { - plannedOps = await Promise.all(plannerResult.plan.operations); - if (plannedOps.length === 0 && !noOperationsExpected) { + resolved = await Promise.all(plannerResult.plan.operations); + if (resolved.length === 0 && !noOperationsExpected) { return notOk({ reason: 'noOperations', error: errorPlanProducedNoOperations(origin, destination), @@ -201,31 +196,43 @@ async function runPlannerLeg( } catch (e) { if (CliStructuredError.is(e) && e.code === 'MIGRATION.UNFILLED_PLACEHOLDER') { hasPlaceholders = true; - // The operations that DID resolve still matter: the destructive-consent - // check must see them, or a placeholder would smuggle a destructive - // baseline past the prompt. Writers stay gated on hasPlaceholders. A - // planner whose `operations` accessor throws synchronously on an - // unfilled placeholder (rather than rejecting one op's promise) exposes - // no operations at all; the check then sees none. + // The operations that DID resolve still matter: the data-loss questions + // must see them, or a placeholder would smuggle a loss past them. + // Writers stay gated on hasPlaceholders. A planner whose `operations` + // accessor throws synchronously on an unfilled placeholder (rather than + // rejecting one op's promise) exposes no operations at all. try { const settled = await Promise.allSettled(plannerResult.plan.operations); - plannedOps = settled.flatMap((entry) => - entry.status === 'fulfilled' ? [entry.value] : [], - ); + resolved = settled.map((entry) => (entry.status === 'fulfilled' ? entry.value : undefined)); } catch { - plannedOps = []; + resolved = []; } } else { throw e; } } + const plannedOps = resolved.filter((op): op is MigrationPlanOperation => op !== undefined); + const position = resolvedPositions(resolved); return ok({ plannedOps, migrationTsContent: plannerResult.plan.renderTypeScript(resolveImportSpecifier), hasPlaceholders, + dataLoss: plannerResult.dataLoss.map(({ operationIndex, subject }) => ({ + operationIndex: position(operationIndex), + subject, + label: + resolved[operationIndex]?.label ?? + subjectText(subject, { origin: fromContract ?? EMPTY_ORIGIN, renames: [] }), + })), appliedStatements: reportAppliedStatements( - plannerResult.appliedStatements, + plannerResult.appliedStatements.map((applied) => ({ + ...applied, + operationIndexes: applied.operationIndexes.flatMap((index) => { + const at = position(index); + return at === undefined ? [] : [at]; + }), + })), fromContract, contract, 0, @@ -233,6 +240,22 @@ async function runPlannerLeg( }); } +/** + * Maps a position in the plan's operations to the position of that operation among those that + * resolved, which is what a result's `operations` lists: `undefined` for an unfilled placeholder. + */ +function resolvedPositions( + resolved: readonly (MigrationPlanOperation | undefined)[], +): (planIndex: number) => number | undefined { + const positions: (number | undefined)[] = []; + let next = 0; + for (const op of resolved) { + positions.push(op === undefined ? undefined : next); + if (op !== undefined) next += 1; + } + return (planIndex) => positions[planIndex]; +} + /** * The operations a result lists. For a leg with unfilled placeholders these are the operations * that resolved; the placeholders themselves are not operations yet. @@ -264,51 +287,67 @@ async function writePlannedMigrationPackage( } /** - * The consent check for an auto-baseline write, mirroring `db update`'s - * destructive-changes refusal: a baseline leg carrying destructive operations - * is only written when the caller consents to that exact plan by its hash. - * Runs before the baseline and delta packages are written, so a refusal - * leaves the app-space migrations directory untouched (the extension seed - * phase runs earlier and unconditionally, as it does for no-op runs). - * A leg with unfilled placeholders is still checked over the operations - * that did resolve. Returns `null` when the write may proceed. + * Asks about every operation of the planned legs that would lose data, until each is answered, + * before anything is written. A delete answer consents to the loss of its subject. A rename typed + * at the prompt is a statement the plan did not have, so the delta is planned again with it, and + * the loss it answered must be gone. */ -function refuseUnconsentedDestructiveBaseline( - leg: PlannerSuccess, - baselineToHash: string, - consent: { readonly planHash: string } | undefined, -): CliStructuredError | null { - const ops = leg.plannedOps; - const destructiveOps = ops.filter((op) => op.operationClass === 'destructive'); - if (destructiveOps.length === 0) { - return null; - } - const planHash = computePlanHash({ - operations: ops.map((op) => ({ - id: op.id, - label: op.label, - operationClass: op.operationClass, - })), - destination: { storageHash: baselineToHash }, +async function answerPlannedDataLoss(input: { + readonly baseline: PlannerSuccess | undefined; + readonly delta: PlannerSuccess | undefined; + readonly renames: readonly StatementText[]; + readonly consents: readonly StatementText[]; + readonly origin: ContractWithDomain; + readonly originKnown: boolean; + readonly keepDataByHand: ((subject: MigrationSubject) => string) | undefined; + readonly destination: ContractWithDomain; + readonly answer: AnswerPlanQuestions; + readonly planDelta: ( + renames: readonly StatementText[], + ) => Promise>; +}): Promise< + Result< + { + readonly delta: PlannerSuccess | undefined; + readonly deletes: readonly AppliedStatementReport[]; + }, + CliStructuredError + > +> { + const legs = (delta: PlannerSuccess | undefined) => ({ + delta, + dataLoss: [...(input.baseline?.dataLoss ?? []), ...(delta?.dataLoss ?? [])], + accessWidening: [], }); - if (consent === undefined) { - const verdict: DestructiveBaselineVerdict = { - destructiveOperations: destructiveOps.map((op) => ({ id: op.id, label: op.label })), - planHash, - }; - return errorDestructiveChanges( - `The baseline migration contains ${destructiveOps.length} destructive operation(s) that require confirmation`, - { - why: 'The migrations directory is empty, so planning writes a baseline derived from the `db` ref — and that baseline contains operations that would remove data when the migration is applied.', - fix: 'Re-run `prisma migration plan` and type the project directory name when asked, or pass `--no-interactive --confirm ` where there is nobody to ask.', - meta: { ...verdict }, - }, - ); - } - if (consent.planHash !== planHash) { - return errorConsentPlanMismatch({ consentedPlanHash: consent.planHash, planHash }); - } - return null; + const answered = await askPlanQuestions({ + plan: legs(input.delta), + askAccess: false, + renames: input.renames, + preAnswers: input.consents, + consentAll: { delete: false, allow: false }, + origin: input.origin, + originKnown: input.originKnown, + keepDataByHand: input.keepDataByHand, + destination: input.destination, + answer: input.answer, + replan: async (renames) => { + const replanned = await input.planDelta(renames); + return replanned.ok ? ok(legs(replanned.value)) : replanned; + }, + }); + if (!answered.ok) return answered; + const { delta } = answered.value.plan; + const deletes = answered.value.consented.map((consented) => + reportConsentStatement( + consented, + (delta?.dataLoss ?? []) + .filter( + (loss) => migrationSubjectKey(loss.subject) === migrationSubjectKey(consented.subject), + ) + .flatMap(({ operationIndex }) => (operationIndex === undefined ? [] : [operationIndex])), + ), + ); + return ok({ delta, deletes }); } export interface MigrationPlanResult { @@ -624,19 +663,9 @@ async function executeMigrationPlanCommandInner( for (const record of seedResult.seeded) { callbacks?.onSeeded?.(record); } - const seededThisRun = seedResult.seeded.flatMap((r) => + const emittedExtensionDirs = seedResult.seeded.flatMap((r) => r.newMigrationDirs.map((dirName) => ({ spaceId: r.spaceId, dirName })), ); - const carried = options.carryEmittedExtensionDirs ?? []; - const emittedExtensionDirs = [ - ...carried, - ...seededThisRun.filter( - (entry) => - !carried.some( - (prior) => prior.spaceId === entry.spaceId && prior.dirName === entry.dirName, - ), - ), - ]; // Check for no-op (same hash means no changes). Auto-baseline is exempt: // an empty graph with db ref at the current contract still needs a @@ -709,6 +738,34 @@ async function executeMigrationPlanCommandInner( hash: toStorageHash, isEmitted: options.to === undefined, }; + const answerDataLoss = ( + baseline: PlannerSuccess | undefined, + delta: PlannerSuccess | undefined, + planDelta: ( + resolved: readonly ResolvedMigrationStatement[], + ) => Promise>, + ) => + answerPlannedDataLoss({ + baseline, + delta, + renames: statementTexts.filter((statement) => statement.verb === 'rename'), + consents: statementTexts.filter((statement) => statement.verb !== 'rename'), + origin: fromContract ?? EMPTY_ORIGIN, + originKnown: true, + keepDataByHand: keepDataByHandFor(migrations, fromContract ?? EMPTY_ORIGIN), + destination: toContract, + answer: options.answerQuestions, + planDelta: async (renames) => { + const resolved = resolveStatements({ + statements: renames, + origin: { kind: 'contract', contract: fromContract ?? EMPTY_ORIGIN }, + destination: toContract, + }); + if (!resolved.ok) return resolved; + const leg = await planDelta(resolved.value); + return leg.ok ? leg : notOk(leg.failure.error); + }, + }); if (isAutoBaseline && fromHash !== null && fromContract !== null && fromContractInStore) { const deltaTimestamp = new Date(); @@ -736,37 +793,46 @@ async function executeMigrationPlanCommandInner( return notOk(baselineLeg.failure.error); } - const consentFailure = refuseUnconsentedDestructiveBaseline( + const planBaselineDelta = (resolved: readonly ResolvedMigrationStatement[]) => + runPlannerLeg( + planner, + migrations, + frameworkComponents, + aggregate.app.contract(), + fromOrigin, + planDestination, + resolved, + false, + aggregate.app.spaceId, + aggregate, + snapshotsImportPathFrom(deltaPackageDir, migrationsDir), + resolveImportSpecifier, + ); + const plannedDelta = + fromHash === toStorageHash ? undefined : await planBaselineDelta(statements); + // A refused delta writes nothing. A delta with no operations still writes the baseline, so + // the `migration new --from` its error advises has the history to start from. + if ( + plannedDelta !== undefined && + !plannedDelta.ok && + plannedDelta.failure.reason === 'refused' + ) { + return notOk(plannedDelta.failure.error); + } + + const answered = await answerDataLoss( baselineLeg.value, - fromHash, - options.consent, + plannedDelta?.ok === true ? plannedDelta.value : undefined, + planBaselineDelta, ); - if (consentFailure !== null) { - return notOk(consentFailure); + if (!answered.ok) { + return notOk(answered.failure); } - const deltaLeg = - fromHash === toStorageHash - ? undefined - : await runPlannerLeg( - planner, - migrations, - frameworkComponents, - aggregate.app.contract(), - fromOrigin, - planDestination, - statements, - false, - aggregate.app.spaceId, - aggregate, - snapshotsImportPathFrom(deltaPackageDir, migrationsDir), - resolveImportSpecifier, - ); - // A refused delta writes nothing. A delta with no operations still writes the baseline, so - // the `migration new --from` its error advises has the history to start from. - if (deltaLeg !== undefined && !deltaLeg.ok && deltaLeg.failure.reason === 'refused') { - return notOk(deltaLeg.failure.error); - } + plannedDelta === undefined || !plannedDelta.ok || answered.value.delta === undefined + ? plannedDelta + : ok(answered.value.delta); + const deletes = answered.value.deletes; await writePlannedMigrationPackage( baselinePackageDir, @@ -801,7 +867,7 @@ async function executeMigrationPlanCommandInner( emittedExtensionDirs, ...(warnings.length > 0 ? { warnings } : {}), pendingPlaceholders: true, - appliedStatements: statementsWithoutOperations, + appliedStatements: [...statementsWithoutOperations, ...deletes], summary: 'Planned baseline with placeholder(s) — edit migration.ts then run `node migration.ts` to self-emit', timings: { total: Date.now() - startTime }, @@ -826,7 +892,7 @@ async function executeMigrationPlanCommandInner( emittedExtensionDirs, ...(preview !== undefined ? { preview } : {}), ...(warnings.length > 0 ? { warnings } : {}), - appliedStatements: statementsWithoutOperations, + appliedStatements: [...statementsWithoutOperations, ...deletes], summary: buildAutoBaselinePlanSummary(baselineOps.length, 0, emittedExtensionDirs.length), timings: { total: Date.now() - startTime }, }; @@ -857,7 +923,7 @@ async function executeMigrationPlanCommandInner( emittedExtensionDirs, ...(warnings.length > 0 ? { warnings } : {}), pendingPlaceholders: true, - appliedStatements: deltaLeg.value.appliedStatements, + appliedStatements: [...deltaLeg.value.appliedStatements, ...deletes], summary: 'Planned baseline + migration with placeholder(s) — edit migration.ts then run `node migration.ts` to self-emit', timings: { total: Date.now() - startTime }, @@ -890,7 +956,7 @@ async function executeMigrationPlanCommandInner( emittedExtensionDirs, ...(preview !== undefined ? { preview } : {}), ...(warnings.length > 0 ? { warnings } : {}), - appliedStatements: deltaLeg.value.appliedStatements, + appliedStatements: [...deltaLeg.value.appliedStatements, ...deletes], summary: buildAutoBaselinePlanSummary( baselineOps.length, deltaOps.length, @@ -906,23 +972,31 @@ async function executeMigrationPlanCommandInner( const dirName = formatMigrationDirName(timestamp, slug); const packageDir = join(appMigrationsDir, dirName); - const deltaLeg = await runPlannerLeg( - planner, - migrations, - frameworkComponents, - aggregate.app.contract(), - fromOrigin, - planDestination, - statements, - statements.length > 0 && fromHash === toStorageHash, - aggregate.app.spaceId, - aggregate, - snapshotsImportPathFrom(packageDir, migrationsDir), - resolveImportSpecifier, - ); - if (!deltaLeg.ok) { - return notOk(deltaLeg.failure.error); + const planDelta = (resolved: readonly ResolvedMigrationStatement[]) => + runPlannerLeg( + planner, + migrations, + frameworkComponents, + aggregate.app.contract(), + fromOrigin, + planDestination, + resolved, + resolved.length > 0 && fromHash === toStorageHash, + aggregate.app.spaceId, + aggregate, + snapshotsImportPathFrom(packageDir, migrationsDir), + resolveImportSpecifier, + ); + const plannedDelta = await planDelta(statements); + if (!plannedDelta.ok) { + return notOk(plannedDelta.failure.error); + } + const answered = await answerDataLoss(undefined, plannedDelta.value, planDelta); + if (!answered.ok) { + return notOk(answered.failure); } + const deltaLeg = ok(answered.value.delta ?? plannedDelta.value); + const deletes = answered.value.deletes; if (!deltaLeg.value.hasPlaceholders && deltaLeg.value.plannedOps.length === 0) { const result: MigrationPlanResult = { @@ -933,7 +1007,7 @@ async function executeMigrationPlanCommandInner( operations: [], emittedExtensionDirs, ...(warnings.length > 0 ? { warnings } : {}), - appliedStatements: deltaLeg.value.appliedStatements, + appliedStatements: [...deltaLeg.value.appliedStatements, ...deletes], summary: 'No changes to plan: the statements need no operations', timings: { total: Date.now() - startTime }, }; @@ -961,7 +1035,7 @@ async function executeMigrationPlanCommandInner( ...(warnings.length > 0 ? { warnings } : {}), pendingPlaceholders: true, ...(fromDefaulted ? { fromDefaulted } : {}), - appliedStatements: deltaLeg.value.appliedStatements, + appliedStatements: [...deltaLeg.value.appliedStatements, ...deletes], summary: 'Planned migration with placeholder(s) — edit migration.ts then run `node migration.ts` to self-emit', timings: { total: Date.now() - startTime }, @@ -988,7 +1062,7 @@ async function executeMigrationPlanCommandInner( ...(preview !== undefined ? { preview } : {}), ...(fromDefaulted ? { fromDefaulted } : {}), ...(warnings.length > 0 ? { warnings } : {}), - appliedStatements: deltaLeg.value.appliedStatements, + appliedStatements: [...deltaLeg.value.appliedStatements, ...deletes], summary: buildPlanSummary(plannedOps.length, emittedExtensionDirs.length), timings: { total: Date.now() - startTime }, }; diff --git a/packages/1-framework/3-tooling/cli/src/control-api/operations/plan-identity.ts b/packages/1-framework/3-tooling/cli/src/control-api/operations/plan-identity.ts deleted file mode 100644 index 5d8817a270a2..000000000000 --- a/packages/1-framework/3-tooling/cli/src/control-api/operations/plan-identity.ts +++ /dev/null @@ -1,24 +0,0 @@ -import { createHash } from 'node:crypto'; -import { canonicalizeJson } from '@internal/framework-components/utils'; - -/** - * Content identity of a `db update` plan: what it does (the ordered display - * operations) and where it lands (the destination storage hash). A - * `DESTRUCTIVE_CHANGES` refusal names the refused plan by this hash, and the - * apply that carries the consent back recomputes it to prove it is still - * applying the plan that was consented to. - */ -export function computePlanHash(plan: { - readonly operations: ReadonlyArray<{ - readonly id: string; - readonly label: string; - readonly operationClass: string; - }>; - readonly destination: { readonly storageHash: string }; -}): string { - const canonical = canonicalizeJson({ - operations: plan.operations, - destination: { storageHash: plan.destination.storageHash }, - }); - return createHash('sha256').update(canonical).digest('hex'); -} diff --git a/packages/1-framework/3-tooling/cli/src/control-api/operations/ref-resolution.ts b/packages/1-framework/3-tooling/cli/src/control-api/operations/ref-resolution.ts index 28eaa661426d..ddd404e444b3 100644 --- a/packages/1-framework/3-tooling/cli/src/control-api/operations/ref-resolution.ts +++ b/packages/1-framework/3-tooling/cli/src/control-api/operations/ref-resolution.ts @@ -17,6 +17,8 @@ import { mapRefResolutionError, requireLiveDatabase, } from '../../utils/cli-errors'; +import { statementFlag } from '../statements/statement-flag'; +import type { StatementText } from '../statements/statement-text'; export interface RefResolutionContext { readonly graph: MigrationGraph; @@ -62,8 +64,8 @@ export function retryCommandFor(args: { readonly from?: string | undefined; readonly to: string | undefined; readonly advanceRef?: string | undefined; - /** The `--rename` statements as given, in order. */ - readonly renames?: readonly string[] | undefined; + /** The statements as given: each verb's values in order, `rename` first. */ + readonly statements?: readonly StatementText[] | undefined; /** The command runs without a database when `--from` names a contract. */ readonly canRunOffline: boolean; }): string { @@ -76,7 +78,7 @@ export function retryCommandFor(args: { ...(suggestsOffline ? ['--from '] : []), ...(args.to === undefined ? [] : [`--to ${args.to}`]), ...(args.advanceRef === undefined ? [] : [`--advance-ref ${args.advanceRef}`]), - ...(args.renames ?? []).map((rename) => `--rename ${rename}`), + ...(args.statements ?? []).map(statementFlag), ...(needsConnection ? ['--db $DATABASE_URL'] : []), ].join(' '); } diff --git a/packages/1-framework/3-tooling/cli/src/control-api/statements/parse-consent.ts b/packages/1-framework/3-tooling/cli/src/control-api/statements/parse-consent.ts new file mode 100644 index 000000000000..47a6d8940efd --- /dev/null +++ b/packages/1-framework/3-tooling/cli/src/control-api/statements/parse-consent.ts @@ -0,0 +1,34 @@ +import type { CliStructuredError } from '@internal/errors/control'; +import { notOk, ok, type Result } from '@internal/utils/result'; +import { errorStatementInvalid } from '../../utils/cli-errors'; +import type { StatementText } from './statement-text'; + +/** A verb whose statement consents to what a plan does to its subject. */ +export type ConsentVerb = 'delete' | 'allow'; + +export interface ParsedConsent extends StatementText { + readonly verb: ConsentVerb; +} + +function formsFix(verb: ConsentVerb): string { + return verb === 'delete' + ? 'Write the statement as --delete , where the name is Model, namespace.Model, Model.field or namespace.Model.field, or the storage name the refusal gave, for example --delete Legacy or --delete User.nickname.' + : 'Write the statement as --allow , where the name is the model the refusal gave, for example --allow User.'; +} + +/** + * Checks the text of one delete or allow statement. The text stays unresolved: it names what a + * plan would do to a subject, so it is matched by equality against the plan's subjects, not + * resolved against the contracts. Any name a subject can have is accepted, since a storage name + * may contain any character a quoted identifier can. + */ +export function parseConsentStatement( + statement: StatementText & { readonly verb: ConsentVerb }, +): Result { + if (statement.text.trim() === '') { + return notOk( + errorStatementInvalid(statement, 'The statement names nothing.', formsFix(statement.verb)), + ); + } + return ok({ verb: statement.verb, text: statement.text }); +} diff --git a/packages/1-framework/3-tooling/cli/src/control-api/statements/plan-questions.ts b/packages/1-framework/3-tooling/cli/src/control-api/statements/plan-questions.ts new file mode 100644 index 000000000000..d25ef495ae6e --- /dev/null +++ b/packages/1-framework/3-tooling/cli/src/control-api/statements/plan-questions.ts @@ -0,0 +1,538 @@ +import type { ContractWithDomain } from '@internal/contract/types'; +import type { CliStructuredError } from '@internal/errors/control'; +import { + type MigrationSubject, + type ModelCoordinate, + migrationSubjectKey, + modelDisplayName, + type ResolvedMigrationStatement, + type TargetMigrationsCapability, +} from '@internal/framework-components/control'; +import { InternalError } from '@internal/utils/internal-error'; +import { notOk, ok, type Result } from '@internal/utils/result'; +import { + errorStatementAnswersNoQuestion, + errorStatementDidNotResolveLoss, + STORE_ORIGIN_SNAPSHOT_STEPS, +} from '../../utils/cli-errors'; +import type { ConsentVerb } from './parse-consent'; +import { resolveStatements } from './resolve-statements'; +import type { StatementText } from './statement-text'; + +/** A verb a plan question accepts. */ +export type PlanQuestionVerb = 'rename' | ConsentVerb; + +/** + * What a command asks about one operation of a plan: the question, its subject written as the + * statement names it, and the verbs that answer it. A command hands it to its prompt. + */ +export interface PlanQuestion { + readonly question: string; + readonly subject: string; + readonly verbs: readonly PlanQuestionVerb[]; + readonly forms: { readonly rename?: string }; + readonly validate: (verb: PlanQuestionVerb, text: string) => string | undefined; +} + +export interface PlanAnswer { + readonly verb: PlanQuestionVerb; + readonly text: string; +} + +/** + * Asks every question at once and returns one answer per question, in question order, with a verb + * and text the question accepts. To refuse, throw. It is called at least once per apply, with an + * empty list when nothing is in question, including under `acceptDataLoss: true`; return `[]` + * then. + */ +export type AnswerPlanQuestions = ( + questions: readonly PlanQuestion[], +) => Promise; + +/** + * An operation of a plan with its subject and the label of that operation. `operationIndex` is + * its position among the operations the result lists; `undefined` when the plan could not resolve + * it. + */ +export interface PlannedSubject { + readonly operationIndex: number | undefined; + readonly label: string; + readonly subject: MigrationSubject; +} + +/** An operation of a plan that changes who can read or write its subject's rows. */ +export interface PlannedAccessChange extends PlannedSubject { + readonly widens: boolean; +} + +/** What a plan would lose, and whose access it would change. */ +export interface PlannedQuestions { + readonly dataLoss: readonly PlannedSubject[]; + readonly accessWidening: readonly PlannedAccessChange[]; +} + +function modelText(contract: ContractWithDomain, coordinate: ModelCoordinate): string { + return Object.keys(contract.domain.namespaces).length > 1 + ? modelDisplayName(coordinate) + : coordinate.model; +} + +/** The contracts a statement names things in, and the renames the plan applies so far. */ +export interface StatementContracts { + readonly origin: ContractWithDomain; + /** The destination contract; needed only to name a field of a renamed model. */ + readonly destination?: ContractWithDomain; + readonly renames: readonly StatementText[]; +} + +/** The model a rename statement of `contracts` gives `coordinate` in the destination, if any. */ +function renamedModel( + coordinate: ModelCoordinate, + contracts: StatementContracts, +): ModelCoordinate | undefined { + if (contracts.destination === undefined || contracts.renames.length === 0) return undefined; + const resolved = resolveStatements({ + statements: contracts.renames, + origin: { kind: 'contract', contract: contracts.origin }, + destination: contracts.destination, + }); + if (!resolved.ok) return undefined; + return resolved.value.find( + (statement) => + statement.entity === 'model' && + statement.from.namespaceId === coordinate.namespaceId && + statement.from.model === coordinate.model, + )?.to; +} + +/** + * The subject as a statement names it: `Model` or `Model.field`, with the namespace when its + * contract has several, or the storage name when no model stores what the operation is about. A + * field of a model the plan renames is named through the model's new name, as a rename of the + * field is written. + */ +export function subjectText(subject: MigrationSubject, contracts: StatementContracts): string { + switch (subject.kind) { + case 'model': + return modelText(contracts.origin, subject); + case 'field': { + const renamed = renamedModel(subject, contracts); + const model = + renamed === undefined || contracts.destination === undefined + ? modelText(contracts.origin, subject) + : modelText(contracts.destination, renamed); + return `${model}.${subject.field}`; + } + case 'storage': + return subject.name; + } +} + +/** The statement's description, written as its question names the subject: `delete field "User.nickname"`. */ +export function consentDescription( + verb: ConsentVerb, + subject: MigrationSubject, + text: string, +): string { + return `${verb} ${subject.kind} "${text}"`; +} + +function lossText(subject: MigrationSubject, text: string, originKnown: boolean): string { + switch (subject.kind) { + case 'model': + return `would lose the data of model "${text}"`; + case 'field': + return `would lose the values of field "${text}"`; + case 'storage': + return originKnown + ? `would lose the data in "${text}", which no model of the origin contract stores` + : `would lose the data in "${text}", named by its storage name because the origin contract is unknown; --delete loses its rows`; + } +} + +/** + * How to keep the rows of a storage subject that was renamed, when the origin contract is unknown: + * store the snapshot of the contract the database is at, as `MIGRATION.STATEMENT_ORIGIN_UNKNOWN` + * advises, then answer with a rename. The refusal gives it once, on its first storage question. The + * engine does not put the CLI's name in a question's text, so the steps name `prisma`. + */ +export const ORIGIN_SNAPSHOT_RECOVERY = [ + 'If it was renamed, keep its data instead: store the snapshot of the contract the database is at, then run this command again and answer with --rename.', + ...STORE_ORIGIN_SNAPSHOT_STEPS.map((step) => step.replaceAll('{bin}', 'prisma')), +].join(' '); + +function sameCoordinate(statement: ResolvedMigrationStatement, subject: MigrationSubject): boolean { + if (subject.kind === 'storage') return false; + const { from } = statement; + if (from.namespaceId !== subject.namespaceId || from.model !== subject.model) return false; + return subject.kind === 'field' + ? statement.entity === 'field' && statement.from.field === subject.field + : statement.entity === 'model'; +} + +function namesSubject(verb: ConsentVerb, subject: string) { + return (text: string): string | undefined => + text === subject ? undefined : `${verb} names "${subject}": --${verb} ${subject}.`; +} + +/** + * How the target says to keep a subject's data by hand when its planner carries out no rename, or + * `undefined` when it carries them out. + */ +export function keepDataByHandFor( + migrations: { readonly renameStatements?: TargetMigrationsCapability['renameStatements'] }, + origin: ContractWithDomain, +): ((subject: MigrationSubject) => string) | undefined { + const refused = migrations.renameStatements; + return refused === undefined ? undefined : (subject) => refused.keepDataByHand(subject, origin); +} + +/** A rename as a statement is written: a field's new name keeps the model, `User.nickname:User.`. */ +function renameForm(subject: MigrationSubject, text: string): string { + return subject.kind === 'field' + ? `${text}:${text.slice(0, text.lastIndexOf('.'))}.` + : `${text}:`; +} + +/** Whether a rename statement of the plan already gives the subject a new name. */ +function renamedByPlan( + subject: MigrationSubject, + contracts: StatementContracts & { readonly destination: ContractWithDomain }, +): boolean { + if (subject.kind === 'storage' || contracts.renames.length === 0) return false; + const resolved = resolveStatements({ + statements: contracts.renames, + origin: { kind: 'contract', contract: contracts.origin }, + destination: contracts.destination, + }); + return resolved.ok && resolved.value.some((statement) => sameCoordinate(statement, subject)); +} + +/** Whether the destination still has the subject's model or field, under the name a rename gives it. */ +function inDestination( + subject: MigrationSubject, + contracts: StatementContracts & { readonly destination: ContractWithDomain }, +): boolean { + if (subject.kind === 'storage') return false; + const model = renamedModel(subject, contracts) ?? subject; + const destinationModel = + contracts.destination.domain.namespaces[model.namespaceId]?.models[model.model]; + if (destinationModel === undefined) return false; + return subject.kind === 'model' || Object.hasOwn(destinationModel.fields, subject.field); +} + +/** + * The question for one operation that would lose data. A model or field the destination no longer + * has, and no rename of the plan renamed, may be renamed instead of deleted; a rename is resolved + * against the two contracts after the plan's other renames, and its old name must be the subject. + * A subject the destination keeps, such as a field whose type changes, and data no model stores, + * can only be deleted. Where the target's planner carries out no rename, `keepDataByHand` says how + * to keep the data instead. + */ +export function dataLossQuestion( + loss: PlannedSubject, + contracts: StatementContracts & { + readonly destination: ContractWithDomain; + readonly originKnown: boolean; + readonly keepDataByHand: ((subject: MigrationSubject) => string) | undefined; + }, +): PlanQuestion { + const subject = subjectText(loss.subject, contracts); + const renamable = + contracts.keepDataByHand === undefined && + loss.subject.kind !== 'storage' && + !inDestination(loss.subject, contracts) && + !renamedByPlan(loss.subject, contracts); + const deleteNames = namesSubject('delete', subject); + return { + question: [ + `${loss.label} ${lossText(loss.subject, subject, contracts.originKnown)}.`, + ...(contracts.keepDataByHand === undefined ? [] : [contracts.keepDataByHand(loss.subject)]), + ].join(' '), + subject, + verbs: renamable ? ['rename', 'delete'] : ['delete'], + forms: renamable ? { rename: renameForm(loss.subject, subject) } : {}, + validate: (verb, text) => { + if (verb !== 'rename') return deleteNames(text); + const resolved = resolveStatements({ + statements: [...contracts.renames, { verb: 'rename', text }], + origin: { kind: 'contract', contract: contracts.origin }, + destination: contracts.destination, + }); + if (!resolved.ok) return `${resolved.failure.message}. ${resolved.failure.why ?? ''}`.trim(); + const statement = resolved.value.at(-1); + return statement !== undefined && sameCoordinate(statement, loss.subject) + ? undefined + : `The rename's old name is not "${subject}". Write it as ${renameForm(loss.subject, subject)}.`; + }, + }; +} + +/** + * The question for one operation that changes who can read or write its subject's rows: it widens + * access, as disabling row-level security does, or changes it either way, as dropping a policy + * does. Each such operation is its own question, answered by its own `allow`. + */ +export function accessWideningQuestion( + widening: PlannedAccessChange, + contracts: StatementContracts, +): PlanQuestion { + const subject = subjectText(widening.subject, contracts); + return { + question: `${widening.label} would ${widening.widens ? 'widen' : 'change'} who can read and write its rows.`, + subject, + verbs: ['allow'], + forms: {}, + validate: (_verb, text) => namesSubject('allow', subject)(text), + }; +} + +/** A subject the user consented to, with the statement's text as its question wrote it. */ +export interface ConsentedSubject { + readonly verb: ConsentVerb; + readonly subject: MigrationSubject; + readonly text: string; + /** The position of the operation an `allow` consented to; a `delete` covers every loss of its subject. */ + readonly operationIndex: number | undefined; +} + +const keyOf = (planned: PlannedSubject) => migrationSubjectKey(planned.subject); + +/** + * The pre-answers that answer a question about `text`: every `delete` of the subject, or the + * first `allow` of it no earlier question took, since an `allow` answers one operation. + */ +function givenFor( + verb: ConsentVerb, + text: string, + preAnswers: readonly StatementText[], + taken: ReadonlySet, +): readonly StatementText[] { + const matching = preAnswers.filter( + (statement) => statement.verb === verb && statement.text === text, + ); + if (verb === 'delete') return matching; + const first = matching.find((statement) => !taken.has(statement)); + return first === undefined ? [] : [first]; +} + +/** + * A `delete` answers for its subject; an `allow` for its one operation, known by its label and + * subject, since a re-plan moves every operation's position. + */ +function questionKey(verb: ConsentVerb, entry: PlannedSubject): string { + return verb === 'allow' ? `allow:${entry.label}:${keyOf(entry)}` : `delete:${keyOf(entry)}`; +} + +function checkAnswers(questions: readonly PlanQuestion[], answers: readonly PlanAnswer[]): void { + if (answers.length !== questions.length) { + const noun = (count: number, word: string) => `${count} ${word}${count === 1 ? '' : 's'}`; + throw new InternalError( + `answerQuestions gave ${noun(answers.length, 'answer')} for ${noun(questions.length, 'question')}; answer each question in order, or throw to refuse.`, + { cause: { questions: questions.map(({ subject }) => subject), answers } }, + ); + } + questions.forEach((question, index) => { + const answer = answers[index]; + if (answer === undefined) return; + const cause = { cause: { question: question.subject, verbs: question.verbs, answer } }; + if (!question.verbs.includes(answer.verb)) { + throw new InternalError( + `answerQuestions answered "${question.question}" with ${answer.verb}, which it does not accept; it accepts ${question.verbs.join(' or ')}.`, + cause, + ); + } + const rejected = question.validate(answer.verb, answer.text); + if (rejected !== undefined) { + throw new InternalError( + `answerQuestions answered "${question.question}" wrongly: ${rejected}`, + cause, + ); + } + }); +} + +function withRecoveryOnFirstStorageQuestion( + asked: readonly { readonly loss: PlannedSubject; readonly question: PlanQuestion }[], + recoveryApplies: boolean, +): readonly PlanQuestion[] { + const first = recoveryApplies + ? asked.findIndex(({ loss }) => loss.subject.kind === 'storage') + : -1; + return asked.map(({ question }, index) => + index === first + ? { ...question, question: `${question.question} ${ORIGIN_SNAPSHOT_RECOVERY}` } + : question, + ); +} + +/** + * Asks about every operation of a plan that would lose data and, when `askAccess`, every one that + * would widen access, until each is answered. A `delete` or `allow` text in `preAnswers`, or + * `consentAll` for its verb, answers its question without asking. A rename typed at the prompt is a statement + * the plan did not have, so the plan is made again with it, and the operation it answered must be + * gone; a loss the new plan has on the same subject, such as a type change on the renamed field, + * is asked in the next round. The first round asks even when nothing is in question, so a statement no question consumed is + * refused before anything is done. + */ +export async function askPlanQuestions(input: { + readonly plan: TPlan; + readonly askAccess: boolean; + readonly renames: readonly StatementText[]; + readonly preAnswers: readonly StatementText[]; + readonly consentAll: { readonly [verb in ConsentVerb]: boolean }; + readonly origin: ContractWithDomain; + readonly originKnown: boolean; + readonly keepDataByHand: ((subject: MigrationSubject) => string) | undefined; + readonly destination: ContractWithDomain; + readonly answer: AnswerPlanQuestions; + readonly replan: (renames: readonly StatementText[]) => Promise>; +}): Promise< + Result< + { + readonly plan: TPlan; + readonly renames: readonly StatementText[]; + readonly consented: readonly ConsentedSubject[]; + }, + TFailure | CliStructuredError + > +> { + let plan = input.plan; + let renames = input.renames; + const consented = new Map(); + const usedPreAnswers = new Set(); + for (let round = 0; ; round += 1) { + const contracts = { + origin: input.origin, + originKnown: input.originKnown, + keepDataByHand: input.keepDataByHand, + destination: input.destination, + renames, + }; + const pending = ( + verb: ConsentVerb, + planned: readonly TEntry[], + ): readonly TEntry[] => [ + ...new Map( + planned + .filter((entry) => !consented.has(questionKey(verb, entry))) + .map((entry) => [questionKey(verb, entry), entry]), + ).values(), + ]; + const consent = (verb: ConsentVerb, entry: PlannedSubject, text: string) => + consented.set(questionKey(verb, entry), { + verb, + subject: entry.subject, + text, + operationIndex: entry.operationIndex, + }); + for (const [verb, entries] of [ + ['delete', pending('delete', plan.dataLoss)], + ['allow', input.askAccess ? pending('allow', plan.accessWidening) : []], + ] as const) { + for (const entry of entries) { + const text = subjectText(entry.subject, contracts); + const given = givenFor(verb, text, input.preAnswers, usedPreAnswers); + for (const statement of given) usedPreAnswers.add(statement); + if (input.consentAll[verb] || given.length > 0) consent(verb, entry, text); + } + } + const losses = pending('delete', plan.dataLoss); + const widenings = input.askAccess ? pending('allow', plan.accessWidening) : []; + if (losses.length + widenings.length === 0 && round > 0) break; + const questions = [ + ...withRecoveryOnFirstStorageQuestion( + losses.map((loss) => ({ loss, question: dataLossQuestion(loss, contracts) })), + !input.originKnown && input.keepDataByHand === undefined, + ), + ...widenings.map((widening) => accessWideningQuestion(widening, contracts)), + ]; + const answers = await input.answer(questions); + checkAnswers(questions, answers); + const typedRenames: { readonly text: string; readonly loss: PlannedSubject }[] = []; + answers.forEach((answer, index) => { + const entry = [...losses, ...widenings][index]; + const question = questions[index]; + if (entry === undefined || question === undefined) return; + if (answer.verb === 'rename') typedRenames.push({ text: answer.text, loss: entry }); + else consent(answer.verb, entry, question.subject); + }); + if (typedRenames.length === 0) { + if (losses.length + widenings.length === 0) break; + continue; + } + renames = [...renames, ...typedRenames.map(({ text }) => ({ verb: 'rename' as const, text }))]; + const replanned = await input.replan(renames); + if (!replanned.ok) return replanned; + plan = replanned.value; + for (const entry of plan.accessWidening) { + const allowed = consented.get(questionKey('allow', entry)); + if (allowed !== undefined) { + consented.set(questionKey('allow', entry), { + ...allowed, + operationIndex: entry.operationIndex, + }); + } + } + const operationOf = (entry: PlannedSubject) => `${entry.label}:${keyOf(entry)}`; + const stillPlanned = new Set(plan.dataLoss.map(operationOf)); + const unresolved = typedRenames.find(({ loss }) => stillPlanned.has(operationOf(loss))); + if (unresolved !== undefined) { + return notOk( + errorStatementDidNotResolveLoss( + { verb: 'rename', text: unresolved.text }, + subjectText(unresolved.loss.subject, contracts), + ), + ); + } + } + const unused = input.preAnswers.filter((statement) => !usedPreAnswers.has(statement)); + if (unused.length > 0) { + return notOk( + errorStatementAnswersNoQuestion( + unused, + questionSubjects(plan, input.askAccess, { ...input, renames }), + ), + ); + } + return ok({ plan, renames, consented: [...consented.values()] }); +} + +function questionSubjects( + plan: PlannedQuestions, + askAccess: boolean, + contracts: StatementContracts, +): readonly string[] { + const subjects = [...plan.dataLoss, ...(askAccess ? plan.accessWidening : [])].map((entry) => + subjectText(entry.subject, contracts), + ); + return [...new Set(subjects)]; +} + +/** + * Refuses the `delete` and `allow` statements that name no subject a plan asks about, without + * asking anything: a run that only plans still says which statements would consent to nothing. + */ +export function refuseUnusedConsents(input: { + readonly plan: PlannedQuestions; + readonly statements: readonly StatementText[]; + readonly contracts: StatementContracts; +}): Result { + const subjects = questionSubjects(input.plan, true, input.contracts); + const used = new Set(); + for (const [verb, entries] of [ + ['delete', input.plan.dataLoss], + ['allow', input.plan.accessWidening], + ] as const) { + for (const entry of entries) { + const text = subjectText(entry.subject, input.contracts); + for (const statement of givenFor(verb, text, input.statements, used)) used.add(statement); + } + } + const unused = input.statements.filter( + (statement) => statement.verb !== 'rename' && !used.has(statement), + ); + return unused.length === 0 + ? ok(undefined) + : notOk(errorStatementAnswersNoQuestion(unused, subjects)); +} diff --git a/packages/1-framework/3-tooling/cli/src/control-api/statements/report-applied-statements.ts b/packages/1-framework/3-tooling/cli/src/control-api/statements/report-applied-statements.ts index 101143dbcaf9..60456e16da1d 100644 --- a/packages/1-framework/3-tooling/cli/src/control-api/statements/report-applied-statements.ts +++ b/packages/1-framework/3-tooling/cli/src/control-api/statements/report-applied-statements.ts @@ -3,19 +3,38 @@ import { type AppliedMigrationStatement, describeMigrationStatement, type MigrationStatementJson, + type MigrationSubjectJson, migrationStatementJson, + migrationSubjectJson, } from '@internal/framework-components/control'; +import type { ConsentVerb } from './parse-consent'; +import { type ConsentedSubject, consentDescription } from './plan-questions'; + +/** A delete or allow statement in JSON output: the subject whose data or access it consents to. */ +export interface ConsentStatementJson { + readonly kind: ConsentVerb; + readonly subject: MigrationSubjectJson; +} /** - * A statement a plan applied, as the CLI reports it: the statement as JSON output writes it, the - * positions of its operations in the result's `operations`, and its description. + * A statement a plan applied, as the CLI reports it: its verb, the statement as JSON output writes + * it, the positions of its operations in the result's `operations`, and its description. */ -export interface AppliedStatementReport { - readonly statement: MigrationStatementJson; - readonly operationIndexes: readonly number[]; - /** The statement in domain names, for example `rename model "Profile" to "User"`. */ - readonly description: string; -} +export type AppliedStatementReport = + | { + readonly verb: 'rename'; + readonly statement: MigrationStatementJson; + readonly operationIndexes: readonly number[]; + /** The statement in domain names, for example `rename model "Profile" to "User"`. */ + readonly description: string; + } + | { + readonly verb: ConsentVerb; + readonly statement: ConsentStatementJson; + readonly operationIndexes: readonly number[]; + /** The statement as its question named the subject, for example `delete model "Legacy"`. */ + readonly description: string; + }; /** * `operationOffset` is the number of operations the result lists before the plan that applied the @@ -28,8 +47,22 @@ export function reportAppliedStatements( operationOffset: number, ): readonly AppliedStatementReport[] { return applied.map((entry) => ({ + verb: 'rename', statement: migrationStatementJson(entry.statement), operationIndexes: entry.operationIndexes.map((index) => operationOffset + index), description: describeMigrationStatement(entry.statement, fromContract ?? contract, contract), })); } + +/** A delete or allow statement a plan applied, with the positions of the operations it consented to. */ +export function reportConsentStatement( + consented: ConsentedSubject, + operationIndexes: readonly number[], +): AppliedStatementReport { + return { + verb: consented.verb, + statement: { kind: consented.verb, subject: migrationSubjectJson(consented.subject) }, + operationIndexes, + description: consentDescription(consented.verb, consented.subject, consented.text), + }; +} diff --git a/packages/1-framework/3-tooling/cli/src/control-api/statements/resolve-statements.ts b/packages/1-framework/3-tooling/cli/src/control-api/statements/resolve-statements.ts index b272fcc2db36..8105d8d9cc3b 100644 --- a/packages/1-framework/3-tooling/cli/src/control-api/statements/resolve-statements.ts +++ b/packages/1-framework/3-tooling/cli/src/control-api/statements/resolve-statements.ts @@ -19,6 +19,7 @@ import { errorStatementOriginUnknown, errorStatementUnresolved, } from '../../utils/cli-errors'; +import { parseConsentStatement } from './parse-consent'; import { mixedSidesFix, type ParsedRename, @@ -693,17 +694,30 @@ function resolveStatement( } /** - * Resolves the statements a user gave, in order, against the origin and + * Resolves the rename statements a user gave, in order, against the origin and * destination contracts. Each resolved statement names its old and new entity - * by namespace, model and field. + * by namespace, model and field. A delete or allow statement is checked and + * left out: it names what a plan would do to a subject, so it is matched against + * the plan, and it needs no origin contract. */ export function resolveStatements( input: ResolveStatementsInput, ): Result { - if (input.statements.length === 0) return ok([]); - if (input.origin.kind === 'missing') { - return notOk(errorStatementOriginUnknown(input.origin)); + const parsed: ParsedRename[] = []; + for (const entry of input.statements) { + if (entry.verb === 'delete' || entry.verb === 'allow') { + const statement = parseConsentStatement({ ...entry, verb: entry.verb }); + if (!statement.ok) return statement; + continue; + } + if (input.origin.kind === 'missing') { + return notOk(errorStatementOriginUnknown(input.origin)); + } + const statement = parseRenameStatement({ ...entry, verb: entry.verb }); + if (!statement.ok) return statement; + parsed.push(statement.value); } + if (parsed.length === 0 || input.origin.kind === 'missing') return ok([]); const state: ResolutionState = { origin: { name: 'origin', contract: input.origin.contract }, destination: { name: 'destination', contract: input.destination }, @@ -711,12 +725,6 @@ export function resolveStatements( renamedFrom: new Set(), renamedTo: new Set(), }; - const parsed: ParsedRename[] = []; - for (const entry of input.statements) { - const statement = parseRenameStatement(entry); - if (!statement.ok) return statement; - parsed.push(statement.value); - } const resolved: ResolvedMigrationStatement[] = []; for (const [index, statement] of parsed.entries()) { const result = resolveStatement(state, statement, parsed.slice(index + 1)); diff --git a/packages/1-framework/3-tooling/cli/src/control-api/statements/statement-flag.ts b/packages/1-framework/3-tooling/cli/src/control-api/statements/statement-flag.ts new file mode 100644 index 000000000000..4f5dddc7faae --- /dev/null +++ b/packages/1-framework/3-tooling/cli/src/control-api/statements/statement-flag.ts @@ -0,0 +1,20 @@ +import type { StatementText } from './statement-text'; + +const SHELL_SAFE = /^[A-Za-z0-9_.:/@%+,-]+$/; + +/** The value as a shell passes it through unchanged, single-quoted when needed. */ +export function shellQuoted(value: string): string { + return SHELL_SAFE.test(value) ? value : `'${value.replaceAll("'", `'\\''`)}'`; +} + +/** + * A statement written as a flag a shell passes through unchanged: the value is single-quoted when + * it has a character a shell would read, and joined with `=` when it starts with `-`, which a + * parser would otherwise read as a flag. + */ +export function statementFlag(statement: StatementText): string { + const value = shellQuoted(statement.text); + return statement.text.startsWith('-') + ? `--${statement.verb}=${value}` + : `--${statement.verb} ${value}`; +} diff --git a/packages/1-framework/3-tooling/cli/src/control-api/statements/statement-text.ts b/packages/1-framework/3-tooling/cli/src/control-api/statements/statement-text.ts index f6e2cf9706f5..053892c83eee 100644 --- a/packages/1-framework/3-tooling/cli/src/control-api/statements/statement-text.ts +++ b/packages/1-framework/3-tooling/cli/src/control-api/statements/statement-text.ts @@ -1,9 +1,10 @@ /** The verbs a statement can have. */ -export type StatementVerb = 'rename'; +export type StatementVerb = 'rename' | 'delete' | 'allow'; /** * A statement as the user wrote it, before it is parsed: its verb and its text, for example - * `{ verb: 'rename', text: 'Profile:User' }` for `--rename Profile:User`. + * `{ verb: 'rename', text: 'Profile:User' }` for `--rename Profile:User` or + * `{ verb: 'delete', text: 'Legacy' }` for `--delete Legacy`. */ export interface StatementText { readonly verb: StatementVerb; diff --git a/packages/1-framework/3-tooling/cli/src/control-api/testing/fixture-client.ts b/packages/1-framework/3-tooling/cli/src/control-api/testing/fixture-client.ts index f42474dcec4d..2472e55e0597 100644 --- a/packages/1-framework/3-tooling/cli/src/control-api/testing/fixture-client.ts +++ b/packages/1-framework/3-tooling/cli/src/control-api/testing/fixture-client.ts @@ -125,6 +125,8 @@ export function defaultControlClientFixtures(): ControlClientFixtures { const applySuccess = { mode: 'apply' as const, appliedStatements: [], + dataLoss: [], + accessWidening: [], plan: { operations: [fixtureOperation] }, destination: contract, execution: { operationsPlanned: 1, operationsExecuted: 1 }, diff --git a/packages/1-framework/3-tooling/cli/src/control-api/types.ts b/packages/1-framework/3-tooling/cli/src/control-api/types.ts index 7fe6d7ea92c5..6a9f681debd9 100644 --- a/packages/1-framework/3-tooling/cli/src/control-api/types.ts +++ b/packages/1-framework/3-tooling/cli/src/control-api/types.ts @@ -13,8 +13,11 @@ import type { ControlFamilyDescriptor, ControlTargetDescriptor, CoreSchemaView, + MigrationAccessChange, + MigrationOperationSubject, MigrationPlannerConflict, MigrationPlanOperation, + MigrationPlanSubjects, OperationPreview, VerifyDatabaseResult, VerifyDatabaseSchemaResult, @@ -24,6 +27,7 @@ import type { Result } from '@internal/utils/result'; import type { ExecuteDbSignResult } from './operations/db-sign'; import type { ExecuteDbVerifyResult } from './operations/db-verify'; import type { RenderContractDtsOptions, RenderContractDtsResult } from './render-contract-dts'; +import type { AnswerPlanQuestions } from './statements/plan-questions'; import type { AppliedStatementReport } from './statements/report-applied-statements'; import type { StatementText } from './statements/statement-text'; @@ -210,22 +214,10 @@ export interface DbUpdateOptions { * The type is driver-specific (e.g., string URL for Postgres). */ readonly connection?: unknown; - /** - * When true, allows applying plans that contain destructive operations - * (e.g., DROP TABLE, DROP COLUMN, ALTER TYPE). - * When false (default), the operation returns a failure if the plan - * includes destructive operations, which `db update` turns into its consent - * prompt: the user types the database name, or passes `--confirm ` - * where there is nobody to ask. - */ + /** Consents to every operation that would lose data, without asking. */ readonly acceptDataLoss?: boolean; - /** - * Consent to the destructive plan a prior `DESTRUCTIVE_CHANGES` refusal - * named, identified by that refusal's `planHash`. The apply recomputes its - * plan and refuses with `CONSENT_PLAN_MISMATCH` when the fresh plan is not - * the one that was consented to. - */ - readonly consent?: { readonly planHash: string }; + /** Consents to every operation that would widen who can read or write rows, without asking. */ + readonly acceptAccessWidening?: boolean; /** * On-disk migrations directory. Always required — every `db update` * routes through the per-space flow, which reads on-disk @@ -234,11 +226,19 @@ export interface DbUpdateOptions { */ readonly migrationsDir: string; /** - * The statements as the user wrote them, in the order given. They resolve - * against the contract the database marker names, read from the snapshot - * store, and the destination contract. + * The statements as the user wrote them, in the order given. A `rename` + * resolves against the contract the database marker names, read from the + * snapshot store, and the destination contract; a `delete` or `allow` + * answers the question about the subject it names. */ readonly statements?: readonly StatementText[]; + /** + * Asks, before an apply, what each operation that would lose data means and + * whether each that would widen who can read or write rows may run, for the + * questions no statement answered. The `db update` command asks through its + * prompt. + */ + readonly answerQuestions: AnswerPlanQuestions; /** Optional progress callback for observing operation progress */ readonly onProgress?: OnControlProgress; } @@ -450,10 +450,18 @@ export interface DbInitFailure { */ export type DbInitResult = Result; +/** An operation an apply asks about, with its subject written as the question writes it: `User.nickname`. */ +export interface AskedSubject extends MigrationOperationSubject { + readonly text: string; +} + +/** An operation an apply asks about because it changes who can read or write its subject's rows. */ +export interface AskedAccessChange extends AskedSubject, MigrationAccessChange {} + /** * Successful dbUpdate result. */ -export interface DbUpdateSuccess { +export interface DbUpdateSuccess extends MigrationPlanSubjects { readonly mode: 'plan' | 'apply'; readonly plan: { readonly operations: ReadonlyArray<{ @@ -496,40 +504,7 @@ export interface DbUpdateSuccess { /** * Failure codes for dbUpdate operation. */ -export type DbUpdateFailureCode = - | 'PLANNING_FAILED' - | 'RUNNER_FAILED' - | 'DESTRUCTIVE_CHANGES' - | 'CONSENT_PLAN_MISMATCH'; - -/** One planned operation whose class is destructive, as the refusal names it. */ -export interface DestructivePlanOperation { - readonly id: string; - readonly label: string; -} - -/** - * The verdict a `DESTRUCTIVE_CHANGES` refusal carries: what the plan would - * destroy, the database it would destroy it in, and the identity of the plan - * that was refused. Consent is granted against `planHash` — the caller passes - * it back as `DbUpdateOptions.consent`. - */ -export interface DestructiveChangesVerdict { - readonly destructiveOperations: ReadonlyArray; - /** The connected database's name, when the driver can name it. */ - readonly databaseName: string | undefined; - /** Content hash of the refused plan. */ - readonly planHash: string; -} - -/** - * Why an apply carrying consent was refused: the plan recomputed for the - * apply is not the plan that was consented to. - */ -export interface ConsentPlanMismatchVerdict { - readonly consentedPlanHash: string; - readonly planHash: string; -} +export type DbUpdateFailureCode = 'PLANNING_FAILED' | 'RUNNER_FAILED'; /** * Failure details for dbUpdate operation. @@ -541,10 +516,6 @@ export interface DbUpdateFailure { readonly conflicts: ReadonlyArray | undefined; readonly warnings?: ReadonlyArray; readonly meta: Record | undefined; - /** Present exactly when `code` is `'DESTRUCTIVE_CHANGES'`. */ - readonly destructiveChanges?: DestructiveChangesVerdict; - /** Present exactly when `code` is `'CONSENT_PLAN_MISMATCH'`. */ - readonly consentPlanMismatch?: ConsentPlanMismatchVerdict; /** Underlying failure or error for diagnostics; never serialized into envelopes. */ readonly cause?: unknown; } diff --git a/packages/1-framework/3-tooling/cli/src/exports/control-api.ts b/packages/1-framework/3-tooling/cli/src/exports/control-api.ts index 97b2c37c955e..07b76bd13077 100644 --- a/packages/1-framework/3-tooling/cli/src/exports/control-api.ts +++ b/packages/1-framework/3-tooling/cli/src/exports/control-api.ts @@ -11,6 +11,9 @@ // Re-export core control plane types for consumer convenience export type { ControlStack, + MigrationOperationSubject, + MigrationPlanSubjects, + MigrationSubject, ResolvedMigrationStatement, VerifyDatabaseResult, VerifyDatabaseSchemaResult, @@ -142,6 +145,12 @@ export type { RenderContractDtsResult, RenderContractDtsSuccess, } from '../control-api/render-contract-dts'; +export type { + AnswerPlanQuestions, + PlanAnswer, + PlanQuestion, + PlanQuestionVerb, +} from '../control-api/statements/plan-questions'; export type { AppliedStatementReport } from '../control-api/statements/report-applied-statements'; export { type ResolveStatementsInput, @@ -151,6 +160,7 @@ export { export type { StatementText, StatementVerb } from '../control-api/statements/statement-text'; // CLI-specific types export type { + AskedSubject, ContractEmitOptions, ContractEmitResult, ControlActionName, diff --git a/packages/1-framework/3-tooling/cli/src/orm/db/consent.ts b/packages/1-framework/3-tooling/cli/src/orm/db/consent.ts deleted file mode 100644 index 545bd08173af..000000000000 --- a/packages/1-framework/3-tooling/cli/src/orm/db/consent.ts +++ /dev/null @@ -1,59 +0,0 @@ -import { CliStructuredError } from '@prisma/cli-engine/protocol'; -import type { DestructivePlanOperation } from '../../control-api/types'; - -/** - * A token nobody can type is not consent: an empty one makes the engine's - * type-to-confirm accept a bare Enter, and `--confirm ""` grant outright. - */ -export function errorConsentTokenUnresolved(targetId: string): CliStructuredError { - return new CliStructuredError( - 'CLI.CONSENT_TOKEN_UNRESOLVED', - 'The database this run would change has no name to confirm.', - { - why: `The connected database reports no name, and the target id "${targetId}" yields none either, so the consent prompt has nothing to ask you to type and the destructive operations cannot be authorised.`, - nextActions: [ - { - kind: 'user-choice', - label: - 'Name the database in `db.connection` (or pass `--db `) and run `{bin} db update` again.', - }, - ], - }, - ); -} - -/** A prompt that names nothing cannot be consented to knowingly. */ -export function errorConsentOperationsMissing(options?: { - readonly previewCommand?: string; -}): CliStructuredError { - return new CliStructuredError( - 'CLI.CONSENT_OPERATIONS_MISSING', - 'The plan was refused as destructive but named no operations to confirm.', - { - why: 'The consent prompt has to list what is about to be destroyed, and the refusal carried no operations to list.', - nextActions: [ - { - kind: 'run-command', - label: 'Preview the plan', - command: options?.previewCommand ?? '{bin} db update --dry-run', - }, - ], - }, - ); -} - -/** The indented per-operation list every destructive-consent question renders. */ -export function destructiveOperationList(operations: readonly DestructivePlanOperation[]): string { - return operations.map((operation) => ` - ${operation.label}`).join('\n'); -} - -/** The question the user answers before anything is dropped. */ -export function destructiveConsentQuestion( - operations: readonly DestructivePlanOperation[], - token: string, -): string { - return [ - `Apply ${operations.length} destructive operation(s) to ${token}? Data they remove cannot be recovered:`, - destructiveOperationList(operations), - ].join('\n'); -} diff --git a/packages/1-framework/3-tooling/cli/src/orm/db/migration-blocks.ts b/packages/1-framework/3-tooling/cli/src/orm/db/migration-blocks.ts index 466431422b25..5fd651d371f5 100644 --- a/packages/1-framework/3-tooling/cli/src/orm/db/migration-blocks.ts +++ b/packages/1-framework/3-tooling/cli/src/orm/db/migration-blocks.ts @@ -1,6 +1,7 @@ import type { OperationPreview } from '@internal/framework-components/control'; import type { Block, TreeNode } from '@prisma/cli-engine'; import type { NextAction } from '@prisma/cli-engine/protocol'; +import { statementFlag } from '../../control-api/statements/statement-flag'; import type { PerSpaceExecutionEntry } from '../../control-api/types'; import { type MigrationCommandResult, @@ -153,6 +154,33 @@ function applySummaryText(result: MigrationCommandResult): string { return `Applied ${executed} operation(s)${across}`; } +/** + * The questions an apply would ask, as a dry run lists them: each operation that would lose data, + * answered by `--delete` or `--rename`, and each that would widen access, answered by `--allow`. + */ +function questionBlocks(result: MigrationCommandResult): readonly Block[] { + const entries = [ + ...(result.dataLoss ?? []).map((entry) => ({ entry, verb: 'delete' as const })), + ...(result.accessWidening ?? []).map((entry) => ({ entry, verb: 'allow' as const })), + ]; + if (entries.length === 0) { + return []; + } + return [ + { + kind: 'tree', + roots: [ + { + label: 'An apply asks about', + children: entries.map(({ entry, verb }) => ({ + label: `${result.plan.operations[entry.operationIndex]?.label ?? 'an operation'}: ${statementFlag({ verb, text: entry.text })}${entry.answered ? ' (answered)' : ''}`, + })), + }, + ], + }, + ]; +} + function planBlocks(result: MigrationCommandResult): readonly Block[] { const planned = result.plannedAdvanceRef; return [ @@ -160,6 +188,7 @@ function planBlocks(result: MigrationCommandResult): readonly Block[] { ...plannerWarningBlocks(result), ...operationBlocks(result), ...appliedStatementBlocks(result.appliedStatements ?? []), + ...questionBlocks(result), { kind: 'fields', rows: [ diff --git a/packages/1-framework/3-tooling/cli/src/orm/db/prepare.ts b/packages/1-framework/3-tooling/cli/src/orm/db/prepare.ts index 2dec9e5b6997..1db207b8f348 100644 --- a/packages/1-framework/3-tooling/cli/src/orm/db/prepare.ts +++ b/packages/1-framework/3-tooling/cli/src/orm/db/prepare.ts @@ -74,8 +74,11 @@ export async function prepareMigrationRun(inputs: { readonly db: string | undefined; readonly commandName: string; readonly createClient: CreateControlClient; - /** The command the missing-connection error suggests; defaults to `{bin} --db `. */ - readonly retryCommand?: string; + /** + * The command the missing-connection error suggests; defaults to `{bin} --db `. + * A function is called only when the error is raised. + */ + readonly retryCommand?: string | (() => string); }): Promise> { const { config, cwd, commandName } = inputs; const contractPath = contractPathFor(config); @@ -103,7 +106,10 @@ export async function prepareMigrationRun(inputs: { why: `Database connection is required for ${commandName} (set db.connection in prisma.config.ts, or pass --db )`, commandName, missingFlags: ['--db'], - ...ifDefined('retryCommand', inputs.retryCommand), + ...ifDefined( + 'retryCommand', + typeof inputs.retryCommand === 'function' ? inputs.retryCommand() : inputs.retryCommand, + ), }), ), ); diff --git a/packages/1-framework/3-tooling/cli/src/orm/db/update.ts b/packages/1-framework/3-tooling/cli/src/orm/db/update.ts index 5995bd1319a5..0b2215f1916a 100644 --- a/packages/1-framework/3-tooling/cli/src/orm/db/update.ts +++ b/packages/1-framework/3-tooling/cli/src/orm/db/update.ts @@ -1,4 +1,5 @@ import { ormConfigSection } from '@internal/config-loader'; +import { migrationSubjectJson } from '@internal/framework-components/control'; import { ifDefined } from '@internal/utils/defined'; import { isStructuredError } from '@internal/utils/structured-error'; import type { Block, Presentations } from '@prisma/cli-engine'; @@ -22,8 +23,9 @@ import { preflightRefAdvancement, } from '../../control-api/operations/ref-advancement'; import { retryCommandFor } from '../../control-api/operations/ref-resolution'; -import { renameStatements } from '../../control-api/statements/statement-text'; -import type { CreateControlClient, DbUpdateResult, DbUpdateSuccess } from '../../control-api/types'; +import { shellQuoted, statementFlag } from '../../control-api/statements/statement-flag'; +import type { StatementText } from '../../control-api/statements/statement-text'; +import type { AskedSubject, CreateControlClient, DbUpdateSuccess } from '../../control-api/types'; import { CliStructuredError, errorContractValidationFailed } from '../../utils/cli-errors'; import { closeQuietly } from '../../utils/command-helpers'; import { RECORDED_CONTRACT_REF_FORMS } from '../../utils/contract-ref-forms'; @@ -33,12 +35,9 @@ import { defineOrmCommand } from '../define-command'; import { dbFlag } from '../flags'; import { baseDirFor, migrationsDirFor } from '../migration/paths'; import { normalizeError } from '../normalize-error'; +import { promptPlanQuestions } from '../plan-question-prompt'; import { controlProgressReporter } from '../progress'; -import { - destructiveConsentQuestion, - errorConsentOperationsMissing, - errorConsentTokenUnresolved, -} from './consent'; +import { ormStatementVerbs } from '../statement-verbs'; import { migrationResultBlocks, migrationResultNextActions } from './migration-blocks'; import { prepareMigrationRun } from './prepare'; @@ -48,6 +47,10 @@ function updatePresentations(inputs: { readonly database: string | undefined; readonly to: string | undefined; readonly dryRun: boolean; + /** Whether the run named its database with `--db`, which the suggested apply repeats as a placeholder. */ + readonly dbGiven: boolean; + readonly advanceRef: string | undefined; + readonly statements: readonly StatementText[]; }): Presentations { const { document, database, dryRun } = inputs; return { @@ -66,16 +69,57 @@ function updatePresentations(inputs: { ...migrationResultBlocks(document), ], json: () => document, - next: () => migrationResultNextActions(document, '{bin} db update'), + next: () => + migrationResultNextActions( + document, + [ + '{bin} db update', + ...(inputs.to === undefined ? [] : [`--to ${shellQuoted(inputs.to)}`]), + ...(inputs.dbGiven ? ['--db '] : []), + ...(inputs.advanceRef === undefined + ? [] + : [`--advance-ref ${shellQuoted(inputs.advanceRef)}`]), + ...inputs.statements.map(statementFlag), + ].join(' '), + ), }; } +/** + * The entries a dry run lists, each marked answered when a flag it was given answers its question: + * a `--delete` answers every loss of its subject, and an `--allow` one operation, in order. + */ +function subjectEntriesJson( + entries: readonly AskedSubject[], + verb: 'delete' | 'allow', + given: readonly StatementText[], +) { + const flagsLeft = new Map(); + for (const statement of given) { + if (statement.verb === verb) { + flagsLeft.set(statement.text, (flagsLeft.get(statement.text) ?? 0) + 1); + } + } + return entries.map((entry) => { + const left = flagsLeft.get(entry.text) ?? 0; + if (left > 0 && verb === 'allow') flagsLeft.set(entry.text, left - 1); + return { + operationIndex: entry.operationIndex, + subject: migrationSubjectJson(entry.subject), + text: entry.text, + answered: left > 0, + }; + }); +} + function updateDocument(inputs: { readonly value: DbUpdateSuccess; readonly targetId: string; readonly advancedRef: { readonly name: string; readonly hash: string } | null; readonly plannedAdvanceRef: { readonly name: string; readonly hash: string } | null; readonly startedAt: number; + /** The statements the run was given, which a dry run's questions are marked answered by. */ + readonly statements: readonly StatementText[]; }): MigrationCommandResult { const { value } = inputs; return { @@ -112,6 +156,12 @@ function updateDocument(inputs: { }), ...ifDefined('perSpace', value.perSpace), appliedStatements: value.appliedStatements, + ...(value.mode === 'plan' + ? { + dataLoss: subjectEntriesJson(value.dataLoss, 'delete', inputs.statements), + accessWidening: subjectEntriesJson(value.accessWidening, 'allow', inputs.statements), + } + : {}), ...ifDefined('warnings', value.warnings), advancedRef: inputs.advancedRef, plannedAdvanceRef: inputs.plannedAdvanceRef, @@ -120,6 +170,15 @@ function updateDocument(inputs: { }; } +/** The delete and allow values the engine still holds for the questions, read without taking them. */ +function unansweredConsents(statements: { + readonly values: () => readonly { readonly verb: string; readonly text: string }[]; +}): readonly StatementText[] { + return statements + .values() + .flatMap(({ verb, text }) => (verb === 'delete' || verb === 'allow' ? [{ verb, text }] : [])); +} + export function createDbUpdateCommand(createClient: CreateControlClient) { return defineOrmCommand({ help: { @@ -127,14 +186,16 @@ export function createDbUpdateCommand(createClient: CreateControlClient) { description: 'Compares the database to the emitted contract and applies the changes that\n' + 'close the gap, whether or not the database was bootstrapped with `db init`.\n' + - 'An operation that would destroy data is applied only with your consent: the\n' + - 'command asks you to type the database name. A run with nobody to ask — a CI\n' + - 'job, or `--no-interactive` — takes it from `--confirm ` instead.\n' + - 'Use --dry-run to see the operations without applying them.', + 'Before it applies an operation that would lose data, it asks what the\n' + + 'operation means: --rename keeps the data under a new name, --delete lets it\n' + + 'go. Before it widens who can read or write rows, it asks for --allow. Where\n' + + 'nobody can answer, it refuses and lists every question. Use --dry-run to see\n' + + 'the operations and the questions without applying anything.', examples: [ 'db update', 'db update --dry-run', - 'db update --no-interactive --confirm appdb', + 'db update --delete Legacy', + 'db update --rename Profile:User --allow User', 'db update --to production', ], }, @@ -150,16 +211,23 @@ export function createDbUpdateCommand(createClient: CreateControlClient) { brief: 'Advance the named ref to the post-command contract hash', placeholder: 'name', }), - rename: flag.repeated({ - brief: - 'Rename a model or field: Model, namespace.Model, Model.field or namespace.Model.field on each side; repeat for several, applied in order', - placeholder: 'old:new', - }), }, }, + statements: ormStatementVerbs, needs: { config: ormConfigSection }, handler: async (args, ctx) => { const startedAt = Date.now(); + const renames = ctx.statements + .take('rename') + .map(({ text }) => ({ verb: 'rename' as const, text })); + // A dry run asks nothing, so it takes the delete and allow values and the control API + // checks each against the subjects an apply would ask about. + const previewConsents = args.flags.dryRun + ? [ + ...ctx.statements.take('delete').map(({ text }) => ({ verb: 'delete' as const, text })), + ...ctx.statements.take('allow').map(({ text }) => ({ verb: 'allow' as const, text })), + ] + : []; let destination: ResolveContractRefToSnapshotSuccess | undefined; if (args.flags.to !== undefined) { const resolved = await resolveContractRefToSnapshot({ @@ -181,13 +249,14 @@ export function createDbUpdateCommand(createClient: CreateControlClient) { db: args.flags.db, commandName: 'db update', createClient, - retryCommand: retryCommandFor({ - commandName: args.flags.dryRun ? 'db update --dry-run' : 'db update', - to: args.flags.to, - advanceRef: args.flags.advanceRef, - renames: args.flags.rename, - canRunOffline: false, - }), + retryCommand: () => + retryCommandFor({ + commandName: args.flags.dryRun ? 'db update --dry-run' : 'db update', + to: args.flags.to, + advanceRef: args.flags.advanceRef, + statements: [...renames, ...previewConsents, ...unansweredConsents(ctx.statements)], + canRunOffline: false, + }), }); if (!prepared.ok) { return notOk(prepared.failure); @@ -220,56 +289,23 @@ export function createDbUpdateCommand(createClient: CreateControlClient) { try { await client.connect(dbConnection); - const update = (consent?: { readonly planHash: string }): Promise => - client.dbUpdate({ - contract: contractJson, - mode, - migrationsDir, - statements: renameStatements(args.flags.rename), - ...(consent === undefined ? {} : { consent }), - onProgress: controlProgressReporter(ctx.report), - }); - - // A successful run shows the planner's warnings in its blocks. A refused - // or failed one has no result to carry them, and an errored envelope - // renders no meta, so they are reported as events to reach both channels. - // The refusal's warnings matter most of all: they are what the user is - // told just before consenting. - const reported = new Set(); + // A refused or failed run has no result to carry the planner's warnings, + // and an errored envelope renders no meta, so they are reported as events + // to reach both channels. const reportPlannerWarnings = (warnings: readonly { readonly summary: string }[]): void => { for (const warning of warnings) { - if (!reported.has(warning.summary)) { - reported.add(warning.summary); - ctx.report({ kind: 'message', severity: 'warn', text: warning.summary }); - } + ctx.report({ kind: 'message', severity: 'warn', text: warning.summary }); } }; - let result = await update(); - // The destructive verdict is the planner's, so it arrives only after the - // connection is open. Consent is asked for on that same connection and - // the apply is re-run carrying the refused plan's hash — the control API - // refuses if the plan it is about to apply is no longer that plan. Only - // an apply can ask: a dry run has nothing to authorise. - if (mode === 'apply' && !result.ok && result.failure.code === 'DESTRUCTIVE_CHANGES') { - reportPlannerWarnings(result.failure.warnings ?? []); - const refusal = result.failure.destructiveChanges; - if (refusal === undefined || refusal.destructiveOperations.length === 0) { - return notOk(normalizeError(errorConsentOperationsMissing())); - } - const token = refusal.databaseName ?? ''; - if (token.trim().length === 0) { - return notOk(normalizeError(errorConsentTokenUnresolved(ctx.config.target.targetId))); - } - const granted = await ctx.prompt.consent( - destructiveConsentQuestion(refusal.destructiveOperations, token), - { token }, - ); - if (!granted) { - return notOk(normalizeError(mapDbUpdateFailure(result.failure))); - } - result = await update({ planHash: refusal.planHash }); - } + const result = await client.dbUpdate({ + contract: contractJson, + mode, + migrationsDir, + statements: [...renames, ...previewConsents], + answerQuestions: promptPlanQuestions(ctx.prompt), + onProgress: controlProgressReporter(ctx.report), + }); if (!result.ok) { reportPlannerWarnings(result.failure.warnings ?? []); return notOk(normalizeError(mapDbUpdateFailure(result.failure))); @@ -300,9 +336,10 @@ export function createDbUpdateCommand(createClient: CreateControlClient) { advancedRef: advanced.value.advancedRef, plannedAdvanceRef: advanced.value.plannedAdvanceRef, startedAt, + statements: previewConsents, }); } catch (error) { - // A refused, mistyped or cancelled consent is the engine's own error, and + // A refused, mistyped or cancelled answer is the engine's own error, and // the engine settles it: cancellation exits 3, everything else 2. Catching // it here would restate it as this command's failure and lose that. if (error instanceof EngineStructuredError) { @@ -347,6 +384,9 @@ export function createDbUpdateCommand(createClient: CreateControlClient) { database: prepared.value.database, to: args.flags.to, dryRun: args.flags.dryRun, + dbGiven: args.flags.db !== undefined, + advanceRef: args.flags.advanceRef, + statements: [...renames, ...previewConsents], }), ), ); diff --git a/packages/1-framework/3-tooling/cli/src/orm/define-command.ts b/packages/1-framework/3-tooling/cli/src/orm/define-command.ts index 1d996d0d5819..1731289790ad 100644 --- a/packages/1-framework/3-tooling/cli/src/orm/define-command.ts +++ b/packages/1-framework/3-tooling/cli/src/orm/define-command.ts @@ -8,6 +8,7 @@ import type { NeedsSpec, PositionalSpec, SpawnDeclarations, + StatementSpec, } from '@prisma/cli-engine'; import { defineCommand } from '@prisma/cli-engine'; import { notOk } from '@prisma/cli-engine/protocol'; @@ -44,6 +45,8 @@ export function defineOrmCommand< readonly needs?: NeedsSpec; readonly exitCodes?: Readonly>; readonly installsPackages?: TInstallsPackages; + /** The statement verbs the command may ask with; see the engine README § Statement prompts. */ + readonly statements?: Readonly>; readonly handler: Handler; } & SpawnDeclarations, ): CommandDefinition { diff --git a/packages/1-framework/3-tooling/cli/src/orm/migration/plan.ts b/packages/1-framework/3-tooling/cli/src/orm/migration/plan.ts index c5459f841ec1..abd2bc4afd54 100644 --- a/packages/1-framework/3-tooling/cli/src/orm/migration/plan.ts +++ b/packages/1-framework/3-tooling/cli/src/orm/migration/plan.ts @@ -1,5 +1,4 @@ import { ormConfigSection } from '@internal/config-loader'; -import { blindCast } from '@internal/utils/casts'; import { ifDefined } from '@internal/utils/defined'; import type { Block, Presentations, Text, TreeNode } from '@prisma/cli-engine'; import { flag } from '@prisma/cli-engine'; @@ -8,25 +7,20 @@ import { notOk, ok } from '@prisma/cli-engine/protocol'; import { join } from 'pathe'; import { createControlClient } from '../../control-api/client'; import type { ContractSpaceSeedPhaseRecord } from '../../control-api/operations/contract-space-seed-phase'; -import type { - DestructiveBaselineVerdict, - MigrationPlanResult, -} from '../../control-api/operations/migration-plan'; +import type { MigrationPlanResult } from '../../control-api/operations/migration-plan'; import { executeMigrationPlanCommand } from '../../control-api/operations/migration-plan'; -import { renameStatements } from '../../control-api/statements/statement-text'; -import type { CreateControlClient, DestructivePlanOperation } from '../../control-api/types'; -import { ERROR_CODE_DESTRUCTIVE_CHANGES } from '../../utils/cli-errors'; +import type { CreateControlClient } from '../../control-api/types'; import { RECORDED_CONTRACT_REF_FORMS, RECORDED_OR_EMPTY_CONTRACT_REF_FORMS, } from '../../utils/contract-ref-forms'; import { previewBlockHeader } from '../../utils/formatters/migrations'; import { runCommandAction } from '../../utils/next-actions'; -import { destructiveOperationList, errorConsentOperationsMissing } from '../db/consent'; import { defineOrmCommand } from '../define-command'; -import { consentToken } from '../init-inputs'; import { normalizeError } from '../normalize-error'; +import { promptPlanQuestions } from '../plan-question-prompt'; import { appliedStatementBlocks } from '../statement-blocks'; +import { ormStatementVerbs } from '../statement-verbs'; import { appMigrationsDirFor, baseDirFor, @@ -262,14 +256,6 @@ function planPresentations(inputs: { }; } -/** The question the user answers before a destructive baseline is written. */ -function destructiveBaselineQuestion(operations: readonly DestructivePlanOperation[]): string { - return [ - `Write a baseline migration containing ${operations.length} destructive operation(s)? Applying it would remove data that cannot be recovered:`, - destructiveOperationList(operations), - ].join('\n'); -} - export function createMigrationPlanCommand(createClient: CreateControlClient) { return defineOrmCommand({ help: { @@ -278,15 +264,16 @@ export function createMigrationPlanCommand(createClient: CreateControlClient) { 'Compares the emitted contract against the latest on-disk migration state\n' + 'and produces a new migration package with the required operations.\n' + 'On an empty migrations directory a baseline package is derived from the\n' + - '`db` ref first; a baseline containing destructive operations is only\n' + - 'written with your consent: the command asks you to type the project\n' + - 'directory name, or takes `--confirm ` where there is nobody\n' + - 'to ask. Offline — does not consult the database.', + '`db` ref first. A plan that would lose data is written only once you say\n' + + 'what each such operation means: --rename to keep the data under a new\n' + + 'name, or --delete to let it go. Where nobody can answer, the command\n' + + 'refuses and lists every question. Offline — does not consult the database.', examples: [ 'migration plan', // biome-ignore lint/plugin/no-family-vocabulary: a migration slug a user would plausibly type, not a schema concept 'migration plan --name add-users-table', 'migration plan --to ^ --name rollback', + 'migration plan --rename Profile:User --delete Legacy', 'migration plan --json', ], }, @@ -301,26 +288,15 @@ export function createMigrationPlanCommand(createClient: CreateControlClient) { brief: `Destination contract reference (${RECORDED_CONTRACT_REF_FORMS}); defaults to the emitted contract`, placeholder: 'contract', }), - rename: flag.repeated({ - brief: - 'Rename a model or field: Model, namespace.Model, Model.field or namespace.Model.field on each side; repeat for several, applied in order', - placeholder: 'old:new', - }), }, }, + statements: { rename: ormStatementVerbs.rename, delete: ormStatementVerbs.delete }, needs: { config: ormConfigSection }, handler: async (args, ctx) => { - // Dirs the seed phase materialised across this invocation's run(s): the - // consented re-run finds them already on disk, so its own seed records - // come back `unchanged` and the accumulated list is threaded back in. - const seededDirs: { spaceId: string; dirName: string }[] = []; const seeded = (record: ContractSpaceSeedPhaseRecord): void => { if (record.action !== 'updated') { return; } - for (const dirName of record.newMigrationDirs) { - seededDirs.push({ spaceId: record.spaceId, dirName }); - } const step = `Seed contract space ${record.spaceId}`; ctx.report({ kind: 'step-started', step, id: record.spaceId }); ctx.report({ @@ -331,66 +307,29 @@ export function createMigrationPlanCommand(createClient: CreateControlClient) { data: { newHash: record.newHash, newMigrationDirs: record.newMigrationDirs }, }); }; - - const plan = (consent?: { readonly planHash: string }) => - executeMigrationPlanCommand( - { - config: ctx.config, - cwd: ctx.cwd, - projectDir: baseDirFor(ctx.config), - ...ifDefined('name', args.flags.name), - ...ifDefined('from', args.flags.from), - ...ifDefined('to', args.flags.to), - statements: renameStatements(args.flags.rename), - ...ifDefined('consent', consent), - ...ifDefined( - 'carryEmittedExtensionDirs', - consent !== undefined && seededDirs.length > 0 ? [...seededDirs] : undefined, - ), - client: createClient({ - family: ctx.config.family, - target: ctx.config.target, - adapter: ctx.config.adapter, - ...ifDefined('driver', ctx.config.driver), - extensions: ctx.config.extensions ?? [], - }), - }, - Date.now(), - { onSeeded: seeded }, - ); - - let planned = await plan(); - // The destructive verdict is the planner's own: an auto-baseline whose - // operations would remove data is refused before anything is written. - // Consent is asked for here and the plan re-run carrying the refused - // plan's hash — the operation layer refuses if the recomputed baseline is - // no longer that plan. Mirrors the `db update` consent flow. - if (!planned.ok && planned.failure.code === ERROR_CODE_DESTRUCTIVE_CHANGES) { - const verdict = blindCast< - Partial, - 'the meta envelope is produced by refuseUnconsentedDestructiveBaseline; presence is checked below' - >(planned.failure.meta ?? {}); - if ( - verdict.destructiveOperations === undefined || - verdict.destructiveOperations.length === 0 || - verdict.planHash === undefined - ) { - return notOk( - normalizeError( - errorConsentOperationsMissing({ previewCommand: '{bin} migration plan' }), - ), - ); - } - const token = consentToken(ctx.cwd); - const granted = await ctx.prompt.consent( - destructiveBaselineQuestion(verdict.destructiveOperations), - { token }, - ); - if (!granted) { - return notOk(normalizeError(planned.failure)); - } - planned = await plan({ planHash: verdict.planHash }); - } + const planned = await executeMigrationPlanCommand( + { + config: ctx.config, + cwd: ctx.cwd, + projectDir: baseDirFor(ctx.config), + ...ifDefined('name', args.flags.name), + ...ifDefined('from', args.flags.from), + ...ifDefined('to', args.flags.to), + statements: ctx.statements + .take('rename') + .map(({ text }) => ({ verb: 'rename' as const, text })), + answerQuestions: promptPlanQuestions(ctx.prompt), + client: createClient({ + family: ctx.config.family, + target: ctx.config.target, + adapter: ctx.config.adapter, + ...ifDefined('driver', ctx.config.driver), + extensions: ctx.config.extensions ?? [], + }), + }, + Date.now(), + { onSeeded: seeded }, + ); if (!planned.ok) { return notOk(normalizeError(planned.failure)); } diff --git a/packages/1-framework/3-tooling/cli/src/orm/plan-question-prompt.ts b/packages/1-framework/3-tooling/cli/src/orm/plan-question-prompt.ts new file mode 100644 index 000000000000..1522e28f5755 --- /dev/null +++ b/packages/1-framework/3-tooling/cli/src/orm/plan-question-prompt.ts @@ -0,0 +1,23 @@ +import type { PromptSurface } from '@prisma/cli-engine'; +import type { AnswerPlanQuestions, PlanAnswer } from '../control-api/statements/plan-questions'; + +/** + * Asks a plan's questions through the engine's statement prompt, all in one batch that refuses a + * statement nothing consumed before the command acts. Flags answer first; where nobody can answer, + * the engine refuses and lists every question. + */ +export function promptPlanQuestions(prompt: PromptSurface): AnswerPlanQuestions { + return async (questions): Promise => { + const answers = await prompt.statements( + questions.map((question) => ({ + question: question.question, + subject: question.subject, + verbs: question.verbs, + forms: question.forms, + validate: question.validate, + })), + { last: true }, + ); + return answers.map(({ verb, text }) => ({ verb, text })); + }; +} diff --git a/packages/1-framework/3-tooling/cli/src/orm/statement-verbs.ts b/packages/1-framework/3-tooling/cli/src/orm/statement-verbs.ts new file mode 100644 index 000000000000..5b51228c38f0 --- /dev/null +++ b/packages/1-framework/3-tooling/cli/src/orm/statement-verbs.ts @@ -0,0 +1,20 @@ +/** + * The statement verbs the ORM commands declare. Each command declares the verbs its questions + * use: `migration plan` takes `rename` and `delete`, and `db update` also `allow`. + */ +export const ormStatementVerbs = { + rename: { + arity: 1, + brief: + 'Rename a model or field instead of dropping it: old:new, each side Model, namespace.Model, Model.field or namespace.Model.field; repeat for several, applied in order', + }, + delete: { + arity: 1, + brief: + 'Let the command lose the data of a model, field or storage name the refusal lists: --delete Model', + }, + allow: { + arity: 1, + brief: 'Let the command widen who can read or write the rows of a model: --allow Model', + }, +} as const; diff --git a/packages/1-framework/3-tooling/cli/src/utils/cli-errors.ts b/packages/1-framework/3-tooling/cli/src/utils/cli-errors.ts index 0e639b997daa..76a0603b3b50 100644 --- a/packages/1-framework/3-tooling/cli/src/utils/cli-errors.ts +++ b/packages/1-framework/3-tooling/cli/src/utils/cli-errors.ts @@ -33,9 +33,6 @@ import type { MigrateFailure } from '../control-api/types'; import { chooseAction, runCommandAction } from './next-actions'; export { - ERROR_CODE_DESTRUCTIVE_CHANGES, - errorConsentPlanMismatch, - errorDestructiveChanges, errorHashMismatch, errorMarkerMissing, errorMarkerRequired, @@ -612,6 +609,59 @@ export function errorStatementUnresolved( ); } +/** + * A rename typed in answer to a data-loss question was planned, and the plan still loses the + * subject's data: the rename does not account for the operation the question was about. + */ +export function errorStatementDidNotResolveLoss( + statement: WrittenStatement, + subject: string, +): ActionableCliError { + const fix = `Answer the question about "${subject}" with --delete ${subject} if its data may be lost, or give a rename whose old name stores that data.`; + return new ActionableCliError( + 'MIGRATION.STATEMENT_DID_NOT_RESOLVE_LOSS', + `"--${statement.verb} ${statement.text}" does not keep the data of "${subject}"`, + { + why: `The plan was made again with the rename, and it still loses the data of "${subject}".`, + fix, + nextActions: [chooseAction(fix)], + meta: { statement: statement.text, verb: statement.verb, subject }, + }, + ); +} + +/** + * `delete` or `allow` statements given to a programmatic call match no question the plan asks, + * so they consent to nothing. + */ +export function errorStatementAnswersNoQuestion( + statements: readonly WrittenStatement[], + subjects: readonly string[], +): ActionableCliError { + const written = statements.map(({ verb, text }) => `"--${verb} ${text}"`); + const listed = + written.length === 1 + ? `Statement ${written[0]} answers` + : `Statements ${written.slice(0, -1).join(', ')} and ${written.at(-1)} answer`; + const fix = + subjects.length === 0 + ? 'Leave the statements out: the plan asks no question.' + : `Leave them out, or name a subject the plan asks about: ${subjects.join(', ')}.`; + return new ActionableCliError( + 'MIGRATION.STATEMENT_ANSWERS_NO_QUESTION', + `${listed} no question of the plan`, + { + why: 'A delete or allow statement consents to an operation the plan asks about, and these name no subject the plan would lose data from or widen access to.', + fix, + nextActions: [chooseAction(fix)], + meta: { + statements: statements.map(({ verb, text }) => ({ verb, text })), + subjects, + }, + }, + ); +} + /** * Without statements, `db update` drops a renamed table or column and creates it again under the * new name, so any advice to run without them says so. @@ -620,7 +670,7 @@ const WITHOUT_STATEMENTS_DROPS = 'A plan made without statements drops the storage of each renamed model or field with its data and creates it again under the new name, and asks for consent before it does.'; /** How to store the snapshot of the contract the database is at, so statements can resolve. */ -const STORE_ORIGIN_SNAPSHOT_STEPS = [ +export const STORE_ORIGIN_SNAPSHOT_STEPS = [ '1. Put the contract source back to the version the database is at, and run `{bin} contract emit`.', '2. Run `{bin} db update --advance-ref --dry-run`, with the same `--db` as this command if it has one, and check that it plans no operations. Then run it again without `--dry-run`: the database matches that contract, so this changes nothing in it, and it stores the contract snapshot. If the dry run plans operations, the database has drifted from that contract; settle that before you go on.', '3. Put the new contract source back, run `{bin} contract emit`, and run this command again with `--advance-ref `.', diff --git a/packages/1-framework/3-tooling/cli/src/utils/db-init-failure.ts b/packages/1-framework/3-tooling/cli/src/utils/db-init-failure.ts index 51a0468a26d2..4824c96c58e4 100644 --- a/packages/1-framework/3-tooling/cli/src/utils/db-init-failure.ts +++ b/packages/1-framework/3-tooling/cli/src/utils/db-init-failure.ts @@ -14,7 +14,7 @@ import { import { runCommandAction } from './next-actions'; const DB_INIT_ADDITIVE_ONLY_FIX = - '`db init` applies only additive changes. Run `{bin} db update`, which also applies widening and destructive ones after you confirm them by typing the database name, or pass `--no-interactive --confirm ` where there is nobody to ask.'; + '`db init` applies only additive changes. Run `{bin} db update`, which also applies widening and destructive ones once you answer what each operation that would lose data means, with `--delete` or `--rename`.'; const DB_INIT_NEEDS_MIGRATION_FIX = '`db init` applies only additive changes, and `db update` does not apply data operations. Plan a migration with `{bin} migration plan`, which can include them, and apply it with `{bin} db migrate`.'; @@ -42,7 +42,7 @@ function adviceForRefusedClasses(refused: ReadonlySet): fix: DB_INIT_ADDITIVE_ONLY_FIX, nextAction: runCommandAction( refused.has('destructive') - ? 'Apply the change with db update, which lists the destructive operations and asks you to confirm them' + ? 'Apply the change with db update, which asks what each operation that would lose data means' : 'Apply the change with db update', '{bin} db update', ), diff --git a/packages/1-framework/3-tooling/cli/src/utils/db-update-failure.ts b/packages/1-framework/3-tooling/cli/src/utils/db-update-failure.ts index a34d26975a10..fd31e338b8f0 100644 --- a/packages/1-framework/3-tooling/cli/src/utils/db-update-failure.ts +++ b/packages/1-framework/3-tooling/cli/src/utils/db-update-failure.ts @@ -2,20 +2,11 @@ import { ifDefined } from '@internal/utils/defined'; import { assertNever } from '@internal/utils/internal-error'; import type { DbUpdateFailure } from '../control-api/types'; import type { CliStructuredError } from './cli-errors'; -import { - errorConsentPlanMismatch, - errorDestructiveChanges, - errorMigrationPlanningFailed, - errorRunnerFailed, -} from './cli-errors'; +import { errorMigrationPlanningFailed, errorRunnerFailed } from './cli-errors'; /** * A `db update` failure as the CLI's structured error. * - * `DESTRUCTIVE_CHANGES` is handled by the command, which asks for consent and - * calls again; it is mapped here for the case the control API returns it from a - * call that already carried consent, which would otherwise degrade silently. - * * The `assertNever` is deliberate: a control-API failure code this does not * handle must stop the command rather than degrade into a generic message. */ @@ -47,30 +38,6 @@ export function mapDbUpdateFailure(failure: DbUpdateFailure): CliStructuredError }); } - if (failure.code === 'DESTRUCTIVE_CHANGES') { - return errorDestructiveChanges(failure.summary, { - ...ifDefined('why', failure.why), - fix: 'Re-run `{bin} db update` and type the database name when asked, or pass `--no-interactive --confirm ` where there is nobody to ask. Use `--dry-run` to preview the operations first.', - ...(failure.destructiveChanges - ? { - meta: { - destructiveOperations: failure.destructiveChanges.destructiveOperations, - databaseName: failure.destructiveChanges.databaseName, - planHash: failure.destructiveChanges.planHash, - }, - } - : {}), - }); - } - - if (failure.code === 'CONSENT_PLAN_MISMATCH') { - return errorConsentPlanMismatch({ - consentedPlanHash: failure.consentPlanMismatch?.consentedPlanHash ?? '', - planHash: failure.consentPlanMismatch?.planHash ?? '', - ...ifDefined('why', failure.why), - }); - } - const exhaustive: never = failure.code; return assertNever(exhaustive, `Unhandled DbUpdateFailure code: ${String(exhaustive)}`); } diff --git a/packages/1-framework/3-tooling/cli/src/utils/formatters/migrations.ts b/packages/1-framework/3-tooling/cli/src/utils/formatters/migrations.ts index f6868580bca6..66ff1226cd97 100644 --- a/packages/1-framework/3-tooling/cli/src/utils/formatters/migrations.ts +++ b/packages/1-framework/3-tooling/cli/src/utils/formatters/migrations.ts @@ -1,5 +1,6 @@ import type { MigrationPlannerConflict, + MigrationSubjectJson, OperationPreview, } from '@internal/framework-components/control'; import type { AppliedStatementReport } from '../../control-api/statements/report-applied-statements'; @@ -45,6 +46,16 @@ export function previewBlockHeader(preview: OperationPreview): string { /** * Shared CLI output type for migration commands (db init, db update). */ +/** An operation of a plan, by its position in `plan.operations`, and its subject. */ +export interface PlannedSubjectJson { + readonly operationIndex: number; + readonly subject: MigrationSubjectJson; + /** The subject as the apply's question writes it, and as `--delete` or `--allow` takes it. */ + readonly text: string; + /** Whether a `--delete` or `--allow` the dry run was given answers this question. */ + readonly answered: boolean; +} + export interface MigrationCommandResult { readonly ok: true; readonly mode: 'plan' | 'apply'; @@ -82,8 +93,12 @@ export interface MigrationCommandResult { * into a single ambiguous list. See {@link PerSpaceExecutionEntry}. */ readonly perSpace?: ReadonlyArray; - /** `db update` only: the `--rename` statements the plan applied, in order; empty when none were given. */ + /** `db update` only: the statements the plan applied, in order; empty when none were given. */ readonly appliedStatements?: readonly AppliedStatementReport[]; + /** `db update --dry-run` only: the operations an apply would ask about because they lose data. */ + readonly dataLoss?: readonly PlannedSubjectJson[]; + /** `db update --dry-run` only: the operations an apply would ask about because they widen access. */ + readonly accessWidening?: readonly PlannedSubjectJson[]; readonly advancedRef?: { readonly name: string; readonly hash: string } | null; readonly plannedAdvanceRef?: { readonly name: string; readonly hash: string } | null; readonly summary: string; diff --git a/packages/1-framework/3-tooling/cli/test/config-types.test.ts b/packages/1-framework/3-tooling/cli/test/config-types.test.ts index bc9c620498ef..84a0d7896f34 100644 --- a/packages/1-framework/3-tooling/cli/test/config-types.test.ts +++ b/packages/1-framework/3-tooling/cli/test/config-types.test.ts @@ -34,6 +34,7 @@ describe('defineConfig', () => { create: () => ({ familyId: 'sql', deserializeContract: (contract: unknown) => contract as Contract, + storageNameOf: (operation) => operation.id, verify: async () => ({ ok: true, summary: 'test', diff --git a/packages/1-framework/3-tooling/cli/test/control-api/client.test.ts b/packages/1-framework/3-tooling/cli/test/control-api/client.test.ts index 6b3efa7019cf..b5c23577c308 100644 --- a/packages/1-framework/3-tooling/cli/test/control-api/client.test.ts +++ b/packages/1-framework/3-tooling/cli/test/control-api/client.test.ts @@ -719,6 +719,8 @@ describe('ControlClient progress emission', () => { plan: () => ({ kind: 'success', appliedStatements: [], + dataLoss: [], + accessWidening: [], plan: { targetId: 'postgres', destination: { storageHash: 'dest' }, @@ -785,6 +787,7 @@ describe('ControlClient progress emission', () => { mode: 'apply', connection: 'postgres://test', migrationsDir: '/tmp/__test-client-migrations', + answerQuestions: async () => [], acceptDataLoss: true, onProgress: (event) => events.push(event), }); @@ -847,6 +850,7 @@ describe('ControlClient progress emission', () => { mode: 'plan', connection: 'postgres://test', migrationsDir: '/tmp/__test-client-migrations', + answerQuestions: async () => [], }); expect(result.ok).toBe(true); @@ -873,6 +877,7 @@ describe('ControlClient progress emission', () => { mode: 'plan', connection: 'postgres://test', migrationsDir: '/tmp/__test-client-migrations', + answerQuestions: async () => [], }); await client.close(); diff --git a/packages/1-framework/3-tooling/cli/test/control-api/db-update-statement-positions.test.ts b/packages/1-framework/3-tooling/cli/test/control-api/db-update-statement-positions.test.ts index 40d1dfd705fd..fdc9a1abac13 100644 --- a/packages/1-framework/3-tooling/cli/test/control-api/db-update-statement-positions.test.ts +++ b/packages/1-framework/3-tooling/cli/test/control-api/db-update-statement-positions.test.ts @@ -59,6 +59,7 @@ const AUDIT_CONTRACT: Contract = createSqlContract({ const AUDIT_HEAD = AUDIT_CONTRACT.storage.storageHash; const AUDIT_OPS: readonly MigrationPlanOperation[] = [ { id: 'table.audit_log', label: 'Create table audit_log', operationClass: 'additive' }, + { id: 'dropTable.audit_old', label: 'Drop table audit_old', operationClass: 'destructive' }, { id: 'index.audit_log.at', label: 'Create index on audit_log', operationClass: 'additive' }, ]; const AUDIT_METADATA: Omit = { @@ -74,6 +75,8 @@ const RENAME_OP: MigrationPlanOperation = { operationClass: 'widening', }; +const LOST = { kind: 'storage', name: 'audit_log' } as const; + function markerAt(storageHash: string): ContractMarkerRecord { return { storageHash, @@ -98,6 +101,7 @@ const family = { introspect: async () => ({ tables: {} }), deserializeContract: (json: unknown) => json as Contract, toOperationPreview: () => ({ statements: [] }), + storageNameOf: (operation: MigrationPlanOperation) => `stored ${operation.id}`, } as unknown as ControlFamilyInstance<'sql', unknown>; const migrations = { @@ -111,6 +115,8 @@ const migrations = { statement, operationIndexes: [0], })), + dataLoss: [{ operationIndex: 0, subject: LOST }], + accessWidening: [{ operationIndex: 0, subject: LOST, widens: true }], plan: { targetId: 'postgres', origin: options.origin, @@ -193,6 +199,7 @@ describe('executeDbUpdate statement positions', () => { migrationsDir: await migrationsDirWithPendingExtension(), targetId: 'postgres', extensions: [auditExtension], + answerQuestions: async () => [], statements: renameStatements(['Profile:User']), }); @@ -207,5 +214,28 @@ describe('executeDbUpdate statement positions', () => { applied.operationIndexes.map((index) => operations[index]?.id), ), ).toEqual([[RENAME_OP.id]]); + expect( + [result.value.dataLoss, result.value.accessWidening].map((entries) => + entries.map(({ operationIndex, subject }) => ({ + operation: operations[operationIndex]?.id, + subject, + })), + ), + ).toEqual([ + [ + { + operation: 'dropTable.audit_old', + subject: { kind: 'storage', name: 'stored dropTable.audit_old' }, + }, + { operation: RENAME_OP.id, subject: LOST }, + ], + [{ operation: RENAME_OP.id, subject: LOST }], + ]); + const listed = new Set(result.value.dataLoss.map(({ operationIndex }) => operationIndex)); + expect( + operations.flatMap((operation, index) => + operation.operationClass === 'destructive' && !listed.has(index) ? [operation.id] : [], + ), + ).toEqual([]); }); }); diff --git a/packages/1-framework/3-tooling/cli/test/control-api/db-update-statements.test.ts b/packages/1-framework/3-tooling/cli/test/control-api/db-update-statements.test.ts index 2561b03a134a..47aba69c43da 100644 --- a/packages/1-framework/3-tooling/cli/test/control-api/db-update-statements.test.ts +++ b/packages/1-framework/3-tooling/cli/test/control-api/db-update-statements.test.ts @@ -1,10 +1,11 @@ import { rmSync, writeFileSync } from 'node:fs'; -import type { Contract, ContractMarkerRecord } from '@internal/contract/types'; +import { asNamespaceId, type Contract, type ContractMarkerRecord } from '@internal/contract/types'; import type { ControlAdapterInstance, ControlDriverInstance, ControlFamilyInstance, MigrationPlannerResult, + MigrationPlannerSuccessResult, ResolvedMigrationStatement, TargetMigrationsCapability, } from '@internal/framework-components/control'; @@ -88,7 +89,10 @@ interface PlannerCall { readonly statements: readonly ResolvedMigrationStatement[]; } -function recordingMigrations(operationClass: 'additive' | 'destructive' = 'additive') { +function recordingMigrations( + operationClass: 'additive' | 'destructive' = 'additive', + dataLoss: MigrationPlannerSuccessResult['dataLoss'] = [], +) { const calls: PlannerCall[] = []; const execute = vi.fn().mockResolvedValue( ok({ @@ -106,6 +110,8 @@ function recordingMigrations(operationClass: 'additive' | 'destructive' = 'addit const renamed = options.statements.length > 0; return { kind: 'success', + dataLoss, + accessWidening: [], appliedStatements: options.statements.map((statement) => ({ statement, operationIndexes: [0], @@ -175,6 +181,7 @@ function update(options: { migrations: options.migrations, frameworkComponents: [], migrationsDir: options.migrationsDir, + answerQuestions: async () => [], targetId: 'postgres', statements: renameStatements(options.renames), ...(options.verifySnapshotContent === undefined @@ -370,6 +377,35 @@ describe('executeDbUpdate with statements', () => { ).rejects.toMatchObject({ code: 'MIGRATION.STATEMENT_UNRESOLVED' }); }); + it('names a dry run subject as the question would, through the renamed model', async () => { + const { migrations } = recordingMigrations('additive', [ + { + operationIndex: 0, + subject: { + kind: 'field', + namespaceId: asNamespaceId('app'), + model: 'Profile', + field: 'nickname', + }, + }, + ]); + + const result = await update({ + migrationsDir: await migrationsDirWithSnapshot(origin), + marker: markerAt(ORIGIN_HASH), + renames: ['Profile:User'], + migrations, + }); + + expect(result.ok && result.value.dataLoss).toEqual([ + { + operationIndex: 0, + subject: { kind: 'field', namespaceId: 'app', model: 'Profile', field: 'nickname' }, + text: 'User.nickname', + }, + ]); + }); + it('reads no snapshot without statements, and plans from no origin contract', async () => { const withSnapshot = recordingMigrations(); const result = await update({ diff --git a/packages/1-framework/3-tooling/cli/test/control-api/db-update.test.ts b/packages/1-framework/3-tooling/cli/test/control-api/db-update.test.ts index 4fa65427d0e5..b82f805b80e7 100644 --- a/packages/1-framework/3-tooling/cli/test/control-api/db-update.test.ts +++ b/packages/1-framework/3-tooling/cli/test/control-api/db-update.test.ts @@ -7,13 +7,21 @@ import type { MigrationRunnerResult, TargetMigrationsCapability, } from '@internal/framework-components/control'; +import { InternalError } from '@internal/utils/internal-error'; import { notOk, ok } from '@internal/utils/result'; import { describe, expect, it, vi } from 'vitest'; import { executeDbUpdate } from '../../src/control-api/operations/db-update'; +import { + ORIGIN_SNAPSHOT_RECOVERY, + type PlanQuestion, +} from '../../src/control-api/statements/plan-questions'; import type { ControlProgressEvent } from '../../src/control-api/types'; const FAKE_MIGRATIONS_DIR = '/tmp/__test-db-update-migrations'; +const noQuestions = async (questions: readonly PlanQuestion[]) => + questions.map((question) => ({ verb: 'delete' as const, text: question.subject })); + function markerRecord(fields: { readonly storageHash: string; readonly profileHash?: string; @@ -61,6 +69,8 @@ function createMockMigrations(overrides?: { }) { const planResult: MigrationPlannerResult = overrides?.planResult ?? { kind: 'success', + dataLoss: [], + accessWidening: [], appliedStatements: [], plan: { targetId: 'postgres', @@ -128,6 +138,7 @@ describe('executeDbUpdate', () => { migrations: createMockMigrations(), frameworkComponents: [], migrationsDir: FAKE_MIGRATIONS_DIR, + answerQuestions: noQuestions, targetId: 'postgres', }); @@ -160,6 +171,7 @@ describe('executeDbUpdate', () => { }), frameworkComponents: [], migrationsDir: FAKE_MIGRATIONS_DIR, + answerQuestions: noQuestions, targetId: 'postgres', }); @@ -176,6 +188,8 @@ describe('executeDbUpdate', () => { const migrations = createMockMigrations({ planResult: { kind: 'success', + dataLoss: [], + accessWidening: [], appliedStatements: [], plan: { targetId: 'postgres', @@ -215,6 +229,7 @@ describe('executeDbUpdate', () => { migrations, frameworkComponents: [], migrationsDir: FAKE_MIGRATIONS_DIR, + answerQuestions: noQuestions, targetId: 'postgres', }); @@ -251,6 +266,7 @@ describe('executeDbUpdate', () => { }), frameworkComponents: [], migrationsDir: FAKE_MIGRATIONS_DIR, + answerQuestions: noQuestions, targetId: 'postgres', }); @@ -291,6 +307,7 @@ describe('executeDbUpdate', () => { }), frameworkComponents: [], migrationsDir: FAKE_MIGRATIONS_DIR, + answerQuestions: noQuestions, targetId: 'postgres', }); @@ -328,6 +345,8 @@ describe('executeDbUpdate', () => { migrations: createMockMigrations({ planResult: { kind: 'success', + dataLoss: [], + accessWidening: [], appliedStatements: [], plan: { targetId: 'postgres', @@ -346,6 +365,7 @@ describe('executeDbUpdate', () => { }), frameworkComponents: [], migrationsDir: FAKE_MIGRATIONS_DIR, + answerQuestions: noQuestions, targetId: 'postgres', }); @@ -367,6 +387,8 @@ describe('executeDbUpdate', () => { const migrations = createMockMigrations({ planResult: { kind: 'success', + dataLoss: [], + accessWidening: [], appliedStatements: [], plan: { targetId: 'postgres', @@ -400,6 +422,7 @@ describe('executeDbUpdate', () => { migrations, frameworkComponents: [], migrationsDir: FAKE_MIGRATIONS_DIR, + answerQuestions: noQuestions, targetId: 'postgres', }); @@ -415,6 +438,8 @@ describe('executeDbUpdate', () => { it('allows additive, widening, and destructive operation classes', async () => { const planFn = vi.fn().mockReturnValue({ kind: 'success', + dataLoss: [], + accessWidening: [], appliedStatements: [], plan: { targetId: 'postgres', @@ -451,6 +476,7 @@ describe('executeDbUpdate', () => { migrations, frameworkComponents: [], migrationsDir: FAKE_MIGRATIONS_DIR, + answerQuestions: noQuestions, targetId: 'postgres', }); @@ -463,11 +489,17 @@ describe('executeDbUpdate', () => { ); }); - describe('destructive changes gate', () => { - function createDestructiveMigrations() { + describe('questions before an apply', () => { + const NICKNAME = { kind: 'storage', name: 'user.nickname' } as const; + const USER = { kind: 'storage', name: 'user' } as const; + + function createDestructiveMigrations(executeSpy?: ReturnType) { return createMockMigrations({ + ...(executeSpy === undefined ? {} : { executeSpy }), planResult: { kind: 'success', + dataLoss: [{ operationIndex: 0, subject: NICKNAME }], + accessWidening: [{ operationIndex: 2, subject: USER, widens: true }], appliedStatements: [], plan: { targetId: 'postgres', @@ -483,6 +515,11 @@ describe('executeDbUpdate', () => { label: 'Add column bio to user', operationClass: 'additive', }, + { + id: 'rls.user.disable', + label: 'Disable row-level security on user', + operationClass: 'widening', + }, ], renderTypeScript: () => { throw new Error('not used in db update tests'); @@ -491,143 +528,243 @@ describe('executeDbUpdate', () => { }, runnerResult: ok({ perSpaceResults: [ - { space: 'app', value: { operationsPlanned: 2, operationsExecuted: 2 } }, + { space: 'app', value: { operationsPlanned: 3, operationsExecuted: 3 } }, ], }), }); } - it('returns DESTRUCTIVE_CHANGES in apply mode without acceptDataLoss', async () => { - const result = await executeDbUpdate({ + function applyInputs(overrides: Partial[0]> = {}) { + return { driver: createMockDriver(), adapter: STUB_ADAPTER, familyInstance: createMockFamilyInstance({ readAllMarkers: async () => new Map([['app', markerRecord({ storageHash: 'origin' })]]), }), contract: dummyContract, - mode: 'apply', + mode: 'apply' as const, migrations: createDestructiveMigrations(), frameworkComponents: [], migrationsDir: FAKE_MIGRATIONS_DIR, - targetId: 'postgres', - }); + targetId: 'postgres' as const, + answerQuestions: noQuestions, + ...overrides, + }; + } - expect(result.ok).toBe(false); - if (!result.ok) { - expect(result.failure.code).toBe('DESTRUCTIVE_CHANGES'); - expect(result.failure.summary).toContain('destructive'); - expect(result.failure.destructiveChanges).toEqual({ - destructiveOperations: [ - { id: 'dropColumn.user.nickname', label: 'Drop column nickname from user' }, + it('asks about each loss and each access widening in one batch before applying', async () => { + const asked: { question: string; subject: string; verbs: readonly string[] }[] = []; + const execute = vi.fn().mockResolvedValue( + ok({ + perSpaceResults: [ + { space: 'app', value: { operationsPlanned: 3, operationsExecuted: 3 } }, ], - databaseName: 'appdb', - planHash: expect.stringMatching(/^[0-9a-f]{64}$/), - }); - } + }), + ); + const result = await executeDbUpdate( + applyInputs({ + migrations: createDestructiveMigrations(execute), + answerQuestions: async (questions) => { + expect(execute).not.toHaveBeenCalled(); + asked.push( + ...questions.map(({ question, subject, verbs }) => ({ question, subject, verbs })), + ); + return questions.map((question) => ({ + verb: question.verbs.includes('allow') ? 'allow' : 'delete', + text: question.subject, + })); + }, + }), + ); + + expect(asked).toEqual([ + { + question: `Drop column nickname from user would lose the data in "user.nickname", named by its storage name because the origin contract is unknown; --delete loses its rows. ${ORIGIN_SNAPSHOT_RECOVERY}`, + subject: 'user.nickname', + verbs: ['delete'], + }, + { + question: + 'Disable row-level security on user would widen who can read and write its rows.', + subject: 'user', + verbs: ['allow'], + }, + ]); + expect(result.ok && result.value.appliedStatements).toEqual([ + { + verb: 'delete', + statement: { kind: 'delete', subject: NICKNAME }, + operationIndexes: [0], + description: 'delete storage "user.nickname"', + }, + { + verb: 'allow', + statement: { kind: 'allow', subject: USER }, + operationIndexes: [2], + description: 'allow storage "user"', + }, + ]); + expect(execute).toHaveBeenCalledTimes(1); }); - it('applies when the consent names the refused plan', async () => { - const sharedInputs = () => ({ - driver: createMockDriver(), - adapter: STUB_ADAPTER, - familyInstance: createMockFamilyInstance({ - readAllMarkers: async () => new Map([['app', markerRecord({ storageHash: 'origin' })]]), + it('takes delete and allow statements as answers without asking', async () => { + const answerQuestions = vi.fn(noQuestions); + const result = await executeDbUpdate( + applyInputs({ + statements: [ + { verb: 'delete', text: 'user.nickname' }, + { verb: 'allow', text: 'user' }, + ], + answerQuestions, }), - contract: dummyContract, - mode: 'apply' as const, - migrations: createDestructiveMigrations(), - frameworkComponents: [], - migrationsDir: FAKE_MIGRATIONS_DIR, - targetId: 'postgres' as const, - }); + ); - const refused = await executeDbUpdate(sharedInputs()); - expect(refused.ok).toBe(false); - const planHash = !refused.ok ? refused.failure.destructiveChanges?.planHash : undefined; - expect(planHash).toBeDefined(); + expect(result.ok).toBe(true); + expect(answerQuestions).toHaveBeenCalledWith([]); + }); - const result = await executeDbUpdate({ - ...sharedInputs(), - consent: { planHash: planHash as string }, + it('refuses delete and allow statements that answer no question, before applying', async () => { + const execute = vi.fn(); + const error = await executeDbUpdate( + applyInputs({ + migrations: createDestructiveMigrations(execute), + statements: [ + { verb: 'delete', text: 'user.nickname' }, + { verb: 'delete', text: 'Nope' }, + { verb: 'allow', text: 'user' }, + { verb: 'allow', text: 'post' }, + ], + }), + ).catch((caught: unknown) => caught); + + expect(error).toMatchObject({ + code: 'MIGRATION.STATEMENT_ANSWERS_NO_QUESTION', + message: 'Statements "--delete Nope" and "--allow post" answer no question of the plan', + meta: { + statements: [ + { verb: 'delete', text: 'Nope' }, + { verb: 'allow', text: 'post' }, + ], + subjects: ['user.nickname', 'user'], + }, }); + expect(execute).not.toHaveBeenCalled(); + }); + + it('answers every data-loss question with acceptDataLoss: true, and still asks about access', async () => { + const asked: string[] = []; + const result = await executeDbUpdate( + applyInputs({ + acceptDataLoss: true, + answerQuestions: async (questions) => { + asked.push(...questions.map(({ subject }) => subject)); + return questions.map(({ subject }) => ({ verb: 'allow', text: subject })); + }, + }), + ); expect(result.ok).toBe(true); - if (result.ok) { - expect(result.value.mode).toBe('apply'); - expect(result.value.execution).toBeDefined(); - } + expect(asked).toEqual(['user']); }); - it('refuses a consent naming a plan it is no longer about to apply', async () => { - const consentedPlanHash = 'f'.repeat(64); - const result = await executeDbUpdate({ - driver: createMockDriver(), - adapter: STUB_ADAPTER, - familyInstance: createMockFamilyInstance({ - readAllMarkers: async () => new Map([['app', markerRecord({ storageHash: 'origin' })]]), - }), - contract: dummyContract, - mode: 'apply', - consent: { planHash: consentedPlanHash }, - migrations: createDestructiveMigrations(), - frameworkComponents: [], - migrationsDir: FAKE_MIGRATIONS_DIR, - targetId: 'postgres', - }); + it('answers every question with acceptDataLoss and acceptAccessWidening', async () => { + const answerQuestions = vi.fn(noQuestions); + const result = await executeDbUpdate( + applyInputs({ acceptDataLoss: true, acceptAccessWidening: true, answerQuestions }), + ); - expect(result.ok).toBe(false); - if (!result.ok) { - expect(result.failure.code).toBe('CONSENT_PLAN_MISMATCH'); - expect(result.failure.consentPlanMismatch).toEqual({ - consentedPlanHash, - planHash: expect.stringMatching(/^[0-9a-f]{64}$/), - }); - } + expect(result.ok).toBe(true); + expect(answerQuestions).toHaveBeenCalledWith([]); }); - it('proceeds to runner in apply mode with acceptDataLoss: true', async () => { - const result = await executeDbUpdate({ - driver: createMockDriver(), - adapter: STUB_ADAPTER, - familyInstance: createMockFamilyInstance({ - readAllMarkers: async () => new Map([['app', markerRecord({ storageHash: 'origin' })]]), + it('rejects an answer callback that leaves a question unanswered, before applying', async () => { + const execute = vi.fn(); + const answers = [{ verb: 'delete', text: 'user.nickname' }] as const; + const error = await executeDbUpdate( + applyInputs({ + migrations: createDestructiveMigrations(execute), + answerQuestions: async () => answers, }), - contract: dummyContract, - mode: 'apply', - acceptDataLoss: true, - migrations: createDestructiveMigrations(), - frameworkComponents: [], - migrationsDir: FAKE_MIGRATIONS_DIR, - targetId: 'postgres', + ).catch((caught: unknown) => caught); + + expect(error).toBeInstanceOf(InternalError); + expect(error).toMatchObject({ + message: + 'answerQuestions gave 1 answer for 2 questions; answer each question in order, or throw to refuse.', + cause: { questions: ['user.nickname', 'user'], answers }, }); + expect(execute).not.toHaveBeenCalled(); + }); - expect(result.ok).toBe(true); - if (result.ok) { - expect(result.value.mode).toBe('apply'); - expect(result.value.execution).toBeDefined(); - } + it('rejects an answer the question does not accept, before applying', async () => { + const execute = vi.fn(); + const error = await executeDbUpdate( + applyInputs({ + migrations: createDestructiveMigrations(execute), + answerQuestions: async () => [ + { verb: 'delete', text: 'user.nickname' }, + { verb: 'delete', text: 'user' }, + ], + }), + ).catch((caught: unknown) => caught); + + expect(error).toBeInstanceOf(InternalError); + expect(error).toMatchObject({ + message: + 'answerQuestions answered "Disable row-level security on user would widen who can read and write its rows." with delete, which it does not accept; it accepts allow.', + cause: { question: 'user', verbs: ['allow'], answer: { verb: 'delete', text: 'user' } }, + }); + expect(execute).not.toHaveBeenCalled(); }); - it('returns success in plan mode regardless of destructive operations', async () => { - const result = await executeDbUpdate({ - driver: createMockDriver(), - adapter: STUB_ADAPTER, - familyInstance: createMockFamilyInstance({ - readAllMarkers: async () => new Map([['app', markerRecord({ storageHash: 'origin' })]]), + it('refuses, without asking, delete and allow statements that answer no question in plan mode', async () => { + const answerQuestions = vi.fn(noQuestions); + const error = await executeDbUpdate( + applyInputs({ + mode: 'plan', + answerQuestions, + statements: [ + { verb: 'delete', text: 'user.nickname' }, + { verb: 'allow', text: 'post' }, + ], }), - contract: dummyContract, - mode: 'plan', - migrations: createDestructiveMigrations(), - frameworkComponents: [], - migrationsDir: FAKE_MIGRATIONS_DIR, - targetId: 'postgres', + ).catch((caught: unknown) => caught); + + expect(error).toMatchObject({ + code: 'MIGRATION.STATEMENT_ANSWERS_NO_QUESTION', + meta: { statements: [{ verb: 'allow', text: 'post' }] }, }); + expect(answerQuestions).not.toHaveBeenCalled(); + }); + + it('takes delete and allow statements that answer questions in plan mode', async () => { + const result = await executeDbUpdate( + applyInputs({ + mode: 'plan', + statements: [ + { verb: 'delete', text: 'user.nickname' }, + { verb: 'allow', text: 'user' }, + ], + }), + ); expect(result.ok).toBe(true); - if (result.ok) { - expect(result.value.mode).toBe('plan'); - expect(result.value.plan.operations).toHaveLength(2); - } + }); + + it('asks nothing in plan mode, and lists the questions an apply would ask', async () => { + const answerQuestions = vi.fn(noQuestions); + const result = await executeDbUpdate(applyInputs({ mode: 'plan', answerQuestions })); + + expect(answerQuestions).not.toHaveBeenCalled(); + expect( + result.ok && { + dataLoss: result.value.dataLoss, + accessWidening: result.value.accessWidening, + }, + ).toEqual({ + dataLoss: [{ operationIndex: 0, subject: NICKNAME, text: 'user.nickname' }], + accessWidening: [{ operationIndex: 2, subject: USER, widens: true, text: 'user' }], + }); }); }); @@ -641,6 +778,8 @@ describe('executeDbUpdate', () => { createPlanner: () => ({ plan: vi.fn().mockReturnValue({ kind: 'success', + dataLoss: [], + accessWidening: [], appliedStatements: [], plan: { targetId: 'postgres', @@ -674,6 +813,7 @@ describe('executeDbUpdate', () => { migrations, frameworkComponents: [], migrationsDir: FAKE_MIGRATIONS_DIR, + answerQuestions: noQuestions, targetId: 'postgres', }); @@ -705,6 +845,7 @@ describe('executeDbUpdate', () => { migrations: createMockMigrations(), frameworkComponents: [], migrationsDir: FAKE_MIGRATIONS_DIR, + answerQuestions: noQuestions, targetId: 'postgres', onProgress: (event) => events.push(event), }); @@ -732,6 +873,7 @@ describe('executeDbUpdate', () => { migrations: createMockMigrations(), frameworkComponents: [], migrationsDir: FAKE_MIGRATIONS_DIR, + answerQuestions: noQuestions, targetId: 'postgres', onProgress: (event) => events.push(event), }); @@ -765,6 +907,7 @@ describe('executeDbUpdate', () => { }), frameworkComponents: [], migrationsDir: FAKE_MIGRATIONS_DIR, + answerQuestions: noQuestions, targetId: 'postgres', onProgress: (event) => events.push(event), }); @@ -793,6 +936,7 @@ describe('executeDbUpdate', () => { }), frameworkComponents: [], migrationsDir: FAKE_MIGRATIONS_DIR, + answerQuestions: noQuestions, targetId: 'postgres', onProgress: (event) => events.push(event), }); @@ -811,6 +955,7 @@ describe('executeDbUpdate', () => { migrations: createMockMigrations(), frameworkComponents: [], migrationsDir: FAKE_MIGRATIONS_DIR, + answerQuestions: noQuestions, targetId: 'postgres', }); diff --git a/packages/1-framework/3-tooling/cli/test/control-api/migrate-plan-space-path.test.ts b/packages/1-framework/3-tooling/cli/test/control-api/migrate-plan-space-path.test.ts index 14faa9d92908..b1e1217234e5 100644 --- a/packages/1-framework/3-tooling/cli/test/control-api/migrate-plan-space-path.test.ts +++ b/packages/1-framework/3-tooling/cli/test/control-api/migrate-plan-space-path.test.ts @@ -49,6 +49,7 @@ const aggregate = { targetId: 'postgres', app: { spaceId: 'app' } } as Pick< describe('planSpacePath — empty-graph spaces', () => { it('an all-external extension space with no marker resolves declaratively to its head ref', () => { const outcome = planSpacePath({ + storageNameOf: (operation) => operation.id, space: makeEmptyGraphSpace('supabase'), aggregate, targetHash: HEAD_HASH, @@ -66,6 +67,7 @@ describe('planSpacePath — empty-graph spaces', () => { it('an extension space with a stale marker advances declaratively to the new head', () => { const outcome = planSpacePath({ + storageNameOf: (operation) => operation.id, space: makeEmptyGraphSpace('supabase'), aggregate, targetHash: HEAD_HASH, @@ -84,6 +86,7 @@ describe('planSpacePath — empty-graph spaces', () => { it('an extension space already at head stays an at-head resolution (no marker rewrite)', () => { const outcome = planSpacePath({ + storageNameOf: (operation) => operation.id, space: makeEmptyGraphSpace('supabase'), aggregate, targetHash: HEAD_HASH, @@ -96,6 +99,7 @@ describe('planSpacePath — empty-graph spaces', () => { it('an extension space whose head requires invariants stays unsatisfiable', () => { const outcome = planSpacePath({ + storageNameOf: (operation) => operation.id, space: makeEmptyGraphSpace('supabase', ['ext:install-v1']), aggregate, targetHash: HEAD_HASH, @@ -113,6 +117,7 @@ describe('planSpacePath — empty-graph spaces', () => { // the space is Prisma 8-managed. A managed element with no authored // graph is an authoring bug and must fail loudly. const outcome = planSpacePath({ + storageNameOf: (operation) => operation.id, space: makeEmptyGraphSpace('broken-extension', [], 'managed'), aggregate, targetHash: HEAD_HASH, @@ -129,6 +134,7 @@ describe('planSpacePath — empty-graph spaces', () => { it('the APP space with an empty graph and a pending target stays never-planned', () => { const outcome = planSpacePath({ + storageNameOf: (operation) => operation.id, space: makeEmptyGraphSpace('app'), aggregate, targetHash: HEAD_HASH, @@ -152,6 +158,7 @@ describe('planSpacePath — empty-graph spaces', () => { deserializeContract: (json) => json as Contract, }); const outcome = planSpacePath({ + storageNameOf: (operation) => operation.id, space, aggregate, targetHash: EMPTY_CONTRACT_HASH, diff --git a/packages/1-framework/3-tooling/cli/test/control-api/migration-plan-prologue.test.ts b/packages/1-framework/3-tooling/cli/test/control-api/migration-plan-prologue.test.ts index 963d6fd11be6..d0cca924efb4 100644 --- a/packages/1-framework/3-tooling/cli/test/control-api/migration-plan-prologue.test.ts +++ b/packages/1-framework/3-tooling/cli/test/control-api/migration-plan-prologue.test.ts @@ -37,7 +37,13 @@ describe('executeMigrationPlanCommand — mutation-prologue guard', () => { ); const result = await executeMigrationPlanCommand( - { config, cwd: '/project', projectDir: '/project', client: stubClient }, + { + config, + cwd: '/project', + projectDir: '/project', + client: stubClient, + answerQuestions: async () => [], + }, Date.now(), ); @@ -59,7 +65,13 @@ describe('executeMigrationPlanCommand — mutation-prologue guard', () => { ); const result = await executeMigrationPlanCommand( - { config, cwd: '/project', projectDir: '/project', client: stubClient }, + { + config, + cwd: '/project', + projectDir: '/project', + client: stubClient, + answerQuestions: async () => [], + }, Date.now(), ); diff --git a/packages/1-framework/3-tooling/cli/test/control-api/statements/ask-plan-questions.test.ts b/packages/1-framework/3-tooling/cli/test/control-api/statements/ask-plan-questions.test.ts new file mode 100644 index 000000000000..46dda0df1d95 --- /dev/null +++ b/packages/1-framework/3-tooling/cli/test/control-api/statements/ask-plan-questions.test.ts @@ -0,0 +1,314 @@ +import { asNamespaceId } from '@internal/contract/types'; +import { ok } from '@internal/utils/result'; +import { describe, expect, it } from 'vitest'; +import { + askPlanQuestions, + ORIGIN_SNAPSHOT_RECOVERY, + type PlannedQuestions, + type PlannedSubject, + type PlanQuestion, +} from '../../../src/control-api/statements/plan-questions'; +import { contractOf } from './statement-fixtures'; + +const origin = contractOf({ app: { models: { T: { fields: ['id', 'ratio'] } } } }); +const destination = contractOf({ app: { models: { T: { fields: ['id', 'ratio2'] } } } }); +const ratio: PlannedSubject['subject'] = { + kind: 'field', + namespaceId: asNamespaceId('app'), + model: 'T', + field: 'ratio', +}; + +function planWith( + ...losses: readonly { label: string; operationIndex: number }[] +): PlannedQuestions { + return { + dataLoss: losses.map((loss) => ({ ...loss, subject: ratio })), + accessWidening: [], + }; +} + +const dropRatio = { label: 'Drop column "ratio" from "T"', operationIndex: 1 }; +const convertRatio = { label: 'Alter type of "T"."ratio2" to int4', operationIndex: 1 }; + +function run(replanned: PlannedQuestions, typed: readonly (readonly string[])[]) { + const asked: (readonly string[])[] = []; + let round = 0; + return { + asked, + result: askPlanQuestions({ + plan: planWith(dropRatio), + askAccess: false, + renames: [], + preAnswers: [], + consentAll: { delete: false, allow: false }, + origin, + originKnown: true, + keepDataByHand: undefined, + destination, + answer: async (questions: readonly PlanQuestion[]) => { + asked.push(questions.map(({ question }) => question)); + const answers = typed[round] ?? []; + round += 1; + return answers.map((answer) => { + const [verb, text] = answer.split(' '); + return { verb: verb === 'rename' ? 'rename' : 'delete', text: text ?? '' } as const; + }); + }, + replan: async () => ok(replanned), + }), + }; +} + +describe('a rename typed at the prompt', () => { + it('resolves its loss when the operation asked about is gone, and asks about a new loss on the same field', async () => { + const { asked, result } = run(planWith(convertRatio), [ + ['rename T.ratio:T.ratio2'], + ['delete T.ratio'], + ]); + + expect((await result).ok).toBe(true); + expect(asked).toEqual([ + ['Drop column "ratio" from "T" would lose the values of field "T.ratio".'], + ['Alter type of "T"."ratio2" to int4 would lose the values of field "T.ratio".'], + ]); + }); + + it('fails when the operation asked about is still in the plan', async () => { + const { result } = run(planWith(dropRatio), [['rename T.ratio:T.ratio2']]); + + const outcome = await result; + expect(outcome.ok).toBe(false); + expect(!outcome.ok && outcome.failure).toMatchObject({ + code: 'MIGRATION.STATEMENT_DID_NOT_RESOLVE_LOSS', + }); + }); +}); + +describe('questions about storage when the origin contract is unknown', () => { + it('give the steps that store the snapshot once, on the first such question', async () => { + const asked: string[] = []; + const storage = (name: string, operationIndex: number) => ({ + operationIndex, + label: `Drop table ${name}`, + subject: { kind: 'storage', name } as const, + }); + await askPlanQuestions({ + plan: { dataLoss: [storage('Legacy', 0), storage('Profile', 1)], accessWidening: [] }, + askAccess: false, + renames: [], + preAnswers: [], + consentAll: { delete: false, allow: false }, + origin, + originKnown: false, + keepDataByHand: undefined, + destination, + answer: async (questions) => { + asked.push(...questions.map(({ question }) => question)); + return questions.map(({ subject }) => ({ verb: 'delete', text: subject })); + }, + replan: async () => ok({ dataLoss: [], accessWidening: [] }), + }); + + expect(asked).toEqual([ + `Drop table Legacy would lose the data in "Legacy", named by its storage name because the origin contract is unknown; --delete loses its rows. ${ORIGIN_SNAPSHOT_RECOVERY}`, + 'Drop table Profile would lose the data in "Profile", named by its storage name because the origin contract is unknown; --delete loses its rows.', + ]); + expect(ORIGIN_SNAPSHOT_RECOVERY).toContain('--dry-run'); + expect(ORIGIN_SNAPSHOT_RECOVERY).toContain('`prisma db update --advance-ref --dry-run`'); + expect(ORIGIN_SNAPSHOT_RECOVERY).toContain( + 'with the same `--db` as this command if it has one', + ); + }); +}); + +describe('questions about access', () => { + it('ask once per operation, and say a policy drop changes access where disabling row-level security widens it', async () => { + const user = { kind: 'model', namespaceId: asNamespaceId('app'), model: 'T' } as const; + const asked: string[] = []; + const result = await askPlanQuestions({ + plan: { + dataLoss: [], + accessWidening: [ + { + operationIndex: 0, + label: 'Drop RLS policy "readers" on "T"', + subject: user, + widens: false, + }, + { + operationIndex: 1, + label: 'Disable row-level security on "T"', + subject: user, + widens: true, + }, + ], + }, + askAccess: true, + renames: [], + preAnswers: [], + consentAll: { delete: false, allow: false }, + origin, + originKnown: true, + keepDataByHand: undefined, + destination, + answer: async (questions) => { + asked.push(...questions.map(({ question }) => question)); + return questions.map(({ subject }) => ({ verb: 'allow', text: subject })); + }, + replan: async () => ok({ dataLoss: [], accessWidening: [] }), + }); + + expect(asked).toEqual([ + 'Drop RLS policy "readers" on "T" would change who can read and write its rows.', + 'Disable row-level security on "T" would widen who can read and write its rows.', + ]); + expect( + result.ok && + result.value.consented.map(({ verb, text, operationIndex }) => ({ + verb, + text, + operationIndex, + })), + ).toEqual([ + { verb: 'allow', text: 'T', operationIndex: 0 }, + { verb: 'allow', text: 'T', operationIndex: 1 }, + ]); + }); +}); + +describe('allow statements given as pre-answers', () => { + const t = { kind: 'model', namespaceId: asNamespaceId('app'), model: 'T' } as const; + const plan = { + dataLoss: [], + accessWidening: [ + { operationIndex: 0, label: 'Drop RLS policy "readers" on "T"', subject: t, widens: false }, + { operationIndex: 1, label: 'Disable row-level security on "T"', subject: t, widens: true }, + ], + }; + const ask = (preAnswers: readonly { verb: 'allow'; text: string }[]) => { + const asked: string[] = []; + const result = askPlanQuestions({ + plan, + askAccess: true, + renames: [], + preAnswers, + consentAll: { delete: false, allow: false }, + origin, + originKnown: true, + keepDataByHand: undefined, + destination, + answer: async (questions) => { + asked.push(...questions.map(({ question }) => question)); + return questions.map(({ subject }) => ({ verb: 'allow', text: subject })); + }, + replan: async () => ok({ dataLoss: [], accessWidening: [] }), + }); + return { asked, result }; + }; + + it('answer one operation each, in order, and leave the rest to be asked', async () => { + const { asked, result } = ask([{ verb: 'allow', text: 'T' }]); + + expect((await result).ok).toBe(true); + expect(asked).toEqual([ + 'Disable row-level security on "T" would widen who can read and write its rows.', + ]); + }); + + it('answer every operation when there is one per operation', async () => { + const { asked, result } = ask([ + { verb: 'allow', text: 'T' }, + { verb: 'allow', text: 'T' }, + ]); + + expect((await result).ok).toBe(true); + expect(asked).toEqual([]); + }); + + it('refuse an allow left over once each operation has one', async () => { + const { result } = ask([ + { verb: 'allow', text: 'T' }, + { verb: 'allow', text: 'T' }, + { verb: 'allow', text: 'T' }, + ]); + + const outcome = await result; + expect(!outcome.ok && outcome.failure).toMatchObject({ + code: 'MIGRATION.STATEMENT_ANSWERS_NO_QUESTION', + }); + }); +}); + +describe('an access change consented before a typed rename', () => { + it('is asked once, and reported at its position in the new plan', async () => { + const t = { kind: 'model', namespaceId: asNamespaceId('app'), model: 'T' } as const; + const policyDrop = { label: 'Drop RLS policy "readers" on "T"', subject: t, widens: false }; + let rounds = 0; + const result = await askPlanQuestions({ + plan: { + dataLoss: [{ ...dropRatio, operationIndex: 0, subject: ratio }], + accessWidening: [{ ...policyDrop, operationIndex: 1 }], + }, + askAccess: true, + renames: [], + preAnswers: [], + consentAll: { delete: false, allow: false }, + origin, + originKnown: true, + keepDataByHand: undefined, + destination, + answer: async (questions) => { + rounds += 1; + return questions.map(({ subject, verbs }) => + verbs.includes('allow') + ? { verb: 'allow', text: subject } + : { verb: 'rename', text: 'T.ratio:T.ratio2' }, + ); + }, + replan: async () => + ok({ dataLoss: [], accessWidening: [{ ...policyDrop, operationIndex: 3 }] }), + }); + + expect(rounds).toBe(1); + expect( + result.ok && + result.value.consented.map(({ verb, operationIndex }) => ({ verb, operationIndex })), + ).toEqual([{ verb: 'allow', operationIndex: 3 }]); + }); +}); + +describe('questions about storage on a target that carries out no rename', () => { + it('say how to keep the data by hand, without the steps that make a rename possible', async () => { + const asked: string[] = []; + await askPlanQuestions({ + plan: { + dataLoss: [ + { + operationIndex: 0, + label: 'Drop collection events', + subject: { kind: 'storage', name: 'events' }, + }, + ], + accessWidening: [], + }, + askAccess: false, + renames: [], + preAnswers: [], + consentAll: { delete: false, allow: false }, + origin, + originKnown: false, + keepDataByHand: () => 'Rename the collection by hand first.', + destination, + answer: async (questions) => { + asked.push(...questions.map(({ question }) => question)); + return questions.map(({ subject }) => ({ verb: 'delete', text: subject })); + }, + replan: async () => ok({ dataLoss: [], accessWidening: [] }), + }); + + expect(asked).toEqual([ + 'Drop collection events would lose the data in "events", named by its storage name because the origin contract is unknown; --delete loses its rows. Rename the collection by hand first.', + ]); + }); +}); diff --git a/packages/1-framework/3-tooling/cli/test/control-api/statements/data-loss-question.test.ts b/packages/1-framework/3-tooling/cli/test/control-api/statements/data-loss-question.test.ts new file mode 100644 index 000000000000..5231178ba9a7 --- /dev/null +++ b/packages/1-framework/3-tooling/cli/test/control-api/statements/data-loss-question.test.ts @@ -0,0 +1,115 @@ +import { asNamespaceId } from '@internal/contract/types'; +import { describe, expect, it } from 'vitest'; +import { dataLossQuestion } from '../../../src/control-api/statements/plan-questions'; +import { contractOf } from './statement-fixtures'; + +const origin = contractOf({ app: { models: { User: { fields: ['email'] } } } }); +const destination = contractOf({ app: { models: { User: { fields: ['email'] } } } }); +const drop = { + operationIndex: 0, + label: 'Drop column age from user', + subject: { kind: 'storage', name: 'public.user.age' }, +} as const; + +describe('dataLossQuestion for data no model names', () => { + it('says no model of a known origin contract stores it', () => { + const question = dataLossQuestion(drop, { + origin, + destination, + renames: [], + originKnown: true, + keepDataByHand: undefined, + }); + + expect({ question: question.question, verbs: question.verbs }).toEqual({ + question: + 'Drop column age from user would lose the data in "public.user.age", which no model of the origin contract stores.', + verbs: ['delete'], + }); + }); + + it('says the name is a storage name when the origin contract is unknown', () => { + const question = dataLossQuestion(drop, { + origin, + destination, + renames: [], + originKnown: false, + keepDataByHand: undefined, + }); + + expect(question.question).toBe( + 'Drop column age from user would lose the data in "public.user.age", named by its storage name because the origin contract is unknown; --delete loses its rows.', + ); + }); +}); + +describe('dataLossQuestion verbs', () => { + const unbound = asNamespaceId('app'); + const destinationWithoutNickname = contractOf({ + app: { models: { User: { fields: ['email'] } } }, + }); + const withNickname = contractOf({ app: { models: { User: { fields: ['email', 'nickname'] } } } }); + + const keepByHand = () => 'Rename it by hand first.'; + + function questionFor( + field: string, + destination: typeof origin, + keepDataByHand: (() => string) | undefined, + ) { + return dataLossQuestion( + { + operationIndex: 0, + label: 'An operation', + subject: { kind: 'field', namespaceId: unbound, model: 'User', field }, + }, + { origin: withNickname, destination, renames: [], originKnown: true, keepDataByHand }, + ); + } + + function verbsFor(field: string, destination: typeof origin, renamesPlannable: boolean) { + return questionFor(field, destination, renamesPlannable ? undefined : keepByHand).verbs; + } + + it('offers rename for a field the destination no longer has', () => { + expect(verbsFor('nickname', destinationWithoutNickname, true)).toEqual(['rename', 'delete']); + }); + + it('offers only delete for a field that keeps its name, such as one whose type changes', () => { + expect(verbsFor('email', destinationWithoutNickname, true)).toEqual(['delete']); + }); + + it('offers only delete for a field a rename statement of the plan already renamed', () => { + const destination = contractOf({ app: { models: { User: { fields: ['email', 'handle'] } } } }); + const question = dataLossQuestion( + { + operationIndex: 1, + label: 'Recreate table User', + subject: { kind: 'field', namespaceId: unbound, model: 'User', field: 'nickname' }, + }, + { + origin: withNickname, + destination, + renames: [{ verb: 'rename', text: 'User.nickname:User.handle' }], + originKnown: true, + keepDataByHand: undefined, + }, + ); + expect(question.verbs).toEqual(['delete']); + }); + + it('offers only delete where the planner cannot carry out a rename, and says how to keep the data by hand', () => { + const question = questionFor('nickname', destinationWithoutNickname, keepByHand); + expect({ verbs: question.verbs, question: question.question }).toEqual({ + verbs: ['delete'], + question: + 'An operation would lose the values of field "User.nickname". Rename it by hand first.', + }); + }); + + it('writes the rename form of a field with its model, as a field rename is written', () => { + expect(questionFor('nickname', destinationWithoutNickname, undefined).forms).toEqual({ + rename: 'User.nickname:User.', + }); + }); +}); diff --git a/packages/1-framework/3-tooling/cli/test/control-api/statements/resolve-delete-statements.test.ts b/packages/1-framework/3-tooling/cli/test/control-api/statements/resolve-delete-statements.test.ts new file mode 100644 index 000000000000..a8bc5315e6de --- /dev/null +++ b/packages/1-framework/3-tooling/cli/test/control-api/statements/resolve-delete-statements.test.ts @@ -0,0 +1,84 @@ +import { describe, expect, it } from 'vitest'; +import { + resolveStatements, + type StatementOrigin, +} from '../../../src/control-api/statements/resolve-statements'; +import type { StatementText } from '../../../src/control-api/statements/statement-text'; +import { contractOf, expectFailure, expectValue } from './statement-fixtures'; + +const origin = contractOf({ app: { models: { Profile: {}, Legacy: {} } } }); +const destination = contractOf({ app: { models: { User: {} } } }); + +const missingOrigin: StatementOrigin = { + kind: 'missing', + hash: 'sha256:abc', + snapshotDirectory: 'migrations/snapshots', + unreadable: undefined, +}; + +function remove(text: string): StatementText { + return { verb: 'delete', text }; +} + +function rename(text: string): StatementText { + return { verb: 'rename', text }; +} + +describe('resolveStatements, delete statements', () => { + it('passes a delete through unresolved, since it is matched against the plan', () => { + const result = resolveStatements({ + statements: [remove('Legacy'), rename('Profile:User'), remove('audit_log')], + origin: { kind: 'contract', contract: origin }, + destination, + }); + + expect(expectValue(result)).toEqual([ + { + kind: 'rename', + entity: 'model', + from: { namespaceId: 'app', model: 'Profile' }, + to: { namespaceId: 'app', model: 'User' }, + }, + ]); + }); + + it('needs no origin contract for a delete', () => { + const result = resolveStatements({ + statements: [remove('Legacy'), remove('User.name')], + origin: missingOrigin, + destination, + }); + + expect(expectValue(result)).toEqual([]); + }); + + it('refuses a rename beside a delete when the origin contract is unknown', () => { + const result = resolveStatements({ + statements: [remove('Legacy'), rename('Profile:User')], + origin: missingOrigin, + destination, + }); + + expectFailure(result, 'MIGRATION.STATEMENT_ORIGIN_UNKNOWN', 'sha256:abc'); + }); + + it('refuses a delete with no name', () => { + const result = resolveStatements({ + statements: [remove(' ')], + origin: { kind: 'contract', contract: origin }, + destination, + }); + expectFailure(result, 'MIGRATION.STATEMENT_INVALID', 'names nothing'); + }); + + it('accepts any name a subject can have, including storage names with a ":"', () => { + const result = resolveStatements({ + statements: ['public.a:b', 'public.User.nick:name', 'dropTable.audit_old', 'rawSql'].map( + remove, + ), + origin: missingOrigin, + destination, + }); + expect(expectValue(result)).toEqual([]); + }); +}); diff --git a/packages/1-framework/3-tooling/cli/test/control-api/testing/fixture-client.test.ts b/packages/1-framework/3-tooling/cli/test/control-api/testing/fixture-client.test.ts index f1417fd5c73d..4062f573bc58 100644 --- a/packages/1-framework/3-tooling/cli/test/control-api/testing/fixture-client.test.ts +++ b/packages/1-framework/3-tooling/cli/test/control-api/testing/fixture-client.test.ts @@ -38,6 +38,7 @@ describe('createFixtureControlClient', () => { contract: {}, mode: 'plan', migrationsDir: 'migrations', + answerQuestions: async () => [], }); expect(dbUpdate.ok).toBe(true); @@ -100,7 +101,11 @@ describe('createFixtureControlClient', () => { }); await client.connect('postgres://fixture'); - const dbInit = await client.dbInit({ contract: {}, mode: 'plan', migrationsDir: 'migrations' }); + const dbInit = await client.dbInit({ + contract: {}, + mode: 'plan', + migrationsDir: 'migrations', + }); expect(dbInit.assertNotOk().summary).toBe('planner exploded'); expect(await client.readMarker()).toBeNull(); @@ -154,8 +159,25 @@ describe('createFixtureControlClient', () => { ['verify', (c) => c.verify({ contract: {} })], ['schemaVerify', (c) => c.schemaVerify({ contract: {} })], ['dbSign', (c) => c.dbSign({ contract: {} as never, migrationsDir: 'migrations' })], - ['dbInit', (c) => c.dbInit({ contract: {}, mode: 'plan', migrationsDir: 'migrations' })], - ['dbUpdate', (c) => c.dbUpdate({ contract: {}, mode: 'plan', migrationsDir: 'migrations' })], + [ + 'dbInit', + (c) => + c.dbInit({ + contract: {}, + mode: 'plan', + migrationsDir: 'migrations', + }), + ], + [ + 'dbUpdate', + (c) => + c.dbUpdate({ + contract: {}, + mode: 'plan', + migrationsDir: 'migrations', + answerQuestions: async () => [], + }), + ], [ 'dbVerify', (c) => diff --git a/packages/1-framework/3-tooling/cli/test/orm/db-init.test.ts b/packages/1-framework/3-tooling/cli/test/orm/db-init.test.ts index 753c3bfbd476..11f2b0e32cc3 100644 --- a/packages/1-framework/3-tooling/cli/test/orm/db-init.test.ts +++ b/packages/1-framework/3-tooling/cli/test/orm/db-init.test.ts @@ -448,7 +448,7 @@ describe('db init', () => { { kind: 'run-command', label: - 'Apply the change with db update, which lists the destructive operations and asks you to confirm them', + 'Apply the change with db update, which asks what each operation that would lose data means', command: 'prisma-test db update', }, ], diff --git a/packages/1-framework/3-tooling/cli/test/orm/db-update-consent.test.ts b/packages/1-framework/3-tooling/cli/test/orm/db-update-consent.test.ts index ff33f3b0d04e..350daa383e85 100644 --- a/packages/1-framework/3-tooling/cli/test/orm/db-update-consent.test.ts +++ b/packages/1-framework/3-tooling/cli/test/orm/db-update-consent.test.ts @@ -1,10 +1,16 @@ import { rmSync, writeFileSync } from 'node:fs'; -import { notOk, ok } from '@internal/utils/result'; -import type { EngineEvent, StreamEvent } from '@prisma/cli-engine'; +import { asNamespaceId } from '@internal/contract/types'; +import { ok } from '@internal/utils/result'; +import type { StreamEvent } from '@prisma/cli-engine'; import { join } from 'pathe'; -import stripAnsi from 'strip-ansi'; import { afterAll, beforeEach, describe, expect, it, vi } from 'vitest'; -import type { ControlClient } from '../../src/control-api/types'; +import { + accessWideningQuestion, + dataLossQuestion, + type PlanAnswer, + type PlanQuestion, +} from '../../src/control-api/statements/plan-questions'; +import type { ControlClient, DbUpdateOptions } from '../../src/control-api/types'; import { BIN_GROUPS, createBinCommands } from '../../src/orm/cli'; import { createOrmTestCli } from '../helpers/orm-test-cli'; import { createTestProjectDir, writeProjectManifest } from '../utils/test-project-dir'; @@ -39,14 +45,77 @@ const DESCRIPTOR = { familyId: 'sql', targetId: 'postgres', version: '1.0.0', cr const DEST_HASH = 'd'.repeat(64); const MARKER_HASH = 'a'.repeat(64); const CONNECTION = 'postgres://user:secret@localhost:5432/appdb'; -const TOKEN = 'appdb'; -const PLAN_HASH = 'c'.repeat(64); -const FRESH_PLAN_HASH = 'e'.repeat(64); -const QUESTION = 'Apply 1 destructive operation(s) to appdb?'; let projectDir: string; const projectDirs: string[] = []; +/** The contracts the questions name subjects in: the origin has Legacy and User, the destination User. */ +const origin = { + domain: { + namespaces: { + app: { + models: { + Legacy: { fields: {}, relations: {}, storage: {} }, + User: { fields: {}, relations: {}, storage: {} }, + }, + }, + }, + }, +}; +const destination = { + domain: { namespaces: { app: { models: { User: { fields: {}, relations: {}, storage: {} } } } } }, +}; +const contracts = { + origin, + destination, + renames: [], + originKnown: true, + keepDataByHand: undefined, +}; + +/** One question of each kind, across two spaces: an extension's recorded drop is a storage subject. */ +function planQuestions(): readonly PlanQuestion[] { + return [ + dataLossQuestion( + { + operationIndex: 0, + label: 'Drop table "audit_old"', + subject: { kind: 'storage', name: 'audit_old' }, + }, + contracts, + ), + dataLossQuestion( + { + operationIndex: 1, + label: 'Drop table "Legacy"', + subject: { kind: 'model', namespaceId: asNamespaceId('app'), model: 'Legacy' }, + }, + contracts, + ), + accessWideningQuestion( + { + operationIndex: 2, + label: 'Disable row-level security on "User"', + subject: { kind: 'model', namespaceId: asNamespaceId('app'), model: 'User' }, + widens: true, + }, + contracts, + ), + ]; +} + +/** The control API's shape: an apply asks its questions, then applies; a dry run asks nothing. */ +function askThenApply(questions: () => readonly PlanQuestion[]) { + return async (options: DbUpdateOptions) => { + if (options.mode === 'apply') { + answers.push(...(await options.answerQuestions(questions()))); + } + return ok(applySuccess(options.mode)); + }; +} + +const answers: PlanAnswer[] = []; + beforeEach(() => { projectDir = createTestProjectDir('orm-db-update-consent'); projectDirs.push(projectDir); @@ -56,9 +125,10 @@ beforeEach(() => { JSON.stringify({ storage: { storageHash: MARKER_HASH } }), ); writeFileSync(join(projectDir, 'contract.d.ts'), 'export type Contract = never;\n'); + answers.splice(0); mocks.connect.mockReset().mockResolvedValue(undefined); mocks.close.mockReset().mockResolvedValue(undefined); - mocks.dbUpdate.mockReset().mockImplementation(refuseUntilConsented()); + mocks.dbUpdate.mockReset().mockImplementation(askThenApply(planQuestions)); mocks.renderContractDts .mockReset() .mockResolvedValue(ok({ contractDts: 'export type Contract = never;\n' })); @@ -86,73 +156,22 @@ function ormConfig(overrides: Record = {}): Record { +function applySuccess(mode: 'plan' | 'apply'): Record { return { - mode: 'apply', + mode, destination: { storageHash: DEST_HASH }, plan: { - operations: [{ id: 'op-2', label: 'drop relation legacy', operationClass: 'destructive' }], + operations: [{ id: 'op-2', label: 'Drop table "Legacy"', operationClass: 'destructive' }], }, - execution: { operationsPlanned: 1, operationsExecuted: 1 }, + ...(mode === 'apply' ? { execution: { operationsPlanned: 1, operationsExecuted: 1 } } : {}), marker: { storageHash: MARKER_HASH }, - perSpace: [ - { - spaceId: 'app', - kind: 'app', - operations: [{ id: 'op-2', label: 'drop relation legacy', operationClass: 'destructive' }], - marker: { storageHash: MARKER_HASH }, - }, - ], + appliedStatements: [], + dataLoss: [], + accessWidening: [], summary: 'Database updated', }; } -function destructiveRefusal(overrides: Record = {}): Record { - return { - code: 'DESTRUCTIVE_CHANGES', - summary: 'Planned 1 destructive operation(s) that require confirmation', - why: 'Destructive operations require confirmation', - conflicts: undefined, - meta: undefined, - destructiveChanges: { - destructiveOperations: [{ id: 'op-2', label: 'drop relation legacy' }], - databaseName: TOKEN, - planHash: PLAN_HASH, - }, - ...overrides, - }; -} - -/** The refusal an apply carrying a stale consent comes back with. */ -function planMismatchFailure(): Record { - return { - code: 'CONSENT_PLAN_MISMATCH', - summary: 'The plan changed between consent and apply', - why: 'The freshly computed plan is not the plan that was consented to', - conflicts: undefined, - meta: undefined, - consentPlanMismatch: { consentedPlanHash: PLAN_HASH, planHash: FRESH_PLAN_HASH }, - }; -} - -function warnTexts(events: readonly EngineEvent[]): readonly string[] { - return events.flatMap((event) => - event.kind === 'message' && event.severity === 'warn' ? [event.text] : [], - ); -} - -/** - * The control API's own shape: an apply without consent refuses when the plan - * carries destructive operations, and the call consenting to that exact plan - * (by its hash) applies. - */ -function refuseUntilConsented() { - return (options: { readonly consent?: { readonly planHash: string } }) => - Promise.resolve( - options.consent?.planHash === PLAN_HASH ? ok(applySuccess()) : notOk(destructiveRefusal()), - ); -} - function harness(config: Record = ormConfig()) { return createOrmTestCli({ commands, groups, orm: config }); } @@ -162,327 +181,172 @@ function envelopeOf(json: readonly StreamEvent[]): unknown { return terminal?.kind === 'result' ? terminal.envelope : undefined; } -function applyCalls(): readonly { readonly consent?: { readonly planHash: string } }[] { - return mocks.dbUpdate.mock.calls.map((call) => call[0]); -} - -describe('db update consent', () => { - describe('interactively', () => { - it('applies the destructive plan once the database name is typed', async () => { - const run = await harness().run(['db', 'update', '--json'], { - cwd: projectDir, - isTty: { stdin: true }, - answers: [TOKEN], - }); - - expect(run.exitCode).toBe(0); - expect(envelopeOf(run.json)).toMatchObject({ ok: true, result: { mode: 'apply' } }); - expect(mocks.connect).toHaveBeenCalledTimes(1); - expect(applyCalls().map((call) => call.consent)).toEqual([ - undefined, - { planHash: PLAN_HASH }, - ]); - }); - - /** - * Typed on stdin rather than scripted through `answers`: a scripted answer - * is returned without the prompt ever being rendered, so the question only - * reaches stderr when the run reads it from the stream. - */ - it('names the database and every destructive operation in the question', async () => { - const run = await harness().run(['db', 'update'], { - cwd: projectDir, - isTty: { stdin: true, stdout: true, stderr: true }, - stdin: `${TOKEN}\n`, - }); - - const rendered = stripAnsi(run.stderr); - expect(rendered).toContain(QUESTION); - expect(rendered).toContain('drop relation legacy'); - }); +describe('db update questions', () => { + it('refuses where nobody can answer, listing every question across spaces with its flags', async () => { + const run = await harness().run(['db', 'update', '--json'], { cwd: projectDir }); + + expect(run.exitCode).toBe(2); + const envelope = envelopeOf(run.json) as { + readonly error: { readonly code: string; readonly nextActions: unknown }; + }; + expect(envelope.error.code).toBe('CLI.CONSENT_REQUIRED'); + const actions = JSON.stringify(envelope.error.nextActions); + for (const flag of [ + '--delete audit_old', + '--delete Legacy', + "--rename 'Legacy:'", + '--allow User', + ]) { + expect(actions).toContain(flag); + } + expect(actions).not.toContain('--rename audit_old'); + expect(actions).not.toContain('--delete User'); + }); - it('shows the planner`s warnings before it asks', async () => { - const warning = 'Column user.legacy is dropped without a backfill'; - mocks.dbUpdate - .mockReset() - .mockImplementation((options: { consent?: { planHash: string } }) => - Promise.resolve( - options.consent?.planHash === PLAN_HASH - ? ok(applySuccess()) - : notOk(destructiveRefusal({ warnings: [{ summary: warning }] })), - ), - ); - - const run = await harness().run(['db', 'update'], { - cwd: projectDir, - isTty: { stdin: true, stdout: true, stderr: true }, - stdin: `${TOKEN}\n`, - }); - - const rendered = stripAnsi(run.stderr); - expect(warnTexts(run.events)).toContain(warning); - expect(rendered).toContain(QUESTION); - expect(rendered.indexOf(warning)).toBeLessThan(rendered.indexOf(QUESTION)); - }); + it('gives a value the subject it equals when another subject is its prefix', async () => { + mocks.dbUpdate.mockImplementation( + askThenApply(() => [ + dataLossQuestion( + { + operationIndex: 0, + label: 'Drop table "Legacy"', + subject: { kind: 'model', namespaceId: asNamespaceId('app'), model: 'Legacy' }, + }, + contracts, + ), + dataLossQuestion( + { + operationIndex: 1, + label: 'Drop table "Legacy:x"', + subject: { kind: 'storage', name: 'Legacy:x' }, + }, + contracts, + ), + ]), + ); - it('applies nothing when the answer is not the database name', async () => { - const run = await harness().run(['db', 'update', '--json'], { - cwd: projectDir, - isTty: { stdin: true }, - answers: ['no'], - }); - - expect(run.exitCode).toBe(2); - expect(envelopeOf(run.json)).toMatchObject({ - ok: false, - error: { code: 'CLI.PROMPT_INVALID' }, - }); - expect(applyCalls()).toHaveLength(1); - }); + const run = await harness().run( + ['db', 'update', '--delete', 'Legacy:x', '--delete', 'Legacy', '--json'], + { cwd: projectDir }, + ); - it('exits 3 when the prompt is cancelled', async () => { - const run = await harness().run(['db', 'update', '--json'], { - cwd: projectDir, - isTty: { stdin: true }, - stdin: '', - }); - - expect(run.exitCode).toBe(3); - expect(envelopeOf(run.json)).toMatchObject({ - ok: false, - error: { code: 'CLI.PROMPT_CANCELLED' }, - }); - expect(applyCalls()).toHaveLength(1); - }); + expect(run.exitCode).toBe(0); + expect(answers).toEqual([ + { verb: 'delete', text: 'Legacy' }, + { verb: 'delete', text: 'Legacy:x' }, + ]); }); - describe('non-interactively', () => { - it('refuses without --confirm, naming the token to pass', async () => { - const run = await harness().run(['db', 'update', '--json'], { cwd: projectDir }); + it('applies once every question is answered with flags', async () => { + const run = await harness().run( + ['db', 'update', '--delete', 'audit_old', '--delete', 'Legacy', '--allow', 'User', '--json'], + { cwd: projectDir }, + ); - expect(run.exitCode).toBe(2); - expect(envelopeOf(run.json)).toMatchObject({ - ok: false, - error: { code: 'CLI.CONSENT_REQUIRED', meta: { consentToken: TOKEN } }, - }); - expect(applyCalls()).toHaveLength(1); - }); + expect(run.exitCode).toBe(0); + expect(answers).toEqual([ + { verb: 'delete', text: 'audit_old' }, + { verb: 'delete', text: 'Legacy' }, + { verb: 'allow', text: 'User' }, + ]); + }); - it('applies when --confirm carries the database name', async () => { - const run = await harness().run(['db', 'update', '--confirm', TOKEN, '--json'], { - cwd: projectDir, - }); - - expect(run.exitCode).toBe(0); - expect(envelopeOf(run.json)).toMatchObject({ ok: true, result: { mode: 'apply' } }); - expect(applyCalls().map((call) => call.consent)).toEqual([ - undefined, - { planHash: PLAN_HASH }, - ]); - }); + it('refuses an apply that widens access without --allow', async () => { + const run = await harness().run( + ['db', 'update', '--delete', 'audit_old', '--delete', 'Legacy', '--json'], + { cwd: projectDir }, + ); - it('refuses when --confirm carries another name', async () => { - const run = await harness().run(['db', 'update', '--confirm', 'otherdb', '--json'], { - cwd: projectDir, - }); - - expect(run.exitCode).toBe(2); - expect(envelopeOf(run.json)).toMatchObject({ - ok: false, - error: { code: 'CLI.CONSENT_REQUIRED' }, - }); - expect(applyCalls()).toHaveLength(1); + expect(envelopeOf(run.json)).toMatchObject({ + error: { + code: 'CLI.CONSENT_REQUIRED', + meta: { unanswered: [{ subject: 'User', verbs: ['allow'] }] }, + }, }); }); - describe('--yes', () => { - it('does not accept data loss on its own', async () => { - const run = await harness().run(['db', 'update', '--yes', '--json'], { - cwd: projectDir, - isTty: { stdin: true }, - }); - - expect(run.exitCode).toBe(2); - expect(envelopeOf(run.json)).toMatchObject({ - ok: false, - error: { code: 'CLI.CONSENT_REQUIRED' }, - }); - expect(applyCalls()).toHaveLength(1); + it('takes typed answers', async () => { + const run = await harness().run(['db', 'update', '--json'], { + cwd: projectDir, + isTty: { stdin: true }, + answers: ['delete', 'delete', 'allow'], }); - it('does not stop --confirm from granting', async () => { - const run = await harness().run(['db', 'update', '--yes', '--confirm', TOKEN, '--json'], { - cwd: projectDir, - }); - - expect(run.exitCode).toBe(0); - expect(applyCalls().map((call) => call.consent)).toEqual([ - undefined, - { planHash: PLAN_HASH }, - ]); - }); + expect(run.exitCode).toBe(0); + expect(answers.map(({ verb }) => verb)).toEqual(['delete', 'delete', 'allow']); }); - describe('the consent token', () => { - it('asks with the database name the refusal carries', async () => { - mocks.dbUpdate.mockReset().mockResolvedValue( - notOk( - destructiveRefusal({ - destructiveChanges: { - destructiveOperations: [{ id: 'op-2', label: 'drop relation legacy' }], - databaseName: 'otherdb', - planHash: PLAN_HASH, - }, - }), - ), - ); - - const run = await harness().run(['db', 'update', '--json'], { cwd: projectDir }); - - expect(envelopeOf(run.json)).toMatchObject({ - ok: false, - error: { meta: { consentToken: 'otherdb' } }, - }); + it('does not take --confirm or --yes as an answer', async () => { + const confirmed = await harness().run(['db', 'update', '--confirm', 'appdb', '--json'], { + cwd: projectDir, }); - }); - - describe('when the question would be unanswerable', () => { - it('refuses rather than ask for a blank token', async () => { - mocks.dbUpdate.mockReset().mockResolvedValue( - notOk( - destructiveRefusal({ - destructiveChanges: { - destructiveOperations: [{ id: 'op-2', label: 'drop relation legacy' }], - databaseName: undefined, - planHash: PLAN_HASH, - }, - }), - ), - ); - - const run = await harness().run(['db', 'update', '--json'], { - cwd: projectDir, - isTty: { stdin: true }, - answers: [TOKEN], - }); - - expect(run.exitCode).toBe(2); - expect(envelopeOf(run.json)).toMatchObject({ - ok: false, - error: { code: 'CLI.CONSENT_TOKEN_UNRESOLVED' }, - }); - expect(applyCalls()).toHaveLength(1); + const yes = await harness().run(['db', 'update', '--yes', '--json'], { + cwd: projectDir, + isTty: { stdin: true }, }); - it('refuses rather than ask about operations the refusal did not name', async () => { - mocks.dbUpdate.mockReset().mockResolvedValue( - notOk( - destructiveRefusal({ - destructiveChanges: { - destructiveOperations: [], - databaseName: TOKEN, - planHash: PLAN_HASH, - }, - }), - ), - ); - - const run = await harness().run(['db', 'update', '--json'], { - cwd: projectDir, - isTty: { stdin: true }, - answers: [TOKEN], - }); - - expect(run.exitCode).toBe(2); - expect(envelopeOf(run.json)).toMatchObject({ - ok: false, - error: { code: 'CLI.CONSENT_OPERATIONS_MISSING' }, - }); - expect(applyCalls()).toHaveLength(1); - }); + expect([envelopeOf(confirmed.json), envelopeOf(yes.json)]).toMatchObject([ + { error: { code: 'CLI.CONSENT_REQUIRED' } }, + { error: { code: 'CLI.CONSENT_REQUIRED' } }, + ]); }); - describe('when the applied plan is not the plan consented to', () => { - it('surfaces the control API`s typed mismatch refusal as an error', async () => { - mocks.dbUpdate - .mockReset() - .mockImplementation((options: { consent?: { planHash: string } }) => - Promise.resolve( - options.consent === undefined - ? notOk(destructiveRefusal()) - : notOk(planMismatchFailure()), - ), - ); - - const run = await harness().run(['db', 'update', '--json'], { - cwd: projectDir, - isTty: { stdin: true }, - answers: [TOKEN], - }); - - expect(run.exitCode).toBe(2); - expect(envelopeOf(run.json)).toMatchObject({ - ok: false, - error: { code: 'MIGRATION.CONSENT_PLAN_MISMATCH' }, - }); - expect(applyCalls().map((call) => call.consent)).toEqual([ - undefined, - { planHash: PLAN_HASH }, - ]); - }); + it('refuses a --delete no question asks about', async () => { + mocks.dbUpdate.mockReset().mockImplementation(askThenApply(() => [])); - it('carries the consented plan through the re-run untouched', async () => { - const run = await harness().run(['db', 'update', '--json'], { - cwd: projectDir, - isTty: { stdin: true }, - answers: [TOKEN], - }); - - expect(run.exitCode).toBe(0); - expect(warnTexts(run.events)).toEqual([]); + const run = await harness().run(['db', 'update', '--delete', 'Nope', '--json'], { + cwd: projectDir, }); - }); - describe('--dry-run', () => { - it('never asks, even when the plan call comes back destructive', async () => { - const run = await harness().run(['db', 'update', '--dry-run', '--json'], { - cwd: projectDir, - isTty: { stdin: true }, - answers: [TOKEN], - }); - - expect(run.exitCode).toBe(2); - expect(envelopeOf(run.json)).toMatchObject({ - ok: false, - error: { code: 'MIGRATION.DESTRUCTIVE_CHANGES' }, - }); - expect(mocks.dbUpdate).toHaveBeenCalledWith(expect.objectContaining({ mode: 'plan' })); - expect(applyCalls()).toHaveLength(1); - }); + expect(envelopeOf(run.json)).toMatchObject({ error: { code: 'CLI.CONSENT_UNUSED' } }); }); - describe('when nothing destructive is planned', () => { - beforeEach(() => { - mocks.dbUpdate.mockReset().mockResolvedValue(ok(applySuccess())); - }); - - it('applies without asking, even non-interactively', async () => { - const run = await harness().run(['db', 'update', '--json'], { cwd: projectDir }); + it('asks nothing on a dry run, and hands it the delete and allow flags to check', async () => { + const run = await harness().run( + ['db', 'update', '--dry-run', '--delete', 'Legacy', '--allow', 'User', '--json'], + { cwd: projectDir }, + ); - expect(run.exitCode).toBe(0); - expect(applyCalls()).toHaveLength(1); - expect(applyCalls()[0]?.consent).toBeUndefined(); - expect(run.stderr).not.toContain('confirm'); - }); + expect(run.exitCode).toBe(0); + expect(answers).toEqual([]); + expect(mocks.dbUpdate).toHaveBeenCalledWith( + expect.objectContaining({ + mode: 'plan', + statements: [ + { verb: 'delete', text: 'Legacy' }, + { verb: 'allow', text: 'User' }, + ], + }), + ); + }); - it('plans without asking under --dry-run', async () => { - const run = await harness().run(['db', 'update', '--dry-run', '--json'], { - cwd: projectDir, - }); + it('keeps every statement in the retry command when no connection is configured', async () => { + const run = await harness({ ...ormConfig(), db: undefined }).run( + [ + 'db', + 'update', + '--rename', + 'Profile:User', + '--delete', + 'Legacy', + '--allow', + 'User', + '--json', + ], + { cwd: projectDir }, + ); - expect(run.exitCode).toBe(0); - expect(mocks.dbUpdate).toHaveBeenCalledWith(expect.objectContaining({ mode: 'plan' })); - expect(applyCalls()).toHaveLength(1); + expect(envelopeOf(run.json)).toMatchObject({ + error: { + code: 'CONFIG.DB_CONNECTION_REQUIRED', + nextActions: [ + expect.objectContaining({ + label: expect.stringContaining( + 'prisma-test db update --rename Profile:User --delete Legacy --allow User --db $DATABASE_URL', + ), + }), + ], + }, }); }); }); diff --git a/packages/1-framework/3-tooling/cli/test/orm/db-update-to-resolution.test.ts b/packages/1-framework/3-tooling/cli/test/orm/db-update-to-resolution.test.ts index f43beed9efa6..93d59f703cc4 100644 --- a/packages/1-framework/3-tooling/cli/test/orm/db-update-to-resolution.test.ts +++ b/packages/1-framework/3-tooling/cli/test/orm/db-update-to-resolution.test.ts @@ -265,6 +265,9 @@ describe('db update --to bundle resolution', () => { preview: undefined, }, destination: { storageHash: HASH_B }, + appliedStatements: [], + dataLoss: [], + accessWidening: [], summary: 'Plan ready', }), ); @@ -287,6 +290,9 @@ describe('db update --to bundle resolution', () => { destination: { storageHash: HASH_B }, operations: [{ id: 'table.users', label: 'Create users', operationClass: 'additive' }], }, + appliedStatements: [], + dataLoss: [], + accessWidening: [], advancedRef: null, plannedAdvanceRef: { name: 'db', hash: HASH_B }, summary: 'Plan ready', diff --git a/packages/1-framework/3-tooling/cli/test/orm/db-update.test.ts b/packages/1-framework/3-tooling/cli/test/orm/db-update.test.ts index a40cd9cc2cb8..f9bf2e14f5dd 100644 --- a/packages/1-framework/3-tooling/cli/test/orm/db-update.test.ts +++ b/packages/1-framework/3-tooling/cli/test/orm/db-update.test.ts @@ -97,6 +97,8 @@ function applySuccess(): Record { return { mode: 'apply', appliedStatements: [], + dataLoss: [], + accessWidening: [], destination: { storageHash: DEST_HASH }, plan: { operations: [ @@ -125,6 +127,8 @@ function planSuccess(): Record { return { mode: 'plan', appliedStatements: [], + dataLoss: [], + accessWidening: [], destination: { storageHash: DEST_HASH }, plan: { operations: [{ id: 'op-2', label: 'drop relation legacy', operationClass: 'destructive' }], @@ -206,6 +210,94 @@ describe('db update --rename', () => { expect(run.presented?.data).toMatchObject({ appliedStatements: [PROFILE_TO_USER] }); }); + it('lists what an apply would ask about on a dry run, in its output and its JSON', async () => { + const legacy = { kind: 'model', namespaceId: 'app', model: 'Legacy' } as const; + mocks.dbUpdate.mockResolvedValue( + ok({ ...planSuccess(), dataLoss: [{ operationIndex: 0, subject: legacy, text: 'Legacy' }] }), + ); + const human = await harness().run(['db', 'update', '--dry-run'], { + cwd: projectDir, + isTty: { stdout: true }, + }); + const json = await harness().run(['db', 'update', '--dry-run', '--json'], { cwd: projectDir }); + + expect(human.presented?.presentation.human).toContainEqual({ + kind: 'tree', + roots: [ + { + label: 'An apply asks about', + children: [{ label: 'drop relation legacy: --delete Legacy' }], + }, + ], + }); + expect(json.presented?.data).toMatchObject({ + dataLoss: [{ operationIndex: 0, subject: legacy, text: 'Legacy' }], + accessWidening: [], + }); + }); + + it('marks the questions the flags of a dry run answer, and repeats the flags in the apply it suggests', async () => { + const legacy = { kind: 'model', namespaceId: 'app', model: 'Legacy' } as const; + const audit = { kind: 'storage', name: "audit log's" } as const; + mocks.dbUpdate.mockResolvedValue( + ok({ + ...planSuccess(), + dataLoss: [ + { operationIndex: 0, subject: legacy, text: 'Legacy' }, + { operationIndex: 0, subject: audit, text: "audit log's" }, + ], + }), + ); + const run = await harness().run( + ['db', 'update', '--dry-run', '--rename', 'Profile:User', '--delete', 'Legacy'], + { cwd: projectDir, isTty: { stdout: true } }, + ); + + expect(run.presented?.presentation.human).toContainEqual({ + kind: 'tree', + roots: [ + { + label: 'An apply asks about', + children: [ + { label: 'drop relation legacy: --delete Legacy (answered)' }, + { label: "drop relation legacy: --delete 'audit log'\\''s'" }, + ], + }, + ], + }); + expect(run.presented?.presentation.next).toEqual([ + { + kind: 'run-command', + label: 'Apply the planned operations', + command: 'prisma-test db update --rename Profile:User --delete Legacy', + }, + ]); + }); + + it('suggests the apply with the database as a placeholder and the ref it advances', async () => { + mocks.dbUpdate.mockResolvedValue(ok(planSuccess())); + const run = await harness().run( + [ + 'db', + 'update', + '--dry-run', + '--db', + 'postgres://user:secret@localhost/appdb', + '--advance-ref', + 'qa', + ], + { cwd: projectDir, isTty: { stdout: true } }, + ); + + expect(run.presented?.presentation.next).toEqual([ + { + kind: 'run-command', + label: 'Apply the planned operations', + command: 'prisma-test db update --db --advance-ref qa', + }, + ]); + }); + it('lists the applied statements on a dry run', async () => { mocks.dbUpdate.mockResolvedValue( ok({ ...planSuccess(), appliedStatements: [{ ...PROFILE_TO_USER, operationIndexes: [] }] }), diff --git a/packages/1-framework/3-tooling/cli/test/orm/fixtures/offline-project.ts b/packages/1-framework/3-tooling/cli/test/orm/fixtures/offline-project.ts index cc18987c85a6..90e92c0de560 100644 --- a/packages/1-framework/3-tooling/cli/test/orm/fixtures/offline-project.ts +++ b/packages/1-framework/3-tooling/cli/test/orm/fixtures/offline-project.ts @@ -40,9 +40,30 @@ export async function removeOfflineProjects(): Promise { * An emitted contract. With `models`, it carries an application domain whose * one namespace `app` declares those models, so statements can resolve. */ +/** A model of a fixture contract: its name, or its name and the names of its scalar fields. */ +export type FixtureModel = string | { readonly name: string; readonly fields: readonly string[] }; + +function modelEntry(model: FixtureModel): readonly [string, Record] { + const name = typeof model === 'string' ? model : model.name; + const fields = typeof model === 'string' ? [] : model.fields; + return [ + name, + { + fields: Object.fromEntries( + fields.map((field) => [ + field, + { nullable: false, type: { kind: 'scalar', codecId: 'pg/text@1' } }, + ]), + ), + relations: {}, + storage: {}, + }, + ]; +} + export function contractJson( storageHash: string, - models?: readonly string[], + models?: readonly FixtureModel[], ): Record { return { storage: { storageHash, namespaces: {} }, @@ -56,9 +77,7 @@ export function contractJson( domain: { namespaces: { app: { - models: Object.fromEntries( - models.map((model) => [model, { fields: {}, relations: {}, storage: {} }]), - ), + models: Object.fromEntries(models.map(modelEntry)), }, }, }, @@ -68,7 +87,7 @@ export function contractJson( export async function createOfflineProject(options: { readonly storageHash: string; - readonly models?: readonly string[]; + readonly models?: readonly FixtureModel[]; }): Promise { const dir = createTestProjectDir('orm-offline'); created.push(dir); @@ -143,7 +162,7 @@ export async function seedMigrationPackage(options: { export async function seedContractSnapshot(options: { readonly migrationsDir: string; readonly storageHash: string; - readonly models?: readonly string[]; + readonly models?: readonly FixtureModel[]; }): Promise { await writeContractSnapshot(options.migrationsDir, options.storageHash, { contractJson: contractJson(options.storageHash, options.models), @@ -180,6 +199,32 @@ export interface FakePlannerScript { readonly statementsReceived?: unknown[][]; /** Fails every `plan` call that is given statements, as a planner that refuses them does. */ readonly refuseStatements?: boolean; + /** What every `plan` call reports would lose data. */ + readonly dataLoss?: readonly ScriptedDataLoss[]; + /** What each successive `plan` call reports would lose data; overrides `dataLoss`. */ + readonly dataLossByPlan?: ReadonlyArray; + /** Reports `dataLoss` only from a `plan` call given no statements, as a rename removes a loss. */ + readonly statementsResolveDataLoss?: boolean; + /** Reports `dataLoss` only from a `plan` call given fewer statements than this. */ + readonly statementsResolvingDataLoss?: number; + /** Where in the operations the placeholder that `throwOnOperations` rejects sits; last by default. */ + readonly placeholderAt?: number; + /** Makes the plan's `operations` accessor throw this synchronously. */ + readonly operationsAccessorThrows?: unknown; +} + +/** A `dataLoss` entry of a scripted plan: the position of an operation and what it loses. */ +export interface ScriptedDataLoss { + readonly operationIndex: number; + readonly subject: + | { readonly kind: 'model'; readonly namespaceId: string; readonly model: string } + | { + readonly kind: 'field'; + readonly namespaceId: string; + readonly model: string; + readonly field: string; + } + | { readonly kind: 'storage'; readonly name: string }; } function fakePlanner(script: FakePlannerScript): Record { @@ -191,6 +236,12 @@ function fakePlanner(script: FakePlannerScript): Record { throw script.throwOnPlan; } const operations = script.operationsByPlan?.[planCalls] ?? script.operations; + const dataLoss = + (script.statementsResolveDataLoss === true && options.statements.length > 0) || + options.statements.length >= + (script.statementsResolvingDataLoss ?? Number.POSITIVE_INFINITY) + ? [] + : (script.dataLossByPlan?.[planCalls] ?? script.dataLoss ?? []); planCalls += 1; const [refused] = script.refuseStatements === true ? options.statements : []; if (refused !== undefined) { @@ -202,18 +253,26 @@ function fakePlanner(script: FakePlannerScript): Record { return script.conflicts === undefined ? { kind: 'success', + dataLoss, + accessWidening: [], appliedStatements: options.statements.map((statement) => ({ statement, operationIndexes: (operations ?? [ADDITIVE_OP]).map((_, index) => index), })), plan: { - operations: - script.throwOnOperations === undefined - ? (operations ?? [ADDITIVE_OP]).map((op) => Promise.resolve(op)) - : [ - ...(operations ?? []).map((op) => Promise.resolve(op)), - Promise.reject(script.throwOnOperations), - ], + get operations() { + if (script.operationsAccessorThrows !== undefined) { + throw script.operationsAccessorThrows; + } + if (script.throwOnOperations === undefined) { + return (operations ?? [ADDITIVE_OP]).map((op) => Promise.resolve(op)); + } + const resolved = (operations ?? []).map((op) => Promise.resolve(op)); + const placeholder = Promise.reject(script.throwOnOperations); + placeholder.catch(() => undefined); + const at = script.placeholderAt ?? resolved.length; + return [...resolved.slice(0, at), placeholder, ...resolved.slice(at)]; + }, renderTypeScript: () => '// planned migration\n', }, } diff --git a/packages/1-framework/3-tooling/cli/test/orm/init-inputs.test.ts b/packages/1-framework/3-tooling/cli/test/orm/init-inputs.test.ts index e0ff35f52152..6f19681a893a 100644 --- a/packages/1-framework/3-tooling/cli/test/orm/init-inputs.test.ts +++ b/packages/1-framework/3-tooling/cli/test/orm/init-inputs.test.ts @@ -49,6 +49,12 @@ function recordingPrompt(): { readonly prompt: PromptSurface; readonly textCalls return opts?.default ?? ''; }, browserWait: async () => undefined, + statement: async () => { + throw new Error('unexpected statement prompt'); + }, + statements: async () => { + throw new Error('unexpected statement prompt'); + }, }; return { prompt, textCalls }; } diff --git a/packages/1-framework/3-tooling/cli/test/orm/init-prisma7-fixtures.ts b/packages/1-framework/3-tooling/cli/test/orm/init-prisma7-fixtures.ts index 52dd4b2e8bac..baaaab0aed38 100644 --- a/packages/1-framework/3-tooling/cli/test/orm/init-prisma7-fixtures.ts +++ b/packages/1-framework/3-tooling/cli/test/orm/init-prisma7-fixtures.ts @@ -72,6 +72,12 @@ export function scriptedPrompt(answers: Record = {}): { return opts?.default ?? ''; }, browserWait: async () => undefined, + statement: async () => { + throw new Error('unexpected statement prompt'); + }, + statements: async () => { + throw new Error('unexpected statement prompt'); + }, }; return { prompt, calls }; } diff --git a/packages/1-framework/3-tooling/cli/test/orm/migration-plan.test.ts b/packages/1-framework/3-tooling/cli/test/orm/migration-plan.test.ts index 76d752f1c243..231205172693 100644 --- a/packages/1-framework/3-tooling/cli/test/orm/migration-plan.test.ts +++ b/packages/1-framework/3-tooling/cli/test/orm/migration-plan.test.ts @@ -1,5 +1,6 @@ import { existsSync } from 'node:fs'; -import { readdir, readFile } from 'node:fs/promises'; +import { readdir, readFile, rm } from 'node:fs/promises'; +import type { PrismaNextConfig } from '@internal/config/config-types'; import { contractSnapshotDir } from '@internal/migration-tools/contract-snapshot-store'; import { computeMigrationHash } from '@internal/migration-tools/hash'; import { writeRef } from '@internal/migration-tools/refs'; @@ -8,6 +9,9 @@ import { structuredError } from '@internal/utils/structured-error'; import { createTestCli } from '@prisma/cli-engine/testing'; import { basename, dirname, join } from 'pathe'; import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { executeMigrationPlanCommand } from '../../src/control-api/operations/migration-plan'; +import type { AnswerPlanQuestions } from '../../src/control-api/statements/plan-questions'; +import type { StatementText } from '../../src/control-api/statements/statement-text'; import { BIN_GROUPS } from '../../src/orm/cli'; import { errorUnfilledPlaceholder } from '../../src/utils/cli-errors'; import { createOrmTestCli } from '../helpers/orm-test-cli'; @@ -422,164 +426,506 @@ describe('migration plan', () => { }); }); - describe('auto-baseline consent', () => { - /** An empty graph whose db ref demands a destructive baseline. */ - async function destructiveBaselineProject(): Promise { - const project = await createOfflineProject({ storageHash: HASH_TO }); - await seedContractSnapshot({ migrationsDir: project.migrationsDir, storageHash: HASH_FROM }); + describe('data loss', () => { + const LEGACY_LOSS = { + operationIndex: 1, + subject: { kind: 'model', namespaceId: 'app', model: 'Legacy' }, + } as const; + const AUDIT_LOSS = { + operationIndex: 2, + subject: { kind: 'storage', name: 'audit_log' }, + } as const; + const DROP_AUDIT_OP = { + ...DESTRUCTIVE_OP, + id: 'table.drop_audit', + label: 'Drop table "audit_log"', + }; + const losingScript: FakePlannerScript = { + operations: [ADDITIVE_OP, DESTRUCTIVE_OP], + dataLoss: [LEGACY_LOSS], + }; + + /** A project whose database sits at HASH_FROM, with models Legacy and User, and whose contract is HASH_TO. */ + async function losingProject(): Promise { + const project = await createOfflineProject({ + storageHash: HASH_TO, + models: ['User', 'Archive'], + }); + await seedMigrationPackage({ + appMigrationsDir: project.appMigrationsDir, + dirName: '20260101T0000_initial', + from: null, + to: HASH_FROM, + }); + await seedContractSnapshot({ + migrationsDir: project.migrationsDir, + storageHash: HASH_FROM, + models: ['User', 'Legacy'], + }); await seedDbRef({ appMigrationsDir: project.appMigrationsDir, storageHash: HASH_FROM }); return project; } - const destructiveScript = { operations: [ADDITIVE_OP, DESTRUCTIVE_OP] } as const; - it('refuses non-interactively without --confirm and writes nothing', async () => { - const project = await destructiveBaselineProject(); + /** An empty graph whose db ref demands a baseline, then a delta that loses data. */ + async function losingBaselineProject(): Promise { + const project = await createOfflineProject({ + storageHash: HASH_TO, + models: ['User', 'Archive'], + }); + await seedContractSnapshot({ + migrationsDir: project.migrationsDir, + storageHash: HASH_FROM, + models: ['User', 'Legacy'], + }); + await seedDbRef({ appMigrationsDir: project.appMigrationsDir, storageHash: HASH_FROM }); + return project; + } - const run = await harness(project, { script: destructiveScript }).run( - ['migration', 'plan', '--json'], - { cwd: project.dir }, + function envelopeOf(run: { readonly json: readonly unknown[] }) { + const terminal = run.json.at(-1) as + | { readonly kind: string; readonly envelope?: unknown } + | undefined; + return terminal?.kind === 'result' ? terminal.envelope : undefined; + } + + function planThroughControlApi( + project: OfflineProject, + statements: readonly StatementText[], + answerQuestions: AnswerPlanQuestions, + ) { + return executeMigrationPlanCommand( + { + config: { + ...offlineConfig({ + project, + script: { + operations: [ADDITIVE_OP, DESTRUCTIVE_OP, DROP_AUDIT_OP], + dataLoss: [LEGACY_LOSS, AUDIT_LOSS], + }, + }), + baseDir: project.dir, + } as unknown as PrismaNextConfig, + cwd: project.dir, + projectDir: project.dir, + statements, + answerQuestions, + client: { renderContractDts: renderContractDtsMock }, + }, + Date.now(), ); + } + + it('takes delete statements given to the control API as answers', async () => { + const project = await losingProject(); + const answerQuestions = vi.fn(async () => []); + + const result = await planThroughControlApi( + project, + [ + { verb: 'delete', text: 'Legacy' }, + { verb: 'delete', text: 'audit_log' }, + ], + answerQuestions, + ); + + expect(result.ok && result.value.appliedStatements).toMatchObject([ + { verb: 'delete', description: 'delete model "Legacy"' }, + { verb: 'delete', description: 'delete storage "audit_log"' }, + ]); + expect(answerQuestions).toHaveBeenCalledWith([]); + }); + + it('refuses delete and allow statements given to the control API that answer no question', async () => { + const project = await losingProject(); + + const result = await planThroughControlApi( + project, + [ + { verb: 'delete', text: 'Legacy' }, + { verb: 'delete', text: 'audit_log' }, + { verb: 'delete', text: 'Nope' }, + { verb: 'allow', text: 'User' }, + ], + async () => [], + ); + + expect(!result.ok && result.failure.toEnvelope()).toMatchObject({ + code: 'MIGRATION.STATEMENT_ANSWERS_NO_QUESTION', + meta: { + statements: [ + { verb: 'delete', text: 'Nope' }, + { verb: 'allow', text: 'User' }, + ], + }, + }); + expect(await plannedDirs(project)).toEqual(['20260101T0000_initial']); + }); + + it('refuses where nobody can answer, listing every operation with the flags that answer it', async () => { + const project = await losingProject(); + + const run = await harness(project, { + script: { + operations: [ADDITIVE_OP, DESTRUCTIVE_OP, DROP_AUDIT_OP], + dataLoss: [LEGACY_LOSS, AUDIT_LOSS], + }, + }).run(['migration', 'plan', '--json'], { cwd: project.dir }); expect(run.exitCode).toBe(2); - const terminal = run.json.at(-1); - const envelope = - terminal !== undefined && terminal.kind === 'result' ? terminal.envelope : undefined; - expect(envelope).toMatchObject({ - ok: false, - error: { code: 'CLI.CONSENT_REQUIRED', meta: { consentToken: basename(project.dir) } }, + const envelope = envelopeOf(run) as { + readonly error: { readonly code: string; readonly nextActions: readonly unknown[] }; + }; + expect(envelope.error.code).toBe('CLI.CONSENT_REQUIRED'); + const actions = JSON.stringify(envelope.error.nextActions); + expect(actions).toContain('--delete Legacy'); + expect(actions).toContain("--rename 'Legacy:'"); + expect(actions).toContain('--delete audit_log'); + expect(actions).not.toContain('--rename audit_log'); + expect(JSON.stringify(envelope)).toContain( + 'Drop table \\"legacy\\" would lose the data of model \\"Legacy\\".', + ); + expect(await plannedDirs(project)).toEqual(['20260101T0000_initial']); + }); + + it('writes the plan once --delete names the subject, and reports the statement', async () => { + const project = await losingProject(); + + const run = await harness(project, { script: losingScript }).run( + ['migration', 'plan', '--delete', 'Legacy', '--json'], + { cwd: project.dir }, + ); + + expect(run.exitCode).toBe(0); + expect(run.presented?.data).toMatchObject({ + appliedStatements: [ + { + verb: 'delete', + statement: { + kind: 'delete', + subject: { kind: 'model', namespaceId: 'app', model: 'Legacy' }, + }, + operationIndexes: [1], + description: 'delete model "Legacy"', + }, + ], }); - expect(await plannedDirs(project)).toEqual([]); + expect((await plannedDirs(project)).length).toBe(2); }); - it('names every destructive operation in the question', async () => { - const project = await destructiveBaselineProject(); + it('lists the delete under Statements applied, and keeps the marker on the operation', async () => { + const project = await losingProject(); + + const run = await harness(project, { script: losingScript }).run( + ['migration', 'plan', '--delete', 'Legacy'], + { cwd: project.dir, isTty: { stdout: true } }, + ); - const run = await harness(project, { script: destructiveScript }).run(['migration', 'plan'], { + expect(run.presented?.presentation.human).toContainEqual({ + kind: 'tree', + roots: [ + { + label: 'Statements applied', + children: [{ label: 'delete model "Legacy" (1 operation)' }], + }, + ], + }); + expect(JSON.stringify(run.presented?.presentation.human)).toContain( + '{"label":"Drop table \\"legacy\\"","status":"warn"}', + ); + }); + + it('plans a --rename from the flag in the first plan, which then loses nothing', async () => { + const project = await losingProject(); + const statementsReceived: unknown[][] = []; + + const run = await harness(project, { + script: { ...losingScript, statementsResolveDataLoss: true, statementsReceived }, + }).run(['migration', 'plan', '--rename', 'Legacy:Archive', '--json'], { cwd: project.dir }); + + expect(run.exitCode).toBe(0); + expect(statementsReceived).toEqual([ + [ + { + kind: 'rename', + entity: 'model', + from: { namespaceId: 'app', model: 'Legacy' }, + to: { namespaceId: 'app', model: 'Archive' }, + }, + ], + ]); + }); + + it('plans again with a rename typed at the prompt, and writes the plan', async () => { + const project = await losingProject(); + const statementsReceived: unknown[][] = []; + + const run = await harness(project, { + script: { ...losingScript, statementsResolveDataLoss: true, statementsReceived }, + }).run(['migration', 'plan', '--json'], { cwd: project.dir, - isTty: { stdin: true, stdout: true, stderr: true }, - stdin: `${basename(project.dir)}\n`, + isTty: { stdin: true }, + answers: ['rename Legacy:Archive'], }); - expect(run.stderr).toContain('Drop table "legacy"'); + expect(run.exitCode).toBe(0); + expect(statementsReceived.map((statements) => statements.length)).toEqual([0, 1]); + expect(run.presented?.data).toMatchObject({ + appliedStatements: [{ verb: 'rename', description: 'rename model "Legacy" to "Archive"' }], + }); + expect((await plannedDirs(project)).length).toBe(2); }); - it('writes the baseline once consent is typed', async () => { - const project = await destructiveBaselineProject(); + it('fails when a typed rename does not remove the loss it answered', async () => { + const project = await losingProject(); - const run = await harness(project, { script: destructiveScript }).run( - ['migration', 'plan', '--name', 'delta', '--json'], - { cwd: project.dir, isTty: { stdin: true }, answers: [basename(project.dir)] }, + const run = await harness(project, { script: losingScript }).run( + ['migration', 'plan', '--json'], + { + cwd: project.dir, + isTty: { stdin: true }, + answers: ['rename Legacy:Archive'], + }, ); - const dirs = await plannedDirs(project); - expect(run.exitCode).toBe(0); - expect(dirs.map((entry) => entry.replace(/^\d+T\d+_/, ''))).toEqual(['baseline', 'delta']); + expect(run.exitCode).not.toBe(0); + expect(envelopeOf(run)).toMatchObject({ + error: { + code: 'MIGRATION.STATEMENT_DID_NOT_RESOLVE_LOSS', + meta: { statement: 'Legacy:Archive', subject: 'Legacy' }, + }, + }); + expect(await plannedDirs(project)).toEqual(['20260101T0000_initial']); }); - it('writes the baseline when --confirm carries the project directory name', async () => { - const project = await destructiveBaselineProject(); + it('writes the plan when delete is typed at the prompt', async () => { + const project = await losingProject(); - const run = await harness(project, { script: destructiveScript }).run( - ['migration', 'plan', '--confirm', basename(project.dir), '--json'], - { cwd: project.dir }, + const run = await harness(project, { script: losingScript }).run( + ['migration', 'plan', '--json'], + { + cwd: project.dir, + isTty: { stdin: true }, + answers: ['delete'], + }, ); expect(run.exitCode).toBe(0); - expect((await plannedDirs(project)).length).toBe(2); + expect(run.presented?.data).toMatchObject({ + appliedStatements: [{ verb: 'delete', description: 'delete model "Legacy"' }], + }); }); - it('still asks when the destructive baseline also carries a placeholder', async () => { - const project = await destructiveBaselineProject(); + it('fails on a typed answer the question rejects, with scripted input', async () => { + const project = await losingProject(); - const run = await harness(project, { - script: { - operations: [ADDITIVE_OP, DESTRUCTIVE_OP], - throwOnOperations: errorUnfilledPlaceholder('backfill'), + const run = await harness(project, { script: losingScript }).run( + ['migration', 'plan', '--json'], + { + cwd: project.dir, + isTty: { stdin: true }, + answers: ['delete Nope'], }, - }).run(['migration', 'plan', '--json'], { cwd: project.dir }); + ); - expect(run.exitCode).toBe(2); - const terminal = run.json.at(-1); - const envelope = - terminal !== undefined && terminal.kind === 'result' ? terminal.envelope : undefined; - expect(envelope).toMatchObject({ ok: false, error: { code: 'CLI.CONSENT_REQUIRED' } }); + expect(envelopeOf(run)).toMatchObject({ error: { code: 'CLI.PROMPT_INVALID' } }); + expect(await plannedDirs(project)).toEqual(['20260101T0000_initial']); + }); + + it('refuses a --delete nothing asked for before writing anything', async () => { + const project = await losingProject(); + + const run = await harness(project).run(['migration', 'plan', '--delete', 'Nope', '--json'], { + cwd: project.dir, + }); + + expect(envelopeOf(run)).toMatchObject({ error: { code: 'CLI.CONSENT_UNUSED' } }); + expect(await plannedDirs(project)).toEqual(['20260101T0000_initial']); + }); + + it('does not take --yes as an answer', async () => { + const project = await losingProject(); + + const run = await harness(project, { script: losingScript }).run( + ['migration', 'plan', '--yes', '--json'], + { cwd: project.dir, isTty: { stdin: true } }, + ); + + expect(envelopeOf(run)).toMatchObject({ error: { code: 'CLI.CONSENT_REQUIRED' } }); + expect(await plannedDirs(project)).toEqual(['20260101T0000_initial']); + }); + + it('does not take --confirm as consent', async () => { + const project = await losingProject(); + + const run = await harness(project, { script: losingScript }).run( + ['migration', 'plan', '--confirm', basename(project.dir), '--json'], + { cwd: project.dir }, + ); + + expect(envelopeOf(run)).toMatchObject({ error: { code: 'CLI.CONSENT_REQUIRED' } }); + expect(await plannedDirs(project)).toEqual(['20260101T0000_initial']); + }); + + it('asks before writing either package of an auto-baseline whose delta loses data', async () => { + const project = await losingBaselineProject(); + const script: FakePlannerScript = { + operationsByPlan: [[ADDITIVE_OP], [ADDITIVE_OP, DESTRUCTIVE_OP]], + dataLossByPlan: [[], [LEGACY_LOSS]], + }; + + const refused = await harness(project, { script }).run(['migration', 'plan', '--json'], { + cwd: project.dir, + }); + expect(envelopeOf(refused)).toMatchObject({ error: { code: 'CLI.CONSENT_REQUIRED' } }); expect(await plannedDirs(project)).toEqual([]); + + const answered = await harness(project, { script }).run( + ['migration', 'plan', '--delete', 'Legacy', '--json'], + { cwd: project.dir }, + ); + expect(answered.exitCode).toBe(0); + expect((await plannedDirs(project)).length).toBe(2); }); - it('writes the placeholder baseline once --confirm grants consent', async () => { - const project = await destructiveBaselineProject(); + it('writes a placeholder plan once its loss is answered, positioning the loss among the listed operations', async () => { + const project = await losingBaselineProject(); const run = await harness(project, { script: { operations: [ADDITIVE_OP, DESTRUCTIVE_OP], throwOnOperations: errorUnfilledPlaceholder('backfill'), + placeholderAt: 1, + dataLoss: [{ ...LEGACY_LOSS, operationIndex: 2 }], }, - }).run(['migration', 'plan', '--confirm', basename(project.dir), '--json'], { - cwd: project.dir, - }); + }).run(['migration', 'plan', '--delete', 'Legacy', '--json'], { cwd: project.dir }); expect(run.exitCode).toBe(0); - expect(run.presented?.data).toMatchObject({ pendingPlaceholders: true }); + const data = run.presented?.data as { + readonly pendingPlaceholders: boolean; + readonly operations: readonly { readonly id: string }[]; + readonly appliedStatements: readonly { readonly operationIndexes: readonly number[] }[]; + }; + expect(data.pendingPlaceholders).toBe(true); + expect( + data.appliedStatements.map(({ operationIndexes }) => + operationIndexes.map((index) => data.operations[index]?.id), + ), + ).toEqual([[DESTRUCTIVE_OP.id]]); expect((await plannedDirs(project)).length).toBe(2); }); - it('reports extension dirs the refused first run seeded once consent is granted', async () => { - const EXT_HASH = `f00d${'5'.repeat(60)}`; - const extMetadataBase = { - from: null, - to: EXT_HASH, - providedInvariants: [], - createdAt: '2026-01-01T00:00:00.000Z', + it('still asks about a loss when the operations accessor throws on a placeholder', async () => { + const project = await losingProject(); + const script: FakePlannerScript = { + operations: [ADDITIVE_OP, DESTRUCTIVE_OP], + operationsAccessorThrows: errorUnfilledPlaceholder('backfill'), + dataLoss: [LEGACY_LOSS], }; - const project = await destructiveBaselineProject(); + + const refused = await harness(project, { script }).run(['migration', 'plan', '--json'], { + cwd: project.dir, + }); + expect(envelopeOf(refused)).toMatchObject({ error: { code: 'CLI.CONSENT_REQUIRED' } }); + + const answered = await harness(project, { script }).run( + ['migration', 'plan', '--delete', 'Legacy', '--json'], + { cwd: project.dir }, + ); + expect(answered.exitCode).toBe(0); + expect(answered.presented?.data).toMatchObject({ pendingPlaceholders: true }); + }); + + it('asks a question a re-plan brings up in a second batch', async () => { + const project = await losingProject(); const run = await harness(project, { - script: destructiveScript, - overrides: { - extensions: [ - { - kind: 'extension', - id: 'cipherstash', - familyId: 'sql', - targetId: 'postgres', - version: '1.0.0', - create: () => ({}), - contractSpace: { - contractJson: contractJson(EXT_HASH), - headRef: { hash: EXT_HASH, invariants: [] }, - migrations: [ - { - dirName: '0001_seed', - metadata: { - ...extMetadataBase, - migrationHash: computeMigrationHash(extMetadataBase, []), - }, - ops: [], - }, - ], - }, - }, - ], + script: { + operations: [ADDITIVE_OP, DESTRUCTIVE_OP, DROP_AUDIT_OP], + dataLossByPlan: [[LEGACY_LOSS], [AUDIT_LOSS]], }, - }).run(['migration', 'plan', '--confirm', basename(project.dir), '--json'], { + }).run(['migration', 'plan', '--json'], { cwd: project.dir, + isTty: { stdin: true }, + answers: ['rename Legacy:Archive', 'delete'], }); expect(run.exitCode).toBe(0); expect(run.presented?.data).toMatchObject({ - emittedExtensionDirs: [{ spaceId: 'cipherstash', dirName: '0001_seed' }], + appliedStatements: [ + { verb: 'rename', description: 'rename model "Legacy" to "Archive"' }, + { verb: 'delete', description: 'delete storage "audit_log"', operationIndexes: [2] }, + ], }); }); - it('never asks when the baseline is purely additive', async () => { - const project = await destructiveBaselineProject(); - - const run = await harness(project).run(['migration', 'plan', '--json'], { - cwd: project.dir, + it('names a field of a renamed model through its new name, and takes a rename written that way', async () => { + const project = await createOfflineProject({ + storageHash: HASH_TO, + models: [{ name: 'User', fields: ['id', 'handle'] }], }); + await seedMigrationPackage({ + appMigrationsDir: project.appMigrationsDir, + dirName: '20260101T0000_initial', + from: null, + to: HASH_FROM, + }); + await seedContractSnapshot({ + migrationsDir: project.migrationsDir, + storageHash: HASH_FROM, + models: [{ name: 'Profile', fields: ['id', 'nickname'] }], + }); + await seedDbRef({ appMigrationsDir: project.appMigrationsDir, storageHash: HASH_FROM }); + const script: FakePlannerScript = { + operations: [ADDITIVE_OP, DESTRUCTIVE_OP], + dataLoss: [ + { + operationIndex: 1, + subject: { kind: 'field', namespaceId: 'app', model: 'Profile', field: 'nickname' }, + }, + ], + statementsResolvingDataLoss: 2, + }; - expect(run.exitCode).toBe(0); - expect((await plannedDirs(project)).length).toBe(2); + const refused = await harness(project, { script }).run( + ['migration', 'plan', '--rename', 'Profile:User', '--json'], + { cwd: project.dir }, + ); + const actions = JSON.stringify( + (envelopeOf(refused) as { readonly error: { readonly nextActions: unknown } }).error + .nextActions, + ); + expect(actions).toContain('--delete User.nickname'); + expect(actions).toContain("--rename 'User.nickname:User.'"); + + const deleted = await harness(project, { script }).run( + ['migration', 'plan', '--rename', 'Profile:User', '--delete', 'User.nickname', '--json'], + { cwd: project.dir }, + ); + expect(deleted.presented?.data).toMatchObject({ + appliedStatements: [ + { verb: 'rename' }, + { verb: 'delete', description: 'delete field "User.nickname"' }, + ], + }); + for (const dir of await plannedDirs(project)) { + if (dir !== '20260101T0000_initial') { + await rm(join(project.appMigrationsDir, dir), { recursive: true }); + } + } + + const renamed = await harness(project, { script }).run( + [ + 'migration', + 'plan', + '--rename', + 'Profile:User', + '--rename', + 'User.nickname:User.handle', + '--json', + ], + { cwd: project.dir }, + ); + expect(renamed.exitCode).toBe(0); }); }); @@ -590,7 +936,7 @@ describe('migration plan', () => { const run = await harness(project, { script: { operations: [ADDITIVE_OP, DESTRUCTIVE_OP] }, - }).run(['migration', 'plan', '--confirm', basename(project.dir)], { + }).run(['migration', 'plan'], { cwd: project.dir, isTty: { stdout: true }, }); diff --git a/packages/1-framework/3-tooling/config-loader/package.json b/packages/1-framework/3-tooling/config-loader/package.json index eeb6ec44e2d3..1d6b83be86a0 100644 --- a/packages/1-framework/3-tooling/config-loader/package.json +++ b/packages/1-framework/3-tooling/config-loader/package.json @@ -26,7 +26,7 @@ "tinyglobby": "catalog:" }, "devDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/test-utils": "workspace:8.0.0-rc.16", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@repo/tsdown": "workspace:8.0.0-rc.16", @@ -36,7 +36,7 @@ "vitest": "catalog:" }, "peerDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "typescript": ">=5.9" }, "peerDependenciesMeta": { diff --git a/packages/1-framework/3-tooling/migration/src/aggregate/planner-types.ts b/packages/1-framework/3-tooling/migration/src/aggregate/planner-types.ts index f8b7c5459291..f31af6face40 100644 --- a/packages/1-framework/3-tooling/migration/src/aggregate/planner-types.ts +++ b/packages/1-framework/3-tooling/migration/src/aggregate/planner-types.ts @@ -8,6 +8,7 @@ import type { MigrationPlan, MigrationPlannerConflict, MigrationPlanOperation, + MigrationPlanSubjects, ResolvedMigrationStatement, TargetMigrationsCapability, } from '@internal/framework-components/control'; @@ -93,6 +94,8 @@ export interface PlannerInput string; } /** See {@link PlannerInput.appSpace}. */ @@ -101,24 +104,6 @@ export interface AppSpacePlanningInputs { readonly statements: readonly ResolvedMigrationStatement[]; } -/** - * Per-space output of the planner. The runner ingests this - * shape directly via a thin `toRunnerInput` adapter at the CLI. - * - * - `plan`: ready-to-execute `MigrationPlan` with `targetId` already - * set from `aggregate.targetId`. - * - `displayOps`: same operation list, surfaced separately so plan-mode - * output can render without touching the runner-bound `plan`. - * - `destinationContract`: the typed contract value the runner uses - * for post-apply verification. For the app space, the user's - * contract; for extension spaces, the on-disk `contract.json`. - * - `strategy`: which operation produced this plan — `'resolve-recorded-path'` - * (walked a recorded migration path), `'plan-from-diff'` (fabricated a - * plan from a state diff), or `'declared-state'` (the space ships no - * migration packages at all; the aggregate declares the no-op directly - * without invoking either). Surfaced for diagnostics; not consumed by - * the runner. - */ /** * Per-edge metadata for the chain assembled by resolving a contract * space's recorded migration path. Lets `migrate` surface a per-migration @@ -147,7 +132,29 @@ export interface AggregateMigrationEdgeRef { readonly destinationContractJson?: unknown; } -export interface PerSpacePlan { +/** + * Per-space output of the planner. The runner ingests this + * shape directly via a thin `toRunnerInput` adapter at the CLI. + * + * - `plan`: ready-to-execute `MigrationPlan` with `targetId` already + * set from `aggregate.targetId`. + * - `displayOps`: same operation list, surfaced separately so plan-mode + * output can render without touching the runner-bound `plan`. + * - `destinationContract`: the typed contract value the runner uses + * for post-apply verification. For the app space, the user's + * contract; for extension spaces, the on-disk `contract.json`. + * - `strategy`: which operation produced this plan — `'resolve-recorded-path'` + * (walked a recorded migration path), `'plan-from-diff'` (fabricated a + * plan from a state diff), or `'declared-state'` (the space ships no + * migration packages at all; the aggregate declares the no-op directly + * without invoking either). Surfaced for diagnostics; not consumed by + * the runner. + * - `dataLoss` and `accessWidening`: what the operations of `plan` lose + * and whose access they widen. A recorded path names each destructive + * operation by its storage name, since no planner mapped it to a model, + * and lists no widening: a written migration is reviewed before it runs. + */ +export interface PerSpacePlan extends MigrationPlanSubjects { readonly plan: MigrationPlan; readonly displayOps: readonly MigrationPlanOperation[]; readonly destinationContract: Contract; diff --git a/packages/1-framework/3-tooling/migration/src/aggregate/planner.ts b/packages/1-framework/3-tooling/migration/src/aggregate/planner.ts index 307e448c8b79..efeed19e76c8 100644 --- a/packages/1-framework/3-tooling/migration/src/aggregate/planner.ts +++ b/packages/1-framework/3-tooling/migration/src/aggregate/planner.ts @@ -113,6 +113,7 @@ export async function planMigration 0 ? { warnings: plannerResult.warnings } : {}), diff --git a/packages/1-framework/3-tooling/migration/src/aggregate/strategies/resolve-recorded-path.ts b/packages/1-framework/3-tooling/migration/src/aggregate/strategies/resolve-recorded-path.ts index 22e82a44437e..30eedb1a947e 100644 --- a/packages/1-framework/3-tooling/migration/src/aggregate/strategies/resolve-recorded-path.ts +++ b/packages/1-framework/3-tooling/migration/src/aggregate/strategies/resolve-recorded-path.ts @@ -1,4 +1,4 @@ -import type { MigrationPlan } from '@internal/framework-components/control'; +import type { MigrationPlan, MigrationPlanOperation } from '@internal/framework-components/control'; import { InternalError } from '@internal/utils/internal-error'; import { EMPTY_CONTRACT_HASH } from '../../constants'; import { findPathWithDecision } from '../../migration-graph'; @@ -33,6 +33,8 @@ export interface ResolveRecordedPathInputs { * strategy itself does not interpret it. */ readonly refName?: string; + /** The name the database knows the object an operation acts on by; names what a drop loses. */ + readonly storageNameOf: (operation: MigrationPlanOperation) => string; } /** @@ -115,6 +117,17 @@ export function resolveRecordedPath(input: ResolveRecordedPathInputs): ResolveRe destinationContract: space.contract(), strategy: 'resolve-recorded-path', appliedStatements: [], + dataLoss: pathOps.flatMap((operation, operationIndex) => + operation.operationClass === 'destructive' + ? [ + { + operationIndex, + subject: { kind: 'storage' as const, name: input.storageNameOf(operation) }, + }, + ] + : [], + ), + accessWidening: [], migrationEdges: edgeRefs, pathDecision: outcome.decision, }, diff --git a/packages/1-framework/3-tooling/migration/test/aggregate/planner.test.ts b/packages/1-framework/3-tooling/migration/test/aggregate/planner.test.ts index 6950201333a4..91f5fa74e68b 100644 --- a/packages/1-framework/3-tooling/migration/test/aggregate/planner.test.ts +++ b/packages/1-framework/3-tooling/migration/test/aggregate/planner.test.ts @@ -96,7 +96,13 @@ describe('planMigration', () => { app: makeSpace({ spaceId: 'app' }), }); const stubPlan = makeSyntheticPlan('placeholder-target-id-from-stub'); - const planner = makeStubPlanner({ kind: 'success', plan: stubPlan, appliedStatements: [] }); + const planner = makeStubPlanner({ + kind: 'success', + plan: stubPlan, + appliedStatements: [], + dataLoss: [], + accessWidening: [], + }); const result = await planMigration({ aggregate, @@ -110,6 +116,7 @@ describe('planMigration', () => { callerPolicy: { ignoreGraphFor: new Set(['app']) }, operationPolicy: POLICY, appSpace: { fromContract: null, statements: [] }, + storageNameOf: (operation) => operation.id, }); expect(result.ok).toBe(true); @@ -140,6 +147,8 @@ describe('planMigration', () => { kind: 'success', plan: makeSyntheticPlan('postgres'), appliedStatements: [applied], + dataLoss: [], + accessWidening: [], }; }, emptyMigration: () => { @@ -156,6 +165,7 @@ describe('planMigration', () => { callerPolicy: { ignoreGraphFor: new Set(['app']) }, operationPolicy: POLICY, appSpace: { fromContract: origin, statements: [statement] }, + storageNameOf: (operation) => operation.id, }); expect(received).toEqual({ fromContract: origin, statements: [statement] }); @@ -174,7 +184,13 @@ describe('planMigration', () => { const planner: MigrationPlanner<'sql', 'postgres'> = { plan: () => { planned = true; - return { kind: 'success', plan: makeSyntheticPlan('postgres'), appliedStatements: [] }; + return { + kind: 'success', + plan: makeSyntheticPlan('postgres'), + appliedStatements: [], + dataLoss: [], + accessWidening: [], + }; }, emptyMigration: () => { throw new Error('not used'); @@ -190,6 +206,7 @@ describe('planMigration', () => { callerPolicy: { ignoreGraphFor: new Set() }, operationPolicy: POLICY, appSpace: { fromContract: null, statements: [statement] }, + storageNameOf: (operation) => operation.id, }); expect(result.assertNotOk()).toEqual({ @@ -220,7 +237,13 @@ describe('planMigration', () => { fromContract: options.fromContract, statements: options.statements, }); - return { kind: 'success', plan: makeSyntheticPlan('postgres'), appliedStatements: [] }; + return { + kind: 'success', + plan: makeSyntheticPlan('postgres'), + appliedStatements: [], + dataLoss: [], + accessWidening: [], + }; }, emptyMigration: () => { throw new Error('not used'); @@ -236,6 +259,7 @@ describe('planMigration', () => { callerPolicy: { ignoreGraphFor: new Set(['app', 'cipherstash']) }, operationPolicy: POLICY, appSpace: { fromContract: origin, statements: [statement] }, + storageNameOf: (operation) => operation.id, }); expect(result.ok).toBe(true); @@ -259,7 +283,13 @@ describe('planMigration', () => { }); const stubPlan = makeSyntheticPlan('postgres'); - const planner = makeStubPlanner({ kind: 'success', plan: stubPlan, appliedStatements: [] }); + const planner = makeStubPlanner({ + kind: 'success', + plan: stubPlan, + appliedStatements: [], + dataLoss: [], + accessWidening: [], + }); const result = await planMigration({ aggregate, @@ -273,6 +303,7 @@ describe('planMigration', () => { callerPolicy: { ignoreGraphFor: new Set(['app']) }, operationPolicy: POLICY, appSpace: { fromContract: null, statements: [] }, + storageNameOf: (operation) => operation.id, }); expect(result.ok).toBe(true); @@ -305,6 +336,8 @@ describe('planMigration', () => { kind: 'success', plan: makeSyntheticPlan('postgres'), appliedStatements: [], + dataLoss: [], + accessWidening: [], }), ); const planner: MigrationPlanner<'sql', 'postgres'> = { @@ -326,6 +359,7 @@ describe('planMigration', () => { callerPolicy: { ignoreGraphFor: new Set() }, operationPolicy: POLICY, appSpace: { fromContract: null, statements: [] }, + storageNameOf: (operation) => operation.id, }); expect(result.ok).toBe(true); @@ -364,6 +398,8 @@ describe('planMigration', () => { kind: 'success', plan: makeSyntheticPlan('postgres'), appliedStatements: [], + dataLoss: [], + accessWidening: [], }); const result = await planMigration({ @@ -380,6 +416,7 @@ describe('planMigration', () => { callerPolicy: { ignoreGraphFor: new Set(['app', 'cipherstash']) }, operationPolicy: POLICY, appSpace: { fromContract: null, statements: [] }, + storageNameOf: (operation) => operation.id, }); expect(result.ok).toBe(false); @@ -404,6 +441,8 @@ describe('planMigration', () => { kind: 'success', plan: makeSyntheticPlan('postgres'), appliedStatements: [], + dataLoss: [], + accessWidening: [], }); const result = await planMigration({ @@ -420,6 +459,7 @@ describe('planMigration', () => { callerPolicy: { ignoreGraphFor: new Set(['app']) }, operationPolicy: POLICY, appSpace: { fromContract: null, statements: [] }, + storageNameOf: (operation) => operation.id, }); expect(result.ok).toBe(false); @@ -460,6 +500,8 @@ describe('planMigration', () => { kind: 'success', plan: makeSyntheticPlan('postgres'), appliedStatements: [], + dataLoss: [], + accessWidening: [], }); const result = await planMigration({ @@ -474,6 +516,7 @@ describe('planMigration', () => { callerPolicy: { ignoreGraphFor: new Set(['app']) }, operationPolicy: POLICY, appSpace: { fromContract: null, statements: [] }, + storageNameOf: (operation) => operation.id, }); expect(result.ok).toBe(false); @@ -514,6 +557,8 @@ describe('planMigration', () => { kind: 'success', plan: makeSyntheticPlan('postgres'), appliedStatements: [], + dataLoss: [], + accessWidening: [], }); const result = await planMigration({ @@ -528,6 +573,7 @@ describe('planMigration', () => { callerPolicy: { ignoreGraphFor: new Set(['app']) }, operationPolicy: POLICY, appSpace: { fromContract: null, statements: [] }, + storageNameOf: (operation) => operation.id, }); expect(result.ok).toBe(true); @@ -559,6 +605,7 @@ describe('planMigration', () => { callerPolicy: { ignoreGraphFor: new Set(['app']) }, operationPolicy: POLICY, appSpace: { fromContract: null, statements: [] }, + storageNameOf: (operation) => operation.id, }); expect(result.ok).toBe(false); diff --git a/packages/1-framework/3-tooling/migration/test/aggregate/strategies/plan-from-diff.test.ts b/packages/1-framework/3-tooling/migration/test/aggregate/strategies/plan-from-diff.test.ts index 4fc6ffb106af..c4f1a81c403b 100644 --- a/packages/1-framework/3-tooling/migration/test/aggregate/strategies/plan-from-diff.test.ts +++ b/packages/1-framework/3-tooling/migration/test/aggregate/strategies/plan-from-diff.test.ts @@ -55,7 +55,13 @@ describe('planFromDiff', () => { plan: ({ schema, ownership }) => { observedSchema = schema; observedOwnership = ownership; - return { kind: 'success', plan: makeStubPlan('placeholder'), appliedStatements: [] }; + return { + kind: 'success', + plan: makeStubPlan('placeholder'), + appliedStatements: [], + dataLoss: [], + accessWidening: [], + }; }, emptyMigration: () => { throw new Error('not used'); @@ -149,6 +155,8 @@ describe('planFromDiff', () => { kind: 'success', plan: { ...makeStubPlan('placeholder'), origin: options.origin }, appliedStatements: [], + dataLoss: [], + accessWidening: [], }; }, emptyMigration: () => { @@ -219,4 +227,44 @@ describe('planFromDiff', () => { if (outcome.kind !== 'failure') return; expect(outcome.conflicts).toEqual([{ kind: 'typeMismatch', summary: 'incompatible' }]); }); + + it('passes through what each operation loses and whose access it widens', async () => { + const subject = { kind: 'storage', name: 'audit_log' } as const; + const stubPlanner: MigrationPlanner<'sql', 'postgres'> = { + plan: () => ({ + kind: 'success', + plan: makeStubPlan('placeholder'), + appliedStatements: [], + dataLoss: [{ operationIndex: 0, subject }], + accessWidening: [{ operationIndex: 0, subject, widens: true }], + }), + emptyMigration: () => { + throw new Error('not used'); + }, + }; + const outcome = await planFromDiff({ + aggregateTargetId: 'postgres', + currentMarker: null, + space: makeSpace('app', {}), + ownership: STUB_OWNERSHIP, + schemaIntrospection: { tables: {} }, + adapter: STUB_ADAPTER, + migrations: { + createPlanner: () => stubPlanner, + createRunner: () => { + throw new Error('runner not used'); + }, + contractToSchema: () => ({ tables: {} }), + }, + frameworkComponents: [], + operationPolicy: POLICY, + fromContract: null, + statements: [], + }); + + expect(outcome.kind === 'ok' && outcome.result).toMatchObject({ + dataLoss: [{ operationIndex: 0, subject }], + accessWidening: [{ operationIndex: 0, subject }], + }); + }); }); diff --git a/packages/1-framework/3-tooling/migration/test/aggregate/strategies/resolve-recorded-path.test.ts b/packages/1-framework/3-tooling/migration/test/aggregate/strategies/resolve-recorded-path.test.ts index 1b649fa3786b..94898cc07421 100644 --- a/packages/1-framework/3-tooling/migration/test/aggregate/strategies/resolve-recorded-path.test.ts +++ b/packages/1-framework/3-tooling/migration/test/aggregate/strategies/resolve-recorded-path.test.ts @@ -20,6 +20,29 @@ function makeSpace( } describe('resolveRecordedPath', () => { + it('names each destructive operation of the path by its storage name, and widens no access', () => { + const headHash = 'cipher-head'; + const pkg = createAttestedPackage('20260101T0000_init', { from: null, to: headHash }, [ + { id: 'table.events', label: 'Create table events', operationClass: 'additive' }, + { id: 'table.legacy', label: 'Drop table legacy', operationClass: 'destructive' }, + { id: 'rls.events', label: 'Disable RLS on events', operationClass: 'widening' }, + ]); + + const outcome = resolveRecordedPath({ + aggregateTargetId: 'postgres', + space: makeSpace([pkg], headHash), + currentMarker: null, + storageNameOf: (operation) => `storage of ${operation.id}`, + }); + + expect(outcome.kind === 'ok' && outcome.result).toMatchObject({ + dataLoss: [ + { operationIndex: 1, subject: { kind: 'storage', name: 'storage of table.legacy' } }, + ], + accessWidening: [], + }); + }); + it('walks the shortest path from the live marker to the on-disk head ref', () => { const headHash = 'cipher-head'; const pkg = createAttestedPackage('20260101T0000_init', { from: null, to: headHash }); @@ -28,6 +51,7 @@ describe('resolveRecordedPath', () => { aggregateTargetId: 'postgres', space: makeSpace([pkg], headHash), currentMarker: null, + storageNameOf: (operation) => operation.id, }); expect(outcome.kind).toBe('ok'); @@ -55,6 +79,7 @@ describe('resolveRecordedPath', () => { aggregateTargetId: 'postgres', space: makeSpace([baseline, delta], headHash), currentMarker: null, + storageNameOf: (operation) => operation.id, }); expect(outcome.kind).toBe('ok'); @@ -74,6 +99,7 @@ describe('resolveRecordedPath', () => { aggregateTargetId: 'postgres', space: makeSpace([pkg], headHash), currentMarker: null, + storageNameOf: (operation) => operation.id, }); expect(outcome.kind).toBe('ok'); @@ -93,6 +119,7 @@ describe('resolveRecordedPath', () => { aggregateTargetId: 'postgres', space: makeSpace([pkg], headHash), currentMarker: null, + storageNameOf: (operation) => operation.id, }); expect(outcome.kind).toBe('unreachable'); @@ -108,6 +135,7 @@ describe('resolveRecordedPath', () => { aggregateTargetId: 'postgres', space, currentMarker: null, + storageNameOf: (operation) => operation.id, }); expect(outcome.kind).toBe('unsatisfiable'); @@ -124,6 +152,7 @@ describe('resolveRecordedPath', () => { space: makeSpace([pkg], headHash), currentMarker: null, refName: 'prod', + storageNameOf: (operation) => operation.id, }); expect(outcome.kind).toBe('ok'); @@ -139,6 +168,7 @@ describe('resolveRecordedPath', () => { aggregateTargetId: 'postgres', space: makeSpace([pkg], headHash), currentMarker: null, + storageNameOf: (operation) => operation.id, }); expect(outcome.kind).toBe('ok'); @@ -154,6 +184,7 @@ describe('resolveRecordedPath', () => { aggregateTargetId: 'postgres', space: makeSpace([pkg], headHash), currentMarker: { storageHash: headHash, invariants: [] }, + storageNameOf: (operation) => operation.id, }); expect(outcome.kind).toBe('ok'); @@ -170,6 +201,7 @@ describe('resolveRecordedPath', () => { aggregateTargetId: 'postgres', space: makeSpace([], EMPTY_CONTRACT_HASH), currentMarker: null, + storageNameOf: (operation) => operation.id, }); expect(outcome.kind).toBe('ok'); diff --git a/packages/2-mongo-family/9-family/src/core/control-instance.ts b/packages/2-mongo-family/9-family/src/core/control-instance.ts index fd29d57963ed..c1bf3a1bc6bf 100644 --- a/packages/2-mongo-family/9-family/src/core/control-instance.ts +++ b/packages/2-mongo-family/9-family/src/core/control-instance.ts @@ -38,6 +38,7 @@ import type { import type { MongoControlExtensionDescriptor } from './control-types'; import { MongoContractSerializer } from './ir/mongo-contract-serializer'; import { mongoOperationsToPreview } from './operation-preview'; +import { mongoStorageNameOf } from './operation-storage-name'; import { mongoSchemaToView } from './schema-to-view'; import { verifyMongoSchema } from './schema-verify/verify-mongo-schema'; @@ -383,5 +384,9 @@ export function createMongoFamilyInstance(controlStack: ControlStack): MongoCont toOperationPreview(operations: readonly MigrationPlanOperation[]): OperationPreview { return mongoOperationsToPreview(operations); }, + + storageNameOf(operation: MigrationPlanOperation): string { + return mongoStorageNameOf(operation); + }, }; } diff --git a/packages/2-mongo-family/9-family/src/core/operation-storage-name.ts b/packages/2-mongo-family/9-family/src/core/operation-storage-name.ts new file mode 100644 index 000000000000..2c2157a04e07 --- /dev/null +++ b/packages/2-mongo-family/9-family/src/core/operation-storage-name.ts @@ -0,0 +1,24 @@ +import type { MigrationPlanOperation } from '@internal/framework-components/control'; + +function collectionOfStep(step: unknown): string | undefined { + if (typeof step !== 'object' || step === null) return undefined; + const command: unknown = Reflect.get(step, 'command'); + if (typeof command !== 'object' || command === null) return undefined; + const collection: unknown = Reflect.get(command, 'collection'); + return typeof collection === 'string' ? collection : undefined; +} + +/** + * The collection an operation's command names, as the database knows it: the name the Mongo + * planner gives a dropped collection. An operation whose commands name no collection is named by + * its id. + */ +export function mongoStorageNameOf(operation: MigrationPlanOperation): string { + const execute: unknown = Reflect.get(operation, 'execute'); + const steps: readonly unknown[] = Array.isArray(execute) ? execute : []; + for (const step of steps) { + const collection = collectionOfStep(step); + if (collection !== undefined) return collection; + } + return operation.id; +} diff --git a/packages/2-mongo-family/9-family/test/operation-storage-name.test.ts b/packages/2-mongo-family/9-family/test/operation-storage-name.test.ts new file mode 100644 index 000000000000..d92698fbea8b --- /dev/null +++ b/packages/2-mongo-family/9-family/test/operation-storage-name.test.ts @@ -0,0 +1,28 @@ +import type { MigrationPlanOperation } from '@internal/framework-components/control'; +import { DropCollectionCommand } from '@internal/mongo-query-ast/control'; +import { describe, expect, it } from 'vitest'; +import { mongoStorageNameOf } from '../src/core/operation-storage-name'; + +function operation(execute: readonly unknown[]): MigrationPlanOperation { + const op = { id: 'collection.events.drop', label: 'Drop collection events: all of it' }; + return { ...op, operationClass: 'destructive', execute } as MigrationPlanOperation; +} + +describe('mongoStorageNameOf', () => { + it('names the collection the command names, as the planner names a dropped collection', () => { + expect([ + mongoStorageNameOf( + operation([{ description: 'drop', command: new DropCollectionCommand('events') }]), + ), + mongoStorageNameOf( + operation([ + { description: 'drop', command: { kind: 'dropCollection', collection: 'logs' } }, + ]), + ), + ]).toEqual(['events', 'logs']); + }); + + it('names an operation whose commands name no collection by its id, never its label', () => { + expect(mongoStorageNameOf(operation([]))).toBe('collection.events.drop'); + }); +}); diff --git a/packages/2-sql/9-family/src/core/control-instance.ts b/packages/2-sql/9-family/src/core/control-instance.ts index 51bbc04078ba..a6cb5eca2897 100644 --- a/packages/2-sql/9-family/src/core/control-instance.ts +++ b/packages/2-sql/9-family/src/core/control-instance.ts @@ -63,6 +63,7 @@ import { } from './diff/schema-verify'; import { sqlFamilyError } from './errors'; import { SqlContractSerializer } from './ir/sql-contract-serializer'; +import { storageNameOfOperation } from './migrations/operation-storage-name'; import type { SqlSchemaDiffFn } from './migrations/schema-differ'; import type { SqlControlAdapterDescriptor, @@ -325,6 +326,8 @@ export interface SqlControlFamilyInstance bootstrapControlTableQueries(): readonly DdlNode[]; toOperationPreview(operations: readonly MigrationPlanOperation[]): OperationPreview; + + storageNameOf(operation: MigrationPlanOperation): string; } export type SqlFamilyInstance = SqlControlFamilyInstance; @@ -966,6 +969,10 @@ export function createSqlFamilyInstance( return sqlOperationsToPreview(operations); }, + storageNameOf(operation: MigrationPlanOperation): string { + return storageNameOfOperation(operation); + }, + toSchemaView(schema: SqlSchemaIRNode): CoreSchemaView { // Walk the schema-IR tree's own structure (root → namespaces → tables) // into one flat list of table nodes. A root that exposes a `namespaces` diff --git a/packages/2-sql/9-family/src/core/migrations/field-event-planner.ts b/packages/2-sql/9-family/src/core/migrations/field-event-planner.ts index 8a9f541e5532..b35c3be02e65 100644 --- a/packages/2-sql/9-family/src/core/migrations/field-event-planner.ts +++ b/packages/2-sql/9-family/src/core/migrations/field-event-planner.ts @@ -71,9 +71,25 @@ interface FieldEntry { readonly newField: StorageColumn | undefined; } +/** A call a codec hook returned, with the column of the field event it was returned for. */ +export interface FieldEventCall { + readonly call: OpFactoryCall; + readonly namespaceId: string; + /** The table and column as the new contract names them. */ + readonly tableName: string; + readonly columnName: string; +} + export function planFieldEventOperations( options: PlanFieldEventOperationsOptions, ): readonly OpFactoryCall[] { + return planFieldEventCalls(options).map(({ call }) => call); +} + +/** The calls the codec hooks return, each with the field event's column. */ +export function planFieldEventCalls( + options: PlanFieldEventOperationsOptions, +): readonly FieldEventCall[] { const priorContract = options.priorContract; const newContract = options.newContract; const renamedTo = new Map( @@ -145,7 +161,7 @@ export function planFieldEventOperations( } } - const calls: OpFactoryCall[] = []; + const calls: FieldEventCall[] = []; appendCalls('added', added, options.codecHooks, calls, (e) => e.newField?.codecId); appendCalls('dropped', dropped, options.codecHooks, calls, (e) => e.priorField?.codecId); appendCalls('altered', altered, options.codecHooks, calls, (e) => e.newField?.codecId); @@ -170,7 +186,7 @@ function appendCalls( event: FieldEvent, entries: readonly FieldEntry[], codecHooks: ReadonlyMap, - calls: OpFactoryCall[], + calls: FieldEventCall[], pickCodecId: (entry: FieldEntry) => string | undefined, ): void { for (const entry of entries) { @@ -180,7 +196,14 @@ function appendCalls( if (!hook?.onFieldEvent) continue; const ctx = buildContext(event, entry); const emitted = hook.onFieldEvent(event, ctx); - for (const call of emitted) calls.push(call); + for (const call of emitted) { + calls.push({ + call, + namespaceId: entry.namespaceId, + tableName: entry.tableName, + columnName: entry.fieldName, + }); + } } } diff --git a/packages/2-sql/9-family/src/core/migrations/operation-storage-name.ts b/packages/2-sql/9-family/src/core/migrations/operation-storage-name.ts new file mode 100644 index 000000000000..12d60742d5d9 --- /dev/null +++ b/packages/2-sql/9-family/src/core/migrations/operation-storage-name.ts @@ -0,0 +1,44 @@ +import type { MigrationPlanOperation } from '@internal/framework-components/control'; +import { UNBOUND_NAMESPACE_ID } from '@internal/framework-components/ir'; + +interface StorageDetails { + readonly objectType: string; + readonly name: string; + readonly schema?: string; + readonly table?: string; +} + +function isStorageDetails(details: unknown): details is StorageDetails { + if (typeof details !== 'object' || details === null) return false; + const [objectType, name, schema, table] = ['objectType', 'name', 'schema', 'table'].map( + (key): unknown => Reflect.get(details, key), + ); + return ( + typeof objectType === 'string' && + typeof name === 'string' && + (schema === undefined || typeof schema === 'string') && + (table === undefined || typeof table === 'string') + ); +} + +function storageDetailsOf(operation: MigrationPlanOperation): StorageDetails | undefined { + const target: unknown = Reflect.get(operation, 'target'); + if (typeof target !== 'object' || target === null) return undefined; + const details: unknown = Reflect.get(target, 'details'); + return isStorageDetails(details) ? details : undefined; +} + +/** + * The name the database knows an operation's object by, read from the operation's target + * details: `schema.table.column` for a column, `schema.name` for anything else, without the + * schema in the unbound namespace. An operation without details is named by its id. + */ +export function storageNameOfOperation(operation: MigrationPlanOperation): string { + const details = storageDetailsOf(operation); + if (details === undefined) return operation.id; + const schema = + details.schema === undefined || details.schema === UNBOUND_NAMESPACE_ID ? [] : [details.schema]; + const table = + details.objectType === 'column' && details.table !== undefined ? [details.table] : []; + return [...schema, ...table, details.name].join('.'); +} diff --git a/packages/2-sql/9-family/src/core/migrations/operation-subjects.ts b/packages/2-sql/9-family/src/core/migrations/operation-subjects.ts new file mode 100644 index 000000000000..e6c77bcc9696 --- /dev/null +++ b/packages/2-sql/9-family/src/core/migrations/operation-subjects.ts @@ -0,0 +1,194 @@ +import { asNamespaceId, type ContractWithDomain } from '@internal/contract/types'; +import type { + MigrationAccessChange, + MigrationOperationSubject, + MigrationPlanSubjects, + MigrationSubject, + ModelCoordinate, + ResolvedMigrationStatement, +} from '@internal/framework-components/control'; +import { UNBOUND_NAMESPACE_ID } from '@internal/framework-components/ir'; +import type { SqlModelStorage } from '@internal/sql-contract/types'; +import { + fieldRenameStorageEffect, + isSqlModelStorage, + modelRenameStorageEffect, +} from './statement-planning'; + +/** + * What an operation is about, as the plan names it after its rename statements: a table, or a + * column of one, in a storage namespace, or neither, such as a type. `storageName` names the + * subject when no model of the origin contract stores it. + */ +export interface SubjectStorage { + readonly storageName: string; + readonly table: + | { + readonly namespaceId: string; + readonly table: string; + readonly column: string | undefined; + } + | undefined; +} + +/** What a codec hook's call is about: the column of the field event it was returned for. */ +export function fieldEventStorage(field: { + readonly namespaceId: string; + readonly tableName: string; + readonly columnName: string; +}): SubjectStorage { + const qualifier = field.namespaceId === UNBOUND_NAMESPACE_ID ? [] : [field.namespaceId]; + return { + storageName: [...qualifier, field.tableName, field.columnName].join('.'), + table: { namespaceId: field.namespaceId, table: field.tableName, column: field.columnName }, + }; +} + +/** + * Names for destructive calls a target cannot map to storage: each call's factory name, with a + * `#n` ordinal when several calls of the plan share it, so each name picks out one call. + */ +export function unknownCallNames( + calls: readonly TCall[], +): ReadonlyMap { + const counts = new Map(); + for (const call of calls) counts.set(call.factoryName, (counts.get(call.factoryName) ?? 0) + 1); + const seen = new Map(); + return new Map( + calls.map((call) => { + const ordinal = (seen.get(call.factoryName) ?? 0) + 1; + seen.set(call.factoryName, ordinal); + const shared = (counts.get(call.factoryName) ?? 0) > 1; + return [call, shared ? `${call.factoryName}#${ordinal}` : call.factoryName]; + }), + ); +} + +/** A call of a plan: how many operations it lowers to, and what the first of them is about. */ +export interface CallSubjects { + readonly operationCount: number; + /** What the call's operation loses, one entry per model, field or storage name. */ + readonly dataLoss: readonly SubjectStorage[]; + /** What the call's operation changes access to, and whether it widens it. */ + readonly accessWidening: readonly (SubjectStorage & { readonly widens: boolean })[]; +} + +interface SubjectContext { + readonly fromContract: ContractWithDomain | null; + readonly contract: ContractWithDomain; + /** The rename statements the plan applied, whose renames a target names the new way. */ + readonly statements: readonly ResolvedMigrationStatement[]; +} + +function originTable( + context: SubjectContext & { readonly fromContract: ContractWithDomain }, + namespaceId: string, + table: string, +): string { + for (const statement of context.statements) { + if (statement.entity !== 'model') continue; + const effect = modelRenameStorageEffect(statement, context.fromContract, context.contract); + if (!effect.ok || effect.value.kind !== 'renameTable') continue; + const { rename } = effect.value; + if (rename.namespaceId === namespaceId && rename.to === table) return rename.from; + } + return table; +} + +function originColumn( + context: SubjectContext & { readonly fromContract: ContractWithDomain }, + namespaceId: string, + table: string, + column: string, +): string { + for (const statement of context.statements) { + if (statement.entity !== 'field') continue; + const effect = fieldRenameStorageEffect(statement, context.fromContract, context.contract); + if (!effect.ok || effect.value.kind !== 'renameColumn') continue; + const rename = effect.value; + if ( + rename.table.namespaceId === namespaceId && + rename.table.table === table && + rename.to === column + ) { + return rename.from; + } + } + return column; +} + +interface StoringModel { + readonly coordinate: ModelCoordinate; + readonly isRoot: boolean; + readonly fields: SqlModelStorage['fields']; +} + +/** The models of `contract` that store `table`: a model and its variants share one. */ +function modelsStoring( + contract: ContractWithDomain, + namespaceId: string, + table: string, +): readonly StoringModel[] { + return Object.entries(contract.domain.namespaces).flatMap(([modelNamespace, namespace]) => + Object.entries(namespace.models).flatMap(([model, definition]) => { + const { storage } = definition; + if (!isSqlModelStorage(storage)) return []; + if (storage.namespaceId !== namespaceId || storage.table !== table) return []; + return [ + { + coordinate: { namespaceId: asNamespaceId(modelNamespace), model }, + isRoot: definition.base === undefined, + fields: storage.fields, + }, + ]; + }), + ); +} + +function subjectOf(target: SubjectStorage, context: SubjectContext): MigrationSubject { + const storage: MigrationSubject = { kind: 'storage', name: target.storageName }; + const { fromContract } = context; + if (target.table === undefined || fromContract === null) return storage; + const withOrigin = { ...context, fromContract }; + const { namespaceId } = target.table; + const table = originTable(withOrigin, namespaceId, target.table.table); + const models = modelsStoring(fromContract, namespaceId, table); + if (target.table.column === undefined) { + const model = models.find(({ isRoot }) => isRoot) ?? models[0]; + return model === undefined ? storage : { kind: 'model', ...model.coordinate }; + } + const column = originColumn(withOrigin, namespaceId, table, target.table.column); + for (const model of [...models].sort((a, b) => Number(b.isRoot) - Number(a.isRoot))) { + const field = Object.entries(model.fields).find(([, stored]) => stored.column === column)?.[0]; + if (field !== undefined) return { kind: 'field', ...model.coordinate, field }; + } + return storage; +} + +/** + * The subject of each operation that loses data and each that widens access, at its position in + * the plan. A subject is a model or a field of the origin contract when one stores it, found under + * its origin name when a rename statement renamed its table or column earlier in the plan. + */ +export function subjectsOfCalls( + calls: readonly CallSubjects[], + context: SubjectContext, +): MigrationPlanSubjects { + const dataLoss: MigrationOperationSubject[] = []; + const accessWidening: MigrationAccessChange[] = []; + let operationIndex = 0; + for (const call of calls) { + for (const target of call.dataLoss) { + dataLoss.push({ operationIndex, subject: subjectOf(target, context) }); + } + for (const target of call.accessWidening) { + accessWidening.push({ + operationIndex, + subject: subjectOf(target, context), + widens: target.widens, + }); + } + operationIndex += call.operationCount; + } + return { dataLoss, accessWidening }; +} diff --git a/packages/2-sql/9-family/src/core/migrations/plan-helpers.ts b/packages/2-sql/9-family/src/core/migrations/plan-helpers.ts index 8fce797dfe22..b34d8549e183 100644 --- a/packages/2-sql/9-family/src/core/migrations/plan-helpers.ts +++ b/packages/2-sql/9-family/src/core/migrations/plan-helpers.ts @@ -1,4 +1,7 @@ -import type { AppliedMigrationStatement } from '@internal/framework-components/control'; +import type { + AppliedMigrationStatement, + MigrationPlanSubjects, +} from '@internal/framework-components/control'; import { blindCast } from '@internal/utils/casts'; import type { NotOk, Ok } from '@internal/utils/result'; import { notOk, ok } from '@internal/utils/result'; @@ -120,12 +123,15 @@ export function createMigrationPlan( export function plannerSuccess( plan: SqlMigrationPlan, appliedStatements: readonly AppliedMigrationStatement[], + subjects: MigrationPlanSubjects, warnings?: readonly SqlPlannerConflict[], ): SqlPlannerSuccessResult { return Object.freeze({ kind: 'success', plan, appliedStatements: Object.freeze([...appliedStatements]), + dataLoss: Object.freeze([...subjects.dataLoss]), + accessWidening: Object.freeze([...subjects.accessWidening]), ...(warnings && warnings.length > 0 ? { warnings: Object.freeze( diff --git a/packages/2-sql/9-family/src/core/migrations/statement-planning.ts b/packages/2-sql/9-family/src/core/migrations/statement-planning.ts index 903a71e51dac..5ca1f6a0b836 100644 --- a/packages/2-sql/9-family/src/core/migrations/statement-planning.ts +++ b/packages/2-sql/9-family/src/core/migrations/statement-planning.ts @@ -63,7 +63,8 @@ export interface ColumnOnOneSide { readonly kind: 'columnOnOneSide'; } -function isSqlModelStorage(storage: ModelStorageBase): storage is SqlModelStorage { +/** Whether a model's storage names a table, as a model a SQL contract stores in a table does. */ +export function isSqlModelStorage(storage: ModelStorageBase): storage is SqlModelStorage { return ( typeof storage['table'] === 'string' && typeof storage['namespaceId'] === 'string' && diff --git a/packages/2-sql/9-family/src/exports/control.ts b/packages/2-sql/9-family/src/exports/control.ts index 04a20558d0e2..72217d9595c7 100644 --- a/packages/2-sql/9-family/src/exports/control.ts +++ b/packages/2-sql/9-family/src/exports/control.ts @@ -37,8 +37,24 @@ export { partitionCallsByControlPolicy, partitionIssuesByControlPolicy, } from '../core/migrations/control-policy'; -export type { PlanFieldEventOperationsOptions } from '../core/migrations/field-event-planner'; -export { planFieldEventOperations } from '../core/migrations/field-event-planner'; +export type { + FieldEventCall, + PlanFieldEventOperationsOptions, +} from '../core/migrations/field-event-planner'; +export { + planFieldEventCalls, + planFieldEventOperations, +} from '../core/migrations/field-event-planner'; +export { storageNameOfOperation } from '../core/migrations/operation-storage-name'; +export type { + CallSubjects, + SubjectStorage, +} from '../core/migrations/operation-subjects'; +export { + fieldEventStorage, + subjectsOfCalls, + unknownCallNames, +} from '../core/migrations/operation-subjects'; export { createMigrationPlan, plannerFailure, diff --git a/packages/2-sql/9-family/test/field-event-planner.test.ts b/packages/2-sql/9-family/test/field-event-planner.test.ts index 17afee1c7b73..5f4664f920b2 100644 --- a/packages/2-sql/9-family/test/field-event-planner.test.ts +++ b/packages/2-sql/9-family/test/field-event-planner.test.ts @@ -1,10 +1,57 @@ import type { MigrationPlanOperation } from '@internal/framework-components/control'; import { UNBOUND_NAMESPACE_ID } from '@internal/framework-components/ir'; import { describe, expect, it } from 'vitest'; -import { planFieldEventOperations } from '../src/core/migrations/field-event-planner'; +import { + planFieldEventCalls, + planFieldEventOperations, +} from '../src/core/migrations/field-event-planner'; import type { CodecControlHooks } from '../src/core/migrations/types'; import { col, contract, makeOp, recordingHook, table } from './field-event-fixtures'; +describe('planFieldEventCalls', () => { + it('returns each call with the column of the field event it was returned for', () => { + const fromContract = contract({ + User: table({ + id: col({ codecId: 'pg/text@1' }), + email: col({ codecId: 'cs/string@1' }), + bio: col({ codecId: 'cs/string@1' }), + }), + }); + const newContract = contract({ User: table({ id: col({ codecId: 'pg/text@1' }) }) }); + const cs = recordingHook((call) => [makeOp(`drop-${call.fieldName}`)]); + + const calls = planFieldEventCalls({ + priorContract: fromContract, + newContract, + codecHooks: new Map([['cs/string@1', cs.hook]]), + tableRenames: [], + columnRenames: [], + }); + + expect( + calls.map(({ call, namespaceId, tableName, columnName }) => ({ + factoryName: call.factoryName, + namespaceId, + tableName, + columnName, + })), + ).toEqual([ + { + factoryName: 'drop-bio', + namespaceId: UNBOUND_NAMESPACE_ID, + tableName: 'User', + columnName: 'bio', + }, + { + factoryName: 'drop-email', + namespaceId: UNBOUND_NAMESPACE_ID, + tableName: 'User', + columnName: 'email', + }, + ]); + }); +}); + describe('planFieldEventOperations', () => { it("fires 'added' once per added field on the new field's codec", () => { const fromContract = contract({ diff --git a/packages/2-sql/9-family/test/migrations.types.test.ts b/packages/2-sql/9-family/test/migrations.types.test.ts index 7324d46e2431..acc494a93166 100644 --- a/packages/2-sql/9-family/test/migrations.types.test.ts +++ b/packages/2-sql/9-family/test/migrations.types.test.ts @@ -162,8 +162,9 @@ describe('planner helpers', () => { operations: [], providedInvariants: [], }); - const success = plannerSuccess(plan, []); - expect(success).toEqual({ kind: 'success', plan, appliedStatements: [] }); + const noSubjects = { dataLoss: [], accessWidening: [] }; + const success = plannerSuccess(plan, [], noSubjects); + expect(success).toEqual({ kind: 'success', plan, appliedStatements: [], ...noSubjects }); expect(Object.isFrozen(success)).toBe(true); const warning = { @@ -171,9 +172,14 @@ describe('planner helpers', () => { summary: 'control policy suppressed: createTable(users)', meta: { controlPolicy: 'external', factoryName: 'createTable' }, }; - const successWithWarnings = plannerSuccess(plan, [], [warning]); + const successWithWarnings = plannerSuccess(plan, [], noSubjects, [warning]); expect(successWithWarnings.warnings).toEqual([warning]); - expect(plannerSuccess(plan, [], [])).toEqual({ kind: 'success', plan, appliedStatements: [] }); + expect(plannerSuccess(plan, [], noSubjects, [])).toEqual({ + kind: 'success', + plan, + appliedStatements: [], + ...noSubjects, + }); const conflict = { kind: 'typeMismatch', diff --git a/packages/2-sql/9-family/test/operation-storage-name.test.ts b/packages/2-sql/9-family/test/operation-storage-name.test.ts new file mode 100644 index 000000000000..0686fefe80dd --- /dev/null +++ b/packages/2-sql/9-family/test/operation-storage-name.test.ts @@ -0,0 +1,32 @@ +import type { MigrationPlanOperation } from '@internal/framework-components/control'; +import { UNBOUND_NAMESPACE_ID } from '@internal/framework-components/ir'; +import { describe, expect, it } from 'vitest'; +import { storageNameOfOperation } from '../src/core/migrations/operation-storage-name'; + +function operation(details: Record | undefined): MigrationPlanOperation { + const op = { id: 'drop.thing', label: 'Drop a thing: carefully', operationClass: 'destructive' }; + return { + ...op, + ...(details === undefined ? {} : { target: { id: 'postgres', details } }), + } as MigrationPlanOperation; +} + +describe('storageNameOfOperation', () => { + it('names a column through its table and schema, and anything else through its schema', () => { + expect([ + storageNameOfOperation( + operation({ schema: 'public', objectType: 'column', name: 'nickname', table: 'User' }), + ), + storageNameOfOperation(operation({ schema: 'public', objectType: 'table', name: 'Legacy' })), + storageNameOfOperation(operation({ schema: 'public', objectType: 'type', name: 'mood' })), + storageNameOfOperation( + operation({ schema: UNBOUND_NAMESPACE_ID, objectType: 'table', name: 'Legacy' }), + ), + storageNameOfOperation(operation({ objectType: 'column', name: 'nickname', table: 'User' })), + ]).toEqual(['public.User.nickname', 'public.Legacy', 'public.mood', 'Legacy', 'User.nickname']); + }); + + it('names an operation without target details by its id, never its label', () => { + expect(storageNameOfOperation(operation(undefined))).toBe('drop.thing'); + }); +}); diff --git a/packages/2-sql/9-family/test/operation-subjects.test.ts b/packages/2-sql/9-family/test/operation-subjects.test.ts new file mode 100644 index 000000000000..b8383d007b47 --- /dev/null +++ b/packages/2-sql/9-family/test/operation-subjects.test.ts @@ -0,0 +1,223 @@ +import { asNamespaceId } from '@internal/contract/types'; +import { UNBOUND_NAMESPACE_ID } from '@internal/framework-components/ir'; +import { describe, expect, it } from 'vitest'; +import { + type CallSubjects, + fieldEventStorage, + type SubjectStorage, + subjectsOfCalls, + unknownCallNames, +} from '../src/core/migrations/operation-subjects'; +import { contractOf, renameField, renameModel } from './statement-fixtures'; + +const app = asNamespaceId('app'); + +function table(name: string, storageName = name): SubjectStorage { + return { storageName, table: { namespaceId: 'app', table: name, column: undefined } }; +} + +function column(tableName: string, columnName: string): SubjectStorage { + return { + storageName: `${tableName}.${columnName}`, + table: { namespaceId: 'app', table: tableName, column: columnName }, + }; +} + +function losing(...targets: readonly SubjectStorage[]): CallSubjects { + return { operationCount: 1, dataLoss: targets, accessWidening: [] }; +} + +const unchanged: CallSubjects = { operationCount: 1, dataLoss: [], accessWidening: [] }; + +const origin = contractOf({ + Legacy: { table: 'legacy', fields: { id: 'id' } }, + User: { table: 'user', fields: { id: 'id', name: 'full_name' } }, +}); + +describe('subjectsOfCalls', () => { + it('names the model of a dropped table and the field of a dropped column', () => { + expect( + subjectsOfCalls([unchanged, losing(table('legacy')), losing(column('user', 'full_name'))], { + fromContract: origin, + contract: contractOf({ User: { table: 'user', fields: { id: 'id' } } }), + statements: [], + }), + ).toEqual({ + dataLoss: [ + { operationIndex: 1, subject: { kind: 'model', namespaceId: app, model: 'Legacy' } }, + { + operationIndex: 2, + subject: { kind: 'field', namespaceId: app, model: 'User', field: 'name' }, + }, + ], + accessWidening: [], + }); + }); + + it('counts every operation of a call with companions before the next call', () => { + expect( + subjectsOfCalls( + [{ operationCount: 3, dataLoss: [], accessWidening: [] }, losing(table('legacy'))], + { fromContract: origin, contract: origin, statements: [] }, + ).dataLoss, + ).toEqual([ + { operationIndex: 3, subject: { kind: 'model', namespaceId: app, model: 'Legacy' } }, + ]); + }); + + it('finds a table and a column renamed earlier in the plan under their origin names', () => { + const destination = contractOf({ + Account: { table: 'account', fields: { id: 'id', displayName: 'display_name' } }, + }); + const from = contractOf({ + User: { table: 'user', fields: { id: 'id', name: 'full_name' } }, + }); + expect( + subjectsOfCalls([losing(column('account', 'display_name'))], { + fromContract: from, + contract: destination, + statements: [ + renameModel('User', 'Account'), + renameField('User', 'name', 'displayName', 'Account'), + ], + }).dataLoss, + ).toEqual([ + { + operationIndex: 0, + subject: { kind: 'field', namespaceId: app, model: 'User', field: 'name' }, + }, + ]); + }); + + it('names storage the origin contract does not declare by its storage name', () => { + expect( + subjectsOfCalls( + [losing(table('audit_log', 'public.audit_log'), column('user', 'nickname'))], + { + fromContract: origin, + contract: origin, + statements: [], + }, + ).dataLoss, + ).toEqual([ + { operationIndex: 0, subject: { kind: 'storage', name: 'public.audit_log' } }, + { operationIndex: 0, subject: { kind: 'storage', name: 'user.nickname' } }, + ]); + }); + + it('names every subject by its storage name when the plan has no origin contract', () => { + expect( + subjectsOfCalls([losing(table('legacy'), column('user', 'full_name'))], { + fromContract: null, + contract: origin, + statements: [], + }).dataLoss, + ).toEqual([ + { operationIndex: 0, subject: { kind: 'storage', name: 'legacy' } }, + { operationIndex: 0, subject: { kind: 'storage', name: 'user.full_name' } }, + ]); + }); + + it('names a subject with no table, such as a type, by its storage name', () => { + expect( + subjectsOfCalls([losing({ storageName: 'public.mood', table: undefined })], { + fromContract: origin, + contract: origin, + statements: [], + }).dataLoss, + ).toEqual([{ operationIndex: 0, subject: { kind: 'storage', name: 'public.mood' } }]); + }); + + it('lists access-widening operations with the model of their table', () => { + expect( + subjectsOfCalls( + [ + unchanged, + { + operationCount: 1, + dataLoss: [], + accessWidening: [{ ...table('user'), widens: false }], + }, + ], + { fromContract: origin, contract: origin, statements: [] }, + ), + ).toEqual({ + dataLoss: [], + accessWidening: [ + { + operationIndex: 1, + subject: { kind: 'model', namespaceId: app, model: 'User' }, + widens: false, + }, + ], + }); + }); + + describe('models that share a table', () => { + const sharing = contractOf({ + Admin: { table: 'users', base: 'User', fields: { id: 'id', level: 'level' } }, + Guest: { table: 'users', base: 'User', fields: { id: 'id' } }, + User: { table: 'users', fields: { id: 'id', email: 'email' } }, + }); + + it('names the root model of a dropped table, and the model with a field for a dropped column', () => { + expect( + subjectsOfCalls([losing(table('users')), losing(column('users', 'email'))], { + fromContract: sharing, + contract: sharing, + statements: [], + }).dataLoss.map(({ subject }) => subject), + ).toEqual([ + { kind: 'model', namespaceId: app, model: 'User' }, + { kind: 'field', namespaceId: app, model: 'User', field: 'email' }, + ]); + }); + + it('names a field only a variant has through the variant', () => { + expect( + subjectsOfCalls([losing(column('users', 'level'))], { + fromContract: sharing, + contract: sharing, + statements: [], + }).dataLoss.map(({ subject }) => subject), + ).toEqual([{ kind: 'field', namespaceId: app, model: 'Admin', field: 'level' }]); + }); + }); +}); + +describe('unknownCallNames', () => { + it('numbers the calls that share a factory name, and leaves a unique one alone', () => { + const calls = [ + { factoryName: 'dropSearchConfig' }, + { factoryName: 'rebuildThing' }, + { factoryName: 'dropSearchConfig' }, + ]; + expect([...unknownCallNames(calls).values()]).toEqual([ + 'dropSearchConfig#1', + 'rebuildThing', + 'dropSearchConfig#2', + ]); + }); +}); + +describe('fieldEventStorage', () => { + it('names the field event column, qualified by its namespace outside the unbound one', () => { + const subjects = subjectsOfCalls( + [ + losing( + fieldEventStorage({ namespaceId: 'app', tableName: 'user', columnName: 'full_name' }), + fieldEventStorage({ namespaceId: 'app', tableName: 'user', columnName: 'nickname' }), + ), + ], + { fromContract: origin, contract: origin, statements: [] }, + ); + expect(subjects.dataLoss.map(({ subject }) => subject)).toEqual([ + { kind: 'field', namespaceId: app, model: 'User', field: 'name' }, + { kind: 'storage', name: 'app.user.nickname' }, + ]); + expect( + fieldEventStorage({ namespaceId: UNBOUND_NAMESPACE_ID, tableName: 'user', columnName: 'bio' }) + .storageName, + ).toBe('user.bio'); + }); +}); diff --git a/packages/2-sql/9-family/test/statement-fixtures.ts b/packages/2-sql/9-family/test/statement-fixtures.ts index ba8ffbb4d7ef..561930baeb6c 100644 --- a/packages/2-sql/9-family/test/statement-fixtures.ts +++ b/packages/2-sql/9-family/test/statement-fixtures.ts @@ -3,6 +3,7 @@ import { type Contract, type ContractModelBase, type ControlPolicy, + crossRef, profileHash, type StorageHashBase, } from '@internal/contract/types'; @@ -29,6 +30,8 @@ export interface ModelSpec { readonly control?: ControlPolicy; /** Field name to column name; `null` for a relation field, which has no column. */ readonly fields?: Readonly>; + /** The model this one is a variant of, in the same namespace. */ + readonly base?: string; } const text = { dataType: 'test/text', codecId: 'test/text@1', nullable: false }; @@ -57,6 +60,7 @@ function model(namespaceId: string, spec: ModelSpec): ContractModelBase { }, ]), ), + ...(spec.base === undefined ? {} : { base: crossRef(spec.base, namespaceId) }), storage: spec.table === null ? {} diff --git a/packages/3-extensions/mongo/README.md b/packages/3-extensions/mongo/README.md index af9ce3f2eceb..7e7a43d6aea0 100644 --- a/packages/3-extensions/mongo/README.md +++ b/packages/3-extensions/mongo/README.md @@ -163,11 +163,20 @@ Control-plane client factory. Collapses the family + target + adapter + driver w ```typescript import { createMongoControlClient } from '@internal/mongo/control'; +import contract from './src/prisma/contract.json' with { type: 'json' }; const control = createMongoControlClient({ connection: process.env['MONGODB_URL']!, }); -await control.dbUpdate({ migrations: { dir: 'migrations/app' } }); +await control.dbUpdate({ + contract, + mode: 'apply', + migrationsDir: 'migrations', + answerQuestions: async (questions) => { + if (questions.length > 0) throw new Error('db update would lose data or widen access'); + return []; + }, +}); ``` ### `@internal/mongo/bson` diff --git a/packages/3-extensions/paradedb/package.json b/packages/3-extensions/paradedb/package.json index a1e20871d11c..eca119f31de0 100644 --- a/packages/3-extensions/paradedb/package.json +++ b/packages/3-extensions/paradedb/package.json @@ -34,7 +34,7 @@ "@internal/operations": "workspace:8.0.0-rc.16", "@internal/sql-contract-ts": "workspace:8.0.0-rc.16", "@internal/target-postgres": "workspace:8.0.0-rc.16", - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/test-utils": "workspace:8.0.0-rc.16", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@repo/tsdown": "workspace:8.0.0-rc.16", diff --git a/packages/3-extensions/pgvector/package.json b/packages/3-extensions/pgvector/package.json index 45805ed55d74..776b0a531c98 100644 --- a/packages/3-extensions/pgvector/package.json +++ b/packages/3-extensions/pgvector/package.json @@ -39,7 +39,7 @@ "@internal/postgres": "workspace:8.0.0-rc.16", "@internal/postgres-codec-testkit": "workspace:8.0.0-rc.16", "@internal/sql-contract-ts": "workspace:8.0.0-rc.16", - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/test-utils": "workspace:8.0.0-rc.16", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@repo/tsdown": "workspace:8.0.0-rc.16", diff --git a/packages/3-extensions/postgis/package.json b/packages/3-extensions/postgis/package.json index 51121ac2261b..b6014b9ee210 100644 --- a/packages/3-extensions/postgis/package.json +++ b/packages/3-extensions/postgis/package.json @@ -37,7 +37,7 @@ "@internal/operations": "workspace:8.0.0-rc.16", "@internal/postgres": "workspace:8.0.0-rc.16", "@internal/sql-contract-ts": "workspace:8.0.0-rc.16", - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/test-utils": "workspace:8.0.0-rc.16", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@repo/tsdown": "workspace:8.0.0-rc.16", diff --git a/packages/3-extensions/sqlite/README.md b/packages/3-extensions/sqlite/README.md index 46103aad4af5..c693d17c1a22 100644 --- a/packages/3-extensions/sqlite/README.md +++ b/packages/3-extensions/sqlite/README.md @@ -67,11 +67,20 @@ Control-plane client factory. Collapses the family + target + adapter + driver w ```typescript import { createSqliteControlClient } from '@internal/sqlite/control'; +import contract from './src/prisma/contract.json' with { type: 'json' }; const control = createSqliteControlClient({ connection: 'path/to/app.db', }); -await control.dbUpdate({ migrations: { dir: 'migrations/app' } }); +await control.dbUpdate({ + contract, + mode: 'apply', + migrationsDir: 'migrations', + answerQuestions: async (questions) => { + if (questions.length > 0) throw new Error('db update would lose data or widen access'); + return []; + }, +}); ``` ### `@internal/sqlite/migration` diff --git a/packages/3-extensions/supabase/package.json b/packages/3-extensions/supabase/package.json index f46c554f425f..1c6dcf8a98f4 100644 --- a/packages/3-extensions/supabase/package.json +++ b/packages/3-extensions/supabase/package.json @@ -48,7 +48,7 @@ "@internal/operations": "workspace:8.0.0-rc.16", "@internal/psl-printer": "workspace:8.0.0-rc.16", "@internal/sql-contract-psl": "workspace:8.0.0-rc.16", - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/test-utils": "workspace:8.0.0-rc.16", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@repo/tsdown": "workspace:8.0.0-rc.16", diff --git a/packages/3-extensions/supabase/test/skeleton.integration.test.ts b/packages/3-extensions/supabase/test/skeleton.integration.test.ts index e28d4418a2f7..8717c5d2fc4b 100644 --- a/packages/3-extensions/supabase/test/skeleton.integration.test.ts +++ b/packages/3-extensions/supabase/test/skeleton.integration.test.ts @@ -624,6 +624,11 @@ describe('supabase RLS behavioral e2e — filtering + drift-fails-verify', () => // Plan: exactly one drop per policy — no enablement change anywhere. const planResult = await client.dbUpdate({ + answerQuestions: async (questions) => + questions.map((question) => ({ + verb: question.verbs.includes('allow') ? ('allow' as const) : ('delete' as const), + text: question.subject, + })), contract: noPolicyContract, mode: 'plan', migrationsDir, @@ -638,6 +643,11 @@ describe('supabase RLS behavioral e2e — filtering + drift-fails-verify', () => ); const applyResult = await client.dbUpdate({ + answerQuestions: async (questions) => + questions.map((question) => ({ + verb: question.verbs.includes('allow') ? ('allow' as const) : ('delete' as const), + text: question.subject, + })), contract: noPolicyContract, mode: 'apply', migrationsDir, @@ -696,6 +706,11 @@ describe('supabase RLS behavioral e2e — filtering + drift-fails-verify', () => // Plan: exactly one rename — no drop, no create, no enablement change. const planResult = await client.dbUpdate({ + answerQuestions: async (questions) => + questions.map((question) => ({ + verb: question.verbs.includes('allow') ? ('allow' as const) : ('delete' as const), + text: question.subject, + })), contract: renamedContract, mode: 'plan', migrationsDir, @@ -707,6 +722,11 @@ describe('supabase RLS behavioral e2e — filtering + drift-fails-verify', () => expect(planIds).toEqual([`rlsPolicy.public.profile.${POLICY_WIRE_NAME}.rename`]); const applyResult = await client.dbUpdate({ + answerQuestions: async (questions) => + questions.map((question) => ({ + verb: question.verbs.includes('allow') ? ('allow' as const) : ('delete' as const), + text: question.subject, + })), contract: renamedContract, mode: 'apply', migrationsDir, diff --git a/packages/3-mongo-target/1-mongo-target/src/core/migrations/control-target.ts b/packages/3-mongo-target/1-mongo-target/src/core/migrations/control-target.ts index d8e1460cc605..21479ec048f3 100644 --- a/packages/3-mongo-target/1-mongo-target/src/core/migrations/control-target.ts +++ b/packages/3-mongo-target/1-mongo-target/src/core/migrations/control-target.ts @@ -20,7 +20,7 @@ import { notOk, ok } from '@internal/utils/result'; import { mongoTargetDescriptorMeta } from '../descriptor-meta'; import type { MongoTargetContract } from '../mongo-target-contract'; import { MongoTargetContractSerializer } from '../mongo-target-contract-serializer'; -import { MongoMigrationPlanner } from './mongo-planner'; +import { keepDataByHand, MongoMigrationPlanner } from './mongo-planner'; import { MongoMigrationRunner, type MongoMigrationRunnerExecuteOptions } from './mongo-runner'; import { MongoTargetSchemaVerifier } from './mongo-target-schema-verifier'; import { entityNamesDeclaredBy, scopeVerifyResultToSpace } from './scope-verify-result'; @@ -42,6 +42,7 @@ export const mongoTargetDescriptor: MongoControlTargetDescriptor) { return new MongoMigrationPlanner(); }, diff --git a/packages/3-mongo-target/1-mongo-target/src/core/migrations/migration-factories.ts b/packages/3-mongo-target/1-mongo-target/src/core/migrations/migration-factories.ts index 8f46a5d5d0db..87c5dcf53702 100644 --- a/packages/3-mongo-target/1-mongo-target/src/core/migrations/migration-factories.ts +++ b/packages/3-mongo-target/1-mongo-target/src/core/migrations/migration-factories.ts @@ -169,10 +169,6 @@ export function createIndex( }; } -/** - * Dropping an index loses no documents, but this stays 'destructive' on MongoDB until it moves to - * 'widening' as it has on the SQL targets. - */ export function dropIndex( collectionName: string, keys: ReadonlyArray, @@ -183,7 +179,7 @@ export function dropIndex( return { id: buildIndexOpId('drop', collectionName, keys), label: `Drop index on ${collectionName} (${formatKeys(keys)})`, - operationClass: 'destructive', + operationClass: 'widening', precheck: [ { description: `index exists on ${collectionName}`, @@ -261,7 +257,7 @@ export function setValidation( return { id: `collection.${collectionName}.setValidation`, label: `Set validation on ${collectionName}`, - operationClass: 'destructive', + operationClass: 'widening', precheck: [], execute: [ { @@ -287,7 +283,7 @@ export function collMod( return { id: meta?.id ?? `collection.${collectionName}.collMod`, label: meta?.label ?? `Modify collection ${collectionName}`, - operationClass: meta?.operationClass ?? 'destructive', + operationClass: meta?.operationClass ?? 'widening', precheck: options.validator != null ? [ diff --git a/packages/3-mongo-target/1-mongo-target/src/core/migrations/mongo-planner.ts b/packages/3-mongo-target/1-mongo-target/src/core/migrations/mongo-planner.ts index 51f9cd6e95b9..308f72a53f45 100644 --- a/packages/3-mongo-target/1-mongo-target/src/core/migrations/mongo-planner.ts +++ b/packages/3-mongo-target/1-mongo-target/src/core/migrations/mongo-planner.ts @@ -1,15 +1,16 @@ -import type { Contract, ContractWithDomain } from '@internal/contract/types'; +import { asNamespaceId, type Contract, type ContractWithDomain } from '@internal/contract/types'; import { contractToMongoSchemaIR } from '@internal/family-mongo/control'; import type { TargetBoundComponentDescriptor } from '@internal/framework-components/components'; import { describeMigrationStatement, - type MigrationOperationClass, type MigrationOperationPolicy, + type MigrationOperationSubject, type MigrationPlanner, type MigrationPlannerConflict, type MigrationPlannerResult, type MigrationPlanWithAuthoringSurface, type MigrationScaffoldContext, + type MigrationSubject, type ModelCoordinate, type PlanOrigin, type ResolvedMigrationStatement, @@ -67,103 +68,10 @@ function validatorsEqual( ); } -function classifyValidatorUpdate( - origin: MongoSchemaValidator, - dest: MongoSchemaValidator, -): 'widening' | 'destructive' { - // Moving to a stricter action or level narrows the accepted value space. - if (origin.validationAction !== dest.validationAction && dest.validationAction === 'error') { - return 'destructive'; - } - if (origin.validationLevel !== dest.validationLevel && dest.validationLevel === 'strict') { - return 'destructive'; - } - - if (canonicalize(origin.jsonSchema) === canonicalize(dest.jsonSchema)) { - return 'widening'; - } - - // Check whether the schema change only adds non-required properties (widening). - return isWideningSchemaChange(origin.jsonSchema, dest.jsonSchema) ? 'widening' : 'destructive'; -} - function isPlainObject(v: unknown): v is Record { return typeof v === 'object' && v !== null && !Array.isArray(v); } -/** - * Returns true when `dest` is a structural superset of `origin` for the common - * additive case: adding non-required properties to a top-level object schema. - * Anything uncertain falls through to the safe `destructive` default. - */ -function isWideningSchemaChange( - origin: Record, - dest: Record, -): boolean { - // Only handle top-level object schemas. - if (origin['bsonType'] !== 'object' || dest['bsonType'] !== 'object') { - return false; - } - - // Any change to keys besides 'required' and 'properties' is uncertain → destructive. - const allKeys = new Set([...Object.keys(origin), ...Object.keys(dest)]); - for (const key of allKeys) { - if (key === 'required' || key === 'properties') continue; - if (canonicalize(origin[key]) !== canonicalize(dest[key])) return false; - } - - // dest.required must be a subset of origin.required — no new required fields. - const originRequired = new Set( - Array.isArray(origin['required']) ? origin['required'] : [], - ); - const destRequired = Array.isArray(dest['required']) ? dest['required'] : []; - for (const field of destRequired) { - if (!originRequired.has(field)) return false; - } - - // All properties that existed in origin must still exist unchanged. - // New properties in dest (absent from origin) are allowed — widening. - const originProps = isPlainObject(origin['properties']) ? origin['properties'] : {}; - const destProps = isPlainObject(dest['properties']) ? dest['properties'] : {}; - for (const field of Object.keys(originProps)) { - if (!Object.hasOwn(destProps, field)) return false; // Property removed → destructive. - if (!admitsEverything(destProps[field], originProps[field])) return false; // Property narrowed → destructive. - } - - return true; -} - -function bsonTypeList(value: unknown): readonly unknown[] | undefined { - if (value === undefined) return undefined; - return Array.isArray(value) ? value : [value]; -} - -/** - * Whether every value `origin` admits, `dest` admits too, for the property schemas Prisma derives: `{}` admits every value, a `bsonType` list may gain types, and an array's `items` may admit more. Any other difference is treated as narrowing. - */ -function admitsEverything(dest: unknown, origin: unknown): boolean { - if (canonicalize(dest) === canonicalize(origin)) return true; - if (!isPlainObject(dest) || !isPlainObject(origin)) return false; - if (Object.keys(dest).length === 0) return true; - for (const key of new Set([...Object.keys(origin), ...Object.keys(dest)])) { - if (key === 'bsonType') { - const destTypes = bsonTypeList(dest['bsonType']); - const originTypes = bsonTypeList(origin['bsonType']); - if (destTypes === undefined) continue; - if (originTypes === undefined || !originTypes.every((type) => destTypes.includes(type))) { - return false; - } - continue; - } - if (key === 'items') { - if (!admitsEverything(dest['items'] ?? {}, origin['items'] ?? {})) return false; - continue; - } - if (canonicalize(origin[key]) !== canonicalize(dest[key])) return false; - } - return true; -} - function propertiesOf(schema: Record): Record { return isPlainObject(schema['properties']) ? schema['properties'] : {}; } @@ -224,12 +132,63 @@ function collectionOf(contract: ContractWithDomain | null, coordinate: ModelCoor return typeof collection === 'string' ? collection : coordinate.model; } +/** + * The model of `fromContract` that stores its documents in `collection`, if any: the root model + * when variants share the collection with it. + */ +function modelStoredIn( + fromContract: ContractWithDomain, + collection: string, +): ModelCoordinate | undefined { + const storing = Object.entries(fromContract.domain.namespaces).flatMap( + ([namespaceId, namespace]) => + Object.entries(namespace.models).flatMap(([model, definition]) => { + const coordinate = { namespaceId: asNamespaceId(namespaceId), model }; + return collectionOf(fromContract, coordinate) === collection + ? [{ coordinate, isRoot: definition.base === undefined }] + : []; + }), + ); + return (storing.find(({ isRoot }) => isRoot) ?? storing[0])?.coordinate; +} + +/** Each collection drop of the plan, with the model whose documents it loses. */ +function collectionDrops( + calls: readonly OpFactoryCall[], + fromContract: ContractWithDomain | null, +): readonly MigrationOperationSubject[] { + return calls.flatMap((call, operationIndex) => { + if (!(call instanceof DropCollectionCall)) return []; + const model = fromContract === null ? undefined : modelStoredIn(fromContract, call.collection); + const subject: MigrationSubject = + model === undefined + ? { kind: 'storage', name: call.collection } + : { kind: 'model', ...model }; + return [{ operationIndex, subject }]; + }); +} + const NOT_IN_THIS_RELEASE = 'MongoDB cannot carry out rename statements in this release.'; function shellString(name: string): string { return JSON.stringify(name); } +/** + * How to keep the documents of a subject a plan would drop, since the planner carries out no + * rename: rename the collection by hand before a plan that drops it is applied. Right for both + * `db update`, which then finds nothing to drop, and `migration plan`, whose written migration + * still drops it. + */ +export function keepDataByHand( + subject: MigrationSubject, + fromContract: ContractWithDomain, +): string { + const collection = + subject.kind === 'storage' ? subject.name : collectionOf(fromContract, subject); + return `If it was renamed, keep its documents instead: rename collection "${collection}" by hand on each database before a plan that drops it is applied there, for example with db.getCollection(${shellString(collection)}).renameCollection("") in mongosh. db update then drops nothing; a migration written by migration plan still drops "${collection}", so remove that operation from its migration.ts, or do not apply it where the collection was renamed.`; +} + /** * What a plan made without the statement does to the data, and how to keep it. Right for both * `db update` and `migration plan`, since the planner does not know which command it serves. @@ -451,6 +410,8 @@ export class MongoMigrationPlanner implements MigrationPlanner<'mongo', 'mongo'> options.snapshotsImportPath, ), appliedStatements: [], + dataLoss: collectionDrops(result.calls, options.fromContract), + accessWidening: [], }; } @@ -483,9 +444,6 @@ function planValidatorDiffCall( if (validatorsEqual(originValidator, destValidator)) return undefined; if (destValidator) { - const operationClass: MigrationOperationClass = originValidator - ? classifyValidatorUpdate(originValidator, destValidator) - : 'destructive'; return new CollModCall( collName, { @@ -498,7 +456,7 @@ function planValidatorDiffCall( label: originValidator ? validatorUpdateLabel(collName, originValidator, destValidator) : `Add validator on ${collName}`, - operationClass, + operationClass: 'widening', }, ); } @@ -536,7 +494,7 @@ function planMutableOptionsDiffCall( { id: `options.${collName}.update`, label: `Update mutable options on ${collName}`, - operationClass: desiredCSPPI.enabled ? 'widening' : 'destructive', + operationClass: 'widening', }, ); } diff --git a/packages/3-mongo-target/1-mongo-target/src/core/migrations/op-factory-call.ts b/packages/3-mongo-target/1-mongo-target/src/core/migrations/op-factory-call.ts index a56000fb7d98..49e30382823a 100644 --- a/packages/3-mongo-target/1-mongo-target/src/core/migrations/op-factory-call.ts +++ b/packages/3-mongo-target/1-mongo-target/src/core/migrations/op-factory-call.ts @@ -112,7 +112,7 @@ export class CreateIndexCall extends OpFactoryCallNode { export class DropIndexCall extends OpFactoryCallNode { readonly factoryName = 'dropIndex' as const; - readonly operationClass = 'destructive' as const; + readonly operationClass = 'widening' as const; readonly collection: string; readonly keys: ReadonlyArray; readonly label: string; @@ -195,7 +195,7 @@ export class CollModCall extends OpFactoryCallNode { this.collection = collection; this.options = options; this.meta = meta; - this.operationClass = meta?.operationClass ?? 'destructive'; + this.operationClass = meta?.operationClass ?? 'widening'; this.label = meta?.label ?? `Modify collection ${collection}`; this.freeze(); } diff --git a/packages/3-mongo-target/1-mongo-target/test/db-update-statements.test.ts b/packages/3-mongo-target/1-mongo-target/test/db-update-statements.test.ts index 2dd443e16e02..06e6bbbd8c9f 100644 --- a/packages/3-mongo-target/1-mongo-target/test/db-update-statements.test.ts +++ b/packages/3-mongo-target/1-mongo-target/test/db-update-statements.test.ts @@ -66,6 +66,11 @@ describe('db update with statements on MongoDB', () => { }); const result = await executeDbUpdate({ + answerQuestions: async (questions) => + questions.map((question) => ({ + verb: question.verbs.includes('allow') ? ('allow' as const) : ('delete' as const), + text: question.subject, + })), driver: { close: async () => {}, databaseName: async () => 'appdb', diff --git a/packages/3-mongo-target/1-mongo-target/test/migration-e2e.test.ts b/packages/3-mongo-target/1-mongo-target/test/migration-e2e.test.ts index afa2e7c59b79..cfdfa6f9cdbb 100644 --- a/packages/3-mongo-target/1-mongo-target/test/migration-e2e.test.ts +++ b/packages/3-mongo-target/1-mongo-target/test/migration-e2e.test.ts @@ -284,7 +284,7 @@ describe('migration file E2E', () => { { id: 'validator.users.add', label: 'Add validator on users', - operationClass: 'destructive', + operationClass: 'widening', }, ), ]; @@ -306,7 +306,7 @@ describe('migration file E2E', () => { expect(ops).toHaveLength(1); expect(ops[0].id).toBe('validator.users.add'); expect(ops[0].label).toBe('Add validator on users'); - expect(ops[0].operationClass).toBe('destructive'); + expect(ops[0].operationClass).toBe('widening'); expect(ops[0].execute[0].command.kind).toBe('collMod'); }); diff --git a/packages/3-mongo-target/1-mongo-target/test/migration-factories.test.ts b/packages/3-mongo-target/1-mongo-target/test/migration-factories.test.ts index 3e2347c07d95..88d001d20a0a 100644 --- a/packages/3-mongo-target/1-mongo-target/test/migration-factories.test.ts +++ b/packages/3-mongo-target/1-mongo-target/test/migration-factories.test.ts @@ -124,7 +124,7 @@ describe('dropIndex', () => { expect(op.id).toBe(buildIndexOpId('drop', 'users', keys)); expect(op.label).toBe('Drop index on users (email:1)'); - expect(op.operationClass).toBe('destructive'); + expect(op.operationClass).toBe('widening'); }); it('includes precheck that index exists', () => { @@ -282,7 +282,7 @@ describe('setValidation', () => { expect(op.id).toBe('collection.users.setValidation'); expect(op.label).toBe('Set validation on users'); - expect(op.operationClass).toBe('destructive'); + expect(op.operationClass).toBe('widening'); }); it('wraps schema in $jsonSchema validator', () => { @@ -378,7 +378,7 @@ describe('collMod', () => { expect(op.id).toBe('collection.users.collMod'); expect(op.label).toBe('Modify collection users'); - expect(op.operationClass).toBe('destructive'); + expect(op.operationClass).toBe('widening'); }); it('applies meta overrides for id, label, and operationClass', () => { diff --git a/packages/3-mongo-target/1-mongo-target/test/mongo-ops-serializer.test.ts b/packages/3-mongo-target/1-mongo-target/test/mongo-ops-serializer.test.ts index 9f81bf6b75a6..c127697b9a0a 100644 --- a/packages/3-mongo-target/1-mongo-target/test/mongo-ops-serializer.test.ts +++ b/packages/3-mongo-target/1-mongo-target/test/mongo-ops-serializer.test.ts @@ -67,7 +67,7 @@ function makeDropIndexOp(): MongoMigrationPlanOperation { return { id: 'index.users.drop(email:1)', label: 'Drop index on users (email ascending)', - operationClass: 'destructive', + operationClass: 'widening', precheck: [ { description: 'index exists on users', @@ -129,7 +129,7 @@ describe('serializeMongoOps / deserializeMongoOps', () => { expect(deserialized).toHaveLength(1); const op = asDdlOp(deserialized[0]!); expect(op.id).toBe('index.users.drop(email:1)'); - expect(op.operationClass).toBe('destructive'); + expect(op.operationClass).toBe('widening'); const cmd = op.execute[0]!.command as DropIndexCommand; expect(cmd.kind).toBe('dropIndex'); @@ -638,7 +638,7 @@ describe('serializeMongoOps / deserializeMongoOps', () => { const op: MongoMigrationPlanOperation = { id: 'validator.users.update', label: 'Update validator on users', - operationClass: 'destructive', + operationClass: 'widening', precheck: [], execute: [ { diff --git a/packages/3-mongo-target/1-mongo-target/test/mongo-planner.data-loss.test.ts b/packages/3-mongo-target/1-mongo-target/test/mongo-planner.data-loss.test.ts new file mode 100644 index 000000000000..46c548a6ad51 --- /dev/null +++ b/packages/3-mongo-target/1-mongo-target/test/mongo-planner.data-loss.test.ts @@ -0,0 +1,164 @@ +import { asNamespaceId } from '@internal/contract/types'; +import type { MigrationOperationPolicy } from '@internal/framework-components/control'; +import { MongoCollection, type MongoContract } from '@internal/mongo-contract'; +import { MongoSchemaCollection, MongoSchemaIndex, MongoSchemaIR } from '@internal/mongo-schema-ir'; +import { expectDataLossMatchesDestructive } from '@repo/test-utils/data-loss-expectations'; +import { describe, expect, it } from 'vitest'; +import { keepDataByHand, MongoMigrationPlanner } from '../src/core/migrations/mongo-planner'; +import type { PlannerProducedMongoMigration } from '../src/core/migrations/planner-produced-migration'; + +const ALL_CLASSES_POLICY: MigrationOperationPolicy = { + allowedOperationClasses: ['additive', 'widening', 'destructive', 'data'], +}; + +/** A contract whose models each store their documents in the named collection. */ +function contractWith( + models: Readonly>, + seed: string, + bases: Readonly> = {}, +): MongoContract { + return { + target: 'mongo', + targetFamily: 'mongo', + profileHash: 'test-profile', + capabilities: {}, + extensions: {}, + meta: {}, + roots: {}, + domain: { + namespaces: { + __unbound__: { + models: Object.fromEntries( + Object.entries(models).map(([model, collection]) => [ + model, + { + fields: {}, + relations: {}, + storage: { collection }, + ...(bases[model] === undefined + ? {} + : { base: { namespace: '__unbound__', model: bases[model] } }), + }, + ]), + ), + }, + }, + }, + storage: { + storageHash: `storage-${seed}`, + namespaces: { + __unbound__: { + id: '__unbound__', + kind: 'mongo-namespace', + entries: { + collection: Object.fromEntries( + Object.values(models).map((collection) => [collection, new MongoCollection({})]), + ), + }, + }, + }, + }, + } as unknown as MongoContract; +} + +function plan(fromContract: MongoContract | null) { + const result = new MongoMigrationPlanner().plan({ + contract: contractWith({ User: 'users' }, 'to'), + schema: new MongoSchemaIR([ + new MongoSchemaCollection({ name: 'users' }), + new MongoSchemaCollection({ + name: 'events', + indexes: [new MongoSchemaIndex({ keys: [{ field: 'at', direction: 1 }] })], + }), + ]), + policy: ALL_CLASSES_POLICY, + fromContract, + origin: null, + statements: [], + frameworkComponents: [], + snapshotsImportPath: '../../snapshots', + }); + if (result.kind !== 'success') throw new Error(JSON.stringify(result.conflicts)); + expectDataLossMatchesDestructive( + result, + (result.plan as PlannerProducedMongoMigration).operations, + ); + return { + operations: (result.plan as PlannerProducedMongoMigration).operations.map( + (operation) => operation.label, + ), + dataLoss: result.dataLoss, + accessWidening: result.accessWidening, + }; +} + +describe('MongoDB planner, data loss', () => { + it('names the model whose collection a drop loses, and only the collection drop', () => { + const planned = plan(contractWith({ User: 'users', Event: 'events' }, 'from')); + const dropIndex = planned.operations.findIndex((label) => label.startsWith('Drop index')); + const dropCollection = planned.operations.indexOf('Drop collection events'); + + expect({ dropIndex: dropIndex >= 0, ...planned }).toMatchObject({ + dropIndex: true, + dataLoss: [ + { + operationIndex: dropCollection, + subject: { kind: 'model', namespaceId: asNamespaceId('__unbound__'), model: 'Event' }, + }, + ], + accessWidening: [], + }); + }); + + it('names the collection by its name when the plan has no origin contract', () => { + const planned = plan(null); + expect( + planned.dataLoss.map(({ operationIndex, subject }) => ({ + operation: planned.operations[operationIndex], + subject, + })), + ).toEqual([ + { operation: 'Drop collection events', subject: { kind: 'storage', name: 'events' } }, + ]); + }); + + it('names the root model when a variant shares the dropped collection', () => { + const fromContract = contractWith( + { Meeting: 'events', User: 'users', Event: 'events' }, + 'from', + { Meeting: 'Event' }, + ); + const planned = plan(fromContract); + + expect( + planned.dataLoss.map(({ operationIndex, subject }) => ({ + operation: planned.operations[operationIndex], + subject, + })), + ).toEqual([ + { + operation: 'Drop collection events', + subject: { kind: 'model', namespaceId: asNamespaceId('__unbound__'), model: 'Event' }, + }, + ]); + }); +}); + +describe('keepDataByHand', () => { + it('says to rename the collection a dropped model stores its documents in', () => { + expect( + keepDataByHand( + { kind: 'model', namespaceId: asNamespaceId('__unbound__'), model: 'Event' }, + contractWith({ Event: 'events' }, 'from'), + ), + ).toBe( + 'If it was renamed, keep its documents instead: rename collection "events" by hand on each database before a plan that drops it is applied there, for example with db.getCollection("events").renameCollection("") in mongosh. db update then drops nothing; a migration written by migration plan still drops "events", so remove that operation from its migration.ts, or do not apply it where the collection was renamed.', + ); + }); + + it('names a collection no model stores by its name', () => { + expect(keepDataByHand({ kind: 'storage', name: 'audit' }, contractWith({}, 'from'))).toBe( + 'If it was renamed, keep its documents instead: rename collection "audit" by hand on each database before a plan that drops it is applied there, for example with db.getCollection("audit").renameCollection("") in mongosh. db update then drops nothing; a migration written by migration plan still drops "audit", so remove that operation from its migration.ts, or do not apply it where the collection was renamed.', + ); + }); +}); diff --git a/packages/3-mongo-target/1-mongo-target/test/mongo-planner.test.ts b/packages/3-mongo-target/1-mongo-target/test/mongo-planner.test.ts index a0ec1ff56219..6f2c1c167d05 100644 --- a/packages/3-mongo-target/1-mongo-target/test/mongo-planner.test.ts +++ b/packages/3-mongo-target/1-mongo-target/test/mongo-planner.test.ts @@ -149,7 +149,7 @@ describe('MongoMigrationPlanner', () => { expect(plan.operations).toHaveLength(1); const op = plan.operations[0] as MongoMigrationPlanOperation; - expect(op.operationClass).toBe('destructive'); + expect(op.operationClass).toBe('widening'); expect(op.execute).toHaveLength(1); expect(op.execute[0]!.command.kind).toBe('dropIndex'); const cmd = op.execute[0]!.command as DropIndexCommand; @@ -193,7 +193,7 @@ describe('MongoMigrationPlanner', () => { expect(plan.operations).toHaveLength(2); const drop = plan.operations[0] as MongoMigrationPlanOperation; const create = plan.operations[1] as MongoMigrationPlanOperation; - expect(drop.operationClass).toBe('destructive'); + expect(drop.operationClass).toBe('widening'); expect(create.operationClass).toBe('additive'); }); @@ -267,7 +267,11 @@ describe('MongoMigrationPlanner', () => { ]); const plan = planSuccess(planner, contract, origin); expect(plan.operations).toHaveLength(3); - expect(plan.operations.every((op) => op.operationClass === 'destructive')).toBe(true); + expect(plan.operations.map((op) => op.operationClass)).toEqual([ + 'widening', + 'widening', + 'destructive', + ]); expect(plan.operations[2]!.id).toBe('collection.users.drop'); }); @@ -295,7 +299,7 @@ describe('MongoMigrationPlanner', () => { const plan = planSuccess(planner, contract, origin); expect(plan.operations).toHaveLength(2); - expect(plan.operations[0]!.operationClass).toBe('destructive'); + expect(plan.operations[0]!.operationClass).toBe('widening'); expect(plan.operations[1]!.operationClass).toBe('additive'); }); @@ -313,7 +317,7 @@ describe('MongoMigrationPlanner', () => { }); describe('policy gating', () => { - it('returns conflicts when destructive operations are disallowed', () => { + it('returns conflicts when the policy disallows an index drop', () => { const contract = makeContract({ users: {} }); const origin = irWithCollection('users', [ascIndex('email')]); const result = planner.plan({ @@ -364,7 +368,7 @@ describe('MongoMigrationPlanner', () => { expect(result.conflicts).toHaveLength(3); }); - it('rejects destructive validator add with additive-only policy', () => { + it('rejects a validator add, which is widening, with additive-only policy', () => { const contract = makeContract({ users: { validator: { @@ -388,7 +392,7 @@ describe('MongoMigrationPlanner', () => { expect(result.kind).toBe('failure'); if (result.kind !== 'failure') throw new Error('Expected failure'); expect(result.conflicts).toHaveLength(1); - expect(result.conflicts[0]!.summary).toContain('destructive'); + expect(result.conflicts[0]!.summary).toContain('widening'); }); it('allows widening validator removal with widening policy', () => { @@ -618,6 +622,7 @@ describe('MongoMigrationPlanner', () => { const cmd = collModOps[0]!.execute[0]!.command as CollModCommand; expect(cmd.validator).toEqual({ $jsonSchema: { bsonType: 'object' } }); expect(cmd.validationLevel).toBe('strict'); + expect(collModOps[0]!.operationClass).toBe('widening'); }); it('validator add has precheck (collection exists) and postcheck (validator applied)', () => { @@ -744,9 +749,9 @@ describe('MongoMigrationPlanner', () => { ).toMatchObject({ operationClass: 'widening' }); }); - it('is destructive when the bsonType list loses a type, as changing Bson to Json does', () => { + it('is widening when the bsonType list loses a type, as changing Bson to Json does', () => { expect(validatorUpdate({}, { bsonType: JSON_TYPES })).toMatchObject({ - operationClass: 'destructive', + operationClass: 'widening', label: 'Update validator on items (changed: meta)', }); }); @@ -823,7 +828,7 @@ describe('MongoMigrationPlanner', () => { expect(collModOps[0]!.operationClass).toBe('widening'); }); - it('classifies the open->closed transition (adding additionalProperties:false) as destructive', () => { + it('classifies the open->closed transition (adding additionalProperties:false) as widening', () => { const contract = makeContract({ users: { validator: { @@ -857,7 +862,7 @@ describe('MongoMigrationPlanner', () => { (op) => op.execute[0]?.command.kind === 'collMod', ); expect(collModOps).toHaveLength(1); - expect(collModOps[0]!.operationClass).toBe('destructive'); + expect(collModOps[0]!.operationClass).toBe('widening'); }); it('classifies adding a non-required property to an already-closed schema as widening', () => { @@ -901,7 +906,7 @@ describe('MongoMigrationPlanner', () => { expect(collModOps[0]!.operationClass).toBe('widening'); }); - it('classifies narrowing an existing property type as destructive', () => { + it('classifies narrowing an existing property type as widening', () => { const contract = makeContract({ users: { validator: { @@ -932,10 +937,10 @@ describe('MongoMigrationPlanner', () => { (op) => op.execute[0]?.command.kind === 'collMod', ); expect(collModOps).toHaveLength(1); - expect(collModOps[0]!.operationClass).toBe('destructive'); + expect(collModOps[0]!.operationClass).toBe('widening'); }); - it('classifies adding a field to required as destructive', () => { + it('classifies adding a field to required as widening', () => { const contract = makeContract({ users: { validator: { @@ -974,7 +979,7 @@ describe('MongoMigrationPlanner', () => { (op) => op.execute[0]?.command.kind === 'collMod', ); expect(collModOps).toHaveLength(1); - expect(collModOps[0]!.operationClass).toBe('destructive'); + expect(collModOps[0]!.operationClass).toBe('widening'); }); it('treats reordered jsonSchema keys as equivalent (no operation emitted)', () => { @@ -1032,7 +1037,7 @@ describe('MongoMigrationPlanner', () => { expect(collModOps[0]!.operationClass).toBe('widening'); }); - it('classifies validationAction warn->error as destructive', () => { + it('classifies validationAction warn->error as widening', () => { const contract = makeContract({ users: { validator: { @@ -1057,7 +1062,7 @@ describe('MongoMigrationPlanner', () => { (op) => op.execute[0]?.command.kind === 'collMod', ); expect(collModOps).toHaveLength(1); - expect(collModOps[0]!.operationClass).toBe('destructive'); + expect(collModOps[0]!.operationClass).toBe('widening'); }); it('classifies validationLevel strict->moderate as widening', () => { @@ -1088,7 +1093,7 @@ describe('MongoMigrationPlanner', () => { expect(collModOps[0]!.operationClass).toBe('widening'); }); - it('classifies validationLevel moderate->strict as destructive', () => { + it('classifies validationLevel moderate->strict as widening', () => { const contract = makeContract({ users: { validator: { @@ -1113,10 +1118,10 @@ describe('MongoMigrationPlanner', () => { (op) => op.execute[0]?.command.kind === 'collMod', ); expect(collModOps).toHaveLength(1); - expect(collModOps[0]!.operationClass).toBe('destructive'); + expect(collModOps[0]!.operationClass).toBe('widening'); }); - it('classifies mixed widening+destructive changes as destructive', () => { + it('classifies a validator that both relaxes and tightens as widening', () => { const contract = makeContract({ users: { validator: { @@ -1141,7 +1146,7 @@ describe('MongoMigrationPlanner', () => { (op) => op.execute[0]?.command.kind === 'collMod', ); expect(collModOps).toHaveLength(1); - expect(collModOps[0]!.operationClass).toBe('destructive'); + expect(collModOps[0]!.operationClass).toBe('widening'); }); it('no-ops when validators are identical', () => { @@ -1437,7 +1442,7 @@ describe('MongoMigrationPlanner', () => { expect(collModOps[0]!.operationClass).toBe('widening'); }); - it('classifies disabling changeStreamPreAndPostImages as destructive', () => { + it('classifies disabling changeStreamPreAndPostImages as widening', () => { const contract = makeContract({ events: { options: { changeStreamPreAndPostImages: { enabled: false } }, @@ -1456,7 +1461,7 @@ describe('MongoMigrationPlanner', () => { (op) => op.execute[0]?.command.kind === 'collMod', ); expect(collModOps).toHaveLength(1); - expect(collModOps[0]!.operationClass).toBe('destructive'); + expect(collModOps[0]!.operationClass).toBe('widening'); }); it('emits { enabled: false } when destination removes changeStreamPreAndPostImages', () => { @@ -1474,7 +1479,7 @@ describe('MongoMigrationPlanner', () => { (op) => op.execute[0]?.command.kind === 'collMod', ); expect(collModOps).toHaveLength(1); - expect(collModOps[0]!.operationClass).toBe('destructive'); + expect(collModOps[0]!.operationClass).toBe('widening'); const cmd = collModOps[0]!.execute[0]!.command as CollModCommand; expect(cmd.changeStreamPreAndPostImages).toEqual({ enabled: false }); }); @@ -1622,7 +1627,7 @@ describe('MongoMigrationPlanner', () => { expect((calls[0] as CollModCall).meta).toMatchObject({ id: 'validator.users.update', label: 'Update validator on users', - operationClass: 'destructive', + operationClass: 'widening', }); }); diff --git a/packages/3-mongo-target/1-mongo-target/test/op-factory-call.test.ts b/packages/3-mongo-target/1-mongo-target/test/op-factory-call.test.ts index 93cb225abc94..db2f8cc600ca 100644 --- a/packages/3-mongo-target/1-mongo-target/test/op-factory-call.test.ts +++ b/packages/3-mongo-target/1-mongo-target/test/op-factory-call.test.ts @@ -48,13 +48,13 @@ describe('Mongo call classes', () => { ); }); - it('DropIndexCall freezes, labels destructively, and lowers to dropIndex(...)', () => { + it('DropIndexCall freezes, labels as widening, and lowers to dropIndex(...)', () => { const call = new DropIndexCall('users', [{ field: 'legacy', direction: -1 }]); expect(Object.isFrozen(call)).toBe(true); expect(call).toMatchObject({ factoryName: 'dropIndex', - operationClass: 'destructive', + operationClass: 'widening', label: 'Drop index on users (legacy:-1)', }); @@ -87,13 +87,13 @@ describe('Mongo call classes', () => { expect(call.toOp()).toEqual(dropCollection('users')); }); - it('CollModCall defaults operationClass to destructive, uses a default label, and lowers to collMod(...)', () => { + it('CollModCall defaults operationClass to widening, uses a default label, and lowers to collMod(...)', () => { const call = new CollModCall('users', { validator: { $jsonSchema: { type: 'object' } } }); expect(Object.isFrozen(call)).toBe(true); expect(call).toMatchObject({ factoryName: 'collMod', - operationClass: 'destructive', + operationClass: 'widening', label: 'Modify collection users', }); diff --git a/packages/3-targets/3-targets/postgres/src/core/migrations/issue-planner.ts b/packages/3-targets/3-targets/postgres/src/core/migrations/issue-planner.ts index 02fa9ae01b66..4ee7c2bc8f96 100644 --- a/packages/3-targets/3-targets/postgres/src/core/migrations/issue-planner.ts +++ b/packages/3-targets/3-targets/postgres/src/core/migrations/issue-planner.ts @@ -94,6 +94,7 @@ import { postgresPlannerStrategies, type StrategyContext, } from './planner-strategies'; +import { liveColumnWidensSafely } from './safe-widenings'; export type { CallMigrationStrategy, StrategyContext }; @@ -687,11 +688,13 @@ function mapColumnNodeIssue( if (columnTypeChanged(expected, actual)) { const { qualifiedTargetType, formatTypeExpected } = renderColumnAlterType(expected, types); calls.push( - new AlterColumnTypeCall(schemaName, tableName, expected.name, { - qualifiedTargetType, - formatTypeExpected, - rawTargetTypeForLabel: qualifiedTargetType, - }), + new AlterColumnTypeCall( + schemaName, + tableName, + expected.name, + { qualifiedTargetType, formatTypeExpected, rawTargetTypeForLabel: qualifiedTargetType }, + liveColumnWidensSafely(expected, actual, types) ? 'widening' : 'destructive', + ), ); } if (expected.nullable !== actual.nullable) { diff --git a/packages/3-targets/3-targets/postgres/src/core/migrations/op-factory-call.ts b/packages/3-targets/3-targets/postgres/src/core/migrations/op-factory-call.ts index 3de2f614c797..e5574f91c52a 100644 --- a/packages/3-targets/3-targets/postgres/src/core/migrations/op-factory-call.ts +++ b/packages/3-targets/3-targets/postgres/src/core/migrations/op-factory-call.ts @@ -67,6 +67,7 @@ import { import type { PostgresColumnDefault } from '../types'; import { boundSchema } from './bound-schema'; import { + type AlterColumnTypeClass, addNotNullColumnDirect, alterColumnType, dropColumn, @@ -388,10 +389,6 @@ export type RenameCompanionCall = RenameConstraintCall | RenameIndexCall; export class RenameTableCall extends PostgresOpFactoryCallNode { readonly factoryName = 'renameTable' as const; - // `widening` for the same reason as `RenameConstraintCall`: a rename is - // neither additive creation nor destructive, and the class vocabulary has no - // neutral middle class, so this is the class that plans under every - // allowance set except additive-only init. readonly operationClass = 'widening' as const; readonly schemaName: string; readonly oldTableName: string; @@ -644,7 +641,7 @@ export interface AlterColumnTypeOptions { export class AlterColumnTypeCall extends PostgresOpFactoryCallNode { readonly factoryName = 'alterColumnType' as const; - readonly operationClass = 'destructive' as const; + readonly operationClass: AlterColumnTypeClass; readonly schemaName: string; readonly tableName: string; readonly columnName: string; @@ -656,12 +653,14 @@ export class AlterColumnTypeCall extends PostgresOpFactoryCallNode { tableName: string, columnName: string, options: AlterColumnTypeOptions, + operationClass: AlterColumnTypeClass = 'destructive', ) { super(); this.schemaName = schemaName; this.tableName = tableName; this.columnName = columnName; this.options = options; + this.operationClass = operationClass; this.label = `Alter type of "${tableName}"."${columnName}" to ${options.rawTargetTypeForLabel}`; this.freeze(); } @@ -674,7 +673,14 @@ export class AlterColumnTypeCall extends PostgresOpFactoryCallNode { { meta: { factory: 'AlterColumnTypeCall' } }, ); } - return alterColumnType(this.schemaName, this.tableName, this.columnName, this.options, lowerer); + return alterColumnType( + this.schemaName, + this.tableName, + this.columnName, + this.options, + lowerer, + this.operationClass, + ); } renderTypeScript(): string { @@ -685,6 +691,9 @@ export class AlterColumnTypeCall extends PostgresOpFactoryCallNode { opts.push(`table: ${jsonToTsSource(this.tableName)}`); opts.push(`column: ${jsonToTsSource(this.columnName)}`); opts.push(`options: ${jsonToTsSource(this.options)}`); + if (this.operationClass !== 'destructive') { + opts.push(`operationClass: ${jsonToTsSource(this.operationClass)}`); + } return `this.alterColumnType({ ${opts.join(', ')} })`; } @@ -695,7 +704,7 @@ export class AlterColumnTypeCall extends PostgresOpFactoryCallNode { export class SetNotNullCall extends PostgresOpFactoryCallNode { readonly factoryName = 'setNotNull' as const; - readonly operationClass = 'destructive' as const; + readonly operationClass = 'widening' as const; readonly schemaName: string; readonly tableName: string; readonly columnName: string; @@ -1201,10 +1210,6 @@ export class DropConstraintCall extends PostgresOpFactoryCallNode { export class RenameConstraintCall extends PostgresOpFactoryCallNode { readonly factoryName = 'renameConstraint' as const; - // `widening` is chosen so the rename plans under every allowance set except - // additive-only init — a rename is neither additive-creation nor - // destructive, and the class vocabulary has no neutral middle class. It is - // NOT that a rename widens anything; this is the accepted typology tradeoff. readonly operationClass = 'widening' as const; readonly schemaName: string; readonly tableName: string; @@ -1456,10 +1461,6 @@ export class CreateIndexCall extends PostgresOpFactoryCallNode { export class RenameIndexCall extends PostgresOpFactoryCallNode { readonly factoryName = 'renameIndex' as const; - // `widening` is chosen so the rename plans under every allowance set except - // additive-only init — a rename is neither additive-creation nor - // destructive, and the class vocabulary has no neutral middle class. It is - // NOT that a rename widens anything; this is the accepted typology tradeoff. readonly operationClass = 'widening' as const; readonly schemaName: string; readonly tableName: string; @@ -2082,10 +2083,6 @@ export class DisableRowLevelSecurityCall extends PostgresOpFactoryCallNode { export class RenamePostgresRlsPolicyCall extends PostgresOpFactoryCallNode { readonly factoryName = 'renameRlsPolicy' as const; - // `widening` is chosen so the rename plans under every allowance set except - // additive-only init — a rename is neither additive-creation nor - // destructive, and the class vocabulary has no neutral middle class. It is - // NOT that a rename widens anything; this is the accepted typology tradeoff. readonly operationClass = 'widening' as const; readonly schemaName: string; readonly tableName: string; diff --git a/packages/3-targets/3-targets/postgres/src/core/migrations/operation-subjects.ts b/packages/3-targets/3-targets/postgres/src/core/migrations/operation-subjects.ts new file mode 100644 index 000000000000..facf28678bdd --- /dev/null +++ b/packages/3-targets/3-targets/postgres/src/core/migrations/operation-subjects.ts @@ -0,0 +1,145 @@ +import type { Contract } from '@internal/contract/types'; +import { + type CallSubjects, + type FieldEventCall, + fieldEventStorage, + type SubjectStorage, + storageNameOfOperation, + unknownCallNames, +} from '@internal/family-sql/control'; +import type { OpFactoryCall } from '@internal/framework-components/control'; +import { UNBOUND_NAMESPACE_ID } from '@internal/framework-components/ir'; +import type { SqlStorage } from '@internal/sql-contract/types'; +import { parseWireName } from '@internal/sql-schema-ir/naming'; +import { resolveNamespaceIdForDdlSchema } from './control-policy'; +import { + AlterColumnTypeCall, + CreatePostgresRlsPolicyCall, + DisableRowLevelSecurityCall, + DropColumnCall, + DropPostgresRlsPolicyCall, + DropTableCall, + RawSqlCall, +} from './op-factory-call'; + +interface Locator { + /** The contract that names the namespaces: the origin contract when the plan has one. */ + readonly contract: Contract; + /** The codec hooks' calls, each with the column of the field event it was returned for. */ + readonly fieldEvents: ReadonlyMap; +} + +function storageNamespaceId(locator: Locator, schemaName: string): string { + return schemaName === UNBOUND_NAMESPACE_ID + ? UNBOUND_NAMESPACE_ID + : resolveNamespaceIdForDdlSchema(locator.contract, schemaName); +} + +function qualified(schemaName: string, name: string): string { + return schemaName === UNBOUND_NAMESPACE_ID ? name : `${schemaName}.${name}`; +} + +function target( + locator: Locator, + schemaName: string, + table: string, + column: string | undefined, +): SubjectStorage { + return { + storageName: qualified(schemaName, column === undefined ? table : `${table}.${column}`), + table: { namespaceId: storageNamespaceId(locator, schemaName), table, column }, + }; +} + +function rawSqlTarget(locator: Locator, call: RawSqlCall): SubjectStorage { + const details = call.op.target.details; + if (details === undefined) return { storageName: call.op.id, table: undefined }; + if (details.objectType === 'table') + return target(locator, details.schema, details.name, undefined); + if (details.objectType === 'column' && details.table !== undefined) { + return target(locator, details.schema, details.table, details.name); + } + return { storageName: storageNameOfOperation(call.op), table: undefined }; +} + +function knownTarget(locator: Locator, call: OpFactoryCall): SubjectStorage | undefined { + if (call instanceof DropTableCall) { + return target(locator, call.schemaName, call.tableName, undefined); + } + if (call instanceof DropColumnCall || call instanceof AlterColumnTypeCall) { + return target(locator, call.schemaName, call.tableName, call.columnName); + } + if (call instanceof RawSqlCall) return rawSqlTarget(locator, call); + const fieldEvent = locator.fieldEvents.get(call); + return fieldEvent === undefined ? undefined : fieldEventStorage(fieldEvent); +} + +/** + * A policy's identity across a replacement: its name, or the prefix of a generated (wire) name, + * whose hash changes with the policy's body. + */ +function policyKey(schemaName: string, tableName: string, policyName: string): string { + return JSON.stringify([schemaName, tableName, parseWireName(policyName)?.prefix ?? policyName]); +} + +/** + * The policies the plan creates. A drop of one of them, by name or by generated-name prefix, is + * half of a replacement, which leaves a policy in place, so it widens no access. + */ +function createdPolicies(calls: readonly OpFactoryCall[]): ReadonlySet { + return new Set( + calls.flatMap((call) => + call instanceof CreatePostgresRlsPolicyCall + ? [policyKey(call.schemaName, call.tableName, call.policy.name)] + : [], + ), + ); +} + +function accessWideningOf( + locator: Locator, + call: OpFactoryCall, + replaced: ReadonlySet, +): CallSubjects['accessWidening'] { + if (call instanceof DisableRowLevelSecurityCall) { + return [{ ...target(locator, call.schemaName, call.tableName, undefined), widens: true }]; + } + if ( + call instanceof DropPostgresRlsPolicyCall && + !replaced.has(policyKey(call.schemaName, call.tableName, call.policyName)) + ) { + return [{ ...target(locator, call.schemaName, call.tableName, undefined), widens: false }]; + } + return []; +} + +function operationCountOf(call: OpFactoryCall): number { + return 'companions' in call && Array.isArray(call.companions) ? 1 + call.companions.length : 1; +} + +/** What each call of a Postgres plan loses, and whose access it widens. */ +export function postgresCallSubjects( + calls: readonly OpFactoryCall[], + locator: Locator, +): readonly CallSubjects[] { + const replaced = createdPolicies(calls); + const destructive = calls.filter((call) => call.operationClass === 'destructive'); + const known = new Map(destructive.map((call) => [call, knownTarget(locator, call)])); + const unknownNames = unknownCallNames( + destructive.filter((call) => known.get(call) === undefined), + ); + return calls.map((call) => { + const lossTarget = known.get(call); + const unknownName = unknownNames.get(call); + return { + operationCount: operationCountOf(call), + dataLoss: + lossTarget !== undefined + ? [lossTarget] + : unknownName !== undefined + ? [{ storageName: unknownName, table: undefined }] + : [], + accessWidening: accessWideningOf(locator, call, replaced), + }; + }); +} diff --git a/packages/3-targets/3-targets/postgres/src/core/migrations/operations/columns.ts b/packages/3-targets/3-targets/postgres/src/core/migrations/operations/columns.ts index 13f70ff6470d..1bcad634f180 100644 --- a/packages/3-targets/3-targets/postgres/src/core/migrations/operations/columns.ts +++ b/packages/3-targets/3-targets/postgres/src/core/migrations/operations/columns.ts @@ -112,6 +112,9 @@ export async function renameColumn( }; } +/** A type change is `widening` when every value of the old type converts to the new type unchanged. */ +export type AlterColumnTypeClass = 'widening' | 'destructive'; + /** * `qualifiedTargetType` is the new column type as it appears in the * `ALTER COLUMN TYPE` clause (schema-qualified for user-defined types, raw @@ -131,6 +134,7 @@ export async function alterColumnType( readonly using?: string; }, lowerer: ExecuteRequestLowerer, + operationClass: AlterColumnTypeClass = 'destructive', ): Promise { const qualified = qualifyTableName(schemaName, tableName); const usingClause = options.using @@ -152,7 +156,7 @@ export async function alterColumnType( return { id: `alterType.${tableName}.${columnName}`, label: `Alter type of "${tableName}"."${columnName}" to ${options.rawTargetTypeForLabel}`, - operationClass: 'destructive', + operationClass, target: targetDetails('column', columnName, schemaName, tableName), precheck: [step(`ensure column "${columnName}" exists`, present.sql, present.params)], execute: [ @@ -198,7 +202,7 @@ export async function setNotNull( return { id: `alterNullability.setNotNull.${tableName}.${columnName}`, label: `Set NOT NULL on "${tableName}"."${columnName}"`, - operationClass: 'destructive', + operationClass: 'widening', target: targetDetails('column', columnName, schemaName, tableName), precheck: [ step(`ensure column "${columnName}" exists`, present.sql, present.params), diff --git a/packages/3-targets/3-targets/postgres/src/core/migrations/operations/indexes.ts b/packages/3-targets/3-targets/postgres/src/core/migrations/operations/indexes.ts index 616aed1e7559..c832c5a659e9 100644 --- a/packages/3-targets/3-targets/postgres/src/core/migrations/operations/indexes.ts +++ b/packages/3-targets/3-targets/postgres/src/core/migrations/operations/indexes.ts @@ -70,12 +70,7 @@ export async function createIndex( }; } -/** - * `ALTER INDEX … RENAME TO`. `widening` for the same typology reason as the - * RLS policy rename: a rename is neither additive creation nor destructive, - * and the class vocabulary has no neutral middle class — it is NOT that a - * rename widens anything. - */ +/** `ALTER INDEX … RENAME TO`: `widening`, because a rename keeps every row and value. */ export function renameIndexOperationId( schemaName: string, tableName: string, diff --git a/packages/3-targets/3-targets/postgres/src/core/migrations/planner-strategies.ts b/packages/3-targets/3-targets/postgres/src/core/migrations/planner-strategies.ts index 3d1b69857878..4217ff5ed53a 100644 --- a/packages/3-targets/3-targets/postgres/src/core/migrations/planner-strategies.ts +++ b/packages/3-targets/3-targets/postgres/src/core/migrations/planner-strategies.ts @@ -36,7 +36,7 @@ import type { TargetBoundComponentDescriptor } from '@internal/framework-compone import type { SchemaDiffIssue } from '@internal/framework-components/control'; import { issueOutcome } from '@internal/framework-components/control'; import { UNBOUND_NAMESPACE_ID } from '@internal/framework-components/ir'; -import { type SqlTypeLookups, sqlDataTypeOfCodec } from '@internal/sql-contract/data-type'; +import type { SqlTypeLookups } from '@internal/sql-contract/data-type'; import type { SqlStorage, StorageTable, StorageTypeInstance } from '@internal/sql-contract/types'; import * as contractFree from '@internal/sql-relational-core/contract-free'; import { @@ -45,8 +45,6 @@ import { type SqlSchemaIR, } from '@internal/sql-schema-ir/types'; import { blindCast } from '@internal/utils/casts'; -import { InternalError } from '@internal/utils/internal-error'; -import { pgFloat4, pgFloat8, pgInt2, pgInt4, pgInt8 } from '../data-types'; import { isPostgresSchema } from '../postgres-schema'; import { renderColumnAlterType, @@ -68,6 +66,7 @@ import { } from './op-factory-call'; import { buildSchemaLookupMap, hasForeignKey, hasUniqueConstraint } from './planner-schema-lookup'; import { buildTargetDetails, type PostgresPlanTargetDetails } from './planner-target-details'; +import { columnDataType, isSafeTypeWidening } from './safe-widenings'; /** * Look up a storage table by its explicit namespace coordinate. Returns @@ -236,29 +235,13 @@ export const notNullBackfillCallStrategy: CallMigrationStrategy = (issues, ctx) }; }; -const SAFE_WIDENINGS = new Set([ - `${pgInt2.id}→${pgInt4.id}`, - `${pgInt2.id}→${pgInt8.id}`, - `${pgInt4.id}→${pgInt8.id}`, - `${pgFloat4.id}→${pgFloat8.id}`, -]); - -/** The id of the data type a column node's codec represents. */ -function columnDataType(column: SqlColumnIR, types: SqlTypeLookups): string { - if (column.codecRef === undefined) { - throw new InternalError( - `Column "${column.name}" carries no codec, so its data type is unknown; a type change is planned only between columns built from contracts.`, - ); - } - return sqlDataTypeOfCodec(column.codecRef.codecId, types).id; -} - /** - * Handles `not-equal` column issues whose TYPE differs. `fromContract` is - * only supplied by `migration plan` — for reconciliation (`db update` / - * `db init`, `fromContract === null`) this strategy never fires, mirroring - * the legacy `typeChangeCallStrategy`'s requirement of a prior contract: - * `mapNodeIssueToCall`'s in-place ALTER covers reconciliation directly. + * Handles `not-equal` column issues whose TYPE differs, for a plan that has an + * origin contract and may include `data` operations: `migration plan`. Under + * `db update` or `db init`, whose policies leave out `data`, it never fires, + * whether or not an origin contract is known: it could only plan the in-place + * ALTER that `mapNodeIssueToCall` plans, so leaving that to the mapper keeps + * the plan, and its order, the same either way. * * A single node issue can carry BOTH type and nullability drift (Postgres * alters in place, so the differ emits one `not-equal` column issue where @@ -270,8 +253,9 @@ function columnDataType(column: SqlColumnIR, types: SqlTypeLookups): string { * partially handled. */ export const typeChangeCallStrategy: CallMigrationStrategy = (issues, ctx) => { - if (ctx.fromContract === null) return { kind: 'no_match' }; - const dataAllowed = ctx.policy.allowedOperationClasses.includes('data'); + if (ctx.fromContract === null || !ctx.policy.allowedOperationClasses.includes('data')) { + return { kind: 'no_match' }; + } const matched: SchemaDiffIssue[] = []; const calls: PostgresOpFactoryCall[] = []; @@ -285,8 +269,7 @@ export const typeChangeCallStrategy: CallMigrationStrategy = (issues, ctx) => { const fromType = columnDataType(actual, ctx.types); const toType = columnDataType(expected, ctx.types); - const isSafeWidening = SAFE_WIDENINGS.has(`${fromType}→${toType}`); - if (!isSafeWidening && !dataAllowed) continue; + const isSafeWidening = isSafeTypeWidening(fromType, toType); const ddlSchemaName = issueSchemaName(issue); const tableName = issueTableName(issue); @@ -301,7 +284,9 @@ export const typeChangeCallStrategy: CallMigrationStrategy = (issues, ctx) => { rawTargetTypeForLabel: qualifiedTargetType, }; if (isSafeWidening) { - calls.push(new AlterColumnTypeCall(schemaName, tableName, expected.name, alterOpts)); + calls.push( + new AlterColumnTypeCall(schemaName, tableName, expected.name, alterOpts, 'widening'), + ); } else { calls.push( new DataTransformCall( @@ -316,7 +301,7 @@ export const typeChangeCallStrategy: CallMigrationStrategy = (issues, ctx) => { if (expected.nullable !== actual.nullable) { if (expected.nullable) { calls.push(new DropNotNullCall(schemaName, tableName, expected.name)); - } else if (dataAllowed) { + } else { calls.push( new DataTransformCall( `handle-nulls-${tableName}-${expected.name}`, @@ -325,8 +310,6 @@ export const typeChangeCallStrategy: CallMigrationStrategy = (issues, ctx) => { ), new SetNotNullCall(schemaName, tableName, expected.name), ); - } else { - calls.push(new SetNotNullCall(schemaName, tableName, expected.name)); } } } diff --git a/packages/3-targets/3-targets/postgres/src/core/migrations/planner.ts b/packages/3-targets/3-targets/postgres/src/core/migrations/planner.ts index be8d7f14cdd8..e68e81317db8 100644 --- a/packages/3-targets/3-targets/postgres/src/core/migrations/planner.ts +++ b/packages/3-targets/3-targets/postgres/src/core/migrations/planner.ts @@ -13,16 +13,19 @@ import { extractCodecControlHooks, partitionCallsByControlPolicy, partitionIssuesByControlPolicy, - planFieldEventOperations, + planFieldEventCalls, plannerFailure, planStatements, sqlTypeLookupsOf, + subjectsOfCalls, } from '@internal/family-sql/control'; import type { ExecuteRequestLowerer } from '@internal/family-sql/control-adapter'; import type { TargetBoundComponentDescriptor } from '@internal/framework-components/components'; import type { AppliedMigrationStatement, + MigrationAccessChange, MigrationOperationClass, + MigrationOperationSubject, MigrationPlanner, MigrationPlanWithAuthoringSurface, MigrationScaffoldContext, @@ -76,6 +79,7 @@ import { RenamePostgresRlsPolicyCall, type RenameTableCall, } from './op-factory-call'; +import { postgresCallSubjects } from './operation-subjects'; import { TypeScriptRenderablePostgresMigration } from './planner-produced-postgres-migration'; import { postgresPlannerStrategies } from './planner-strategies'; import { resolveDdlSchemaForNamespaceStorage } from './resolve-ddl-schema'; @@ -130,6 +134,8 @@ export type PostgresPlanResult = readonly plan: TypeScriptRenderablePostgresMigration; readonly warnings?: readonly SqlPlannerConflict[]; readonly appliedStatements: readonly AppliedMigrationStatement[]; + readonly dataLoss: readonly MigrationOperationSubject[]; + readonly accessWidening: readonly MigrationAccessChange[]; } | SqlPlannerFailureResult; @@ -400,13 +406,14 @@ export class PostgresMigrationPlanner implements MigrationPlanner<'sql', 'postgr // within-group sorting by `(tableName, fieldName)` so re-emits are // byte-stable. The hook fires only at the application emitter — // extension-space planning never reaches this helper. - const fieldEventOps = planFieldEventOperations({ - priorContract: options.fromContract, + const fieldEventCalls = planFieldEventCalls({ + priorContract: options.origin === null ? null : options.fromContract, newContract: options.contract, codecHooks, tableRenames: statements.value.tableRenames, columnRenames: statements.value.columnRenames, }); + const fieldEventOps = fieldEventCalls.map(({ call }) => call); // Codec hook ops are target-agnostic `OpFactoryCall`; Postgres planning // lifts them at this integration boundary (see field-event-planner JSDoc). const fieldEventPostgresCalls = blindCast< @@ -458,6 +465,17 @@ export class PostgresMigrationPlanner implements MigrationPlanner<'sql', 'postgr this.#lowerer, ), appliedStatements: statements.value.appliedStatements, + ...subjectsOfCalls( + postgresCallSubjects(calls, { + contract: options.fromContract ?? options.contract, + fieldEvents: new Map(fieldEventCalls.map((fieldEvent) => [fieldEvent.call, fieldEvent])), + }), + { + fromContract: options.fromContract, + contract: options.contract, + statements: options.statements, + }, + ), ...(warnings.length > 0 ? { warnings: Object.freeze(warnings) } : {}), }); } diff --git a/packages/3-targets/3-targets/postgres/src/core/migrations/postgres-migration.ts b/packages/3-targets/3-targets/postgres/src/core/migrations/postgres-migration.ts index be4713a13036..1694629659ef 100644 --- a/packages/3-targets/3-targets/postgres/src/core/migrations/postgres-migration.ts +++ b/packages/3-targets/3-targets/postgres/src/core/migrations/postgres-migration.ts @@ -53,6 +53,7 @@ import { SetDefaultCall, SetNotNullCall, } from './op-factory-call'; +import type { AlterColumnTypeClass } from './operations/columns'; import type { RenamableConstraintKind } from './operations/constraints'; import { type DataTransformOptions, dataTransform } from './operations/data-transform'; import { installExtension } from './operations/dependencies'; @@ -510,12 +511,14 @@ export abstract class PostgresMigration< readonly table: string; readonly column: string; readonly options: AlterColumnTypeOptions; + readonly operationClass?: AlterColumnTypeClass; }): Promise> { return new AlterColumnTypeCall( options.schema, options.table, options.column, options.options, + options.operationClass, ).toOp(this.controlAdapterFor('alterColumnType')); } diff --git a/packages/3-targets/3-targets/postgres/src/core/migrations/safe-widenings.ts b/packages/3-targets/3-targets/postgres/src/core/migrations/safe-widenings.ts new file mode 100644 index 000000000000..264290ec4350 --- /dev/null +++ b/packages/3-targets/3-targets/postgres/src/core/migrations/safe-widenings.ts @@ -0,0 +1,52 @@ +import { type SqlTypeLookups, sqlDataTypeOfCodec } from '@internal/sql-contract/data-type'; +import type { SqlColumnIR } from '@internal/sql-schema-ir/types'; +import { InternalError } from '@internal/utils/internal-error'; +import { pgFloat4, pgFloat8, pgInt2, pgInt4, pgInt8 } from '../data-types'; + +/** The type changes that keep every value: each pair is `from→to` by data type id. */ +const SAFE_WIDENINGS = new Set([ + `${pgInt2.id}→${pgInt4.id}`, + `${pgInt2.id}→${pgInt8.id}`, + `${pgInt4.id}→${pgInt8.id}`, + `${pgFloat4.id}→${pgFloat8.id}`, +]); + +const WIDENABLE_TYPES = [pgInt2, pgInt4, pgInt8, pgFloat4, pgFloat8]; + +export function isSafeTypeWidening(fromDataType: string, toDataType: string): boolean { + return SAFE_WIDENINGS.has(`${fromDataType}→${toDataType}`); +} + +/** The id of the data type a column node's codec represents. */ +export function columnDataType(column: SqlColumnIR, types: SqlTypeLookups): string { + if (column.codecRef === undefined) { + throw new InternalError( + `Column "${column.name}" carries no codec, so its data type is unknown; a type change is planned only between columns built from contracts.`, + ); + } + return sqlDataTypeOfCodec(column.codecRef.codecId, types).id; +} + +/** The widenable data type a migration writes as `text`, which introspection reports normalised. */ +function widenableDataTypeWritten(text: string): string | undefined { + return WIDENABLE_TYPES.find((type) => + type.sql.texts.some((candidate) => candidate.written === true && candidate.text === text), + )?.id; +} + +/** + * Whether changing an introspected column to the contract's column keeps every value. The live + * column carries no codec, so its type is read from the normalised name introspection reports. + */ +export function liveColumnWidensSafely( + expected: SqlColumnIR, + actual: SqlColumnIR, + types: SqlTypeLookups, +): boolean { + if (expected.many === true || actual.many === true) return false; + const from = + actual.resolvedNativeType === undefined + ? undefined + : widenableDataTypeWritten(actual.resolvedNativeType); + return from !== undefined && isSafeTypeWidening(from, columnDataType(expected, types)); +} diff --git a/packages/3-targets/3-targets/postgres/test/migrations/node-issue-planner.test.ts b/packages/3-targets/3-targets/postgres/test/migrations/node-issue-planner.test.ts index 286bcaf2627a..f6b1a2f3c06c 100644 --- a/packages/3-targets/3-targets/postgres/test/migrations/node-issue-planner.test.ts +++ b/packages/3-targets/3-targets/postgres/test/migrations/node-issue-planner.test.ts @@ -176,6 +176,100 @@ describe('buildPostgresPlanDiff + planNodeIssues (one-differ path)', () => { expect(calls.map((c) => c.factoryName)).toEqual(['alterColumnType', 'setNotNull']); }); + describe('a type change of an introspected column, with no origin contract', () => { + function alterClass( + live: { readonly nativeType: string; readonly many?: boolean }, + contractColumn: TableSpec['columns'][string], + ) { + const contract = makeContract({ + user: { + columns: { + id: { dataType: 'pg/uuid', codecId: 'pg/uuid@1', nullable: false }, + age: contractColumn, + }, + primaryKey: { columns: ['id'] }, + foreignKeys: [], + uniques: [], + indexes: [], + }, + }); + const actual = rootOf({ + user: new PostgresTableSchemaNode({ + name: 'user', + columns: { + id: { name: 'id', nativeType: 'uuid', nullable: false, resolvedNativeType: 'uuid' }, + age: { + name: 'age', + nativeType: live.nativeType, + nullable: false, + resolvedNativeType: live.nativeType, + ...(live.many === undefined ? {} : { many: live.many }), + }, + }, + primaryKey: { columns: ['id'] }, + foreignKeys: [], + uniques: [], + indexes: [], + policies: [], + rlsEnabled: false, + }), + }); + const call = planFor(contract, actual).find((c) => c instanceof AlterColumnTypeCall); + return call?.operationClass; + } + + it('is widening for a safe widening read from the written type name', () => { + expect([ + alterClass( + { nativeType: 'int4' }, + { dataType: 'pg/int8', codecId: 'pg/int8@1', nullable: false }, + ), + alterClass( + { nativeType: 'int2' }, + { dataType: 'pg/int4', codecId: 'pg/int4@1', nullable: false }, + ), + alterClass( + { nativeType: 'float4' }, + { dataType: 'pg/float8', codecId: 'pg/float8@1', nullable: false }, + ), + ]).toEqual(['widening', 'widening', 'widening']); + }); + + it('is destructive for a type change that can change values, or one between lists', () => { + expect([ + alterClass( + { nativeType: 'int8' }, + { dataType: 'pg/int4', codecId: 'pg/int4@1', nullable: false }, + ), + alterClass( + { nativeType: 'text' }, + { dataType: 'pg/int4', codecId: 'pg/int4@1', nullable: false }, + ), + alterClass( + { nativeType: 'int4', many: true }, + { + dataType: 'pg/int8', + codecId: 'pg/int8@1', + nullable: false, + many: { elementNullable: false }, + }, + ), + alterClass( + { nativeType: 'float4' }, + { dataType: 'pg/int8', codecId: 'pg/int8@1', nullable: false }, + ), + alterClass( + { nativeType: 'numeric(10,2)' }, + { dataType: 'pg/int8', codecId: 'pg/int8@1', nullable: false }, + ), + alterClass( + { nativeType: 'int4' }, + { dataType: 'pg/float8', codecId: 'pg/float8@1', nullable: false }, + ), + ]).toEqual(Array(6).fill('destructive')); + }); + }); + it('checks a type change of a column that references a storage type against the catalog text of the referenced type', () => { const contract = makeContract( { diff --git a/packages/3-targets/3-targets/postgres/test/migrations/op-factory-call.lowering.test.ts b/packages/3-targets/3-targets/postgres/test/migrations/op-factory-call.lowering.test.ts index 2ac6e997e24d..2b564cd7abe9 100644 --- a/packages/3-targets/3-targets/postgres/test/migrations/op-factory-call.lowering.test.ts +++ b/packages/3-targets/3-targets/postgres/test/migrations/op-factory-call.lowering.test.ts @@ -222,6 +222,21 @@ describe('AlterColumnTypeCall', () => { expect(call.label).toBe('Alter type of "user"."age" to bigint'); }); + it('is destructive unless the planner says the type change is a safe widening', async () => { + const { lowerer } = recordingCheckLowerer(); + const call = new AlterColumnTypeCall('public', 'user', 'age', options); + const op = await call.toOp(lowerer); + expect([call.operationClass, op.operationClass]).toEqual(['destructive', 'destructive']); + }); + + it('is widening when constructed as a safe widening, and renders the class', async () => { + const { lowerer } = recordingCheckLowerer(); + const call = new AlterColumnTypeCall('public', 'user', 'age', options, 'widening'); + const op = await call.toOp(lowerer); + expect([call.operationClass, op.operationClass]).toEqual(['widening', 'widening']); + expect(call.renderTypeScript()).toContain('operationClass: "widening"'); + }); + it('uses an explicit USING clause when provided', async () => { const { lowerer } = recordingCheckLowerer(); const call = new AlterColumnTypeCall('public', 'user', 'age', { @@ -258,6 +273,7 @@ describe('AlterColumnTypeCall', () => { expect(ts).toContain('qualifiedTargetType: "bigint"'); expect(ts).toContain('formatTypeExpected: "bigint"'); expect(ts).toContain('rawTargetTypeForLabel: "bigint"'); + expect(ts).not.toContain('operationClass'); expect(call.importRequirements()).toEqual([]); }); }); @@ -285,6 +301,13 @@ describe('SetNotNullCall', () => { expect(call.label).toBe('Set NOT NULL on "user"."email"'); }); + it('is widening, because its precheck fails rather than losing a NULL value', async () => { + const { lowerer } = recordingCheckLowerer(); + const call = new SetNotNullCall('public', 'user', 'email'); + const op = await call.toOp(lowerer); + expect([call.operationClass, op.operationClass]).toEqual(['widening', 'widening']); + }); + it('toOp() throws when no lowerer is provided', async () => { const call = new SetNotNullCall('public', 'user', 'email'); await expect(async () => call.toOp()).rejects.toThrow('createPostgresMigrationPlanner'); diff --git a/packages/3-targets/3-targets/postgres/test/migrations/operation-subjects.test.ts b/packages/3-targets/3-targets/postgres/test/migrations/operation-subjects.test.ts new file mode 100644 index 000000000000..edaa5399aceb --- /dev/null +++ b/packages/3-targets/3-targets/postgres/test/migrations/operation-subjects.test.ts @@ -0,0 +1,122 @@ +import { asNamespaceId, type Contract, coreHash, profileHash } from '@internal/contract/types'; +import { type FieldEventCall, subjectsOfCalls } from '@internal/family-sql/control'; +import type { OpFactoryCall } from '@internal/framework-components/control'; +import { SqlStorage, StorageTable } from '@internal/sql-contract/types'; +import { applicationDomainOf } from '@repo/test-utils'; +import { describe, expect, it } from 'vitest'; +import { postgresCallSubjects } from '../../src/core/migrations/operation-subjects'; +import { PostgresSchema } from '../../src/core/postgres-schema'; + +const text = { dataType: 'pg/text', codecId: 'pg/text@1', nullable: false }; + +const contract: Contract = { + target: 'postgres', + targetFamily: 'sql', + profileHash: profileHash('subjects'), + storage: new SqlStorage({ + storageHash: coreHash('subjects'), + namespaces: { + public: new PostgresSchema({ + id: 'public', + entries: { + table: { + User: new StorageTable({ + columns: { id: text, email: text, bio: text }, + primaryKey: { columns: ['id'] }, + foreignKeys: [], + uniques: [], + indexes: [], + }), + }, + }, + }), + }, + }), + roots: {}, + domain: applicationDomainOf({ + models: { + User: { + fields: {}, + relations: {}, + storage: { + table: 'User', + namespaceId: 'public', + fields: { id: { column: 'id' }, email: { column: 'email' }, bio: { column: 'bio' } }, + }, + }, + }, + }), + capabilities: {}, + extensions: {}, + meta: {}, +}; + +function destructiveCall(factoryName: string): OpFactoryCall { + return { + factoryName, + operationClass: 'destructive', + label: `Drop the search config: ${factoryName}`, + renderTypeScript: () => '', + importRequirements: () => [], + toOp: () => { + throw new Error('not lowered'); + }, + }; +} + +function subjectsOf(calls: readonly OpFactoryCall[], fieldEvents: readonly FieldEventCall[]) { + return subjectsOfCalls( + postgresCallSubjects(calls, { + contract, + fieldEvents: new Map(fieldEvents.map((fieldEvent) => [fieldEvent.call, fieldEvent])), + }), + { fromContract: contract, contract, statements: [] }, + ).dataLoss; +} + +describe('postgresCallSubjects', () => { + it('names the field of each destructive codec hook call', () => { + const onEmail = destructiveCall('dropSearchConfig'); + const onBio = destructiveCall('dropSearchConfig'); + const field = (call: OpFactoryCall, columnName: string): FieldEventCall => ({ + call, + namespaceId: 'public', + tableName: 'User', + columnName, + }); + + expect(subjectsOf([onEmail, onBio], [field(onEmail, 'email'), field(onBio, 'bio')])).toEqual([ + { + operationIndex: 0, + subject: { + kind: 'field', + namespaceId: asNamespaceId('__unbound__'), + model: 'User', + field: 'email', + }, + }, + { + operationIndex: 1, + subject: { + kind: 'field', + namespaceId: asNamespaceId('__unbound__'), + model: 'User', + field: 'bio', + }, + }, + ]); + }); + + it('numbers destructive calls of one unknown factory, never naming them by label', () => { + expect( + subjectsOf( + [destructiveCall('rebuild'), destructiveCall('rebuild'), destructiveCall('purge')], + [], + ), + ).toEqual([ + { operationIndex: 0, subject: { kind: 'storage', name: 'rebuild#1' } }, + { operationIndex: 1, subject: { kind: 'storage', name: 'rebuild#2' } }, + { operationIndex: 2, subject: { kind: 'storage', name: 'purge' } }, + ]); + }); +}); diff --git a/packages/3-targets/3-targets/postgres/test/migrations/planner.data-loss.test.ts b/packages/3-targets/3-targets/postgres/test/migrations/planner.data-loss.test.ts new file mode 100644 index 000000000000..21c357f35da9 --- /dev/null +++ b/packages/3-targets/3-targets/postgres/test/migrations/planner.data-loss.test.ts @@ -0,0 +1,418 @@ +import { asNamespaceId, type Contract, coreHash, profileHash } from '@internal/contract/types'; +import type { CodecControlHooks } from '@internal/family-sql/control'; +import type { ExecuteRequestLowerer } from '@internal/family-sql/control-adapter'; +import { + APP_SPACE_ID, + planOriginOf, + type ResolvedMigrationStatement, +} from '@internal/framework-components/control'; +import { UNBOUND_NAMESPACE_ID } from '@internal/framework-components/ir'; +import { SqlStorage, StorageTable } from '@internal/sql-contract/types'; +import { applicationDomainOf } from '@repo/test-utils'; +import { expectDataLossMatchesDestructive } from '@repo/test-utils/data-loss-expectations'; +import { describe, expect, it } from 'vitest'; +import { createPostgresMigrationPlanner } from '../../src/core/migrations/planner'; +import { postgresContractToSchema } from '../../src/core/migrations/postgres-contract-to-schema'; +import { PostgresRlsEnablement } from '../../src/core/postgres-rls-enablement'; +import { PostgresRlsPolicy } from '../../src/core/postgres-rls-policy'; +import { PostgresSchema } from '../../src/core/postgres-schema'; +import { postgresTypeComponents } from '../postgres-type-lookups'; +import { emailIndex } from './rename-column-fixtures'; + +const stubLowerer: ExecuteRequestLowerer = { + lower: () => ({ sql: 'stub', params: [] }), + renderColumnDefault: async () => '', + lowerToExecuteRequest: async () => ({ sql: 'stub', params: [] }), +}; + +const ALL_CLASSES = { + allowedOperationClasses: ['additive', 'widening', 'destructive', 'data'] as const, +}; +const unbound = asNamespaceId(UNBOUND_NAMESPACE_ID); + +const columnTypes = { + int4: { dataType: 'pg/int4', codecId: 'pg/int4@1', nullable: false }, + int8: { dataType: 'pg/int8', codecId: 'pg/int8@1', nullable: false }, + text: { dataType: 'pg/text', codecId: 'pg/text@1', nullable: false }, + 'text?': { dataType: 'pg/text', codecId: 'pg/text@1', nullable: true }, +} as const; + +interface TableSpec { + readonly columns: Readonly>; + /** The `using` of the table's one policy, which also turns on row-level security. */ + readonly policy?: string; + /** A column with an index named after the table. */ + readonly indexed?: string; + /** The hash of the policy's generated (wire) name; without it the policy has an exact name. */ + readonly policyHash?: string; +} + +function policyOn(table: string, using: string, hash: string | undefined): PostgresRlsPolicy { + return new PostgresRlsPolicy({ + naming: + hash === undefined + ? { kind: 'exact', name: `${table} readers` } + : { kind: 'wire', prefix: `${table}_readers`, hash }, + tableName: table, + namespaceId: 'public', + operation: 'select', + roles: ['app_user'], + using, + permissive: true, + withCheck: undefined, + }); +} + +/** A model per table, named as the table, each field stored in the column of the same name. */ +function contract(seed: string, tables: Readonly>): Contract { + const specs = Object.entries(tables); + const policies = specs.flatMap(([table, spec]) => + spec.policy === undefined ? [] : [policyOn(table, spec.policy, spec.policyHash)], + ); + const schema = new PostgresSchema({ + id: 'public', + entries: { + table: Object.fromEntries( + specs.map(([table, spec]) => [ + table, + new StorageTable({ + columns: Object.fromEntries( + Object.entries(spec.columns).map(([column, type]) => [column, columnTypes[type]]), + ), + primaryKey: { columns: ['id'] }, + foreignKeys: [], + uniques: [], + indexes: spec.indexed === undefined ? [] : [emailIndex(table, spec.indexed)], + }), + ]), + ), + policy: Object.fromEntries(policies.map((policy) => [policy.name, policy])), + rls: Object.fromEntries( + policies.map((policy) => [ + policy.tableName, + new PostgresRlsEnablement({ tableName: policy.tableName, namespaceId: 'public' }), + ]), + ), + }, + }); + return { + target: 'postgres', + targetFamily: 'sql', + profileHash: profileHash(seed), + storage: new SqlStorage({ storageHash: coreHash(seed), namespaces: { public: schema } }), + roots: {}, + domain: applicationDomainOf({ + models: Object.fromEntries( + specs.map(([table, spec]) => [ + table, + { + fields: Object.fromEntries( + Object.keys(spec.columns).map((column) => [ + column, + { nullable: false, type: { kind: 'scalar', codecId: 'pg/text@1' } }, + ]), + ), + relations: {}, + storage: { + table, + namespaceId: 'public', + fields: Object.fromEntries( + Object.keys(spec.columns).map((column) => [column, { column }]), + ), + }, + }, + ]), + ), + }), + capabilities: {}, + extensions: {}, + meta: {}, + }; +} + +async function planned( + from: Contract, + to: Contract, + fromContract: Contract | null = from, + statements: readonly ResolvedMigrationStatement[] = [], + frameworkComponents: typeof postgresTypeComponents = postgresTypeComponents, +) { + const result = createPostgresMigrationPlanner(stubLowerer).plan({ + contract: to, + schema: postgresContractToSchema(from, postgresTypeComponents), + policy: ALL_CLASSES, + fromContract, + origin: planOriginOf(fromContract), + statements, + frameworkComponents, + spaceId: APP_SPACE_ID, + snapshotsImportPath: '../../snapshots', + }); + if (result.kind !== 'success') throw new Error(JSON.stringify(result.conflicts)); + const settled = await Promise.allSettled(result.plan.operations); + expectDataLossMatchesDestructive( + result, + settled.map((entry) => + entry.status === 'fulfilled' ? entry.value : { operationClass: 'data' }, + ), + ); + const labels = settled.map((entry) => + entry.status === 'fulfilled' ? entry.value.label : 'placeholder', + ); + const labelled = (entries: typeof result.dataLoss) => + entries.map(({ operationIndex, subject }) => ({ + operation: labels[operationIndex], + subject, + })); + return { + labels, + dataLoss: labelled(result.dataLoss), + accessWidening: labelled(result.accessWidening), + }; +} + +describe('Postgres planner, data loss', () => { + it('names the model of a dropped table and the field of a dropped column', async () => { + const from = contract('from', { + Legacy: { columns: { id: 'int4' } }, + User: { columns: { id: 'int4', nickname: 'text' } }, + }); + const to = contract('to', { User: { columns: { id: 'int4' } } }); + + expect((await planned(from, to)).dataLoss).toEqual([ + { + operation: 'Drop table "Legacy"', + subject: { kind: 'model', namespaceId: unbound, model: 'Legacy' }, + }, + { + operation: 'Drop column "nickname" from "User"', + subject: { kind: 'field', namespaceId: unbound, model: 'User', field: 'nickname' }, + }, + ]); + }); + + it('names the field a destructive codec-hook call is about, after the control-policy partition', async () => { + const from = contract('from', { User: { columns: { id: 'int4', nickname: 'text' } } }); + const to = contract('to', { User: { columns: { id: 'int4' } } }); + const nickname = { kind: 'field', namespaceId: unbound, model: 'User', field: 'nickname' }; + + const plan = await planned( + from, + to, + from, + [], + withFieldHook(postgresTypeComponents, 'dropped', 'destructive'), + ); + + expect(plan.dataLoss).toEqual([ + { operation: 'Drop column "nickname" from "User"', subject: nickname }, + { operation: 'Codec hook for dropped User.nickname', subject: nickname }, + ]); + }); + + it('names the field of a type change that can change values, and not of a safe widening', async () => { + const from = contract('from', { + User: { columns: { id: 'int4', age: 'text', score: 'int4' } }, + }); + const to = contract('to', { User: { columns: { id: 'int4', age: 'int4', score: 'int8' } } }); + + expect((await planned(from, to)).dataLoss).toEqual([ + { + operation: 'Alter type of "User"."age" to int4', + subject: { kind: 'field', namespaceId: unbound, model: 'User', field: 'age' }, + }, + ]); + }); + + it('names storage by its schema-qualified name when the plan has no origin contract', async () => { + const from = contract('from', { + Legacy: { columns: { id: 'int4' } }, + User: { columns: { id: 'int4', nickname: 'text' } }, + }); + const to = contract('to', { User: { columns: { id: 'int4' } } }); + + expect((await planned(from, to, null)).dataLoss.map(({ subject }) => subject)).toEqual([ + { kind: 'storage', name: 'public.Legacy' }, + { kind: 'storage', name: 'public.User.nickname' }, + ]); + }); + + it('lists a policy drop and disabling row-level security as widening access to the model', async () => { + const from = contract('from', { + User: { columns: { id: 'int4' }, policy: 'true' }, + }); + const to = contract('to', { User: { columns: { id: 'int4' } } }); + const { dataLoss, accessWidening } = await planned(from, to); + + const user = { kind: 'model', namespaceId: unbound, model: 'User' }; + expect({ dataLoss, accessWidening }).toEqual({ + dataLoss: [], + accessWidening: [ + { operation: 'Disable row-level security on "User"', subject: user }, + { operation: 'Drop RLS policy "User readers" on "User"', subject: user }, + ], + }); + }); + + it('leaves out the drop of a generated-name policy the plan replaces under a new hash', async () => { + const from = contract('from', { + User: { columns: { id: 'int4' }, policy: 'true', policyHash: 'aaaaaaaa' }, + }); + const to = contract('to', { + User: { columns: { id: 'int4' }, policy: 'false', policyHash: 'bbbbbbbb' }, + }); + + const { labels, accessWidening } = await planned(from, to); + expect(labels).toEqual( + expect.arrayContaining([expect.stringContaining('User_readers_aaaaaaaa')]), + ); + expect(accessWidening).toEqual([]); + }); + + it('lists the drop of a generated-name policy the plan does not replace', async () => { + const from = contract('from', { + User: { columns: { id: 'int4' }, policy: 'true', policyHash: 'aaaaaaaa' }, + }); + const to = contract('to', { User: { columns: { id: 'int4' } } }); + + const { accessWidening } = await planned(from, to); + expect(accessWidening.map(({ operation }) => operation)).toEqual( + expect.arrayContaining([expect.stringContaining('User_readers_aaaaaaaa')]), + ); + }); + + it('leaves out the drop of a policy the plan replaces', async () => { + const from = contract('from', { User: { columns: { id: 'int4' }, policy: 'true' } }); + const to = contract('to', { User: { columns: { id: 'int4' }, policy: 'false' } }); + + const { dataLoss, accessWidening } = await planned(from, to); + expect({ dataLoss, accessWidening }).toEqual({ dataLoss: [], accessWidening: [] }); + }); + + it('names the origin model and fields of a table a statement renamed earlier in the plan', async () => { + const from = contract('from', { + User: { + columns: { id: 'int4', handle: 'text', nickname: 'text', age: 'text' }, + indexed: 'handle', + }, + }); + const to = contract('to', { + Account: { columns: { id: 'int4', handle: 'text', age: 'int4' }, indexed: 'handle' }, + }); + const renameUser: ResolvedMigrationStatement = { + kind: 'rename', + entity: 'model', + from: { namespaceId: unbound, model: 'User' }, + to: { namespaceId: unbound, model: 'Account' }, + }; + + const result = await planned(from, to, from, [renameUser]); + + expect(result.labels).toEqual([ + 'Rename table "User" to "Account"', + 'Rename primary key "User_pkey" to "Account_pkey" on "Account"', + expect.stringMatching(/^Rename index "User_handle_idx_\w+" to "Account_handle_idx_\w+"/), + 'Drop column "nickname" from "Account"', + 'placeholder', + 'Alter type of "Account"."age" to int4', + ]); + expect(result.dataLoss).toEqual([ + { + operation: 'Drop column "nickname" from "Account"', + subject: { kind: 'field', namespaceId: unbound, model: 'User', field: 'nickname' }, + }, + { + operation: 'Alter type of "Account"."age" to int4', + subject: { kind: 'field', namespaceId: unbound, model: 'User', field: 'age' }, + }, + ]); + }); +}); + +/** A codec hook that adds an operation of `operationClass` for every field of the `on` event. */ +function withFieldHook( + components: typeof postgresTypeComponents, + on: 'added' | 'dropped', + operationClass: 'additive' | 'destructive', +): typeof postgresTypeComponents { + const hooks: CodecControlHooks = { + onFieldEvent: (event, ctx) => + event === on + ? [ + { + factoryName: 'codecHook', + operationClass, + label: `Codec hook for ${on} ${ctx.tableName}.${ctx.fieldName}`, + renderTypeScript: () => 'codecHook()', + importRequirements: () => [], + toOp: () => ({ + id: `codec.${on}.${ctx.tableName}.${ctx.fieldName}`, + label: `Codec hook for ${on} ${ctx.tableName}.${ctx.fieldName}`, + operationClass, + target: { id: 'postgres' }, + precheck: [], + execute: [], + postcheck: [], + }), + }, + ] + : [], + }; + return [ + ...components, + { + kind: 'adapter', + id: 'field-event-hook', + familyId: 'sql', + targetId: 'postgres', + version: '0.0.0', + types: { codecTypes: { controlPlaneHooks: { 'pg/text@1': hooks } } }, + } as (typeof components)[number], + ]; +} + +describe('Postgres planner under db update', () => { + const DB_UPDATE = { allowedOperationClasses: ['additive', 'widening', 'destructive'] as const }; + + async function dbUpdatePlan( + from: Contract, + to: Contract, + fromContract: Contract | null, + ) { + const result = createPostgresMigrationPlanner(stubLowerer).plan({ + contract: to, + schema: postgresContractToSchema(from, postgresTypeComponents), + policy: DB_UPDATE, + fromContract, + origin: null, + statements: [], + frameworkComponents: withFieldHook(postgresTypeComponents, 'added', 'additive'), + spaceId: APP_SPACE_ID, + snapshotsImportPath: '../../snapshots', + }); + if (result.kind !== 'success') throw new Error(JSON.stringify(result.conflicts)); + const operations = (await Promise.all(result.plan.operations)).map( + ({ id, label, operationClass }) => ({ id, label, operationClass }), + ); + return { origin: result.plan.origin, operations }; + } + + it('plans the same operations with and without the origin contract, codec field-event hooks included, and asserts no origin', async () => { + const from = contract('from', { + Legacy: { columns: { id: 'int4' } }, + User: { + columns: { id: 'int4', nickname: 'text', age: 'text', score: 'int4', email: 'text?' }, + }, + }); + const to = contract('to', { + User: { columns: { id: 'int4', age: 'int4', score: 'int8', email: 'text', bio: 'text?' } }, + }); + + const withOrigin = await dbUpdatePlan(from, to, from); + const without = await dbUpdatePlan(from, to, null); + + expect(withOrigin).toEqual(without); + expect(withOrigin.origin).toBeNull(); + expect(withOrigin.operations.length).toBeGreaterThan(0); + }); +}); diff --git a/packages/3-targets/3-targets/postgres/test/postgres-migration-op-builders.test.ts b/packages/3-targets/3-targets/postgres/test/postgres-migration-op-builders.test.ts index bd5cb106317a..854582538a36 100644 --- a/packages/3-targets/3-targets/postgres/test/postgres-migration-op-builders.test.ts +++ b/packages/3-targets/3-targets/postgres/test/postgres-migration-op-builders.test.ts @@ -11,6 +11,7 @@ import { col, lit } from '@internal/sql-relational-core/contract-free'; import { blindCast } from '@internal/utils/casts'; import { describe, expect, it } from 'vitest'; import type { AlterColumnTypeOptions } from '../src/core/migrations/op-factory-call'; +import type { AlterColumnTypeClass } from '../src/core/migrations/operations/columns'; import type { DataTransformOptions } from '../src/core/migrations/operations/data-transform'; import type { CreateIndexExtras } from '../src/core/migrations/operations/indexes'; import type { ForeignKeySpec } from '../src/core/migrations/operations/shared'; @@ -158,6 +159,7 @@ class ExposedMigration extends PostgresMigration { readonly table: string; readonly column: string; readonly options: AlterColumnTypeOptions; + readonly operationClass?: AlterColumnTypeClass; }): Promise { return this.alterColumnType(options); } @@ -546,4 +548,35 @@ describe('PostgresMigration op-builder methods with a ControlStack', () => { expect(op.execute[0]?.description).toBe('drop table "widget"'); expect(typeof op.execute[0]?.sql).toBe('string'); }); + + it('alterColumnType is destructive unless the migration says it is a widening', async () => { + const m = new ExposedMigration(fakeControlStack()); + const options = { + qualifiedTargetType: 'int8', + formatTypeExpected: 'bigint', + rawTargetTypeForLabel: 'int8', + }; + const lossy = await m.callAlterColumnType({ + schema: 'public', + table: 'widget', + column: 'count', + options, + }); + const widening = await m.callAlterColumnType({ + schema: 'public', + table: 'widget', + column: 'count', + options, + operationClass: 'widening', + }); + + expect([lossy.operationClass, widening.operationClass]).toEqual(['destructive', 'widening']); + }); + + it('setNotNull lowers to a widening operation', async () => { + const m = new ExposedMigration(fakeControlStack()); + const op = await m.callSetNotNull({ schema: 'public', table: 'widget', column: 'name' }); + + expect(op.operationClass).toBe('widening'); + }); }); diff --git a/packages/3-targets/3-targets/sqlite/src/core/migrations/op-factory-call.ts b/packages/3-targets/3-targets/sqlite/src/core/migrations/op-factory-call.ts index 5e1938f76783..7480a82cecf1 100644 --- a/packages/3-targets/3-targets/sqlite/src/core/migrations/op-factory-call.ts +++ b/packages/3-targets/3-targets/sqlite/src/core/migrations/op-factory-call.ts @@ -314,9 +314,6 @@ export function indexReplacementCalls( export class RenameTableCall extends SqliteOpFactoryCallNode { readonly factoryName = 'renameTable' as const; - // `widening`: a rename is neither additive creation nor destructive, and the - // class vocabulary has no neutral middle class, so this is the class that - // plans under every allowance set except additive-only init. readonly operationClass = 'widening' as const; readonly oldTableName: string; /** The new name: the table's contract-side identity after the rename. */ @@ -490,17 +487,26 @@ export class RecreateTableCall extends SqliteOpFactoryCallNode { readonly summary: string; readonly postchecks: readonly RecreatePostcheck[]; readonly label: string; + /** + * The columns whose values the copy can change, because their type changes. The planner reads + * them to name what the recreate loses; they never reach the operation or `migration.ts`. + */ + readonly lossyColumns: readonly string[]; - constructor(args: { - tableName: string; - contractTable: SqliteTableSpec; - schemaColumnNames: readonly string[]; - indexes: readonly SqliteIndexSpec[]; - summary: string; - postchecks: readonly RecreatePostcheck[]; - operationClass: MigrationOperationClass; - }) { + constructor( + args: { + tableName: string; + contractTable: SqliteTableSpec; + schemaColumnNames: readonly string[]; + indexes: readonly SqliteIndexSpec[]; + summary: string; + postchecks: readonly RecreatePostcheck[]; + operationClass: MigrationOperationClass; + }, + lossyColumns: readonly string[] = [], + ) { super(); + this.lossyColumns = Object.freeze([...lossyColumns]); this.tableName = args.tableName; this.contractTable = args.contractTable; this.schemaColumnNames = args.schemaColumnNames; diff --git a/packages/3-targets/3-targets/sqlite/src/core/migrations/operation-subjects.ts b/packages/3-targets/3-targets/sqlite/src/core/migrations/operation-subjects.ts new file mode 100644 index 000000000000..592b98afac95 --- /dev/null +++ b/packages/3-targets/3-targets/sqlite/src/core/migrations/operation-subjects.ts @@ -0,0 +1,95 @@ +import { + type CallSubjects, + type FieldEventCall, + fieldEventStorage, + type SubjectStorage, + storageNameOfOperation, + unknownCallNames, +} from '@internal/family-sql/control'; +import type { OpFactoryCall } from '@internal/framework-components/control'; +import { UNBOUND_NAMESPACE_ID } from '@internal/framework-components/ir'; +import { DropColumnCall, DropTableCall, RawSqlCall, RecreateTableCall } from './op-factory-call'; + +function tableTarget(table: string): SubjectStorage { + return { + storageName: table, + table: { namespaceId: UNBOUND_NAMESPACE_ID, table, column: undefined }, + }; +} + +function columnTarget(table: string, column: string): SubjectStorage { + return { + storageName: `${table}.${column}`, + table: { namespaceId: UNBOUND_NAMESPACE_ID, table, column }, + }; +} + +function operationCountOf(call: OpFactoryCall): number { + return 'companions' in call && Array.isArray(call.companions) ? 1 + call.companions.length : 1; +} + +/** + * The columns a recreate loses: those the new table leaves out, whose values the copy drops, and, + * when the recreate is destructive, those whose type it changes. + */ +function recreateLosses(call: RecreateTableCall): readonly string[] { + const kept = new Set(call.contractTable.columns.map((column) => column.name)); + const dropped = call.schemaColumnNames.filter((column) => !kept.has(column)); + return [...dropped, ...(call.operationClass === 'destructive' ? call.lossyColumns : [])]; +} + +/** + * What each call of a SQLite plan loses. A recreate that leaves a column out loses its values + * itself, so a later drop of that column is not listed again. + */ +export function sqliteCallSubjects( + calls: readonly OpFactoryCall[], + fieldEvents: ReadonlyMap, +): readonly CallSubjects[] { + const lost = new Set(); + const unknownNames = unknownCallNames( + calls.filter( + (call) => + call.operationClass === 'destructive' && + !(call instanceof RecreateTableCall) && + !(call instanceof DropColumnCall) && + !(call instanceof DropTableCall) && + !(call instanceof RawSqlCall) && + !fieldEvents.has(call), + ), + ); + return calls.map((call): CallSubjects => { + const operationCount = operationCountOf(call); + if (call instanceof RecreateTableCall) { + const columns = recreateLosses(call); + for (const column of columns) lost.add(`${call.tableName}.${column}`); + const dataLoss = + columns.length === 0 && call.operationClass === 'destructive' + ? [tableTarget(call.tableName)] + : columns.map((column) => columnTarget(call.tableName, column)); + return { operationCount, dataLoss, accessWidening: [] }; + } + if (call instanceof DropColumnCall) { + const alreadyLost = lost.has(`${call.tableName}.${call.columnName}`); + return { + operationCount, + dataLoss: alreadyLost ? [] : [columnTarget(call.tableName, call.columnName)], + accessWidening: [], + }; + } + if (call instanceof DropTableCall) { + return { operationCount, dataLoss: [tableTarget(call.tableName)], accessWidening: [] }; + } + if (call.operationClass !== 'destructive') { + return { operationCount, dataLoss: [], accessWidening: [] }; + } + const fieldEvent = fieldEvents.get(call); + const lossTarget: SubjectStorage = + call instanceof RawSqlCall + ? { storageName: storageNameOfOperation(call.op), table: undefined } + : fieldEvent !== undefined + ? fieldEventStorage(fieldEvent) + : { storageName: unknownNames.get(call) ?? call.factoryName, table: undefined }; + return { operationCount, dataLoss: [lossTarget], accessWidening: [] }; + }); +} diff --git a/packages/3-targets/3-targets/sqlite/src/core/migrations/planner-strategies.ts b/packages/3-targets/3-targets/sqlite/src/core/migrations/planner-strategies.ts index 4f99e02c21cc..102121f0c5c9 100644 --- a/packages/3-targets/3-targets/sqlite/src/core/migrations/planner-strategies.ts +++ b/packages/3-targets/3-targets/sqlite/src/core/migrations/planner-strategies.ts @@ -65,13 +65,18 @@ export type CallMigrationStrategy = ( * all (table/column not-found-or-not-expected, and index issues — those are * standalone ops, never folded into a recreate). * - * Column drift is a single `not-equal` issue now (type AND nullability - * compared together by `SqlColumnIR.isEqualTo`), so this reads both fields - * off the node pair directly rather than trusting a separate issue kind per - * attribute: a type change is always destructive; a pure nullability change - * is destructive when tightening (NOT NULL required) and widening when - * relaxing. + * Column drift is a single `not-equal` issue (type and nullability compared + * together by `SqlColumnIR.isEqualTo`): a type change is destructive, because + * the copy can change values; a nullability change alone is widening, because + * a tightening copy fails on a NULL rather than losing it. */ +/** The column of a column issue, which `classifyNodeIssue` classes destructive only for a type change. */ +function columnNameOf(issue: SchemaDiffIssue): string { + return blindCast( + issue.expected, + ).name; +} + function classifyNodeIssue(issue: SchemaDiffIssue): 'widening' | 'destructive' | null { const node = issue.expected ?? issue.actual; if (node === undefined) return null; @@ -88,10 +93,7 @@ function classifyNodeIssue(issue: SchemaDiffIssue): 'widening' | 'destructive' | const actual = blindCast( issue.actual, ); - if (columnTypeChanged(expected, actual)) return 'destructive'; - // Type is unchanged, so `not-equal` here means only nullability - // differs: relaxing (NOT NULL → nullable) is safe; tightening is not. - return expected.nullable ? 'widening' : 'destructive'; + return columnTypeChanged(expected, actual) ? 'destructive' : 'widening'; } case RelationalSchemaNodeKind.columnDefault: case RelationalSchemaNodeKind.primaryKey: @@ -105,7 +107,8 @@ function classifyNodeIssue(issue: SchemaDiffIssue): 'widening' | 'destructive' | /** * Groups recreate-eligible issues by table, decides per-table operation class - * (destructive wins over widening), and emits one `RecreateTableCall` per + * (destructive wins over widening, and a recreate that leaves a live column + * out of the new table loses its values, so it is destructive), and emits one `RecreateTableCall` per * table. Returns unchanged-or-smaller issue list — issues the strategy * consumed are removed so `mapNodeIssueToCall` doesn't double-handle them. * @@ -115,7 +118,10 @@ function classifyNodeIssue(issue: SchemaDiffIssue): 'widening' | 'destructive' | * issue only carries that attribute's own node, never the whole table. */ export const recreateTableStrategy: CallMigrationStrategy = (issues, ctx) => { - const byTable = new Map(); + const byTable = new Map< + string, + { issues: SchemaDiffIssue[]; hasDestructive: boolean; lossyColumns: string[] } + >(); const consumed = new Set(); for (const issue of issues) { @@ -123,13 +129,17 @@ export const recreateTableStrategy: CallMigrationStrategy = (issues, ctx) => { if (!cls) continue; const tableName = issue.path[1]; if (tableName === undefined) continue; - const entry = byTable.get(tableName); - if (entry) { - entry.issues.push(issue); - if (cls === 'destructive') entry.hasDestructive = true; - } else { - byTable.set(tableName, { issues: [issue], hasDestructive: cls === 'destructive' }); + const entry = byTable.get(tableName) ?? { + issues: [], + hasDestructive: false, + lossyColumns: [], + }; + entry.issues.push(issue); + if (cls === 'destructive') { + entry.hasDestructive = true; + entry.lossyColumns.push(columnNameOf(issue)); } + byTable.set(tableName, entry); consumed.add(issue); } @@ -140,9 +150,11 @@ export const recreateTableStrategy: CallMigrationStrategy = (issues, ctx) => { const expectedTable = ctx.expected.tables[tableName]; const actualTable = ctx.actual.tables[tableName]; if (!expectedTable || !actualTable) continue; - const operationClass: MigrationOperationClass = entry.hasDestructive - ? 'destructive' - : 'widening'; + const leavesColumnsOut = Object.keys(actualTable.columns).some( + (column) => !Object.hasOwn(expectedTable.columns, column), + ); + const operationClass: MigrationOperationClass = + entry.hasDestructive || leavesColumnsOut ? 'destructive' : 'widening'; // Flatten the expected table node to a self-contained spec — the Call // holds pre-rendered SQL fragments only, no schema-IR node. @@ -156,15 +168,18 @@ export const recreateTableStrategy: CallMigrationStrategy = (issues, ctx) => { })); calls.push( - new RecreateTableCall({ - tableName, - contractTable: tableSpec, - schemaColumnNames: Object.keys(actualTable.columns), - indexes, - summary: buildRecreateSummary(tableName, entry.issues), - postchecks: buildRecreatePostchecks(tableName, entry.issues, tableSpec), - operationClass, - }), + new RecreateTableCall( + { + tableName, + contractTable: tableSpec, + schemaColumnNames: Object.keys(actualTable.columns), + indexes, + summary: buildRecreateSummary(tableName, entry.issues), + postchecks: buildRecreatePostchecks(tableName, entry.issues, tableSpec), + operationClass, + }, + entry.lossyColumns, + ), ); } @@ -194,11 +209,10 @@ export const recreateTableStrategy: CallMigrationStrategy = (issues, ctx) => { * recipe slot in strategy order (backfill first, recreate second), which * matches the required execution order. * - * Mirrors Postgres's `nullableTighteningCallStrategy` / `'data'`-class - * gating. When `'data'` is not in the policy (the default `db update` / - * `db init` path), the strategy short-circuits and the recreate alone - * runs with its current destructive-class gating — preserving today's - * behavior where a tightening blows up at runtime if NULLs are present. + * Mirrors Postgres's `nullableTighteningCallStrategy`. When `'data'` is not + * in the policy (the default `db update` / `db init` path), the strategy + * short-circuits and the recreate alone runs, failing at runtime if NULLs + * are present. */ export const nullabilityTighteningBackfillStrategy: CallMigrationStrategy = (issues, ctx) => { if (!ctx.policy.allowedOperationClasses.includes('data')) { diff --git a/packages/3-targets/3-targets/sqlite/src/core/migrations/planner.ts b/packages/3-targets/3-targets/sqlite/src/core/migrations/planner.ts index 88ff43303ae6..3c8b12fe76c6 100644 --- a/packages/3-targets/3-targets/sqlite/src/core/migrations/planner.ts +++ b/packages/3-targets/3-targets/sqlite/src/core/migrations/planner.ts @@ -10,14 +10,17 @@ import type { import { detectTableNameCaseChanges, extractCodecControlHooks, - planFieldEventOperations, + planFieldEventCalls, plannerFailure, planStatements, + subjectsOfCalls, } from '@internal/family-sql/control'; import type { ExecuteRequestLowerer } from '@internal/family-sql/control-adapter'; import type { TargetBoundComponentDescriptor } from '@internal/framework-components/components'; import type { AppliedMigrationStatement, + MigrationAccessChange, + MigrationOperationSubject, MigrationPlanner, MigrationScaffoldContext, PlanOrigin, @@ -42,6 +45,7 @@ import { planIssues, } from './issue-planner'; import type { RenameColumnCall, RenameTableCall } from './op-factory-call'; +import { sqliteCallSubjects } from './operation-subjects'; import { type SqliteMigrationDestinationInfo, TypeScriptRenderableSqliteMigration, @@ -68,6 +72,8 @@ export type SqlitePlanResult = readonly kind: 'success'; readonly plan: TypeScriptRenderableSqliteMigration; readonly appliedStatements: readonly AppliedMigrationStatement[]; + readonly dataLoss: readonly MigrationOperationSubject[]; + readonly accessWidening: readonly MigrationAccessChange[]; } | SqlPlannerFailureResult; @@ -210,13 +216,14 @@ export class SqliteMigrationPlanner // `(tableName, fieldName)` deterministic for byte-stable re-emits. // Hook fires only at the application emitter — extension-space planning // (M2 R2) never reaches this helper. - const fieldEventOps = planFieldEventOperations({ - priorContract: options.fromContract, + const fieldEventCalls = planFieldEventCalls({ + priorContract: options.origin === null ? null : options.fromContract, newContract: options.contract, codecHooks, tableRenames: statements.value.tableRenames, columnRenames: statements.value.columnRenames, }); + const fieldEventOps = fieldEventCalls.map(({ call }) => call); // Codec-emitted calls already conform to `OpFactoryCall` — render + // toOp + importRequirements ride directly through the same emit path // as structural ops, no `RawSqlCall` wrap. @@ -248,6 +255,17 @@ export class SqliteMigrationPlanner this.#lowerer, ), appliedStatements: statements.value.appliedStatements, + ...subjectsOfCalls( + sqliteCallSubjects( + calls, + new Map(fieldEventCalls.map((fieldEvent) => [fieldEvent.call, fieldEvent])), + ), + { + fromContract: options.fromContract, + contract: options.contract, + statements: options.statements, + }, + ), }; } diff --git a/packages/3-targets/3-targets/sqlite/test/migrations/nullability-backfill.test.ts b/packages/3-targets/3-targets/sqlite/test/migrations/nullability-backfill.test.ts index bce118dbc094..d1854ae723d8 100644 --- a/packages/3-targets/3-targets/sqlite/test/migrations/nullability-backfill.test.ts +++ b/packages/3-targets/3-targets/sqlite/test/migrations/nullability-backfill.test.ts @@ -112,7 +112,7 @@ describe('nullability-tightening backfill', async () => { const result = planner.plan({ contract: tightenedEmailContract(), schema: nullableEmailSchema(), - policy: { allowedOperationClasses: ['additive', 'destructive'] }, + policy: { allowedOperationClasses: ['additive', 'widening'] }, fromContract: null, origin: null, statements: [], @@ -127,14 +127,14 @@ describe('nullability-tightening backfill', async () => { const ops = await Promise.all(result.plan.operations); expect(ops).toHaveLength(1); expect(ops[0]?.id).toBe('recreateTable.users'); - expect(ops[0]?.operationClass).toBe('destructive'); + expect(ops[0]?.operationClass).toBe('widening'); }); it("with 'data' allowed, emits a backfill data-transform stub before the recreate", async () => { const result = planner.plan({ contract: tightenedEmailContract(), schema: nullableEmailSchema(), - policy: { allowedOperationClasses: ['additive', 'destructive', 'data'] }, + policy: { allowedOperationClasses: ['additive', 'widening', 'data'] }, fromContract: null, origin: null, statements: [], diff --git a/packages/3-targets/3-targets/sqlite/test/migrations/op-factory-call.test.ts b/packages/3-targets/3-targets/sqlite/test/migrations/op-factory-call.test.ts index 84faa69ee248..da3eb2e11dfa 100644 --- a/packages/3-targets/3-targets/sqlite/test/migrations/op-factory-call.test.ts +++ b/packages/3-targets/3-targets/sqlite/test/migrations/op-factory-call.test.ts @@ -427,6 +427,30 @@ describe('RecreateTableCall', () => { expect(op.postcheck.some((s) => s.description.includes('type'))).toBe(true); }); + it('keeps its lossy columns out of the operation and the rendered migration', async () => { + const args = { + tableName: 'user', + contractTable: tableSpec([ + colSpec({ name: 'id', typeSql: 'INTEGER', nullable: false }), + colSpec({ name: 'age', typeSql: 'INTEGER', nullable: true }), + ]), + schemaColumnNames: ['id', 'age'], + indexes: [], + summary: 'Recreates table user to apply schema changes: type mismatch on age', + postchecks: [], + operationClass: 'destructive' as const, + }; + const withLossy = new RecreateTableCall(args, ['age']); + const without = new RecreateTableCall(args); + + expect(withLossy.lossyColumns).toEqual(['age']); + expect(without.lossyColumns).toEqual([]); + expect(withLossy.renderTypeScript()).toBe(without.renderTypeScript()); + expect(JSON.stringify(await withLossy.toOp(stubLowerer('CHECK SQL')))).toBe( + JSON.stringify(await without.toOp(stubLowerer('CHECK SQL'))), + ); + }); + it('writes each column default the adapter renders, and checks the recreated table carries it', async () => { const lowerer: ExecuteRequestLowerer = { ...stubLowerer('CHECK SQL'), diff --git a/packages/3-targets/3-targets/sqlite/test/migrations/operation-subjects.test.ts b/packages/3-targets/3-targets/sqlite/test/migrations/operation-subjects.test.ts new file mode 100644 index 000000000000..3e326cf0cce2 --- /dev/null +++ b/packages/3-targets/3-targets/sqlite/test/migrations/operation-subjects.test.ts @@ -0,0 +1,123 @@ +import { asNamespaceId, type Contract, coreHash, profileHash } from '@internal/contract/types'; +import { type FieldEventCall, subjectsOfCalls } from '@internal/family-sql/control'; +import type { OpFactoryCall } from '@internal/framework-components/control'; +import { UNBOUND_NAMESPACE_ID } from '@internal/framework-components/ir'; +import { SqlStorage, StorageTable } from '@internal/sql-contract/types'; +import { applicationDomainOf } from '@repo/test-utils'; +import { describe, expect, it } from 'vitest'; +import { sqliteCallSubjects } from '../../src/core/migrations/operation-subjects'; +import { sqliteCreateNamespace } from '../../src/core/sqlite-unbound-database'; + +const text = { dataType: 'sqlite/text', codecId: 'sqlite/text@1', nullable: false }; + +const contract: Contract = { + target: 'sqlite', + targetFamily: 'sql', + profileHash: profileHash('subjects'), + storage: new SqlStorage({ + storageHash: coreHash('subjects'), + namespaces: { + [UNBOUND_NAMESPACE_ID]: sqliteCreateNamespace({ + id: UNBOUND_NAMESPACE_ID, + entries: { + table: { + User: new StorageTable({ + columns: { id: text, email: text, bio: text }, + primaryKey: { columns: ['id'] }, + foreignKeys: [], + uniques: [], + indexes: [], + }), + }, + }, + }), + }, + }), + roots: {}, + domain: applicationDomainOf({ + models: { + User: { + fields: {}, + relations: {}, + storage: { + table: 'User', + namespaceId: UNBOUND_NAMESPACE_ID, + fields: { id: { column: 'id' }, email: { column: 'email' }, bio: { column: 'bio' } }, + }, + }, + }, + }), + capabilities: {}, + extensions: {}, + meta: {}, +}; + +function destructiveCall(factoryName: string): OpFactoryCall { + return { + factoryName, + operationClass: 'destructive', + label: `Drop the search config: ${factoryName}`, + renderTypeScript: () => '', + importRequirements: () => [], + toOp: () => { + throw new Error('not lowered'); + }, + }; +} + +function subjectsOf(calls: readonly OpFactoryCall[], fieldEvents: readonly FieldEventCall[]) { + return subjectsOfCalls( + sqliteCallSubjects( + calls, + new Map(fieldEvents.map((fieldEvent) => [fieldEvent.call, fieldEvent])), + ), + { fromContract: contract, contract, statements: [] }, + ).dataLoss; +} + +describe('sqliteCallSubjects', () => { + it('names the field of each destructive codec hook call', () => { + const onEmail = destructiveCall('dropSearchConfig'); + const onBio = destructiveCall('dropSearchConfig'); + const field = (call: OpFactoryCall, columnName: string): FieldEventCall => ({ + call, + namespaceId: UNBOUND_NAMESPACE_ID, + tableName: 'User', + columnName, + }); + + expect(subjectsOf([onEmail, onBio], [field(onEmail, 'email'), field(onBio, 'bio')])).toEqual([ + { + operationIndex: 0, + subject: { + kind: 'field', + namespaceId: asNamespaceId('__unbound__'), + model: 'User', + field: 'email', + }, + }, + { + operationIndex: 1, + subject: { + kind: 'field', + namespaceId: asNamespaceId('__unbound__'), + model: 'User', + field: 'bio', + }, + }, + ]); + }); + + it('numbers destructive calls of one unknown factory, never naming them by label', () => { + expect( + subjectsOf( + [destructiveCall('rebuild'), destructiveCall('rebuild'), destructiveCall('purge')], + [], + ), + ).toEqual([ + { operationIndex: 0, subject: { kind: 'storage', name: 'rebuild#1' } }, + { operationIndex: 1, subject: { kind: 'storage', name: 'rebuild#2' } }, + { operationIndex: 2, subject: { kind: 'storage', name: 'purge' } }, + ]); + }); +}); diff --git a/packages/3-targets/3-targets/sqlite/test/migrations/planner-strategies.test.ts b/packages/3-targets/3-targets/sqlite/test/migrations/planner-strategies.test.ts index 2025750d83e2..1040e35c292c 100644 --- a/packages/3-targets/3-targets/sqlite/test/migrations/planner-strategies.test.ts +++ b/packages/3-targets/3-targets/sqlite/test/migrations/planner-strategies.test.ts @@ -157,31 +157,98 @@ describe('recreateTableStrategy', () => { expect((result.calls[0] as RecreateTableCall).operationClass).toBe('widening'); }); - it('relaxing nullability (NOT NULL → nullable) is widening, tightening is destructive', () => { + it('classifies a recreate for a nullability change alone as widening, in either direction', () => { const ctx = makeContext({ expected: new SqlSchemaIR({ tables: { user: expectedUserTable } }), actual: new SqlSchemaIR({ tables: { user: actualUserTable } }), }); + const nullabilityChange = (expectedNullable: boolean) => + issue({ + path: ['database', 'user', 'column:email'], + expected: expectedColumn({ name: 'email', nativeType: 'TEXT', nullable: expectedNullable }), + actual: actualColumn({ name: 'email', nativeType: 'TEXT', nullable: !expectedNullable }), + }); + + const classes = [true, false].map((expectedNullable) => { + const result = recreateTableStrategy([nullabilityChange(expectedNullable)], ctx); + return result.kind === 'match' + ? (result.calls[0] as RecreateTableCall).operationClass + : result.kind; + }); + expect(classes).toEqual(['widening', 'widening']); + }); + + it('classifies a recreate that leaves a live column out as destructive', () => { + const actualWithNickname = table({ + name: 'user', + columns: { + id: actualColumn({ name: 'id', nativeType: 'INTEGER', nullable: false }), + email: actualColumn({ name: 'email', nativeType: 'TEXT', nullable: false }), + nickname: actualColumn({ name: 'nickname', nativeType: 'TEXT', nullable: true }), + }, + primaryKey: primaryKey(['id']), + }); const relaxing = issue({ path: ['database', 'user', 'column:email'], expected: expectedColumn({ name: 'email', nativeType: 'TEXT', nullable: true }), actual: actualColumn({ name: 'email', nativeType: 'TEXT', nullable: false }), }); - const widening = recreateTableStrategy([relaxing], ctx); - expect(widening.kind).toBe('match'); - if (widening.kind !== 'match') return; - expect((widening.calls[0] as RecreateTableCall).operationClass).toBe('widening'); + const result = recreateTableStrategy( + [relaxing], + makeContext({ + expected: new SqlSchemaIR({ tables: { user: expectedUserTable } }), + actual: new SqlSchemaIR({ tables: { user: actualWithNickname } }), + }), + ); - const tightening = issue({ + expect(result.kind === 'match' && (result.calls[0] as RecreateTableCall).operationClass).toBe( + 'destructive', + ); + }); + + it('records the columns whose type changes as the lossy columns of the recreate', () => { + const ctx = makeContext({ + expected: new SqlSchemaIR({ tables: { user: expectedUserTable } }), + actual: new SqlSchemaIR({ tables: { user: actualUserTable } }), + }); + const typeChange = issue({ + path: ['database', 'user', 'column:email'], + expected: expectedColumn({ name: 'email', nativeType: 'TEXT', nullable: true }), + actual: actualColumn({ name: 'email', nativeType: 'INTEGER', nullable: true }), + }); + const pkDrift = issue({ + path: ['database', 'user', 'primary-key'], + expected: primaryKey(['id']), + actual: primaryKey(['id', 'email']), + }); + + const lossy = recreateTableStrategy([typeChange, pkDrift], ctx); + const lossless = recreateTableStrategy([pkDrift], ctx); + + expect( + [lossy, lossless].map((result) => + result.kind === 'match' ? (result.calls[0] as RecreateTableCall).lossyColumns : result.kind, + ), + ).toEqual([['email'], []]); + }); + + it('classifies a recreate for a nullability tightening and a type change as destructive', () => { + const ctx = makeContext({ + expected: new SqlSchemaIR({ tables: { user: expectedUserTable } }), + actual: new SqlSchemaIR({ tables: { user: actualUserTable } }), + }); + const tighteningAndTypeChange = issue({ path: ['database', 'user', 'column:email'], expected: expectedColumn({ name: 'email', nativeType: 'TEXT', nullable: false }), - actual: actualColumn({ name: 'email', nativeType: 'TEXT', nullable: true }), + actual: actualColumn({ name: 'email', nativeType: 'INTEGER', nullable: true }), }); - const destructive = recreateTableStrategy([tightening], ctx); - expect(destructive.kind).toBe('match'); - if (destructive.kind !== 'match') return; - expect((destructive.calls[0] as RecreateTableCall).operationClass).toBe('destructive'); + + const result = recreateTableStrategy([tighteningAndTypeChange], ctx); + + expect(result.kind).toBe('match'); + if (result.kind !== 'match') return; + expect((result.calls[0] as RecreateTableCall).operationClass).toBe('destructive'); }); it('groups issues by table and emits one RecreateTableCall per affected table', () => { diff --git a/packages/3-targets/3-targets/sqlite/test/migrations/planner.data-loss.test.ts b/packages/3-targets/3-targets/sqlite/test/migrations/planner.data-loss.test.ts new file mode 100644 index 000000000000..407e90b1a83d --- /dev/null +++ b/packages/3-targets/3-targets/sqlite/test/migrations/planner.data-loss.test.ts @@ -0,0 +1,325 @@ +import { asNamespaceId, type Contract, coreHash, profileHash } from '@internal/contract/types'; +import type { CodecControlHooks } from '@internal/family-sql/control'; +import { + APP_SPACE_ID, + planOriginOf, + type ResolvedMigrationStatement, +} from '@internal/framework-components/control'; +import { UNBOUND_NAMESPACE_ID } from '@internal/framework-components/ir'; +import { SqlStorage, StorageTable } from '@internal/sql-contract/types'; +import { applicationDomainOf } from '@repo/test-utils'; +import { expectDataLossMatchesDestructive } from '@repo/test-utils/data-loss-expectations'; +import { describe, expect, it } from 'vitest'; +import { sqliteContractToSchema } from '../../src/core/migrations/diff-database-schema'; +import { createSqliteMigrationPlanner } from '../../src/core/migrations/planner'; +import { sqliteCreateNamespace } from '../../src/core/sqlite-unbound-database'; +import { sqliteTestComponents, sqliteTestTypes } from '../sqlite-test-types'; +import { HANDLE_INDEX_HASH, handleIndex, stubLowerer } from './rename-table-fixtures'; + +const unbound = asNamespaceId(UNBOUND_NAMESPACE_ID); +const ALL_CLASSES = { allowedOperationClasses: ['additive', 'widening', 'destructive'] as const }; + +const columnTypes = { + text: { dataType: 'sqlite/text', codecId: 'sqlite/text@1', nullable: false }, + integer: { dataType: 'sqlite/integer', codecId: 'sqlite/integer@1', nullable: false }, + 'text?': { dataType: 'sqlite/text', codecId: 'sqlite/text@1', nullable: true }, +} as const; + +type ColumnType = keyof typeof columnTypes; + +/** A model per table, each field stored in the column of the same name. */ +function contract( + seed: string, + tables: Readonly>>>, + indexed: ReadonlySet = new Set(), +): Contract { + return { + target: 'sqlite', + targetFamily: 'sql', + profileHash: profileHash(seed), + storage: new SqlStorage({ + storageHash: coreHash(seed), + namespaces: { + [UNBOUND_NAMESPACE_ID]: sqliteCreateNamespace({ + id: UNBOUND_NAMESPACE_ID, + entries: { + table: Object.fromEntries( + Object.entries(tables).map(([table, columns]) => [ + table, + new StorageTable({ + columns: Object.fromEntries( + Object.entries(columns).map(([column, type]) => [column, columnTypes[type]]), + ), + primaryKey: { columns: ['id'] }, + uniques: [], + indexes: indexed.has(table) ? [handleIndex(table)] : [], + foreignKeys: [], + }), + ]), + ), + }, + }), + }, + }), + roots: {}, + domain: applicationDomainOf({ + models: Object.fromEntries( + Object.entries(tables).map(([table, columns]) => [ + table, + { + fields: Object.fromEntries( + Object.keys(columns).map((column) => [ + column, + { nullable: false, type: { kind: 'scalar', codecId: 'sqlite/text@1' } }, + ]), + ), + relations: {}, + storage: { + table, + namespaceId: UNBOUND_NAMESPACE_ID, + fields: Object.fromEntries( + Object.keys(columns).map((column) => [column, { column }]), + ), + }, + }, + ]), + ), + }), + capabilities: {}, + extensions: {}, + meta: {}, + }; +} + +async function planned( + from: Contract, + to: Contract, + options: { + readonly fromContract?: Contract | null; + readonly statements?: readonly ResolvedMigrationStatement[]; + } = {}, +) { + const fromContract = options.fromContract === undefined ? from : options.fromContract; + const result = createSqliteMigrationPlanner(stubLowerer).plan({ + contract: to, + schema: sqliteContractToSchema(from, sqliteTestTypes), + policy: ALL_CLASSES, + fromContract, + origin: planOriginOf(fromContract), + statements: options.statements ?? [], + frameworkComponents: sqliteTestComponents, + spaceId: APP_SPACE_ID, + snapshotsImportPath: '../../snapshots', + }); + if (result.kind !== 'success') throw new Error(JSON.stringify(result.conflicts)); + const operations = await Promise.all(result.plan.operations); + const originModel = (table: string) => + (options.statements ?? []).find( + (statement) => statement.entity === 'model' && statement.to.model === table, + )?.from.model ?? table; + expectDataLossMatchesDestructive(result, operations, { + subjectOf: ({ target }) => + target.details?.objectType === 'column' && target.details.table !== undefined + ? { + kind: 'field', + namespaceId: unbound, + model: originModel(target.details.table), + field: target.details.name, + } + : undefined, + }); + const labels = operations.map((op) => op.label); + return { + labels, + dataLoss: result.dataLoss.map(({ operationIndex, subject }) => ({ + operation: labels[operationIndex], + subject, + })), + accessWidening: result.accessWidening, + classes: Object.fromEntries(operations.map((op) => [op.label, op.operationClass])), + }; +} + +describe('SQLite planner, data loss', () => { + it('names the model of a dropped table and the field of a dropped column', async () => { + const from = contract('from', { + Legacy: { id: 'integer' }, + User: { id: 'integer', email: 'text', nickname: 'text' }, + }); + const to = contract('to', { User: { id: 'integer', email: 'text' } }); + + const { dataLoss, accessWidening } = await planned(from, to); + expect({ dataLoss, accessWidening }).toEqual({ + dataLoss: [ + { + operation: 'Drop column nickname on User', + subject: { kind: 'field', namespaceId: unbound, model: 'User', field: 'nickname' }, + }, + { + operation: 'Drop table Legacy', + subject: { kind: 'model', namespaceId: unbound, model: 'Legacy' }, + }, + ], + accessWidening: [], + }); + }); + + it('names the field whose type a recreate changes, at the recreate', async () => { + const from = contract('from', { User: { id: 'integer', age: 'text' } }); + const to = contract('to', { User: { id: 'integer', age: 'integer' } }); + + expect((await planned(from, to)).dataLoss).toEqual([ + { + operation: 'Recreate table User', + subject: { kind: 'field', namespaceId: unbound, model: 'User', field: 'age' }, + }, + ]); + }); + + it('names a column a recreate leaves out once, at the recreate, which is destructive', async () => { + const from = contract('from', { User: { id: 'integer', email: 'text?', nickname: 'text' } }); + const to = contract('to', { User: { id: 'integer', email: 'text' } }); + const result = await planned(from, to); + + expect(result.classes['Recreate table User']).toBe('destructive'); + expect(result.dataLoss).toEqual([ + { + operation: 'Recreate table User', + subject: { kind: 'field', namespaceId: unbound, model: 'User', field: 'nickname' }, + }, + ]); + }); + + it('names storage by the names the database has when the plan has no origin contract', async () => { + const from = contract('from', { + Legacy: { id: 'integer' }, + User: { id: 'integer', email: 'text', nickname: 'text' }, + }); + const to = contract('to', { User: { id: 'integer', email: 'text' } }); + + expect((await planned(from, to, { fromContract: null })).dataLoss).toEqual([ + { + operation: 'Drop column nickname on User', + subject: { kind: 'storage', name: 'User.nickname' }, + }, + { operation: 'Drop table Legacy', subject: { kind: 'storage', name: 'Legacy' } }, + ]); + }); + + it('names the origin model and fields of a table a statement renamed earlier in the plan', async () => { + const from = contract( + 'from', + { User: { id: 'integer', handle: 'text', nickname: 'text', age: 'text' } }, + new Set(['User']), + ); + const to = contract( + 'to', + { Account: { id: 'integer', handle: 'text', age: 'integer' } }, + new Set(['Account']), + ); + const renameUser: ResolvedMigrationStatement = { + kind: 'rename', + entity: 'model', + from: { namespaceId: unbound, model: 'User' }, + to: { namespaceId: unbound, model: 'Account' }, + }; + + const result = await planned(from, to, { statements: [renameUser] }); + + expect(result.labels).toEqual([ + 'Rename table User to Account', + `Drop index User_handle_idx_${HANDLE_INDEX_HASH} on Account`, + `Create index Account_handle_idx_${HANDLE_INDEX_HASH} on Account`, + 'Recreate table Account', + 'Drop column nickname on Account', + ]); + expect(result.dataLoss).toEqual([ + { + operation: 'Recreate table Account', + subject: { kind: 'field', namespaceId: unbound, model: 'User', field: 'nickname' }, + }, + { + operation: 'Recreate table Account', + subject: { kind: 'field', namespaceId: unbound, model: 'User', field: 'age' }, + }, + ]); + }); +}); + +/** A codec hook that adds an operation for every field it is told was added. */ +function withAddedFieldHook(components: typeof sqliteTestComponents): typeof sqliteTestComponents { + const hooks: CodecControlHooks = { + onFieldEvent: (event, ctx) => + event === 'added' + ? [ + { + factoryName: 'codecAdded', + operationClass: 'additive', + label: `Codec hook for added ${ctx.tableName}.${ctx.fieldName}`, + renderTypeScript: () => 'codecAdded()', + importRequirements: () => [], + toOp: () => ({ + id: `codec.added.${ctx.tableName}.${ctx.fieldName}`, + label: `Codec hook for added ${ctx.tableName}.${ctx.fieldName}`, + operationClass: 'additive', + target: { id: 'sqlite' }, + precheck: [], + execute: [], + postcheck: [], + }), + }, + ] + : [], + }; + return [ + ...components, + { + kind: 'adapter', + id: 'field-event-hook', + familyId: 'sql', + targetId: 'sqlite', + version: '0.0.0', + types: { codecTypes: { controlPlaneHooks: { 'sqlite/text@1': hooks } } }, + } as (typeof components)[number], + ]; +} + +describe('SQLite planner under db update', () => { + async function dbUpdatePlan( + from: Contract, + to: Contract, + fromContract: Contract | null, + ) { + const result = createSqliteMigrationPlanner(stubLowerer).plan({ + contract: to, + schema: sqliteContractToSchema(from, sqliteTestTypes), + policy: ALL_CLASSES, + fromContract, + origin: null, + statements: [], + frameworkComponents: withAddedFieldHook(sqliteTestComponents), + spaceId: APP_SPACE_ID, + snapshotsImportPath: '../../snapshots', + }); + if (result.kind !== 'success') throw new Error(JSON.stringify(result.conflicts)); + const operations = (await Promise.all(result.plan.operations)).map( + ({ id, label, operationClass }) => ({ id, label, operationClass }), + ); + return { origin: result.plan.origin, operations }; + } + + it('plans the same operations with and without the origin contract, codec field-event hooks included, and asserts no origin', async () => { + const from = contract('from', { + Legacy: { id: 'integer' }, + User: { id: 'integer', email: 'text?', nickname: 'text', age: 'text' }, + }); + const to = contract('to', { User: { id: 'integer', email: 'text', age: 'integer' } }); + + const withOrigin = await dbUpdatePlan(from, to, from); + const without = await dbUpdatePlan(from, to, null); + + expect(withOrigin).toEqual(without); + expect(withOrigin.origin).toBeNull(); + expect(withOrigin.operations.length).toBeGreaterThan(0); + }); +}); diff --git a/packages/3-targets/6-adapters/postgres/test/migrations/db-init-update.cli.integration.test.ts b/packages/3-targets/6-adapters/postgres/test/migrations/db-init-update.cli.integration.test.ts index 36babf6906e8..2df20c890f31 100644 --- a/packages/3-targets/6-adapters/postgres/test/migrations/db-init-update.cli.integration.test.ts +++ b/packages/3-targets/6-adapters/postgres/test/migrations/db-init-update.cli.integration.test.ts @@ -371,6 +371,11 @@ describe('db init / db update aggregate pipeline (CLI) - postgres', { concurrent } const updateResult = await executeDbUpdate({ + answerQuestions: async (questions) => + questions.map((question) => ({ + verb: question.verbs.includes('allow') ? ('allow' as const) : ('delete' as const), + text: question.subject, + })), driver: drv, adapter: controlAdapter, familyInstance, @@ -467,6 +472,11 @@ describe('db init / db update aggregate pipeline (CLI) - postgres', { concurrent } const updateResult = await executeDbUpdate({ + answerQuestions: async (questions) => + questions.map((question) => ({ + verb: question.verbs.includes('allow') ? ('allow' as const) : ('delete' as const), + text: question.subject, + })), driver: drv, adapter: controlAdapter, familyInstance, diff --git a/packages/3-targets/6-adapters/postgres/test/migrations/op-factory-call.lowering.test.ts b/packages/3-targets/6-adapters/postgres/test/migrations/op-factory-call.lowering.test.ts index 27f353565820..740f9e766123 100644 --- a/packages/3-targets/6-adapters/postgres/test/migrations/op-factory-call.lowering.test.ts +++ b/packages/3-targets/6-adapters/postgres/test/migrations/op-factory-call.lowering.test.ts @@ -154,7 +154,7 @@ describe('renderOps', () => { }, { id: 'alterNullability.setNotNull.user.email', - operationClass: 'destructive', + operationClass: 'widening', details: schemaObject('column', 'email', 'user'), }, { diff --git a/packages/3-targets/6-adapters/sqlite/test/migrations/db-init-update.cli.test.ts b/packages/3-targets/6-adapters/sqlite/test/migrations/db-init-update.cli.test.ts index c81432a8a11c..2134af0f6f11 100644 --- a/packages/3-targets/6-adapters/sqlite/test/migrations/db-init-update.cli.test.ts +++ b/packages/3-targets/6-adapters/sqlite/test/migrations/db-init-update.cli.test.ts @@ -372,6 +372,11 @@ describe('db init / db update aggregate pipeline (CLI) - sqlite', { } const updateResult = await executeDbUpdate({ + answerQuestions: async (questions) => + questions.map((question) => ({ + verb: question.verbs.includes('allow') ? ('allow' as const) : ('delete' as const), + text: question.subject, + })), driver: testDb!.driver, adapter: controlAdapter, familyInstance, @@ -583,6 +588,11 @@ describe('db init / db update aggregate pipeline (CLI) - sqlite', { // a no-op — proves the aggregate path's marker / hash check still // short-circuits when n=1. const updateResult = await executeDbUpdate({ + answerQuestions: async (questions) => + questions.map((question) => ({ + verb: question.verbs.includes('allow') ? ('allow' as const) : ('delete' as const), + text: question.subject, + })), driver: testDb!.driver, adapter: controlAdapter, familyInstance, @@ -660,6 +670,11 @@ describe('db init / db update aggregate pipeline (CLI) - sqlite', { } const updateResult = await executeDbUpdate({ + answerQuestions: async (questions) => + questions.map((question) => ({ + verb: question.verbs.includes('allow') ? ('allow' as const) : ('delete' as const), + text: question.subject, + })), driver: testDb!.driver, adapter: controlAdapter, familyInstance, diff --git a/packages/9-public/@prisma/orm-toolchain/package.json b/packages/9-public/@prisma/orm-toolchain/package.json index d08844035299..92dfb761b13f 100644 --- a/packages/9-public/@prisma/orm-toolchain/package.json +++ b/packages/9-public/@prisma/orm-toolchain/package.json @@ -49,7 +49,7 @@ "typescript": "catalog:" }, "peerDependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "typescript": ">=5.9", "vite": "^7.0.0 || ^8.0.0" }, diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index d2c087df3f0d..3e37f65012fe 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -201,8 +201,8 @@ importers: version: 8.21.3 devDependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/tsconfig': specifier: workspace:8.0.0-rc.16 version: link:../../packages/0-config/tsconfig @@ -238,8 +238,8 @@ importers: version: 2.24.0(express@5.2.1)(hono@4.11.4)(next@16.3.0(@types/node@26.1.2)(react-dom@19.2.8(react@19.2.8))(react@19.2.8))(react-router@8.3.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8))(react@19.2.8)(vite@8.1.4(@types/node@26.1.2)(esbuild@0.28.2)(jiti@2.7.0)(tsx@4.23.12)(yaml@2.9.0)) devDependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@prisma/compute-sdk': specifier: ^0.43.0 version: 0.43.0(@prisma/management-api-sdk@1.76.0)(encoding@0.1.13)(rollup@4.59.0) @@ -293,8 +293,8 @@ importers: version: 8.22.0 devDependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@prisma/dev': specifier: 'catalog:' version: 0.25.2(typescript@5.9.3)(vite@8.1.4(@types/node@26.1.2)(esbuild@0.28.2)(jiti@2.7.0)(tsx@4.23.12)(yaml@2.9.0)) @@ -345,8 +345,8 @@ importers: version: 7.5.0(socks@2.8.9) devDependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/tsconfig': specifier: workspace:8.0.0-rc.16 version: link:../../packages/0-config/tsconfig @@ -388,8 +388,8 @@ importers: version: 19.2.8(react@19.2.8) devDependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/test-utils': specifier: workspace:8.0.0-rc.16 version: link:../../test/utils @@ -434,8 +434,8 @@ importers: version: link:../../packages/9-public/@prisma/orm-postgres devDependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/test-utils': specifier: workspace:8.0.0-rc.16 version: link:../../test/utils @@ -471,8 +471,8 @@ importers: version: 8.22.0 devDependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/test-utils': specifier: workspace:8.0.0-rc.16 version: link:../../test/utils @@ -517,8 +517,8 @@ importers: specifier: 5.20260804.1 version: 5.20260804.1 '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/tsconfig': specifier: workspace:8.0.0-rc.16 version: link:../../packages/0-config/tsconfig @@ -569,8 +569,8 @@ importers: version: 19.2.8(react@19.2.8) devDependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@prisma/dev': specifier: 'catalog:' version: 0.25.2(typescript@5.9.3)(vite@8.1.4(@types/node@26.1.2)(esbuild@0.28.2)(jiti@2.7.0)(tsx@4.23.12)(yaml@2.9.0)) @@ -633,8 +633,8 @@ importers: version: 17.4.2 devDependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/tsconfig': specifier: workspace:8.0.0-rc.16 version: link:../../packages/0-config/tsconfig @@ -682,8 +682,8 @@ importers: version: 5.0.0(leaflet@1.9.4)(react-dom@19.2.8(react@19.2.8))(react@19.2.8) devDependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/test-utils': specifier: workspace:8.0.0-rc.16 version: link:../../test/utils @@ -734,8 +734,8 @@ importers: version: 8.22.0 devDependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@prisma/dev': specifier: 'catalog:' version: 0.25.2(typescript@5.9.3)(vite@8.1.4(@types/node@26.1.2)(esbuild@0.28.2)(jiti@2.7.0)(tsx@4.23.12)(yaml@2.9.0)) @@ -798,8 +798,8 @@ importers: version: 8.3.0(react-dom@19.2.8(react@19.2.8))(react@19.2.8) devDependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@prisma/dev': specifier: 'catalog:' version: 0.25.2(typescript@5.9.3)(vite@8.1.4(@types/node@26.1.2)(esbuild@0.28.2)(jiti@2.7.0)(tsx@4.23.12)(yaml@2.9.0)) @@ -904,8 +904,8 @@ importers: version: 4.3.3 devDependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/test-utils': specifier: workspace:8.0.0-rc.16 version: link:../../test/utils @@ -953,8 +953,8 @@ importers: version: 8.22.0 devDependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/test-utils': specifier: workspace:8.0.0-rc.16 version: link:../../test/utils @@ -1428,8 +1428,8 @@ importers: specifier: workspace:8.0.0-rc.16 version: link:../../../2-sql/5-runtime '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/test-utils': specifier: workspace:8.0.0-rc.16 version: link:../../../../test/utils @@ -1547,8 +1547,8 @@ importers: version: 0.2.17 devDependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/test-utils': specifier: workspace:8.0.0-rc.16 version: link:../../../../test/utils @@ -3246,8 +3246,8 @@ importers: specifier: workspace:8.0.0-rc.16 version: link:../../3-targets/3-targets/postgres '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/test-utils': specifier: workspace:8.0.0-rc.16 version: link:../../../test/utils @@ -3334,8 +3334,8 @@ importers: specifier: workspace:8.0.0-rc.16 version: link:../../2-sql/2-authoring/contract-ts '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/test-utils': specifier: workspace:8.0.0-rc.16 version: link:../../../test/utils @@ -3416,8 +3416,8 @@ importers: specifier: workspace:8.0.0-rc.16 version: link:../../2-sql/2-authoring/contract-ts '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/test-utils': specifier: workspace:8.0.0-rc.16 version: link:../../../test/utils @@ -3792,8 +3792,8 @@ importers: specifier: workspace:8.0.0-rc.16 version: link:../../2-sql/2-authoring/contract-psl '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/test-utils': specifier: workspace:8.0.0-rc.16 version: link:../../../test/utils @@ -5333,8 +5333,8 @@ importers: packages/9-public/@prisma/orm-toolchain: dependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@prisma/orm-framework': specifier: workspace:8.0.0-rc.16 version: link:../orm-framework @@ -5479,8 +5479,8 @@ importers: test/e2e/framework/test/fixtures: dependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@prisma/orm-extension-arktype-json': specifier: workspace:8.0.0-rc.16 version: link:../../../../../packages/9-public/@prisma/orm-extension-arktype-json @@ -5497,8 +5497,8 @@ importers: test/e2e/framework/test/sqlite/fixtures: dependencies: '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@prisma/orm-sqlite': specifier: workspace:8.0.0-rc.16 version: link:../../../../../../packages/9-public/@prisma/orm-sqlite @@ -5684,8 +5684,8 @@ importers: specifier: workspace:8.0.0-rc.16 version: link:../../packages/0-shared/publish-surface '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@repo/test-utils': specifier: workspace:8.0.0-rc.16 version: link:../utils @@ -5804,8 +5804,8 @@ importers: specifier: workspace:8.0.0-rc.16 version: link:../../../../../../packages/1-framework/0-foundation/utils '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@prisma/orm-extension-pgvector': specifier: workspace:8.0.0-rc.16 version: link:../../../../../../packages/9-public/@prisma/orm-extension-pgvector @@ -5879,8 +5879,8 @@ importers: specifier: workspace:8.0.0-rc.16 version: link:../../../../../../packages/1-framework/0-foundation/utils '@prisma/cli-engine': - specifier: 0.6.3 - version: 0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) + specifier: 0.7.0 + version: 0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4) '@prisma/orm-extension-pgvector': specifier: workspace:8.0.0-rc.16 version: link:../../../../../../packages/9-public/@prisma/orm-extension-pgvector @@ -7661,8 +7661,8 @@ packages: '@prisma/adapter-pg@7.10.0': resolution: {integrity: sha512-N7nwSor0HO1Kz6xBv0TPAjAPysKK0fac6p4fVN3ensLOuzc/83Fgmln5k92eK/cvzqdkSR/2kkAqlbcdwVrwpw==} - '@prisma/cli-engine@0.6.3': - resolution: {integrity: sha512-rNPu6ulo72abHznDZ76sEa+mcPZ8k9tm5cQTwflb5EzEWq3f0T6/3NQ9GIJsNAX/oJK9H1pKl08tLgTjK8Vc7A==} + '@prisma/cli-engine@0.7.0': + resolution: {integrity: sha512-CL21NYQQEAR9IEApPfUzMhv/VtbsJs4efRSxlvzHoFT69vynr0Wuu5KNNbZgIlONOEfULqP0djFCD4VaA/HVPg==} engines: {node: '>=22.12.0'} peerDependencies: '@prisma/management-api-sdk': ^1.55.0 @@ -13430,7 +13430,7 @@ snapshots: transitivePeerDependencies: - pg-native - '@prisma/cli-engine@0.6.3(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4)': + '@prisma/cli-engine@0.7.0(@prisma/management-api-sdk@1.76.0)(magicast@0.5.4)': dependencies: '@clack/prompts': 1.5.0 '@prisma/management-api-sdk': 1.76.0 diff --git a/projects/migration-statements/deferred.md b/projects/migration-statements/deferred.md index 5fe9c97830dc..9d0718e7c125 100644 --- a/projects/migration-statements/deferred.md +++ b/projects/migration-statements/deferred.md @@ -10,15 +10,17 @@ A user keeps `model User` and changes `@@map("users")` to `@@map("app_users")`. **Options:** let a same-coordinate model statement (`--rename User:User`) mean "this model's storage was renamed"; or add a storage-level statement; or accept the hand-written route for this case. Decide with the operator before slice 2 writes the refusal text, because the refusal will otherwise suggest a `--rename` that cannot resolve. +**Decided 2026-10-07 by Will:** no statement for this case in this project. Slice 2's refusal keeps slice 1's manual steps for both commands. A statement that can name storage objects rather than models and fields is wanted later; Will floated `--rename table/users:app_users` as an illustration only, and the syntax needs a critical design discussion and a survey of established prior art before anything is chosen. + ## A model move across namespaces (`auth.User:public.User`) **Found:** slice 1, dispatch 3 (2026-10-06). Resolves but is refused with `statementRejected`; needs a `set schema` operation with its working-schema step and companion names. Scheduled for slice 3 with the namespace renames (recorded in `plan.md`). -## A codec's `onFieldEvent` hook sees different events under `db update` with and without statements +## A codec's `onFieldEvent` hook never sees the prior contract under `db update` -**Found:** slice 1, dispatch 5 review (2026-10-06). +**Found:** slice 1, dispatch 5 review (2026-10-06). Settled by slice 2, dispatch 4 (2026-10-08). -`db update` passes the origin contract to the planner only when `--rename` is given. Without statements, field-event planning has no prior contract and reports every column as added, as before this slice; with statements it reports only real changes. No codec in the repository implements `onFieldEvent` today, so nothing observable differs. If a codec starts relying on the hook under `db update`, decide whether `db update` should always supply the origin contract (which needs the runner's origin handling kept separate, as slice 1 did with `origin: null`). +Since slice 2, `db update` reads the origin contract on every run with a snapshot, but the planners give field-event planning the prior contract only when the plan has an origin. `db init` and `db update` plan with `origin: null`, so the hook reports every column as added on every run, with or without a snapshot, as before slice 1, and a plain `db update` plans the same operations whether or not the snapshot exists. No codec in the repository implements `onFieldEvent` today. If a codec needs real `added`, `dropped` and `altered` events under `db update`, the plan's operations would then depend on the snapshot, and that needs a decision. ## Which field name a MongoDB statement uses @@ -26,14 +28,40 @@ A user keeps `model User` and changes `@@map("users")` to `@@map("app_users")`. On SQL a statement names the model's field (`User.fullName`), and the storage bridge maps it to a column. On MongoDB every authoring surface keys a model's `fields` by the stored name (`@map("_id")` gives `fields._id`), so a statement resolved today against a Mongo contract names the stored field (`User._id`). Slice 4 must decide whether Mongo statements name the model's field or the stored field, and make the resolver agree on both families. -## MongoDB still classes index drops as destructive - -**Found:** slice 1, whole-slice architect review (2026-10-07). For slice 2 and slice 4. - -Project requirement 10 says drops that lose no data are widening on every target. Slice 1 reclassified them on Postgres and SQLite only; the MongoDB planner still classes an index drop as `destructive`. Slice 2's refusal must not refuse a drop no statement can resolve, so either slice 2 reclassifies Mongo's index drops first or slice 4 does it before Mongo statements ship. - ## A SQLite column rename followed by a table rebuild builds the replacement index twice **Found:** slice 1, whole-slice code review (2026-10-07), finding F04. When one plan renames a column on SQLite and a later step rebuilds the same table, the column rename's companion index replacement (drop and create) runs, then the rebuild creates the index again. Rows are kept; one index build is wasted. Removing the companion would need the planner to rewrite statement calls after the diff, and then re-running `migration.ts` (whose `renameColumn` cannot see the later rebuild) would no longer reproduce `ops.json`. Fixing it properly needs operations that carry dependency information, which the design notes already name as a direction. + +## SQLite `db update` cannot add a required field to a table that has rows + +**Found:** slice 2, dispatch 5 (2026-10-08). Pre-existing, not caused by the refusal. + +The Postgres planner has a temporary-default recipe for a required column added to a populated table under `db update`; SQLite has none, so the runner fails with `MIGRATION.RUNNER_FAILED` ("Cannot add a NOT NULL column with default value NULL"). The slice 2 SQLite journey adds the field to an empty table and says so. Resolving it means a SQLite temporary-default recipe or a table rebuild with a backfill; `--backfill` (slice 3) covers `migration plan`, not `db update`. + +## `--delete ` consenting for every model in the namespace + +**Found:** slice 2, whole-slice architect review (2026-10-08). For slice 3. + +Project decision 6 lets a namespace delete cover everything in the namespace while the refusal lists each model. The engine answers a question only by a value equal to its subject or starting with it, so a namespace-wide answer has to be expanded by the ORM before the questions are asked. Slice 3 adds the namespace noun to `rename` and `delete` together and does that expansion there. Until then every `delete` names one model, field or storage object. + + +## A MongoDB validator that requires a new field leaves existing documents unwritable + +**Found:** slice 2 manual QA (2026-10-08), F9. For slice 4. + +Adding a required field to a populated collection, or making an optional field required, updates the collection's validator with no question and no warning; it is `widening`, since the validator applies only to later writes. Every existing document that lacks the field then fails any update with `Document failed validation`. Nothing is lost, but nothing tells the user that existing documents need the field. A backfill statement would let the user say how to fill them. + +## The Postgres runner's NOT NULL failure gives advice about drift + +**Found:** slice 2 manual QA (2026-10-08), F10. Pre-existing text. + +`db update` making a column NOT NULL on a table with a NULL fails at the precheck (`ensure no NULL values in "note"`) with nothing lost, but the next action says to reconcile schema drift, and the `why` says only "Migration runner failed". The advice should say how many rows hold NULL and how to fix them, or keep the field optional. + +## The refusal's next actions are one line per flag, not grouped by subject + +**Found:** slice 2 manual QA (2026-10-08), F11. Engine side. + +`CLI.CONSENT_REQUIRED` lists one `user-choice` next action per flag form ("Run the command again with --delete Legacy", another for the rename form), not grouped by subject and not a complete runnable command, and its summary says "N subjects need a statement", which is project vocabulary. Grouping the forms per subject and printing a full command would need the engine to know the invocation; raise it on prisma/prisma-cli once the statement prompt has shipped. + diff --git a/projects/migration-statements/manual-qa-reports/2026-10-08-qa-slice-2.md b/projects/migration-statements/manual-qa-reports/2026-10-08-qa-slice-2.md new file mode 100644 index 000000000000..c049c076527f --- /dev/null +++ b/projects/migration-statements/manual-qa-reports/2026-10-08-qa-slice-2.md @@ -0,0 +1,249 @@ +# Manual QA — slice 2, the refusal and `--delete` — 2026-10-08 + +Branch `tml-3476-statement-refusal` at `7cbc9f9e8a`. Script: `projects/migration-statements/slices/refusal/manual-qa.md`, all twelve scenarios. + +## Summary + +| # | Scenario | Verdict | +| - | -------- | ------- | +| 1 | Postgres, terminal: answer the questions by typing | ✅ | +| 2 | The same non-interactively | ✅ (F11) | +| 3 | SQLite `db update`: interactive, `--confirm`, `--delete` | ⚠️ (F3, F4) | +| 4 | Stray or misspelled `--delete`; `--delete` on a dry run | ⚠️ (F5, F6) | +| 5 | Postgres, drop a policy through `db update` | ⚠️ (F7, F8) | +| 6 | Required field and NOT NULL, Postgres and MongoDB | ⚠️ (F9, F10) | +| 7 | `db update --db` with no snapshot | ✅ (F5, F12, F13) | +| 8 | A field on a renamed model | ⚠️ (F2) | +| 9 | MongoDB drop and `--rename` | ⚠️ (F1) | +| 10 | Upgrade fragments (judgement) | ✅ (F15) | +| 11 | Skill replay (judgement) | ✅ (F16) | +| 12 | Exploratory | ⚠️ (F14, F17) | + +No blocker by the script's definition. F1 comes closest: see its note. + +## Findings + +1. ⚠️ **F1 — On MongoDB, a refusal for a dropped collection offers only `--delete`, and says nothing about the hand rename that keeps the documents.** Contract renames `Event` to `Log` (scenario 9). Command: `prisma db update` (also `migration plan`). Output: + ``` + ✘ [CLI.CONSENT_REQUIRED] "Event" needs a statement, and the session is not interactive. + why: Drop collection Event would lose the data of model "Event". + → Pass --delete Event + ``` + In a terminal the question offers only `(delete)` (seen for the reverse rename, `Log` back to `Event`: `Drop collection Log would lose the data of model "Log". (delete)`), and the only way out is Ctrl-C. A user who renamed the model is offered exactly one route, and it drops the documents. The route that keeps them, `renameCollection` by hand and then `db update` with no statement (verified: `(no operations)`, 2 documents kept in `Log`), appears only if the user thinks to try `--rename Event:Log` and reads that error. I expected the refusal to say: "If it was renamed, rename the collection by hand first (see `--rename`'s error)". This is not a blocker, because the `why` says the data is lost and `--delete` is the explicit consent to that. But by the script's literal rule ("an error whose advice loses data if followed") it is close to one. +2. ⚠️ **F2 — The `` placeholder for a field leads to a rename that fails.** Command: `prisma db update --rename Profile:User` with `Profile.nickname` dropped and `User.handle` added (scenario 8). The refusal says `→ Pass --rename 'User.nickname:'`, and the question says `(rename User.nickname: or delete)`. The natural reading is `rename User.nickname:handle`. Typed at the prompt it fails: `Cannot resolve statement "--rename User.nickname:handle". The destination contract has no model "handle" (models: User).` The form that works is `User.nickname:User.handle`. For a field the placeholder should be `User.`, or a bare field name should resolve within the model. The re-ask did work, and the full form planned two renames and kept the rows. +3. ⚠️ **F3 — `--confirm ` is ignored without a word.** Command: `prisma db update --confirm dev.db` and `--confirm ./dev.db --no-interactive` (SQLite, scenario 3), `--confirm template1` (Postgres, scenario 5), `--confirm s9` (MongoDB), and `migration plan --confirm postgres`. Each gives the same `CLI.CONSENT_REQUIRED` as a run without `--confirm`. Nothing was applied, so no data is at risk. But a script author whose run used to pass sees a refusal that never mentions the flag they passed. I expected a line such as "`--confirm` no longer consents to data loss; use the flags below". The app upgrade fragment does say this. +4. ⚠️ **F4 — A second run of `--delete` gives advice that points the wrong way.** Command: `prisma db update --delete Legacy`, run again after it applied (SQLite). Output: `✘ [CLI.CONSENT_UNUSED] --delete Legacy was given but nothing in this run asked about Legacy.` / `→ Remove the flag, or spell the subject the way the command names it.` The spelling is right; the table is already gone. Compare the rename re-run in the same scenario, which is now exemplary: `The origin contract already has "User" and has no "Profile", so this rename has already happened. … → Leave out --rename Profile:User.` +5. ⚠️ **F5 — A misspelled or wrongly shaped `--delete` is not reported while a question is unanswered.** Commands (SQLite, scenario 4): `--delete Legcy`, `--delete legacy`, `--delete Legacy.note`; Postgres: `migration plan … --rename Profile:User --delete Legcy`; scenario 7: `--delete Legacy` and `--delete legacy_rows` where the subject is `public.legacy_rows`. Each gives the plain `CLI.CONSENT_REQUIRED` for `Legacy` and never says that the flag the user passed matched nothing. The user can work it out by comparing the suggested flag with their own, but the script expected `CLI.CONSENT_UNUSED`. `CONSENT_UNUSED` appears only when every question is answered (`--delete Foo` with `--delete Legacy`, or `--delete Foo` on a plan with no loss). In a terminal it is worse: with `--delete Legcy` the CLI asks about `Legacy`, I answered `delete`, and only then did it fail with `CONSENT_UNUSED` (nothing applied, `Legacy` kept). The unused flag should be reported before the question, and next to the refusal. +6. ⚠️ **F6 — `CONSENT_UNUSED` does not name the subjects the plan does ask about; the dry-run equivalent does.** Apply: `--delete Foo` → `→ Remove the flag, or spell the subject the way the command names it.` Dry run: `--dry-run --delete Foo` → `MIGRATION.STATEMENT_ANSWERS_NO_QUESTION … → Leave them out, or name a subject the plan asks about: Legacy.` The dry-run wording is the one to copy. Note: the script expected `--delete` on a dry run to fail with `CONSENT_UNUSED`. It does not: `--dry-run --delete Legacy` succeeds, marks the question `(answered)` and suggests `prisma db update --delete Legacy`. This matches commit `b3d90528c0` and the app fragment, so the script is out of date here, not the CLI. +7. ⚠️ **F7 — Dropping a permissive row-level-security policy narrows access, but the CLI says it widens it.** Command: `prisma db update` after removing `policy_select tenant_read` (permissive, `using (tenant_id = 1)`, role `app_user`) while `@@rls` stays (scenario 5). Question: `Drop RLS policy "tenant_read_f8d5e783" on "user" would widen who can read and write its rows.` Before the apply, `SET ROLE app_user; SELECT count(*) FROM "user"` → `1`; after `--allow User` → `0`. With row-level security on, permissive policies grant access, so dropping one takes access away. Disabling row-level security does widen it (`0` → `2` rows after the `to2` step). Asking before a policy drop is defensible; the stated reason is wrong, and `--allow` reads as "allow wider access". +8. ⚠️ **F8 — When one model has two access questions, the question names only one operation.** Contract removes the policy and `@@rls` together. Dry run lists both under `An apply asks about`. The refusal and the prompt say only `Drop RLS policy … would widen …`, not `Disable row-level security`, which is the operation that actually widens access. One `--allow User` applied both (`allow model "User" (2 operations)`). The `why` should list every operation behind the subject. The interactive question also has no way to say no except Ctrl-C. +9. ⚠️ **F9 — On MongoDB, a new required field leaves every existing document unwritable, and nothing says so.** Command: `prisma db update` adding `size Int` to a populated `Item` (scenario 6). Output: `Update validator on Item (added: size)`, no question, no warning. Afterwards `updateOne({note:'a'},{$set:{note:'aa'}})` fails with `Document failed validation`, because no existing document has `size`. The same happens when `note String?` becomes `note String` with a `null` and a missing `note` in the collection. No data is lost, and the reclassification to `widening` is by design. But the old classification at least warned; now a user gets no hint that existing documents need a backfill. A line such as "3 existing documents do not satisfy the new validator" would help. +10. ⚠️ **F10 — The NOT NULL failure on Postgres is clear about the cause but its advice is about drift.** Command: `prisma db update` with `note String?` → `note String` and one NULL (scenario 6). Output: + ``` + ✘ [MIGRATION.RUNNER_FAILED] Operation alterNullability.setNotNull.Item.note failed during precheck: ensure no NULL values in "note" + why: Migration runner failed + → Inspect the reported conflict, reconcile schema drift if needed, then re-run `prisma db update` + ``` + Nothing changed (column still nullable, both rows intact), and after setting the NULL the same command applied. The `why` says nothing, and there is no drift. I expected: "N rows have NULL in `note`; set a value or keep the field optional." SQLite gives `NOT NULL constraint failed: _prisma_new_Item.note` with the same advice, also without data loss. +11. ⚠️ (minor) **F11 — The refusal's next steps are flags, not commands, and the alternatives are not grouped.** `nextActions` are `user-choice` entries such as `Pass --delete Legacy`, four for two subjects, with nothing marking which ones are alternatives for the same subject. `meta.unanswered[]` groups them correctly. A `run-command` with the user's command plus the flag would be easier to paste. The summary `2 subjects need a statement` uses project words ("subject", "statement") that a first-time user does not know. +12. ⚠️ (minor) **F12 — The no-snapshot refusal is one long paragraph.** Command: `prisma db update --db ` with no snapshot (scenario 7). The `why` runs to about 120 words with steps 1–3 inline. The steps are correct: I followed them (emit the old contract, `--advance-ref qa --dry-run` → `(no operations)`, apply, emit the new contract, `--advance-ref qa --rename Legacy:Old`) and the table was renamed with both rows kept. The `STATEMENT_ORIGIN_UNKNOWN` error gives the same steps as separate `→` lines, which reads far better. +13. ⚠️ (minor) **F13 — A dry run with `--db` suggests an apply without `--db`.** Command: `prisma db update --db --advance-ref qa --dry-run`. Last line: `→ Apply the planned operations: prisma db update`. Followed literally, it runs against the config's database, not the one just previewed, and without the ref. The answered `--delete` flags are carried over (F6 note); `--db` and `--advance-ref` should be too. +14. ⚠️ **F14 — A type change is answered with `delete` and reported as a deleted field, even when it keeps every value.** Postgres `age Int` → `age String`: `Alter type of "User"."age" to text would lose the values of field "User.age"` → `Pass --delete User.age`, reported as `delete field "User.age"`. SQLite `qty Int` → `qty String` with `--delete Item.qty`: values `1`, `2` kept as text, report says `delete field`. The spec defers `--convert` to slice 3, so this is known, but the wording tells the user they are deleting data they are not. +15. ⚠️ (minor) **F15 — Extension fragment: one rule from the spec is missing.** The spec says the transient DROP of a policy replacement (drop then create a changed policy) is not listed in `accessWidening`. The extension fragment does not say this, so a third-party planner author would list it and make every policy edit ask. Every type and import path the fragment names exists in the built packages' `.d.mts` (checked: `MigrationSubject`, `MigrationOperationSubject`, `MigrationPlanSubjects`, `subjectsOfCalls`, `CallSubjects`, `FieldEventCall`, `storageNameOfOperation`, `AnswerPlanQuestions`, `PlanQuestion`, `PlanAnswer`, `AskedSubject`, `AlterColumnTypeCall`, `RecreateTableCall`). +16. ⚠️ (minor) **F16 — The skill says `CONSENT_UNUSED` covers a typo; a typo next to an unanswered question gives `CONSENT_REQUIRED` (F5).** `skills/prisma-8/references/migrations.md` line 75. +17. ⚠️ (minor) **F17 — The CLI ignores SIGTERM while a question is open.** Command: `prisma migration plan --name t --from 20261008T0244_initial` in a pseudo-terminal, left at the question, then SIGTERM. It was still running 5 s later and needed SIGKILL (in the first attempt it was still running after 20 s). Ctrl-C works (exit 3, `CLI.PROMPT_CANCELLED`, nothing written). A CI job with a TTY and a timeout would hang until killed. This may come from the engine's prompt, not this slice. + +## Setup + +- `git branch --show-current` → `tml-3476-statement-refusal`; `git status --short` → clean at the start and at the end. +- `pnpm install --frozen-lockfile` → exit 0. `pnpm build` → `87 successful, 87 total`. `@prisma/cli-engine` resolved from the CLI package is `0.7.0` (`pkg.pr.new/@prisma/cli-engine@c560ced`). +- CLI: `packages/1-framework/3-tooling/cli/dist/bin.mjs` through `wip/qa/prisma` (runs it with the shell's Node v26.8.1). Human output with `--format human` when not in a terminal. +- Projects: one per scenario under `wip/qa/` (`s1`–`s9`, `s6m`, `s7src`, `s12`, `s12b`, `s12c`). Postgres projects from `prisma orm init --target postgres --authoring psl --skip-install`; SQLite and MongoDB configs written by hand from `examples/prisma-8-demo-sqlite` and `examples/mongo-demo`. Packages resolve through `wip/qa/node_modules` symlinks to the workspace facades. +- Postgres: one `@prisma/dev` server per scenario (`wip/qa/pg-server.mjs`). Careful: it is one PGlite backend, so a `SET ROLE` in `psql` carries into the CLI's next connection. One run in scenario 5 was spoiled that way (`permission denied for database template1`) and is not counted. MongoDB: `MongoMemoryReplSet` (`wip/qa/mongo-server.mjs`), queried with `wip/qa/mq.mjs`. +- Terminal: `wip/qa/drive.py`, a Python `pty` driver that waits for a pattern and types an answer. **Renderer: clack.** A wrong answer is explained and asked again; the line reader was never used. +- All five Postgres servers and the replica set were stopped by PID at the end. + +## 1. Postgres, terminal — ✅ + +`from`: `Profile { id; name @unique; handle; posts Post[] }`, `Post` → `Profile`, `Legacy { id; note }`. `to`: `Profile` renamed `User`, `Legacy` removed. `migration plan --name initial`, `db migrate`, 3 profiles, 2 posts, 2 legacy rows. + +`prisma migration plan --name tidy` without `--from` refused with `MIGRATION.PLAN_ORIGIN_UNKNOWN` (existing behaviour, as in slice 1), so I used `--from 20261008T0234_initial`. Typed answers: + +``` +◆ Drop table "Legacy" would lose the data of model "Legacy". (rename Legacy: or delete) +│ remove +◆ Start the answer with rename or delete. +│ delete +◆ Drop table "Profile" would lose the data of model "Profile". (rename Profile: or delete) +│ rename Profile:Nope +◆ Cannot resolve statement "--rename Profile:Nope". The destination contract has no model "Nope" (models: public.Post, public.User). +│ rename Profile:User +✔ Planned 4 operation(s) +├─ Rename table "Profile" to "User" +├─ Rename primary key "Profile_pkey" to "User_pkey" on "User" +├─ Rename unique constraint "Profile_name_key" to "User_name_key" on "User" +└─ ⚠ Drop table "Legacy" +Statements applied +├─ rename model "Profile" to "User" (3 operations) +└─ delete model "Legacy" (1 operation) +``` + +The questions are clear. Both wrong answers were explained and asked again. The second error quotes the flag form (`--rename Profile:Nope`) of what I typed at the prompt (`rename Profile:Nope`), which is a little confusing (minor). `migration.ts` has `...this.renameTable(...)` and `this.dropTable(... 'Legacy')`; `ops.json` classes are `widening` ×3 and `destructive` for the drop. `db migrate` applied it with no question (as specified: a written migration is reviewed before it runs). `User` has alice, bob, carol; `Post` rows kept; `Legacy` and `Profile` gone; a re-plan from `tidy` shows no change. + +Ctrl-C at the first question: `✘ [CLI.PROMPT_CANCELLED] The prompt "Drop table "Legacy" would lose …" was cancelled before it was answered.`, exit 3, nothing written. The nested quotes in the message are cosmetic. + +## 2. Non-interactive — ✅ + +``` +$ prisma migration plan --name tidy --from 20261008T0234_initial < /dev/null +✘ [CLI.CONSENT_REQUIRED] 2 subjects need a statement, and the session is not interactive: "Legacy", "Profile". + why: Drop table "Legacy" would lose the data of model "Legacy". + Drop table "Profile" would lose the data of model "Profile". +→ Pass --rename 'Legacy:' +→ Pass --delete Legacy +→ Pass --rename 'Profile:' +→ Pass --delete Profile +[exit 2] +``` + +The `migrations/` listing was identical before and after. JSON: `meta.unanswered: [{subject: "Legacy", verbs: ["rename","delete"]}, {subject: "Profile", …}]`. Pasting `--rename Profile:User --delete Legacy` planned the same 4 operations as scenario 1; `db migrate` applied it; 3 rows in `User`, `Legacy` gone. With only `--rename Profile:User` the refusal lists only `Legacy`. Minor: F11. + +## 3. SQLite `db update` — ⚠️ + +`from`: `Profile`, `Legacy` (ids without `@default(autoincrement())`, see "Not covered"); `to`: `Legacy` removed; 2 rows each. + +Interactive (copy `s3a`): `yes` → `Start the answer with rename or delete.`; `delete Profile` → `delete names "Legacy": --delete Legacy.` (understandable, but it shows the flag where the prompt wants `delete`); `delete` → applied, `delete model "Legacy" (1 operation)`, `Profile` kept 2 rows. + +`--confirm dev.db` and `--confirm ./dev.db --no-interactive` → the plain `CONSENT_REQUIRED`, `Legacy` still has 2 rows (F3). `--delete Legacy` → `⚠ Drop table Legacy`, `Statements applied └─ delete model "Legacy"`, `Profile` intact. Same command again → `CONSENT_UNUSED … nothing in this run asked about Legacy` (F4). + +DoD check on a copy (`s3b`, `Profile` → `User` and `Legacy` removed): `--rename Profile:User --delete Legacy` applied both; the second run failed on the first statement: `STATEMENT_UNRESOLVED … this rename has already happened. For db update, the database is already at the new name. → Leave out --rename Profile:User.` Good. + +## 4. Stray and misspelled `--delete` — ⚠️ + +| Command (SQLite `db update`, Postgres `migration plan`) | Result | +| --- | --- | +| `--delete Legacy --delete Profile` (Profile kept) | `CONSENT_UNUSED --delete Profile was given but nothing in this run asked about Profile.` nothing applied | +| `--delete Foo` on a plan with no loss | `CONSENT_UNUSED`, same wording | +| `--delete Legcy`, `--delete legacy`, `--delete Legacy.note` | `CONSENT_REQUIRED` for `Legacy`; the unused flag is not mentioned (F5) | +| Same, in a terminal | asks about `Legacy`, then after `delete` fails `CONSENT_UNUSED --delete Legcy`; nothing applied (F5) | +| `migration plan … --delete Legacy --delete Foo` | `CONSENT_UNUSED`; nothing written | +| `migration plan … --delete Legacy --delete Legacy` | `CONSENT_UNUSED … the question about Legacy was already answered by another flag. → Give one flag per question.` Good | +| `--dry-run --delete Legacy` | succeeds; `An apply asks about └─ Drop table Legacy: --delete Legacy (answered)`; `→ Apply …: prisma db update --delete Legacy` (F6 note) | +| `--dry-run --delete Foo` | `MIGRATION.STATEMENT_ANSWERS_NO_QUESTION … name a subject the plan asks about: Legacy.` (F6) | + +Nothing was written or applied in any failing row. + +## 5. Postgres policy drop — ⚠️ + +Project from the `contract-rls-wire.prisma` journey fixture (`User @@map("user") @@rls`, `policy_select tenant_read`), role `app_user`, 2 rows. + +``` +$ prisma db update --dry-run +App space +└─ Drop RLS policy "tenant_read_f8d5e783" on "user" +An apply asks about +└─ Drop RLS policy "tenant_read_f8d5e783" on "user": --allow User +$ prisma db update +✘ [CLI.CONSENT_REQUIRED] "User" needs a statement, and the session is not interactive. + why: Drop RLS policy "tenant_read_f8d5e783" on "user" would widen who can read and write its rows. +→ Pass --allow User +$ prisma db update --confirm template1 → same refusal +``` + +The dry run asks nothing. It shows the operation under `An apply asks about`, not under a heading named "access widening". In a terminal: `delete` → `Start the answer with allow.`; Ctrl-C → exit 3. `--allow User` applied: `Statements applied └─ allow model "User" (1 operation)`. Removing `@@rls` next, the terminal answer `allow` applied `Disable row-level security`. Policy and RLS removed together: one `--allow User` covered both (F8). F7 is about the wording. + +## 6. Required fields and NOT NULL — ⚠️ + +Postgres, `Item { id; note String? }`, 2 rows (one `note` NULL): + +- Adding `size Int`: no question. `ADD COLUMN "size" int4 DEFAULT (0) NOT NULL; … DROP DEFAULT`. Both rows get `0`. The output says only `Add column size to Item`, not that existing rows got `0` (existing behaviour). +- `note String?` → `String` with a NULL: the dry run shows `Set NOT NULL` and no question; the apply fails (F10), no data lost; after `UPDATE … SET note='b'` it applies. + +MongoDB, `Item { id; note String? }`, 3 documents (`note:'a'`, `note:null`, no `note`): + +- Adding `size Int` and then making `note` required: no question either time, validator updated, documents unchanged (F9). + +## 7. `db update --db` with no snapshot — ✅ + +`s7src` (`Keep`, `Legacy @@map("legacy_rows")`, 2 rows) applied with its own `db` ref; `s7` is a separate directory with no `migrations/` and a contract without `Legacy`. + +``` +$ prisma db update --db --dry-run +└─ Drop table "legacy_rows": --delete public.legacy_rows +$ prisma db update --db +✘ [CLI.CONSENT_REQUIRED] "public.legacy_rows" needs a statement, and the session is not interactive. + why: Drop table "legacy_rows" would lose the data in "public.legacy_rows", named by its storage name because the origin contract is unknown; --delete loses its rows. If it was renamed, keep its data instead: … 1. … 2. … 3. … +→ Pass --delete public.legacy_rows +$ prisma db update --db --delete public.legacy_rows +✔ Applied 1 operation(s) … Statements applied └─ delete storage "public.legacy_rows" (1 operation) +``` + +It names the storage name and says the origin is unknown. `--rename Legacy:Old` → `STATEMENT_ORIGIN_UNKNOWN` with the steps as `→` lines. I followed the steps on a recreated table (see F12): the snapshot was stored, then `--advance-ref qa` without statements asked about `Legacy` by model name, and `--rename Legacy:Old` renamed `legacy_rows` to `Old` with rows `1|x`, `2|y` kept. `--delete Legacy` and `--delete legacy_rows` were ignored silently (F5). + +## 8. A field on a renamed model — ⚠️ + +SQLite, `Profile { id; name; nickname }` → `User { id; name; handle }`, 2 rows. + +``` +$ prisma db update --rename Profile:User +✘ [CLI.CONSENT_REQUIRED] "User.nickname" needs a statement … +→ Pass --rename 'User.nickname:' +→ Pass --delete User.nickname +$ prisma db update --rename Profile:User --rename Profile.nickname:User.handle +✘ [MIGRATION.STATEMENT_INVALID] … Name a field's model as the destination contract names it: --rename User.nickname:User.handle. +$ prisma db update --rename Profile:User --rename User.nickname:User.handle +├─ Rename table Profile to User +├─ Rename column nickname on User to handle +Statements applied +├─ rename model "Profile" to "User" (1 operation) +└─ rename field "User.nickname" to "User.handle" (1 operation) +``` + +Rows kept. The `Statements applied` block names things as I typed them. `--delete Profile.nickname` (old model name) is ignored silently (F5). F2 covers the placeholder. In a terminal, after `rename Profile:User` was answered, a question about the field followed, and the full form applied. `--rename Profile:User --delete User.nickname` failed at apply on SQLite with `Cannot add a NOT NULL column with default value NULL` for `handle` and rolled back (`Profile` intact); see "Not covered". + +## 9. MongoDB — ⚠️ + +`Keep`, `Event` (2 documents). With `Event` removed: `db update`, `db update --confirm s9` and `migration plan --name drop-event` are all refused with `→ Pass --delete Event`, and nothing is written (slice 1's F4 auto-baseline leak is fixed). `migration plan --delete Event` wrote the auto-baseline and `drop_event` (`⚠ Drop collection Event`, `delete model "Event"`). `db update --delete Event` dropped it. + +With `Event` renamed to `Log`: `db update --rename Event:Log` and `migration plan … --rename Event:Log` → `PLANNING_FAILED`, with advice that states the loss and gives `renameCollection`. Followed: `(no operations)`, `Database already matches contract`, 2 documents in `Log`. Without `--rename` see F1. + +## 10. Upgrade fragments — ✅ (judgement) + +App (`upgrade-instructions/pending/migration-statement-refusal/app/instructions.md`), read as a script author with `db update --confirm mydb`: it says plainly that `--confirm` answers nothing, gives the replacement per case (`--delete `, `--delete `, `--rename`, the storage name schema-qualified on Postgres, `--allow `), and how to find the subjects (`--no-interactive --json` → `meta.unanswered[]`, or `--dry-run --json`). Each claim I checked holds: the dry-run `STATEMENT_ANSWERS_NO_QUESTION`, the storage-name subject without a snapshot, field-of-renamed-model naming, `--rename` not offered for a type change or on MongoDB. + +Extension, read as a planner author: every field and type to add is named with its import path, and each exists in the built packages (F15 has the list). Detection misses are stated per change. One spec rule is missing (F15). + +## 11. Skill replay — ✅ (judgement) + +"How do I delete a field?" → § *Answer the data-loss questions*: `--delete User.nickname`, with `--delete Legacy` and storage names. "How do I answer a refusal?" → read `meta.unanswered[]` or `nextActions`, add the flags, "Don't reach for `--delete` by reflex: if the model or field was renamed, `--rename` keeps its rows", and Pitfall 8 gives the field form `Person.email:Person.emailAddress`. The code table has `CONSENT_REQUIRED` (with `--confirm` and `--yes` answering nothing), `CONSENT_UNUSED`, `STATEMENT_DID_NOT_RESOLVE_LOSS`. Minor: F16. + +## 12. Exploratory — ⚠️ + +| Probe | Result | +| --- | --- | +| Five subjects in one plan (2 models, 2 fields, a type change), Postgres | All listed; five `--delete`s planned; `Statements applied` lists each ✅ | +| `--delete User` when only `User.bio`/`User.name` drop | `CONSENT_UNUSED` ✅ (a model delete does not cover its fields) | +| Auto-baseline with a drop (MongoDB, empty graph, `db` ref) | Refused, nothing written; with `--delete` baseline + drop written ✅ | +| `--json` in a terminal | Asks (on stderr: with stdout redirected to a file the question still showed); stdout is one clean JSON result; `appliedStatements[].verb` is `delete` ✅ | +| `--yes`, with and without a terminal | `CONSENT_REQUIRED … which --yes cannot give` ✅ | +| Ctrl-C at a question | exit 3, `CLI.PROMPT_CANCELLED`, nothing written ✅ | +| SIGTERM at a question | ignored (F17) | +| Placeholder plan with a drop (type change needs a data transform) | operations and `Statements applied` both shown (slice 1's F9 fixed) ✅ | +| Rollback `--from a --to initial` | refused for the reverse type change; `--delete User.age` plans it ✅ | +| Rename at the prompt that leaves a field drop (SQLite `Legacy{note}` → `Archive{title}`) | after `rename Legacy:Archive` it plans again and asks about `Archive.note`; `rename Archive.note:Archive.title` applied, rows kept ✅ | +| SQLite nullability-only tightening | `Recreate table Item`, no question; with a NULL fails, rows kept ✅ | +| SQLite and Postgres type change Int → String | asked, answered with `--delete`, values kept (F14) | + +## Not covered, and observations outside this slice + +- `MIGRATION.STATEMENT_DID_NOT_RESOLVE_LOSS`: I found no answer that reaches it, because the prompt rejects a rename whose old side is not the subject before planning again. +- A destructive operation in an extension contract space; `migration plan` on SQLite; `db migrate` of a plan with a `--delete` on SQLite and MongoDB. +- SQLite: `id Int @id @default(autoincrement())` fails the first `db update` with `MIGRATION.RUNNER_FAILED … Database schema does not satisfy contract (2 failures)` (`column:id default` expected `autoincrement()`). I dropped the default to proceed. It looks like an existing verifier bug, unrelated to this slice; worth checking on `main`. +- SQLite: adding a required `String` to a populated table fails (`Cannot add a NOT NULL column with default value NULL`), where Postgres fills `0` for an `Int`. Existing behaviour; it means scenario 6's "required field, no refusal" passes on SQLite only for an empty table. +- Every Postgres `db update` prints `⚠ Planner warnings - control policy suppressed: namespace "__unbound__" …`, as in slice 1. +- `contract emit` in the scratch projects warns `contract.d.ts imports types from packages that are not installed: @internal/target-postgres`; caused by the symlinked setup. +- `--allow` is offered only by `db update`; `migration plan` writes the policy drop with no question, as specified. + +## Cleanup + +All `@prisma/dev` servers (PIDs 12048, 13122, 13707, 14041, 14271) and the replica set (12050) were stopped by PID. One CLI process left by my driver at a question (12403, started by me) was killed with SIGKILL after SIGTERM failed (F17). `wip/qa/` is left in place with the projects, helper scripts (`prisma`, `pg-server.mjs`, `mongo-server.mjs`, `mq.mjs`, `drive.py`) and the terminal logs (`logs-*`). diff --git a/projects/migration-statements/plan.md b/projects/migration-statements/plan.md index c7395c5741c6..0090e6f8d0d8 100644 --- a/projects/migration-statements/plan.md +++ b/projects/migration-statements/plan.md @@ -26,7 +26,9 @@ Four slices. The first three stack: the statement surface with renames, then the **Builds on.** Slice 1. Two things slice 1 leaves for it: the missing-origin check in the CLI statement resolver runs over the whole input and must move inside the per-statement loop so that `--delete` works without an origin contract; and the `@@map`-only rename gap in `deferred.md` must be decided before the refusal text is written. Decided 2026-10-07 on prisma/orm#30638: dropping a row-level-security policy and disabling row-level security stay `widening` (they lose no data), and slice 2 adds a separate consent question before `db update` applies an operation that widens who can read or write rows, answered per operation like the data-loss consent and refused in a non-interactive run unless consented. -**Hands to.** The refusal shape every later verb hooks its statements into; the per-operation consent model; `--confirm` gone. +**Design settled 2026-10-07 (Will).** Consent is a statement the user answers with, not a token copied back: the CLI engine gains `ctx.prompt.statement`/`statements`, answered on the command line by verb flags (`--delete Legacy`, `--rename Legacy:Archive`, `--allow User`) and interactively by typing the statement when a human runs the command. The engine change is its own slice, built in prisma/prisma-cli first: [`slices/engine-statement-prompt/spec.md`](./slices/engine-statement-prompt/spec.md). Slice 2's spec is [`slices/refusal/spec.md`](./slices/refusal/spec.md). The `@@map`-only gap gets no statement (see `deferred.md`). Operations that lose no data (`SET NOT NULL`, safe type widenings, MongoDB validator and index changes) stop being destructive. + +**Hands to.** The refusal shape every later verb hooks its statements into; the per-operation consent model; `--confirm` no longer consents to data loss. ### Slice 3 — Convert and backfill scaffold the placeholder migration, and the remaining nouns @@ -44,7 +46,7 @@ Four slices. The first three stack: the statement surface with renames, then the **Outcome.** The Mongo planner takes the same resolved statements: collection rename, document rewrites for field and value object field renames, drops and unsets for deletes, a data transform scaffold for convert, under the slice 2 refusal and consent model. No family vocabulary enters the framework. -**Builds on.** Slice 2. +**Builds on.** Slice 2. Slice 2 left a temporary member for it: `TargetMigrationsCapability.refusesRenameStatements`, set by MongoDB so a data-loss question offers no `rename` there. Slice 4 deletes the member, its one CLI read and its doc mention when the Mongo planner carries out renames. **Hands to.** Project close-out for Mongo. diff --git a/projects/migration-statements/slices/engine-statement-prompt/spec.md b/projects/migration-statements/slices/engine-statement-prompt/spec.md new file mode 100644 index 000000000000..c2b105039d2d --- /dev/null +++ b/projects/migration-statements/slices/engine-statement-prompt/spec.md @@ -0,0 +1,110 @@ +# Slice spec — The CLI engine asks for a statement, and a verb flag answers it + +**Project:** [`projects/migration-statements/`](../../spec.md) · **Engine slice, built in prisma/prisma-cli** (`packages/cli-engine`) · **Linear:** TML-3476 (slice 2 depends on it) · **Branch (prisma-cli):** `engine-statement-prompts` + +## At a glance + +An ORM command finds that applying its plan would drop the `Legacy` table. It asks the engine: + +```ts +const answer = await ctx.prompt.statement( + 'Table "Legacy" would be dropped and its rows lost. What do you mean?', + { + subject: 'Legacy', + verbs: ['rename', 'delete'], + validate: (verb, text) => resolve(verb, text), // ok or an error message + }, +); +// answer: { verb: 'delete', text: 'Legacy' } or { verb: 'rename', text: 'Legacy:Archive' } +``` + +Run by a script or an agent with no answer on the command line, the command fails before it changes anything: + +```text +✖ [CLI.CONSENT_REQUIRED] "Legacy" needs a statement, and the session is not interactive. +→ Pass --delete Legacy, or --rename Legacy: +``` + +Run with `--delete Legacy`, the question is answered before anything renders and no prompt is shown. Run by a human in a terminal, the question is asked and the human types `delete` or `rename Legacy:Archive`; a wrong answer is explained and asked again. `--yes` never answers it. `--delete Legacy` given to a run that never asks about `Legacy` is an error, not silence. + +## Chosen design + +### The primitive + +`PromptSurface` gains `statement(question, { subject, verbs, validate })`, beside `consent`. It is a consent: structurally undefaultable, never satisfied by `--yes`, never satisfied by Enter. + +- `subject` is the string the answer is about, in the command's own vocabulary (for the ORM: a model or field coordinate such as `Legacy` or `User.name`). +- `verbs` lists the verb flags that may answer it, in the order the refusal should print them. +- `validate(verb, text)` returns `undefined` to accept, or a message string to reject. The engine never interprets the text; the command does. +- It returns `{ verb, text }`. + +### How it is answered + +1. **Command line first.** For each verb in `verbs`, the engine looks through the run's unconsumed values of that verb flag for one that *names the subject*. The subject check is `text === subject` or `text.startsWith(subject + ':')`. The first match is validated; if `validate` accepts it, it is consumed and returned without rendering anything. If `validate` rejects it, the run fails with `CLI.PROMPT_INVALID` carrying the message, because a wrong flag cannot be corrected by re-prompting. +2. **Non-interactive, or `--yes`:** throw `CLI.CONSENT_REQUIRED`. The message names the subject; `nextActions` carries one `user-choice` per verb, written as the flag to pass (`--delete Legacy`, `--rename Legacy:`); `meta` carries `{ subject, verbs }`. +3. **Interactive:** render the question and read a line. The answer is ` ` or just `` (then `text` is the subject). An unknown verb, or a `validate` rejection, re-prompts under clack with the message; under the line renderer it fails with `CLI.PROMPT_INVALID`, as `consent` does today. + +### The batch form + +A command that has several questions asks them together: `ctx.prompt.statements([q1, q2, ...])`, each entry the same shape as a `statement` call, returning the answers in order. It exists because a refusal must name every unanswered question at once (project requirement 4), and a command cannot tell whether the session is interactive (the engine owns that). + +- Every question is first answered from the command line as above. +- Non-interactively, or under `--yes`, one `CLI.CONSENT_REQUIRED` is thrown that lists every question still unanswered, each with its flag forms in `nextActions`, and `meta.unanswered: [{ subject, verbs }]`. +- Interactively, the unanswered questions are asked one after another, in order. +- `statement(q)` is `statements([q])[0]`. +- `statements(questions, { last: true })` says this is the run's final ask: when it has answered its questions and values are still unconsumed, it throws `CLI.CONSENT_UNUSED` there, before the command does anything with the answers. The end-of-run check stays for commands that never say `last`. +- A verb's `arity` is the number of argv values each occurrence takes; a wrong count is `CLI.INVALID_ARGUMENTS`. Verbs are one lowercase word. A declaration carries a `brief` for help, because the product owns its help text. A `server-command` may not declare statements. + +### Taking a verb's values up front + +Some statements are inputs to the command's work, not answers to a question: the ORM plans *with* its `rename` statements and only then learns what still loses data. So a handler can take a verb's values before asking anything: `ctx.statements.take('rename')` returns that verb's values in argv order, as `{ verb, values, text }[]`, and consumes them, so the end-of-run leftover check does not report them. A verb not declared by the command is a construction error. Values of other verbs stay for the questions. The ordinary rule that handlers never see consent values holds for `--confirm`; a statement is the command's own declared input. + +A question's `subject` may contain `:`. A value answers the question whose subject it equals; failing that, the question with the longest subject the value starts with (followed by `:`). Questions whose subjects are prefixes of one another are asked in one batch so that rule can apply. A verb a handler takes is not listed in a later question's `verbs`, and `take` comes before the `last` batch. + +### Verb flags + +- The engine knows no verbs. A command declares the statements it may ask for, with their arity: `defineCommand({ statements: { rename: { arity: 1 }, delete: { arity: 1 } }, ... })`. For that command only, the engine parses `--` followed by `arity` values, repeatable, keeps the values in argv order, and hands them out through `ctx.prompt.statement`. The values stay out of the handler's flags; a handler reads a verb's values only through `ctx.statements.take`. (Will's ruling, 2026-10-07: the verb is the ORM's, declared per command, not registered on the family or the CLI, so no other command parses it.) +- A command cannot declare an ordinary flag with the same name as one of its statements, and a `statement()` call naming a verb the command did not declare is a construction error. +- `arity` is 1 for every ORM verb today; it exists so the engine never assumes the shape of a statement. +- Order across flags is kept: the engine records the position of every verb-flag value in argv, so a product that needs `--rename A:B --delete C` in the order written can read it. This slice only needs to consume values; it exposes the ordered list on the run state so the ORM's statement list can be built from it later. +- At the end of a run that settled successfully, an unconsumed verb-flag value fails the run with a new `CLI.CONSENT_UNUSED`: "`--delete Legacy` was given but nothing in this run asked about `Legacy`". A run that failed for another reason reports that reason only. On a command that declares statements, an unused `--confirm` value is also `CLI.CONSENT_UNUSED` (the ORM's QA found a defensive `--confirm ` silently ignored); on every other command a leftover `--confirm` stays silent as before, because shipped commands accept the flag on paths that never ask (a follow-up issue in prisma-cli records them). + +### What stays + +`consent(question, { token })` and `--confirm` are unchanged. The other products keep using them. + +## Coherence rationale + +One primitive, its flags and its error codes ship together because a prompt without its flag-driven equivalent breaks the engine's own rule, and a flag that can be given but never consumed is the silent-typo bug the ORM project refuses to ship. Order-keeping is in because it costs one array at parse time now and would cost a second parse of argv later. + +## Scope + +**In:** `statement` on `PromptSurface` and both renderers (clack, line); verb-flag registration and reservation; consumption and `CLI.CONSENT_UNUSED`; `CLI.CONSENT_REQUIRED` for a statement; tests in `packages/cli-engine/tests/prompts.test.ts` style, with scripted `answers`; the engine `README`, `docs/product/cli-style-guide.md` (consent section: a statement is the second consent form) and `docs/reference/error-reference.md`; `pnpm bump-cli-engine-version minor`. + +**Out:** any ORM command; `--confirm` leftovers on commands that declare no statements (prisma/prisma-cli#339); `select`-style menus; help text for the ORM's verbs (the ORM owns its briefs). + +## Pre-investigated edge cases + +| Case | Disposition | +| --- | --- | +| Two questions about the same subject | Two flag values needed; each `statement` call consumes one. Same as `consent`. | +| A subject that contains `:` | The subject check uses the subject string as given; the command picks subjects without `:`. Documented on the option. | +| `--delete Legacy --json` in a TTY | Format never decides interactivity (engine ruling). Flag answers; nothing is rendered. | +| A verb flag value given twice | Both kept; the second is unconsumed and reported by `CLI.CONSENT_UNUSED`. | +| `validate` throws | Treated as a bug: the engine does not catch it. | + +## Slice done conditions + +- `pnpm --filter @prisma/cli-engine test`, `typecheck` and `lint` green; the conformance suite (`pnpm check:conformance`) green. +- A probe command in the tests is driven four ways: flag answer, non-interactive refusal naming both verbs, interactive scripted answer `delete`, interactive scripted answer `rename Legacy:Archive`, plus the unused-flag failure and the rejected-flag failure. +- The engine version bumped minor; the published package is what the ORM's slice 2 pins. + +## Open questions + +None. Will decided on 2026-10-07: consents carry a verb; the answer is a free-text statement the command validates; a human in a terminal is asked interactively. + +## References + +- Project spec [`../../spec.md`](../../spec.md) decisions 2 and 6, requirement 12; [`../../plan.md`](../../plan.md) § Stretch goal. +- prisma-cli: `packages/cli-engine/src/execution/prompts.ts`, `shared-flags.ts`, `clack-renderer.ts`, `src/context.ts`; `docs/product/cli-style-guide.md` § consent; ADR 0004 (engine version pinning). +- `wip/slice-2-consent-options.md`: the option evaluation Will chose from. diff --git a/projects/migration-statements/slices/refusal/dispatches/02-data-loss-subjects.md b/projects/migration-statements/slices/refusal/dispatches/02-data-loss-subjects.md new file mode 100644 index 000000000000..d3b759b8389f --- /dev/null +++ b/projects/migration-statements/slices/refusal/dispatches/02-data-loss-subjects.md @@ -0,0 +1,32 @@ +# Dispatch 2 — The planner result names what each destructive operation would lose + +**Slice:** [`../spec.md`](../spec.md) § "The structured refusal" · **Plan:** [`../plan.md`](../plan.md) dispatch 2 · **Builds on:** dispatch 1 + +## Outcome + +The planner's success result says, for every operation that loses data and for every operation that widens access to rows, which domain thing it is about, so the CLI can ask the user about it in contract vocabulary without any family code in the framework. + +## What to build + +1. **Framework type** (`packages/1-framework/1-core/framework-components/src/control/migration-statements.ts`): `MigrationStatementSubject = { kind: 'model' | 'field', namespaceId, model, field? } | { kind: 'storage', name }`, with its JSON form next to `migrationStatementJson` (leave `namespaceId` out for the unbound namespace, as statements do). `MigrationPlannerSuccessResult` gains `dataLoss: readonly { operationIndex: number; subject: MigrationStatementSubject }[]` and `accessWidening` of the same shape. Both required; every planner fills them. +2. **SQL family** (`packages/2-sql/9-family/src/core/migrations/`): after a plan is assembled, walk its operations; for each `destructive` one, map its table or column back to a model or field of `fromContract` through `model.storage.table` and `fields[f].column` (reverse of `modelTable`/`fieldColumn` in `statement-planning.ts`), with the rename statements already applied, so a table renamed earlier in the plan is found under its old name in the origin. No `fromContract`, or a table no model stores: `{ kind: 'storage', name }`. Index entries by the operation that loses the data; on SQLite a table rebuild that drops columns is that operation, not the later `dropColumn`, so list the dropped field once, at the rebuild's index. `accessWidening`: the `DropPostgresRlsPolicyCall` and `DisableRowLevelSecurityCall` operations, subject the table's model, excluding the transient DROP of a policy replacement (see `gradePolicyReplacement` and the replacement path in `planner.ts`). +3. **MongoDB**: `dropCollection` entries with the model whose collection it is; `accessWidening` empty. +4. **Safe type widenings on `db update`** (dispatch 1 left this open; the reviewer's finding is in `../reviews/code-review.md` round 1 item 7): in `mapColumnNodeIssue` (`packages/3-targets/3-targets/postgres/src/core/migrations/issue-planner.ts` ~L654-705) the live column has no codec but carries a normalised `resolvedNativeType`. Map that name to a data type id through the written text of the parameterless types (`data-types.ts` ~L140-200) and reuse `SAFE_WIDENINGS`; require `many` false on both sides. A safe pair gets `'widening'`. Test through the issue planner with an introspected `int4` column and an `int8` contract column. +5. **Aggregate and `db update` plumbing** (`packages/1-framework/3-tooling/migration/src/aggregate/`): `PerSpacePlan` and `planFromDiff` carry `dataLoss` and `accessWidening` through; `db-run.ts`'s plan result exposes them with each entry's `operationIndex` offset the way `appliedStatements` positions are (see `operationsBefore`). +6. **Per-statement origin check**: in `resolve-statements.ts`, the `origin.kind === 'missing'` check moves inside the per-statement loop so it applies to `rename` only. Add the `delete` verb to `StatementVerb` and the parser (text is a coordinate or, when the refusal said so, a storage name; both pass through unresolved, since `delete` is matched against the plan in dispatch 3, not resolved against a contract). `describeMigrationStatement` writes `delete model "Legacy"`. + +## Not in this dispatch + +Any prompt, flag, or refusal text; the CLI's matching of `delete` answers to `dataLoss`; the engine pin. Those are dispatch 3. + +## Tests + +Family and target tests for every mapping case: model subject, field subject, renamed-earlier table, storage fallback with and without `fromContract`, SQLite rebuild indexing, Postgres policy drop and RLS disable with the replacement DROP excluded, Mongo collection. The `db update` safe-widening test. Aggregate plumbing and offset tests. Resolver tests for `delete` and for `rename` with a missing origin failing per statement. + +## Halt conditions + +Stop and report if the SQL operations do not carry enough (`target.details`) to find the table and column for some destructive operation; if mapping the live native type needs more than the written-text lookup described; or if excluding the policy-replacement DROP needs the planner to carry new state. + +## Gate + +The standard gate from `../plan.md`, plus `pnpm lint:framework-vocabulary` (the framework type must add no family word) and the two rename-statements journeys. diff --git a/projects/migration-statements/slices/refusal/dispatches/03-refusal-on-migration-plan.md b/projects/migration-statements/slices/refusal/dispatches/03-refusal-on-migration-plan.md new file mode 100644 index 000000000000..3b67a5248d0b --- /dev/null +++ b/projects/migration-statements/slices/refusal/dispatches/03-refusal-on-migration-plan.md @@ -0,0 +1,39 @@ +# Dispatch 3 — The engine pin, the verbs, and the refusal on `migration plan` + +**Slice:** [`../spec.md`](../spec.md) § "The structured refusal", § "`--delete` and how statements consent" · **Plan:** [`../plan.md`](../plan.md) dispatch 3 · **Builds on:** dispatch 2 and the engine change prisma/prisma-cli#337 + +## Outcome + +`migration plan` refuses every plan that would lose data until the user has said what each operation means, with the engine's statement prompt doing the asking: flags on the command line, one refusal that lists every unanswered operation when nobody can answer, and a typed answer when a human runs it. The blanket consent is gone from the command. + +## The engine + +- `@prisma/cli-engine` 0.7.0 (prisma/prisma-cli#337; ADR 0006 there; engine README § statements). Until it is published, pin the PR's preview build in `packages/1-framework/3-tooling/cli/package.json` (`https://pkg.pr.new/@prisma/cli-engine@337`, both the dependency and the peer) and say so in the commit; the pin is switched to `0.7.0` in its own commit when it is on the registry. The PR must not merge on the preview pin. +- A command declares `statements: { rename: { arity: 1, brief }, delete: { arity: 1, brief } }`. The engine parses those flags for that command, keeps the values in argv order on the run state, and never shows them to the handler. Remove the `--rename` flag declaration slice 1 added to `migration plan` (keeping it is a construction error). +- `ctx.prompt.statements(questions, { last: true })` returns `{ verb, text, values }` per question, answered from the flags first; non-interactively it throws one `CLI.CONSENT_REQUIRED` that lists every unanswered question with its flag forms; interactively it asks each in turn and re-asks on a rejected answer. `last: true` throws `CLI.CONSENT_UNUSED` for any leftover value before the command acts. +- The run's ordered statement values are the source of the statement list that slice 1 built from `args.flags.rename`; read them through the engine's API for that (see the engine README; the implementer finds the exact accessor). + +## What to build + +1. **Declare and read.** `migration plan` declares `rename` and `delete` with briefs. The statement list `{ verb, text }[]` is built from the engine's ordered values; `rename` entries resolve as in slice 1; `delete` entries are kept for matching. +2. **Plan, then ask.** Plan with the `rename` statements. For each `dataLoss` entry of the delta leg (and of the baseline leg in an auto-baseline), build one question: text from the operation's label plus what is lost (`Table "Legacy" would be dropped and its rows lost.`); `subject` written as the coordinate (`Legacy`, `User.name`, or the storage name, the `kind: 'storage'` case saying so in the question); `verbs: ['rename', 'delete']` for a model or field subject, `['delete']` for storage; `forms: { rename: ':' }`; `validate`: a `delete` answer must equal the subject; a `rename` answer must parse, resolve against the two contracts with the slice 1 resolver, and have its old side equal to the subject. Ask all questions in one `statements(..., { last: true })` call. Since the engine answers from flags before prompting, a `--delete Legacy` given up front is consumed here. +3. **Re-plan with new renames.** A `rename` answer that came from the prompt is a statement the plan did not have: append it, plan again, and expect no `dataLoss` entry for that subject; if one remains, fail with the new `MIGRATION.STATEMENT_DID_NOT_RESOLVE_LOSS` (error reference entry). A `rename` answered from a flag was already in the statement list, so it was in the first plan; only a prompt answer triggers a re-plan. +4. **Write only when answered.** Nothing is written until every question is answered. The auto-baseline consent (`refuseUnconsentedDestructiveBaseline`, `consentToken`, `destructiveBaselineQuestion`, the `consent`/`planHash` round trip through `MigrationPlanOptions`) is removed; `MIGRATION.DESTRUCTIVE_CHANGES` and `MIGRATION.CONSENT_PLAN_MISMATCH` stop being raised by this command (keep the codes if `db update` still raises them until dispatch 4; retire them there). +5. **Output.** `Statements applied` lists `delete` lines (`delete model "Legacy" (1 operation)`) with the operation positions of the entry each consented to; JSON `appliedStatements` entries carry `verb`. `dataLoss` is not separately printed on success. The `⚠` marker on destructive operations stays. +6. **Docs.** The CLI README `migration plan` section: statements, the refusal, the interactive answer, `--confirm` no longer used; the error reference for `CLI.CONSENT_REQUIRED`/`CONSENT_UNUSED` as this command raises them (link the engine's entries) and the new code; the Migration System doc § Statements and § `migration plan`. + +## Not in this dispatch + +`db update` (dispatch 4); `allow`; journeys and upgrade fragments (dispatch 5); retiring the two old error codes from the errors package while `db update` still uses them. + +## Tests + +CLI tests through `createOrmTestCli` with the engine's scripted `answers` and `isTty` runtime options: refusal lists every operation with both flag forms; `--delete Legacy` writes the plan and reports the statement; `--rename Legacy:Archive` on the flag resolves the loss in the first plan; a typed `rename Legacy:Archive` re-plans and succeeds; a typed `delete`; a rejected typed answer is re-asked (clack) or fails (line reader, the test runtime); `--delete Nope` fails with `CLI.CONSENT_UNUSED` before any package is written; `--yes` does not answer; a storage subject offers only `delete`; an auto-baseline with a destructive delta asks before writing either package; the old `--confirm ` no longer consents (the engine still parses it; nothing asks). Update `migration-plan.test.ts` and `migration-plan-statements.test.ts` where they used the old consent. + +## Halt conditions + +Stop and report if the engine gives no way to read the ordered statement values from a handler; if `createOrmTestCli` cannot drive the engine's interactive path; or if the preview package cannot be installed with `pnpm install` in this workspace. + +## Gate + +Standard, plus `pnpm check:error-reference`, `pnpm lint:framework-vocabulary`, both rename-statements journeys, `test/integration/test/cli.migration-plan-ref-aware.e2e.test.ts`, `cli-journeys/data-transform-strategies.e2e.test.ts` and `migration-plan-details.e2e.test.ts` (they plan drops without an auto-baseline and will now need statements or will assert the refusal). diff --git a/projects/migration-statements/slices/refusal/dispatches/04-refusal-on-db-update.md b/projects/migration-statements/slices/refusal/dispatches/04-refusal-on-db-update.md new file mode 100644 index 000000000000..f2a606c7ff77 --- /dev/null +++ b/projects/migration-statements/slices/refusal/dispatches/04-refusal-on-db-update.md @@ -0,0 +1,45 @@ +# Dispatch 4 — The refusal and `allow` on `db update` + +**Slice:** [`../spec.md`](../spec.md) § "`--delete` and how statements consent", § "Access widening (`allow`)" · **Plan:** [`../plan.md`](../plan.md) dispatch 4 · **Builds on:** dispatch 3 + +## Step 0 — already done in dispatch 3 + +The slice 1 branch is merged (`0bf6536f64`). Skip to the outcome. + +## Carried from dispatch 3's review (D3-8) + +A delete is reported through the origin model (`delete field "Profile.nickname"`) although the question wrote it as `User.nickname`. Describe it the way the question writes it, in both commands' `Statements applied` and JSON, with a test. + +## Old step 0 (kept for the record) + +`bot/tml-3475-statement-renames` moved after this branch was cut (the merge-queue test fix, a merge of `main`, and two follow-ups). Merge it into `tml-3476-statement-refusal` with `git merge --no-ff`; do not rebase. Resolve conflicts keeping both sides; the base's rewritten test "writes nothing when the delta planner refuses after the baseline was planned" is the one dispatch 3 already satisfies. Run the gate on the merge before starting the rest. + +## Outcome + +`db update` asks the same per-operation questions as `migration plan`, with one engine mechanism; in apply mode it also asks, with the verb `allow`, before it widens who can read or write rows; the blanket consent and its errors are gone from the CLI. + +## What to build + +1. **Declare and read.** `db update` declares `statements: { rename, delete, allow }` with briefs, drops the slice 1 `--rename` flag declaration, builds the rename list from `ctx.statements.take('rename')`, and passes the `delete` and `allow` answers through the same `answerDataLoss`-style callback as `migration plan` (share the question builder in `control-api/statements/data-loss-questions.ts`; add an `accessWidening` question builder next to it). +2. **Plan, then ask, then apply.** `executeDbUpdate` plans once (the current pre-plan), asks for every `dataLoss` entry across all spaces (an extension space's entry has a storage subject and offers `delete` only) and, in apply mode, every `accessWidening` entry (verb `allow` only; subject the model; question text from the label: `Disable row-level security on "User" would widen who can read and write its rows.`), all in one `statements(..., { last: true })` call, then applies. A typed `rename` re-plans as on `migration plan`. A dry run (`--dry-run`) plans, lists `dataLoss` and `accessWidening` in its output, and asks nothing; a `--delete` or `--allow` given to a dry run is unused and fails with `CLI.CONSENT_UNUSED`, which is the engine's rule, so say so in the README. +3. **Remove the blanket consent.** `guardDestructiveChanges`, the consent prompt and helpers in `orm/db/consent.ts`, `acceptDataLoss`'s CLI path, `DESTRUCTIVE_CHANGES` and `CONSENT_PLAN_MISMATCH` from `DbUpdateFailureCode`, `computePlanHash`'s consent use, and `MIGRATION.DESTRUCTIVE_CHANGES` / `MIGRATION.CONSENT_PLAN_MISMATCH` from the errors package and the error reference (nothing raises them any more after dispatch 3). `DB_INIT_ADDITIVE_ONLY_FIX` in `db-init-failure.ts` stops telling users to pass `--confirm`. The control API keeps `acceptDataLoss` for programmatic callers and gains `statements`. +4. **Retry command.** `retryCommandFor` carries every verb's values (`rename`, `delete`, `allow`) in order. +5. **No origin snapshot.** With statements but no snapshot, `rename` fails as in slice 1; subjects are storage names, `delete` consents to them, and the refusal says the names are storage names because the origin contract is unknown. +6. **Output.** `Statements applied` and JSON `appliedStatements` carry `delete` and `allow` lines with their positions, offset per space as the positions already are. JSON `dataLoss` and `accessWidening` appear on the dry-run document. +7. **Docs.** CLI README `db update`: statements, the questions, `allow`, the dry run, `--confirm` no longer consents; the Migration System doc § `db update` (including that recorded extension migrations contribute no `accessWidening` question, because a written migration is reviewed before it runs); the error reference; the CLI Style Guide's consent section: statements are the data-loss form, `consent`/`--confirm` stays for one-thing-at-stake commands. + +## Not in this dispatch + +Journeys, upgrade fragments, the skill references (dispatch 5). + +## Tests + +CLI tests through the mocked client and the control-API tests with a mock family: refusal lists every operation across spaces; `--delete` applies; `--allow User` applies a policy drop and its absence refuses in apply mode and is not asked in dry run; a typed answer; `--confirm ` no longer consents (nothing asks for it); `CLI.CONSENT_UNUSED` for a stray value; retry command carries all verbs; no-snapshot storage subjects. Update `db-update-consent.test.ts`, `db-update.test.ts`, `db-update-to-resolution.test.ts` and `control-api/db-update.test.ts`. + +## Halt conditions + +Stop and report if sharing the question builder between the two commands needs the control API to know about the engine's prompt types (it must not: the callback boundary stays at the command); if the dry run's `CONSENT_UNUSED` behaviour under the engine cannot be made to say something useful; or if retiring the two error codes breaks a published surface a fragment must name. + +## Gate + +Standard, plus `pnpm check:error-reference`, `pnpm lint:framework-vocabulary`, both rename-statements journeys, `cli-journeys/db-update-workflows.e2e.test.ts`, `drift-schema.e2e.test.ts`, `mongo-db-update-consent.e2e.test.ts`, `rename-table-migration.sqlite.e2e.test.ts`, `cli.db-update.e2e.test.ts` and `cli.db-update.preflight-gaps.e2e.test.ts` (they pass `--confirm` today and must change to statements). diff --git a/projects/migration-statements/slices/refusal/dispatches/05-journeys-docs-fragments.md b/projects/migration-statements/slices/refusal/dispatches/05-journeys-docs-fragments.md new file mode 100644 index 000000000000..3cfb1bcf2819 --- /dev/null +++ b/projects/migration-statements/slices/refusal/dispatches/05-journeys-docs-fragments.md @@ -0,0 +1,49 @@ +# Dispatch 5 — Journeys, docs, upgrade fragments + +**Slice:** [`../spec.md`](../spec.md) § "Slice done conditions" · **Plan:** [`../plan.md`](../plan.md) dispatch 5 · **Builds on:** dispatch 4 + +## Outcome + +The slice done conditions hold through journeys on all three targets; every document and skill that describes consent describes statements; the upgrade fragments name every behaviour change and every removed export, and are validated by execution. + +## Journeys (`test/integration/test/cli-journeys/`) + +Next to the `rename-statements-migration*` journeys, one Postgres file, one SQLite file and one MongoDB file, labelled `S` steps as the existing ones are: + +1. **Rename plus delete through `migration plan`:** rename a model and a field and remove another model; without `--delete`, `migration plan --json` exits 2 with `CLI.CONSENT_REQUIRED` whose `nextActions` contain `--delete `; with that exact flag it writes the plan; `migrate` applies it on tables with rows, a unique, a foreign key from another table, a secondary index and (Postgres) a check; rows and objects present under the new names, the deleted table gone, a further plan empty, `db verify --schema-only` clean. +2. **The same through `db update`:** no prompt; `--confirm ` alone is refused with `CLI.CONSENT_REQUIRED`; a second `db update` with the same statements fails on the first statement (`MIGRATION.STATEMENT_UNRESOLVED`). +3. **A required field:** adding a non-nullable field plans and applies on all three targets with no refusal. +4. **Postgres only, `allow`:** `db update` that drops a policy refuses in apply mode without `--allow ` (`nextActions` names it), applies with it, and a dry run lists it under `accessWidening` without asking. +5. **MongoDB:** a collection drop is refused and `--delete ` consents; a `rename` is still refused by the Mongo planner with `statementRefused`. +6. **No snapshot (Postgres and SQLite, one each):** `db update --db ` with a dropped model refuses with a storage subject and the note that the origin is unknown; `--delete ` applies; the SQLite form is the unqualified name, the Postgres form is `public.user`. + +Add the planner test the review asked for: a codec hook that emits a destructive call, proving the hook-call-to-field match survives the control-policy partitions (`../plan.md` open items). + +## Carried from dispatch 4's review (D4-6) + +A programmatic dry run (`mode: 'plan'`) ignores a `delete` or `allow` statement that matches no subject, where apply mode fails with `MIGRATION.STATEMENT_ANSWERS_NO_QUESTION`. Check them in plan mode too, without asking, so both modes refuse; test it. The new code goes in the app upgrade fragment's control-API list. + +## Docs and skills + +- `skills/prisma-8/references/migrations.md`, `debug.md` and `SKILL.md`: replace `MIGRATION.DESTRUCTIVE_CHANGES` and `--confirm ` with the statements and the refusal; the error-code table lists `CLI.CONSENT_REQUIRED`, `CLI.CONSENT_UNUSED` and `MIGRATION.STATEMENT_DID_NOT_RESOLVE_LOSS`; `skills/journey-tests/*` that mention `--confirm` for these commands. +- `docs/commands/SUMMARY.md` and `docs/reference/error-reference.md`: already touched in dispatches 3 and 4; re-read for stale consent wording. +- `docs/architecture docs/subsystems/7. Migration System.md` § Statements: one coherent section for `rename`, `delete`, `allow`, the questions and the interactive answer; § `db update`: recorded extension migrations contribute no `accessWidening` question. +- The slice spec and the project spec: nothing to change unless the implementation diverged; report any divergence instead of editing them. + +## Upgrade fragments (`upgrade-instructions/pending/migration-statement-refusal/`) + +**app/instructions.md** (extend the fragment dispatch 1 created): +- `migration plan` refuses every plan that loses data, not only auto-baselines; the way out is `--delete ` or `--rename`; detection: scripts and CI that run `migration plan` on a contract change that drops something (no reliable regex; say so and give the `nextActions` form). +- `db update --confirm ` no longer consents; detection regex on `--confirm` next to `db update`; replacement `--delete ` per operation, `--allow ` before an access widening. +- `db update` reads the origin snapshot on every run (behaviour note, no action). +- Control API: `executeMigrationPlanCommand` and `executeDbUpdate` require `answerQuestions`; `consent`, `carryEmittedExtensionDirs` and `acceptDataLoss`'s CLI path are gone; `retryCommandFor` takes `statements`. Detection on each identifier. +- Removed from `@prisma/orm-framework/errors/execution`: `ERROR_CODE_DESTRUCTIVE_CHANGES`, `errorDestructiveChanges`, `ERROR_CODE_CONSENT_PLAN_MISMATCH`, `errorConsentPlanMismatch`. Removed from `@prisma/orm-toolchain/cli/control-api`: `'DESTRUCTIVE_CHANGES'` and `'CONSENT_PLAN_MISMATCH'` in `DbUpdateFailureCode`, `DbUpdateFailure.destructiveChanges` / `.consentPlanMismatch`, `DestructivePlanOperation`, `DestructiveChangesVerdict`, `ConsentPlanMismatchVerdict`. Removed CLI codes `CLI.CONSENT_TOKEN_UNRESOLVED`, `CLI.CONSENT_OPERATIONS_MISSING`. + +**extension/instructions.md** (new): +- `MigrationPlannerSuccessResult` requires `dataLoss` and `accessWidening` (`MigrationOperationSubject[]`); `MigrationStatementSubject`; `PerSpacePlan` carries both; `ControlFamilyInstance.storageNameOf` is required; `StatementVerb` gains `delete`; `DeleteMigrationStatement`; `RecreateTableCall` (SQLite) takes `lossyColumns`; `AlterColumnTypeCall` (Postgres) takes `operationClass`; `planFieldEventCalls`; the class changes from dispatch 1 as they affect a planner an extension implements. Detection regexes per identifier, with the import paths. + +Validate both by execution per `skills-contrib/record-upgrade-instructions/SKILL.md` § validation (restore `examples/` and `packages/3-extensions/` to the base, apply the fragment, no diff outside tests; test dirs stay at base; run the example tests). The engine pin must not be in the fragments: it is a dependency bump the PR body states. + +## Gate + +Standard, plus `pnpm check:upgrade-coverage --mode pr --prev bot/tml-3475-statement-renames` (must pass now), `pnpm lint:skills`, `pnpm lint:docs`, every journey named above and every journey dispatch 4 changed, and the example tests the fragment validation runs. diff --git a/projects/migration-statements/slices/refusal/dispatches/06-whole-slice-review-fixes.md b/projects/migration-statements/slices/refusal/dispatches/06-whole-slice-review-fixes.md new file mode 100644 index 000000000000..b2c1d74a47bd --- /dev/null +++ b/projects/migration-statements/slices/refusal/dispatches/06-whole-slice-review-fixes.md @@ -0,0 +1,41 @@ +# Dispatch 6 — Whole-slice review fixes + +**Reviews:** [`../reviews/system-design-review.md`](../reviews/system-design-review.md) (S01–S20) and [`../reviews/code-review-final.md`](../reviews/code-review-final.md) (F01–F10, with the probe log). Read both before starting. Fix on new commits; no push. + +## Decisions already taken + +- **S01 / F02, namespace-wide `--delete`:** deferred to slice 3; recorded in the slice spec and `deferred.md` (`4fe1ebe66c`). Nothing to build. Update the app upgrade fragment and the README if they promise it. +- **S06 renaming of the class value, S20 (`allow` outside the four verbs), S07, S08:** close-out ADR or later slices. Not here. +- **S14, S16, F03, F10:** engine side; handled on prisma/prisma-cli#337 (`values()` and the ADR are done; shell-quoting of flag forms and the prompt fixes are requested). + +## Fix in this dispatch + +**Correctness** +- **F01 (high):** `answerPlanQuestions` must decide whether a typed rename resolved its loss by the asked-about operation being gone from the re-plan, not by subject equality; a loss left on the same subject is asked in the next round. The probe: `T.ratio Float` → `T.ratio2 Int`; `--rename T.ratio:T.ratio2 --delete T.ratio` on the command line works, the same rename typed fails with `STATEMENT_DID_NOT_RESOLVE_LOSS` and the advice then drops the column. Test both paths. +- **S02 (high):** `executeMigrationPlanCommand` parses `delete`/`allow` statements and drops them; pass them as pre-answers exactly as `executeDbUpdate` does, and refuse unused ones (`STATEMENT_ANSWERS_NO_QUESTION`). Test. +- **F04 / S17:** re-pin the root override to the current #337 head `84ce08e` (has `949a03a` longest-subject matching, `take`, `values()`), run `pnpm install`, and add a CLI test with subjects `Legacy` and `Legacy:x` in one plan where `--delete Legacy:x` answers the right question. +- **F05:** the dry run's "An apply asks about" list and the JSON subjects (`subjectDisplay` in `migration-blocks.ts`) must print the same text the questions use (`User.nickname` after a rename; no `public.` prefix where the question has none). Carry the question's subject text in each entry. Test on Postgres with a renamed model. +- **F06:** a dry run given the correct `--delete`/`--allow` flags must not fail with `CONSENT_UNUSED`; check them in plan mode with `refuseUnusedConsents` and consume the matching ones (use `ctx.statements.values()` to read, then the engine's matching by asking the questions with a plan-mode answerer that accepts flag answers and never prompts; say which you chose). The exact apply command must preview cleanly. Test. +- **S05:** fill `PerSpacePlan.dataLoss` for recorded paths in the aggregate planner (`resolve-recorded-path.ts` and `planner.ts`), using `storageNameOf`, and delete the strategy check in `db-run.ts`. +- **S09:** split `acceptAccessWidening` from `acceptDataLoss` on the control API; `acceptDataLoss: true` no longer answers `allow` questions. Fragment and README. +- **F09:** a policy replacement whose policies have generated (wire) names is a create-then-drop with different names; exclude it from `accessWidening` by pairing drop and create on the same table and the same wire-name prefix (`parseWireName`), as the explicit-name case already pairs by name. Test both. + +**Advice text** +- **F07:** with no origin snapshot, the refusal's next actions include the snapshot recovery steps (slice 1's `STATEMENT_ORIGIN_UNKNOWN` advice) before `--delete `, and say that `--delete` loses the rows. Test the text. +- **F08:** offer `rename` only where it can succeed: not for a lossy type change on a field that keeps its name, not on MongoDB. Test the verbs per case. + +**Names and shape (architect)** +- **S04:** `MigrationStatementSubject` → `MigrationSubject` (and its JSON form), everywhere, docs and fragment included. +- **S03:** export `AnswerPlanQuestions`, `PlanQuestion`, `PlanAnswer`, `PlanQuestionVerb` from `exports/control-api.ts`. +- **S10:** either give `DeleteMigrationStatement` and the delete branch of `describeMigrationStatement` a production caller (the `Statements applied` delete line should use it) or remove them; fix the Migration System doc line 223 accordingly. +- **S12:** rename the loop so it is not the same word as the callback type; move the access-widening question builder out of `data-loss-questions.ts` (or rename the module `plan-questions.ts`); one `sameSubject` helper replaces the four `JSON.stringify` comparisons. +- **S11:** declare the `{ dataLoss, accessWidening }` pair once (`PlanSubjects`) and reuse it; rename one of the two `planSubjects` functions; rename `SubjectTarget` to what it is. +- **S13:** the error reference says `CLI.CONSENT_UNUSED` (engine, CLI flags) and `MIGRATION.STATEMENT_ANSWERS_NO_QUESTION` (control API, programmatic) are the same condition at two boundaries. +- **S15:** one shared verb-declaration object for both commands; fix the engine spec sentence that says the verbs are registered once for the ORM (that file is the orchestrator's: report the sentence, do not edit it). +- **S06 docs:** the five "no neutral middle class" comments in the Postgres and SQLite sources, and the Style Guide's "drops are destructive", say what `widening` means now. +- **S19:** the class doc says `data` operations run what their author wrote and may lose data by design; only the planner's own classification promises data loss ⇔ `destructive`. +- **S18:** a shared test helper that asserts, for a planner's result, that every `destructive` operation has exactly one `dataLoss` entry and no `widening` one has any; use it in the three targets' data-loss tests. + +## Gate + +The standard gate, `lint:throws`, `check:error-reference`, `check:upgrade-coverage --mode pr --prev bot/tml-3475-statement-renames`, `lint:docs`, `lint:skills`, every journey dispatches 3 to 5 named, and the probes F01, F04, F05, F06, F07 re-run through the built CLI with their outputs pasted in the report. diff --git a/projects/migration-statements/slices/refusal/manual-qa.md b/projects/migration-statements/slices/refusal/manual-qa.md new file mode 100644 index 000000000000..e322f0d97ea8 --- /dev/null +++ b/projects/migration-statements/slices/refusal/manual-qa.md @@ -0,0 +1,42 @@ +# Manual QA — slice 2, the refusal and `--delete` (TML-3476) + +> **Be the user.** Run the built CLI against real databases in a scratch project, read every message as someone who has never seen this feature, and judge what the journeys cannot: whether a refusal tells you what to type next, whether the interactive questions make sense, and whether anything loses data without you saying so. +> +> **Out of scope.** Do not re-run `pnpm test`, the journeys or CI lints. Decide from what the commands print and what the database holds, not from the source. +> +> **Spec:** `projects/migration-statements/slices/refusal/spec.md` · **Project spec:** `projects/migration-statements/spec.md` · **Engine:** `wip/prisma-cli` README § statements (prisma/prisma-cli#337) + +## What this script is testing + +**The problem.** A plan that dropped a table or column ran after one yes/no consent for the whole plan (`--confirm `), or, on `migration plan`, with no consent at all unless it was an automatic baseline. + +**What changed.** Both commands refuse any plan that loses data until the user has said what each operation means. Non-interactively the refusal lists every operation with the flag that answers it (`--delete Legacy`, `--rename Legacy:`); in a terminal each operation is a question the user answers by typing `delete` or `rename Legacy:Archive`. `--confirm` no longer consents to data loss. Before `db update` drops a row-level-security policy or disables row-level security it asks the same way with `allow`. Operations that lose no data (`SET NOT NULL`, safe type widenings, MongoDB index and validator changes) no longer count as destructive. + +**Audiences.** End users in a terminal (the interactive path is the one the journeys cannot drive), scripts and agents (the refusal), extension authors (the upgrade fragment), agents using the `prisma-8` skill. + +## Scenarios + +| # | Scenario | What it proves | Isolation | +| - | -------- | -------------- | --------- | +| 1 | Postgres, terminal: remove a model and rename another, run `migration plan` with no flags, answer the questions by typing | The question text is understandable; `delete` and `rename X:Y` both work typed; a wrong answer is explained and re-asked; the written plan matches the answers | tmpdir, real TTY | +| 2 | The same non-interactively (`< /dev/null` or `--no-interactive`) | The refusal lists every operation and the exact flags; pasting them works; nothing was written by the refused run | tmpdir | +| 3 | SQLite, `db update` interactive, then `--confirm ` only, then `--delete` | `--confirm` alone is refused with `CLI.CONSENT_REQUIRED`, which names the leftover `--confirm` and the flags that answer; `--confirm` with every question answered is `CLI.CONSENT_UNUSED`; `--delete` applies; a second run with the same statements fails and says why | tmpdir, real TTY | +| 4 | A stray or misspelled `--delete`, and `--delete` on a dry run | A stray `--delete` with every question answered is `CLI.CONSENT_UNUSED`; a misspelled one next to an unanswered question is named in the `CLI.CONSENT_REQUIRED` refusal; nothing is written or applied. On a dry run a matching `--delete` is accepted and its question marked `(answered)`, and one that matches nothing is `MIGRATION.STATEMENT_ANSWERS_NO_QUESTION` | tmpdir | +| 5 | Postgres, drop a policy through `db update` | The dry run lists it under access widening and asks nothing; the apply asks with `allow`; `--allow User` applies | tmpdir | +| 6 | Add a required field to a populated table, Postgres and MongoDB; tighten a column to NOT NULL with and without NULLs | No question; the NULL case fails clearly without losing data | tmpdir | +| 7 | `db update --db ` with no snapshot, dropping a model | The refusal names the table by its storage name and says the origin is unknown; `--delete ` applies | tmpdir | +| 8 | A field on a renamed model: `--rename Profile:User` with `Profile.nickname` dropped | The refusal suggests `--rename 'User.nickname:User.'`; that form resolves; the `Statements applied` block names things the way you typed them | tmpdir | +| 9 | MongoDB: drop a collection through both commands; `--rename` | Refused until `--delete Event`, and the question says how to keep the documents by hand (rename the collection in `mongosh` before a plan that drops it is applied; a migration written by `migration plan` still drops it); `--rename` refused with advice that loses nothing | tmpdir | +| 10 | Read the app and extension upgrade fragments as their audiences **(judgement)** | A script author knows what to replace `--confirm` with; a planner author knows every field and type to add | read-only | +| 11 | Ask the skill how to delete a field and how to answer a refusal **(judgement)** | `skills/prisma-8/references/migrations.md` leads to `--delete`, and to `--rename` when the user meant a rename | read-only | +| 12 | Exploratory: 20 minutes | Two `--delete`s, `--delete` for a model whose field is also dropped, an auto-baseline plan with a drop, `--json` in a TTY (does it still prompt to stderr?), `--yes`, Ctrl-C at a question (exit 3), a placeholder plan with a drop | tmpdir | + +## Pre-flight + +1. `git branch --show-current` is `tml-3476-statement-refusal`; `git status --short` shows no source changes. +2. `pnpm install --frozen-lockfile` and `pnpm build`. The engine is the preview build of prisma/prisma-cli#337 through the root `pnpm.overrides`; confirm `node_modules/@prisma/cli-engine/package.json` says 0.7.0. +3. Scratch projects under `wip/qa/`, one per scenario, from the built CLI's `prisma orm init`. Postgres through `@prisma/dev`; SQLite files; MongoDB through `mongodb-memory-server`. Scenarios 1 and 3 need a real terminal (`script`, `expect`, or run them yourself in a pty); the plain line reader fails on a wrong answer where clack re-asks, note which renderer you got. + +## Report + +Write `projects/migration-statements/manual-qa-reports/-qa-slice-2.md`: a verdict per scenario (✅ ⚠️ 🛑), every finding numbered with the exact command, output and what you expected, and a note of anything the script did not cover. A finding is a blocker when an error's advice loses data if followed, or when data is lost with no statement. diff --git a/projects/migration-statements/slices/refusal/plan.md b/projects/migration-statements/slices/refusal/plan.md new file mode 100644 index 000000000000..3f344ed2b162 --- /dev/null +++ b/projects/migration-statements/slices/refusal/plan.md @@ -0,0 +1,58 @@ +# Slice plan — Both commands refuse data loss until the user states what each operation means + +**Spec:** [`spec.md`](./spec.md) · **Linear:** [TML-3476](https://linear.app/prisma-company/issue/TML-3476) · **Branch:** `tml-3476-statement-refusal` · **Depends on:** `@prisma/cli-engine` 0.7 from [`../engine-statement-prompt/`](../engine-statement-prompt/spec.md) + +Every dispatch runs on Opus. Validation gate unless stated: `pnpm typecheck`, `pnpm --filter lint`, `pnpm --filter test`, `pnpm lint:deps` when imports change, `pnpm fixtures:check` when classes or rendering change, `pnpm check:error-reference` when error codes change, `pnpm lint:framework-vocabulary` when `packages/1-framework` changes. Never the full integration or package suites locally; run named files, and both rename-statements journeys plus the new ones. + +Dispatches 1 and 2 do not touch the prompt and start before the engine ships. Dispatches 3 to 5 need the engine pin. + +## Dispatches + +### 1 — Destructive means data loss, on every target + +**Outcome.** `SET NOT NULL`, the known safe type widenings, SQLite rebuilds caused only by a nullability tightening, MongoDB index drops, validator tightenings and change-stream image settings are `widening`; drop table, drop column, drop collection, and type changes that can lose values stay `destructive`. The `MigrationOperationClass` doc and the Migration System doc § Operation Model say what each class means. Example migrations and planner goldens are regenerated where a class changed; `examples/prisma7-adoption/test/handover.test.ts` expectations follow. + +**Builds on.** Slice 1. + +**Hands to.** A `destructive` set the refusal can refuse without blocking a required column. + +### 2 — The planner result names what each destructive operation would lose + +**Outcome.** The framework type `MigrationSubject` beside `AppliedMigrationStatement`; the planner success result carries `dataLoss: { operationIndex, subject }[]` and `accessWidening: { operationIndex, subject }[]`; the SQL family maps a dropped or type-changed table or column back to the origin contract's model or field when `fromContract` is present, else to a storage subject; MongoDB maps a dropped collection to its model; the transient DROP of a policy replacement is not listed. The aggregate planner and `planFromDiff` pass both lists through. The missing-origin check in `resolve-statements.ts` moves inside the per-statement loop. + +**Builds on.** Dispatch 1. + +**Hands to.** The structured refusal, ready for the CLI to render and for the prompt to consume. + +### 3 — The engine pin, the verbs, and the refusal on `migration plan` + +**Outcome.** `@prisma/cli-engine` 0.7 pinned; each command declares its verbs from one shared table (`rename`, `delete`, and on `db update` `allow`); `migration plan` plans, turns every `dataLoss` entry into a `ctx.prompt.statements` question, re-plans with any `rename` answers, matches `delete` answers, and writes only when nothing is left unanswered. The auto-baseline consent (`refuseUnconsentedDestructiveBaseline`, `consentToken`, the plan-hash round trip) is gone; `MIGRATION.DESTRUCTIVE_CHANGES` and `CONSENT_PLAN_MISMATCH` are retired from this command; `MIGRATION.STATEMENT_DID_NOT_RESOLVE_LOSS` is new. `Statements applied` and JSON `appliedStatements` carry `delete` lines with `verb`. Statement order comes from the engine's ordered verb-flag list. + +**Builds on.** Dispatch 2 and the engine. + +**Hands to.** The refusal shape on one command; the `delete` path end to end offline. + +### 4 — The refusal and `allow` on `db update` + +**Outcome.** `db update` asks the same questions per destructive operation and, in apply mode, an `allow` question per access-widening operation; `guardDestructiveChanges`, the consent prompt in `orm/db/consent.ts`, `acceptDataLoss`'s CLI path, `MIGRATION.DESTRUCTIVE_CHANGES` and `CONSENT_PLAN_MISMATCH` are gone from the CLI (the control API keeps `acceptDataLoss` and gains `statements`); a dry run lists `dataLoss` and asks nothing; `retryCommandFor` carries every verb. Without an origin snapshot, `delete` works on storage subjects and the refusal says why the names are storage names. + +**Builds on.** Dispatch 3. + +**Hands to.** Both commands on the one consent model. + +### 5 — Journeys, docs, upgrade fragments + +**Outcome.** The slice done conditions hold through journeys on Postgres, SQLite and MongoDB (rename plus delete through both commands; the refusal's `nextActions` flag then succeeds; `--confirm ` no longer consents; a required field plans without refusal; `--allow` on a policy drop). The CLI README's two command sections, the Migration System doc § Statements and § `db update`, the error reference, the CLI Style Guide's consent section, and `skills/prisma-8` describe statements as the data-loss consent. The app upgrade fragment records that `--confirm` no longer consents to data loss and that `migration plan` refuses every plan that loses data; the extension fragment records `dataLoss`, `accessWidening` and `MigrationSubject`. Every `--confirm` in tests and journeys for these two commands is replaced. + +**Builds on.** Dispatch 4. + +**Hands to.** Slice DoD. + +## Open items + +- The PR must not merge while `pkg.pr.new` appears in any `package.json` or the root `pnpm.overrides`: the engine pin switches to the published `@prisma/cli-engine` 0.7.0 first. +- Dispatch 5's app upgrade fragment names the control-API break: `executeMigrationPlanCommand` requires `answerDataLoss`; `consent` and `carryEmittedExtensionDirs` are gone. + +- Dispatch 5 adds a planner test where a codec hook emits a destructive call, proving the hook-call-to-field match (by object identity) survives the control-policy partitions. + +- Whether the Mongo planner's `statementRefused` for `rename` needs its wording changed now that `delete` exists: dispatch 3 reads the text and decides. diff --git a/projects/migration-statements/slices/refusal/spec.md b/projects/migration-statements/slices/refusal/spec.md new file mode 100644 index 000000000000..fa6d056fe390 --- /dev/null +++ b/projects/migration-statements/slices/refusal/spec.md @@ -0,0 +1,108 @@ +# Slice spec — Both commands refuse data loss until the user states what each operation means + +**Project:** [`projects/migration-statements/`](../../spec.md) · **Slice 2** · **Linear:** [TML-3476](https://linear.app/prisma-company/issue/TML-3476) · **Branch:** `tml-3476-statement-refusal` (on top of slice 1, prisma/orm#30638) · **Depends on:** the engine slice [`../engine-statement-prompt/spec.md`](../engine-statement-prompt/spec.md), published as `@prisma/cli-engine` 0.7. + +## At a glance + +A user removes the `Legacy` model and renames `Profile` to `User`, then plans without saying what they mean about `Legacy`: + +```text +$ prisma migration plan --name tidy-users --rename Profile:User +✖ [CLI.CONSENT_REQUIRED] 1 operation would lose data, and the session is not interactive. + Table "Legacy" would be dropped and its rows lost. +→ To rename it: --rename Legacy: +→ To delete it: --delete Legacy +``` + +With `--delete Legacy` the plan is written, and `Drop table "Legacy"` is listed as `destructive` with the statement that allowed it. Run by a human in a terminal, the same command asks: + +```text +? Table "Legacy" would be dropped and its rows lost. What do you mean? + (delete, or rename Legacy:) › delete +``` + +`prisma db update` behaves the same way against a live database. `--confirm ` no longer consents to data loss on either command, and there is no blanket consent: each operation that loses data has its own statement. Before `db update` drops a row-level-security policy or disables row-level security on a table, it asks the same way with the verb `allow`: `--allow User` consents to one operation that changes who can read or write `User`'s rows. + +## Chosen design + +### What is refused + +An operation is refused when its class is `destructive`, and after this slice an operation is `destructive` only when it loses rows or values: + +| Operation | Class after this slice | Subject | +| --- | --- | --- | +| Drop table (Postgres, SQLite), drop collection (MongoDB) | destructive | the model | +| Drop column (Postgres, SQLite) | destructive | the field | +| Alter column type outside the known safe widenings (Postgres); SQLite table rebuild caused by a type change | destructive | the field | +| `SET NOT NULL`, safe type widenings (`int2`→`int4`→`int8`, `float4`→`float8`), SQLite rebuilds caused only by a nullability tightening | widening | none | +| MongoDB index drop, validator tightening, change-stream image settings | widening | none | + +`widening` means an operation that changes existing structure without losing data; its doc comment says so. A refused non-drop (the type change) is consented with `--delete ` in this slice, as the project spec's DoD says; `--convert` replaces that in slice 3. + +### The structured refusal + +The family planner's success result gains `dataLoss: readonly MigrationOperationSubject[]`, one entry per destructive operation in plan order: `{ operationIndex, subject }`. `subject` is a `MigrationSubject`: `{ kind: 'model' | 'field', namespaceId, model, field? }` when the family can map the operation's table or column back to the origin contract (`fromContract`), else `{ kind: 'storage', name }` with the storage name as the planner knows it. The framework type lives beside `AppliedMigrationStatement` and carries no family vocabulary. The SQL family maps through `model.storage.table` and `fields[f].column`; MongoDB through the collection name. A table no contract declared (drift) is a `storage` subject. + +The CLI turns each entry into one engine question (`ctx.prompt.statements`): question text from the operation's label, `subject` written as the coordinate (`Legacy`, `User.name`, or the storage name), `verbs: ['rename', 'delete']` for a model or field subject and `['delete']` for a storage subject. The engine handles the command line, the non-interactive refusal that lists every question, and the interactive prompt. The CLI never reads the TTY. + +### `--delete` and how statements consent + +- `--delete ` is an engine statement flag (each command declares its verbs from one shared table: `migration plan` declares `rename` and `delete`, `db update` all three). Its text is a model or field coordinate in slice 1's grammar, or the storage name when the refusal said so. A namespace coordinate (`--delete billing`) is deferred to slice 3, with the namespace renames (decided 2026-10-08 at the whole-slice review: the engine answers a question only by a value that equals its subject or starts with it, so a namespace-wide consent needs the ORM to pre-answer from the namespace, which slice 3 builds with its namespace nouns). In this slice every `delete` names one model, field or storage object. +- A `delete` never reaches the planner. Planning runs first with the `rename` statements; the plan's `dataLoss` entries are then matched against the answers. `validate` for a `delete` answer checks the text equals the subject; the text is any non-empty string, because a storage name may contain `:`. `validate` for a `rename` answer resolves the statement with slice 1's resolver against the two contracts and accepts only when its old side is the subject. +- A `rename` answered at the prompt or on the command line that passes validation is added to the statement list and the plan is run again, exactly as if it had been typed on the command line; the second plan must have no `dataLoss` entry for that subject, else the run fails (`MIGRATION.STATEMENT_DID_NOT_RESOLVE_LOSS`). Order between a run's `--rename` and `--delete` values comes from the engine's ordered verb-flag list. +- The missing-origin check in `resolve-statements.ts` moves inside the per-statement loop: with no origin contract, `rename` statements fail with `MIGRATION.STATEMENT_ORIGIN_UNKNOWN` and `delete` statements still work, because they are matched against the plan, not resolved against a contract. +- `--confirm` is no longer asked for by either command. `guardDestructiveChanges`, the plan-hash consent round trip, `MIGRATION.DESTRUCTIVE_CHANGES` and `CONSENT_PLAN_MISMATCH` go, on both commands, including `migration plan`'s auto-baseline consent, which becomes the same per-operation refusal. The control API keeps `acceptDataLoss` as the programmatic equivalent and gains `statements` as the preferred form. +- `migration plan` refuses before writing anything (slice 1's rule stands). An auto-baseline whose delta is refused writes nothing. + +### Access widening (`allow`) + +`db update` asks before applying an operation that changes who can read or write rows: disabling row-level security (which widens access) and dropping a row-level-security policy (which changes it; dropping a permissive policy narrows). Both stay `widening` as operation classes, and each operation is its own question, answered by its own `--allow `. The family's success result lists them in `accessWidening: readonly { operationIndex, subject, widens }[]` with the model as subject (`MigrationAccessChange`; `widens` is true for disabling row-level security, false for a policy drop); the CLI asks with verb `allow` only (`--allow User`). `migration plan` does not ask: the written migration is reviewed before it runs. The transient DROP in a policy replacement (DROP then CREATE of a changed policy) is not listed. + +### Output + +The `Statements applied` block gains `delete` and `allow` lines (`delete model "Legacy" (1 operation)`), and JSON `appliedStatements` entries carry `verb`. `describeMigrationStatement` writes them. A destructive operation in the human plan output keeps its `⚠` marker. + +### MongoDB + +The Mongo planner keeps refusing `rename` statements. Since `delete` never reaches a planner, a MongoDB project consents to a collection drop with `--delete ` and is not blocked before slice 4. + +## Coherence rationale + +The refusal, the verb that answers it, and the removal of the blanket consent ship together because a release with both consents is contradictory and a refusal with no way out is unshippable (project transitional-shape constraint 2). The reclassification ships here because the refusal would otherwise block adding a required field. The `allow` consent ships here because it is the same prompt shape and Will tied it to this slice on prisma/orm#30638. + +## Scope + +**In:** the reclassification on all three targets; `dataLoss` and `accessWidening` on the planner result and the family mapping to subjects; the `delete` and `allow` verbs through the engine; the per-statement origin check; removal of the blanket consent paths and their errors; output and JSON; the CLI README (`migration plan`, `db update`), the Migration System doc § Statements and § Operation Model, the error reference, the CLI Style Guide's consent section (statements are the data-loss form); upgrade fragments (app: `--confirm` no longer consents to data loss, `migration plan` refuses every plan that loses data; extension: `dataLoss`/`accessWidening` on the success result, `MigrationSubject`); journeys on Postgres, SQLite and MongoDB; the engine pin bumped to 0.7. + +**Out:** `--convert`, `--backfill`, value and namespace renames (slice 3); MongoDB rename statements (slice 4); the `@@map`-only rename gap (decided 2026-10-07: no statement; the refusal keeps slice 1's manual steps); any storage-object statement syntax (deferred). + +## Pre-investigated edge cases + +| Case | Disposition | +| --- | --- | +| `--delete` names a model that is not dropped | `CLI.CONSENT_UNUSED` from the engine. | +| `--delete User` when only `User.name` is dropped | Unused; the refusal said `User.name`. A model delete does not cover its fields. | +| `--rename Legacy:Archive` where `Archive` is not in the destination | Slice 1's `STATEMENT_UNRESOLVED`, before planning. | +| `db update --db ` with no snapshot | `rename` fails as in slice 1; subjects are storage names; `--delete users` works. The refusal says the names are storage names because the origin contract is unknown. | +| A destructive operation inside an extension contract space | Listed with its storage subject; the app's statements cannot name it. | +| `migration plan --to ^` (rollback) that drops a table | Refused like any plan; `--delete` consents. | +| Dry run (`db update --dry-run`) | Plans and lists `dataLoss`; asks nothing. | +| Same `--delete` twice | Second is unused. | + +## Slice done conditions + +- Project DoD journeys restricted to rename plus delete, on Postgres, SQLite and MongoDB: `migration plan --rename ... --delete Legacy` then `migrate`; the same through `db update` with no prompt; both commands refuse without `--delete`, and the refusal's `nextActions` contain the exact flag that then succeeds; `--confirm ` no longer consents; a second `db update` with the same statements fails on the first statement. +- A journey that adds a required field plans and applies with no refusal on all three targets. +- A `db update` on Postgres that drops a policy refuses without `--allow ` and applies with it. +- `pnpm fixtures:check` with no churn except the example migrations whose classes changed; `lint:framework-vocabulary` count unchanged. + +## Open questions + +None. + +## References + +- [`../../spec.md`](../../spec.md) decisions 1, 2, 6; requirements 4, 10, 12. [`../../plan.md`](../../plan.md) slice 2. [`../../deferred.md`](../../deferred.md). +- [`../engine-statement-prompt/spec.md`](../engine-statement-prompt/spec.md). +- `docs/CLI Style Guide.md` § Destructive operation confirmation; `docs/architecture docs/subsystems/7. Migration System.md` § Operation Model. +- prisma-cli `packages/cli-engine/src/execution/prompts.ts`. diff --git a/skills/journey-tests/02b-rename-with-statement.md b/skills/journey-tests/02b-rename-with-statement.md index b77ee773f4cb..8c4af1a0570b 100644 --- a/skills/journey-tests/02b-rename-with-statement.md +++ b/skills/journey-tests/02b-rename-with-statement.md @@ -12,6 +12,7 @@ - [ ] Runs `contract emit`. - [ ] Runs `migration plan --name rename-user-email --rename User.email:User.emailAddress`. - [ ] Runs `migration show ` and confirms the plan renames the column instead of dropping and adding it. +- [ ] If a plan without the statement is refused with `CLI.CONSENT_REQUIRED` for `User.email`, answers with `--rename User.email:User.emailAddress`, not `--delete User.email`. - [ ] Runs `db migrate`. ## Success criteria diff --git a/skills/prisma-8/SKILL.md b/skills/prisma-8/SKILL.md index bb1e3106f551..2fc69748f104 100644 --- a/skills/prisma-8/SKILL.md +++ b/skills/prisma-8/SKILL.md @@ -59,7 +59,7 @@ Open the reference whose triggers match the task. If more than one matches, open | --- | --- | --- | | Adopt / set up / first steps | [`references/quickstart.md`](references/quickstart.md) | new project, existing database, "what can I do with Prisma 8", "where do I start", "just ran createprisma", `npx create-prisma`, first steps, first query, `prisma orm init` greenfield setup, `contract infer` + `db sign` brownfield adoption, connect-write-read first arc, day-to-day commands (`contract emit`, `db init`, `db update`, `migration plan`, `db migrate`, `db schema`, `db verify`), flags `--target` / `--authoring` / `--schema-path` / `--probe-db` / `--output` | | Edit the data contract | [`references/contract.md`](references/contract.md) | schema, models, fields, attributes, relations, indexes, enums, value objects (composite types), type aliases, namespaces (Postgres schemas), cross-contract foreign keys (cross-space FK), polymorphic types (`@@discriminator` / `@@base`), extension namespaces (`pgvector.Vector(...)`, `postgis.Geometry(...)`), `prisma.config.ts` / `definePrismaConfig` / `ormConfig`, `prisma contract emit`, PSL, `contract.prisma`, `contract.ts`, `contract.json`, `contract.d.ts`, `@internal/postgres/config`, `@internal/postgres/contract-builder`, `@internal/mongo/config`, `extensions:`, pgvector, postgis, paradedb, Temporal / `temporal-polyfill` / `RUNTIME.TEMPORAL_UNAVAILABLE`, `@@control`, control policy (managed / tolerated / external / observed), soft delete, validations, callbacks | -| Author migrations | [`references/migrations.md`](references/migrations.md) | `db update` vs `migration plan`, `db migrate`, `migration new`, `migration show`, `db update --dry-run`, `db verify`, `db sign`, data migration, `dataTransform`, placeholder sentinels in framework-rendered `migration.ts`, `MIGRATION.HASH_MISMATCH`, `MIGRATION.UNFILLED_PLACEHOLDER`, `MIGRATION.DESTRUCTIVE_CHANGES` / `--confirm `, schema drift | +| Author migrations | [`references/migrations.md`](references/migrations.md) | `db update` vs `migration plan`, `db migrate`, `migration new`, `migration show`, `db update --dry-run`, `db verify`, `db sign`, data migration, `dataTransform`, placeholder sentinels in framework-rendered `migration.ts`, `MIGRATION.HASH_MISMATCH`, `MIGRATION.UNFILLED_PLACEHOLDER`, data-loss questions (`--delete`, `--rename`, `--allow`, `CLI.CONSENT_REQUIRED`), schema drift | | Migration graph, refs, plan origin | [`references/migration-model.md`](references/migration-model.md) | migration graph, refs, `migration ref set` / `list` / `delete`, the `db` ref, `--advance-ref`, `--no-advance-ref`, `migration plan --from`, `from: (baseline)` in plan output, greenfield / from-scratch plan, baseline, first migration before deploy (Composer / CD-managed databases), chaining migrations, retrofitting migrations onto an existing database, `MIGRATION.HASH_NOT_IN_GRAPH`, `MIGRATION.PATH_UNREACHABLE` at plan/chain time | | Review migrations on deploy | [`references/migration-review.md`](references/migration-review.md) | "what migrations are going to run", "what runs on deploy / merge", merge conflict, diamond convergence, concurrent migrations, migration status, ref management for CI, staging / production environment refs, `MIGRATION.MARKER_NOT_IN_HISTORY`, `MIGRATION.MISSING_INVARIANTS`, `db migrate --show` | | Write queries | [`references/queries.md`](references/queries.md) | query, where, select, project, orderBy, limit, offset, take, skip, include, lookup, first, all, count, aggregate, groupBy, create, update, delete, upsert, returning, transaction, `db.orm`, `db.sql`, `runtime.query(plan)` vs `runtime.execute(plan)`, `db.prepare` / prepared statements, streaming / `for await`, many-to-many `include` and nested `connect`, `and` / `or` / `not` from `@prisma/orm-postgres/orm-client`, `db.query.from(...)` (Mongo pipeline), namespace-aware accessors, `.all()` Thenable, single-use iterators (`RUNTIME.ITERATOR_CONSUMED`), target-declared aggregate types (`count`, integer `sum`, and integer `avg` are `number`; `count` and integer `sum` throw outside ±(2^53 − 1) rather than round, while `avg` is a fraction already and carries no guard; `countBigInt` / `sumBigInt` / `avgDecimal` are the lossless forms, `avgDecimal` on PostgreSQL only), custom collection class (`extends Collection`), `with` and scopes (`Scope`, `Filtered`, `Ordered`), shared query fragments (`db.orm.scope(fields, body)` for a filter on every model with given fields, `CodecField`, `db.orm.public.Post.scope(...)` for a shared `select` / `include`, `orderByField` for an order or sort parameter from a request), drizzle-style, kysely-style. Naming types (§ *Naming model and result types*): "type of my model", "return type of a query", `ResultType`, `Scalars`, `Shape`, `models` / `Models` namespace in `contract.d.ts`, `Models.public_User`, Prisma 7 `Prisma.User` / `GetPayload` / `UserGetPayload` equivalents. Postgres/SQLite specifics: [`references/queries-postgres.md`](references/queries-postgres.md); Mongo specifics: [`references/queries-mongo.md`](references/queries-mongo.md) | diff --git a/skills/prisma-8/references/debug.md b/skills/prisma-8/references/debug.md index 6a5dfec6fd1e..bad5d318bbdd 100644 --- a/skills/prisma-8/references/debug.md +++ b/skills/prisma-8/references/debug.md @@ -96,7 +96,9 @@ The single source of truth: read the envelope, find the row by `code`, follow th | `CONTRACT.TARGET_MISMATCH` | `db verify`, runtime startup | Contract target ≠ config target; align them (see `meta.expected` / `meta.actual`). | | `CONTRACT.SCHEMA_VERIFICATION_FAILED` | `db verify`, `db sign` (both exit 4 with the finding) | Live schema does not satisfy the contract; `meta.issues` lists the drifted paths per `meta.space`. Run `db update` to reconcile, or adjust the contract. | | `MIGRATION.RUNNER_FAILED` | `db migrate`, `db update`, `db init` | Wrapper for an unclassified apply failure; `why` and `meta` carry the detail. Reconcile the reported failure, then re-run. Previously applied migrations are preserved. | -| `MIGRATION.DESTRUCTIVE_CHANGES` | `db update` when run with nobody to ask (`--no-interactive`, CI) | Consent is the database name: interactively you type it; non-interactively pass `--confirm `. `--yes` does **not** grant it. `--dry-run` previews. **Only `db update` has this flow** — `db migrate` does not gate destructive ops on a flag. | +| `CLI.CONSENT_REQUIRED` | `migration plan` / `db update` when a plan would lose data (or, on `db update`, widen access) and nobody can answer (`--no-interactive`, CI, `--yes`) | `meta.unanswered[]` and `nextActions` list each question with its flags. Re-run with `--delete `, `--rename :` or `--allow ` as each subject means. `--confirm` and `--yes` answer none of them. `db migrate` does not ask. | +| `CLI.CONSENT_UNUSED` | `migration plan` / `db update` | Every question was answered, and a `--delete` or `--allow` was left over: a typo, a subject the plan does not touch, or a `--confirm` (which consents to nothing). Fix or drop the flag. A typo next to a question nobody answered is reported as `CLI.CONSENT_REQUIRED` instead, which names the flag that matched nothing. On `db update --dry-run` the same mistake is `MIGRATION.STATEMENT_ANSWERS_NO_QUESTION`. | +| `MIGRATION.STATEMENT_DID_NOT_RESOLVE_LOSS` | A rename typed at the data-loss prompt | The rename still loses the subject's data. Answer with `delete`, or give a rename whose old name is the subject. | | `MIGRATION.NO_INVARIANT_PATH` / `MIGRATION.UNKNOWN_INVARIANT` | `db migrate` | Concurrent-migration and invariant flows — `references/migration-review.md`. | | `DRIVER.NOT_CONNECTED` with message `Runtime is closed` | A query, prepared statement or `db.runtime().connection()` that starts after `db.close()` and after the runtime was idle for one turn of the event loop, or after the `await using` scope that held a serverless connection ended | The work started after the runtime closed. Almost always it is a lazy read (`db.orm...all()`, `db.runtime().query(...)`) returned without `await`, which starts only when the caller awaits it, or work that waited on something other than the database first. Write `return await ...`, or move the call before `close()`. See `references/runtime.md` § *Workflow — Serverless and per-request runtimes*. | | `await db.close()` or the end of an `await using` scope never settles in a test | A test that installed fake timers (`vi.useFakeTimers()`, Jest's modern fake timers) before the Prisma runtime module was imported | `close()` waits one turn of the event loop through a `setTimeout(0)` that the runtime module takes when it loads. Install fake timers after the imports, or advance the fake clock (`await vi.runAllTimersAsync()`) before awaiting the close. | diff --git a/skills/prisma-8/references/migration-review.md b/skills/prisma-8/references/migration-review.md index 5247ad5fc7c4..19e212c29633 100644 --- a/skills/prisma-8/references/migration-review.md +++ b/skills/prisma-8/references/migration-review.md @@ -177,7 +177,7 @@ The whole job is one command: `migration status --to staging --db $URL` is what a human or agent runs to answer *"what will run on deploy?"* before merging: it reports the path from the live marker to the ref without changing anything. `db migrate --show --db $URL` gives the same path as an ordered preview; `migration log --db $URL` gives the applied history after a deploy. -`db migrate` is interactive-free and has no destructive-op confirmation prompt — the safety rails that prompt for destructive changes live on `db update` (see the `references/migrations.md` skill). Whatever the planner put in the migration graph is what `db migrate` runs; review happens at `migration plan` and at `migration status` time, before the apply step. +`db migrate` is interactive-free and asks nothing — the questions about data loss are asked by `migration plan` when it writes the package, and by `db update` (see the `references/migrations.md` skill). Whatever the planner put in the migration graph is what `db migrate` runs; review happens at `migration plan` and at `migration status` time, before the apply step. ## Common Pitfalls diff --git a/skills/prisma-8/references/migrations.md b/skills/prisma-8/references/migrations.md index f0c3759d0783..0ff2871ffbb6 100644 --- a/skills/prisma-8/references/migrations.md +++ b/skills/prisma-8/references/migrations.md @@ -30,8 +30,8 @@ Once the contract changes, you choose how the change reaches the database. This ## Key Concepts -- **`db update` (quick path).** Reads the emitted contract, diffs against the live DB, applies the change. Optional `--dry-run` prints the plan without executing. A destructive operation is applied only with consent: interactively you type the database name; non-interactively pass `--confirm ` (`--yes` does not grant it). **Writes no migration directory.** Operations needing data transforms are not handled by this path — `db update` excludes the `data` operation class entirely and short-circuits where a data transform would be required. Use only against a database that has no shared history with anyone else (your local dev DB). -- **`migration plan` (formal path).** Reads the emitted contract, diffs it against a resolved origin — explicit `--from`, else the `db` ref, else the empty database; there is no "head of the graph" to chain from (see `references/migration-model.md`) — and writes a new migration package under `migrations/app/_/`. If any operation needs a data transform, the package's `migration.ts` contains `placeholder(...)` calls you fill in. +- **`db update` (quick path).** Reads the emitted contract, diffs against the live DB, applies the change. Optional `--dry-run` prints the plan without executing. Before an apply it asks what each operation that would lose data means, answered by `--delete ` or `--rename :`, and whether each operation that would widen who can read or write a model's rows may run, answered by `--allow ` (see *Answer the data-loss questions* below). **Writes no migration directory.** Operations needing data transforms are not handled by this path — `db update` excludes the `data` operation class entirely and short-circuits where a data transform would be required. Use only against a database that has no shared history with anyone else (your local dev DB). +- **`migration plan` (formal path).** Reads the emitted contract, diffs it against a resolved origin — explicit `--from`, else the `db` ref, else the empty database; there is no "head of the graph" to chain from (see `references/migration-model.md`) — and writes a new migration package under `migrations/app/_/`. A plan that would lose data is written only once `--delete` or `--rename` says what each such operation means. If any operation needs a data transform, the package's `migration.ts` contains `placeholder(...)` calls you fill in. - **The `app/` segment in migration paths is the consuming application's contract-space id.** Every migration *you* author lives under `migrations/app/`. Extensions your contract depends on get their own sibling directories (`migrations//`) — those are managed by the extension package and you don't write into them. The `app/` segment lands automatically the first time you run `migration plan` / `db init` against an app-level config. - **Migration package files** (inside each `migrations/app//`): - `migration.json` — manifest (metadata + `migrationHash`). @@ -71,7 +71,9 @@ Treat the rendered import lines as framework-managed on both targets: | `MIGRATION.INVALID_DEFAULT_EXPORT` | Loading `migration.ts` | The file's default export is not a `Migration` subclass or factory function. Restore the planner-emitted scaffold from version control or re-run `migration plan` for a clean package. | | `MIGRATION.DATA_TRANSFORM_CONTRACT_MISMATCH` | Building a data-transform query plan | The query builder was instantiated with a contract reference different from the `endContract` passed to `this.dataTransform(...)`. Use the `endContract` imported at module scope for both. | | `MIGRATION.HASH_MISMATCH` *Migration package is corrupt* | `db migrate` (or any read of the package) | `ops.json` / `migration.json` were edited without self-emitting. Run `node migrations/app//migration.ts` to re-emit, then re-run `db migrate`. | -| `MIGRATION.DESTRUCTIVE_CHANGES` | `db update` run non-interactively without consent | Re-run with `--confirm ` (the database name from the connection), or `--dry-run` to preview. | +| `CLI.CONSENT_REQUIRED` | `migration plan` / `db update` run with nobody to answer (`--no-interactive`, CI, `--yes`) | The plan would lose data or (on `db update`) widen access. `meta.unanswered` and `nextActions` list each question with the flags that answer it: `--delete `, `--rename :`, `--allow `. Re-run with the ones you mean. `--confirm` and `--yes` answer none of them. | +| `CLI.CONSENT_UNUSED` | `migration plan` / `db update` | Every question was answered, and a `--delete` or `--allow` was left over: a typo, a subject the plan does not touch, or a `--confirm` (which consents to nothing). Fix or drop the flag. A typo next to a question nobody answered is reported as `CLI.CONSENT_REQUIRED` instead, which names the flag that matched nothing. On `db update --dry-run` the same mistake is `MIGRATION.STATEMENT_ANSWERS_NO_QUESTION`. | +| `MIGRATION.STATEMENT_DID_NOT_RESOLVE_LOSS` | A rename typed at the data-loss prompt | The plan was made again with the rename and still loses the subject's data. Answer with `delete`, or give a rename whose old name is the subject. | | `CONTRACT.MARKER_MISMATCH` | `db verify` (finding, exit 4) | The marker disagrees with the contract hash (**Postgres:** `prisma_contract.marker`; **Mongo:** `_prisma_migrations`). The DB is at a different contract version than the code thinks. Either run a migration forward, or — if the DB is correct and the marker is stale after a manual fix-up — run `db sign`. | | `CONTRACT.MARKER_MISSING` | `db verify` (finding, exit 4), runtime startup (warning) | The DB has no marker yet. Run `prisma db init --db ` to baseline an empty database, `db update --db ` to apply the current contract directly, or `db sign --db ` if the schema already matches the contract. | | `MIGRATION.STATEMENT_INVALID` | `migration plan` / `db update` with `--rename` | The statement is malformed, mixes a model and a field, renames something to itself, swaps two names, or repeats a name. The error's next step is the statement to type instead; use it. | @@ -147,7 +149,7 @@ Canonical detail: [Migration System § Contract resolution through the snapshot ## Workflow — `db update` (quick path) -The concept: `db update` resolves the destination (`emitted contract`) against the live DB and applies the difference. Preview with `--dry-run`. Destructive ops need consent: interactively the command asks you to type the database name; with `--no-interactive` (CI) it reads `--confirm ` instead, and refuses with `MIGRATION.DESTRUCTIVE_CHANGES` if neither is given. `--yes` accepts prompt defaults and never grants this consent. The path excludes operations of the `data` class entirely — if the diff requires a data transform, `db update` fails with a planning error and you switch to `migration plan` to author the transform. +The concept: `db update` resolves the destination (`emitted contract`) against the live DB and applies the difference. Preview with `--dry-run`, which asks nothing and lists the questions an apply would ask under `dataLoss` and `accessWidening`. An apply that would lose data or widen access asks first (see *Answer the data-loss questions*). The path excludes operations of the `data` class entirely — if the diff requires a data transform, `db update` fails with a planning error and you switch to `migration plan` to author the transform. Run after a contract edit: @@ -167,7 +169,7 @@ Inspect the JSON output to drive the next move: pnpm prisma db update --db $DATABASE_URL --json ``` -The JSON contains `plan.operations[]` with each `operationClass`, plus (in apply mode) `execution.operationsExecuted` and the post-apply `marker.storageHash`. If the command failed because of destructive operations, the error envelope's `meta.destructiveOperations[]` lists exactly what would have been dropped. +The JSON contains `plan.operations[]` with each `operationClass`, plus (in apply mode) `execution.operationsExecuted` and the post-apply `marker.storageHash`. If the command was refused because nobody answered its questions, the error is `CLI.CONSENT_REQUIRED`, and its `meta.unanswered[]` lists each subject with the flags that answer it. ## Workflow — `migration plan` + `db migrate` (formal path) @@ -212,7 +214,7 @@ Fill in any data transforms (see *Fill a placeholder*), self-emit if you edited pnpm prisma db migrate --db $DATABASE_URL ``` -`db migrate` runs without prompting — destructive-op confirmation lives on `db update`, not here. Review destructive ops in the plan output or in `migration show` *before* applying. +`db migrate` runs without prompting: the data-loss questions were answered when `migration plan` wrote the package. Review destructive ops in the plan output or in `migration show` *before* applying. ## Workflow — Fill a placeholder @@ -362,7 +364,7 @@ On Postgres the operations are **methods on the `Migration` base class**, each t **Postgres** operations (representative set, all `this.({...})`): - Tables: `createTable`, `renameTable`, `dropTable`. `renameTable` reads the migration's start and end contracts and returns the table rename plus a rename of every constraint and index named after the old table, so spread it: `...this.renameTable({ table: 'userProfile', to: 'UserProfile' })`. -- Columns: `addColumn` (`column: col(name, nativeType, { codecRef })`), `renameColumn` (also renames the constraints and indexes named after the column, so spread it: `...this.renameColumn({ table: 'User', column: 'email', to: 'emailAddress' })`), `dropColumn`, `alterColumnType`, `setNotNull`, `dropNotNull`, `setDefault` (`column: col(name, nativeType, { default: lit(value) or fn(expression), codecRef })`; `fn('autoincrement()')` on a smallint, integer or bigint column creates or reuses the `{table}_{column}_seq` sequence, attaches it, and starts it past the column's largest value), `dropDefault`. +- Columns: `addColumn` (`column: col(name, nativeType, { codecRef })`), `renameColumn` (also renames the constraints and indexes named after the column, so spread it: `...this.renameColumn({ table: 'User', column: 'email', to: 'emailAddress' })`), `dropColumn`, `alterColumnType` (pass `operationClass: 'widening'` only when every value of the old type converts to the new type unchanged, such as `int4` to `int8`; the default is `'destructive'`), `setNotNull`, `dropNotNull`, `setDefault` (`column: col(name, nativeType, { default: lit(value) or fn(expression), codecRef })`; `fn('autoincrement()')` on a smallint, integer or bigint column creates or reuses the `{table}_{column}_seq` sequence, attaches it, and starts it past the column's largest value), `dropDefault`. - Constraints: `addPrimaryKey`, `addForeignKey`, `addUnique`, `addCheckConstraint`, `renameConstraint` (`kind`: `primaryKey`, `unique`, `foreignKey` or `checkConstraint`), `renameCheckConstraint`, `dropCheckConstraint`, `dropConstraint`. - Indexes: `createIndex`, `renameIndex`, `dropIndex`. - Enums: `createNativeEnumType`, `addNativeEnumValue`, `dropNativeEnumType`. @@ -470,22 +472,22 @@ pnpm prisma db migrate --db $DATABASE_URL If self-emit itself fails (e.g. the contract has moved on and the operations no longer make sense against the migration's end contract), the package is stale. Either restore it from version control or delete it and re-plan with `migration plan`. -## Workflow — Resolve a destructive-operation prompt (`db update` only) +## Workflow — Answer the data-loss questions (`migration plan` and `db update`) -The concept: when `db update` would drop columns or tables, it stops and asks before applying. The prompt is `db update`-specific — `db migrate` does *not* prompt and runs whatever the migration package contains, so review the plan or call `migration show` before `db migrate`. +The concept: the planner cannot tell whether a dropped model or field was meant to go, or was renamed. So before `migration plan` writes a package, or `db update` applies, the command asks one question per model, field or storage name an operation would lose data from. `db update` also asks one per model whose rows an operation would open up, such as dropping a row-level-security policy. Each question is answered by a statement: -When `db update` reports destructive operations interactively, the warning lists them. The prompt is: +- `--delete ` lets the data go: `--delete Legacy`, `--delete User.nickname`. A subject no model stores is named by its storage name (`public.audit_log` on Postgres, `audit_log` on SQLite) and can only be deleted. +- `--rename :` keeps it under a new name: `--rename Profile:User`, or for a field `--rename User.nickname:User.handle` (the new name keeps its model). A field of a renamed model is named through the new name (`--delete User.nickname` after `--rename Profile:User`). +- `--allow ` (`db update` only) lets an operation that changes who can read or write the model's rows run: one flag per operation, so a policy drop and disabling row-level security on one model take `--allow User --allow User`. -> Apply destructive changes? This cannot be undone. - -Routing: +```bash +pnpm prisma migration plan --name drop_legacy --delete Legacy +pnpm prisma db update --db $DATABASE_URL --rename Profile:User --delete Legacy --allow User +``` -- Answer yes if the data is no longer needed. -- Answer no, then either: - - Re-shape the migration via `migration plan` and hand-edit `migration.ts` to preserve the data (e.g. copy-to-new-column, then drop), or - - Skip the destructive operation by reverting the contract change. +In a terminal the command asks each question in turn; type `delete`, `allow`, or `rename :`. Where nobody can answer (CI, `--no-interactive`, `--yes`), it fails with `CLI.CONSENT_REQUIRED`; read `meta.unanswered[]` or the `nextActions`, decide what each subject means, and re-run with those flags. `--confirm` and `--yes` answer none of these questions. Don't reach for `--delete` by reflex: if the model or field was renamed, `--rename` keeps its rows. -Interactively, consent is typing the database name back (the prompt names it). In non-interactive contexts (CI, `--no-interactive`), the destructive-op response is returned as `MIGRATION.DESTRUCTIVE_CHANGES` — `meta.destructiveOperations[]` lists what would have been dropped. Re-run with `--confirm ` to grant consent (`--yes` does not), or address each operation individually. +`db migrate` does *not* ask: whatever the package contains runs, so review the plan or call `migration show` before `db migrate`. ## Common Pitfalls @@ -506,7 +508,7 @@ Interactively, consent is typing the database name back (the prompt names it). I - **Runtime-apply migrations.** Prisma 8 doesn't apply pending migrations from your app's startup code (the "Drizzle pattern" for serverless / edge). Workaround: run `prisma db migrate` from your deploy pipeline before the app starts. If you need runtime-apply built-in, file a feature request via the `references/feedback.md` skill. - **Seeds-as-first-class.** Prisma 8 doesn't ship a `prisma db seed` equivalent. Workaround: write a TypeScript script that imports your `db` instance and runs your setup queries; invoke it from `package.json`'s scripts. If you need first-class seeding, file a feature request via the `references/feedback.md` skill. - **Migration squashing.** Prisma 8 doesn't squash older migrations into a baseline. They accumulate; for very large histories, manual baseline-and-truncate is the path. If you need built-in squashing, file a feature request via the `references/feedback.md` skill. -- **Renames beyond models and fields, and renames on MongoDB.** `--rename` covers models and fields on Postgres and SQLite. It does not rename enum values, namespaces or value objects, and does not move a model to another namespace. On MongoDB, `--rename` fails with `MIGRATION.PLANNING_FAILED` and plans nothing. Workaround: follow the error's advice and make the rename by hand in `mongosh` on each database, before a plan made without `--rename` is applied there: `renameCollection` for a model, and for a field turn off the collection's validator, drop each unique index that includes the field, then `updateMany({}, { $rename: ... })`. Applying the plan then creates the new indexes and turns the validator back on. There is no Mongo rename operation to write in `migration.ts`. If you need one of these, file a feature request via the `references/feedback.md` skill. +- **Renames beyond models and fields, and renames on MongoDB.** `--rename` covers models and fields on Postgres and SQLite. It does not rename enum values, namespaces or value objects, and does not move a model to another namespace. On MongoDB, `--rename` fails with `MIGRATION.PLANNING_FAILED` and plans nothing. A data-loss question about a dropped collection offers only `--delete` and says how to keep the documents by hand; a migration written by `migration plan` still drops the collection, so remove that operation or do not apply it where the collection was renamed. Workaround: follow the error's advice and make the rename by hand in `mongosh` on each database, before a plan made without `--rename` is applied there: `renameCollection` for a model, and for a field turn off the collection's validator, drop each unique index that includes the field, then `updateMany({}, { $rename: ... })`. Applying the plan then creates the new indexes and turns the validator back on. There is no Mongo rename operation to write in `migration.ts`. If you need one of these, file a feature request via the `references/feedback.md` skill. ## Graph and history commands @@ -538,4 +540,4 @@ The CLI collects anonymous usage data by default. To opt out, set `PRISMA_DISABL - [ ] Used `db verify` only when diagnosing drift — not as a routine post-apply step. - [ ] Did NOT use `db update` against a shared or production database. - [ ] Did NOT edit `ops.json` directly. -- [ ] Did NOT skip a destructive-op prompt without inspecting `meta.destructiveOperations[]`; granted consent with the database name (or `--confirm `), not `--yes`. +- [ ] Answered each data-loss question deliberately: `--rename` for a rename, `--delete` only for data that may go, `--allow` only for a widening that is meant. diff --git a/test/e2e/framework/test/fixtures/package.json b/test/e2e/framework/test/fixtures/package.json index b70d7616a897..3537adbfb23f 100644 --- a/test/e2e/framework/test/fixtures/package.json +++ b/test/e2e/framework/test/fixtures/package.json @@ -5,7 +5,7 @@ "version": "8.0.0-rc.16", "description": "The Postgres fixture of the e2e suite. Its own manifest because emission reads the nearest one to decide which package names generated files import, and this suite exercises two databases whose halves must not name each other's facade.", "dependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@prisma/orm-extension-arktype-json": "workspace:8.0.0-rc.16", "@prisma/orm-extension-pgvector": "workspace:8.0.0-rc.16", "@prisma/orm-postgres": "workspace:8.0.0-rc.16", diff --git a/test/e2e/framework/test/sqlite/fixtures/package.json b/test/e2e/framework/test/sqlite/fixtures/package.json index a6e27e3a34b7..fb5a4b48b0bb 100644 --- a/test/e2e/framework/test/sqlite/fixtures/package.json +++ b/test/e2e/framework/test/sqlite/fixtures/package.json @@ -5,7 +5,7 @@ "version": "8.0.0-rc.16", "description": "The SQLite fixture of the e2e suite. Its own manifest because emission reads the nearest one to decide which package names generated files import, and this suite exercises two databases whose halves must not name each other's facade.", "dependencies": { - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@prisma/orm-sqlite": "workspace:8.0.0-rc.16" } } diff --git a/test/e2e/framework/test/sqlite/migrations/destructive.test.ts b/test/e2e/framework/test/sqlite/migrations/destructive.test.ts index 0263639b48a2..bb32ade336ee 100644 --- a/test/e2e/framework/test/sqlite/migrations/destructive.test.ts +++ b/test/e2e/framework/test/sqlite/migrations/destructive.test.ts @@ -134,23 +134,6 @@ describe('SQLite Migration E2E - Destructive column changes', () => { ); }); - it('tightens nullability (nullable to NOT NULL)', async () => { - await applyMigration( - { - origin: defineContract({ - models: { User: model('User', { fields: { id: int.id(), name: text.optional() } }) }, - }), - destination: defineContract({ - models: { User: model('User', { fields: { id: int.id(), name: text } }) }, - }), - policy: ALL, - }, - async ({ schema }) => { - expect(schema.tables['User']!.columns['name']!.nullable).toBe(false); - }, - ); - }); - it('drops a column and preserves remaining data', async () => { await applyMigration( { diff --git a/test/e2e/framework/test/sqlite/migrations/harness.ts b/test/e2e/framework/test/sqlite/migrations/harness.ts index c90d5397b62a..68697789e354 100644 --- a/test/e2e/framework/test/sqlite/migrations/harness.ts +++ b/test/e2e/framework/test/sqlite/migrations/harness.ts @@ -118,14 +118,45 @@ function formatFailure(f: MigrationRunnerFailure): string { return parts.join('\n'); } +interface MigrationOptions { + origin?: Contract; + destination: Contract; + policy?: MigrationOperationPolicy; + seed?: (driver: Driver) => Promise; +} + +export interface FailedMigrationResult { + readonly driver: Driver; + readonly failure: MigrationRunnerFailure; +} + export async function applyMigration( - options: { - origin?: Contract; - destination: Contract; - policy?: MigrationOperationPolicy; - seed?: (driver: Driver) => Promise; - }, + options: MigrationOptions, runAssertions: (result: MigrationResult) => Promise, +): Promise { + await runMigration(options, runAssertions, async ({ failure }) => { + throw new Error(`Destination runner failed: ${formatFailure(failure)}`); + }); +} + +/** Applies the destination plan, expects the runner to fail, and hands the failure and the database to `assertFailure`. */ +export async function applyMigrationExpectingFailure( + options: MigrationOptions, + assertFailure: (result: FailedMigrationResult) => Promise, +): Promise { + await runMigration( + options, + async () => { + throw new Error('Expected the destination runner to fail, but it succeeded'); + }, + assertFailure, + ); +} + +async function runMigration( + options: MigrationOptions, + onSuccess: (result: MigrationResult) => Promise, + onFailure: (result: FailedMigrationResult) => Promise, ): Promise { const testDb = createTestDb(); const { driver } = testDb; @@ -200,8 +231,10 @@ export async function applyMigration( }, ], }); - if (!runResult.ok) - throw new Error(`Destination runner failed: ${formatFailure(runResult.failure)}`); + if (!runResult.ok) { + await onFailure({ driver, failure: runResult.failure }); + return; + } const freshSchema = await adapter.introspect(driver); const vr = familyInstance.verifySchema({ @@ -219,7 +252,7 @@ export async function applyMigration( for (const [name, tbl] of Object.entries(freshSchema.tables)) { if (!CONTROL_TABLES.has(name)) userTables[name] = tbl; } - await runAssertions({ + await onSuccess({ driver, schema: new SqlSchemaIR({ ...freshSchema, tables: userTables }), operationsExecuted: runResult.value.perSpaceResults[0]?.value.operationsExecuted ?? 0, diff --git a/test/e2e/framework/test/sqlite/migrations/widening.test.ts b/test/e2e/framework/test/sqlite/migrations/widening.test.ts index 40b971475521..079849399186 100644 --- a/test/e2e/framework/test/sqlite/migrations/widening.test.ts +++ b/test/e2e/framework/test/sqlite/migrations/widening.test.ts @@ -1,7 +1,7 @@ import { datetimeColumn } from '@prisma/orm-sqlite/adapter/column-types'; import { defineContract, field, model, now, sql } from '@prisma/orm-sqlite/contract-builder'; import { describe, expect, it } from 'vitest'; -import { applyMigration, int, text } from './harness'; +import { applyMigration, applyMigrationExpectingFailure, int, text } from './harness'; describe('SQLite Migration E2E - Widening operations (recreate-table)', () => { const WIDENING = { allowedOperationClasses: ['additive', 'widening'] } as const; @@ -34,6 +34,53 @@ describe('SQLite Migration E2E - Widening operations (recreate-table)', () => { ); }); + it('tightens nullability (nullable to NOT NULL)', async () => { + await applyMigration( + { + origin: defineContract({ + models: { User: model('User', { fields: { id: int.id(), name: text.optional() } }) }, + }), + destination: defineContract({ + models: { User: model('User', { fields: { id: int.id(), name: text } }) }, + }), + policy: WIDENING, + }, + async ({ schema }) => { + expect(schema.tables['User']!.columns['name']!.nullable).toBe(false); + }, + ); + }); + + it('fails a nullability tightening on a NULL and keeps the row and the nullable column', async () => { + await applyMigrationExpectingFailure( + { + origin: defineContract({ + models: { User: model('User', { fields: { id: int.id(), name: text.optional() } }) }, + }), + destination: defineContract({ + models: { User: model('User', { fields: { id: int.id(), name: text } }) }, + }), + policy: WIDENING, + seed: async (driver) => { + await driver.query('INSERT INTO "User" (id, name) VALUES (?, ?)', [1, null]); + }, + }, + async ({ driver }) => { + const rows = (await driver.query('SELECT id, name FROM "User"')).rows; + const columns = ( + await driver.query<{ name: string; notnull: number }>('PRAGMA table_info("User")') + ).rows.map(({ name, notnull }) => ({ name, notnull })); + expect({ rows, columns }).toEqual({ + rows: [{ id: 1, name: null }], + columns: [ + { name: 'id', notnull: 1 }, + { name: 'name', notnull: 0 }, + ], + }); + }, + ); + }); + it('changes a column default', async () => { await applyMigration( { diff --git a/test/integration/package.json b/test/integration/package.json index a3db598f219c..91e68a3678ad 100644 --- a/test/integration/package.json +++ b/test/integration/package.json @@ -82,7 +82,7 @@ "@internal/middleware-cache": "workspace:8.0.0-rc.16", "@internal/mongo-lowering": "workspace:8.0.0-rc.16", "@internal/publish-surface": "workspace:8.0.0-rc.16", - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@repo/test-utils": "workspace:8.0.0-rc.16", "@repo/tsconfig": "workspace:8.0.0-rc.16", "@repo/tsdown": "workspace:8.0.0-rc.16", diff --git a/test/integration/test/authoring/psl.pgvector-dbinit.test.ts b/test/integration/test/authoring/psl.pgvector-dbinit.test.ts index 642a3dad65de..6fc4b3662d36 100644 --- a/test/integration/test/authoring/psl.pgvector-dbinit.test.ts +++ b/test/integration/test/authoring/psl.pgvector-dbinit.test.ts @@ -222,6 +222,11 @@ model Document { await withPgvectorControlClient(connectionString, async (client) => { const update = await client.dbUpdate({ + answerQuestions: async (questions) => + questions.map((question) => ({ + verb: question.verbs.includes('allow') ? ('allow' as const) : ('delete' as const), + text: question.subject, + })), contract: emittedContract, mode: 'apply', migrationsDir, diff --git a/test/integration/test/cli-journeys/README.md b/test/integration/test/cli-journeys/README.md index 4e78494d769c..4c3044e2bbd0 100644 --- a/test/integration/test/cli-journeys/README.md +++ b/test/integration/test/cli-journeys/README.md @@ -27,11 +27,14 @@ pnpm test:journeys | `multi-step-migration.e2e.test.ts` | Planning two migrations (base → additive → v3) without applying either, then batch-applying both at once. Verifies pending/applied status reporting | | `migration-plan-details.e2e.test.ts` | **Plan JSON envelope**: full `--json` output shape with operations, attestation round-trip (plan → verify). **Destructive planning**: drop-column migration produces destructive operation class | | `migration-apply-edge-cases.e2e.test.ts` | **No path**: apply fails when contract changed without planning. **Resume**: partial failure (NOT NULL violation) leaves marker at last success, re-apply resumes. **Destructive apply**: single drop-column migration verifies column removed + marker updated. **Multi-step destructive**: three-migration batch (create → add → drop) in one apply | -| `db-update-workflows.e2e.test.ts` | **Direct update**: `db update` without migrations (additive-only, dry-run, noop). **Destructive update**: drops a column, tests `--no-interactive` rejection, `--json` error envelope, and `--json -y` auto-accept. **Re-init conflict**: `db init` on an already-initialized DB with a different contract fails; recovery via `db update` | +| `db-update-workflows.e2e.test.ts` | **Direct update**: `db update` without migrations (additive-only, dry-run, noop). **Destructive update**: drops a column; `--no-interactive` and `--yes` are refused with `CLI.CONSENT_REQUIRED` naming the question, `--confirm` answers nothing, and `--delete User.email` applies. **Re-init conflict**: `db init` on an already-initialized DB with a different contract fails; recovery via `db update` | | `contract-infer-workflow.e2e.test.ts` | **PSL inference workflow**: refresh `contract.prisma` from the live database with `contract infer`, emit from the inferred PSL, verify the schema, and confirm a second infer is stable | | `brownfield-adoption.e2e.test.ts` | **Adopt Prisma on existing DB**: infer a PSL contract from the live DB, emit matching artifacts, `db verify --schema-only`, sign, verify, and then evolve via `db update`. **Schema mismatch**: emit a contract that doesn't match the DB, observe sign / schema-only verify failures, fix contract, retry | | `rename-table-migration.e2e.test.ts` | **Rename a table and keep its rows**: seed `userProfile` with a unique constraint, a foreign key, an index, row-level security and a policy, then drop the model's `@@map`. Journey R1 confirms the bare plan is refused by the case-change guard, which points at `this.renameTable`, and that a `renameTable` call naming a table the end contract lacks fails when `migration.ts` builds its operations. A migration created with `migration new` and `...this.renameTable({ table: 'userProfile', to: 'UserProfile' })` renames the table and each constraint and index named after it; `migrate` keeps the rows, the policy and RLS; `db verify --schema-only` is clean; a plan with no schema change is empty; and a later migration that removes the unique constraint, the foreign key and the index applies. Journey R5 changes the table name and its foreign key target in one schema change: a rename-only migration for it fails at `migrate`, which verifies against the end contract; made as a rename-only change and then a planned foreign key change, it keeps the rows and verifies clean | | `rename-table-migration.sqlite.e2e.test.ts` | The SQLite twin of the first rename journey, driven through a file database and the SQLite facade config: the rename plus a drop and a create of each index named after the old table, then the same empty plan and removal migration. A second journey follows the by-hand path of a `db update` project: the case guard gives the two-statement rename through `_prisma_rename_UserProfile`, and after running it `db update` drops and recreates the indexes named after the old table, keeps the rows and verifies clean | +| `delete-statements-migration.e2e.test.ts` | **Statements that consent to data loss and widened access (Postgres)**: journey S1 renames `Profile` to `User` and its field, and removes `Legacy`, through `migration plan`: refused with `CLI.CONSENT_REQUIRED` naming `--delete Legacy`, planned with it, and migrated on tables with rows, a unique, a foreign key, an index and a check. S2 does the same through `db update`, where `--confirm ` is refused, and the same statements fail on a second run (`MIGRATION.STATEMENT_UNRESOLVED`). S3 adds a required field to a table with rows without asking. S4 drops a policy: a dry run lists it under `accessWidening`, the apply is refused without `--allow User` and applies with it. S5 drops a model with `db update --db ` and no snapshot: the subject is `public.Legacy`, the question says the origin contract is unknown, and `--delete public.Legacy` applies | +| `delete-statements-migration.sqlite.e2e.test.ts` | **The SQLite twin**: S1 and S2 as on Postgres; S3 adds a required field to an empty table (SQLite's `db update` has no temporary default for existing rows); S4 removes the snapshot store, so the dropped table is asked about as `Legacy`, its unqualified storage name | +| `delete-statements-migration.mongo.e2e.test.ts` | **The MongoDB twin**: dropping a collection is refused by `migration plan` and `db update` until `--delete Event`; `--confirm ` is refused; a required field applies without asking; a `--rename` is refused by the planner with `statementRefused` | | `rename-statements-migration.e2e.test.ts` | **Rename statements on the command line (Postgres)**: seed `Profile` with rows, a unique field, a secondary index, a check, row-level security with a policy, and a foreign key from `Post`; the destination renames the model to `User` and its field `name` to `fullName`. Journey S1 plans `migration plan --rename Profile:User --rename User.name:User.fullName`: only widening renames of the table, the column and the objects named after them, both statements reported as applied, and re-running the written `migration.ts` reproduces `ops.json` and `migration.json` byte for byte; after `migrate` the rows and every object are under the new names, a further plan is empty and `db verify --schema-only` is clean. Journey S2 applies the same statements with `db update` and no consent, checks that `db update --dry-run` without statements then plans nothing, and refuses the statements on a second run (`MIGRATION.STATEMENT_UNRESOLVED`). Journey S3 covers the errors: unresolved names, a field named through its old model (the message gives the corrected statement), and `db update --db ` with no snapshot to resolve against (`MIGRATION.STATEMENT_ORIGIN_UNKNOWN`, advising `--advance-ref`) | | `rename-statements-migration.sqlite.e2e.test.ts` | The SQLite twin of journeys S1 and S2, driven through a file database and the SQLite facade config: the table and column renames, the index on the column dropped and created under its destination name, the foreign key from `Post` following the table, the byte-identical re-emit, the empty `db update --dry-run` after the renames, and the second `db update` refused | @@ -50,7 +53,7 @@ pnpm test:journeys | File | What it covers | |---|---| -| `drift-schema.e2e.test.ts` | **Manual schema change with unchanged marker**: a DBA drops a column; `db verify` now fails by default because it runs schema verification, while `db verify --marker-only` reproduces marker-only verification. **Extra column drift**: DBA adds a column via manual DDL; tolerant `db verify` / `db verify --schema-only` pass, strict `db verify` fails; recover by expanding the contract and running `db update` | +| `drift-schema.e2e.test.ts` | **Manual schema change with unchanged marker**: a DBA drops a column; `db verify` now fails by default because it runs schema verification, while `db verify --marker-only` reproduces marker-only verification. **Extra column drift**: DBA adds a column via manual DDL; tolerant `db verify` / `db verify --schema-only` pass, strict `db verify` fails; recover by expanding the contract and running `db update`, which asks about the unmanaged column by its storage name and applies with `--delete public.user.age` | | `drift-marker.e2e.test.ts` | **Missing marker**: contract emitted but `db init` never run — `db verify` fails, `db verify --schema-only` shows the schema mismatch, init recovers. **Stale marker**: contract changed without updating DB — verify fails, schema-only verify shows the missing column, `db update` recovers. **Mixed-mode evolution**: iterate through multiple contract versions using `db update` (no migration files). **Corrupt marker**: marker row overwritten with garbage — verify fails, `db verify --schema-only` passes (schema intact), `db sign` recovers | | `drift-migration-dag.e2e.test.ts` | **Chain breakage**: after building a migration history, a migration directory is deleted from disk. `migration apply` fails (no path to destination), recovery by re-planning the missing edge | diff --git a/test/integration/test/cli-journeys/data-transform-strategies.e2e.test.ts b/test/integration/test/cli-journeys/data-transform-strategies.e2e.test.ts index 13338cb41803..e11a53920395 100644 --- a/test/integration/test/cli-journeys/data-transform-strategies.e2e.test.ts +++ b/test/integration/test/cli-journeys/data-transform-strategies.e2e.test.ts @@ -50,6 +50,8 @@ interface Scenario { readonly seedSql: string; readonly targetContract: ContractVariant; readonly migrationName: string; + /** The statements the plan needs, such as consent to a type change that can lose values. */ + readonly statements?: readonly string[]; readonly dirToken: string; readonly placeholderId: string; readonly assertScaffold: (scaffold: string) => void; @@ -135,6 +137,7 @@ const scenarios: readonly Scenario[] = [ seedSql: `INSERT INTO "public"."user" (id, email, score) VALUES (1, 'alice@example.com', '10'), (2, 'bob@test.org', '20')`, targetContract: 'contract-typechange-int', migrationName: 'retype-score-to-int', + statements: ['--delete', 'User.score'], dirToken: 'retype_score_to_int', placeholderId: 'typechange-user-score', assertScaffold: (scaffold) => { @@ -194,6 +197,7 @@ withTempDir(({ createTempDir }) => { scenario.migrationName, '--from', latestMigrationDirName(ctx), + ...(scenario.statements ?? []), ]); expect(planResult.exitCode, `plan: ${planResult.stderr}`).toBe(0); diff --git a/test/integration/test/cli-journeys/db-update-workflows.e2e.test.ts b/test/integration/test/cli-journeys/db-update-workflows.e2e.test.ts index 2fe703e0896d..677ec11c8a7a 100644 --- a/test/integration/test/cli-journeys/db-update-workflows.e2e.test.ts +++ b/test/integration/test/cli-journeys/db-update-workflows.e2e.test.ts @@ -4,9 +4,9 @@ * D — Direct update without migrations: swap to an additive contract, dry-run * to preview changes, apply, confirm noop on re-run, then verify. * - * E — Destructive update with confirmation: swap to a contract that drops a - * column, test that --no-interactive blocks destructive changes, --json - * returns an error envelope, and --json -y auto-accepts and succeeds. + * E — Destructive update answered by a statement: swap to a contract that + * drops a column, test that --no-interactive and --yes refuse it and name + * the question, that --confirm answers nothing, and that --delete applies. * * Marker-aware violations (orphan markers, declared-but-unmigrated * extension spaces) are caught by the contract-space verifier — see @@ -18,7 +18,6 @@ import stripAnsi from 'strip-ansi'; import { describe, expect, it } from 'vitest'; import { withTempDir } from '../utils/cli-test-helpers'; import { - consentTokenFor, type JourneyContext, parseJsonOutput, runContractEmit, @@ -107,11 +106,12 @@ withTempDir(({ createTempDir }) => { const dryRun = await runDbUpdate(ctx, ['--dry-run']); expect(dryRun.exitCode, 'E.02: db update dry-run').toBe(0); - // E.03: db update --no-interactive — nobody to ask, so consent is required + // E.03: db update --no-interactive — nobody to ask, so the question is unanswered const noInteractive = await runDbUpdate(ctx, ['--json', '--no-interactive']); expect(noInteractive.exitCode, 'E.03: non-interactive destructive fails').toBe(2); expect(parseJsonOutput(noInteractive), 'E.03: consent error').toMatchObject({ code: 'CLI.CONSENT_REQUIRED', + meta: { unanswered: [{ subject: 'User.email', verbs: ['rename', 'delete'] }] }, }); // E.04: db update --yes — --yes accepts declared defaults, never data loss @@ -121,14 +121,25 @@ withTempDir(({ createTempDir }) => { code: 'CLI.CONSENT_REQUIRED', }); - // E.05: db update --confirm — the non-interactive grant + // E.05: db update --confirm — answers no question const confirmed = await runDbUpdate(ctx, [ '--json', + '--no-interactive', '--confirm', - consentTokenFor(db.connectionString), + 'postgres', ]); - expect(confirmed.exitCode, 'E.05: --confirm applies').toBe(0); - expect(parseJsonOutput(confirmed), 'E.05: success envelope').toMatchObject({ ok: true }); + expect(confirmed.exitCode, 'E.05: --confirm does not answer').toBe(2); + expect(parseJsonOutput(confirmed), 'E.05: consent error').toMatchObject({ + code: 'CLI.CONSENT_REQUIRED', + }); + + // E.06: db update --delete User.email — the statement that answers it + const deleted = await runDbUpdate(ctx, ['--json', '--delete', 'User.email']); + expect(deleted.exitCode, 'E.06: --delete applies').toBe(0); + expect(parseJsonOutput(deleted), 'E.06: the delete is applied').toMatchObject({ + ok: true, + appliedStatements: [{ verb: 'delete', description: 'delete field "User.email"' }], + }); }, timeouts.spinUpPpgDev, ); diff --git a/test/integration/test/cli-journeys/delete-statements-migration.e2e.test.ts b/test/integration/test/cli-journeys/delete-statements-migration.e2e.test.ts new file mode 100644 index 000000000000..175b952b2f3c --- /dev/null +++ b/test/integration/test/cli-journeys/delete-statements-migration.e2e.test.ts @@ -0,0 +1,455 @@ +/** + * Statements that consent to data loss and widened access (Postgres). + * + * A plan that would lose data asks what each such operation means: `--delete ` lets the data go, `--rename :` keeps it. `db update` also asks, with `--allow `, before it widens who can read or write a model's rows. Where nobody can answer, the command fails with `CLI.CONSENT_REQUIRED`, whose next actions name the flags. + * + * Journey S1 renames `Profile` to `User` and its field `name` to `fullName`, and removes `Legacy`, through `migration plan`: refused without `--delete Legacy`, planned with it, and applied by `migrate` on tables with rows, a unique, a foreign key from `Post`, a secondary index and a check. Journey S2 does the same through `db update`, where `--confirm ` no longer consents, and the same statements fail on a second run. Journey S3 adds a required field with no refusal. Journey S4 drops a policy: refused without `--allow User`, applied with it, and listed under `accessWidening` by a dry run. Journey S5 drops a model with `db update --db ` and no snapshot, so the subject is its storage name. + */ + +import { writeFileSync } from 'node:fs'; +import { join } from 'node:path'; +import { describe, expect, it } from 'vitest'; +import { withTempDir } from '../utils/cli-test-helpers'; +import { + engineError, + getMigrationDirs, + type JourneyContext, + latestMigrationDirName, + parseJsonOutput, + planMigrationAndSelfEmit, + runContractEmit, + runDbUpdate, + runDbVerify, + runMigrate, + runMigrationPlan, + setupJourney, + sql, + timeouts, + useDevDatabase, +} from '../utils/journey-test-helpers'; + +const USERS = ` model User { + id Int @id + fullName String @unique + handle String + tenantId Int @map("tenant_id") + posts Post[] + + @@index([fullName, handle]) + @@check(name: "handle_present", expression: "(length(handle) > 0)") + @@rls + } + + model Post { + id Int @id + profileId Int + profile User @relation(fields: [profileId], references: [id]) + } +`; + +const POLICY = (target: string) => ` + policy_select tenant_read { + target = ${target} + roles = [app_user] + using = "(tenant_id = 1)" + } +`; + +const LEGACY = ` + model Legacy { + id Int @id + note String + } +`; + +const FROM_PSL = `// use prisma-8 + +namespace public { + model Profile { + id Int @id + name String @unique + handle String + tenantId Int @map("tenant_id") + posts Post[] + + @@index([name, handle]) + @@check(name: "handle_present", expression: "(length(handle) > 0)") + @@rls + } + + model Post { + id Int @id + profileId Int + profile Profile @relation(fields: [profileId], references: [id]) + } +${LEGACY}${POLICY('Profile')}} +`; + +const TO_PSL = `// use prisma-8 + +namespace public { +${USERS}${POLICY('User')}} +`; + +const TO_WITH_LEGACY_PSL = `// use prisma-8 + +namespace public { +${USERS}${LEGACY}${POLICY('User')}} +`; + +const TO_REQUIRED_FIELD_PSL = TO_PSL.replace( + ' handle String\n', + ' handle String\n nickname String\n', +); + +const TO_NO_POLICY_PSL = `// use prisma-8 + +namespace public { +${USERS}} +`; + +const RENAMES = ['--rename', 'Profile:User', '--rename', 'User.name:User.fullName'] as const; + +interface AppliedStatementReport { + readonly description: string; +} + +async function createAppUserRole(connectionString: string): Promise { + await sql( + connectionString, + 'DO $$ BEGIN CREATE ROLE app_user; EXCEPTION WHEN duplicate_object THEN NULL; END $$', + ); +} + +async function seedRows(connectionString: string): Promise { + await sql( + connectionString, + `INSERT INTO "public"."Profile" (id, name, handle, tenant_id) + VALUES (1, 'alice', 'al', 1), (2, 'bob', 'bo', 1); + INSERT INTO "public"."Post" (id, "profileId") VALUES (10, 1); + INSERT INTO "public"."Legacy" (id, note) VALUES (1, 'old')`, + ); +} + +async function emitContract(ctx: JourneyContext, psl: string, label: string): Promise { + writeFileSync(join(ctx.testDir, 'contract.prisma'), psl, 'utf-8'); + const emit = await runContractEmit(ctx); + expect(emit.exitCode, `${label}: emit: ${emit.stderr}`).toBe(0); +} + +function nextActionsOf(result: Parameters[0]): string { + return JSON.stringify(engineError(result)?.nextActions ?? []); +} + +async function expectRenamedWithoutLegacy( + ctx: JourneyContext, + connectionString: string, + label: string, +): Promise { + const rows = await sql( + connectionString, + `SELECT id, "fullName", handle FROM "public"."User" ORDER BY id`, + ); + expect(rows.rows, `${label}: rows present under the new names`).toEqual([ + { id: 1, fullName: 'alice', handle: 'al' }, + { id: 2, fullName: 'bob', handle: 'bo' }, + ]); + const live = await sql( + connectionString, + `SELECT + to_regclass('"public"."Profile"') AS profile, + to_regclass('"public"."Legacy"') AS legacy, + (SELECT array_agg(conname::text || ':' || contype::text ORDER BY conname) FROM pg_constraint + WHERE conrelid = '"public"."User"'::regclass) AS constraints, + (SELECT array_agg(indexname::text ORDER BY indexname) FROM pg_indexes + WHERE schemaname = 'public' AND tablename = 'User') AS indexes, + (SELECT array_agg(confrelid::regclass::text) FROM pg_constraint + WHERE conrelid = '"public"."Post"'::regclass AND contype = 'f') AS post_references`, + ); + expect( + live.rows[0], + `${label}: old and deleted tables gone, objects under the new names`, + ).toEqual({ + profile: null, + legacy: null, + constraints: [ + 'User_fullName_key:u', + 'User_pkey:p', + expect.stringMatching(/^handle_present(_[0-9a-f]+)?:c$/), + ], + indexes: ['User_fullName_handle_idx_46490beb', 'User_fullName_key', 'User_pkey'], + post_references: ['"User"'], + }); + const verify = await runDbVerify(ctx, ['--schema-only']); + expect(verify.exitCode, `${label}: db verify --schema-only: ${verify.stderr}`).toBe(0); +} + +function databaseName(connectionString: string): string { + return decodeURIComponent(new URL(connectionString).pathname.slice(1)); +} + +withTempDir(({ createTempDir }) => { + describe('Journey S1: rename and delete with migration plan', () => { + const db = useDevDatabase(); + + it( + 'refuses the plan until --delete names the dropped model, then migrates it', + async () => { + const ctx = setupJourney({ + connectionString: db.connectionString, + createTempDir, + contractMode: 'psl', + }); + await createAppUserRole(db.connectionString); + await emitContract(ctx, FROM_PSL, 'S1.01'); + const initial = await planMigrationAndSelfEmit(ctx, ['--name', 'initial']); + expect(initial.exitCode, `S1.02: plan initial: ${initial.stderr}`).toBe(0); + const applyInitial = await runMigrate(ctx); + expect(applyInitial.exitCode, `S1.03: migrate initial: ${applyInitial.stderr}`).toBe(0); + await seedRows(db.connectionString); + const origin = latestMigrationDirName(ctx); + await emitContract(ctx, TO_PSL, 'S1.04'); + + const refused = await runMigrationPlan(ctx, ['--from', origin, ...RENAMES, '--json']); + expect(refused.exitCode, 'S1.05: the plan loses Legacy, so it is refused').toBe(2); + expect(engineError(refused)?.code, 'S1.05: nobody to answer').toBe('CLI.CONSENT_REQUIRED'); + expect(nextActionsOf(refused), 'S1.05: names the flag that answers').toContain( + '--delete Legacy', + ); + expect(getMigrationDirs(ctx), 'S1.05: nothing written').toHaveLength(1); + + const plan = await runMigrationPlan(ctx, [ + '--name', + 'tidy-users', + '--from', + origin, + ...RENAMES, + '--delete', + 'Legacy', + '--json', + ]); + expect(plan.exitCode, `S1.06: plan with --delete: ${plan.stderr}`).toBe(0); + expect( + parseJsonOutput<{ appliedStatements: readonly AppliedStatementReport[] }>( + plan, + ).appliedStatements.map((entry) => entry.description), + 'S1.06: the renames, then the delete', + ).toEqual([ + 'rename model "Profile" to "User"', + 'rename field "User.name" to "User.fullName"', + 'delete model "Legacy"', + ]); + + const apply = await runMigrate(ctx); + expect(apply.exitCode, `S1.07: migrate: ${apply.stderr}`).toBe(0); + await expectRenamedWithoutLegacy(ctx, db.connectionString, 'S1.08'); + + const planned = latestMigrationDirName(ctx); + const fresh = await runMigrationPlan(ctx, ['--from', planned, '--json']); + expect(fresh.exitCode, `S1.09: plan with no schema change: ${fresh.stderr}`).toBe(0); + expect(parseJsonOutput<{ noOp: boolean }>(fresh).noOp, 'S1.09: plan is empty').toBe(true); + }, + timeouts.spinUpPpgDev, + ); + }); + + describe('Journey S2: rename and delete with db update', () => { + const db = useDevDatabase(); + + it( + 'refuses --confirm, applies with --delete, and refuses the same statements on a second run', + async () => { + const ctx = setupJourney({ + connectionString: db.connectionString, + createTempDir, + contractMode: 'psl', + }); + await createAppUserRole(db.connectionString); + await emitContract(ctx, FROM_PSL, 'S2.01'); + const create = await runDbUpdate(ctx, ['--json']); + expect(create.exitCode, `S2.02: db update creates the tables: ${create.stderr}`).toBe(0); + await seedRows(db.connectionString); + await emitContract(ctx, TO_PSL, 'S2.03'); + + const confirmed = await runDbUpdate(ctx, [ + ...RENAMES, + '--no-interactive', + '--confirm', + databaseName(db.connectionString), + '--json', + ]); + expect(confirmed.exitCode, 'S2.04: --confirm does not consent').toBe(2); + expect(engineError(confirmed)?.code, 'S2.04: the question is unanswered').toBe( + 'CLI.CONSENT_REQUIRED', + ); + expect(nextActionsOf(confirmed), 'S2.04: names the flag that answers').toContain( + '--delete Legacy', + ); + + const update = await runDbUpdate(ctx, [...RENAMES, '--delete', 'Legacy', '--json']); + expect(update.exitCode, `S2.05: db update with --delete: ${update.stderr}`).toBe(0); + expect( + parseJsonOutput<{ appliedStatements: readonly AppliedStatementReport[] }>( + update, + ).appliedStatements.map((entry) => entry.description), + 'S2.05: the renames, then the delete', + ).toEqual([ + 'rename model "Profile" to "User"', + 'rename field "User.name" to "User.fullName"', + 'delete model "Legacy"', + ]); + await expectRenamedWithoutLegacy(ctx, db.connectionString, 'S2.06'); + + const again = await runDbUpdate(ctx, [...RENAMES, '--delete', 'Legacy', '--json']); + expect(again.exitCode, 'S2.07: the same statements fail on a second run').not.toBe(0); + expect(engineError(again)?.code, 'S2.07: the first statement no longer resolves').toBe( + 'MIGRATION.STATEMENT_UNRESOLVED', + ); + }, + timeouts.spinUpPpgDev, + ); + }); + + describe('Journey S3: a required field', () => { + const db = useDevDatabase(); + + it( + 'adds a non-nullable field to a table with rows without asking', + async () => { + const ctx = setupJourney({ + connectionString: db.connectionString, + createTempDir, + contractMode: 'psl', + }); + await createAppUserRole(db.connectionString); + await emitContract(ctx, TO_PSL, 'S3.01'); + const create = await runDbUpdate(ctx, ['--json']); + expect(create.exitCode, `S3.02: db update creates User: ${create.stderr}`).toBe(0); + await sql( + db.connectionString, + `INSERT INTO "public"."User" (id, "fullName", handle, tenant_id) VALUES (1, 'alice', 'al', 1)`, + ); + await emitContract(ctx, TO_REQUIRED_FIELD_PSL, 'S3.03'); + + const update = await runDbUpdate(ctx, ['--no-interactive', '--json']); + expect(update.exitCode, `S3.04: db update adds the field: ${update.stderr}`).toBe(0); + const rows = await sql(db.connectionString, `SELECT id, nickname FROM "public"."User"`); + expect(rows.rows, 'S3.04: the existing row has the backfilled value').toEqual([ + { id: 1, nickname: '' }, + ]); + }, + timeouts.spinUpPpgDev, + ); + }); + + describe('Journey S4: a policy drop widens access', () => { + const db = useDevDatabase(); + + it( + 'lists the widening on a dry run, refuses without --allow, and applies with it', + async () => { + const ctx = setupJourney({ + connectionString: db.connectionString, + createTempDir, + contractMode: 'psl', + }); + await createAppUserRole(db.connectionString); + await emitContract(ctx, TO_PSL, 'S4.01'); + const create = await runDbUpdate(ctx, ['--json']); + expect(create.exitCode, `S4.02: db update creates User: ${create.stderr}`).toBe(0); + await emitContract(ctx, TO_NO_POLICY_PSL, 'S4.03'); + + const dryRun = await runDbUpdate(ctx, ['--dry-run', '--json']); + expect(dryRun.exitCode, `S4.04: dry run: ${dryRun.stderr}`).toBe(0); + const listed = parseJsonOutput<{ accessWidening: readonly { text: string }[] }>( + dryRun, + ).accessWidening; + expect(listed, 'S4.04: the dry run lists the widening as --allow takes it').toEqual([ + expect.objectContaining({ + subject: expect.objectContaining({ model: 'User' }), + text: 'User', + }), + ]); + + const refused = await runDbUpdate(ctx, ['--no-interactive', '--json']); + expect(refused.exitCode, 'S4.05: the apply is refused without --allow').toBe(2); + expect(engineError(refused)?.code, 'S4.05: the question is unanswered').toBe( + 'CLI.CONSENT_REQUIRED', + ); + expect(nextActionsOf(refused), 'S4.05: names the flag that answers').toContain( + '--allow User', + ); + const kept = await sql( + db.connectionString, + `SELECT count(*)::int AS policies FROM pg_policies WHERE tablename = 'User'`, + ); + expect(kept.rows, 'S4.05: the policy is still there').toEqual([{ policies: 1 }]); + + const allowed = await runDbUpdate(ctx, [ + ...listed.flatMap(({ text }) => ['--allow', text]), + '--json', + ]); + expect(allowed.exitCode, `S4.06: db update --allow User: ${allowed.stderr}`).toBe(0); + const dropped = await sql( + db.connectionString, + `SELECT count(*)::int AS policies FROM pg_policies WHERE tablename = 'User'`, + ); + expect(dropped.rows, 'S4.06: the policy is gone').toEqual([{ policies: 0 }]); + }, + timeouts.spinUpPpgDev, + ); + }); + + describe('Journey S5: db update --db with no snapshot', () => { + const db = useDevDatabase(); + + it( + 'names the dropped table by its storage name, and deletes it by that name', + async () => { + const ctx = setupJourney({ + connectionString: db.connectionString, + createTempDir, + contractMode: 'psl', + }); + await createAppUserRole(db.connectionString); + await emitContract(ctx, TO_WITH_LEGACY_PSL, 'S5.01'); + const create = await runDbUpdate(ctx, ['--db', db.connectionString, '--json']); + expect(create.exitCode, `S5.02: db update --db creates the tables: ${create.stderr}`).toBe( + 0, + ); + await emitContract(ctx, TO_PSL, 'S5.03'); + + const refused = await runDbUpdate(ctx, [ + '--db', + db.connectionString, + '--no-interactive', + '--json', + ]); + expect(refused.exitCode, 'S5.04: the drop is refused').toBe(2); + const refusal = engineError(refused); + expect(refusal?.code, 'S5.04: the question is unanswered').toBe('CLI.CONSENT_REQUIRED'); + expect(refusal?.meta, 'S5.04: the subject is the storage name').toMatchObject({ + unanswered: [{ subject: 'public.Legacy', verbs: ['delete'] }], + }); + expect(JSON.stringify(refusal), 'S5.04: the question says why').toContain( + 'named by its storage name because the origin contract is unknown', + ); + + const deleted = await runDbUpdate(ctx, [ + '--db', + db.connectionString, + '--delete', + 'public.Legacy', + '--json', + ]); + expect(deleted.exitCode, `S5.05: --delete public.Legacy: ${deleted.stderr}`).toBe(0); + const legacy = await sql( + db.connectionString, + `SELECT to_regclass('"public"."Legacy"') AS legacy`, + ); + expect(legacy.rows, 'S5.05: the table is gone').toEqual([{ legacy: null }]); + }, + timeouts.spinUpPpgDev, + ); + }); +}); diff --git a/test/integration/test/cli-journeys/delete-statements-migration.mongo.e2e.test.ts b/test/integration/test/cli-journeys/delete-statements-migration.mongo.e2e.test.ts new file mode 100644 index 000000000000..202cc2113ef4 --- /dev/null +++ b/test/integration/test/cli-journeys/delete-statements-migration.mongo.e2e.test.ts @@ -0,0 +1,305 @@ +/** + * Statements that consent to data loss (MongoDB). + * + * Dropping a collection loses its documents, so `migration plan` and `db update` ask what the drop means; `--delete ` answers. The MongoDB planner refuses rename statements in this release. + * + * Journey S1 removes `Event` through `migration plan`: refused without `--delete Event`, planned with it, and applied by `migrate`. Journey S2 does the same through `db update`, where `--confirm ` no longer consents. Journey S3 adds a required field with no refusal. Journey S4 gives a rename, which the planner refuses with `statementRefused`. + */ + +import { copyFileSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs'; +import { rm } from 'node:fs/promises'; +import { join } from 'node:path'; +import { timeouts } from '@repo/test-utils'; +import { MongoClient } from 'mongodb'; +import { MongoMemoryReplSet } from 'mongodb-memory-server'; +import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest'; +import { fixtureAppDir } from '../utils/cli-test-helpers'; +import { + engineError, + type JourneyContext, + latestMigrationDirName, + parseJsonOutput, + runContractEmit, + runDbUpdate, + runMigrate, + runMigrationPlan, +} from '../utils/journey-test-helpers'; + +const FIXTURES_DIR = join(fixtureAppDir, 'fixtures/mongo-cli-journeys'); + +interface AppliedStatementReport { + readonly description: string; +} + +function withDatabase(baseUri: string, dbName: string): string { + const url = new URL(baseUri); + url.pathname = `/${dbName}`; + return url.toString(); +} + +function setupProject(connectionString: string): JourneyContext { + const testDir = join( + fixtureAppDir, + `test-mongo-delete-${Date.now()}-${Math.random().toString(36).slice(2)}`, + ); + const outputDir = join(testDir, 'output'); + mkdirSync(outputDir, { recursive: true }); + mkdirSync(join(testDir, 'migrations'), { recursive: true }); + writeFileSync( + join(testDir, 'package.json'), + JSON.stringify({ + name: 'mongo-delete-journey', + private: true, + type: 'module', + dependencies: { '@prisma/orm-mongo': 'workspace:0.16.0' }, + }), + 'utf-8', + ); + copyFileSync(join(FIXTURES_DIR, 'contract-with-events.ts'), join(testDir, 'contract.ts')); + const config = readFileSync(join(FIXTURES_DIR, 'prisma.config.with-db.ts'), 'utf-8').replace( + /\{\{DB_URL\}\}/g, + () => connectionString, + ); + const configPath = join(testDir, 'prisma.config.ts'); + writeFileSync(configPath, config, 'utf-8'); + return { testDir, configPath, outputDir }; +} + +const USERS_PSL = `// use prisma-8 + +model User { + id ObjectId @id @map("_id") + email String + name String + + @@map("users") +} +`; + +const USERS_WITH_NICKNAME_PSL = USERS_PSL.replace( + ' name String\n', + ' name String\n nickname String\n', +); + +/** A project whose contract is PSL, which emits a validator for each collection. */ +function setupPslProject(connectionString: string): JourneyContext { + const testDir = join( + fixtureAppDir, + `test-mongo-delete-psl-${Date.now()}-${Math.random().toString(36).slice(2)}`, + ); + const outputDir = join(testDir, 'output'); + mkdirSync(outputDir, { recursive: true }); + writeFileSync( + join(testDir, 'package.json'), + JSON.stringify({ + name: 'mongo-delete-psl-journey', + private: true, + type: 'module', + dependencies: { '@prisma/orm-mongo': 'workspace:0.16.0' }, + }), + 'utf-8', + ); + writeFileSync(join(testDir, 'contract.prisma'), USERS_PSL, 'utf-8'); + const configPath = join(testDir, 'prisma.config.ts'); + writeFileSync( + configPath, + [ + "import { defineConfig as ormConfig } from '@prisma/orm-mongo/config';", + "import { definePrismaConfig } from '@prisma/cli-engine';", + '', + 'export default definePrismaConfig({', + ' orm: ormConfig({', + " contract: './contract.prisma',", + " output: 'output',", + ` db: { connection: ${JSON.stringify(connectionString)} },`, + ' }),', + '});', + '', + ].join('\n'), + ); + return { testDir, configPath, outputDir }; +} + +function useFixture(ctx: JourneyContext, name: string): void { + copyFileSync(join(FIXTURES_DIR, name), join(ctx.testDir, 'contract.ts')); +} + +async function emit(ctx: JourneyContext, label: string): Promise { + const emitted = await runContractEmit(ctx); + expect(emitted.exitCode, `${label}: emit: ${emitted.stderr}`).toBe(0); +} + +function nextActionsOf(result: Parameters[0]): string { + return JSON.stringify(engineError(result)?.nextActions ?? []); +} + +describe('Journeys (MongoDB): statements that consent to data loss', { + timeout: timeouts.spinUpMongoMemoryServer, +}, () => { + let replSet: MongoMemoryReplSet; + let client: MongoClient; + const created = new Set(); + + beforeAll(async () => { + replSet = await MongoMemoryReplSet.create({ + instanceOpts: [ + { launchTimeout: timeouts.spinUpMongoMemoryServer, storageEngine: 'wiredTiger' }, + ], + replSet: { count: 1, storageEngine: 'wiredTiger' }, + }); + client = new MongoClient(replSet.getUri()); + await client.connect(); + }, timeouts.spinUpMongoMemoryServer); + + afterEach(async () => { + for (const dir of created) { + await rm(dir, { recursive: true, force: true }).catch(() => {}); + } + created.clear(); + }); + + afterAll(async () => { + await client?.close().catch(() => {}); + await replSet?.stop().catch(() => {}); + }, timeouts.spinUpMongoMemoryServer); + + async function collections(dbName: string): Promise { + const listed = await client.db(dbName).listCollections().toArray(); + return listed + .map(({ name }) => name) + .filter((name) => !name.startsWith('_prisma')) + .sort(); + } + + it('S1: migration plan refuses the drop until --delete names the model, then migrates it', async () => { + const dbName = 'mongo_delete_plan'; + const ctx = setupProject(withDatabase(replSet.getUri(), dbName)); + created.add(ctx.testDir); + await emit(ctx, 'S1.01'); + const initial = await runMigrationPlan(ctx, ['--name', 'initial']); + expect(initial.exitCode, `S1.02: plan initial: ${initial.stderr}`).toBe(0); + const applyInitial = await runMigrate(ctx); + expect(applyInitial.exitCode, `S1.03: migrate initial: ${applyInitial.stderr}`).toBe(0); + await client.db(dbName).collection('events').insertOne({ name: 'launch' }); + const origin = latestMigrationDirName(ctx); + useFixture(ctx, 'contract-additive.ts'); + await emit(ctx, 'S1.04'); + + const refused = await runMigrationPlan(ctx, ['--from', origin, '--json']); + expect(refused.exitCode, 'S1.05: the plan drops events, so it is refused').toBe(2); + expect(engineError(refused)?.code, 'S1.05: nobody to answer').toBe('CLI.CONSENT_REQUIRED'); + expect(nextActionsOf(refused), 'S1.05: names the flag that answers').toContain( + '--delete Event', + ); + + const plan = await runMigrationPlan(ctx, [ + '--name', + 'drop-events', + '--from', + origin, + '--delete', + 'Event', + '--json', + ]); + expect(plan.exitCode, `S1.06: plan with --delete: ${plan.stderr}`).toBe(0); + expect( + parseJsonOutput<{ appliedStatements: readonly AppliedStatementReport[] }>( + plan, + ).appliedStatements.map((entry) => entry.description), + 'S1.06: the delete is applied', + ).toEqual(['delete model "Event"']); + + const apply = await runMigrate(ctx); + expect(apply.exitCode, `S1.07: migrate: ${apply.stderr}`).toBe(0); + expect(await collections(dbName), 'S1.07: the collection is gone').toEqual(['users']); + }); + + it('S2: db update refuses --confirm and drops the collection with --delete', async () => { + const dbName = 'mongo_delete_update'; + const ctx = setupProject(withDatabase(replSet.getUri(), dbName)); + created.add(ctx.testDir); + await emit(ctx, 'S2.01'); + const create = await runDbUpdate(ctx, ['--no-interactive', '--json']); + expect(create.exitCode, `S2.02: db update creates the collections: ${create.stderr}`).toBe(0); + await client.db(dbName).collection('events').insertOne({ name: 'launch' }); + useFixture(ctx, 'contract-additive.ts'); + await emit(ctx, 'S2.03'); + + const confirmed = await runDbUpdate(ctx, ['--no-interactive', '--confirm', dbName, '--json']); + expect(confirmed.exitCode, 'S2.04: --confirm does not consent').toBe(2); + expect(engineError(confirmed)?.code, 'S2.04: the question is unanswered').toBe( + 'CLI.CONSENT_REQUIRED', + ); + expect(nextActionsOf(confirmed), 'S2.04: names the flag that answers').toContain( + '--delete Event', + ); + expect(await collections(dbName), 'S2.04: nothing dropped').toContain('events'); + + const deleted = await runDbUpdate(ctx, ['--no-interactive', '--delete', 'Event', '--json']); + expect(deleted.exitCode, `S2.05: db update --delete Event: ${deleted.stderr}`).toBe(0); + expect( + parseJsonOutput<{ appliedStatements: readonly AppliedStatementReport[] }>( + deleted, + ).appliedStatements.map((entry) => entry.description), + 'S2.05: the delete is applied', + ).toEqual(['delete model "Event"']); + expect(await collections(dbName), 'S2.05: the collection is gone').toEqual(['users']); + }); + + it('S3: db update adds a required field to a collection with documents without asking', async () => { + const dbName = 'mongo_delete_required'; + const ctx = setupPslProject(withDatabase(replSet.getUri(), dbName)); + created.add(ctx.testDir); + await emit(ctx, 'S3.01'); + const create = await runDbUpdate(ctx, ['--no-interactive', '--json']); + expect(create.exitCode, `S3.02: db update creates users: ${create.stderr}`).toBe(0); + await client.db(dbName).collection('users').insertOne({ email: 'a@example.com', name: 'a' }); + writeFileSync(join(ctx.testDir, 'contract.prisma'), USERS_WITH_NICKNAME_PSL, 'utf-8'); + await emit(ctx, 'S3.03'); + + const update = await runDbUpdate(ctx, ['--no-interactive', '--json']); + expect(update.exitCode, `S3.04: db update adds the field: ${update.stdout}`).toBe(0); + const [users] = await client + .db(dbName) + .listCollections({ name: 'users' }, { nameOnly: false }) + .toArray(); + expect(users, 'S3.05: the validator requires the new field').toMatchObject({ + options: { + validator: { $jsonSchema: { required: expect.arrayContaining(['nickname']) } }, + }, + }); + const documents = await client + .db(dbName) + .collection('users') + .find({}, { projection: { _id: 0 } }) + .toArray(); + expect(documents, 'S3.05: the existing document is kept').toEqual([ + { email: 'a@example.com', name: 'a' }, + ]); + }); + + it('S4: the planner refuses a rename statement', async () => { + const dbName = 'mongo_delete_rename'; + const ctx = setupProject(withDatabase(replSet.getUri(), dbName)); + created.add(ctx.testDir); + await emit(ctx, 'S4.01'); + const create = await runDbUpdate(ctx, ['--no-interactive', '--json']); + expect(create.exitCode, `S4.02: db update creates the collections: ${create.stderr}`).toBe(0); + useFixture(ctx, 'contract-events-renamed.ts'); + await emit(ctx, 'S4.03'); + + const renamed = await runDbUpdate(ctx, [ + '--no-interactive', + '--rename', + 'Event:Occasion', + '--json', + ]); + expect(renamed.exitCode, 'S4.04: the rename is refused').not.toBe(0); + const refusal = engineError(renamed); + expect(refusal?.code, 'S4.04: planning failed').toBe('MIGRATION.PLANNING_FAILED'); + expect(JSON.stringify(refusal?.meta), 'S4.04: the conflict is the refused statement').toContain( + 'statementRefused', + ); + expect(await collections(dbName), 'S4.04: nothing changed').toEqual(['events', 'users']); + }); +}); diff --git a/test/integration/test/cli-journeys/delete-statements-migration.sqlite.e2e.test.ts b/test/integration/test/cli-journeys/delete-statements-migration.sqlite.e2e.test.ts new file mode 100644 index 000000000000..826b9a2627b5 --- /dev/null +++ b/test/integration/test/cli-journeys/delete-statements-migration.sqlite.e2e.test.ts @@ -0,0 +1,332 @@ +/** + * Statements that consent to data loss (SQLite). + * + * The SQLite twin of `delete-statements-migration.e2e.test.ts`. SQLite has no checks and no row-level security, so the fixture has a unique field, a secondary index and a foreign key from `Post`, and there is no access to widen. + * + * Journey S1 renames `Profile` to `User` and its field `name` to `fullName`, and removes `Legacy`, through `migration plan`: refused without `--delete Legacy`, planned with it, and applied by `migrate`. Journey S2 does the same through `db update`, where `--confirm ` no longer consents, and the same statements fail on a second run. Journey S3 adds a required field with no refusal, to an empty table: SQLite's `db update` has no temporary default to fill existing rows, so on a table with rows the runner fails with `MIGRATION.RUNNER_FAILED` rather than asking. Journey S4 drops a model with no snapshot of the database's contract, so the subject is the table's unqualified name. + */ + +import { readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { DatabaseSync } from 'node:sqlite'; +import { join } from 'pathe'; +import { describe, expect, it } from 'vitest'; +import { withTempDir, writeProjectManifest } from '../utils/cli-test-helpers'; +import { + engineError, + getMigrationDirs, + type JourneyContext, + latestMigrationDirName, + parseJsonOutput, + planMigrationAndSelfEmit, + runContractEmit, + runDbUpdate, + runDbVerify, + runMigrate, + runMigrationPlan, + sqlitePslConfigFixture, + timeouts, +} from '../utils/journey-test-helpers'; + +const USERS = `model User { + id Int @id + fullName String @unique + handle String + posts Post[] + + @@index([fullName, handle]) +} + +model Post { + id Int @id + profileId Int + profile User @relation(fields: [profileId], references: [id]) +} +`; + +const LEGACY = ` +model Legacy { + id Int @id + note String +} +`; + +const FROM_PSL = `// use prisma-8 + +model Profile { + id Int @id + name String @unique + handle String + posts Post[] + + @@index([name, handle]) +} + +model Post { + id Int @id + profileId Int + profile Profile @relation(fields: [profileId], references: [id]) +} +${LEGACY}`; + +const TO_PSL = `// use prisma-8 + +${USERS}`; + +const TO_WITH_LEGACY_PSL = `${TO_PSL}${LEGACY}`; + +const TO_REQUIRED_FIELD_PSL = TO_PSL.replace( + ' handle String\n', + ' handle String\n nickname String\n', +); + +const RENAMES = ['--rename', 'Profile:User', '--rename', 'User.name:User.fullName'] as const; + +interface AppliedStatementReport { + readonly description: string; +} + +type SqliteJourney = JourneyContext & { readonly dbPath: string }; + +function setupSqliteJourney(createTempDir: () => string): SqliteJourney { + const testDir = createTempDir(); + const dbPath = join(testDir, 'journey.db'); + const configPath = join(testDir, 'prisma.config.ts'); + const config = readFileSync(sqlitePslConfigFixture, 'utf-8').replace('{{DB_PATH}}', () => dbPath); + writeFileSync(configPath, config, 'utf-8'); + writeFileSync(join(testDir, 'contract.prisma'), FROM_PSL, 'utf-8'); + writeProjectManifest(testDir); + return { testDir, configPath, outputDir: testDir, dbPath }; +} + +function withDatabase(dbPath: string, run: (db: DatabaseSync) => T): T { + const db = new DatabaseSync(dbPath); + try { + return run(db); + } finally { + db.close(); + } +} + +function seedRows(dbPath: string): void { + withDatabase(dbPath, (db) => { + db.exec( + `INSERT INTO "Profile" (id, name, handle) VALUES (1, 'alice', 'al'), (2, 'bob', 'bo'); + INSERT INTO "Post" (id, "profileId") VALUES (10, 1); + INSERT INTO "Legacy" (id, note) VALUES (1, 'old')`, + ); + }); +} + +function tables(dbPath: string): readonly unknown[] { + return withDatabase(dbPath, (db) => + db + .prepare(`SELECT name FROM sqlite_master WHERE type = 'table' ORDER BY name`) + .all() + .map((row) => row['name']), + ); +} + +async function emitContract(ctx: JourneyContext, psl: string, label: string): Promise { + writeFileSync(join(ctx.testDir, 'contract.prisma'), psl, 'utf-8'); + const emit = await runContractEmit(ctx); + expect(emit.exitCode, `${label}: emit: ${emit.stderr}`).toBe(0); +} + +function nextActionsOf(result: Parameters[0]): string { + return JSON.stringify(engineError(result)?.nextActions ?? []); +} + +async function expectRenamedWithoutLegacy(ctx: SqliteJourney, label: string): Promise { + const state = withDatabase(ctx.dbPath, (db) => ({ + rows: db.prepare(`SELECT id, "fullName", handle FROM "User" ORDER BY id`).all(), + indexes: db + .prepare( + `SELECT name FROM sqlite_master WHERE type = 'index' AND tbl_name = 'User' ORDER BY name`, + ) + .all() + .map((row) => row['name']), + postReferences: db + .prepare(`SELECT "table", "from", "to" FROM pragma_foreign_key_list('Post')`) + .all(), + })); + expect(state, `${label}: rows and objects under the new names`).toEqual({ + rows: [ + { id: 1, fullName: 'alice', handle: 'al' }, + { id: 2, fullName: 'bob', handle: 'bo' }, + ], + indexes: [ + expect.stringMatching(/^User_fullName_handle_idx_[0-9a-f]+$/), + 'sqlite_autoindex_User_1', + ], + postReferences: [{ table: 'User', from: 'profileId', to: 'id' }], + }); + const remaining = tables(ctx.dbPath); + expect(remaining.includes('Profile'), `${label}: the old table is gone`).toBe(false); + expect(remaining.includes('Legacy'), `${label}: the deleted table is gone`).toBe(false); + const verify = await runDbVerify(ctx, ['--schema-only']); + expect(verify.exitCode, `${label}: db verify --schema-only: ${verify.stderr}`).toBe(0); +} + +const STATEMENT_DESCRIPTIONS = [ + 'rename model "Profile" to "User"', + 'rename field "User.name" to "User.fullName"', + 'delete model "Legacy"', +]; + +withTempDir(({ createTempDir }) => { + describe('Journey S1 (SQLite): rename and delete with migration plan', () => { + it( + 'refuses the plan until --delete names the dropped model, then migrates it', + async () => { + const ctx = setupSqliteJourney(createTempDir); + await emitContract(ctx, FROM_PSL, 'S1.01'); + const initial = await planMigrationAndSelfEmit(ctx, ['--name', 'initial']); + expect(initial.exitCode, `S1.02: plan initial: ${initial.stderr}`).toBe(0); + const applyInitial = await runMigrate(ctx); + expect(applyInitial.exitCode, `S1.03: migrate initial: ${applyInitial.stderr}`).toBe(0); + seedRows(ctx.dbPath); + const origin = latestMigrationDirName(ctx); + await emitContract(ctx, TO_PSL, 'S1.04'); + + const refused = await runMigrationPlan(ctx, ['--from', origin, ...RENAMES, '--json']); + expect(refused.exitCode, 'S1.05: the plan loses Legacy, so it is refused').toBe(2); + expect(engineError(refused)?.code, 'S1.05: nobody to answer').toBe('CLI.CONSENT_REQUIRED'); + expect(nextActionsOf(refused), 'S1.05: names the flag that answers').toContain( + '--delete Legacy', + ); + expect(getMigrationDirs(ctx), 'S1.05: nothing written').toHaveLength(1); + + const plan = await runMigrationPlan(ctx, [ + '--name', + 'tidy-users', + '--from', + origin, + ...RENAMES, + '--delete', + 'Legacy', + '--json', + ]); + expect(plan.exitCode, `S1.06: plan with --delete: ${plan.stderr}`).toBe(0); + expect( + parseJsonOutput<{ appliedStatements: readonly AppliedStatementReport[] }>( + plan, + ).appliedStatements.map((entry) => entry.description), + 'S1.06: the renames, then the delete', + ).toEqual(STATEMENT_DESCRIPTIONS); + + const apply = await runMigrate(ctx); + expect(apply.exitCode, `S1.07: migrate: ${apply.stderr}`).toBe(0); + await expectRenamedWithoutLegacy(ctx, 'S1.08'); + + const fresh = await runMigrationPlan(ctx, [ + '--from', + latestMigrationDirName(ctx), + '--json', + ]); + expect(fresh.exitCode, `S1.09: plan with no schema change: ${fresh.stderr}`).toBe(0); + expect(parseJsonOutput<{ noOp: boolean }>(fresh).noOp, 'S1.09: plan is empty').toBe(true); + }, + timeouts.spinUpPpgDev, + ); + }); + + describe('Journey S2 (SQLite): rename and delete with db update', () => { + it( + 'refuses --confirm, applies with --delete, and refuses the same statements on a second run', + async () => { + const ctx = setupSqliteJourney(createTempDir); + await emitContract(ctx, FROM_PSL, 'S2.01'); + const create = await runDbUpdate(ctx, ['--json']); + expect(create.exitCode, `S2.02: db update creates the tables: ${create.stderr}`).toBe(0); + seedRows(ctx.dbPath); + await emitContract(ctx, TO_PSL, 'S2.03'); + + const confirmed = await runDbUpdate(ctx, [ + ...RENAMES, + '--no-interactive', + '--confirm', + 'journey.db', + '--json', + ]); + expect(confirmed.exitCode, 'S2.04: --confirm does not consent').toBe(2); + expect(engineError(confirmed)?.code, 'S2.04: the question is unanswered').toBe( + 'CLI.CONSENT_REQUIRED', + ); + expect(nextActionsOf(confirmed), 'S2.04: names the flag that answers').toContain( + '--delete Legacy', + ); + + const update = await runDbUpdate(ctx, [...RENAMES, '--delete', 'Legacy', '--json']); + expect(update.exitCode, `S2.05: db update with --delete: ${update.stderr}`).toBe(0); + expect( + parseJsonOutput<{ appliedStatements: readonly AppliedStatementReport[] }>( + update, + ).appliedStatements.map((entry) => entry.description), + 'S2.05: the renames, then the delete', + ).toEqual(STATEMENT_DESCRIPTIONS); + await expectRenamedWithoutLegacy(ctx, 'S2.06'); + + const again = await runDbUpdate(ctx, [...RENAMES, '--delete', 'Legacy', '--json']); + expect(again.exitCode, 'S2.07: the same statements fail on a second run').not.toBe(0); + expect(engineError(again)?.code, 'S2.07: the first statement no longer resolves').toBe( + 'MIGRATION.STATEMENT_UNRESOLVED', + ); + }, + timeouts.spinUpPpgDev, + ); + }); + + describe('Journey S3 (SQLite): a required field', () => { + it( + 'adds a non-nullable field to an empty table without asking', + async () => { + const ctx = setupSqliteJourney(createTempDir); + await emitContract(ctx, TO_PSL, 'S3.01'); + const create = await runDbUpdate(ctx, ['--json']); + expect(create.exitCode, `S3.02: db update creates User: ${create.stderr}`).toBe(0); + await emitContract(ctx, TO_REQUIRED_FIELD_PSL, 'S3.03'); + + const update = await runDbUpdate(ctx, ['--no-interactive', '--json']); + expect(update.exitCode, `S3.04: db update adds the field: ${update.stdout}`).toBe(0); + const columns = withDatabase(ctx.dbPath, (db) => + db + .prepare(`SELECT name FROM pragma_table_info('User') ORDER BY name`) + .all() + .map((row) => row['name']), + ); + expect(columns, 'S3.04: the field has a column').toContain('nickname'); + }, + timeouts.spinUpPpgDev, + ); + }); + + describe('Journey S4 (SQLite): no snapshot of the database contract', () => { + it( + 'names the dropped table by its unqualified storage name, and deletes it by that name', + async () => { + const ctx = setupSqliteJourney(createTempDir); + await emitContract(ctx, TO_WITH_LEGACY_PSL, 'S4.01'); + const create = await runDbUpdate(ctx, ['--json']); + expect(create.exitCode, `S4.02: db update creates the tables: ${create.stderr}`).toBe(0); + rmSync(join(ctx.testDir, 'migrations', 'snapshots'), { recursive: true, force: true }); + await emitContract(ctx, TO_PSL, 'S4.03'); + + const refused = await runDbUpdate(ctx, ['--no-interactive', '--json']); + expect(refused.exitCode, 'S4.04: the drop is refused').toBe(2); + const refusal = engineError(refused); + expect(refusal?.code, 'S4.04: the question is unanswered').toBe('CLI.CONSENT_REQUIRED'); + expect(refusal?.meta, 'S4.04: the subject is the table name').toMatchObject({ + unanswered: [{ subject: 'Legacy', verbs: ['delete'] }], + }); + expect(JSON.stringify(refusal), 'S4.04: the question says why').toContain( + 'named by its storage name because the origin contract is unknown', + ); + + const deleted = await runDbUpdate(ctx, ['--delete', 'Legacy', '--json']); + expect(deleted.exitCode, `S4.05: --delete Legacy: ${deleted.stderr}`).toBe(0); + expect(tables(ctx.dbPath).includes('Legacy'), 'S4.05: the table is gone').toBe(false); + }, + timeouts.spinUpPpgDev, + ); + }); +}); diff --git a/test/integration/test/cli-journeys/drift-schema.e2e.test.ts b/test/integration/test/cli-journeys/drift-schema.e2e.test.ts index a48ce1fab83d..467a65145b9f 100644 --- a/test/integration/test/cli-journeys/drift-schema.e2e.test.ts +++ b/test/integration/test/cli-journeys/drift-schema.e2e.test.ts @@ -18,9 +18,9 @@ import stripAnsi from 'strip-ansi'; import { describe, expect, it } from 'vitest'; import { withTempDir } from '../utils/cli-test-helpers'; import { - consentTokenFor, engineDiagnosticCodes, type JourneyContext, + parseJsonOutput, runContractEmit, runDbInit, runDbSchema, @@ -82,7 +82,7 @@ withTempDir(({ createTempDir }) => { // M.05: db update recovers by re-adding the NOT NULL column with a temporary default, // then dropping that default so future inserts must provide an explicit value. - const update = await runDbUpdate(ctx, ['--confirm', consentTokenFor(db.connectionString)]); + const update = await runDbUpdate(ctx); expect(update.exitCode, 'M.05: db update recovers dropped column drift').toBe(0); // M.06: db verify passes after reconciliation @@ -161,15 +161,18 @@ withTempDir(({ createTempDir }) => { // N.06: db update --no-interactive rejects (drift from unmanaged 'age' column // makes the planner classify this as destructive) - const update = await runDbUpdate(ctx, ['--no-interactive']); + const update = await runDbUpdate(ctx, ['--json', '--no-interactive']); expect(update.exitCode, 'N.06: --no-interactive rejects destructive').toBe(2); + expect(parseJsonOutput(update), 'N.06: the unmanaged column is the question').toMatchObject( + { + code: 'CLI.CONSENT_REQUIRED', + meta: { unanswered: [{ subject: 'public.user.age', verbs: ['delete'] }] }, + }, + ); - // N.07: db update --confirm explicitly accepts the destructive plan - const updateConfirmed = await runDbUpdate(ctx, [ - '--confirm', - consentTokenFor(db.connectionString), - ]); - expect(updateConfirmed.exitCode, 'N.07: db update --confirm accepts').toBe(0); + // N.07: db update --delete names the storage no model stores, and applies + const updateConfirmed = await runDbUpdate(ctx, ['--delete', 'public.user.age']); + expect(updateConfirmed.exitCode, 'N.07: db update --delete applies').toBe(0); // N.08: db verify --schema-only tolerant (passes — all contract columns present; 'age' tolerated as extra) const tolerantAfter = await runDbVerify(ctx, ['--schema-only']); diff --git a/test/integration/test/cli-journeys/migration-apply-edge-cases.e2e.test.ts b/test/integration/test/cli-journeys/migration-apply-edge-cases.e2e.test.ts index 836210e8732e..89120faadb30 100644 --- a/test/integration/test/cli-journeys/migration-apply-edge-cases.e2e.test.ts +++ b/test/integration/test/cli-journeys/migration-apply-edge-cases.e2e.test.ts @@ -184,6 +184,8 @@ withTempDir(({ createTempDir }) => { 'drop-email', '--from', latestMigrationDirName(ctx), + '--delete', + 'User.email', ]); expect(plan1.exitCode, 'plan drop-email').toBe(0); @@ -264,6 +266,10 @@ withTempDir(({ createTempDir }) => { 'drop-email', '--from', latestMigrationDirName(ctx), + '--delete', + 'User.email', + '--delete', + 'User.name', ]); expect(plan2.exitCode, 'plan drop-email').toBe(0); diff --git a/test/integration/test/cli-journeys/migration-plan-details.e2e.test.ts b/test/integration/test/cli-journeys/migration-plan-details.e2e.test.ts index e024bf4ff397..ccd907cc5ea6 100644 --- a/test/integration/test/cli-journeys/migration-plan-details.e2e.test.ts +++ b/test/integration/test/cli-journeys/migration-plan-details.e2e.test.ts @@ -135,6 +135,8 @@ withTempDir(({ createTempDir }) => { 'drop-email', '--from', latestMigrationDirName(ctx), + '--delete', + 'User.email', '--json', ]); expect(planDrop.exitCode, 'I.03: plan drop-email').toBe(0); diff --git a/test/integration/test/cli-journeys/mongo-db-update-consent.e2e.test.ts b/test/integration/test/cli-journeys/mongo-db-update-consent.e2e.test.ts index 55a2755b0483..3fd52b2730c4 100644 --- a/test/integration/test/cli-journeys/mongo-db-update-consent.e2e.test.ts +++ b/test/integration/test/cli-journeys/mongo-db-update-consent.e2e.test.ts @@ -8,9 +8,9 @@ import stripAnsi from 'strip-ansi'; import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest'; import { fixtureAppDir } from '../utils/cli-test-helpers'; import { - consentTokenFor, engineError, type JourneyContext, + parseJsonOutput, runContractEmit, runDbInit, runDbUpdate, @@ -94,7 +94,7 @@ function setupPslProject(connectionString: string): JourneyContext { return { testDir, configPath, outputDir }; } -describe('Journey: Mongo db update confirms destructive changes', { +describe('Journey: Mongo db update asks for consent only to lose data', { timeout: timeouts.spinUpMongoMemoryServer, }, () => { let replSet: MongoMemoryReplSet; @@ -124,7 +124,7 @@ describe('Journey: Mongo db update confirms destructive changes', { await replSet?.stop().catch(() => {}); }, timeouts.spinUpMongoMemoryServer); - it('drops an index once the database name is passed with --confirm', async () => { + it('drops an index without consent, because dropping an index loses no documents', async () => { const connectionString = withDatabase(replSet.getUri(), DB_NAME); const ctx = setupProject(connectionString); created.add(ctx.testDir); @@ -136,19 +136,45 @@ describe('Journey: Mongo db update confirms destructive changes', { copyFileSync(join(FIXTURES_DIR, 'contract-base.ts'), join(ctx.testDir, 'contract.ts')); expect((await runContractEmit(ctx)).exitCode).toBe(0); - const destructive = await runDbUpdate(ctx, [ - '--no-interactive', - '--confirm', - consentTokenFor(connectionString), - ]); + const widening = await runDbUpdate(ctx, ['--no-interactive']); - expect(destructive.exitCode, stripAnsi(destructive.stderr)).toBe(0); + expect(widening.exitCode, stripAnsi(widening.stderr)).toBe(0); const indexKeys = (await client.db(DB_NAME).collection('users').indexes()).map( (index) => index.key, ); expect(indexKeys).toEqual([{ _id: 1 }, { email: 1 }]); }); + it('drops a collection once --delete names its model', async () => { + const dbName = 'mongo_consent_drop_collection'; + const connectionString = withDatabase(replSet.getUri(), dbName); + const ctx = setupProject(connectionString); + created.add(ctx.testDir); + + copyFileSync(join(FIXTURES_DIR, 'contract-with-events.ts'), join(ctx.testDir, 'contract.ts')); + expect((await runContractEmit(ctx)).exitCode).toBe(0); + const additive = await runDbUpdate(ctx, ['--no-interactive']); + expect(additive.exitCode, stripAnsi(additive.stderr)).toBe(0); + + copyFileSync(join(FIXTURES_DIR, 'contract-additive.ts'), join(ctx.testDir, 'contract.ts')); + expect((await runContractEmit(ctx)).exitCode).toBe(0); + + const unconfirmed = await runDbUpdate(ctx, ['--json', '--no-interactive']); + expect(unconfirmed.exitCode).toBe(2); + expect(parseJsonOutput(unconfirmed)).toMatchObject({ + code: 'CLI.CONSENT_REQUIRED', + meta: { unanswered: [{ subject: 'Event', verbs: ['delete'] }] }, + }); + const kept = await client.db(dbName).listCollections({ name: 'events' }).toArray(); + expect(kept.map(({ name }) => name)).toEqual(['events']); + + const destructive = await runDbUpdate(ctx, ['--no-interactive', '--delete', 'Event']); + + expect(destructive.exitCode, stripAnsi(destructive.stderr)).toBe(0); + const collections = await client.db(dbName).listCollections({ name: 'events' }).toArray(); + expect(collections).toEqual([]); + }); + it('points db init at db update when a collection with data needs a validator', async () => { const dbName = 'mongo_init_journey'; const connectionString = withDatabase(replSet.getUri(), dbName); @@ -168,11 +194,7 @@ describe('Journey: Mongo db update confirms destructive changes', { ], }); - const update = await runDbUpdate(ctx, [ - '--no-interactive', - '--confirm', - consentTokenFor(connectionString), - ]); + const update = await runDbUpdate(ctx, ['--no-interactive']); expect(update.exitCode, stripAnsi(update.stderr)).toBe(0); const [collection] = await client.db(dbName).listCollections({ name: 'events' }).toArray(); expect(collection).toMatchObject({ diff --git a/test/integration/test/cli-journeys/rename-table-migration.sqlite.e2e.test.ts b/test/integration/test/cli-journeys/rename-table-migration.sqlite.e2e.test.ts index 25d01fafd5f8..2320a1746202 100644 --- a/test/integration/test/cli-journeys/rename-table-migration.sqlite.e2e.test.ts +++ b/test/integration/test/cli-journeys/rename-table-migration.sqlite.e2e.test.ts @@ -294,7 +294,7 @@ withTempDir(({ createTempDir }) => { db.exec('ALTER TABLE "_prisma_rename_UserProfile" RENAME TO "UserProfile"'); }); - const update = await runDbUpdate(ctx, ['--json', '--confirm', 'journey.db']); + const update = await runDbUpdate(ctx, ['--json']); expect(update.exitCode, `R4.05: db update after the rename: ${update.stdout}`).toBe(0); const state = withDatabase(ctx.dbPath, (db) => ({ rows: db.prepare(`SELECT id, email FROM "UserProfile" ORDER BY id`).all(), diff --git a/test/integration/test/cli-journeys/rollback-cycle.e2e.test.ts b/test/integration/test/cli-journeys/rollback-cycle.e2e.test.ts index 41d3c464e762..d3304a81f048 100644 --- a/test/integration/test/cli-journeys/rollback-cycle.e2e.test.ts +++ b/test/integration/test/cli-journeys/rollback-cycle.e2e.test.ts @@ -79,6 +79,8 @@ withTempDir(({ createTempDir }) => { addPhoneDir, '--to', rollbackTarget, + '--delete', + 'User.phone', '--json', ]); expect(planRollback.exitCode, 'plan rollback --to ^').toBe(0); @@ -91,7 +93,7 @@ withTempDir(({ createTempDir }) => { expect(rollback.to, 'rollback to predecessor C1').toBe(c1Hash); expect( rollback.operations.some((op) => op.operationClass === 'destructive'), - 'reverse delta drops the added column (destructive), no refusal', + 'reverse delta drops the added column (destructive), answered by --delete', ).toBe(true); const contractSource = readFileSync(join(ctx.testDir, 'contract.ts'), 'utf-8'); expect(contractSource, 'contract source untouched (still phone variant)').toContain( diff --git a/test/integration/test/cli.db-update.e2e.test.ts b/test/integration/test/cli.db-update.e2e.test.ts index 5fce1536491a..3078a3b676e1 100644 --- a/test/integration/test/cli.db-update.e2e.test.ts +++ b/test/integration/test/cli.db-update.e2e.test.ts @@ -7,7 +7,6 @@ import { fixtureAppDir, runOnEngine, withTempDir } from './utils/cli-test-helper import { replaceInFileOrThrow } from './utils/contract-fixture-editing'; import { runDbInit } from './utils/db-init-test-helpers'; import { - consentTokenFor, type DbUpdateTestSetup, runDbUpdate, runDbUpdateAllowFailure, @@ -346,9 +345,9 @@ withTempDir(({ createTempDir }) => { timeouts.spinUpPpgDev, ); - // Scenario 7a: Destructive changes require explicit consent + // Scenario 7a: Destructive changes need a statement that answers them it( - 'fails with DESTRUCTIVE_CHANGES when destructive ops are not confirmed', + 'refuses a drop no statement answers, naming the storage it would lose', async () => { await withDevDatabase(async ({ connectionString }) => { const { testSetup, configPath } = await setupDbUpdateFixture( @@ -372,8 +371,9 @@ withTempDir(({ createTempDir }) => { expect(run.exitCode).toBe(2); expect(stripAnsi(run.stderr)).toMatch(/CONSENT_REQUIRED/); + expect(stripAnsi(run.stderr)).toContain('--delete public.project.legacy_notes'); - // Verify the confirmation requirement actually blocked the update — + // Verify the refusal actually blocked the update — // the drift column must still exist await withClient(connectionString, async (client) => { const result = await client.query( @@ -410,8 +410,8 @@ withTempDir(({ createTempDir }) => { const run = await runDbUpdateAllowFailure(testSetup, [ '--config', configPath, - '--confirm', - consentTokenFor(connectionString), + '--delete', + 'public.project.legacy_notes', ]); expect(run.exitCode).not.toBe(0); diff --git a/test/integration/test/cli.db-update.preflight-gaps.e2e.test.ts b/test/integration/test/cli.db-update.preflight-gaps.e2e.test.ts index 78d630e1e58e..d895d5e7fd1a 100644 --- a/test/integration/test/cli.db-update.preflight-gaps.e2e.test.ts +++ b/test/integration/test/cli.db-update.preflight-gaps.e2e.test.ts @@ -20,7 +20,6 @@ import { describe, expect, it } from 'vitest'; import { fixtureAppDir, runOnEngine, withTempDir } from './utils/cli-test-helpers'; import { runDbInit } from './utils/db-init-test-helpers'; import { - consentTokenFor, type DbUpdateTestSetup, runDbUpdate, runDbUpdateAllowFailure, @@ -68,12 +67,7 @@ withTempDir(({ createTempDir }) => { await swapToVariant(testSetup, 'contract-add-fk.ts'); - const run = await runDbUpdate(testSetup, [ - '--config', - configPath, - '--confirm', - consentTokenFor(connectionString), - ]); + const run = await runDbUpdate(testSetup, ['--config', configPath]); expect(run.exitCode).toBe(0); await withClient(connectionString, async (client) => { @@ -115,12 +109,7 @@ withTempDir(({ createTempDir }) => { await swapToVariant(testSetup, 'contract-add-required-unique.ts'); - const run = await runDbUpdate(testSetup, [ - '--config', - configPath, - '--confirm', - consentTokenFor(connectionString), - ]); + const run = await runDbUpdate(testSetup, ['--config', configPath]); expect(run.exitCode).toBe(0); await withClient(connectionString, async (client) => { @@ -167,12 +156,7 @@ withTempDir(({ createTempDir }) => { await swapToVariant(testSetup, 'contract-add-required-unique.ts'); - const run = await runDbUpdateAllowFailure(testSetup, [ - '--config', - configPath, - '--confirm', - consentTokenFor(connectionString), - ]); + const run = await runDbUpdateAllowFailure(testSetup, ['--config', configPath]); expect(run.exitCode).not.toBe(0); await withClient(connectionString, async (client) => { diff --git a/test/integration/test/cli.migration-plan-ref-aware.e2e.test.ts b/test/integration/test/cli.migration-plan-ref-aware.e2e.test.ts index 73ed7b8c40b4..0d1e96409141 100644 --- a/test/integration/test/cli.migration-plan-ref-aware.e2e.test.ts +++ b/test/integration/test/cli.migration-plan-ref-aware.e2e.test.ts @@ -197,7 +197,13 @@ withTempDir(({ createTempDir }) => { await withJourney(createTempDir, connectionString, async (ctx) => { const { dbRefHash } = await seedDevToShipTrap(ctx); - const plan = await runMigrationPlan(ctx, ['--name', 'add-phone', '--json']); + const plan = await runMigrationPlan(ctx, [ + '--name', + 'add-phone', + '--delete', + 'User.name', + '--json', + ]); expect(plan.exitCode).toBe(0); const planJson = parseJsonOutput(plan); expect(planJson.baselineDir).toBeDefined(); @@ -246,7 +252,15 @@ withTempDir(({ createTempDir }) => { await withJourney(createTempDir, connectionString, async (ctx) => { await seedDevToShipTrap(ctx); expect( - (await runAutoBaselinePlanAndEmit(ctx, ['--name', 'trap-close', '--json'])).exitCode, + ( + await runAutoBaselinePlanAndEmit(ctx, [ + '--name', + 'trap-close', + '--delete', + 'User.name', + '--json', + ]) + ).exitCode, ).toBe(0); expect((await runMigrate(ctx)).exitCode).toBe(0); @@ -254,7 +268,7 @@ withTempDir(({ createTempDir }) => { expect((await runContractEmit(ctx)).exitCode).toBe(0); const dbRefHash = readDbRefHash(ctx); - const plan = await runMigrationPlan(ctx, ['--json']); + const plan = await runMigrationPlan(ctx, ['--delete', 'User.name', '--json']); expect(plan.exitCode).toBe(0); const planJson = parseJsonOutput(plan); expect(planJson.baselineDir).toBeUndefined(); @@ -488,7 +502,13 @@ withTempDir(({ createTempDir }) => { await withJourney(createTempDir, connectionString, async (ctx) => { await seedDevToShipTrap(ctx); - const implicit = await runMigrationPlan(ctx, ['--name', 'implicit', '--json']); + const implicit = await runMigrationPlan(ctx, [ + '--name', + 'implicit', + '--delete', + 'User.name', + '--json', + ]); expect(implicit.exitCode).toBe(0); const implicitJson = parseJsonOutput(implicit); @@ -499,6 +519,8 @@ withTempDir(({ createTempDir }) => { 'db', '--name', 'explicit', + '--delete', + 'User.name', '--json', ]); expect(explicit.exitCode).toBe(0); diff --git a/test/integration/test/cross-package/postgres-issue-planner.test.ts b/test/integration/test/cross-package/postgres-issue-planner.test.ts index f9502e62a56a..7003f7acb303 100644 --- a/test/integration/test/cross-package/postgres-issue-planner.test.ts +++ b/test/integration/test/cross-package/postgres-issue-planner.test.ts @@ -196,7 +196,7 @@ describe('planIssues', () => { expect(calls).toHaveLength(3); expect(calls[0]).toMatchObject({ factoryName: 'addColumn' }); expect(calls[1]).toMatchObject({ factoryName: 'dataTransform' }); - expect(calls[2]).toMatchObject({ factoryName: 'setNotNull' }); + expect(calls[2]).toMatchObject({ factoryName: 'setNotNull', operationClass: 'widening' }); }); it('DataTransformCall.toOp() rejects with MIGRATION.UNFILLED_PLACEHOLDER', async () => { @@ -271,7 +271,7 @@ describe('planIssues', () => { const calls = result.value.calls; expect(calls).toHaveLength(2); expect(calls[0]).toMatchObject({ factoryName: 'dataTransform' }); - expect(calls[1]).toMatchObject({ factoryName: 'setNotNull' }); + expect(calls[1]).toMatchObject({ factoryName: 'setNotNull', operationClass: 'widening' }); }); }); @@ -344,7 +344,10 @@ describe('planIssues', () => { if (!result.ok) throw new Error('expected ok'); const calls = result.value.calls; expect(calls).toHaveLength(1); - expect(calls[0]).toMatchObject({ factoryName: 'alterColumnType' }); + expect(calls[0]).toMatchObject({ + factoryName: 'alterColumnType', + operationClass: 'widening', + }); }); it('emits DataTransformCall + AlterColumnTypeCall for unsafe change', () => { @@ -374,7 +377,10 @@ describe('planIssues', () => { const calls = result.value.calls; expect(calls).toHaveLength(2); expect(calls[0]).toMatchObject({ factoryName: 'dataTransform' }); - expect(calls[1]).toMatchObject({ factoryName: 'alterColumnType' }); + expect(calls[1]).toMatchObject({ + factoryName: 'alterColumnType', + operationClass: 'destructive', + }); }); }); diff --git a/test/integration/test/fixtures/cli/cli-e2e-test-app/fixtures/mongo-cli-journeys/contract-events-renamed.ts b/test/integration/test/fixtures/cli/cli-e2e-test-app/fixtures/mongo-cli-journeys/contract-events-renamed.ts new file mode 100644 index 000000000000..f0f257379705 --- /dev/null +++ b/test/integration/test/fixtures/cli/cli-e2e-test-app/fixtures/mongo-cli-journeys/contract-events-renamed.ts @@ -0,0 +1,23 @@ +import { defineContract, field, index, model } from '@internal/mongo/contract-builder'; + +const User = model('User', { + collection: 'users', + fields: { + _id: field.objectId(), + email: field.string(), + name: field.string(), + }, + indexes: [index({ email: 1 }, { unique: true }), index({ name: 1 })], +}); + +const Occasion = model('Occasion', { + collection: 'occasions', + fields: { + _id: field.objectId(), + name: field.string(), + }, +}); + +export const contract = defineContract({ + models: { User, Occasion }, +}); diff --git a/test/integration/test/fixtures/cli/cli-e2e-test-app/fixtures/mongo-cli-journeys/contract-with-events.ts b/test/integration/test/fixtures/cli/cli-e2e-test-app/fixtures/mongo-cli-journeys/contract-with-events.ts new file mode 100644 index 000000000000..3e7afe343440 --- /dev/null +++ b/test/integration/test/fixtures/cli/cli-e2e-test-app/fixtures/mongo-cli-journeys/contract-with-events.ts @@ -0,0 +1,23 @@ +import { defineContract, field, index, model } from '@internal/mongo/contract-builder'; + +const User = model('User', { + collection: 'users', + fields: { + _id: field.objectId(), + email: field.string(), + name: field.string(), + }, + indexes: [index({ email: 1 }, { unique: true }), index({ name: 1 })], +}); + +const Event = model('Event', { + collection: 'events', + fields: { + _id: field.objectId(), + name: field.string(), + }, +}); + +export const contract = defineContract({ + models: { User, Event }, +}); diff --git a/test/integration/test/fixtures/cli/cli-e2e-test-app/package.json b/test/integration/test/fixtures/cli/cli-e2e-test-app/package.json index 3dc88db12414..b3e547b65bfc 100644 --- a/test/integration/test/fixtures/cli/cli-e2e-test-app/package.json +++ b/test/integration/test/fixtures/cli/cli-e2e-test-app/package.json @@ -19,7 +19,7 @@ "@internal/target-mongo": "workspace:8.0.0-rc.16", "@internal/target-postgres": "workspace:8.0.0-rc.16", "@internal/utils": "workspace:8.0.0-rc.16", - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@prisma/orm-extension-pgvector": "workspace:8.0.0-rc.16", "@prisma/orm-mongo": "workspace:8.0.0-rc.16", "@prisma/orm-postgres": "workspace:8.0.0-rc.16", diff --git a/test/integration/test/fixtures/cli/cli-integration-test-app/package.json b/test/integration/test/fixtures/cli/cli-integration-test-app/package.json index f53abfdbbeef..1d4bc33991e2 100644 --- a/test/integration/test/fixtures/cli/cli-integration-test-app/package.json +++ b/test/integration/test/fixtures/cli/cli-integration-test-app/package.json @@ -20,7 +20,7 @@ "@internal/target-mongo": "workspace:8.0.0-rc.16", "@internal/target-postgres": "workspace:8.0.0-rc.16", "@internal/utils": "workspace:8.0.0-rc.16", - "@prisma/cli-engine": "0.6.3", + "@prisma/cli-engine": "0.7.0", "@prisma/orm-extension-pgvector": "workspace:8.0.0-rc.16", "@prisma/orm-mongo": "workspace:8.0.0-rc.16", "@prisma/orm-postgres": "workspace:8.0.0-rc.16", diff --git a/test/integration/test/mongo/migration-e2e.test.ts b/test/integration/test/mongo/migration-e2e.test.ts index ecb1fdabbae4..26d45baab879 100644 --- a/test/integration/test/mongo/migration-e2e.test.ts +++ b/test/integration/test/mongo/migration-e2e.test.ts @@ -372,7 +372,7 @@ describe('MongoDB migration E2E', { timeout: timeouts.spinUpMongoMemoryServer }, expect(dropResult.plan.operations).toHaveLength(1); const dropOp = await dropResult.plan.operations[0]!; - expect(dropOp.operationClass).toBe('destructive'); + expect(dropOp.operationClass).toBe('widening'); expect(dropOp.label).toContain('Drop index'); // Step 3: Apply drop diff --git a/test/integration/test/upgrade-instructions/migration-statement-refusal/detection.test.ts b/test/integration/test/upgrade-instructions/migration-statement-refusal/detection.test.ts new file mode 100644 index 000000000000..88f73d2bf989 --- /dev/null +++ b/test/integration/test/upgrade-instructions/migration-statement-refusal/detection.test.ts @@ -0,0 +1,46 @@ +import { readFileSync } from 'node:fs'; +import { fileURLToPath } from 'node:url'; +import { dirname, join } from 'pathe'; +import { describe, expect, it } from 'vitest'; + +const FRAGMENT_PATH = + 'upgrade-instructions/pending/migration-statement-refusal/app/instructions.md'; + +const here = dirname(fileURLToPath(import.meta.url)); +const fragment = readFileSync(join(here, '../../../../..', FRAGMENT_PATH), 'utf-8'); + +function detectionOf(id: string): readonly RegExp[] { + const entry = fragment.split(/\n {2}- id: /).find((block) => block.startsWith(`${id}\n`)); + if (entry === undefined) throw new Error(`The fragment has no change ${id}`); + return [...entry.matchAll(/^ {8}- '(.*)'$/gm)].map( + ([, pattern]) => new RegExp((pattern ?? '').replaceAll("''", "'")), + ); +} + +function detects(id: string, content: string): boolean { + return detectionOf(id).some((pattern) => pattern.test(content)); +} + +describe('db-update-confirm-no-longer-consents detection', () => { + it('finds --confirm on a db update command line', () => { + expect( + detects('db-update-confirm-no-longer-consents', 'prisma db update --confirm appdb'), + ).toBe(true); + }); + + it('finds --confirm on a db update command continued over several lines', () => { + const script = 'pnpm prisma db update \\\n --no-interactive \\\n --confirm appdb\n'; + expect(detects('db-update-confirm-no-longer-consents', script)).toBe(true); + }); + + it('skips a --confirm that belongs to a later command', () => { + const script = 'pnpm prisma db update --json\npnpm prisma orm init --confirm my-app\n'; + expect(detects('db-update-confirm-no-longer-consents', script)).toBe(false); + }); + + it('skips a longer flag that starts with --confirm', () => { + expect(detects('db-update-confirm-no-longer-consents', 'prisma db update --confirmed')).toBe( + false, + ); + }); +}); diff --git a/test/integration/test/utils/db-update-test-helpers.ts b/test/integration/test/utils/db-update-test-helpers.ts index 804673ed6069..8ee1710727b6 100644 --- a/test/integration/test/utils/db-update-test-helpers.ts +++ b/test/integration/test/utils/db-update-test-helpers.ts @@ -85,18 +85,3 @@ export async function runDbUpdate( * which of the two they mean. */ export const runDbUpdateAllowFailure = runDbUpdate; - -/** - * What `db update` asks the user to type before it destroys anything: the name - * of the connected database, which for these Postgres-backed tests is the - * database segment of the connection URL. A run that means to accept data loss - * passes it as `--confirm`, because `--yes` cannot grant a consent. - */ -export function consentTokenFor(connectionString: string): string { - const parsed = new URL(connectionString); - const name = parsed.pathname.split('/').filter((segment) => segment.length > 0)[0]; - if (name === undefined) { - throw new Error(`Connection URL names no database: ${connectionString}`); - } - return decodeURIComponent(name); -} diff --git a/test/integration/test/utils/journey-test-helpers.ts b/test/integration/test/utils/journey-test-helpers.ts index 82178a70eae4..82abd945eb55 100644 --- a/test/integration/test/utils/journey-test-helpers.ts +++ b/test/integration/test/utils/journey-test-helpers.ts @@ -339,21 +339,6 @@ export async function runDbUpdate( return runOnEngine(ctx, ['db', 'update', ...extraArgs], options); } -/** - * What `db update` asks the user to type before it destroys anything: the name - * of the connected database, which for these Postgres-backed tests is the - * database segment of the connection URL. A run that means to accept data loss - * passes it as `--confirm`, because `--yes` cannot grant a consent. - */ -export function consentTokenFor(connectionString: string): string { - const parsed = new URL(connectionString); - const name = parsed.pathname.split('/').filter((segment) => segment.length > 0)[0]; - if (name === undefined) { - throw new Error(`Connection URL names no database: ${connectionString}`); - } - return decodeURIComponent(name); -} - export async function runDbVerify( ctx: JourneyContext, extraArgs: readonly string[] = [], diff --git a/test/utils/package.json b/test/utils/package.json index af6de60c8183..46527cc090c8 100644 --- a/test/utils/package.json +++ b/test/utils/package.json @@ -55,6 +55,10 @@ "types": "./dist/exports/timeouts.d.mts", "import": "./dist/exports/timeouts.mjs" }, + "./data-loss-expectations": { + "types": "./dist/exports/data-loss-expectations.d.mts", + "import": "./dist/exports/data-loss-expectations.mjs" + }, "./typed-expectations": { "types": "./dist/exports/typed-expectations.d.mts", "import": "./dist/exports/typed-expectations.mjs" diff --git a/test/utils/src/data-loss-expectations.ts b/test/utils/src/data-loss-expectations.ts new file mode 100644 index 000000000000..dcc4ae0f91af --- /dev/null +++ b/test/utils/src/data-loss-expectations.ts @@ -0,0 +1,54 @@ +import { expect } from 'vitest'; + +/** A subject as a planner's `dataLoss` names it: a model, a field or a storage name. */ +interface LossSubject { + readonly kind: string; + readonly namespaceId?: string; + readonly model?: string; + readonly field?: string; + readonly name?: string; +} + +function keyOf(subject: LossSubject): string { + return JSON.stringify([ + subject.kind, + subject.namespaceId, + subject.model, + subject.field, + subject.name, + ]); +} + +/** + * Asserts a planner's `dataLoss` against its own classification: an entry points only at a + * `destructive` operation, and every `destructive` operation has an entry. The one exception is an + * operation whose loss an earlier entry already names, as a SQLite recreate names a column that a + * later drop removes again; `subjectOf` says what such an operation loses, and the operation + * passes only when an earlier entry has that subject. + */ +export function expectDataLossMatchesDestructive< + TOperation extends { readonly operationClass: string }, +>( + result: { + readonly dataLoss: readonly { + readonly operationIndex: number; + readonly subject: LossSubject; + }[]; + }, + operations: readonly TOperation[], + options: { readonly subjectOf?: (operation: TOperation) => LossSubject | undefined } = {}, +): void { + const named = new Set(result.dataLoss.map(({ operationIndex }) => operationIndex)); + const classesNamed = [...named].map((index) => operations[index]?.operationClass); + expect(classesNamed.every((operationClass) => operationClass === 'destructive')).toBe(true); + const namedBefore = (index: number, subject: LossSubject) => + result.dataLoss.some( + (entry) => entry.operationIndex < index && keyOf(entry.subject) === keyOf(subject), + ); + const unnamedDestructive = operations.flatMap((operation, index) => { + if (operation.operationClass !== 'destructive' || named.has(index)) return []; + const subject = options.subjectOf?.(operation); + return subject !== undefined && namedBefore(index, subject) ? [] : [index]; + }); + expect(unnamedDestructive).toEqual([]); +} diff --git a/test/utils/test/data-loss-expectations.test.ts b/test/utils/test/data-loss-expectations.test.ts new file mode 100644 index 000000000000..e497f1303c3b --- /dev/null +++ b/test/utils/test/data-loss-expectations.test.ts @@ -0,0 +1,60 @@ +import { describe, expect, it } from 'vitest'; +import { expectDataLossMatchesDestructive } from '../src/data-loss-expectations'; + +const dropA = { label: 'Drop table A', operationClass: 'destructive' }; +const dropB = { label: 'Drop table B', operationClass: 'destructive' }; +const addIndex = { label: 'Create index', operationClass: 'additive' }; +const tableA = { kind: 'model', namespaceId: 'app', model: 'A' }; +const tableB = { kind: 'model', namespaceId: 'app', model: 'B' }; + +describe('expectDataLossMatchesDestructive', () => { + it('passes when each destructive operation is named', () => { + expectDataLossMatchesDestructive( + { + dataLoss: [ + { operationIndex: 0, subject: tableA }, + { operationIndex: 2, subject: tableB }, + ], + }, + [dropA, addIndex, dropB], + ); + }); + + it('fails on a destructive operation no entry names, after one that is named', () => { + expect(() => + expectDataLossMatchesDestructive({ dataLoss: [{ operationIndex: 0, subject: tableA }] }, [ + dropA, + dropB, + ]), + ).toThrow(); + }); + + it('fails on an entry at an operation that is not destructive', () => { + expect(() => + expectDataLossMatchesDestructive({ dataLoss: [{ operationIndex: 0, subject: tableA }] }, [ + addIndex, + ]), + ).toThrow(); + }); + + it('passes an unnamed destructive operation whose loss an earlier entry names', () => { + expectDataLossMatchesDestructive( + { dataLoss: [{ operationIndex: 0, subject: tableA }] }, + [ + { label: 'Recreate table A', operationClass: 'destructive' }, + { label: 'Drop table A again', operationClass: 'destructive' }, + ], + { subjectOf: (operation) => (operation.label.endsWith('again') ? tableA : undefined) }, + ); + }); + + it('fails an unnamed destructive operation whose loss no earlier entry names', () => { + expect(() => + expectDataLossMatchesDestructive( + { dataLoss: [{ operationIndex: 0, subject: tableA }] }, + [dropA, dropB], + { subjectOf: () => tableB }, + ), + ).toThrow(); + }); +}); diff --git a/test/utils/tsdown.config.ts b/test/utils/tsdown.config.ts index d808612bc255..88c8166e59cd 100644 --- a/test/utils/tsdown.config.ts +++ b/test/utils/tsdown.config.ts @@ -4,6 +4,7 @@ export default defineConfig({ entry: { index: 'src/exports/index.ts', 'contract-factories': 'src/contract-factories.ts', + 'data-loss-expectations': 'src/data-loss-expectations.ts', 'column-descriptors': 'src/column-descriptors.ts', 'lowered-params': 'src/lowered-params.ts', 'operation-descriptors': 'src/operation-descriptors.ts', diff --git a/upgrade-instructions/pending/migration-statement-refusal/app/instructions.md b/upgrade-instructions/pending/migration-statement-refusal/app/instructions.md new file mode 100644 index 000000000000..9c404462c6dd --- /dev/null +++ b/upgrade-instructions/pending/migration-statement-refusal/app/instructions.md @@ -0,0 +1,101 @@ +--- +changes: + - id: destructive-means-data-loss + summary: | + An operation is `destructive` only when it can lose rows or values. Postgres `setNotNull` and the type changes that keep every value (`int2` to `int4` to `int8`, `float4` to `float8`), a SQLite table rebuild that only changes nullability, and MongoDB `dropIndex`, `setValidation`, `collMod` and the planner's validator and change-stream image changes are now `widening`. Running an existing `migration.ts` again writes `widening` for these operations in `ops.json`, and a new `migrationHash`. + detection: + glob: "**/migration.ts" + matches: + - '\bsetNotNull\(' + - '(?` lets the data go, `--rename :` keeps it. Where nobody can answer it fails with `CLI.CONSENT_REQUIRED`, whose `nextActions` name the flags. `--confirm` answers none of these questions. + detection: + glob: "**/*.{sh,bash,zsh,yml,yaml,json,toml,mjs,cjs,js,ts,mts,cts}" + matches: + - '(?` no longer consents to data loss. An apply asks about each operation that would lose data, answered by `--delete ` or `--rename`, and about each that would widen who can read or write a model's rows, answered by `--allow `. Without them it fails with `CLI.CONSENT_REQUIRED` where nobody can answer. + detection: + glob: "**/*.{sh,bash,zsh,yml,yaml,json,toml,mjs,cjs,js,ts,mts,cts}" + matches: + - 'db\s+update(?:[^\n\\]|\\\r?\n|\\.)*(?'` (a value with characters a shell would read is single-quoted). Add the flags you mean to that run. A subject no model of the starting contract stores is named by its storage name and can only be deleted. `--rename` is offered only for a model or field the new contract no longer has, so a field whose type changes is answered with `--delete`, and on MongoDB, whose planner refuses renames in this release, only `--delete` is offered; the question then says how to keep the documents by renaming the collection in `mongosh` before a plan that drops it is applied, and that a migration written by `migration plan` still drops it. A field of a model the plan renames is named through the model's new name: `--rename Profile:User --delete User.nickname`. Do not keep a `--confirm ` for this: it answers nothing. + +## `db-update-confirm-no-longer-consents` + +Detection finds `--confirm` on the same command line as `db update`, including a command continued over several lines with `\`. For each `prisma db update ... --confirm `, replace `--confirm ` with one statement per operation it was there to allow: + +- `--delete ` or `--delete ` for each model or field whose data the update may lose, or `--rename :` / `--rename :` if it was renamed. +- `--delete ` for data no model stores, such as a table or column added by hand; on Postgres the name is schema-qualified (`public.audit_log`). +- `--allow ` before each operation that changes who can read or write that model's rows, such as dropping its row-level-security policy or disabling row-level security; each operation is its own question, so both on one model take `--allow User --allow User`. + +To find the subjects, run the same command with `--no-interactive --json` and read `meta.unanswered[]`, or `--dry-run --json` and read `dataLoss` and `accessWidening`. A `--delete` or `--allow` that answers no question fails the run with `CLI.CONSENT_UNUSED`; on `--dry-run`, which asks nothing, it fails with `MIGRATION.STATEMENT_ANSWERS_NO_QUESTION`, so the exact command you will apply can be previewed with `--dry-run` first. + +## `db-update-reads-origin-snapshot` + +`prisma db update` now reads the snapshot of the contract the database's marker names on every run, not only when `--rename` is given, so it can name what an operation would lose by model and field. Without a snapshot (a database last updated with `--db ` and no `--advance-ref`), each subject is its storage name, and the question says so. The planned operations are the same either way. + +## `control-api-answer-questions` + +These come from `@prisma/orm-toolchain/cli/control-api`. + +- `executeDbUpdate(options)` and the control client's `dbUpdate(options)` require `answerQuestions`, typed `AnswerPlanQuestions` (with `PlanQuestion`, `PlanAnswer` and `PlanQuestionVerb`, all exported from `@prisma/orm-toolchain/cli/control-api`). Before an apply it is called with every question no statement answered, at least once, with an empty list when nothing is in question, and also under `acceptDataLoss: true`. Return one `{ verb, text }` per question, in order, with a verb from `question.verbs` and `question.subject` as the text (or `:` for a rename); throw to refuse. A callback that returns too few answers, or an answer its question rejects, throws an `InternalError`. To keep a script that refuses every data loss, write `answerQuestions: async (questions) => { if (questions.length > 0) throw new Error('db update would lose data or widen access'); return []; }`. +- `acceptDataLoss: true` answers every data-loss question, and `acceptAccessWidening: true` every access-widening question. A caller that passed `acceptDataLoss: true` to apply whatever the plan holds passes both. +- `delete` and `allow` statements in `statements` (`{ verb: 'delete', text: 'Legacy' }`) answer their questions without asking. A `delete` answers every loss of its subject; an `allow` answers one operation, in order, so two access changes on one model take two `allow` statements. One that answers no question fails with `MIGRATION.STATEMENT_ANSWERS_NO_QUESTION`, in plan mode as in apply mode. +- `consent` is gone from `executeDbUpdate`'s and `dbUpdate`'s options: the plan-hash binding it carried no longer exists. Remove it. +- `executeMigrationPlanCommand(options)` requires `answerQuestions`, with the same contract, and loses `consent` and `carryEmittedExtensionDirs`. Remove both. +- `DbUpdateSuccess` gains `dataLoss` and `accessWidening`, the operations of the plan that lose data or widen access, by position, each an `AskedSubject` with the subject's `text` as the question writes it. `AskedSubject`, `MigrationSubject`, `MigrationOperationSubject` and `MigrationPlanSubjects` are exported from `@prisma/orm-toolchain/cli/control-api`. + +## `applied-statement-report-verb` + +`AppliedStatementReport` is now a union on `verb`. A `rename` entry keeps its `statement` as before; a `delete` or `allow` entry's `statement` is `{ kind: 'delete' | 'allow', subject }`, where `subject` is a model, a field, or `{ kind: 'storage', name }`. Code that reads `report.statement.from` or `.to` narrows on `report.verb === 'rename'` first. An exhaustive `switch` over `StatementVerb` gains `case 'delete':` and `case 'allow':`. + +## `consent-errors-removed` + +Remove imports of `ERROR_CODE_DESTRUCTIVE_CHANGES`, `errorDestructiveChanges`, `ERROR_CODE_CONSENT_PLAN_MISMATCH` and `errorConsentPlanMismatch` from `@prisma/orm-framework/errors/execution`; nothing raises those errors any more. Code that checks a `db update` failure for `code === 'DESTRUCTIVE_CHANGES'` or `'CONSENT_PLAN_MISMATCH'`, or reads `failure.destructiveChanges` or `failure.consentPlanMismatch`, handles `CLI.CONSENT_REQUIRED` from the CLI instead, or, through the control API, the refusal its own `answerQuestions` throws. A script that matches `MIGRATION.DESTRUCTIVE_CHANGES`, `MIGRATION.CONSENT_PLAN_MISMATCH`, `CLI.CONSENT_TOKEN_UNRESOLVED` or `CLI.CONSENT_OPERATIONS_MISSING` in CLI output matches `CLI.CONSENT_REQUIRED` instead. diff --git a/upgrade-instructions/pending/migration-statement-refusal/extension/instructions.md b/upgrade-instructions/pending/migration-statement-refusal/extension/instructions.md new file mode 100644 index 000000000000..f0face894f3a --- /dev/null +++ b/upgrade-instructions/pending/migration-statement-refusal/extension/instructions.md @@ -0,0 +1,103 @@ +--- +changes: + - id: planner-success-data-loss + summary: | + A migration planner's success result gains required `dataLoss` and `accessWidening` lists of `MigrationOperationSubject` (from `@prisma/orm-framework/components/control`): the operations that lose data, and those that widen who can read or write rows, each by position with its subject. A planner, or a test double of one, that returns `{ kind: 'success', ... }` adds both, empty when it plans neither. + detection: + glob: "**/*.{ts,mts,cts}" + matches: + - '(? operation.id` in a test. The aggregate planner fills the lists from the planner's result for a space it plans from a diff, and for a space it applies from recorded migrations it lists each destructive operation in `dataLoss` under its storage name, and no access widening, since a written migration is reviewed before it runs. + +## `family-instance-storage-name-of` + +A `ControlFamilyInstance` implementation adds `storageNameOf(operation: MigrationPlanOperation): string`. The aggregate planner calls it to name what a destructive operation of a recorded migration loses, since no planner mapped it to a model. A SQL family returns the name from the operation's target details, `schema.table.column` for a column and `schema.name` for anything else; `storageNameOfOperation` from `@prisma/orm-family-sql/family/control` does that. A test double returns any stable name. + +A target whose planner carries out no rename statement sets `renameStatements: { refused: true, keepDataByHand(subject, fromContract) }` on its `migrations` capability. The CLI then offers no `--rename` in a data-loss question, and ends the question with the text `keepDataByHand` returns: how to keep the subject's data by hand before running the command again. MongoDB's target sets it in this release, and its text says to rename the collection in `mongosh` before a plan that drops it is applied, and that a migration written by `migration plan` still drops it. + +## `operation-classes-and-calls` + +An operation is `destructive` only when it can lose rows or values. Postgres `setNotNull`, and MongoDB `dropIndex`, `setValidation` and `collMod` (with no `operationClass` given), are now `widening`, and so are the op-factory calls a planner builds for them. A planner or test that asserts `operationClass: 'destructive'` for them asserts `'widening'`. + +`AlterColumnTypeCall` takes an optional last constructor argument, `operationClass: 'widening' | 'destructive'`, default `'destructive'`; pass `'widening'` only when every value of the old type converts to the new type unchanged. `RecreateTableCall` takes an optional second constructor argument, `lossyColumns`: the columns whose values the copy can change because their type changes. A planner that builds a recreate for a type change passes them, so `dataLoss` names those fields; they never reach the operation or `migration.ts`. + +## `control-client-db-update-answer-questions` + +In code and documentation that call the control client's `dbUpdate({ ... })` without `answerQuestions`, add a callback. Detection finds files that call `dbUpdate(` and never mention `answerQuestions`; a file with one call that has it and another that lacks it is skipped, so check it by hand. Pass the options `dbUpdate` takes: `contract` (the emitted `contract.json`), `mode` and `migrationsDir`. In a README example that has none of them, add `import contract from './src/prisma/contract.json' with { type: 'json' };` after the control client's import, and write the call with a callback that refuses every data loss and access widening, as `dbUpdate` used to without consent: + +```typescript +await control.dbUpdate({ + contract, + mode: 'apply', + migrationsDir: 'migrations', + answerQuestions: async (questions) => { + if (questions.length > 0) throw new Error('db update would lose data or widen access'); + return []; + }, +}); +``` + +The callback is called at least once per apply, with an empty list when nothing is in question; return `[]` then. To consent, return one `{ verb, text }` per question, in order: a verb from `question.verbs` and `question.subject` as the text.