From ad3c17132c44598ac6c376812fd61daa9e9844a3 Mon Sep 17 00:00:00 2001 From: Joris Wouter Jonkers Date: Tue, 27 Oct 2026 14:51:36 +0100 Subject: [PATCH] feat(model): refuse a fragment written at a model version the toolkit does not read Composition admitted every fragment whatever schemaVersion it was written at, so a fragment at a newer minor composed with the fields the toolkit could not read dropped, and the run exited zero. A participant whose fragment is at another major, or at a newer minor than the toolkit's, is now E_SCHEMA_VERSION_MISMATCH: isolated at the fragment it last composed at, dormant or not, and a Platform document at such a version stops the run. A patch difference never gates. Where the fragment a Project would be held or rolled back at is itself one the toolkit cannot read, the run stops too, since composing it would drop the same fields. The toolkit's model version is release-please's to set, beside package.json, so the range moves with every release. Closes #665 --- docs/requirements.md | 2 +- release-please-config.json | 1 + scripts/lint-codes.ts | 1 - spec/v1/40-composition.md | 11 +- src/application/compose-estate.ts | 104 ++++++++++++++---- src/cli/main.ts | 4 +- test/application/compose.test.ts | 169 ++++++++++++++++++++++++++++++ test/cli/main.test.ts | 9 +- 8 files changed, 274 insertions(+), 27 deletions(-) diff --git a/docs/requirements.md b/docs/requirements.md index 12492b5d8..216fc05dd 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -72,7 +72,7 @@ This ledger holds **115** rows. The compiler's behaviours join it as they land. | REQ-047 | A fragment's share of the images lock holds the aliases its project file names and no other, refusing one the lock it is handed does not hold, and an entry that records no version at its alias; composition unions every share into the one lock resolution reads, taking an alias locked to the same image as one entry and refusing one locked differently at the share of the fragment that changed, which isolates that Project | [test/application/images-lock-shares.test.ts](../test/application/images-lock-shares.test.ts) | | REQ-048 | A Process that declares Kubernetes API access holds it only where the Platform document admits its Application: an admitted one renders a ClusterRole of its declared rules bound to its one ServiceAccount, a mounted token and egress to where the API answers, and the delivery machinery it is declared for renders, rolling and ungated, to its committed tree; a declaration nobody admitted is refused, and so is a holder no project file declares | [test/adapters/spelling.test.ts](../test/adapters/spelling.test.ts) | | REQ-049 | The Vault policy job is rendered where the Platform document names it, a Secret Store answers and the render holds a document to write: one per Project that holds a document: a ServiceAccount, a ConfigMap of exactly that Project's documents and a Job, in that Project's own share and namespace, the ConfigMap and the Job named by the digest of those documents and what the pod runs with, with a policy of its own | [test/model/render.test.ts](../test/model/render.test.ts) | -| REQ-050 | Composition reads the participants list: a participant that published nothing, or nothing within its age, is held at the fragment it last composed at, a fragment the list does not name is left out, and each is recorded in the lock with its code; a Platform document that is missing, stale or unlisted, and a dormant entry past its review, stop the run | [test/application/compose.test.ts](../test/application/compose.test.ts) | +| REQ-050 | Composition reads the participants list: a participant that published nothing, or nothing within its age, or whose fragment was written at a model version the toolkit does not accept (another major, or a newer minor), is held at the fragment it last composed at, a fragment the list does not name is left out, and each is recorded in the lock with its code; a Platform document that is missing, stale, unlisted or written at such a version, a fragment a Project would be held or rolled back at that is written at such a version, and a dormant entry past its review, stop the run | [test/application/compose.test.ts](../test/application/compose.test.ts) | | REQ-051 | Composition returns the pin source of every artifact whose pin moves, one it was handed no pin for among them: an `OCIRepository` naming the artifact by a digest that names none until it is published, verified keyless against the Platform document's signer matched whole, and one Kustomization per Reconcile Unit the artifact holds, each after the units it follows, and, on a Project's first delivery alone, a patch marking every Deployment for recreation, and, on every unit that carries a Rollout, patches labelling each pod template and Job it applies with the Rollout's version; an artifact whose pin stays has one written only where its committed source holds other objects than the one derived, however either is written, and then at the digest it is committed at, a paused Project's with its units as committed | [test/application/compose.test.ts](../test/application/compose.test.ts) | | REQ-052 | An Application that moves its schema with a changelog renders its migration identity and two Jobs named by the revision's tag, suspended and created once: the up, and the down to the tag of the revision the release was proven against, none on a first release; the identity holds the owner credential under a policy and role of its own, and the datastore admits it. A count above one renders a disruption budget, and for a blue-green Process an autoscaler with equal bounds the Canary names; auth's share and its estate-scoped share equal their committed tree byte for byte | [test/model/render.test.ts](../test/model/render.test.ts) | | REQ-053 | A file grant of one key renders as a projected Secret volume carrying that key as the file at `mountAt`, at `fileMode`, its directory mounted read-only into the Process and every sidecar and nothing of it in the environment; a file grant of other than one key, with no mode, or directly in `/` stops the render. The knowledge platform's share and its estate-scoped share equal their committed tree byte for byte | [test/model/render.test.ts](../test/model/render.test.ts) | diff --git a/release-please-config.json b/release-please-config.json index 0b27decb6..83969b8fe 100644 --- a/release-please-config.json +++ b/release-please-config.json @@ -4,6 +4,7 @@ ".": { "release-type": "node", "changelog-path": "CHANGELOG.md", + "extra-files": ["src/cli/main.ts"], "include-component-in-tag": false, "exclude-paths": ["emf"], "bump-minor-pre-major": true, diff --git a/scripts/lint-codes.ts b/scripts/lint-codes.ts index a2c812841..d1528600d 100644 --- a/scripts/lint-codes.ts +++ b/scripts/lint-codes.ts @@ -85,7 +85,6 @@ export const PENDING: readonly Pending[] = [ "E_RAW_SECRET", "E_READER_NOT_DECLARED", "E_ROLL_AFFECTS_OTHER_READERS", - "E_SCHEMA_VERSION_MISMATCH", "E_UNDECLARED_SECRET_PATH", ], }, diff --git a/spec/v1/40-composition.md b/spec/v1/40-composition.md index 64083afc6..50153c6de 100644 --- a/spec/v1/40-composition.md +++ b/spec/v1/40-composition.md @@ -226,9 +226,14 @@ composes as if nothing happened. participant (`E_PARTICIPANT_MISSING`, `E_PARTICIPANT_STALE`), and one whose `schemaVersion` the toolkit does not accept (`E_SCHEMA_VERSION_MISMATCH`), is isolated at its last composed fragment even though its digest did not change: - the error is about that participant alone, so it is the one held back. A - toolkit upgrade that no participant's version accepts therefore holds every - Project still and fails nothing, until each republishes. + the error is about that participant alone, so it is the one held back. Where + the fragment it would be held at, or the one a Rollback names, is itself one + the toolkit does not accept, nothing is left to hold it at, and composing it + would drop what the toolkit cannot read: the run stops with + `E_SCHEMA_VERSION_MISMATCH`, naming the Project and that fragment, until the + fragment is republished or the toolkit reads it. A toolkit upgrade a major + ahead of the fragments therefore stops the run, which the + [version rollout](#version-rollout) avoids by republishing first. - **A fragment nobody expected is left out.** A fragment whose project the participants list does not name is `E_PARTICIPANT_UNLISTED`: it is isolated and not composed at all, whether or not its Project composed before. Nobody diff --git a/src/application/compose-estate.ts b/src/application/compose-estate.ts index d19c430b5..fdee94d03 100644 --- a/src/application/compose-estate.ts +++ b/src/application/compose-estate.ts @@ -191,6 +191,33 @@ const PROJECT_FILE = ".project.yml"; interface Isolation { readonly refused: Fragment | undefined; readonly codes: readonly string[]; + /** Why its fragment was written at a version the toolkit refuses, where it was. */ + readonly written?: string | undefined; +} + +/** What holds a participant back before the union composes, and why. */ +interface Standing { + readonly code: string; + readonly written?: string; +} + +/** + * Whether the toolkit reads the model version a fragment was written at: a + * range, not equality (spec/v1/40-composition.md#versioning). Another major, + * either way, or a newer minor is refused, since an older toolkit would drop + * what it cannot read and exit zero; a patch never gates. + */ +function versionRefusal(written: string, toolkit: string): string | undefined { + const [major, minor] = written.split(".").map(Number) as [number, number]; + const [ownMajor, ownMinor] = toolkit.split(".").map(Number) as [ + number, + number, + ]; + if (major !== ownMajor) + return `written at ${written}, a different major than this toolkit's ${toolkit}`; + return minor > ownMinor + ? `written at ${written}, a newer minor than this toolkit's ${toolkit}` + : undefined; } /** A composition whose Platform document is there to compose with. */ @@ -238,15 +265,21 @@ function declaredBy({ files }: Fragment): string[] { */ function standingOf( input: ComposeInput, - generatedAt: string, + options: Pick, project: string, fragment: Fragment | undefined, -): string | undefined { +): Standing | undefined { const entry = own(input.participants.participants, project); - if (entry === undefined) return "E_PARTICIPANT_UNLISTED"; - if (fragment === undefined) return "E_PARTICIPANT_MISSING"; + if (entry === undefined) return { code: "E_PARTICIPANT_UNLISTED" }; + if (fragment === undefined) return { code: "E_PARTICIPANT_MISSING" }; if (declaredBy(fragment).some((declared) => declared !== project)) - return "E_PARTICIPANT_UNLISTED"; + return { code: "E_PARTICIPANT_UNLISTED" }; + const version = versionRefusal( + fragment.manifest.spec.schemaVersion, + options.schemaVersion, + ); + if (version !== undefined) + return { code: "E_SCHEMA_VERSION_MISMATCH", written: version }; // Dormant exempts a participant from its age, and from nothing else. if (entry.dormant === true) return undefined; const at = own(input.published, project); @@ -255,39 +288,71 @@ function standingOf( `${project}: a fragment whose publish time the caller did not supply`, ); const days = Number((entry.maxAge ?? DEFAULT_MAX_AGE).slice(0, -1)); - return Date.parse(generatedAt) - Date.parse(at) > days * DAY_MS - ? "E_PARTICIPANT_STALE" + return Date.parse(options.generatedAt) - Date.parse(at) > days * DAY_MS + ? { code: "E_PARTICIPANT_STALE" } : undefined; } const PARTICIPANTS = "participants.yml"; /** What stops the run before anything composes: the list's own entries, and the platform's standing. */ -function listRefusals(input: ComposeInput, generatedAt: string): Diagnostic[] { +function listRefusals( + input: ComposeInput, + options: ComposeOptions, +): Diagnostic[] { const platform = input.platform; const project = platform?.manifest.spec.project; const standing = project === undefined - ? "E_PARTICIPANT_MISSING" - : standingOf(input, generatedAt, project, platform); + ? { code: "E_PARTICIPANT_MISSING" } + : standingOf(input, options, project, platform); return [ ...(standing === undefined ? [] : [ { - code: standing, + code: standing.code, document: PARTICIPANTS, path: "/participants", message: project === undefined ? "the Platform document published no fragment" - : `the Platform document's fragment, ${project}, cannot be composed with: ${standing}`, + : `the Platform document's fragment, ${project}, cannot be composed with: ${standing.code}${standing.written === undefined ? "" : `: ${(platform as Fragment).ref} was ${standing.written}`}`, hint: "No render is composed without a current, listed Platform document: publish it, or correct its entry.", } as Diagnostic, ]), ]; } +/** + * A fragment the union would compose that the toolkit cannot read: the one a + * Project is held at, or a Rollback's. Nothing is left to hold it at, and + * composing it would drop what the toolkit cannot read, so the run stops + * (spec/v1/40-composition.md#versioning). + */ +function unreadable( + candidates: ReadonlyMap, + schemaVersion: string, +): Diagnostic[] { + return [...candidates].flatMap(([project, fragment]) => { + const written = versionRefusal( + fragment.manifest.spec.schemaVersion, + schemaVersion, + ); + return written === undefined + ? [] + : [ + { + code: "E_SCHEMA_VERSION_MISMATCH", + document: PARTICIPANTS, + path: "/participants", + message: `${project} can be composed only at ${fragment.ref}, which was ${written}`, + hint: "Republish the fragment at a model version this toolkit reads, or compose with a toolkit that reads it.", + }, + ]; + }); +} + const placed = (directory: string, files: readonly AuthoredFile[]) => files.map(({ name, text }) => ({ name: `${directory}/${name}`, text })); @@ -389,7 +454,7 @@ export function composeEstate( ): Result { // A Platform document that is missing, stale or unlisted stops the run: // there is nothing to isolate. - const refusals = listRefusals(input, options.generatedAt); + const refusals = listRefusals(input, options); return refusals.length > 0 ? { ok: false, diagnostics: refusals } : composed(input as Composing, options); @@ -437,9 +502,10 @@ function composed( ); for (const project of [...new Set([...expected, ...newest.keys()])]) { const fragment = newest.get(project); - const code = standingOf(input, options.generatedAt, project, fragment); - if (code === undefined) continue; - isolated.set(project, { refused: fragment, codes: [code] }); + const standing = standingOf(input, options, project, fragment); + if (standing === undefined) continue; + const { code, written } = standing; + isolated.set(project, { refused: fragment, codes: [code], written }); const earlier = previous[project]?.ref; if (code === "E_PARTICIPANT_UNLISTED" || earlier === undefined) candidates.delete(project); @@ -452,6 +518,8 @@ function composed( } for (;;) { + const stops = unreadable(candidates, options.schemaVersion); + if (stops.length > 0) return { ok: false, diagnostics: stops }; const files = [ ...placed(PLATFORM, input.platform.files), { name: `${PLATFORM}/${CLUSTER_STATE}`, text: input.clusterState.text }, @@ -820,7 +888,7 @@ function conditionsOf( ): Condition[] { const conditions: Condition[] = [...isolated] .sort(([a], [b]) => byText(a, b)) - .map(([project, { refused, codes }]): Condition => { + .map(([project, { refused, codes, written }]): Condition => { if (refused === undefined) return { project, @@ -834,7 +902,7 @@ function conditionsOf( project, condition: "isolated", title: `${project}: ${version} isolated`, - body: `The fragment ${refused.ref} was refused with ${named(codes)}. ${project} stays at the fragment it last composed at, and its pin does not move until a release composes.`, + body: `The fragment ${refused.ref} was refused with ${named(codes)}${written === undefined ? "" : `: it was ${written}`}. ${project} stays at the fragment it last composed at, and its pin does not move until a release composes.`, discord: `${project} ${version} isolated: ${named(codes)}`, }; }); diff --git a/src/cli/main.ts b/src/cli/main.ts index 76d55b7a0..ce05b08ca 100644 --- a/src/cli/main.ts +++ b/src/cli/main.ts @@ -60,8 +60,8 @@ export interface World { */ export const EXIT = { accepted: 0, refused: 1, usage: 2 } as const; -/** The model version this toolkit writes. */ -const SCHEMA_VERSION = "1.0.0"; +/** The model version this toolkit writes and admits up to: the package's own, set by release-please. */ +export const SCHEMA_VERSION = "1.0.0"; // x-release-please-version const MANIFEST = "fragment.yml"; const PLATFORM_DOCUMENT = "platform.intent.yml"; const PROOF = "migration-proof.yml"; diff --git a/test/application/compose.test.ts b/test/application/compose.test.ts index 4286c4a00..d489489ae 100644 --- a/test/application/compose.test.ts +++ b/test/application/compose.test.ts @@ -1820,6 +1820,175 @@ describe("composeEstate, against the participants list", () => { ]); }); + describe("by the model version each fragment was written at", () => { + // Another manifest is another artifact, so another digest. + const writtenAt = (schemaVersion: string, fragment = NOTES): Fragment => ({ + ...fragment, + ref: `${fragment.ref.split("@")[0] as string}@sha256:${hex({ ref: fragment.ref, schemaVersion }, 64)}`, + manifest: { + ...fragment.manifest, + spec: { ...fragment.manifest.spec, schemaVersion }, + }, + }); + const withNotes = (fragment: Fragment) => ({ + fragments: [fragment, ...others], + }); + + it("holds a fragment written at a newer minor than the toolkit's, at the one it last composed at", () => { + const newer = writtenAt("1.1.0", release("notes", "1.1.0")); + const composed = after(withNotes(newer)); + + expect(notes(composed)).toStrictEqual({ + locked: NOTES.ref, + isolated: { refused: newer.ref, codes: ["E_SCHEMA_VERSION_MISMATCH"] }, + moves: false, + }); + expect( + composed.conditions.find(({ project }) => project === "notes")?.body, + ).toBe( + `The fragment ${newer.ref} was refused with E_SCHEMA_VERSION_MISMATCH: it was written at 1.1.0, a newer minor than this toolkit's 1.0.0. notes stays at the fragment it last composed at, and its pin does not move until a release composes.`, + ); + expect( + composed.statuses.find(({ repository }) => + repository.endsWith("/notes"), + )?.description, + ).toBe("notes 1.1.0 isolated: E_SCHEMA_VERSION_MISMATCH"); + }); + + it("holds a fragment written at another major, in either direction", () => { + for (const version of ["2.0.0", "0.9.0"]) { + const other = writtenAt(version); + expect( + after(withNotes(other)).conditions.find( + ({ project }) => project === "notes", + )?.body, + ).toBe( + `The fragment ${other.ref} was refused with E_SCHEMA_VERSION_MISMATCH: it was written at ${version}, a different major than this toolkit's 1.0.0. notes stays at the fragment it last composed at, and its pin does not move until a release composes.`, + ); + } + }); + + it("admits a fragment written at an older minor or another patch", () => { + const toolkit = { ...OPTIONS, schemaVersion: "1.2.3" }; + for (const version of ["1.0.0", "1.2.0", "1.2.9"]) { + const result = composeEstate( + { + ...EXPECTED, + platform: writtenAt("1.2.3", PLATFORM), + fragments: [writtenAt(version), ...others], + held: [], + pins: {}, + clusterState: { + name: "cluster-state.yml", + text: text("platform/cluster-state.yml"), + }, + }, + toolkit, + ); + expect(result.ok && notes(result.value).isolated).toBeUndefined(); + } + }); + + it("holds a dormant participant's fragment too: dormancy exempts its age alone", () => { + expect( + notes( + after({ + ...withNotes(writtenAt("1.1.0")), + ...listing({ dormant: true, reason: "archived" }), + }), + ).isolated?.codes, + ).toStrictEqual(["E_SCHEMA_VERSION_MISMATCH"]); + }); + + // The fragment a Project would be held at is one the toolkit cannot + // read either: composing it would drop what the toolkit cannot read. + const unreadable = (newer: Fragment) => + `E_SCHEMA_VERSION_MISMATCH participants.yml /participants notes can be composed only at ${newer.ref}, which was written at 1.1.0, a newer minor than this toolkit's 1.0.0`; + + it("stops the run where the fragment a Project last composed at is one the toolkit cannot read", () => { + const newer = writtenAt("1.1.0", release("notes", "1.1.0")); + const input = { + ...EXPECTED, + platform: PLATFORM, + fragments: [newer, ...others], + held: [...RELEASES, newer], + pins: {}, + clusterState: { + name: "cluster-state.yml", + text: text("platform/cluster-state.yml"), + }, + }; + const earlier = composeEstate(input, { + ...OPTIONS, + schemaVersion: "1.1.0", + }); + if (!earlier.ok) throw new Error("the 1.1.0 toolkit composes"); + + expect( + refused({ + ...input, + previous: { + lock: earlier.value.lock, + commit: hex("estate-commit", 40), + }, + }), + ).toStrictEqual([unreadable(newer)]); + }); + + it("stops the run where a Rollback names a held fragment the toolkit cannot read", () => { + const newer = writtenAt("1.1.0", release("notes", "1.1.0")); + expect( + refused({ + held: [...RELEASES, newer], + pins: pinned(FIRST, { + notes: { + ...PAUSE, + [PIN_ANNOTATIONS.rollbackVersion]: "1.1.0", + [PIN_ANNOTATIONS.rollbackFragment]: newer.ref, + }, + }), + previous: { lock: FIRST.lock, commit: hex("estate-commit", 40) }, + }), + ).toStrictEqual([unreadable(newer)]); + }); + + it("says how to clear a fragment the toolkit cannot read", () => { + const newer = writtenAt("1.1.0", release("notes", "1.1.0")); + const result = composeEstate( + { + ...EXPECTED, + platform: PLATFORM, + fragments: RELEASES, + held: [...RELEASES, newer], + pins: pinned(FIRST, { + notes: { + ...PAUSE, + [PIN_ANNOTATIONS.rollbackVersion]: "1.1.0", + [PIN_ANNOTATIONS.rollbackFragment]: newer.ref, + }, + }), + clusterState: { + name: "cluster-state.yml", + text: text("platform/cluster-state.yml"), + }, + }, + OPTIONS, + ); + expect( + result.ok ? [] : result.diagnostics.map(({ hint }) => hint), + ).toStrictEqual([ + "Republish the fragment at a model version this toolkit reads, or compose with a toolkit that reads it.", + ]); + }); + + it("stops the run on a Platform document written at a version the toolkit does not accept, naming the rule and the fragment", () => { + const platform = writtenAt("1.1.0", PLATFORM); + expect(refused({ platform })).toStrictEqual([ + `E_SCHEMA_VERSION_MISMATCH participants.yml /participants the Platform document's fragment, jorisjonkers.dev, cannot be composed with: E_SCHEMA_VERSION_MISMATCH: ${platform.ref} was written at 1.1.0, a newer minor than this toolkit's 1.0.0`, + ]); + }); + }); + it("asks for the publish time of every fragment whose age it reads", () => { expect(() => compose({ diff --git a/test/cli/main.test.ts b/test/cli/main.test.ts index c29360324..62c95d2ce 100644 --- a/test/cli/main.test.ts +++ b/test/cli/main.test.ts @@ -16,7 +16,12 @@ import { import { join } from "node:path"; import { describe, expect, it } from "vitest"; import { parse } from "yaml"; -import { main, USAGE_TEXT, type World } from "../../src/cli/main.ts"; +import { + main, + SCHEMA_VERSION, + USAGE_TEXT, + type World, +} from "../../src/cli/main.ts"; import { sha256Hasher } from "../../src/index.ts"; import { temporary } from "../setup.ts"; @@ -547,7 +552,7 @@ describe("deploy-kit compose", () => { }; expect(lock.metadata.generatedAt).toBe("2026-10-02T09:00:00.000Z"); expect(lock.spec.toolkitVersion).toBe("0.2.0"); - expect(lock.spec.schemaVersion).toBe("1.0.0"); + expect(lock.spec.schemaVersion).toBe(SCHEMA_VERSION); const report = JSON.parse( readFileSync(join(root, "first", "composition.json"), "utf8"), ) as {