diff --git a/.changeset/react-declaration-parity-cannot-run-is-not-a-skip.md b/.changeset/react-declaration-parity-cannot-run-is-not-a-skip.md new file mode 100644 index 0000000000..0ec2e2a178 --- /dev/null +++ b/.changeset/react-declaration-parity-cannot-run-is-not-a-skip.md @@ -0,0 +1,15 @@ +--- +'@objectstack/spec': patch +--- + +`check:react-declaration-parity` had no path on which it could go red — a missing manifest now fails instead of skipping, and the ledger says "cannot run here" instead of "deliberately not run". + +Two things stacked. The gate was in **no workflow** (`grep -rl check:react-declaration-parity .github/workflows/` returned nothing), and a manual run without `MANIFEST` printed `⚠ manifest unavailable … — skipping.` and **exited 0**. So the one gate whose history is written up in AGENTS.md as worth keeping — `spec-only` / `registry-only` / `missing` are real signals — could not fail for anybody, ever. Two other issues had already started citing it as the negative example of a gate that cannot be shown to catch anything (#4804 / #4777). + +**Where the manifest comes from, measured before anything was wired.** The right-hand side is objectui's `sdui.manifest.json`, and this repository cannot produce one: its only producer drives a real browser at objectui's built console and reads `window.__MANIFEST` (the registry pulls browser-only deps, so nothing enumerates it from Node). `packages/console/dist/` is gitignored, `scripts/build-console.sh` deliberately does not dump one — it must not drag a browser into the console build — and the published `@objectstack/console` tarball contains no `sdui.manifest.json` either (16.1.0: 513 files, zero `sdui` matches; its `dist/manifest.json` is the PWA manifest, so even the CLI's `@objectstack/console/dist/sdui.manifest.json` fallback resolves to nothing). Wiring a step into `lint.yml` would therefore have wired a **permanently skipping** step — the same defect with CI decoration. + +- **"Could not run" is now a failure, not a skip.** No `MANIFEST`, a path that does not exist, malformed JSON, or a dump declaring zero components each exit **1** with a prescription that names the producer (`pnpm sdui:manifest`, `OBJECTUI_ROOT=../objectui pnpm objectui:build`) and the browser it needs. Deliberately independent of `--strict`: that flag prices a *divergence*, and this is the other thing entirely — no comparison happened. The empty-dump case is new coverage in the same family; objectui's dumper already refuses to *write* one, and this refuses to *read* one instead of reporting every block as missing. +- **The ledger stops implying someone runs it.** `check:generated` moves it out of `NO_GENERATOR` ("runnable, deliberately not run here") into a new `EXTERNAL_INPUT_REQUIRED` bucket that records the missing input and its producer, and prints `cannot run here: check:react-declaration-parity — needs MANIFEST=…; runs in scripts/gen-sdui-manifest.sh`. Following `EXPLICIT_GENERATORS` (#5358/#5807), the classification carries an enforced claim rather than a label: the reconciliation fails if `scripts/gen-sdui-manifest.sh` stops **invoking** the gate on a non-comment line — a gate filed as "runs elsewhere" while running nowhere is the exact hole this bucket exists to expose. +- **The gate is now demonstrably able to fail.** `check-react-blocks-declaration-parity.test.ts` asserts exit codes, not just report text: a fabricated registry-only input and a vanished block each exit non-zero **naming themselves**, all four "could not run" paths exit non-zero, and an accepted state still exits 0 so the red is discriminating. The pre-existing helper swallowed exit codes by design, so every earlier test would have passed against a script that always exited 0 — which is what this gate was. + +Where the manifest should come from in CI (an objectui clone plus a browser in this repo's workflows, a published manifest artifact, or a Node-side dump in objectui) stays an open provenance decision, filed separately. Until it is answered, the honest state is a gate that runs at `pnpm sdui:manifest` and refuses to pretend otherwise. diff --git a/AGENTS.md b/AGENTS.md index 07d51be611..4977de5b5c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -415,8 +415,8 @@ believe it, and before you file a bug about `main` being red. (Two phantom "brea removals" this way while writing this section; `check:generated` now prints this caveat inline when that gate is the one failing.) -`check:liveness`, `check:empty-state`, `check:skill-examples`, -`check:react-declaration-parity`, `check:exported-any` and `check:dual-source-exports` are +`check:liveness`, `check:empty-state`, `check:skill-examples`, `check:exported-any` and +`check:dual-source-exports` are pure checks with no generator — a failure there is a real finding to fix, not an artifact to regenerate. `check:generated` names them as deliberately not run, so its "all up to date" never reads as "everything passed". The last one asks the third question about the @@ -436,6 +436,22 @@ and #4413 shipped four dead blocks straight through a green run of it. Renamed a re-scoped in #4472. The gate is still worth having (`spec-only`, `registry-only` and `missing` are real signals) — just don't read it as proof anything renders. +⚠️ **It is also the one gate `check:generated` cannot run at all**, and it says so in its +own bucket (`EXTERNAL_INPUT_REQUIRED`, "cannot run here") rather than beside the source +audits that are merely *deliberately* not run. Its right-hand side is objectui's +`sdui.manifest.json`, and nothing here can produce one: the registry is a browser app, so +the manifest exists only after `pnpm sdui:manifest` builds objectui at `.objectui-sha` and +enumerates it in a real browser — `packages/console/dist/` is gitignored, the console +build deliberately does not dump one, and the published `@objectstack/console` carries +none either. Until #4690 that combined with a manual run that printed `⚠ manifest +unavailable` and **exited 0**, so no path existed on which this gate could go red; it now +**exits 1** when it has no usable manifest, because "could not run" is a failure, not a +skip (Route & surface ownership §3, *Absence must be loud*). Run it the one way that +works: `pnpm sdui:manifest` (or `OBJECTUI_ROOT=../objectui pnpm objectui:build` first), +which dumps the manifest and runs the ratchet against it. Where the manifest *should* come +from in CI is an open provenance question, tracked separately — do not "fix" the red by +re-adding a skip. + `check:exported-any` is the one of those that also reads the built `dist/*.d.ts`, so the stale-`dist` caveat above applies to it too. It asks the other half of the `api-surface.json` question: that snapshot records an export *exists*, never what it diff --git a/packages/spec/scripts/check-generated-ledger.test.ts b/packages/spec/scripts/check-generated-ledger.test.ts index 80c44fd7de..833a15ca7a 100644 --- a/packages/spec/scripts/check-generated-ledger.test.ts +++ b/packages/spec/scripts/check-generated-ledger.test.ts @@ -84,6 +84,27 @@ describe('check:generated --reconcile-only', () => { expect(output).toContain('1 explicit manual-only generators'); }); + it('files the gate whose input this repo cannot produce as EXTERNAL_INPUT_REQUIRED (#4690)', () => { + // NO_GENERATOR would say "runnable, deliberately not run here" — which is what + // `check:react-declaration-parity` said while it was wired into no workflow and + // skipping by default, i.e. while running nowhere at all. The classification has + // to carry the two facts a reader needs instead: WHICH input is missing, and WHO + // supplies it. + const { status, output } = runReconcile(); + expect(status, output).toBe(0); + expect(output).toContain('1 needing an external input'); + expect(output).toContain('cannot run here: check:react-declaration-parity'); + expect(output).toContain('MANIFEST'); + expect(output).toContain('scripts/gen-sdui-manifest.sh'); + // And the claim is not free: `runBy` must still invoke the gate. The + // reconciliation fails otherwise (a gate classified as "runs elsewhere" while + // running nowhere is the hole this category exists to make visible), so this + // asserts the same fact where the failure message is legible. + const runner = path.resolve(SPEC, '..', '..', 'scripts/gen-sdui-manifest.sh'); + expect(fs.existsSync(runner)).toBe(true); + expect(fs.readFileSync(runner, 'utf8')).toContain('check:react-declaration-parity'); + }); + it('covers the test-layer typecheck gate and its writer (#5286)', () => { // The specific pair that failed CI on this branch. Named here so a later // change that drops either script also has to come back through this file. diff --git a/packages/spec/scripts/check-generated.ts b/packages/spec/scripts/check-generated.ts index a57f75fccb..9adc6c67c3 100644 --- a/packages/spec/scripts/check-generated.ts +++ b/packages/spec/scripts/check-generated.ts @@ -27,7 +27,7 @@ */ import { execSync } from 'node:child_process'; -import { readFileSync } from 'node:fs'; +import { existsSync, readFileSync } from 'node:fs'; import { dirname, join } from 'node:path'; import { fileURLToPath } from 'node:url'; @@ -116,10 +116,6 @@ const GATED: ReadonlyArray<{ const NO_GENERATOR: ReadonlyArray<{ check: string; why: string }> = [ { check: 'check:liveness', why: 'audits whether declared spec properties have a reader — no artifact' }, { check: 'check:empty-state', why: 'audits empty-state coverage — no artifact' }, - { - check: 'check:react-declaration-parity', - why: 'compares the spec schema props against the registry-declared inputs — two declarations, no artifact (and no renderer: #4472)', - }, { check: 'check:skill-examples', why: 'validates skill examples parse — no artifact' }, // Landed in #4177 while this ledger landed in #4183 — neither PR could see the // other, so `main` carried an unclassified script and this reconciliation was @@ -152,6 +148,47 @@ const NO_GENERATOR: ReadonlyArray<{ check: string; why: string }> = [ }, ]; +/** + * Source audits that CANNOT RUN from this repository at all, because the input + * they compare against does not exist here and cannot be produced here. + * + * A separate bucket from `NO_GENERATOR` because the two say different things to a + * reader, and #4690 is what conflating them cost. `NO_GENERATOR` means "runnable, + * deliberately not run in this aggregate — run it yourself and it will answer". + * This one means "you cannot run it here at all, and here is the input it wants + * and who produces it". Sitting in the first list, `check:react-declaration-parity` + * read as the former for the entire time it was the latter: it was wired into no + * workflow, and a manual run without `MANIFEST` printed a `⚠` and exited 0, so no + * path existed on which the gate could go red. Whoever read "deliberately not run" + * reasonably assumed someone, somewhere, was running it. + * + * Encoding WHY in the ledger follows EXPLICIT_GENERATORS (#5807/#5358): a + * classification that records only a name is a classification the next reader has + * to re-derive. `runBy` is what keeps this bucket honest rather than an escape + * hatch — it names the in-repo entry point that DOES run the gate with its input, + * and `reconcileLedger` fails if that file has stopped naming the check. "Cannot + * run here" is a statement about this aggregate; "runs nowhere" would be the defect + * this category is supposed to make visible, not hide. + */ +const EXTERNAL_INPUT_REQUIRED: ReadonlyArray<{ + check: string; + input: string; + runBy: string; + why: string; +}> = [ + { + check: 'check:react-declaration-parity', + input: 'MANIFEST= — objectui\'s registry-inputs dump', + runBy: 'scripts/gen-sdui-manifest.sh', + why: + 'compares the spec schema props against the registry-declared inputs (two declarations, no renderer: #4472). ' + + 'The registry is a browser app, so its manifest exists only after objectui is built at .objectui-sha and ' + + 'enumerated in a real browser — nothing in this repo (console dist is gitignored, the published console ships ' + + 'no sdui.manifest.json) can hand it one. `pnpm sdui:manifest` produces it and runs the ratchet; without it the ' + + 'gate now exits 1 rather than skipping (#4690)', + }, +]; + /** * Generators whose output NOTHING verifies. Recorded rather than ignored: each * one is an artifact that can silently drift from its source, which is the class @@ -203,7 +240,11 @@ const SELF = 'check:generated'; */ function reconcileLedger(scripts: Record): void { const problems: string[] = []; - const declaredChecks = new Set([...GATED.map((g) => g.check), ...NO_GENERATOR.map((n) => n.check)]); + const declaredChecks = new Set([ + ...GATED.map((g) => g.check), + ...NO_GENERATOR.map((n) => n.check), + ...EXTERNAL_INPUT_REQUIRED.map((e) => e.check), + ]); const declaredGens = new Set([ ...GATED.map((g) => g.gen), ...UNGATED_GENERATORS.map((u) => u.gen), @@ -213,8 +254,9 @@ function reconcileLedger(scripts: Record): void { for (const name of Object.keys(scripts)) { if (name === SELF) continue; if (name.startsWith('check:') && !declaredChecks.has(name)) { - problems.push(` \`${name}\` exists in package.json but is in neither GATED nor NO_GENERATOR.\n` + - ` Classify it: does it compare a checked-in artifact against a generator, or audit source?`); + problems.push(` \`${name}\` exists in package.json but is in neither GATED nor NO_GENERATOR (nor EXTERNAL_INPUT_REQUIRED).\n` + + ` Classify it: does it compare a checked-in artifact against a generator, audit source,\n` + + ` or audit source against an input this repo cannot produce (name where it DOES run)?`); } if (name.startsWith('gen:') && !declaredGens.has(name)) { problems.push(` \`${name}\` exists in package.json but no GATED entry names it and it is not in UNGATED_GENERATORS.\n` + @@ -226,6 +268,30 @@ function reconcileLedger(scripts: Record): void { for (const { check } of GATED) if (!scripts[check]) problems.push(` GATED names \`${check}\`, which package.json no longer has.`); for (const { gen } of GATED) if (!scripts[gen]) problems.push(` GATED names \`${gen}\`, which package.json no longer has.`); for (const { check } of NO_GENERATOR) if (!scripts[check]) problems.push(` NO_GENERATOR names \`${check}\`, which package.json no longer has.`); + for (const { check, runBy } of EXTERNAL_INPUT_REQUIRED) { + if (!scripts[check]) problems.push(` EXTERNAL_INPUT_REQUIRED names \`${check}\`, which package.json no longer has.`); + // The claim that makes this category honest rather than an escape hatch: the + // gate cannot run HERE, but it does run SOMEWHERE, and that somewhere is a file + // in this repo that still invokes it. A `runBy` that has stopped naming the + // check is #4690 all over again — a gate classified as "runs elsewhere" while + // running nowhere. + const runner = join(pkgRoot, '..', '..', runBy); + // Named on a line that RUNS it, not merely one that talks about it: these + // runners are shell scripts whose comments discuss the gate at length, and a + // surviving comment is exactly the evidence a deleted invocation leaves behind. + const invokes = existsSync(runner) && + readFileSync(runner, 'utf8') + .split('\n') + .some((line) => line.includes(check) && !line.trim().startsWith('#')); + if (!existsSync(runner)) { + problems.push(` EXTERNAL_INPUT_REQUIRED says \`${check}\` runs via \`${runBy}\`, which does not exist.`); + } else if (!invokes) { + problems.push( + ` EXTERNAL_INPUT_REQUIRED says \`${check}\` runs via \`${runBy}\`, which no longer invokes it.\n` + + ` Either restore the call or reclassify: a gate that runs nowhere is the hole this category records (#4690).`, + ); + } + } for (const { gen } of UNGATED_GENERATORS) if (!scripts[gen]) problems.push(` UNGATED_GENERATORS names \`${gen}\`, which package.json no longer has.`); for (const { gen, gatedBy } of EXPLICIT_GENERATORS) { if (!scripts[gen]) problems.push(` EXPLICIT_GENERATORS names \`${gen}\`, which package.json no longer has.`); @@ -277,8 +343,14 @@ if (reconcileOnly) { console.log( `✓ check:generated ledger reconciles with package.json: ${checks} check: + ${gens} gen: scripts, ` + `all classified (${GATED.length} gated, ${NO_GENERATOR.length} source audits, ` + + `${EXTERNAL_INPUT_REQUIRED.length} needing an external input, ` + `${UNGATED_GENERATORS.length} ungated generators, ${EXPLICIT_GENERATORS.length} explicit ` + `manual-only generators, 1 aggregate).\n` + + // Named, not just counted: this bucket's whole reason for existing is that a + // bare count is what let #4690 read as "someone runs it". + EXTERNAL_INPUT_REQUIRED.map( + (e) => ` ⚠ cannot run here: ${e.check} — needs ${e.input}; runs in ${e.runBy}.\n`, + ).join('') + ` --reconcile-only: no gates were run — this verifies coverage, not artifacts.`, ); process.exit(0); @@ -306,6 +378,15 @@ for (const entry of GATED) { // Narrowing is never silent: say what was deliberately not run. console.log(`\nNot run here (${NO_GENERATOR.length} source audits with no artifact to regenerate): ` + NO_GENERATOR.map((n) => n.check).join(', ')); +// Narrowing is never silent, part three — and this one is a different sentence: +// "deliberately not run" invites the reader to run it, which for these is not an +// option from this repo. Say what the missing input is and who supplies it. +if (EXTERNAL_INPUT_REQUIRED.length) { + console.log(`Cannot run here (${EXTERNAL_INPUT_REQUIRED.length} source audit(s) whose input this repo cannot produce):`); + for (const e of EXTERNAL_INPUT_REQUIRED) { + console.log(` ${e.check} — needs ${e.input}\n runs in ${e.runBy}; ${e.why}`); + } +} if (UNGATED_GENERATORS.length) { console.log(`Generated but ungated (${UNGATED_GENERATORS.length}): ` + UNGATED_GENERATORS.map((u) => u.gen).join(', ') + ' — nothing verifies these are current.'); diff --git a/packages/spec/scripts/check-react-blocks-declaration-parity.test.ts b/packages/spec/scripts/check-react-blocks-declaration-parity.test.ts index a052722d32..497fdf3ffd 100644 --- a/packages/spec/scripts/check-react-blocks-declaration-parity.test.ts +++ b/packages/spec/scripts/check-react-blocks-declaration-parity.test.ts @@ -18,16 +18,26 @@ // declarations agreed and neither was lying. Only the renderer was, and the // renderer is not in scope here. // -// So these tests assert three things in one process run: +// So these tests assert four things in one process run: // 1. the signals it CAN see, in both directions (spec-only / registry-only); // 2. the scope caveat rides along with EVERY report, success included — // whoever forms a belief from this gate is reading a CI log, not a header; // 3. the implementation claim stays gone. This is the executable half of // Prime Directive #10 ("never advertise a capability the runtime doesn't // deliver"): the wording that caused #4472 fails a test if it comes back. +// 4. THE GATE CAN GO RED (#4690). The three above all read the report text, +// which a gate that exits 0 on everything would still produce. #4690 is +// that shape: no workflow ran this gate, and a manual run without MANIFEST +// printed a `⚠` and exited 0 — no path existed on which it could fail, and +// two other issues (#4804/#4777) had started citing it as the negative +// example of a gate nobody can prove catches anything. The last describe +// block therefore asserts EXIT CODES against fabricated violations: a +// registry-only input and a vanished block must each exit non-zero naming +// themselves, an absent/unusable manifest must exit non-zero as "did not +// run", and an accepted state must still exit 0 so the red is discriminating. import { describe, it, expect } from 'vitest'; -import { execFileSync } from 'node:child_process'; +import { execFileSync, spawnSync } from 'node:child_process'; import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; @@ -76,6 +86,55 @@ function run(manifest: Manifest, args: string[] = []): string { } } +/** The committed baseline's shape, per block, as the ratchet reads it. */ +type BaselineFile = { blocks: Record }; + +/** + * Same script, same spawn — but the EXIT CODE is the assertion. + * + * `run()` above deliberately swallows a non-zero exit to return the report text; + * every test using it would pass unchanged against a script that always exited 0, + * which is precisely the #4690 failure mode. So the red-path tests go through this + * helper instead: it never throws, and hands back `status` alongside the output. + * + * `manifest` is written to a tmp file and passed as MANIFEST (a string is written + * verbatim, so malformed JSON can be exercised); `manifestPath` sets MANIFEST to a + * literal path instead; passing neither leaves MANIFEST unset — the case that used + * to skip. MANIFEST is stripped from the inherited env so an ambient one (a + * developer who exported it, `pnpm sdui:manifest` in the same shell) cannot make + * the "no manifest" test silently exercise a real manifest. + */ +function runExit(opts: { + manifest?: Manifest | string; + manifestPath?: string; + baseline?: BaselineFile; + args?: string[]; +}): { status: number | null; output: string } { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'react-parity-exit-')); + try { + const env: NodeJS.ProcessEnv = { ...process.env }; + delete env.MANIFEST; + if (opts.manifest !== undefined) { + const file = path.join(dir, 'sdui.manifest.json'); + fs.writeFileSync(file, typeof opts.manifest === 'string' ? opts.manifest : JSON.stringify(opts.manifest), 'utf8'); + env.MANIFEST = file; + } else if (opts.manifestPath !== undefined) { + env.MANIFEST = opts.manifestPath; + } + const argv = [...(opts.args ?? [])]; + if (opts.baseline) { + const baselineFile = path.join(dir, 'baseline.json'); + fs.writeFileSync(baselineFile, JSON.stringify(opts.baseline), 'utf8'); + argv.push('--baseline', baselineFile); + } + const res = spawnSync(TSX, [SCRIPT, ...argv], { cwd: PKG, env, encoding: 'utf8', maxBuffer: 16 * 1024 * 1024 }); + if (res.error) throw res.error; + return { status: res.status, output: `${res.stdout ?? ''}${res.stderr ?? ''}` }; + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } +} + // `object-form` is a real REACT_BLOCKS entry backed by FormViewSchema. Two of // its schema props are enough to exercise both directions; the rest of the // schema simply shows up as spec-only, which is the soft signal. @@ -165,3 +224,88 @@ describe('check:react-declaration-parity — the retired claim stays retired (Pr expect(header).toMatch(/BOTH SIDES declare and NO RENDERER READS/); }); }); + +describe('check:react-declaration-parity — the gate CAN go red (#4690)', () => { + // `ObjectForm` is the baseline KEY for the `object-form` block (baselines are + // keyed by tag, not schemaType). If the tag ever changes, these tests fail — which + // is the right outcome: the committed baseline would need the same rename. + const TAG = 'ObjectForm'; + const ACCEPTED: BaselineFile = { blocks: { [TAG]: { registryOnly: [], missing: false } } }; + const FABRICATED = 'zzzFabricatedRegistryOnlyInput'; + + /** + * The self-proof #4804/#4777 ask for: a gate that is only ever green on the + * current `main` has not shown it catches anything. Each of these fabricates a + * violation the gate claims to see and asserts a non-zero exit that NAMES it. + */ + it('exits non-zero naming a fabricated registry-only input', { timeout: SPAWN_TIMEOUT_MS }, () => { + const { status, output } = runExit({ + manifest: manifestFor('object-form', [SCHEMA_PROP, FABRICATED]), + baseline: ACCEPTED, + args: ['--strict'], + }); + expect(output).toContain(`new registry-only input(s) not in baseline: ${FABRICATED}`); + expect(status, output).toBe(1); + }); + + it('exits non-zero naming a block that vanished from the manifest', { timeout: SPAWN_TIMEOUT_MS }, () => { + const { status, output } = runExit({ + manifest: manifestFor('something-else', []), + baseline: ACCEPTED, + args: ['--strict'], + }); + expect(output).toContain(`<${TAG}>: block vanished from the manifest`); + expect(status, output).toBe(1); + }); + + /** + * The control. Without it the two reds above could be red for any reason — a + * broken spawn, a bad path — and the suite would still look like proof. + */ + it('exits 0 when the manifest matches the accepted baseline', { timeout: SPAWN_TIMEOUT_MS }, () => { + const { status, output } = runExit({ + manifest: manifestFor('object-form', [SCHEMA_PROP]), + baseline: ACCEPTED, + args: ['--strict'], + }); + expect(output).toMatch(/no new DECLARATION divergence/); + expect(status, output).toBe(0); + }); + + /** + * `--strict` decides what a DIVERGENCE costs. The four cases below are the other + * thing entirely — no comparison happened at all — so they must fail WITHOUT + * `--strict` too. Tying "could not run" to the flag would rebuild #4690's hole one + * flag deeper: every one of these exited 0 before, and the first was the shipped + * default of the manual run. + */ + it('a missing MANIFEST is a failure, not a skip', { timeout: SPAWN_TIMEOUT_MS }, () => { + const { status, output } = runExit({}); + expect(status, output).toBe(1); + expect(output).toMatch(/did NOT run/); + expect(output).not.toMatch(/skipping/); + // The refusal has to be actionable: the manifest's producer lives in another repo. + expect(output).toContain('pnpm sdui:manifest'); + expect(output).toContain('OBJECTUI_ROOT=../objectui'); + }); + + it('a MANIFEST path that does not exist fails loudly, naming the path', { timeout: SPAWN_TIMEOUT_MS }, () => { + const { status, output } = runExit({ manifestPath: '/nonexistent/sdui.manifest.json' }); + expect(status, output).toBe(1); + expect(output).toContain('/nonexistent/sdui.manifest.json does not exist'); + }); + + it('an unreadable manifest fails as a diagnosis, not as an uncaught throw', { timeout: SPAWN_TIMEOUT_MS }, () => { + const { status, output } = runExit({ manifest: '{ not json' }); + expect(status, output).toBe(1); + expect(output).toMatch(/is not readable JSON/); + }); + + it('an empty registry dump is refused rather than compared against nothing', { timeout: SPAWN_TIMEOUT_MS }, () => { + // Every block would report "missing" — catastrophic under --strict, vacuous + // without it. objectui's dumper refuses to WRITE one; this refuses to READ one. + const { status, output } = runExit({ manifest: { components: {} } }); + expect(status, output).toBe(1); + expect(output).toMatch(/declares no components/); + }); +}); diff --git a/packages/spec/scripts/check-react-blocks-declaration-parity.ts b/packages/spec/scripts/check-react-blocks-declaration-parity.ts index 17c0bbc17b..bbd04ac621 100644 --- a/packages/spec/scripts/check-react-blocks-declaration-parity.ts +++ b/packages/spec/scripts/check-react-blocks-declaration-parity.ts @@ -50,12 +50,34 @@ // fixture that stays green while its "renderer" ignores everything, so the // capability cannot be re-assumed by the next reader. // -// The frontend side is the objectui registry-inputs manifest (sdui.manifest.json -// — see objectui scripts/dump-public-manifest.mjs). Provide it with -// MANIFEST=/path/to/sdui.manifest.json. Without it, the check reports "manifest -// unavailable" and exits 0 (same manifest-optional posture as the html-tier gate). +// WHERE THE MANIFEST COMES FROM — AND WHY NOTHING HERE CAN PRODUCE ONE (#4690). +// +// The right-hand side is objectui's registry-inputs manifest (sdui.manifest.json). +// Its only producer is objectui's `scripts/dump-public-manifest.mjs`, which drives a +// real browser (Playwright chromium) at the built console's `dev/manifest-dump.html` +// and reads `window.__MANIFEST`: the registry is a browser app (plugin-map / charts +// pull browser-only deps), so nothing enumerates it from Node. `pnpm sdui:manifest` +// (scripts/gen-sdui-manifest.sh) is the wrapper that builds objectui at +// `.objectui-sha`, dumps the manifest, and then runs THIS gate against it. +// +// This repository carries no manifest to fall back on. Measured, so the next reader +// does not have to re-derive it: `packages/console/dist/` is gitignored (the package +// tracks 4 files, none of them a dist), `scripts/build-console.sh` deliberately does +// not produce one — it must not drag a browser into the console build, and says so — +// and the published `@objectstack/console` tarball has none either (16.1.0: 513 +// files, zero `sdui` matches; its `dist/manifest.json` is the PWA manifest), so even +// the CLI's `@objectstack/console/dist/sdui.manifest.json` fallback resolves to +// nothing. +// +// Therefore "no manifest" never means "nothing to check" here. It means THIS GATE +// DID NOT RUN — reported as exit 1, not as a `⚠` and exit 0. Until #4690 the two +// were confused: the gate was in no workflow at all, and a manual run without a +// MANIFEST skipped and exited 0, so no path existed on which it could ever go red. +// That is the sample AGENTS.md's "Absence must be loud" describes — a verifier that +// silently degrades is worse than no verifier, because it reports success. // // Run: MANIFEST=… pnpm --filter @objectstack/spec check:react-declaration-parity +// (or `pnpm sdui:manifest`, which produces the manifest and runs this for you) // // Baseline ratchet (cheap CI posture). The full spec↔registry divergence has an // accepted baseline (some props are designer-palette-curated, some spec-only are @@ -122,12 +144,70 @@ function manifestInputs(manifest: any, schemaType: string): string[] | null { return inputs.map((i: any) => i?.name).filter(Boolean); } -if (!MANIFEST || !fs.existsSync(MANIFEST)) { - console.log('⚠ react-blocks declaration parity: manifest unavailable (set MANIFEST=…) — skipping.'); - process.exit(0); +/** + * The prescription every "could not run" exit carries. + * + * A refusal is only better than a skip if the reader can act on it. The manifest's + * provenance is two repos away from whoever hits this — it is dumped from objectui's + * registry in a browser — so the exit that replaced the skip has to hand over the + * whole path, not just the missing variable's name. + */ +const MANIFEST_PRESCRIPTION = [ + '', + ' The registry side of this comparison is objectui\'s sdui.manifest.json, and this', + ' repository contains no copy of it: packages/console/dist/ is gitignored, the console', + ' build deliberately does not produce one (it must not pull in a browser), and the', + ' published @objectstack/console ships none either. Produce one, then re-run:', + '', + ' pnpm objectui:build # build + vendor the console at the pinned .objectui-sha', + ' pnpm sdui:manifest # dump the manifest in a browser AND run this ratchet', + '', + ' Against a sibling objectui checkout, point the build at it first:', + '', + ' OBJECTUI_ROOT=../objectui pnpm objectui:build && pnpm sdui:manifest', + '', + ' Or, with a manifest already in hand:', + '', + ' MANIFEST=/path/to/sdui.manifest.json \\', + ' pnpm --filter @objectstack/spec check:react-declaration-parity \\', + ' --baseline react-declaration-parity.baseline.json --strict', + '', + ' (the dump needs a browser: pnpm exec playwright install chromium-headless-shell)', +].join('\n'); + +/** + * Exit loudly because the gate could not run — never because it ran and disagreed. + * + * Deliberately independent of `--strict`: that flag decides what a DIVERGENCE costs, + * and this is the other thing entirely — no comparison happened at all. Making it + * conditional would re-create #4690's hole one flag deeper. + */ +function cannotRun(reason: string): never { + console.error(`✗ react-blocks declaration parity: ${reason}`); + console.error(' This gate did NOT run. That is a failure, not a skip (#4690).'); + console.error(MANIFEST_PRESCRIPTION); + process.exit(1); } -const manifest = JSON.parse(fs.readFileSync(MANIFEST, 'utf8')); +if (!MANIFEST) cannotRun('MANIFEST is not set — there is no registry side to compare against.'); +if (!fs.existsSync(MANIFEST)) cannotRun(`MANIFEST=${MANIFEST} does not exist.`); + +let manifest: any; +try { + manifest = JSON.parse(fs.readFileSync(MANIFEST, 'utf8')); +} catch (err) { + cannotRun(`MANIFEST=${MANIFEST} is not readable JSON — ${(err as Error).message}`); +} +if (!manifest || typeof manifest !== 'object') { + cannotRun(`MANIFEST=${MANIFEST} is not an object — expected { components: { : { inputs: [...] } } }.`); +} +// An empty dump is the same defect wearing a manifest: every block would report as +// "missing", which under --strict reads as a catastrophic regression and without it +// as a green run that compared against nothing. objectui's dumper already refuses to +// write one ("empty manifest — registry not populated"); refuse to read one too. +if (Object.keys((manifest.components ?? manifest) as Record).length === 0) { + cannotRun(`MANIFEST=${MANIFEST} declares no components — an empty registry dump compares against nothing.`); +} let totalSpecOnly = 0; let totalMissingComp = 0; const overlay = (b: (typeof REACT_BLOCKS)[number]) => new Set(b.interactions.map((i) => i.name));