diff --git a/content/docs/guide/ci-cd-pipeline.md b/content/docs/guide/ci-cd-pipeline.md index 4f6ad7d37f..665ec4afc1 100644 --- a/content/docs/guide/ci-cd-pipeline.md +++ b/content/docs/guide/ci-cd-pipeline.md @@ -63,10 +63,17 @@ keys in both directions, so a job added or removed without touching this table i (This section used to open with a hard-coded count and list a seventh job, `dev-server`, that had been deleted three months earlier — [#3451](https://github.com/objectstack-ai/objectui/issues/3451).) +The **What it runs** column is pinned one level further down, by command +([#3653](https://github.com/objectstack-ai/objectui/issues/3653)): every first-party command a job +runs — a `node scripts/*.mjs` invocation, a root `package.json` script, or a `turbo run` task — must +be named in that job's row, and a row may not name one its job does not run. Until that pin landed +this page was judged job by job only, so a `run:` step added to an existing job left every check on +it green — which is how two of `type-check`'s gates came to be missing from this column. + | Job key | Appears as | What it runs | When | |---|---|---|---| | `changeset-check` | Changeset Fixed Group Check | `scripts/check-changeset-fixed.mjs` — every workspace package must be in the changeset `fixed` group or explicitly ignored. It checks group *membership*; it does **not** check whether the PR added a changeset. | Every run | -| `type-check` | Type Check | `scripts/check-type-check-coverage.mjs`, then `pnpm check:spec-symbols`, then `pnpm type-check:scripts`, then `pnpm type-check`, then `pnpm type-check:vitest-setup`. The coverage guard runs first because turbo silently skips packages that have no `type-check` script, so a package without one would otherwise read as passing (#2911). `pnpm type-check:scripts` (`tsconfig.scripts.json`) covers `scripts/**/*.ts`, which `pnpm type-check` cannot reach at all — `scripts/` has no package.json, so turbo never walks it, and the coverage guard decides coverage per *package*. Until [#3494](https://github.com/objectstack-ai/objectui/issues/3494) that left the pin tests in `scripts/__tests__/` — including the one pinning this very page — compiled by nothing. `pnpm type-check:vitest-setup` (`tsconfig.vitest-setup.json`) closes the same gap for the four repo-root `vitest.setup.*` files, uncovered until [#3515](https://github.com/objectstack-ai/objectui/issues/3515); it runs *last*, after `pnpm type-check`, because `vitest.setup.dom.tsx` side-effect-imports four `@object-ui/*` packages and resolves them through the declarations that turbo's `^build` produces. | Every run | +| `type-check` | Type Check | `scripts/check-type-check-coverage.mjs`, then `pnpm check:spec-symbols`, then `pnpm check:i18n-keys`, then `pnpm check:i18n-drift`, then `pnpm type-check:scripts`, then `pnpm type-check`, then `pnpm type-check:vitest-setup`. The coverage guard runs first because turbo silently skips packages that have no `type-check` script, so a package without one would otherwise read as passing (#2911). The two locale gates sit in the middle because both parse the sources with `typescript`: they need the install and nothing built. `pnpm check:i18n-keys` fails when a `t()` call site asks for a key the `en` pack does not define ([#3530](https://github.com/objectstack-ai/objectui/issues/3530)); `pnpm check:i18n-drift` fails when a change to an `en` string is not accompanied by the nine translation packs ([#3650](https://github.com/objectstack-ai/objectui/issues/3650)), and it is why this job's checkout sets `fetch-depth: 0` — it diffs against the merge base, which a depth-1 clone cannot resolve. `pnpm type-check:scripts` (`tsconfig.scripts.json`) covers `scripts/**/*.ts`, which `pnpm type-check` cannot reach at all — `scripts/` has no package.json, so turbo never walks it, and the coverage guard decides coverage per *package*. Until [#3494](https://github.com/objectstack-ai/objectui/issues/3494) that left the pin tests in `scripts/__tests__/` — including the one pinning this very page — compiled by nothing. `pnpm type-check:vitest-setup` (`tsconfig.vitest-setup.json`) closes the same gap for the four repo-root `vitest.setup.*` files, uncovered until [#3515](https://github.com/objectstack-ai/objectui/issues/3515); it runs *last*, after `pnpm type-check`, because `vitest.setup.dom.tsx` side-effect-imports four `@object-ui/*` packages and resolves them through the declarations that turbo's `^build` produces. | Every run | | `test` | Test (shard N/4) | `pnpm test --shard=N/4` across a 4-runner matrix with `fail-fast: false`, so every shard reports its own failures. No coverage instrumentation — v8 adds 40–100% overhead. | **Pull requests only** | | `test-coverage` | Test (coverage) | One unsharded `pnpm test:coverage`, uploaded to Codecov. Nothing blocks on it, which is why it is not sharded. | **Push only** | | `e2e` | Build & E2E | Builds the console with `vite build` (`VITE_BASE_PATH=/console/`), verifies the artifact, then `pnpm test:e2e --project=chromium`. Uploads the Playwright report on failure. | Every run | diff --git a/scripts/__tests__/ci-cd-pipeline-doc.test.ts b/scripts/__tests__/ci-cd-pipeline-doc.test.ts index eb0b5cd2c4..ea8c80e6b2 100644 --- a/scripts/__tests__/ci-cd-pipeline-doc.test.ts +++ b/scripts/__tests__/ci-cd-pipeline-doc.test.ts @@ -268,18 +268,18 @@ describe('ci-cd-pipeline.md — ci.yml job table', () => { const JOB_TABLE_HEADER = '| Job key | Appears as | What it runs | When |'; - /** First column of the job table, in page order. */ - function docJobRows(): { key: string; appearsAs: string }[] { + /** The job table's rows in page order: job key, `Appears as`, `What it runs`. */ + function docJobRows(): { key: string; appearsAs: string; runs: string }[] { const section = coreCiSection(); const at = section.indexOf(JOB_TABLE_HEADER); expect(at, `the job table must keep the header \`${JOB_TABLE_HEADER}\``).toBeGreaterThan(-1); const lines = section.slice(at).split('\n').slice(1); - const rows: { key: string; appearsAs: string }[] = []; + const rows: { key: string; appearsAs: string; runs: string }[] = []; for (const line of lines) { if (!line.startsWith('|')) break; if (/^\|[\s|:-]+\|$/.test(line)) continue; // separator const cells = line.split('|').slice(1, -1).map((c) => c.trim()); - rows.push({ key: cells[0].replace(/`/g, '').trim(), appearsAs: cells[1] ?? '' }); + rows.push({ key: cells[0].replace(/`/g, '').trim(), appearsAs: cells[1] ?? '', runs: cells[2] ?? '' }); } return rows; } @@ -374,4 +374,166 @@ describe('ci-cd-pipeline.md — ci.yml job table', () => { 'still denies it (objectui#3451).', ).toBe(true); }); + + /** + * objectui#3653: every pin above judges `ci.yml` at JOB granularity — the set of + * job keys, the `name:` each one reports under, the absence of a count. None of + * them reads what a job *runs*, so a `run:` step added to an existing job left + * this whole file green. Two such steps had landed unlisted: `pnpm + * check:i18n-keys` (objectui#3530, PR #3547) and `pnpm check:i18n-drift` + * (objectui#3650, PR #3659) both ran in the `type-check` job while its row on the + * page still listed five commands. + * + * Both directions are asserted, because the column is a claim in both: a command + * the job runs and the row omits is a contributor who cannot learn from this page + * that a gate exists; a command the row names and the job does not run is the + * objectui#3451 shape one level down — a page advertising a guardrail that is not + * there. + * + * What counts as a command is deliberately narrower than "every step", and the + * boundary is *derived* rather than hand-listed: a step counts when it names + * something this repository owns — a `scripts/*.mjs` file, a script in the root + * `package.json`, or a `turbo run` task. Environment setup drops out on its own + * because it names none of those (`corepack enable`, `pnpm --version`, `pnpm + * install --frozen-lockfile`, `pnpm exec playwright install`, `pnpm --filter … + * exec vite build`), which keeps this column a summary of the gates rather than a + * transcript of the YAML: the `e2e` job's artifact check and Playwright cache are + * real steps that no reader of this page needs enumerated. + * + * The hole that leaves, stated so nobody mistakes it for coverage: a gate written + * as an inline shell block names no first-party command and is invisible here. + * Gates in this repository land as a root `package.json` script or a + * `scripts/*.mjs` file — both covered — and that is the only reason the narrower + * rule is enough. + * + * Steps are read from `run:` values only, never from the surrounding YAML. The + * `type-check` job's comments alone mention `pnpm type-check`, `turbo run + * type-check` and `pnpm check:i18n-drift`; a scan of the raw block would take all + * three for steps and this pin would then be describing its own comments. + * + * Whether a given step still EXISTS in `ci.yml` is pinned where that step was + * introduced — `check-i18n-call-site-keys.test.ts` and + * `check-i18n-en-drift.test.ts` each hold their own, as do + * `scripts-type-check.test.ts` and `vitest-setup-type-check.test.ts`. This block + * does not repeat those assertions; it pins the *pairing* between the YAML and + * this page, which is the part nothing owned. + */ + describe('what each job runs', () => { + const rootScripts = new Set( + Object.keys( + ( + JSON.parse(fs.readFileSync(path.join(repoRoot, 'package.json'), 'utf8')) as { + scripts: Record; + } + ).scripts, + ), + ); + + /** One job's YAML block, from its key line up to the next thing at that indent. */ + function jobBlock(key: string): string { + const body = ciWorkflow.slice(ciWorkflow.search(/^jobs:[ \t]*$/m)); + const at = body.search(new RegExp(`^ {2}${key}:[ \\t]*$`, 'm')); + expect(at, `ci.yml must still define a \`${key}:\` job`).toBeGreaterThan(-1); + const rest = body.slice(at + 1); + // A job's own comments are indented four spaces or more; the two-space ones + // introduce the *next* job, so stopping at any two-space line is right. + const next = rest.search(/^ {2}\S/m); + return next === -1 ? rest : rest.slice(0, next); + } + + /** Every `run:` step body in a job block — single-line and block scalar alike. */ + function runSteps(block: string): string[] { + const lines = block.split('\n'); + const steps: string[] = []; + for (let i = 0; i < lines.length; i++) { + const at = lines[i].indexOf('run:'); + // `run:` must be the key of the line, not text inside another value. + if (at === -1 || !/^[\s-]*$/.test(lines[i].slice(0, at))) continue; + const value = lines[i].slice(at + 'run:'.length).trim(); + if (!/^[|>][-+]?$/.test(value)) { + steps.push(value); + continue; + } + const body: string[] = []; + for (let j = i + 1; j < lines.length; j++) { + if (lines[j].trim() === '') continue; + if (lines[j].search(/\S/) <= at) break; + body.push(lines[j].trim()); + } + steps.push(body.join('\n')); + } + return steps; + } + + /** + * The first-party commands named in a piece of text — applied to a job's `run:` + * bodies on one side and to its `What it runs` cell on the other, so the two + * sides are compared by the same rule rather than by two spellings of it. + */ + function firstPartyCommands(text: string): Set { + const found = new Set(); + // A gate that lives in this repo's `scripts/` tree. The `node ` prefix is not + // required: ci.yml writes `node scripts/x.mjs`, the page writes the path. + for (const m of text.matchAll(/scripts\/[\w./-]+\.mjs/g)) found.add(m[0]); + // A root `package.json` script. `install`, `--version`, `exec` and `--filter` + // are not scripts, so the setup steps need no exemption list. + for (const m of text.matchAll(/\bpnpm\s+([\w:.-]+)/g)) { + if (rootScripts.has(m[1])) found.add(`pnpm ${m[1]}`); + } + // The build graph, invoked through the task runner instead of a script. + for (const m of text.matchAll(/\bturbo\s+run\s+([\w:-]+)/g)) found.add(`turbo run ${m[1]}`); + return found; + } + + /** `job key -> commands it actually runs`, and the same from the page's table. */ + function commandsByJob(): { key: string; ran: Set; named: Set }[] { + return docJobRows().map((row) => ({ + key: row.key, + ran: firstPartyCommands(runSteps(jobBlock(row.key)).join('\n')), + named: firstPartyCommands(row.runs), + })); + } + + it('names every first-party command the job actually runs', () => { + const jobs = commandsByJob(); + + // A parser that matched nothing would make both directions vacuously green — + // the exact failure the `dev-server` job demonstrated one level up. + const ran = jobs.reduce((n, j) => n + j.ran.size, 0); + expect(ran, 'the ci.yml `run:` parse found implausibly few first-party commands').toBeGreaterThan(8); + + const missing = jobs.flatMap((j) => [...j.ran].filter((c) => !j.named.has(c)).map((c) => `${j.key}: ${c}`)); + + expect( + missing, + `.github/workflows/ci.yml runs commands that the job table in ` + + `content/docs/guide/ci-cd-pipeline.md does not name:\n` + + missing.map((m) => ` - ${m}`).join('\n') + + `\n\nAdd each one to that job's "What it runs" cell, in the order ci.yml runs it. ` + + `A gate nobody wrote down is a build failure contributors meet without knowing what ` + + `produced it — objectui#3653: two locale gates ran in \`type-check\` unlisted, because ` + + `the pins on this page read job keys and job names but never read the steps.`, + ).toEqual([]); + }); + + it('credits no job with a first-party command it does not run', () => { + const jobs = commandsByJob(); + + const named = jobs.reduce((n, j) => n + j.named.size, 0); + expect(named, 'the job table parse found implausibly few commands in "What it runs"').toBeGreaterThan(8); + + const phantom = jobs.flatMap((j) => [...j.named].filter((c) => !j.ran.has(c)).map((c) => `${j.key}: ${c}`)); + + expect( + phantom, + `content/docs/guide/ci-cd-pipeline.md's job table credits jobs with commands that ` + + `.github/workflows/ci.yml does not run there:\n` + + phantom.map((p) => ` - ${p}`).join('\n') + + `\n\nEither the step was removed and the cell is stale, or the command runs in a ` + + `different job and belongs in that row. A "What it runs" cell reads as this job's ` + + `gate list, so naming a command for contrast inside it makes the page claim a ` + + `guardrail — the objectui#3451 mistake, one level down.`, + ).toEqual([]); + }); + }); });