Skip to content

fix(ci): keep a graduation green, and trigger Auto Fix on every generator input - #602

Merged
Vivswan merged 2 commits into
mainfrom
fix/ci-graduation-and-autofix
Nov 12, 2026
Merged

Vivswan merged 2 commits into
mainfrom
fix/ci-graduation-and-autofix

Conversation

@Vivswan

@Vivswan Vivswan commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Before

octokit ships the LFS routes -> the bot graduates lfs.ts -> lint:arch red, and the bot may not write architecture.yml

$ bun .github/scripts/arch-lint.ts
lint:arch: 1 problem
  stale allowance upstream-gaps -> types: no file draws it; remove it from architecture.yml
a pull request edits only a section's docs file -> no Auto Fix run -> docs/reference/sections.md stays stale

does not match   docs/sections/labels.docs.yml
triggers         src/sections/labels/index.ts
$ bun .github/scripts/package-smoke.ts
...
$ npm pack --json --ignore-scripts --pack-destination /tmp/gsac-smoke-XXXXXX/pack

> @vivswan/github-settings-as-code@3.0.0 prepare
> lefthook install || true

package-smoke: JSON Parse error: Unexpected identifier "sync"

After

octokit ships the LFS routes -> the bot graduates lfs.ts -> lint:arch green

$ bun .github/scripts/arch-lint.ts
lint:arch: src/ imports match architecture.yml
a pull request edits only a section's docs file -> Auto Fix runs -> the bot pushes the regenerated page

triggers         docs/sections/labels.docs.yml
triggers         src/sections/labels/index.ts
$ bun .github/scripts/package-smoke.ts
...
$ npm pack --json --ignore-scripts --foreground-scripts=false --pack-destination /tmp/gsac-smoke-XXXXXX/pack
$ npm install /tmp/gsac-smoke-XXXXXX/pack/vivswan-github-settings-as-code-3.0.0.tgz --no-audit --no-fund
...
package smoke: ok

How

  • Tripwire: gap.ts exports OctokitLacks, a constraint a gap file names over its own GAP.
  • Signal: the tripwire still fires as TS2344 inside the gap file, so the graduate script is untouched.
  • Edge: upstream-gaps: [types] leaves architecture.yml, a deliberate edit: a gap file now imports only ./gap.js.
  • Workflow: auto-fix.yml gains five trigger paths and a comment: docs/{sections/*.docs.yml,schema.docs.yml}, test/e2e/openapi/paths.ts, biome.json, biome.local.jsonc. Nothing else about the workflow changed.
  • npm: npm 10 runs prepare under --ignore-scripts and npm 11 skips it. CI packs with npm 11.
  • Library: npm's own --foreground-scripts=false keeps the JSON alone on stdout. No parser was written.

Proof

  • Tests: 621 pass, 0 fail across the 26 test files the change reaches (6 new cases).
  • Smoke: package smoke: ok under npm 10.9.8 and under npm 11.21.2.
  • Gates: typecheck, lint, lint:arch, knip, check:compat, and build:check are green.
Technical details

Trigger paths added

Path Read by How it is read
docs/sections/*.docs.yml build:docs src/sections/docs-registry.ts loads every section's file and shared.docs.yml at import
docs/schema.docs.yml build:docs the same registry loads it at import
test/e2e/openapi/paths.ts build:docs imported by test/e2e/openapi/validate.ts, which was already listed
biome.json, biome.local.jsonc every generator but build:gaps-index .github/scripts/lib/biome-format.ts pipes generated text through biome format
  • Derivation: each generator ran under strace -f -e trace=openat, and every repository file it opened was glob-checked against on.pull_request.paths.
  • Count: 234 repository files are read, of which 36 matched no trigger before and 5 after, as the census script at the end of this section prints.
  • Left out: .gitignore, lint/never-throw.grit, lint/rule-gate.grit. biome opens them while loading its config, and formatter output cannot depend on them.
  • Left out: test/e2e/constants.ts, test/root.ts. They load beside validate.ts, but the bundler drops both: no export of theirs reaches a generator.
  • Hand-listed: the two biome files carry a workflow comment, since the trigger test follows what a generator imports and loads, not what a subprocess opens.
  • Workflow diff: five - "<path>" lines and one two-line comment under on.pull_request.paths. Permissions, jobs, steps, and token use are byte-identical.
Census script and its output
# From the repository root; needs strace. Prints one line for the base commit's workflow and one for this checkout's.
tmp="$(mktemp -d)"
for g in gen-openapi gen-gaps-index gen-docs gen-action-docs gen-inputs-table; do
  strace -f -e trace=openat -o "$tmp/$g.trace" bun ".github/scripts/$g.ts" > /dev/null
done
grep -hv ' = -1 ' "$tmp"/*.trace | grep -o "\"$PWD/[^\"]*\"" | tr -d '"' | sed "s#^$PWD/##" |
  grep -v -e '^node_modules/' -e '^\.git/' -e '^\.git$' | sort -u > "$tmp/opened"
git show faf72dab:.github/workflows/auto-fix.yml > "$tmp/before.yml"
cp .github/workflows/auto-fix.yml "$tmp/after.yml"
for side in before after; do
  bun -e '
    import { readFileSync, statSync } from "node:fs";
    import { parse } from "yaml";
    const [side, dir] = process.argv.slice(1);
    const globs = parse(readFileSync(`${dir}/${side}.yml`, "utf8")).on.pull_request.paths.map((p) => new Bun.Glob(p));
    const read = readFileSync(`${dir}/opened`, "utf8").split("\n").filter((f) => f !== "" && statSync(f, { throwIfNoEntry: false })?.isFile());
    console.log(`${side}: ${read.length} files read, ${read.filter((f) => !globs.some((g) => g.match(f))).length} match no trigger`);
  ' "$side" "$tmp"
done
rm -rf "$tmp"
before: 234 files read, 36 match no trigger
after: 234 files read, 5 match no trigger

The tripwire the script reads

A scratch gap file over a route @octokit/types already ships (GET /repos/{owner}/{repo}), compiled as the script compiles it:

before  src/upstream-gaps/shipped.ts(12,3): error TS2344: Type '"GET /repos/{owner}/{repo}"' does not satisfy the constraint 'never'.
after   src/upstream-gaps/shipped.ts(9,57): error TS2344: Type '{ readonly routes: readonly ["GET /repos/{owner}/{repo}"]; readonly documentedInSpec: false; } & { readonly kind: "octokit"; }' does not satisfy the constraint '{ readonly routes: readonly never[]; }'.
  • Same contract: planGraduation() returns gapFiles: ["src/upstream-gaps/shipped.ts"] and no foreign diagnostic on both sides.
  • Why not MustBeNever inside gap.ts: its argument must be provably never, and for an arbitrary G it is not, so MustBeNever<Extract<G["routes"][number], keyof Endpoints>> is itself a TS2344 at the declaration.
  • Why not defineGap's parameter: the tripwire would fire as TS2322, which a typo in a gap file also raises.
  • MustBeNever itself is untouched: git grep -l MustBeNever -- src test lists 22 files under src/ and 8 under test/.
  • Under src/: its home src/types.ts and 21 other matching files (one of them names it only in a comment), none of them in src/upstream-gaps/.
  • Under test/: 7 users, and test/scripts/graduate-upstream-gaps.test.ts, where the name is text inside a string fixture.
  • What went: its one use in this layer, in lfs.ts, and one mention in a gap.ts comment.
  • Generated: docs/reference/architecture.md loses the upstream_gaps --> types arrow, by bun run build:docs.

The npm difference

  • npm 10 (pacote 19): the directory fetcher runs prepare whatever ignoreScripts says, and npm pack runs scripts in the foreground by default, so the script's stdout precedes the JSON.
  • npm 11 (pacote 21) and npm 12: the fetcher returns early under ignoreScripts.
  • Checked: the pack call prints the JSON alone under npm 10.9.8, 11.21.2, and 12.2.2.
  • Cleanup: the smoke's temp root is removed on the failing run and on the passing one.

How each block was captured

  • Graduation: on a scratch export of each side (git archive <ref> | tar -x), the script's own rewrite, then the index, then the lint:
bun -e 'import { toSpecOnlyGapSource } from "./.github/scripts/graduate-upstream-gaps.ts"; const f = "src/upstream-gaps/lfs.ts"; await Bun.write(f, toSpecOnlyGapSource(await Bun.file(f).text(), f));'
bun .github/scripts/gen-gaps-index.ts
bun .github/scripts/arch-lint.ts
  • Deleted instead of rewritten: rm src/upstream-gaps/lfs.ts in place of the first line prints the same two results.
  • Triggers: the repository test's own matcher, run in each side's checkout:
bun -e 'import { parse } from "yaml"; const paths = parse(await Bun.file(".github/workflows/auto-fix.yml").text()).on.pull_request.paths; for (const path of process.argv.slice(1)) console.log(paths.some((p) => new Bun.Glob(p).match(path)) ? "triggers       " : "does not match ", path);' docs/sections/labels.docs.yml src/sections/labels/index.ts
  • Smoke: to reproduce, install the git hooks with bun install, put any npm 10 on PATH, and run bun .github/scripts/package-smoke.ts.

Tests

  • Command behind the total: bun test test/scripts/{arch-lint,auto-fix-steps,endpoint-docs,gen-docs,generated,graduate-upstream-gaps,nightly-steps,tsc-diagnostics}.test.ts test/library/package-smoke.test.ts test/lint/never-throw.test.ts test/sections/{docs-registry,graphql-queries}.test.ts test/workflows test/docs test/config/package-json.test.ts, after bun run build:schema.
  • test/scripts/graduate-upstream-gaps.test.ts: on a copy of src/, lint:arch stays green with every spec-pinned gap rewritten and with every gap file deleted, beside an untouched control.
  • The same file: a gap over a shipped route fires TS2344 in its own file, plans as a graduation, and its rewrite compiles.
  • test/scripts/auto-fix-steps.test.ts: every module a generator's bundle holds and every docs file the registry loads matches a trigger path.
  • test/library/package-smoke.test.ts: a real npm pack under the smoke's arguments, on a fixture whose prepare prints, yields JSON alone.
  • Comment: auto-fix-steps.ts said the inputs table reads action.yml. It reads INPUT_DECLS from src/flows/inputs.ts.

…ator input

A graduation left lint:arch red. src/upstream-gaps/lfs.ts was the one file drawing the declared edge upstream-gaps -> types, so once the Auto Fix bot rewrote or deleted it, architecture.yml carried a stale allowance the bot may not repair. gap.ts now exports OctokitLacks, a constraint type a gap file names over its own GAP: the tripwire still fires as TS2344 inside the gap file, which is what the graduate script reads, and a gap file imports only ./gap.js. The upstream-gaps: [types] edge is deleted, and the generated module map follows.

Auto Fix did not run for a pull request that edited only a generator input outside src/. Its trigger paths gain docs/sections/*.docs.yml and docs/schema.docs.yml (the docs registry loads them for build:docs), test/e2e/openapi/paths.ts (imported by the validator build:docs reads), and biome.json and biome.local.jsonc (the formatter that every generator but the gaps index pipes its output through). Nothing else in the workflow changed.

package-smoke.ts stopped at npm pack under npm 10, which runs the package's prepare script under --ignore-scripts and prints its output ahead of the JSON. npm 11 skips the script, so CI never saw it. The pack call adds --foreground-scripts=false.

The generator-order comment in auto-fix-steps.ts named action.yml as the inputs table's source. It reads INPUT_DECLS from src/flows/inputs.ts.
Copilot AI balanced review requested due to automatic review settings November 12, 2026 03:36
@github-actions

Copy link
Copy Markdown
Contributor

File size check

0 over a hard cap (fails), 6 warning(s).

File Size Tier Cap
src/engine/diff.ts:56 11 comment lines warn 10
src/github/api-error.ts:9 11 comment lines warn 10
src/problem.ts:73 18 comment lines warn 10
src/sections/contract/errors.ts:13 11 comment lines warn 10
src/sections/contract/module.ts:55 12 comment lines warn 10
src/sections/shared/list/decl.ts:265 14 comment lines warn 10

Split the file, wrap the line, shorten or exempt the comment, or list the path in .file-size-allow.local with a # reason.

5 managed file(s) skipped; repo-platform owns them.

@github-actions

Copy link
Copy Markdown
Contributor

Member shape check

1 finding(s).

File Line Rule
test/sections/contract/validate-hook.test.ts 80 all-or-none

Add the blank lines, make the comment one line directly above the item, comment every field or none, or list the path in .member-shape-allow.local with a # reason.

4 generated file(s) skipped.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block approval of the CI reliability changes.

No open issues 🎉

What changed in this PR

This PR improves CI reliability for upstream-gap graduation, generated-file updates, and library package smoke checks.

Changes:

  • Moves the Octokit graduation constraint into the shared gap module, removing a layer dependency that becomes stale after graduation.
  • Expands Auto Fix triggers to cover documentation and formatter inputs, with regression tests for generator input coverage.
  • Keeps npm pack output parseable under npm 10 by disabling foreground script output, with a regression test.
File Description
test/​scripts/​graduate-upstream-gaps.test.ts Tests architecture validity after graduation and the relocated type-check tripwire.
test/​scripts/​auto-fix-steps.test.ts Checks that generator modules and documentation inputs trigger Auto Fix.
test/​library/​package-smoke.test.ts Tests JSON-only pack output with a printing prepare script.
src/​upstream-gaps/​lfs.ts Uses the shared OctokitLacks constraint.
src/​upstream-gaps/​gap.ts Defines the shared graduation constraint.
docs/​reference/​architecture.md Removes the obsolete dependency from the generated diagram.
architecture.yml Removes the upstream-gaps-to-types allowance.
.github/​workflows/​auto-fix.yml Adds missing generator-input trigger paths.
.github/​scripts/​package-smoke.ts Centralizes pack arguments and disables foreground script output.
.github/​scripts/​auto-fix-steps.ts Corrects a comment identifying the inputs table's source.

Review effort: Balanced

…longs

The comment above the biome trigger paths said no test can see a subprocess's reads. The limit is narrower: the trigger test follows what a generator imports and loads, not what a subprocess opens.
Copilot AI balanced review requested due to automatic review settings November 12, 2026 04:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the CI and automation changes.

No open issues 🎉

Review effort: Balanced

@Vivswan
Vivswan marked this pull request as ready for review November 12, 2026 04:22
@Vivswan Vivswan added the merge-when-green Owner approved: merge once every gate is green label Nov 12, 2026
@Vivswan
Vivswan merged commit 57a85b3 into main Nov 12, 2026
34 checks passed
@Vivswan
Vivswan deleted the fix/ci-graduation-and-autofix branch November 12, 2026 06:02
This was referenced Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-when-green Owner approved: merge once every gate is green

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants