Skip to content

feat: extract acceptance-test-driven CI from saiku as reusable workflows - #1

Merged
buggtb merged 1 commit into
mainfrom
feat/extract-from-saiku
Oct 12, 2026
Merged

buggtb merged 1 commit into
mainfrom
feat/extract-from-saiku

Conversation

@buggtb

@buggtb buggtb commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

What

Extracts saiku's acceptance-test-driven CI (spiculedata/saiku development, incl. #2239) into reusable code:

  • scripts/: repo-agnostic acceptance-spec/-runner/-check/-base-check.mjs plus start-server.sh / stop-server.sh, with their unit tests (plain Node 22+, no dependencies, no package.json).
  • Reusable workflows (on: workflow_call): convention.yml, run-specs.yml, base-check.yml (red on base / green on head, spec-already-green hatch, base build cached on the base SHA).
  • Docs: README.md, docs/spec-format.md, docs/adopting.md.
  • Self-tests: ci.yml (unit tests on node 22 and 24, aggregate job ci), selftest.yml (calls the three workflows on fixtures/consumer, a loopback server).

Callers pin spiculedata/acceptance-spec/.github/workflows/<name>.yml@<full-sha>.

How the scripts are loaded at the pinned ref

Each job checks out this repo with repository: ${{ job.workflow_repository }} and ref: ${{ job.workflow_sha }} (sparse, scripts/ only). Per GitHub's contexts docs these job.* properties describe the workflow file that defines the current job, i.e. the called workflow, unlike the github context (the caller). No ref input and no id-token: write (which job_workflow_sha would need). Each job refuses an empty or non-40-hex value, verifies the checked-out HEAD equals it, then moves scripts/ to $RUNNER_TEMP.

Why

Private repos (inertia, kinetic, keel, later hoaapp; saiku/saiku-cloud eventually) should reuse this with a pinned SHA and a thin caller instead of copying scripts.

Deviations from saiku (all intentional)

  1. Env names: SAIKU_ACCEPTANCE_USER/PASSWORD -> ACCEPTANCE_USER/PASSWORD; SAIKU_BASE_URL is no longer exported (only ACCEPTANCE_BASE_URL). Allow-list now ACCEPTANCE_* only.
  2. No hard-coded /rest/saiku/*: base path from base-path / ACCEPTANCE_BASE_PATH (default none), form-login path from session-path / ACCEPTANCE_SESSION_PATH; auth: session without it is not-run. The /rest/ as-is rule is kept.
  3. kind: "command" added; ui kept as an alias.
  4. ci-feedback integration (--feedback, attach, summariseResults, attachResults) is not extracted: saiku's schema is not generic. The results JSON is uploaded as an artifact.
  5. Waivable test path saiku-ui/e2e/ generalised to (^|/)e2e/; tests/ to tests?/.
  6. ACCEPTANCE_ENFORCE_FROM_ISSUE repo variable becomes the enforce-from-issue input.
  7. base-check.yml is self-contained: it builds and runs both head and base itself (only the changed specs) instead of downloading the specs job's artifact, so it does not depend on another workflow in the caller. Cost: one extra head build on PRs that touch a spec.
  8. Secrets are three named optional workflow secrets (ACCEPTANCE_USER, ACCEPTANCE_PASSWORD, BUILD_TOKEN) because reusable workflows cannot take dynamic secret names. skip-without-build-token reproduces saiku's fork-PR behaviour (no build, api specs not-run).
  9. mode: report|enforce added (all three workflows): failures become warnings.
  10. A base commit without the working directory (new sub-project) skips build/start: api specs not-run.
  11. Third-party action pins are saiku's, unchanged (checkout v7.0.1, setup-node v7.0.0, setup-java v6.0.1, cache v6.1.0, upload-artifact v7.0.2 / v7.0.1, download-artifact v8.0.1).

Unchanged: pull_request only (enforced by a runtime guard and a test), contents: read, no write scope, no GITHUB_TOKEN to specs, allow-listed spec env, api specs limited to the base URL's origin, no redirects followed, a spec that cannot run is never pass.

Verification

  • node --test "scripts/*.test.mjs": 99 tests, all pass locally (node 24). Includes the real start/stop scripts + runner + base check on the loopback fixture, workflow guard tests (SHA pins, no pull_request_target/workflow_run/write scopes, secrets list, no ${{ }} in run:, scripts loaded via job.workflow_*) and a hygiene scan (no tokens, hostnames, IPs, host paths).
  • CI on this PR: all 10 checks green (ci, unit tests node 22 and 24, selftest with convention / run-specs / base-check jobs). In the self-test logs job.workflow_repository / job.workflow_sha resolved to this repo and the merge commit (local ./ call). run-specs passed both fixture specs against the loopback server. base-check reported not-run for the api spec and pass (red on base) for the command spec, as designed for a base commit that does not yet contain the fixture.

Risk

Low for this repo (new code only). The reusable workflows are untested from another repository until merged.

Not verified, check after merge

  1. Cross-repo uses: spiculedata/acceptance-spec/.github/workflows/<name>.yml@<sha> from a private caller: that job.workflow_repository / job.workflow_sha resolve to this repo at the pinned SHA there (the job fails with an explicit error if not), and that the private caller's default token can fetch this public repo. selftest.yml only exercises the local ./ form.
  2. base-check on a real build: here it runs only on the PR that adds the fixture spec (later PRs touch no spec, so it is skipped), and base-side behaviour is covered by the local end-to-end test, not a CI run with a real base build and cache restore/save.
  3. Org-level Actions allow-lists for private callers.
  4. No branch protection / required checks are configured or assumed by this PR.

🤖 Generated with Claude Code

Repository-agnostic scripts (ACCEPTANCE_* env), reusable workflows convention.yml,
run-specs.yml and base-check.yml that load their scripts at the caller's pinned ref via
job.workflow_repository / job.workflow_sha, docs, a loopback fixture, unit and end-to-end
tests, and a self-test workflow.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@buggtb
buggtb merged commit da49765 into main Oct 12, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant