Reusable acceptance-test-driven CI. An issue's acceptance criteria become
acceptance/<issue>/spec.json, committed with the pull request that closes it. CI checks that the spec
exists, runs it against the PR's own build, and (optionally) proves it fails on the base commit and
passes on the PR.
Extracted from spiculedata/saiku so that other repositories pin a commit SHA and keep a thin caller workflow instead of copying scripts. This repository is public: it holds no secrets, and consumers pass anything sensitive through their own repository secrets.
Licensed under the Apache License 2.0 (see LICENSE).
This is the developer's view. Adopting the workflows in a repository is a separate job
(docs/adopting.md); the spec file format is in docs/spec-format.md.
1. Start from the issue. Issue #42 says: "GET /api/version answers 200 with a JSON body that has a string
version." That one sentence is one criterion. Write one criterion per acceptance criterion in the issue; if the
issue is vague, fix the issue, not the spec.
2. Write acceptance/42/spec.json. One directory per issue; the directory name equals the issue field. A
command spec runs an argv you provide, and its exit code is the verdict. It always runs, and it gets the app's
address in ACCEPTANCE_BASE_URL:
{
"issue": 42,
"title": "GET /api/version reports the app version",
"kind": "command",
"criteria": [
{ "id": "C1", "given": "the app is running", "then": "GET /api/version answers 200 with a version string" }
],
"command": ["node", "check.mjs"]
}acceptance/42/check.mjs (any language works; this one needs Node 18+ for fetch):
const base = process.env.ACCEPTANCE_BASE_URL;
if (!base) throw new Error('ACCEPTANCE_BASE_URL is not set');
const res = await fetch(new URL('/api/version', base));
if (res.status !== 200) throw new Error(`C1: expected 200, got ${res.status}`);
const body = await res.json();
if (typeof body.version !== 'string') throw new Error('C1: expected a version string');
console.log(`C1 ok: version ${body.version}`);For a plain HTTP check, an api spec needs no script at all. Another issue, #43 ("GET /api/status answers 200
with state = ok"), as acceptance/43/spec.json. Relative paths sit under base-path, here /api:
{
"issue": 43,
"title": "GET /api/status reports readiness",
"kind": "api",
"criteria": [
{ "id": "C1", "given": "any caller", "then": "GET /api/status answers 200 with state ok" }
],
"steps": [
{
"name": "status is ok",
"criterion": "C1",
"request": { "method": "GET", "path": "/status" },
"expect": { "status": 200, "json": { "state": { "equals": "ok" } } }
}
]
}api specs run only when the workflow has a base-url; without one they are not-run, never a pass.
3. Commit the spec first. On your feature branch, before any implementation:
git switch -c feat/version-endpoint
git add acceptance/42
git commit -m "test: acceptance spec for the version endpoint"Start the app as it is (no /api/version yet) and run the spec: it must fail.
$ node "$AS/acceptance-runner.mjs" run --dir acceptance/42 --base-url http://127.0.0.1:3000 --base-path /api
acceptance/42: fail
x GET /api/version reports the app version - exit code: node check.mjs exited 1: ...
Error: C1: expected 200, got 404
4. Implement. Add the endpoint in a later commit, restart the app and run the same command: acceptance/42: pass.
5. Open the PR with a closing keyword in the body for the issue, for example Closes #42 (a bare #42 or
"related to #42" is not a claim). CI then shows the spec failing on the base build and passing on yours.
The scripts are plain Node 22+ with no dependencies. Check out this repository once, at the commit your workflow
pins, and point AS at its scripts/:
git clone https://github.com/spiculedata/acceptance-spec ~/acceptance-spec
git -C ~/acceptance-spec checkout <the 40-character SHA in your acceptance.yml>
AS=~/acceptance-spec/scriptsRun these from the root of your repository, after your spec is written:
# 1. Validate the spec files and the convention, as the `convention` check does.
# --enforce-from 1 means "every issue owes a spec"; use your real cutoff to mirror CI.
PR_BODY="Closes #42" node "$AS/acceptance-check.mjs" --enforce-from 1
# 2. Run the specs against the app you started (build and start it yourself first).
node "$AS/acceptance-runner.mjs" run --all --base-url http://127.0.0.1:3000 --base-path /api
# One spec only: ... run --dir acceptance/42 --base-url ... --base-path ...Command 1 prints acceptance/42/spec.json present and exits 0, or prints each problem ("criteria" must be a non-empty array, a shell metacharacter in command, a directory that does not match issue) and exits 1.
Command 2 prints pass / fail / not-run per spec and exits 1 if any spec ran and failed.
Add --out results.json to keep the JSON.
To check red-on-base, green-on-head before you push, run the same specs against the base branch and compare, using the decision script CI uses:
git worktree add ../base main && cp -r acceptance ../base/ # base code + your new specs
# start the BASE app, then:
(cd ../base && node "$AS/acceptance-runner.mjs" run --all --base-url http://127.0.0.1:3000 \
--base-path /api --out /tmp/base.json)
# stop it, start YOUR app, then:
node "$AS/acceptance-runner.mjs" run --all --base-url http://127.0.0.1:3000 \
--base-path /api --out /tmp/head.json
git diff --name-status main...HEAD -- 'acceptance/*/spec.json' > /tmp/changed.txt
node "$AS/acceptance-base-check.mjs" --changed /tmp/changed.txt --head /tmp/head.json --base /tmp/base.jsonEach spec reads pass ... red on base, green on head and the exit code is 0 when you are good; a spec that passes on base fails with passes on the base branch.
Three checks (job names are the caller's), plus an optional sticky comment if the repository added the comment
job. The comment is one table, edited in place on every push:
| Row | What it tells you |
|---|---|
| Convention (a closing PR ships its spec) | The PR closes issue N (N at or above the cutoff) and acceptance/N/spec.json exists and is valid. |
| Specs on this PR's build | Every spec in acceptance/ ran against your build. |
| Spec-first proof (red on base, green on this PR) | Each spec you added or changed fails on the base build and passes on yours. |
Each row has one result: Pass, Proven (red on base, green on head), Fail, Passes on base: proves
nothing, Not run, or Nothing to check. The detail is in the job summary (its first line is
## Acceptance specs: <verdict>) and in an annotation titled acceptance <workflow>: <verdict>. Under Fail, the
job log says which spec and why, for example not green on the PR build yet: implement the feature or fix the spec. In report-only mode the comment says so: red does not turn the check red, so read the table, not the check marks.
- Red on base, green on head. A spec you add or change must fail on the base branch's build and pass on the
PR's build. A spec written after the fact passes both and proves nothing, so a spec that already passes on base
fails the check (
passes-on-base) unless a human labels the PRspec-already-green. A spec that is red on the PR build fails with "not green on the PR build yet". not-runis never a pass. It means the spec could not run here: anapispec with no base URL, missing credentials, an unreadable spec, or a fork PR without the build token. Nothing is proven until it runs.- Report vs enforce.
mode: reportturns every failure into a warning, so repositories can roll the checks out without blocking anyone.mode: enforcefails the job. Whether a failed job blocks the merge is the repository's own branch protection. - The cutoff.
enforce-from-issueis a number: issues below it owe no spec (the check says "grandfathered"); issues at or above it do. Set it to the latest issue or PR number plus one (they share one number space), so every existing issue is exempt and only new ones owe a spec. An empty or non-numeric value enforces nothing.
Both are applied by a human, in the PR's labels, and nothing in CI applies them. Agents and bots must never apply either one; if you think one is warranted, say so in the PR and let a person decide.
acceptance-waived: this PR owes no spec because it is documentation-only or tests-only. It is per PR, and the person applying it states the reason in a PR comment. It holds only if every changed file is documentation or a test; a PR that touches anything else is treated as if the label were absent, and the check names the files.spec-already-green: a spec added or changed here legitimately passes on base (a regression guard for behaviour that already works). It excusespasses-on-baseonly; it never rescues a spec that fails on the PR build.
Labels are read from the event payload when the workflow starts: apply one before pushing, or close and reopen the PR. The same goes for editing the PR body to add a closing keyword.
| Symptom | Cause and fix |
|---|---|
this PR closes #N but adds no acceptance/N/spec.json |
Add the spec, or the PR is docs/tests-only and a human should label it. Check the directory is acceptance/<issue number>/, not the PR number. |
| Convention says "not applicable (no-closing-issue)" | The body has no closing keyword. Use Closes #N, Fixes #N or Resolves #N, then push or close and reopen the PR. |
| "grandfathered: issues before #X are not enforced" | The issue is below the cutoff. Nothing to do. |
"criteria" must be a non-empty array and similar |
The spec is invalid; run the validate command above. Fix what it names. |
"command" must not contain shell metacharacters |
command is argv, not a shell string: ["node", "check.mjs"]. Put pipes and && in a script. |
acceptance/N/spec.json: "issue" is X but the directory is acceptance/N |
The issue field must equal the directory name. |
Spec is not-run (no base URL configured) |
An api spec needs the workflow's base-url; locally, pass --base-url. For auth: session|basic the ACCEPTANCE_USER / ACCEPTANCE_PASSWORD secrets (and for session the session-path) must be set. |
passes on the base branch: it proves nothing |
The spec is not testing the change. Rewrite it so it fails without your fix and commit it before the fix. Only if it is a genuine regression guard, ask a human for spec-already-green. |
not green on the PR build yet |
The feature is not working on your build, or the spec is wrong. Run the spec locally against your app. |
A command spec fails with ACCEPTANCE_BASE_URL is not set |
Locally, pass --base-url. In CI the repository's workflow needs a base-url. |
{{env.NAME}} in a request is sent literally |
Only the allow-listed environment (ACCEPTANCE_*, PATH, HOME, ...) resolves; anything else stays as written and the step goes red. |
| Label or body change had no effect | Both are read at trigger time. Push a commit or close and reopen the PR. |
| Red result but the check mark is green | The repository is in mode: report. Read the sticky comment or the job summary. |
The first three do the work; comment.yml is optional; thin-caller-gate.yml and pin-health.yml are guards (see "Guarding a caller's wiring"). All are GitHub reusable workflows (on: workflow_call). Call them from a pull_request workflow,
never pull_request_target, pinned to a full commit SHA:
| Workflow | What it does |
|---|---|
convention.yml |
Fails a PR that closes issue N (N >= enforce-from-issue) without acceptance/N/spec.json. A human-applied acceptance-waived label waives a docs-only or tests-only PR, and the waiver is verified against the changed files. mode: report only warns. |
run-specs.yml |
Builds the caller's app, starts it, and runs every acceptance/<N>/spec.json against it. command specs always run; api specs run only when a base URL is available, otherwise they are not-run, never pass. |
base-check.yml |
Red on base, green on head: every spec a PR adds or changes must fail on the base commit's build and pass on the PR's build. A spec that already passes on base fails the check unless a human applied spec-already-green. A spec that is not green on the PR build fails it with "not green on the PR build yet: implement the feature or fix the spec". Self-contained (builds and runs both sides) and caches the base build on the base SHA. |
comment.yml |
Optional. Posts one sticky PR comment summarising the three verdicts in a small table (see "Seeing the result"). The only workflow here with a write scope, and only when a caller opts in. |
| thin-caller-gate.yml | Guard. A static check that a caller's wiring has not drifted from this design (see "Guarding a caller's wiring"). Read-only, no secrets. |
| pin-health.yml | Guard. Reports how far the pinned SHA is behind main; optionally keeps one tracking issue. Never bumps (see "Guarding a caller's wiring"). |
.github/workflows/acceptance.yml in the consuming repository:
name: acceptance
on:
pull_request:
types: [opened, synchronize, reopened]
permissions:
contents: read
concurrency:
group: acceptance-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
convention:
uses: spiculedata/acceptance-spec/.github/workflows/convention.yml@<full-40-char-sha>
with:
enforce-from-issue: "124" # the live latest issue or PR number at adoption time, plus one
mode: report # report first, then enforce
specs:
uses: spiculedata/acceptance-spec/.github/workflows/run-specs.yml@<full-40-char-sha>
with:
build-command: npm ci && npm run build
start-command: npm start
base-url: http://127.0.0.1:3000
ready-url: http://127.0.0.1:3000/healthz
ready-timeout: 120
base-path: /api
mode: report
secrets:
ACCEPTANCE_USER: ${{ secrets.ACCEPTANCE_USER }} # only if specs use auth: session | basic
ACCEPTANCE_PASSWORD: ${{ secrets.ACCEPTANCE_PASSWORD }}
spec-first:
uses: spiculedata/acceptance-spec/.github/workflows/base-check.yml@<full-40-char-sha>
with:
build-command: npm ci && npm run build
start-command: npm start
base-url: http://127.0.0.1:3000
ready-url: http://127.0.0.1:3000/healthz
base-path: /api
base-cache-paths: |
dist
mode: reportIn mode: report every job concludes green even when a spec is red, so the check marks alone hide it.
Each of the three workflows therefore reports a verdict, three ways, without changing any job's result:
- Job summary. One heading line, for example
## Acceptance specs: fail, then the detail. - Annotation.
::notice(pass, proven, skipped) or::warning(anything else) titledacceptance <workflow>: <verdict>; it shows on the check run's page. - Workflow outputs
verdictandreport-only(trueorfalse).verdictis one of:
verdict |
Meaning |
|---|---|
pass |
Everything that ran passed (convention, run-specs), or every changed spec was excused by spec-already-green. |
red-on-base-green-on-head |
base-check only: every changed spec fails on the base and passes on the PR. The proof. |
fail |
Something is red or could not be proven (also: the job failed before it could report). |
passes-on-base |
base-check only: the only problem is specs that already pass on the base branch. |
not-run |
It could not run here (for example a fork PR without the build token). Never a pass. |
skipped |
Nothing to check: no spec changed, no closing issue, or no specs. |
base-check also has the output build-reused (see share-build in docs/adopting.md).
Job names are not changed (callers key required checks on them); the verdict is not in the name.
Sticky comment (opt in). Add a job that calls comment.yml. It reads only the verdict strings, checks
out none of the caller's code, and is the only job that gets pull-requests: write. It finds its comment
by a hidden marker (and by the Actions bot as author) and edits it in place on every push, so there is one
comment per PR. Fork PRs are skipped (read-only token).
comment:
needs: [convention, specs, spec-first]
if: always()
permissions:
contents: read
pull-requests: write # this job only
uses: spiculedata/acceptance-spec/.github/workflows/comment.yml@<full-40-char-sha>
with:
comment: true
convention-verdict: ${{ needs.convention.outputs.verdict }}
run-specs-verdict: ${{ needs.specs.outputs.verdict }}
base-check-verdict: ${{ needs.spec-first.outputs.verdict }}
report-only: ${{ needs.specs.outputs.report-only }}Leave a verdict input out (or empty) to leave that row out. Callers that do not add this job are unaffected.
Common to run-specs.yml and base-check.yml unless marked.
| Input | Default | Meaning |
|---|---|---|
working-directory |
. |
Directory (relative to the repo root) holding acceptance/; the cwd of build and start. Also on convention.yml. |
build-command |
empty | Shell command that builds the app. Run on head, and on base for base-check. Empty: no build. |
start-command |
empty | Shell command that starts the app and keeps running. Empty: nothing is started and base-url is used as is. |
base-url |
empty | Origin of the server under test. Api specs may only talk to this origin. Empty: api specs are not-run. |
ready-url |
base-url |
Polled until it answers 2xx after start-command. |
ready-timeout |
300 |
Seconds to wait for ready-url. |
base-path |
empty | Default prefix for relative api step paths (ACCEPTANCE_BASE_PATH). |
session-path |
empty | Path of the form login for auth: session specs (ACCEPTANCE_SESSION_PATH). Unset: such specs are not-run. |
skip-without-build-token |
false |
When true and the BUILD_TOKEN secret is empty (fork PRs), skip build and start; api specs are not-run. |
mode |
enforce |
enforce or report (failures become warnings). Also on convention.yml. |
node-version |
22 |
Node for the scripts (and for your build). |
java-version, java-cache |
empty | Optional Temurin JDK (and maven/gradle cache) installed before the build. |
runs-on |
ubuntu-latest |
Runner label. |
timeout-minutes |
25 |
Job timeout (the two run workflows). |
artifact-name |
acceptance-results / acceptance-base-check |
Prefix of the uploaded result artifacts. |
share-build, share-build-paths |
false, empty |
run-specs only. Opt in: upload the head build output (share-build-paths, newline separated, relative to working-directory) as a workflow artifact for base-check. See docs/adopting.md. |
share-build, share-build-artifact-name |
false, acceptance-results |
base-check only. Opt in: reuse that artifact instead of rebuilding head (falls back to building). Order the job after run-specs with needs:. |
comment, *-verdict, report-only, heading |
false, empty |
comment.yml only; see above. |
base-cache-paths |
empty | base-check only. Newline-separated build outputs, relative to working-directory, cached on the base SHA. Empty: the base is always built. |
enforce-from-issue |
required | convention only. Issues numbered below this owe no spec; use the latest issue or PR number plus one. Empty or not a number enforces nothing. |
waivable-paths |
empty | convention only. Newline-separated JavaScript regular expressions (repository-relative paths, forward slashes) that replace the built-in docs-and-tests list for the acceptance-waived label. Give the whole list, documentation patterns included. Use it to state your exact test layout instead of the generous `(^ |
skip-outside-pull-request |
false |
convention only. true: the job is skipped, not failed, on any event but pull_request. For a caller that also runs on push / merge_group with a required aggregate job: do not put an if: on the calling job, since a skipped calling job reports a bare check named after the caller job, which can shadow a required check of that name. |
Secrets (all optional, declared by name because reusable workflows cannot take dynamic secret names):
ACCEPTANCE_USER, ACCEPTANCE_PASSWORD (credentials for auth: session | basic; use a low-privilege
account on a disposable instance) and BUILD_TOKEN (exposed to build-command as $BUILD_TOKEN, for
example a private package registry token). secrets: inherit is never needed.
Two small reusable guards keep a caller thin and current. Both are static or read-only, and neither runs
any of the caller's code. They are separate from the three workflows above, so adopt them after the
caller works (docs/adopting.md, "Adopt the guards").
Static check over .github/workflows/*.yml and the file tree. It fails the PR when:
| Rule | Fails when |
|---|---|
pin-sha |
a uses: of an acceptance-spec workflow is not pinned to a full 40-character lowercase SHA (a tag, a branch or a short SHA) |
single-pin |
the callers pin more than one SHA |
local-copy |
the repository holds a file named like a shared script (acceptance-runner.mjs, acceptance-check.mjs, acceptance-base-check.mjs, acceptance-spec.mjs, start-server.sh, stop-server.sh) outside the allow-local-scripts list |
visible-input |
mode or enforce-from-issue is anything but a literal (enforce/report, digits) or ${{ vars.NAME }} |
no-callers |
no workflow calls the spec repository at all |
It reads nothing but workflow text and file names: contents: read, no secrets, no token use. It always
reports a result on a pull_request, so a path filter on the caller cannot leave a required check pending:
a PR that touches neither .github/workflows/ nor acceptance/ gets a clean skipped verdict without
scanning (only-when-relevant: false scans anyway). Callers that also run on push or merge_group set
skip-outside-pull-request: true, same as convention.yml, instead of putting an if: on the calling job
(a skipped calling job reports a bare check that can shadow a required one). Outputs: verdict
(pass, fail, skipped) and findings. mode: report turns findings into warnings.
# .github/workflows/thin-caller-gate.yml in the caller
name: thin-caller-gate
on:
pull_request:
permissions:
contents: read
jobs:
thin-caller-gate:
uses: spiculedata/acceptance-spec/.github/workflows/thin-caller-gate.yml@<same-40-char-sha-as-the-other-callers>
with:
mode: report # then enforce
# allow-local-scripts: |
# ci/acceptance-runner.mjs # reason: needs the Maven wrapper, tracked in issue 123Compares the SHA the caller pins with this repository's main through the GitHub compare API and writes a
job summary: commits behind, whether the changed files include scripts/ or .github/workflows/ (with the
first few listed), and the diff link. With open-issue: true it also opens, or edits in place, ONE issue
titled "acceptance-spec pin health" (never closed automatically, one issue however often it runs). It never
changes a workflow: bumping is a human decision. Only the opt-in issue job holds issues: write.
# .github/workflows/acceptance-pin-health.yml in the caller
name: acceptance-pin-health
on:
schedule:
- cron: "0 7 * * 1"
workflow_dispatch:
permissions:
contents: read
jobs:
pin-health:
permissions:
contents: read
issues: write # a called workflow cannot hold a scope its caller lacks; used only if open-issue is true
uses: spiculedata/acceptance-spec/.github/workflows/pin-health.yml@<same-40-char-sha-as-the-other-callers>
with:
open-issue: false # default; true opens/updates the tracking issueRun it on schedule and workflow_dispatch, not on pull requests.
See docs/spec-format.md. In short:
{
"issue": 818,
"title": "the widgets endpoint lists a sprocket",
"kind": "api",
"criteria": [{ "id": "C1", "given": "any caller", "then": "GET /widgets answers 200" }],
"steps": [
{
"name": "widgets are listed",
"criterion": "C1",
"request": { "method": "GET", "path": "/widgets" },
"expect": { "status": 200, "json": { "$": { "type": "array" } } }
}
]
}A command spec carries "command": ["node", "check.mjs"] (argv, never a shell string) instead of steps.
One directory per issue, not per PR. The PR body claims an issue with a GitHub closing keyword
(Closes #818).
Kept from saiku:
pull_requestonly. The workflows refuse to run on any other event, and a guard test fails if a workflow here gainspull_request_targetorworkflow_run.permissions: contents: read, no write scope anywhere except the opt-incomment.ymljob (pull-requests: write, no checkout of the caller's code, no PR text, skipped on forks; a guard test enforces this), noGITHUB_TOKENhanded to specs. The firstactions/checkoutpersists no credentials.- Third-party actions are pinned to full commit SHAs (the same pins saiku uses); a test fails otherwise.
- Spec commands run with an allow-listed environment (
PATH,HOME,TMPDIR,LANG,LC_*,CI,NODE_OPTIONS,PLAYWRIGHT_*,ACCEPTANCE_*), not the runner's. Nothing else, no tokens. - Api specs may only address the base URL's origin; redirects are never followed; cookies and credentials go to that origin only.
- Untrusted text (PR body, labels) and caller inputs reach scripts through
env:, never interpolated intorun:(a test enforces it).{{env.NAME}}placeholders in specs resolve from that allow-listed env only. - Secrets are withheld by GitHub from fork PRs; those run only what needs none and record the rest
not-run. - A spec that could not run is never a pass.
Residual risks to know about: build-command and start-command execute PR code, and BUILD_TOKEN is
visible to build-command (same as saiku's Maven token); the specs themselves are authored in the PR under
test, so a same-repository PR author can reach whatever the acceptance account can.
A reusable workflow runs in the caller's repository, so its own scripts/ are not on disk. Each job
checks out this repository at exactly the commit the caller pinned, using the job context:
- uses: actions/checkout@<sha>
with:
repository: ${{ job.workflow_repository }} # spiculedata/acceptance-spec
ref: ${{ job.workflow_sha }} # the SHA in the caller's `uses: ...@<sha>`
sparse-checkout: scriptsGitHub documents job.workflow_repository and job.workflow_sha as the repository and commit SHA of the
workflow file that defines the current job, which for a reusable workflow is the called workflow (the
github context, by contrast, describes the caller). No id-token: write (the job_workflow_sha OIDC claim)
and no ref input are needed, so the scripts cannot drift from the pinned workflow. Each job first
verifies the value is a 40-character SHA, then verifies the checked-out HEAD equals it, then moves
scripts/ to $RUNNER_TEMP so the caller's build never sees it.
Pin the full 40-character commit SHA of a commit on this repository's main, never a branch or tag you do
not control. To bump: pick the new main SHA, replace it in every uses: line of the caller (one
find-and-replace), open a PR, and read the commit range
(git log <old>..<new> -- scripts .github/workflows) before merging.
- Report:
mode: reporteverywhere andenforce-from-issueset to the live latest issue or PR number plus one. Failures are warnings; nothing blocks. - Enforce convention: switch
conventiontoenforce. Only issues numbered at or above the cutoff owe a spec. - Enforce runs and spec-first: switch
run-specsandbase-checktoenforce.
Whether any of these checks blocks a merge depends on the repository's own branch protection or rulesets, which this repository neither sets nor assumes.
Plain Node 22+, no dependencies:
node scripts/acceptance-runner.mjs run --all --base-url http://127.0.0.1:8080 --base-path /api
PR_BODY="Closes #123" ACCEPTANCE_ENFORCE_FROM_ISSUE=1 node scripts/acceptance-check.mjs
node --test "scripts/*.test.mjs"Differences a saiku migration must account for (also listed in the extraction PR):
| saiku | here |
|---|---|
SAIKU_ACCEPTANCE_USER / SAIKU_ACCEPTANCE_PASSWORD |
ACCEPTANCE_USER / ACCEPTANCE_PASSWORD |
SAIKU_BASE_URL in command specs |
ACCEPTANCE_BASE_URL only (saiku's SAIKU_* names are not passed through) |
base path hard-coded to /rest/saiku/api |
base-path: /rest/saiku/api |
session login hard-coded to /rest/saiku/session |
session-path: /rest/saiku/session |
kind ui |
ui still works; command is the same thing |
--feedback ci-feedback.json / attach |
removed (saiku's ci-feedback schema is not generic); post-process the results JSON |
waivable path saiku-ui/e2e/ |
`(^ |
ACCEPTANCE_ENFORCE_FROM_ISSUE repo variable |
enforce-from-issue input (pass ${{ vars.ACCEPTANCE_ENFORCE_FROM_ISSUE }} to keep the variable) |
selftest.yml also runs both guards: thin-caller-gate on fixtures/thin-caller/good (must pass) and on
fixtures/thin-caller/drifted (a branch pin, two SHAs, a local script copy and non-literal inputs; must
produce the fail verdict, and exit 1 in enforce mode), and pin-health on fixtures/pin-health.
Unit tests, the loopback fixture end to end (scripts/e2e-fixture.test.mjs) and the workflow guard tests
run in ci.yml. selftest.yml calls the three reusable workflows against fixtures/consumer from this
same repository with the local ./ form.
Not exercised until the workflows exist at a SHA on main: the cross-repository uses: spiculedata/acceptance-spec/...@<sha> from another repository, in particular that job.workflow_sha /
job.workflow_repository resolve to this repository at the pinned SHA there, and that a private caller can
fetch this public repository with its default token. See docs/adopting.md for the post-merge check.