Skip to content

feat: visible verdicts, clearer messages, optional shared head build - #2

Merged
buggtb merged 2 commits into
mainfrom
feat/clear-results-and-reuse-build
Oct 12, 2026
Merged

buggtb merged 2 commits into
mainfrom
feat/clear-results-and-reuse-build

Conversation

@buggtb

@buggtb buggtb commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

What

  1. Visible verdicts. convention, run-specs and base-check each expose verdict (pass, red-on-base-green-on-head, fail, passes-on-base, not-run, skipped) and report-only (true/false) as workflow_call outputs, write one verdict heading to the job summary, and emit a ::notice/::warning titled acceptance <workflow>: <verdict>. Job results and names are unchanged; mode: report semantics are unchanged.
  2. New comment.yml (opt in). One sticky PR comment with a three-row table in plain words, found by a hidden marker and by the Actions bot as author, edited in place on every push.
  3. Clearer base-check wording. Red on head: "not green on the PR build yet: implement the feature or fix the spec" (plus "this PR may have broken it" if it passes on base). Green on base: "passes on the base branch: it proves nothing about this change (label spec-already-green, applied by a human, excuses it)". Red on base and green on head is the success. Tests, README and docs/spec-format.md updated.
  4. Optional share-build. run-specs (share-build: true, share-build-paths) uploads the head build as a tar artifact; base-check (share-build: true, ordered needs: after it) unpacks it instead of rebuilding head. Same run only, 1 day retention, never on forks, falls back to building. Default off. Trade-offs in docs/adopting.md.

Why

The inertia pilot (report mode) showed red specs hiding behind green checks, a message that blamed "the PR build too" when the feature was simply not done, no result at a glance, and the head being built twice.

Opt-in for a caller (existing callers are unaffected: every new input defaults off, new outputs are additive)

permissions:
  contents: read                 # unchanged

jobs:
  # ...convention / specs / spec-first as today...
  comment:
    needs: [convention, specs, spec-first]
    if: always()
    permissions:
      contents: read
      pull-requests: write       # this job only
    uses: spiculedata/acceptance-spec/.github/workflows/comment.yml@<new full sha>
    with:
      comment: true
      convention-verdict: ${{ needs.convention.outputs.verdict }}
      run-specs-verdict: ${{ needs.specs.outputs.verdict }}
      base-check-verdict: ${{ needs.spec-first.outputs.verdict }}
      report-only: ${{ needs.specs.outputs.report-only }}

Optional build reuse: share-build: true + share-build-paths: <outputs> on run-specs; needs: [specs] + share-build: true on base-check.

Verification

  • node --test "scripts/*.test.mjs": 123 tests pass (new: verdict mapping, caller verdicts, comment text and marker, share-build pack/unpack and path refusal, new wording, guard tests).
  • Guard tests changed deliberately: pull-requests: write is now allowed in exactly two places, the job-level block of comment.yml and of the selftest job that calls it, never at top level, and never mentioned in the three workflows that execute PR code. A new test asserts comment.yml has a single checkout, of this repository's scripts (not the caller's), no build/start inputs, no secrets, and uses github.token once. SHA pinning, pull_request-only, env-not-interpolated tests unchanged and green.
  • selftest end to end on the fixture, green. A verdicts job asserts: consumer run-specs=pass; red fixture run-specs=fail while its job result is success (the report-mode problem, now visible); red-base-check=fail; green-on-base=passes-on-base; consumer base-check=red-on-base-green-on-head (new spec 3 against a new /api/gadgets route); build-reused=true (the share-build path really ran: "Head build reused from run-specs"). Note base-check cases report skipped on later PRs that do not touch their fixture specs.
  • The sticky comment ran for real on this PR (same-repo): created on the first push, edited in place on the second (still exactly one comment).

Risk

Low. Additive outputs and a summary heading; one new optional workflow; share-build and comment are off unless opted in. The only new write scope is pull-requests: write on the caller's comment job, which runs none of the PR's code.

Unverified

  • Cross-repository uses: of comment.yml from another repo (selftest uses the local ./ form), including that job.workflow_sha resolves there.
  • Fork PRs: the comment job and share-build are skipped by condition, not exercised on a real fork. A bot (Dependabot) PR with a read-only token would degrade to a warning, not run.
  • share-build wall-clock benefit is not measured (the fixture build is trivial); it serialises base-check after run-specs, so measure on a real app before adopting.
  • Check-run annotation titles were emitted but not inspected in the UI.

🤖 Generated with Claude Code

…head build

- verdict output + report-only output on all three reusable workflows, one verdict line in the
  job summary and a titled annotation; job names unchanged
- comment.yml: opt-in sticky PR comment, the only job with pull-requests: write, no checkout
  of caller code, skipped on forks
- base-check wording: "not green on the PR build yet" vs "passes on the base branch"
- opt-in share-build: run-specs uploads the head build, base-check reuses it (same run, falls back)
- selftest: red-on-head, green-on-base and shared-build fixture cases, and the comment job

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Acceptance self-test (fixture)

Check Result What it means
Convention (a closing PR ships its spec) Nothing to check Nothing to check on this PR.
Specs on this PR's build Pass Everything that ran passed.
Spec-first proof (red on base, green on this PR) Proven Proven: each new or changed spec fails on the base branch and passes on this PR.

Updated for commit aa615e2, workflow run. This comment is edited in place, never repeated.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@buggtb
buggtb merged commit 65c4575 into main Oct 12, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant