Skip to content

[fork test] gates must fail, not skip - #16

Closed
efiten wants to merge 4 commits into
ci-basefrom
ci/gate-failtest
Closed

efiten wants to merge 4 commits into
ci-basefrom
ci/gate-failtest

Conversation

@efiten

@efiten efiten commented Oct 8, 2026

Copy link
Copy Markdown
Owner

Fork-only. Deliberate failures in shard 2 and image-check; expect '🎭 Playwright E2E Tests' and '🏗️ Build & Publish Docker Image' to end as failure, not skipped.

efiten and others added 4 commits October 8, 2026 19:10
The pipeline ran strictly in sequence: go-test (5-6.5 min), then E2E
(18-19 min), then the image job (7 min), for about 31-33 minutes per run.

- e2e no longer waits for go-test. It builds its own binaries and uses
  nothing go-test produces. build-and-publish now needs both, so
  nothing reaches GHCR unless every test job passed.
- E2E is split over three runners (e2e-shard, balanced on measured
  suite times: ~6 min each). Each shard has its own server, so no
  shared CPU for the timing-sensitive gesture/animation suites. A gate
  job keeps the old "Playwright E2E Tests" check name and folds the
  shard outputs into the same e2e-badges artifact.
- The two-arch build moves to its own image-check job beside the tests
  and runs on every event; the push reuses its build metadata so every
  layer is a cache hit.
- The local `docker compose build` of the staging image is gone: the
  image was never used (ephemeral runner; deploy pulls :edge from GHCR)
  and rebuilt the same Dockerfile for ~2 min. `compose config` still
  validates the file.
- The a11y suite now actually gets AXE_SCREENSHOT_DIR (the env was
  applied to the echo before it, not to node).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Review of the parallel pipeline found that a failed shard or a failed
image check SKIPPED the gate jobs ("Playwright E2E Tests", "Build &
Publish Docker Image"), and branch protection counts a skipped required
check as passing.

- e2e-test and build-and-publish run under !cancelled() and start with
  a Require step that fails unless every needed job passed.
- build-and-publish also needs race-test (skipped is accepted: it only
  runs when ingestor Go files changed; a master push always runs it).
- The routing test executes those Require steps, sees only declared
  needs like GitHub does, and asserts failure (not skipped) for a failed
  go-test, race-test, shard, E2E gate or image check.
- suite() rejects anything but a number in 1..E2E_SHARDS (a missing
  number, 01, 0) and compares numerically.
- m4-icons waits for the lazy distance index itself and fails when the
  API has hops but no map-jump button renders, instead of depending on
  the a11y suite running first on the same shard.
- Only shard 3 builds the e2etest binary and starts the user-management
  server; the GHCR push no longer re-exports the cache image-check just
  wrote; the flake-gate comment matches what it does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@efiten efiten closed this Oct 8, 2026
@efiten
efiten deleted the ci/gate-failtest branch October 8, 2026 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant