Repository navigation
feat(ci): reject an empty test-command before it can pass silently - #27
Merged
Merged
Conversation
2 tasks done
Juanmadiaz45
added a commit
to MicroTodoSuite/microservice-app-gitops
that referenced
this pull request
Sep 21, 2026
## What changes Adds evidence (`evidence/runs/20260921T000000Z-gate-visibility-validation/`) validating T024 (gate presence/visibility/fail-fast) and T038 (full quickstart end-to-end), and links the companion `.github` fix. ## Why T024 asks to validate gate visibility via `act` dry-run and a forced `run-unit=true` run. `run-unit` no longer exists (retired when the architecture centralized on one mandatory `test-command`), and local `act` proved impractical (20+ minutes pulling a runner image without completing on this workstation). Switched to real GitHub Actions runs (disposable draft PRs, closed without merging once observed) and found a real defect: an empty `test-command` made "Run repository tests" report `success` having tested nothing -- a genuine FR-017 violation, not just a stale-task-wording issue. Fixed upstream in [MicroTodoSuite/.github#27](MicroTodoSuite/.github#27). T038 asks to run quickstart end-to-end against a GHCR/cloud-disabled mode that also no longer exists (same reconciliation as T035). Ran every section that still applies against the current repos instead of a mode that can no longer be exercised. ## Tasks - gitops specs/003-reusable-cicd-delivery T024 (delivered, defect found and fixed) - gitops specs/003-reusable-cicd-delivery T038 (delivered, with reconciliation) - [x] The task register is updated in this pull request - [x] Every task ticked here was verified by locating its artifact ## How it is verified - Real, live GitHub Actions runs (not local `act`): `test-command: ""` reported `"Run repository tests": success` before the fix (disposable PR `.github#25`); after the fix (`.github#27`), the same input reports `"Reject an empty test-command": failure` and `"Run repository tests": skipped` (disposable PR `.github#26`). Both throwaway PRs closed without merging once observed. - quickstart §1: `actionlint` clean on `ci.yml`/`release.yml`/`promote.yml`; all three declare `workflow_call`. - quickstart §2: all five services are thin `uses:` callers pinned to the same SHA, no `development.yml` remains. - quickstart §6: `kustomize build | kubeconform -strict` passes for all four onboarded services' local overlays (`Valid: 4, Invalid: 0, Errors: 0` each); base is environment-neutral; shared JWT confirmed by T029's new contract test. - quickstart §7: `validate-gitops.yml` runs on every PR including this one. - Full detail and reconciliation for §3/§4/§5 (superseded by T021/T035's real evidence) in the linked evidence file. - Infrastructure documentation: this validates existing GitHub Actions/Kustomize configuration; no new Terraform/AWS/Azure provider argument was introduced, so neither required MCP server for this repo (`aws-knowledge`, `microsoft-learn`) applies to this specific change. ## Risk and rollback None: adds evidence documentation only. The actual behavior fix lives in the linked `.github#27` PR, reviewed there. Rollback here is reverting this commit. ## What this PR does not do Does not itself change any workflow -- the fix is in `.github#27`. Does not restore a GHCR/cloud-disabled mode that no longer matches the architecture.
2 tasks done
Juanmadiaz45
added a commit
that referenced
this pull request
Sep 21, 2026
) ## What changes Adds a guard step to `stack-tests.yml` that fails visibly when a caller's `stack-command` is empty or blank, before the gate step executes. Also adds `tests/workflows/stack-tests-contract.bats`, the first automated contract coverage for this workflow. ## Why Investigating gitops spec 007 T027 (verify "enabled-but-empty fails visibly" once per gate) after already finding and fixing the identical defect in `ci.yml`'s `test-command` (gitops spec 003 T024, PR #27): `stack-command` has the exact same shape -- `required: true`, no default, silently satisfiable by an empty string, and `run: ${{ inputs.stack-command }}` with an empty value succeeds having run no e2e/perf/dast/pact/conformance gate at all. This workflow backs five separate gates across the frontend repo (e2e, perf, dast, pact, conformance), so the blast radius of a caller accidentally leaving `stack-command` empty was larger than the one already fixed. Confirmed live on real GitHub Actions before the fix (disposable draft PR, closed without merging): a probe run with `stack-command: ""` against a minimal one-service compose stack succeeded. After this fix (disposable draft PR #28, closed without merging): the same input fails at the new guard with `"Reject an empty stack-command": conclusion "failure"` and `"Run probe gate": conclusion "skipped"`. ## Tasks - gitops specs/007-advanced-testing T027 (delivered -- see this repo's evidence in the linked gitops PR) - [x] The task register is updated in the linked gitops pull request - [x] Every task ticked there was verified by locating its artifact ## How it is verified - `tests/workflows/stack-tests-contract.bats` (new file) fails before the guard exists and passes after, following the SDD test-then-feat pair. - `actionlint .github/workflows/stack-tests.yml` is clean. - Live re-verification on real GitHub Actions: disposable draft PR #28 with `stack-command: ""` against a minimal `redis:7-alpine`-only compose stack now fails at the new guard step with `::error::stack-command must not be empty or blank -- an empty run: step succeeds having run no probe gate (FR-017: a gate must fail visibly, never pass silently)`, and the gate step itself is correctly skipped. Closed without merging once observed. - Infrastructure documentation: this is a GitHub Actions workflow change, not Terraform/AWS/Azure; no required MCP server applies to this specific change. ## Risk and rollback Low: adds one guard step; every existing caller (frontend's e2e/perf/dast/pact/conformance workflows) already supplies a real, non-empty `stack-command`, so no caller is currently affected. Rollback is reverting this commit. ## What this PR does not do Does not change `compose-file` or `ready-command`, which are legitimately optional (`default: ""`) and already `if:`-guarded to skip visibly when unset -- only `stack-command` is `required: true` yet was silently satisfiable by an empty string, exactly like `ci.yml`'s `test-command` before PR #27.
Juanmadiaz45
added a commit
to MicroTodoSuite/microservice-app-gitops
that referenced
this pull request
Sep 21, 2026
…cipline (#210) ## What changes Adds evidence (`evidence/runs/20260921T010000Z-advanced-testing-validation/`) validating the six remaining spec 007 tasks the 2026-08-30 reconciliation identified as real work: T011, T012, T013, T018, T026, T027. ## Why Per the reconciliation note in `specs/007-advanced-testing/tasks.md`: T011/T012 were partial (only `frontend<->todos-api` existed), and T013/T018/T026/T027 are observed-verification tasks that cannot be ticked from the existence of code. This closes all six with real, reproducible evidence rather than inference. ## Tasks - gitops specs/007-advanced-testing T011 (delivered) - gitops specs/007-advanced-testing T012 (delivered) - gitops specs/007-advanced-testing T013 (delivered) - gitops specs/007-advanced-testing T018 (delivered) - gitops specs/007-advanced-testing T026 (delivered) - gitops specs/007-advanced-testing T027 (delivered, two real defects found and fixed) - [x] The task register is updated in this pull request - [x] Every task ticked here was verified by locating its artifact ## How it is verified - T011/T012: two new Pact consumer contracts (`microservice-app-frontend` PR #35), authored from real, locally-verified `auth-api`/`users-api` behavior. 3 interactions, 0 failures against the real 5-service compose stack; confirmed green in real CI. - T013: deliberately corrupted a contract's expected value against the live stack -- real failure (exit 1), reverted -- real pass (exit 0). Full transcript in `pact-break-revert.txt`. - T018: for each of the 4 integration gates (todos-api, log-message-processor, users-api, auth-api), ran the real baseline, broke the real interaction code (not the test), confirmed a real failure against the real dependency (Testcontainers Redis x2, real H2, real HTTP boundary), reverted, confirmed green. All 4 repos clean afterward. Full transcript in `integration-gate-breaks.txt`. - T026: byte-for-byte comparison of all 5 services' `ci.yml` callers proves zero per-repo gate duplication; all 5 frontend stack gates route through one shared `stack-tests.yml`. - T027: found and fixed two real "required-but-satisfiable-by-empty-string" defects (`ci.yml`'s `test-command`, already fixed in `.github#27`; and newly found `stack-tests.yml`'s `stack-command`, fixed in `.github#29`) -- confirmed live on real GitHub Actions before and after each fix via disposable draft PRs, closed without merging once observed. - Infrastructure documentation: this validates existing GitHub Actions/application code across 5 repos; no new Terraform/AWS/Azure provider argument was introduced, so neither required MCP server for this repo (`aws-knowledge`, `microsoft-learn`) applies to this specific change. ## Risk and rollback None: adds evidence documentation only. The actual fixes live in `MicroTodoSuite/.github#27` and `#29`, and the new contracts live in `microservice-app-frontend` PR #35 -- all reviewed there. Rollback here is reverting this commit. ## What this PR does not do Does not itself change any workflow or service code -- those changes are in the linked PRs. Does not add a Pact contract for `log-message-processor<->todos-api`, which correctly uses the existing AsyncAPI mechanism instead.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes
Adds a guard step to
ci.ymlthat fails visibly when a caller'stest-commandis empty or blank, before the "Run repository tests" step executes.Why
Investigating gitops spec 003 T024 (validate gate presence/visibility and fail-fast behavior) found a real defect:
test-commandisrequired: truewith no default, but a caller can still satisfy that requirement with an empty string.run: ${{ inputs.test-command }}with an empty resolved value executes an empty shell script, which succeeds (exit 0) having tested nothing -- a green checkmark that doesn't mean what it claims to mean. This is exactly the silent-pass failure mode FR-017 exists to prevent for gate visibility.Confirmed live on real GitHub Actions before the fix (disposable, closed PR #25): a job with
test-command: ""reported"Run repository tests": conclusion "success". After this fix (disposable, closed PR #26): the same input reports"Reject an empty test-command": conclusion "failure"and"Run repository tests": conclusion "skipped".Tasks
gitops specs/003-reusable-cicd-delivery T024 (delivered -- see this repo's evidence in the linked gitops PR)
The task register is updated in the linked gitops pull request
Every task ticked there was verified by locating its artifact
How it is verified
tests/workflows/reusable-workflow-contract.batsfails before the guard exists (ci.yml must fail visibly on an empty test-command...) and passes after, following the SDD test-then-feat pair.actionlint .github/workflows/ci.ymlis clean.test-command: ""now fails at the new guard step with::error::test-command must not be empty or blank -- an empty run: step succeeds having tested nothing (FR-017: a gate must fail visibly, never pass silently), and "Run repository tests" is correctly skipped rather than running. Both disposable probe PRs (DISPOSABLE: probe empty test-command (T024, not for merge) #25 before the fix, DISPOSABLE: re-verify empty test-command fix (T024, not for merge) #26 after) were closed without merging once observed.aws-knowledge,microsoft-learnfor the companion gitops repo) applies to this specific change.Risk and rollback
Low: adds one guard step; every existing caller already supplies a real, non-empty
test-command(confirmed: all five service repos passtest-command: go test ./...or the equivalent for their language), so no caller is currently affected. Rollback is reverting this commit.What this PR does not do
Does not change any caller's
test-commandvalue. Does not touchsource-audit-commandorcontract-command, which are legitimately optional and alreadyif:-guarded to skip visibly when unset -- onlytest-commandisrequired: trueyet was silently satisfiable by an empty string.