Skip to content

fix(ci): stop the release workflow re-asserting HCL the test suite already owns - #34

Merged
ehtishammubarik merged 1 commit into
masterfrom
fix/release-smoke-33
Sep 20, 2026
Merged

ehtishammubarik merged 1 commit into
masterfrom
fix/release-smoke-33

Conversation

@ehtishammubarik

Copy link
Copy Markdown
Owner

The v0.1.1 release failed at test-wheel on 3.10, 3.11 and 3.12 and published nothing, which is the gating working. Run 33878757670.

It was never a regression in the generated output

$ grep -n publicly_accessible /tmp/gen/environments/prod/main.tf
106:  publicly_accessible    = false

The database is private and prod deletion_protection is true. The workflow asserted single-space alignment:

assert "publicly_accessible = false" in prod    # release.yml:170

create_db_subnet_group is 22 characters and joined that block in #13, so terraform fmt realigned the attribute to four spaces:

$ git log --oneline -S"publicly_accessible = false"    -- stackmason/generate.py | tail -1
1d6c660 refactor: rename to stackmason
$ git log --oneline -S"publicly_accessible    = false" -- stackmason/generate.py | tail -1
ab6c7fb fix(generate): make the generated repository actually applyable (#13)

The release workflow has been red since 2026-07-31. Five weeks, undetected, because it runs only on a tag push and no tag was cut in between. The whitespace is incidental; the absence of a feedback loop is the finding.

The assertions should not have existed

tests/test_stackmason.py:252 already owns the property, correctly:

assert re.search(r"^\s+publicly_accessible\s+= false$", ..., re.M)

and release.yml:140 already runs that suite against the installed package with the source tree deleted. The workflow copy was duplication with no feedback loop, so it drifted while the copy next to the code stayed correct. That is the whole mechanism.

What this PR does

release.yml drops the four HCL content assertions; keeps the artifact-level checks the suite genuinely cannot make
adds outputs.tf to the expected-files list, since #29 made it part of every environment
adds a comment with the #13 alignment story, so nobody adds a content assertion back
tests/ makes the deletion_protection assertion whitespace-tolerant, matching the regex style two functions below it

What stays in the smoke test is what is genuinely about the artifact: the entry point resolves from the wheel, a real stackmason new writes real files, the published build still refuses an 0.0.0.0/0 database and writes nothing when it blocks, and no literal credential appears in generated output.

Demonstrated in both directions

Acceptance criterion was that the alignment case must stop failing and a real regression must still fail. Both checked rather than assumed:

realign deletion_protection to one space   ->  1 passed
set prod deletion_protection = false       ->  1 failed
                                               tests/test_stackmason.py:251: AssertionError

Verification

pytest -q         91 passed
coverage          90.90%, floor 88 met
ruff check        All checks passed
ruff format       8 files already formatted
release.yml       parses as YAML
secret-scan       PASS
git-hygiene       PASS

Closes #33

🤖 Generated with Claude Code

https://claude.ai/code/session_012fMM2HhoJ2XzDkfMR2HwSa

…ready owns

The v0.1.1 release failed at test-wheel on all three Pythons, published
nothing, and surfaced a workflow that had been broken for five weeks.

release.yml asserted `publicly_accessible = false` with a single space.
create_db_subnet_group is 22 characters and joined that block in #13, so
terraform fmt realigned the attribute to four spaces and the assertion has
been false since 2026-07-31. Nothing noticed because the workflow runs only
on a tag push, and no tag was cut in that window. The generated output was
correct throughout: the database is private, and prod deletion_protection is
true.

The assertions should not have been there. tests/test_stackmason.py already
covers both properties, and covers them whitespace-tolerantly, and the step
directly above runs that suite against the installed package with the source
tree deleted. The workflow copy was strictly worse duplication: no feedback
loop between releases, so it drifted silently while the copy next to the code
was kept correct.

- release.yml: drop the HCL content assertions. What remains is what the
  suite genuinely cannot see, because it is about the artifact rather than
  the source: the entry point resolves from the wheel, a real `stackmason
  new` writes real files, the published build still refuses an 0.0.0.0/0
  database and writes nothing when it blocks, and the output carries no
  literal credential. Added outputs.tf to the file list, since #29 made it
  part of every environment.
- A comment states why there are no content assertions there, with the #13
  alignment story, so the next person does not add one back.
- tests: made the deletion_protection assertion whitespace-tolerant, matching
  the regex style already used two functions below it for
  publicly_accessible.

Demonstrated both directions rather than assuming:
  realigned the attribute to one space  -> test passes
  set prod deletion_protection = false  -> test fails

Verified: pytest 91 passed, coverage 90.90% against the 88% floor, ruff clean.

Closes #33

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012fMM2HhoJ2XzDkfMR2HwSa
@ehtishammubarik
ehtishammubarik merged commit 4e862a1 into master Sep 20, 2026
10 checks passed
@ehtishammubarik
ehtishammubarik deleted the fix/release-smoke-33 branch September 20, 2026 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The release workflow has been broken since #13, and only a release could reveal it

1 participant