chore: what the v1.1.0 pre-flight found — a discarded dry run, five stale figures, a missing fragment - #150
Merged
Merged
Conversation
The Upload assets step ran when dry_run was not set. So it uploaded on a real release, where GoReleaser has already attached the same archives and checksums to the GitHub Release, and skipped the one path that publishes nothing and therefore has no other output. RELEASE.md has described the intended behaviour all along: a dry run "builds artifacts and uploads them to the run, but publishes nothing". The condition was inverted, so a maintainer validating a build before committing to a tag got a green run with nothing to inspect.
775 boto3 tests, 948 auto-crud operations, 4,496 hand-verified, 148 registered services of which 31 served nothing, and a 93/11 split of model-backed to hand-written services. The measured values are 1,144, 5,193, 4,497, 205 with 4 serving nothing, and 193/12. Nothing caught the drift because nothing reads the file. cmd/devcloud/coverage_test.go gates docs/coverage.md, README.md and docs/README.md against the live registry in both directions; this document is not in that set, and the Go tests that do cite it (internal/config/config_test.go, internal/admin/api_test.go) lock the guaranteed surface by hardcoding keys rather than by parsing prose. The figures now come from the gated page or from a measurement, not from a restatement. This matters more than a stale number usually would: the document ships inside the release archive, so the copy in devcloud_vX.Y.Z_linux_amd64.tar.gz describes the binary beside it.
Release notes come from Changie fragments, not commits, so a merged user-facing change with no fragment is a change that will not appear in the release. PR #149 corrected four false claims in the contributor and getting-started guides and added none. Found by the pre-flight review for v1.1.0. The existing checks cannot see this: RELEASE.md's checklist verifies that every fragment carries an issue number and that the unreleased directory is not empty, neither of which notices a change that never wrote one.
github-actions
Bot
added
documentation
3 of 5 tasks
skyoo2003
added a commit
that referenced
this pull request
Sep 6, 2026
* docs: the docs restated the same numbers in five different values
Eighteen documentation files, rewritten for concision: 3,564 lines to 2,501.
Most of what came out was narrative that re-argued decisions the tables
already state.
The scatter had a cause worth naming. Six figures were published in more
than one place, and no two places agreed:
services 205 / 148 / 101 -> 205 registered, 201 serving
operation tiers 12,407 / 9,030 / -> 4,497 / 5,193 / 2,717
7,475
compat tests 992 / 1,144 / 775 -> 1,144
no Smithy model 11 / 12 -> 11
engine-wired 46 -> 155
Only coverage.md, README.md and docs/README.md are gated against the binary
by cmd/devcloud/coverage_test.go, so those three were right and every
restatement of them had rotted. coverage.md now owns the numbers outright
and the other pages link to it, which is the arrangement the gate already
assumes.
Three statements did not match the code:
- troubleshooting.md told a reader to check GET /devcloud/api/health.
internal/admin/api.go registers five routes and health is not among
them, so the one command offered for "is the server up?" returns 404.
- troubleshooting.md attributed a failed Lambda invoke to Docker not
running. lambda/runtime.go is a stub whose Invoke unconditionally
returns a placeholder payload; no configuration makes it execute a
handler.
- contributing.md listed interface.go, serializer.go and deserializer.go
as codegen output. There are four templates and none of them emits
those; providers parse *http.Request themselves, which architecture.md
already said on the same tree.
crud-engine.md contradicted itself: its protocol table served all five
protocols, and its Scope section two screens later said JSON-only across 46
services. The Scope section was left over from before rest-json landed and
is gone.
compatibility-policy.md said 193 model-backed services and 12 without.
Parsing manifest_gen.go gives 194 and 11 — the figures corrected in #150
were themselves off by one.
The 285-row evidence table in demand.md is untouched: TestDemandSetIsRegistered
parses it, and it is a record of a sample rather than prose.
Verified: all five doc gates pass, every relative link in the tree resolves,
go test ./cmd/... is green.
* chore: add the changelog fragment for the docs rewrite
Release notes come from Changie fragments, not commits, so a docs change
that ships inside the release archive needs one. The issue number is the
PR's, which is why this lands after the PR exists rather than with the
work.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three independent defects found while running the v1.1.0 pre-flight checklist. None of them is caught by an existing check.
Related Issue
None — found by review, not reported. The Changie fragments reference this PR.
Changes
release.yml: a dry run discarded its artifacts. TheUpload assetsstep ran whendry_runwas not set, so it uploaded on a real release — where GoReleaser has already attached the same archives to the GitHub Release — and skipped the one path that publishes nothing and therefore has no other output.RELEASE.mdhas described the intended behaviour all along.docs/compatibility-policy.md: five figures the code had outgrown. 775 boto3 tests, 948auto-crudoperations, 4,496 hand-verified, 148 registered services of which 31 served nothing, and a 93/11 model-backed split. Measured values: 1,144 / 5,193 / 4,497 / 205 with 4 serving nothing / 193 and 12.cmd/devcloud/coverage_test.gogatesdocs/coverage.md,README.mdanddocs/README.mdagainst the live registry; this file is not in that set, and nothing else reads it. It ships inside the release archive, so a stale figure there describes the binary beside it.Test Plan
Full v1.1.0 pre-flight, all four items green:
diff -rqagainst the committed tree (equivalent torm -rf internal/generated && make codegen, without the deletion): no difference.CGO_ENABLED=0 go test ./...: pass.go test -count=1 ./cmd/devcloud/re-run uncached so the published-figure gate is not read from cache.CGO_ENABLED=0 go buildthenDEVCLOUD_BIN=dist/devcloud pytest tests/compatibility/: 1,127 passed, 17 skipped (1,144 collected, which is the figure now published in the policy).grep -L 'Issue: "[0-9]' changes/unreleased/*.yaml: empty.Note that
make test-compatdoes not reproduce the release gate: it passes noDEVCLOUD_BIN, soconftest.pyfalls back togo runwithCGO_ENABLED=1, while the release workflow tests a GoReleaser-builtCGO_ENABLED=0binary. The command above was used instead.Checklist
golangci-linthas nothing to say hereFollow-up not done here
docs/compatibility-policy.mdwent stale silently because no test reads it. Extendingcmd/devcloud/coverage_test.goto gate its figures too — or replacing the non-load-bearing counts with a pointer to the gated page — would close that hole. Left out to keep this PR to the defects themselves.