feat: gate the published coverage figure, and test every registered service

[skyoo2003]

Summary

Milestone 6 of aws-service-coverage-100: CI now gates the published coverage figure in both directions, and the boto3 compatibility suite exercises every registered service instead of a hand-written list of 69. Building the gate exposed that the quality floor's second clause — a passing compatibility test exists for the service — had never been measured: 31 of 205 registered services had no boto3 test at all, 29 of them counted inside the published 201.

Related Issue

N/A — this delivers the final milestone of .claude/prds/aws-service-coverage-100.prd.md. Refs #140, #142, #143.

Changes

The gate (cmd/devcloud/coverage_test.go, new)

• Reads the published figures out of docs/coverage.md, README.md and docs/README.md and compares each to the live registry and fidelity manifest. Removing a service without moving the published number fails; editing the number without the code moving fails identically.
• Also gates the per-operation tier table, the names of the registered-only services, and demand-set membership (all 57 services in docs/demand.md must stay registered, so the count cannot be held steady by swapping one out).
• Replaces the conservative floors it supersedes: minServices = 100 / minOperations = 6000 in fidelity_test.go and minServices = 50 in conformance_test.go. Those were right for catching a mangled generator and could not notice 205 becoming 120.
• No workflow files change — ci.yml already runs go test ./... and compat.yml already runs pytest.

The compatibility suite (tests/compatibility/)

• test_service_smoke.py parametrises over all 205 services from a generated manifest. ENGINE_SERVED_SERVICES (67 entries) and REGISTERED_ONLY_SERVICES (2) are deleted — registering a service and testing it are no longer separate steps.
• _coverage.py (new) resolves DevCloud service IDs to boto3 client names from botocore's own model metadata, refusing a contested name rather than guessing. 204 of 205 resolve automatically; elasticloadbalancing → elb is the single override.
• The floor is asserted per service, not per operation: up to 6 served operations are tried and one success is enough.

Codegen (internal/codegen/gen_compat_manifest.go, new)

• Emits internal/generated/compat/services.json — per registered service, its protocol and served operations, projected from the same fidelity data docs/coverage.md publishes from. It lives under internal/generated so the existing codegen-drift job covers it for free.
• Note for review: the 10,489-line diff in that file is entirely generated. The hand-written change is 68 lines.

Docs

• docs/coverage.md gains a fourth number — Compatibility-tested: 199 — with its ceiling and the reason for each exclusion.
• Fixes drift the gate found on its first run: README.md and docs/README.md had quoted 148 registered / 117 serving since three milestones earlier; docs/demand.md named three demand-set services as serving nothing when Milestone 5 left one; sagemakerruntimehttp2 serves nothing and was never named on a page that claims to name them all.

Test Plan

CGO_ENABLED=0 go test ./...                             → all pass
make codegen; git status --porcelain internal/generated → clean (no drift)
bash scripts/generate-imports.sh; git diff --exit-code  → clean
pytest tests/compatibility/                             → 993 passed in 19.17s (was 854)
golangci-lint run                                       → 0 issues

The gate was proven to bite by mutation, four ways, each reverted immediately:

Mutation	Result
Delete services/omics from imports.go	FAIL — registry 204 vs published 205
Change **205** to **206** in docs/coverage.md	FAIL — published 206 vs binary 205
Revert README.md to 148 registered / 117 serving	FAIL — quotes 148/117, binary has 205/201
Delete the figure from README.md entirely	FAIL — "no longer states the figure in a form this gate can read"

The fourth mutation was run because the first version of the pattern did not match README.md's phrasing at all and passed silently on a stale figure. Zero matches is now a failure.

Findings recorded, deliberately not fixed here

Each is a pre-existing defect the gate surfaced. None is silent — all three are written down in docs/coverage.md or the evidence report.

1. Eight query providers fabricate a success. rds, neptune, docdb, redshift, elasticache, autoscaling, cloudformation and elasticloadbalancingv2 answer any unimplemented query action with HTTP 200 and an empty <{Action}Response/>. They never return plugin.ErrUnhandledOp, so the CRUD engine is never reached, and the body has no {Action}Result wrapper for botocore to read. This contradicts the guarantee docs/coverage.md calls absolute and is the same class as the s3-control P1 fixed in fix: S3 Control requests were served by S3, silently #142 — it deserves its own PR.
2. The four Lex services are registered and reachable by no boto3 caller. All four clients sign as the contested alias lex, which BuildAliases leaves unrouted by design. Published in docs/coverage.md and pinned by test_lex_services_are_unreachable_from_boto3, so fixing the routing fails that test and raises the published figure with it. A correct fix needs a four-way path split — /bots is the first segment for three of the four.
3. Three operations are labelled hand-verified but no route reaches them — appsync/ListApis, eks/ListAccessPolicies, opensearch/ListApplications. Operation-level, not service-level; all three services still meet the floor.

Checklist

• Self-reviewed the code
• Added/updated tests
• Lint/format passes (golangci-lint run → 0 issues)
• Updated documentation (docs/coverage.md, docs/demand.md, README.md, docs/README.md)
• Added a Changie changelog fragment for user-facing changes (Added / Changed / Documentation)