Skip to content

feat: hand-write the eight services the CRUD engine cannot classify - #161

Merged
skyoo2003 merged 1 commit into
mainfrom
feat/phase-2-pilot-the-eight
Sep 12, 2026
Merged

skyoo2003 merged 1 commit into
mainfrom
feat/phase-2-pilot-the-eight

Conversation

@skyoo2003

Copy link
Copy Markdown
Owner

Summary

Eight AWS services hold no CRUD-shaped operation between them, so the generic engine can never classify their 31 operations and scaffolding would leave them registered and serving nothing. Each gets a hand-written provider, plus the one piece of plumbing without which three of them would be registered and reachable by nothing.

Related Issue

Refs #160 (Phase 1 — the models these eight are built from)

Changes

The eight providers — 31 operations, all hand-verified, none unimplemented

Service Ops Protocol Note
eksauth 1 rest-json
inspectorscan 1 rest-json echoes the SBOM — DevCloud runs no scanner
ec2instanceconnect 2 json-1.1
kinesisvideowebrtcstorage 2 rest-json signs as kinesisvideo
marketplacecommerceanalytics 2 json-1.1
cloudsearchdomain 3 rest-json signs as cloudsearch; sqlite-backed, documents round-trip
georoutes 5 rest-json empty result sets — no road graph ships
paymentcryptographydata 15 rest-json signs as payment-cryptography; encrypt→decrypt round-trips

crud.RegisterRoutes — why the contested three are reachable

The gateway splits a shared SigV4 signing name by asking each sibling whether it models the request. That question reads a route table built only from CRUD-classifiable operations, and these services have none — so payment-cryptography-data's traffic was kept by the control plane, and cloudsearch-domain's by a Query provider. RegisterRoutes lets a provider declare the routes it serves by hand. It claims nothing about fidelity: Handle still finds no OpMeta and returns ErrUnclassified. What it changes is which service is asked. Handle is untouched, and no serviceIDOverrides entry was added — the model decides, not a hardcoded winner.

Compatibility harness

_coverage.py's stub builder could not satisfy two of the eight client-side: a string minimum (SSHPublicKey is 80 characters at the least) and a tagged union with no member set. Both read as a service serving nothing when the defect is in the harness — the same way an unsatisfied collection minimum already did, which that file handles and comments on. Nothing in any assertion was weakened.

A pre-existing fabricated success, surfaced and recorded

Padding strings to their minimum made one probe reach the gateway for the first time, and it found workspacesweb.AssociateBrowserSettings answering 200 for an operation nothing implements: the CRUD registry holds only classifiable operations, so UpdatePortal at PUT /portals/{portalArn+} greedily swallows /portals/<arn>/browserSettings. The fix belongs in codegen and was implemented and measured before being reverted — it moves 569 operations between fidelity tiers and takes registered-only from 4 to 1. That is a coverage re-derivation of its own, so it is recorded in test_no_fabricated_success.py's own KNOWN_UNFIXED ledger as a strict xfail with the mechanism named, and left to the scaffolding phase.

Three wire-level defects found by driving real clients

Every unit test here asserts Go-side response maps, which is a layer above the wire. Pointing real boto3 clients at all 31 operations found three defects invisible to them — all fixed, each with a reproducer that was confirmed RED first:

  • cloudsearchdomain.Search was unreachable. botocore rewrites it from the modelled GET /2013-01-01/search?format=sdk&pretty=true into a POST with those terms in a form body. The modelled route matched nothing, so the request stayed with cloudsearch and came back 501. The provider now declares the POST route — the same table backs its own resolution and RegisterRoutes, so the two cannot disagree — and reads q from the form body.
  • GeneratePinData returned PinData: {}. It is a tagged union; botocore raises PinData must have one and only one member set rather than returning, so the 200 was unusable. It now carries the member the caller's generation attribute implies.
  • geo-routes sent PricingBucket in the body. The model binds it to the x-amz-geo-pricing-bucket header, so botocore discarded it in all five operations without erroring. It is a header now.

Published figures, re-derived

Figure Before After
Registered 205 213
Serving ≥1 operation 201 209
Registered-only 4 4 (same four services)
Compatibility-tested 203 211
hand-verified 4,497 4,528
auto-crud / unimplemented 5,193 / 2,717 unchanged
total known 12,407 12,438

The "coverage target is 205 services" blockquote and the whole "The target" section are deliberately untouched — that is the structural rewrite a later phase owns. The target reading 205 next to 213 registered is correct and temporary.

Test Plan

  • go vet ./... — clean
  • golangci-lint run — 0 issues.
  • CGO_ENABLED=0 go build ./... — clean
  • CGO_ENABLED=0 go test ./... — green, including the five published-figure gates
  • make test-compat — 1,155 passed, 1 xfailed (was 1,144 tests before this branch)
  • make stats — Services: 213
  • make codegen run twice, byte-identical
  • make build — 34.8 MiB, budget 45 MiB
  • Coverage of the eight packages + crud: 87–96%, all above the 80% floor
  • All 31 operations driven through real boto3 clients against a running server: 31/31 correct

Two tests were proven to bite rather than trusted:

  • Commenting out one crud.RegisterRoutes call makes TestContestedDataPlanesResolveToThemselves fail with expected: "paymentcryptographydata", actual: "paymentcryptography" — the control plane keeping the request, which is the exact failure the mechanism exists to prevent.
  • Commenting out the POST /2013-01-01/search route makes the new test_handverified_is_reachable.py row fail with cloudsearchdomain.Search … answered 501.

Both files were restored byte-identical and re-verified green.

Checklist

  • Self-reviewed the code
  • Added/updated tests
  • Lint/format passes (golangci-lint run)
  • Updated documentation (if applicable)
  • Added a Changie changelog fragment for user-facing changes

Not one of these eight services' 31 operations carries a CRUD verb prefix,
so the generic engine can never classify them and scaffolding would leave
them registered and serving nothing — the outcome the coverage policy
forbids. Each gets a hand-written provider instead: eks-auth,
inspector-scan, ec2-instance-connect, kinesis-video-webrtc-storage,
marketplace-commerce-analytics, cloudsearch-domain, geo-routes and
payment-cryptography-data.

Three of them sign with a name an already-registered neighbour claims, and
registering those without more would leave them reachable by nothing. The
gateway splits a shared signing name by asking each sibling whether it
models the request, and that question reads a route table built only from
CRUD-classifiable operations — which these services have none of. So
crud.RegisterRoutes lets a provider declare the routes it serves by hand.
It claims nothing about fidelity: Handle still finds no OpMeta and returns
ErrUnclassified. What it changes is which service is asked.

The compatibility harness could not reach two of the eight: botocore
refused the stub input client-side, once on a string minimum and once on a
tagged union with no member set. Both read as a service serving nothing
when the defect is in the harness, the same way an unsatisfied collection
minimum already did, so _coverage.py now handles them too. That made one
probe reach the gateway for the first time and find a pre-existing
fabricated success in workspaces-web, recorded as a strict xfail with its
mechanism named — a greedy CRUD route swallowing a more specific
unclassifiable one, which is a coverage re-derivation of its own.

Driving all 31 operations through real boto3 clients found three defects
no Go test could see, all now fixed and covered:

  - botocore rewrites cloudsearch-domain's Search from the modelled GET to
    a form POST, so the modelled route matched nothing and the request
    stayed with cloudsearch. The provider declares the POST route and reads
    q from the form body.
  - GeneratePinData returned an empty PinData; it is a tagged union, and
    botocore raises rather than returning one with no member set.
  - geo-routes bound PricingBucket into the body, but the model binds it to
    the x-amz-geo-pricing-bucket header, so botocore dropped it.

Registered services 205 -> 213, serving 201 -> 209, hand-verified
operations 4,497 -> 4,528. The registered-only set is still exactly the
known four.
@github-actions github-actions Bot added documentation Improvements or additions to documentation tests Test code and test infrastructure services AWS service implementations labels Sep 12, 2026
@skyoo2003
skyoo2003 merged commit e2dee4e into main Sep 12, 2026
10 checks passed
@skyoo2003
skyoo2003 deleted the feat/phase-2-pilot-the-eight branch September 12, 2026 19:55
skyoo2003 added a commit that referenced this pull request Sep 13, 2026
…e fragments

The 400-character ceiling was wide enough to hold two full sentences plus a
subordinate clause, so it never pushed against the one-sentence rule it was
meant to back — the #163 fragment reached 484 and still read as being in
the spirit of the limit. At 200 the ceiling and the rule push the same way.

Refits the eight unreleased fragments that were over. What came out is
detail the linked issue already carries: the eight service names in #161,
the CI-runner timing in #163, the DynamoDB and Lambda operation counts in
#167.

Also trims RELEASE.md's "right length" example, which was 203 characters
and would have failed the limit the paragraph above it states, and labels
both examples with their length.
skyoo2003 added a commit that referenced this pull request Sep 13, 2026
…g to 200 (#168)

* fix: give six changelog fragments the issue link the release tag checks

Six fragments wrote `Issue:` at the top level instead of under `custom:`,
where `.changie.yaml` declares it. `changie batch` rendered them as
`[#<no value>](.../issues/<no value>)`, which release.yml:234 rejects — the
tag would have failed at the notes-validation step, after the guardrails
had already passed.

The pre-flight grep in RELEASE.md (`grep -L 'Issue: "[0-9]'`) matches both
spellings, so it did not catch this.

Also trims the #163 fragment from 484 to 337 characters, under the 400-char
ceiling `.changie.yaml` sets and `changie new` enforces on fragments it
writes itself. The CI-runner timing detail it dropped is in the issue.

* docs: correct the two protocol-table rows that miscount EC2 as having no model

The protocol table said 12 services have no in-tree Smithy model and exactly
one speaks a protocol the parser cannot read. Counted against
internal/generated/fidelity/manifest_gen.go, it is 11 and two: `ec2` is
model-backed but speaks `aws.protocols#ec2Query`, which parser.go:441-453
does not recognise alongside the five it does.

That put the page at odds with fidelity-manifest.md:106 and
compatibility-policy.md:78, which both say 11, and left a core service's
missing engine coverage unexplained. Rows now sum to 431 either way, so the
arithmetic did not expose it, and no test gates this table.

Adds a paragraph separating what EC2 loses from what a registered-only
service loses: EC2 is served by a hand-written provider and is not in the
registered-only five, but its model-declared tail stays `unimplemented`
rather than falling back to `auto-crud`.

* docs: halve the changelog body ceiling to 200 characters and refit the fragments

The 400-character ceiling was wide enough to hold two full sentences plus a
subordinate clause, so it never pushed against the one-sentence rule it was
meant to back — the #163 fragment reached 484 and still read as being in
the spirit of the limit. At 200 the ceiling and the rule push the same way.

Refits the eight unreleased fragments that were over. What came out is
detail the linked issue already carries: the eight service names in #161,
the CI-runner timing in #163, the DynamoDB and Lambda operation counts in
#167.

Also trims RELEASE.md's "right length" example, which was 203 characters
and would have failed the limit the paragraph above it states, and labels
both examples with their length.
@skyoo2003 skyoo2003 mentioned this pull request Sep 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation services AWS service implementations tests Test code and test infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant