Repository navigation
feat: derive service aliases from models and onboard the AI/ML category - #137
Merged
Merged
Conversation
Onboarding a service needed a hand-written entry in a 63-clause switch in
the gateway whenever its X-Amz-Target prefix did not already resolve.
rekognition could not be onboarded at all: its prefix is
RekognitionService, and normalizeServiceID had no case for it.
RED
normalizeServiceID("rekognitionservice") = "rekognitionservice", want "rekognition"
serviceFromTarget("RekognitionService.ListCollections") = "rekognitionservice", want "rekognition"
plus compile-time: ir.Model has no ShapeName/SigningName/EndpointPrefix/
ARNNamespace/CloudFormationName/SDKID; BuildAliases undefined;
internal/generated/aliases does not exist.
GREEN
ParseSmithyJSON carries the six identifiers a caller can address a service
by. BuildAliases derives alias -> service ID from them and reports, rather
than resolves, the 9 aliases more than one service claims. The gateway
consults a documented override map (contested / legacy / substitution)
then the generated table.
93 aliases that routed before still route to the same service.
rekognition onboards with 0 hand-written lines and answers a boto3 call.
go test ./... green; 777 boto3 compat tests pass (776 + rekognition smoke).
make codegen is idempotent (hashed across two runs).
One regression was caught by the compat suite and not by the unit tests:
mapping "ses" to sesv2 in the override map sent every SES v1 Query call to
the v2 provider. SES v1 and SESv2 publish identical identifiers and only
the protocol separates them, which DetectProtocol already does. The four
aliases involved are now pinned in the golden list.
Refs: .claude/plans/aws-service-coverage-100-milestone-2.plan.md (Tasks 1-3)
All 48 upstream AI/ML models are registered: 42 downloaded, generated and scaffolded with no hand-written code. 105 -> 147 registered services. Three defects the category surfaced, each fixed at its root: smithy.api#Primitive* was unmapped in smithyToGoType, so bedrockagent and omics emitted undefined identifiers and the build failed. All seven prelude primitives are now mapped, not only the two that happened to appear. Every runtime split-out signs with its parent's name, so sagemaker was claimed by 8 services and bedrock by 6, and BuildAliases reported them all as unresolvable. They are not: when exactly one claimant IS the alias, that service names itself and the rest borrow. selfNamedClaimant settles 10 of the 18 collisions from the models, and removes 3 hand-written overrides. The 8 that remain have no self-named claimant and stay a human decision. TestFidelityManifestCoverage required every registered service to serve something, which the 30 new rest-json services cannot. The manifest now carries each service's Protocol, and the invariant is the sharper one: the CRUD engine holds operations for this service but the manifest serves none. forecastquery is json-1.1 and legitimately serves nothing — its whole API is QueryForecast/QueryWhatIfForecast, neither CRUD-shaped. Registered is not covered, and docs/coverage.md never prints the first number without the other two: 148 registered, 117 serving >=1 operation, 31 registered-only. test_registered_only_service_declines_cleanly asserts a registered-only service answers with an AWS error code, never a fabricated 200. Measured, 105 -> 147 services: binary 30.8 -> 31.3 MiB, cold start ~460 ms unchanged, idle RSS 57.3 -> 57.7 MiB. Extrapolated to 431: ~35 MiB binary, ~61 MiB RSS, startup flat. CI: go test ./... 22s, codegen 2s, build 1s, 790 boto3 compat tests in 9s. Refs: .claude/plans/aws-service-coverage-100-milestone-2.plan.md (Tasks 4-7)
`make codegen` emitted a different healthlake/types.go and crudregistry/registry_gen.go on every run. healthlake.json bundles two namespaces — com.amazonaws.healthlake and com.amazon.awshealthlakedatatransformationfrontendservice — with 8 colliding short names. model.Shapes is keyed by short name and the loop iterated a Go map, so the last writer won at random: ResourceArn or ResourceARN, TagList or TagMap, differently each time. It is the only model of 137 with this property, which is why it survived until a whole category was onboarded at once. Sorting alone would be deterministic and still wrong: com.amazon.awshealthlake… sorts before com.amazonaws.healthlake, so it would pick the foreign shape every time. Sorted iteration plus service-namespace precedence — the model's own service defines its API; a second bundled namespace is a dependency. RED: 50 iterations, all "expected Mine, actual Theirs". GREEN: three consecutive `make codegen` runs hash identical. Refs: .claude/plans/aws-service-coverage-100-milestone-2.plan.md (Task 4)
5 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Derives the gateway's service-alias table from the Smithy models instead of a hand-maintained 63-clause switch, then uses that to onboard the entire AI/Machine Learning category — all 48 upstream models, 42 of them new, with zero hand-written code. Registered services go 105 → 148.
This is Milestone 2 of
.claude/prds/aws-service-coverage-100.prd.md("One category proven end to end"), and it closes that PRD's Open Question 2 (service ID aliasing) and Open Question 3 (is 431 the right denominator).Related Issue
Refs #136 — this is the follow-on milestone to "repair AWS service onboarding and measure its cost", which measured 3s/service but only for services whose
X-Amz-Targetprefix already resolved.rekognitionfailed outright there; it works here.Fixes #
Changes
Alias derivation (the blocker)
ir.Modelcarries the six identifiers a caller can address a service by: shape name, signing name, endpoint prefix, ARN namespace, CloudFormation name, SDK id. Named provider-neutrally — a non-AWS source leaves them empty.codegen.BuildAliasesderivesalias → service IDand emitsinternal/generated/aliases/aliases_gen.go. It reports contested aliases rather than resolving them: picking one claimant silently sends the other's traffic to the wrong provider.normalizeServiceIDis now a 36-entry override map (contested / legacy / substitution, each group documented) plus the generated table. It no longer grows per service.selfNamedClaimantsettles the common false collision: every runtime split-out signs with its parent's name, sosagemakeris claimed by 8 services — but exactly one of them issagemaker. That's a fact in the models, not a preference. It resolves 10 of 18 collisions and removed 3 overrides that had been written by hand.AI/ML category
Honesty about what "registered" means
Protocol, so registered and covered are separable numbers.docs/coverage.mdpublishes registered / serving ≥1 / registered-only and never the first alone.TestFidelityManifestCoverageused to require every registered service to serve something. That is false by design oncerest-jsonservices are registered, so it is now the sharper check: the CRUD engine holds operations for this service and the manifest serves none — which is wiring, and nothing else.Three defects the category surfaced, each fixed at its root
smithy.api#Primitive*was unmapped insmithyToGoType, sobedrockagentandomicsemitted undefined identifiers and the build failed. All seven prelude primitives are mapped, not only the two that broke.make codegenwas not idempotent:healthlake.jsonbundles two namespaces with 8 colliding short names, andmodel.Shapesis keyed by short name while iterating a Go map. Fixed with sorted iteration plus service-namespace precedence — sorting alone is deterministic and still wrong, because the foreign namespace sorts first.ses → sesv2sent every SES v1 Query call to the v2 provider. SES v1 and SESv2 publish identical identifiers and only the protocol separates them, whichDetectProtocolalready handles.Test Plan
CGO_ENABLED=0 go test ./...— all packages pass.pytest tests/compatibility— 790 passed, 2 skipped (776 baseline + 14 new).make codegenrun three times, hashed — byte-identical (eca3ebfd…). This is a new guarantee; it was broken before this PR.bash scripts/generate-imports.sh— idempotent.rekognitionend to end: download → codegen → scaffold → build → boto3ListCollectionsreturns 200, with zero hand-written gateway lines.test_registered_only_service_declines_cleanly).Measured (105 → 147 services):
CI cost at 147:
go test ./...22s, compat suite 9s, codegen 2s, build 1s. The single-binary, zero-config property survives the full 431 target.Reviewing this PR: 277 files, but 261 are generated packages, downloaded models, or scaffolds. The reviewable surface is 16 files — everything outside
internal/generated/,smithy-models/, andinternal/services/.Checklist
docs/coverage.md)Known gaps
lexroutes nowhere. All four Lex services sign aslexand none is named it; all arerest-json, so none can be engine-served. Each still routes under its own unambiguous name. Distinguishing them needs URL-path routing, which is only worth writing once one of them has a provider.amazonrdsv19andcognitoare likewise deliberately unrouted — 2 of 3 and 1 of 2 claimants would be wrong. An empty override value means "decided, and the decision is not to guess".Evidence, with RED/GREEN output verbatim for all 23 guarantees:
.claude/tdd/aws-service-coverage-100-milestone-2.tdd.md(local,.claude/is gitignored).