Fix API catalog $ref dereferencing and add $defs deduplication#1
Closed
jpage-godaddy wants to merge 2 commits into
Closed
Fix API catalog $ref dereferencing and add $defs deduplication#1jpage-godaddy wants to merge 2 commits into
jpage-godaddy wants to merge 2 commits into
Conversation
**Fix: unresolved external $refs (was: 587 broken refs across 16 domains)**
The generator was silently preserving unresolved $refs on failure. Root
causes fixed:
- Local refs resolving to nodes that themselves contained $ref were never
recursively dereferenced; now a second dereference pass is applied
- Some spec repos store models at v2/models/ not v2/schemas/models/; added
sibling-directory fallback in resolve_ref()
- Spec files authored on macOS had wrong-case filenames (e.g.
Bulkingestion.yaml vs BulkIngestion.yaml); added case-insensitive path
resolution fallback
- Invalid YAML escape (\.) in double-quoted strings; added preprocessing
on parse retry
- Submodule content (transaction-models) was absent because clone used
--depth 1 without --recurse-submodules; fixed
Added collect_unresolved_refs() post-generation validator that fails the
build loudly with a full list of unresolved refs instead of silently
producing broken output.
Added CI static check in cicd.yml that greps committed schemas for
unresolved external $refs on every PR.
**Improvement: $defs deduplication (was: Metafield inlined 167×, Error 136×)**
External file refs and local #/components/schemas/ refs are now lifted into
a top-level $defs section instead of being fully inlined at every use site.
The same ref URL produces one $defs entry; all occurrences become local
{"$ref": "#/$defs/Name"} pointers. This is standard JSON Schema — any
compliant reader resolves local refs.
Result: orders.json 59,837 → 3,888 lines (-94%), bulk-operations.json
17,592 → 2,512 lines (-86%). Smaller catalog files mean a smaller binary
(schemas are include_str!'d at compile time).
Added indexmap dependency for stable $defs insertion order.
Added 8 regression tests covering all new behaviors.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Escape $defs keys as JSON Pointer segments (RFC 6901) when formatting $ref strings; keys containing ~ or / would otherwise resolve incorrectly - Handle \" inside YAML double-quoted scalars in fix_yaml_invalid_dot_escapes so an escaped quote doesn't prematurely terminate the scalar - Rewrite CI $ref check using `if grep | grep -v; then exit 1; fi` to avoid pipefail triggering on zero matches (grep exits 1 when nothing matches) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Owner
Author
|
Closing; real PR is godaddy#50 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
rust/schemas/api/*.jsonhad 587 unresolved external$refs across 16 of 20 domains, makinggodaddy api describereturn broken schema references. The TypeScript generator produced zero external refs; now the Rust generator matches.$defsdeduplication: instead of fully inlining every referenced schema at each use site, repeated schemas are lifted into a top-level$defssection and replaced with local{"$ref": "#/$defs/Name"}pointers. This is standard JSON Schema — any compliant reader resolves local refs.orders.jsonshrinks from ~60K lines to ~4K lines (−94%).Root causes fixed
$refwere never recursively dereferencedv2/models/notv2/schemas/models/; added sibling-directory fallbackBulkingestion.yamlvsBulkIngestion.yaml); added case-insensitive path fallback\.) in double-quoted strings; added preprocessing on parse retrytransaction-models) was absent because clone used--depth 1without--recurse-submodules; fixedSize reduction from
$defsdeduplicationorders.jsonbulk-operations.jsontransactions.jsonmetafields.jsonSchemas are
include_str!'d at compile time, so smaller files mean a smaller binary.Test plan
cargo test— 114 tests passcargo clippy -- -D warnings— cleancargo fmt --check— cleangrep -r '"[$]ref"' rust/schemas/api/*.json | grep -v '"#/' | wc -lreturns 0godaddy api describe orders list-orders— returns schemas with$ref: "#/$defs/..."local references (no broken file-path refs)🤖 Generated with Claude Code