test: create api fixture validator#605
Open
davidcreated wants to merge 1 commit into
Open
Conversation
Add a fixture validator that checks the JSON fixtures under e2e/fixtures against Zod schemas mirroring the current API/domain shapes (frontend/src/types), catching fixture drift before it reaches tests. - backend/src/testing/fixtureValidator: schemas, registry, validator, and readable diff formatting; classifies findings as errors (missing/mistyped required fields, bad enums, invalid JSON) or warnings (fields no longer in the API shape) - backend/scripts/validate-fixtures.ts CLI with --json and --strict, exits non-zero on drift; wired up as the "fixtures:validate" npm script - Run in CI (node-ci job) right after lint, and as a backend unit test - Fix the stale asset-health fixture, which was missing the required HealthScore.lastUpdated field - Document the check and the fixture update workflow in backend/docs/FIXTURE_VALIDATION.md Closes StellaBridge#561
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements an API Fixture Validator that checks the JSON fixtures under
e2e/fixtures/against Zod schemas mirroring the current API/domain shapes (the canonical types infrontend/src/types). This catches fixture drift — fields added, removed, renamed, or retyped in the API but not in the fixtures — before stale fixtures silently weaken the tests.Closes #561
What's included
backend/src/testing/fixtureValidator/)schemas.ts— Zod schemas mirroringAsset,HealthScore,HealthFactors,Bridge, and the fixture-file envelopes.registry.ts— maps each fixture file to the schema describing its API shape.validator.ts— reads, parses, and validates fixtures; emits readable, path-pointed diffs (e.g.USDC.factors.liquidityDepth).format.ts— CI-friendly text report.error— missing/mistyped required field, invalid enum, unreadable/invalid JSON → fails CI.warning— a property no longer part of the current API shape → fails only under--strict.npm --workspace=backend run fixtures:validate(supports--jsonand--strict), exits non-zero on drift.node-cijob right after lint.backend/tests/testing/fixtureValidator.test.ts(validates every registered fixture + covers missing field, wrong type, bad enum, unexpected property, and nested record paths).backend/docs/FIXTURE_VALIDATION.mdcovering how to run it, the update workflow when the API changes, and how to add a new fixture.Drift fixed in this PR
While building the validator it flagged real drift:
e2e/fixtures/asset-health.jsonwas missing the requiredHealthScore.lastUpdatedfield. Fixed so the committed fixtures match the current shape.Verification
npm --workspace=backend run lint✅npm --workspace=backend run build✅npm --workspace=backend run fixtures:validate→ PASS ✅npm --workspace=backend run test -- tests/testing→ 9/9 passing ✅How to extend
Add a fixture file, add a schema to
schemas.ts, and register it inregistry.ts— it's then covered by both the CLI check and the test suite.