Buy Ajv for schema linting, and serde_json_path for credential paths - #83
Merged
Conversation
Two hand-rolled implementations of things with reference implementations. `json-schema.ts` opened by calling itself "a deliberately useful subset of JSON Schema", and the subset was the problem: no `$ref`, no `minimum`, `pattern`, `uniqueItems`, `minLength`, `patternProperties`, `if`/`then`, `dependentSchemas`. Every one of those passed silently, so a body could be reported clean by us and rejected by the API that published the schema — the failure mode a linter exists to prevent. What stays hand-written is the part that is not JSON Schema, because Ajv is strict about its own spec and the documents we are handed are not: - 3.0's `nullable: true`, folded into 3.1's union type. Ignoring it would report a legitimate `null` as the wrong type. - `type` values that do not exist. One real 3.1 document arrives with `"type": "undefined"` 310 times, plus `emoji`, `icon`, `void` and `http`. Ajv throws on those at compile time, which would cost that schema all of its linting over fields nobody was going to check. Dropping the invented names keeps every valid constraint beside them enforceable. Messages are unchanged, including the "not string" half of a type error that Ajv leaves out — it names what was wanted, and naming what arrived is the other half of not having to go and look. While pinning behaviour I asserted that `additionalProperties` should see properties an `allOf` branch introduces. It should not: it is scoped to the `properties` beside it. Ajv was right and the assertion was wrong, so the test now pins the standard reading, with the surprise written down. `value_at` becomes `serde_json_path`, which reaches what a dotted path cannot: `$..id_token` for a blob whose depth you don't know, `$.keys[?(@.active == true)].secret` for a list you have to pick out of. The dotted form is kept as a fallback rather than migrated, because it is in every rule already on disk and it is *not* valid JSONPath — RFC 9535 wants `[0]` where we accept `.0`, and an object keyed "0" is reachable by one reading and not the other. Query first, walk second, no migration. An ambiguous query reports nothing rather than taking the first match: a rule that silently changes meaning as the document grows is worse than one that says it cannot tell.
`dependency audit` failed on a branch where both audits passed. The log has
no "Process completed with exit code" at all — no run step failed. The only
error is in a post step:
Path Validation Error: Path(s) specified in the action for caching
do(es) not exist, hence no cache is being saved.
The job reads the lockfile and never installs, so there is no pnpm store.
setup-node@v5 infers pnpm from `packageManager` and enables caching anyway,
and the failure needs a cache *miss* to show: a hit restores and skips the
save. So this passed for as long as the key kept hitting, and the first
lockfile change to miss turned a green audit into a red job.
release.yml's draft job already turns this off, for the same reason and
with the same explanation. The audit job wanted it too.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two hand-rolled implementations of things that have reference implementations.
Ajv replaces
src/lib/json-schema.tsThe file opened by describing itself as "a deliberately useful subset of JSON Schema". The subset was the problem: no
$ref,minimum,pattern,uniqueItems,minLength,patternProperties,if/then, ordependentSchemas. All passed silently — so a body could be reported clean by us and rejected by the API that published the schema, which is the one failure a linter exists to prevent.This is also where the message you saw came from. "Request body does not match the OpenAPI schema" is
+page.svelte, and "$.scope does not match any allowed schema" is this file'sanyOfbranch — not the API's. I said earlier that Fiber does no request validation; that was wrong.What stays hand-written is the part that isn't JSON Schema, since Ajv is strict about its own spec and real documents aren't:
nullable: truefolded into 3.1's union type. Ignoring it reports a legitimatenullas the wrong type.typevalues that don't exist. One real 3.1 document has"type": "undefined"310 times, plusemoji,icon,void,http. Ajv throws on those at compile time, costing that schema all of its linting over fields nobody was going to check. Dropping the invented names keeps every valid constraint beside them working.Messages are unchanged, including the "not string" half of a type error that Ajv omits — it names what was wanted; naming what arrived is the other half of not having to go and look.
One correction found by writing the tests: I asserted
additionalPropertiesshould see properties anallOfbranch introduces. It shouldn't — it's scoped to thepropertiesbeside it. Ajv was right; the test now pins the standard reading with the surprise written down.serde_json_pathreplacesauth::value_atReaches what the dotted form can't:
$..id_tokenfor a blob whose nesting depth you don't know,$.keys[?(@.active == true)].secretfor picking the entry that's current instead of pinning an index that moves.The dotted form is kept as a fallback, not migrated. It's in every capture rule already on disk and it is not valid JSONPath — RFC 9535 wants
$.data.tokens[0].valuewhere we accept.0— and an object keyed"0"is reachable by one reading and not the other. Query first, walk second, no migration.An ambiguous query reports nothing rather than taking the first match: a rule that silently changes meaning as the document grows is worse than one that admits it can't tell.
Not bought, with reasons
oas3/openapiv3for parsing — the same document has 374typevalues that aren't JSON Schema types. A typed deserializer rejects or coerces those; walkingserde_json::Valuedegrades field-by-field.oas3is also 3.1-only, and Swagger 2.0 and 3.0 both need to keep working.urlforstore::join_url— RFC 3986 joining drops the base path in 3 of 4 common cases (https://api.example.com/v1+/users→https://api.example.com/users). Base URLs with a path prefix are the norm, so it'd be a regression. The security-critical half (join_url_scoped's origin check) already usesurlvia reqwest.openapi::skeleton— no Rust crate does this; the options are a 0.1.0 wrapper around a JS library or 0.1.0 mock-server crates.http.rsmanual redirect walking — exists precisely because reqwest can't shed a custom credential header cross-origin.Testing
cargo fmt --check,cargo clippy --no-default-features --all-targets -- -D warnings,cargo test(137 passed, 3 new: JSONPath descent and filters, ambiguous-query, numeric object key via fallback),pnpm check(0 errors),pnpm test:e2e(322 passed, 6 new:$ref, constraints beyond the old subset, invented types not disabling the schema, a choice of consts, a property namedtype, and theadditionalPropertiescorrection).