feat(lint): refuse a MISSPELLED format in a json_schema validation rule at publish time (#5178)

.changeset/json-schema-rule-unknown-format-gate.md

@@ -0,0 +1,98 @@
+---
+"@objectstack/lint": patch
+---
+
+fix(lint): a MISSPELLED `format` in a `json_schema` validation rule is refused at publish time (#5178)
+
+#5029 registered `ajv-formats` so a `json_schema` validation rule's `format`
+keyword is really enforced. It did **not** close the other half, and said so:
+under `strict: false` — which is load-bearing, because author-written schemas
+legitimately carry vendor keywords — an **unrecognised** format name is a
+non-event. ajv logs one line at compile time and **drops the keyword**:
+
+```
+$ node -e "const Ajv=require('ajv'); const addFormats=require('ajv-formats');
+  const ajv=new Ajv({allErrors:true,strict:false}); addFormats(ajv);
+  const v=ajv.compile({type:'object',properties:{e:{type:'string',format:'emial'}}});
+  console.log('validates {e: zzz} =', v({e:'zzz'}));"
+unknown format "emial" ignored in schema at path "#/properties/e"
+validates {e: zzz} = true
+```
+
+So an author who types `emial`, `e-mail`, `datetime` for `date-time`,
+`urireference` for `uri-reference`, `ipv_4` for `ipv4` or `Email` for `email`
+gets a rule that is declared, appears in the metadata, appears in every "what
+protects this object" listing, runs on every write, enforces `type` and
+`required` — and enforces nothing for the keyword they actually wrote. The
+record is **accepted**, which is the silent direction: nothing in the metadata,
+the UI or a test run says the constraint is inert. A typo is also the single
+most likely mistake in a hand-written or AI-generated JSON Schema.
+
+**New gate — `validateRuleSchemaFormats`**, a `gating` entry in
+`AUTHORING_RULES`, so it runs on all three authoring commands (`os validate`,
+`os build`, `os lint`) with no per-command wiring. One rule id:
+
+| id | fires when |
+|:---|:---|
+| `validation-rule-json-schema-unknown-format` | a `json_schema` rule's schema names a `format` the runtime's ajv has not registered |
+
+Each finding names the rule, the object, the **RFC 6901 JSON Pointer** to the
+offending keyword, and the **nearest registered name**:
+
+```
+objects.account.validations.support_shape.schema#/properties/email/format
+  `json_schema` validation 'support_shape' on object 'account' names
+  `format: 'emial'` at `#/properties/email/format`, which is not a registered
+  format. … the schema compiles, the rule ships and runs on every write, its
+  `type`/`required` keywords are enforced, and this constraint is enforced on
+  no record, ever. The record is ACCEPTED, so nothing downstream reports the
+  gap either.
+  hint: Did you mean `format: 'email'`? The registered names are: binary, byte,
+  date, date-time, … — the default `ajv-formats` set, the one
+  `rule-validator.ts` registers (#5029).
+```
+
+**The vocabulary is enumerated, never written down.** The registered set is read
+off a live instance of the very ajv the publish gate builds to mirror the
+runtime (`registeredFormatNames()`), not from a hardcoded list. A list would be
+a third opinion that nobody updates: the day `ajv-formats` adds a name, it
+starts refusing a format the write path enforces — a gate that turns working
+metadata red gets switched off, and then protects nothing. Enumerating means the
+gate follows the plugin across an upgrade with no edit at all.
+
+**The walk is JSON-Schema-aware, because `format` is not a magic word.** Every
+subschema position is visited — `properties`, `items` (both the 2020-12 schema
+form and draft-07's tuple array), `anyOf`/`allOf`/`oneOf`/`prefixItems`,
+`$defs`/`definitions`, `additionalProperties`, `patternProperties`,
+`if`/`then`/`else`, `not`, `contains`, `propertyNames`, `dependentSchemas`,
+draft-07 `dependencies` — at any depth, plus the rules nested in a
+`conditional`'s `then`/`otherwise`. Positions that hold arbitrary **data** are
+deliberately never read: a `format` inside `default`, `const`, `enum` or
+`examples` enforces nothing and was never meant to, so reporting it would invent
+a defect out of a legal document. A non-string `format` (`format: 42`) is left
+to `validation-rule-json-schema-uncompilable`, which already refuses it in
+ajv's own words.
+
+**The runtime is untouched, and so is the #4762/#5029 compile parity.** Option 2
+on #5178 (make an unknown format a runtime compile error) was rejected: it fires
+at runtime and fail-**open**, since `checkJsonSchema` catches, logs and skips —
+trading one silent gap for another. And this is a separate judgement laid
+*beside* the existing compile, never folded into it: `validateRuleCompilability`
+still compiles each schema in the runtime's exact environment and still
+**publishes** a typo'd format, because a typo'd format compiles there too. Its
+`#5029` pin ("a MISSPELLED format name is published, not refused") passes
+verbatim. Two questions, two rules — "does ajv accept this schema?" and "will
+this `format` keyword do anything?" — sharing one traversal and one ajv
+environment so they can never disagree about which rules exist or what
+"registered" means.
+
+`ajv` / `ajv-formats` stay **lazy**, and this rule is lazier than its neighbour:
+the registered vocabulary is only fetched once a schema actually names a
+`format`, so a `json_schema` rule that names none loads neither package. Pinned
+by the package's `lazy-deps.test.ts` in all three of its layers.
+
+**Upgrading:** if `os validate` / `os build` / `os lint` newly rejects a
+`json_schema` validation rule, the format name in it was never being enforced —
+fix the spelling to the name the finding suggests, or express the constraint
+with `pattern`, which is enforced. No metadata that was working changes
+behaviour.

packages/lint/src/authoring-rules.ts

@@ -123,6 +123,7 @@ import { validateOrgAxisRedLines } from './validate-org-axis-red-lines.js';
 import { validateSharingRuleEnforceability } from './validate-sharing-rule-enforceability.js';
 import { validateRlsPredicateEnforceability } from './validate-rls-predicate-enforceability.js';
 import { validateRuleCompilability } from './validate-rule-compilability.js';
+import { validateRuleSchemaFormats } from './validate-rule-schema-formats.js';
 import { validateActionLocations } from './validate-action-locations.js';
 import { lintFlowPatterns } from './lint-flow-patterns.js';
 import { lintLivenessProperties } from './lint-liveness-properties.js';
@@ -924,6 +925,28 @@ export const AUTHORING_RULES: readonly AuthoringRule[] = [
     surfaceReason: RUNTIME_OBJECT_WRITES_P2,
     run: (stack) => validateRuleCompilability(stack),
   },
+  // #5178 — the residual half of #5029, which registering `ajv-formats` does
+  // NOT close: under `strict: false` a MISSPELLED format name (`emial`) is
+  // logged once and DROPPED, so the rule compiles, ships, runs on every write
+  // and enforces nothing for the keyword its author wrote — and the record is
+  // accepted, which is the silent direction. Deliberately its own entry rather
+  // than a third finding inside the rule above: that one's whole contract is
+  // compiling in the runtime's exact environment, and a typo'd format compiles
+  // there. This judges the format NAME against the registered set (enumerated
+  // from the same ajv instance, never a hardcoded list) and compiles nothing,
+  // so the #4762/#5029 compile parity is untouched — a judgement beside the
+  // compile, not a divergent compile. Gating for the `lint-flow-patterns.ts`
+  // bar: no reading of the metadata behaves as written.
+  {
+    name: 'validateRuleSchemaFormats',
+    tier: 'gating',
+    input: 'parsed',
+    commands: ALL,
+    source: 'packages/lint/src/validate-rule-schema-formats.ts',
+    surfaces: CLI_ONLY,
+    surfaceReason: RUNTIME_OBJECT_WRITES_P2,
+    run: (stack) => validateRuleSchemaFormats(stack),
+  },
 ];
 
 // ─── Runner ─────────────────────────────────────────────────────────

packages/lint/src/index.ts

@@ -296,8 +296,26 @@ export {
 export type {
   RuleCompilabilityFinding,
   RuleCompilabilitySeverity,
+  WalkedValidationRule,
 } from './validate-rule-compilability.js';
 
+// #5178 — the residual half #5029's `ajv-formats` registration does not close.
+// Under `strict: false` an UNRECOGNISED format name is logged once and the
+// keyword is DROPPED, so `format: 'emial'` leaves the rule declared, running on
+// every write, and enforcing nothing — with the record ACCEPTED. Judged here
+// against the names that ajv instance really has registered, beside the compile
+// above rather than inside it, so the runtime/gate compile parity is untouched.
+export {
+  validateRuleSchemaFormats,
+  nearestRegisteredFormat,
+  MAX_SCHEMA_WALK_DEPTH,
+  VALIDATION_RULE_SCHEMA_UNKNOWN_FORMAT,
+} from './validate-rule-schema-formats.js';
+export type {
+  RuleSchemaFormatFinding,
+  RuleSchemaFormatSeverity,
+} from './validate-rule-schema-formats.js';
+
 export { validateNavAccess, NAV_OBJECT_UNGRANTED } from './validate-nav-access.js';
 export type { NavAccessFinding, NavAccessSeverity } from './validate-nav-access.js';
 

packages/lint/src/lazy-deps.test.ts

@@ -20,6 +20,12 @@
 //     i.e. it must be listed here even though its own weight would not justify
 //     it.
 //
+// #5178 added a second consumer of both — `validate-rule-schema-formats.ts`,
+// which asks that same ajv environment which `format` names it registered. It
+// is LAZIER than the compile gate on purpose: the registered set is only needed
+// once a schema actually names a `format`, so a `json_schema` rule that names
+// none pays nothing, which is the case pinned below.
+//
 // "A react page" was the whole story when this file was written; it is not any
 // more, and the cases below say which trigger they are pinning. Keep them
 // named that way — a test called "until a react page is validated" that also
@@ -117,6 +123,12 @@ describe('lazy dependency loading (kernel boot-path contract)', () => {
       mod.validateRuleCompilability(ruleStack({ type: 'format', name: 'f', field: 'payload', regex: '([', message: 'm' }));
       if (loaded('ajv')) fail('the rule-compilability gate must not load ajv to judge a format rule');
       if (loaded('ajv-formats')) fail('the rule-compilability gate must not load ajv-formats to judge a format rule');
+      // #5178 — and the format-NAME gate asks for the registered set only when
+      // a schema actually names a format, so this one costs nothing either.
+      mod.validateRuleSchemaFormats(
+        ruleStack({ type: 'json_schema', name: 'j0', field: 'payload', schema: { type: 'object' }, message: 'm' }),
+      );
+      if (loaded('ajv')) fail('the format-name gate must not load ajv for a schema that names no format (#5178)');
       const schemaFindings = mod.validateRuleCompilability(
         ruleStack({ type: 'json_schema', name: 'j', field: 'payload', schema: { type: 'not-a-type' }, message: 'm' }),
       );
@@ -125,6 +137,18 @@ describe('lazy dependency loading (kernel boot-path contract)', () => {
       if (!schemaFindings.some((f) => f.rule === 'validation-rule-json-schema-uncompilable')) {
         fail('rule-compilability gate produced no finding');
       }
+      const formatNameFindings = mod.validateRuleSchemaFormats(
+        ruleStack({
+          type: 'json_schema',
+          name: 'jf',
+          field: 'payload',
+          schema: { type: 'object', properties: { e: { type: 'string', format: 'emial' } } },
+          message: 'm',
+        }),
+      );
+      if (!formatNameFindings.some((f) => f.rule === 'validation-rule-json-schema-unknown-format')) {
+        fail('format-name gate produced no finding (#5178)');
+      }
       console.log('OK');
     };
   `;
@@ -162,6 +186,7 @@ describe('lazy dependency loading (kernel boot-path contract)', () => {
       validateHookBodyWrites,
       validateActionBodyWrites,
       validateRuleCompilability,
+      validateRuleSchemaFormats,
     } = await import('./index.js');
 
     // Stacks without a react-source page never touch either dep.
@@ -198,6 +223,15 @@ describe('lazy dependency loading (kernel boot-path contract)', () => {
       ).map((f) => f.rule),
     ).toEqual(['validation-rule-regex-uncompilable']);
 
+    // #5178 — nor does the format-NAME gate on a `json_schema` rule that names
+    // no `format`: it needs the registered vocabulary only when there is a name
+    // to judge, so the vocabulary is fetched last, not first.
+    expect(
+      validateRuleSchemaFormats(
+        ruleStack({ type: 'json_schema', name: 'j0', field: 'payload', schema: { type: 'object' }, message: 'm' }),
+      ),
+    ).toEqual([]);
+
     for (const dep of LAZY_DEPS) {
       expect(depLoaded(req.cache, dep), `${dep} loaded before any react-source or L2-body validation`).toBe(false);
     }
@@ -212,6 +246,20 @@ describe('lazy dependency loading (kernel boot-path contract)', () => {
     expect(depLoaded(req.cache, 'ajv-formats')).toBe(true);
     expect(schemaFindings.map((f) => f.rule)).toEqual(['validation-rule-json-schema-uncompilable']);
 
+    // …and the format-name gate, once a schema does name one, reaches the same
+    // already-loaded environment and still produces its finding (#5178).
+    expect(
+      validateRuleSchemaFormats(
+        ruleStack({
+          type: 'json_schema',
+          name: 'jf',
+          field: 'payload',
+          schema: { type: 'object', properties: { e: { type: 'string', format: 'emial' } } },
+          message: 'm',
+        }),
+      ).map((f) => f.rule),
+    ).toEqual(['validation-rule-json-schema-unknown-format']);
+
     // The first react page with source pays the cost of exactly its own gate's
     // dep — and the gates still work.
     const syntax = validateReactPages({

packages/lint/src/validate-rule-compilability.test.ts

@@ -410,7 +410,13 @@ describe('validateRuleCompilability — parity with the write path (#4762)', ()
     ).toEqual([]);
   });
 
-  it('a MISSPELLED format name is published, not refused — pinned as-is, not changed (#5029)', () => {
+  it('a MISSPELLED format name is published by THIS gate — the compile parity, unchanged (#5029/#5178)', () => {
+    // #5178 closed the gap and this assertion still reads `[]`, deliberately.
+    // The refusal lives in `validate-rule-schema-formats.ts`, a rule that
+    // compiles nothing and judges the format NAME against the registered set.
+    // Merging it into this gate would have made the publish gate disagree with
+    // the write path about what COMPILES, which is the one thing this file
+    // exists to prevent. If this ever goes red, that merge has happened.
     // Under `strict: false` ajv logs `unknown format "emial" ignored` and drops
     // the keyword — in the runtime AND here. So the gate must publish it: a
     // finding would be this gate inventing a verdict the write path does not
@@ -826,8 +832,12 @@ describe('validateRuleCompilability — reads only keys the spec declares (meta-
     // guards what the table lists.
     const receivers = [...new Set([...RULE_CODE.matchAll(/\b([a-z][\w$]*)\??\.[A-Za-z_$]/g)].map((m) => m[1]))];
     const tabled = new Set([...READ_SURFACES.map((r) => r.receiver), ...Object.keys(NOT_SCHEMA_RECEIVERS)]);
-    // Locals whose "keys" are JS methods / this file's own plumbing, not metadata.
-    const PLUMBING = new Set(['findings', 'v', 'out']);
+    // Locals whose "keys" are JS methods / this file's own plumbing, not
+    // metadata. `walked` and `names` are the two accumulators #5178 added when
+    // the rule walk and the registered-format vocabulary became shared handles
+    // (`walkObjectValidationRules`, `registeredFormatNames`) — array `.push` /
+    // `.length` / `.sort`, never a key off authored metadata.
+    const PLUMBING = new Set(['findings', 'v', 'out', 'walked', 'names']);
     expect(receivers.filter((r) => !tabled.has(r) && !PLUMBING.has(r))).toEqual([]);
 
     // …and no excuse outlives the read it excuses. A stale name in either list