fix(objectql,lint)!: json_schema 规则的 format 关键字真正生效 —— 注册 ajv-formats (#5029)

.changeset/json-schema-rule-format-enforced.md

@@ -0,0 +1,96 @@
+---
+"@objectstack/objectql": minor
+"@objectstack/lint": patch
+---
+
+fix(objectql,lint)!: a `json_schema` validation rule's `format` keyword is now ENFORCED — records that passed before can start failing (#5029)
+
+> **⚠️ BEHAVIOUR CHANGE ON DEPLOYED DATA — READ BEFORE UPGRADING.** A `format`
+> keyword inside a `json_schema` validation rule used to enforce **nothing**.
+> It now enforces. If any deployed object carries such a rule, writes that
+> succeeded on the previous version can be **rejected** after this upgrade —
+> including writes from flows, seeds, imports and integrations, not just the UI.
+> Nothing about the metadata changed; the runtime simply started honouring what
+> the metadata always said. See "Before you upgrade" below.
+
+## What was broken
+
+`packages/objectql/src/validation/rule-validator.ts` built its shared ajv as
+`new Ajv({ allErrors: true, strict: false })` and stopped there. In ajv 8 the
+`format` keyword is **not built in** — it ships in the separate `ajv-formats`
+package — and under `strict: false` an unregistered format is not an error: ajv
+logs one line at compile time and **drops the keyword**.
+
+So this rule:
+
+```ts
+{
+  type: 'json_schema',
+  name: 'support_config_shape',
+  field: 'support_config',
+  message: 'Support config is invalid.',
+  schema: {
+    type: 'object',
+    properties: { email: { type: 'string', format: 'email' } },
+    required: ['email'],
+  },
+}
+```
+
+compiled fine, ran on **every** write, enforced `type` and `required` — and
+enforced **nothing at all** for `format`. `{ email: 'not-an-email' }` was
+accepted, for every record, forever. The only signal was a stderr line at
+compile time naming no rule and no object.
+
+This is the #4649 / #4762 family one level in, and the partial failure is what
+made it nasty: the rule visibly rejects a bad `type` / missing `required`
+payload in dev, so it reads as *working* while the `format` half never fires.
+`format` is also one of the most reached-for JSON Schema keywords (`email`,
+`uri`, `uuid`, `date`, `date-time`, `ipv4`), so this was not an exotic corner —
+and it is exactly the shape an AI writing metadata reaches for first.
+
+## What changed
+
+- **`@objectstack/objectql`** now depends on `ajv-formats` and registers it on
+  the shared instance (`addFormats(ajv)`). The **default (full)** format set is
+  used deliberately: `fast` mode trades correctness for speed on precisely the
+  formats authors reach for most, and a format that "mostly" matches is the same
+  declared ≠ enforced defect with a smaller hole.
+- **`@objectstack/lint`** — the #4762 publish gate
+  (`validate-rule-compilability.ts`) compiles every `json_schema` rule with the
+  SAME ajv environment the runtime uses, on purpose, so it registers the same
+  plugin. This is not cosmetic parity: `ajv-formats` also installs the
+  `formatMinimum` / `formatMaximum` keywords, so a gate without it treats them
+  as unknown keywords (`strict: false` ⇒ silently ignored) and would publish a
+  schema the runtime then refuses to compile — a rule that passes review and
+  enforces nothing, which is the failure that gate exists to prevent. The parity
+  test now reads the plugin registration out of the runtime's source, so the two
+  cannot drift apart silently.
+
+**Authoring is unchanged.** `format` stays a legal, publishable JSON Schema
+keyword; the publish gate does not refuse it (option 2 on #5029 was considered
+and rejected — refusing standard JSON Schema would push authors into private
+spellings). What changed is only that the declaration is now true.
+
+## Before you upgrade
+
+1. Find the rules at risk: any `object.validations[]` entry with
+   `type: 'json_schema'` whose `schema` contains a `format` key, at any depth
+   (including inside `$defs` / `$ref` and a `conditional`'s `then` /
+   `otherwise` branch).
+2. For each, audit the existing column against that format. Rows already stored
+   are **not** re-validated — nothing is rejected retroactively, and no
+   migration runs — but the **next write that touches the field** is checked,
+   which includes an unrelated PATCH that merely resends the JSON blob.
+3. If a format was aspirational rather than real, remove that `format` key (or
+   relax it) *before* upgrading. Deleting the keyword is now a meaningful,
+   visible act rather than a no-op.
+
+## Known limitation, recorded deliberately
+
+A **misspelled** format name is still ignored. `format: 'emial'` compiles under
+`strict: false` — ajv logs `unknown format "emial" ignored` and drops it — in
+both the runtime and the publish gate, so a typo still enforces nothing. That
+behaviour is unchanged here and pinned by test in both packages, so it is a
+known boundary rather than an oversight; closing it is an authoring-time
+decision of its own and is tracked separately.

packages/lint/package.json

@@ -31,6 +31,7 @@
     "@objectstack/sdui-parser": "workspace:*",
     "@objectstack/spec": "workspace:*",
     "ajv": "^8.20.0",
+    "ajv-formats": "^3.0.1",
     "sucrase": "^3.35.1",
     "typescript": "^6.0.3"
   },

packages/lint/src/lazy-deps.test.ts

@@ -13,7 +13,12 @@
 //   - `ajv` (~2.4 MB installed), loaded by the #4762 publish gate
 //     (validate-rule-compilability.ts) only when a stack declares a
 //     `json_schema` validation rule — the one rule type whose static artifact
-//     needs a JSON-Schema compiler to judge.
+//     needs a JSON-Schema compiler to judge;
+//   - `ajv-formats`, loaded by the same gate for the same trigger (#5029). Small
+//     on its own, but it `require`s `ajv/dist/compile/codegen`, so an eager
+//     import of it would drag ajv onto the boot path through the back door —
+//     i.e. it must be listed here even though its own weight would not justify
+//     it.
 //
 // "A react page" was the whole story when this file was written; it is not any
 // more, and the cases below say which trigger they are pinning. Keep them
@@ -42,7 +47,7 @@ const distDir = join(srcDir, '..', 'dist');
 
 // Deps that must never load at import time. Extend this list when another
 // heavy, rarely-hit dependency joins the package.
-const LAZY_DEPS = ['typescript', 'sucrase', 'ajv'];
+const LAZY_DEPS = ['typescript', 'sucrase', 'ajv', 'ajv-formats'];
 
 const depLoaded = (cache: Record<string, unknown> | undefined, dep: string) =>
   Object.keys(cache ?? {}).some((p) => p.split(/[/\\]/).join('/').includes(`/node_modules/${dep}/`));
@@ -105,15 +110,18 @@ describe('lazy dependency loading (kernel boot-path contract)', () => {
       const props = mod.validateReactPageProps(${reactStack('function Page(){ return <ObjectForm mode="edit" />; }')});
       if (!loaded('typescript')) fail('typescript was not loaded by a react-page props validation');
       if (!props.some((f) => f.rule === 'react-prop-missing-required')) fail('props gate produced no finding');
-      // #4762 — the JSON-Schema compiler is the third lazy dep. A stack whose
-      // validation rules are all format rules never pays for it.
+      // #4762 — the JSON-Schema compiler is the third lazy dep, and #5029 added
+      // its formats plugin as the fourth. A stack whose validation rules are all
+      // format rules never pays for either.
       const ruleStack = (validation) => ({ objects: [{ name: 'a', fields: { payload: {} }, validations: [validation] }] });
       mod.validateRuleCompilability(ruleStack({ type: 'format', name: 'f', field: 'payload', regex: '([', message: 'm' }));
       if (loaded('ajv')) fail('the rule-compilability gate must not load ajv to judge a format rule');
+      if (loaded('ajv-formats')) fail('the rule-compilability gate must not load ajv-formats to judge a format rule');
       const schemaFindings = mod.validateRuleCompilability(
         ruleStack({ type: 'json_schema', name: 'j', field: 'payload', schema: { type: 'not-a-type' }, message: 'm' }),
       );
       if (!loaded('ajv')) fail('ajv was not loaded by a json_schema validation-rule check');
+      if (!loaded('ajv-formats')) fail('ajv-formats was not loaded by a json_schema validation-rule check (#5029 parity)');
       if (!schemaFindings.some((f) => f.rule === 'validation-rule-json-schema-uncompilable')) {
         fail('rule-compilability gate produced no finding');
       }
@@ -194,11 +202,14 @@ describe('lazy dependency loading (kernel boot-path contract)', () => {
       expect(depLoaded(req.cache, dep), `${dep} loaded before any react-source or L2-body validation`).toBe(false);
     }
 
-    // The first `json_schema` rule pays for ajv — and the gate still works.
+    // The first `json_schema` rule pays for ajv AND its formats plugin (#5029 —
+    // the gate compiles in the runtime's environment, which now registers
+    // formats) — and the gate still works.
     const schemaFindings = validateRuleCompilability(
       ruleStack({ type: 'json_schema', name: 'j', field: 'payload', schema: { type: 'not-a-type' }, message: 'm' }),
     );
     expect(depLoaded(req.cache, 'ajv')).toBe(true);
+    expect(depLoaded(req.cache, 'ajv-formats')).toBe(true);
     expect(schemaFindings.map((f) => f.rule)).toEqual(['validation-rule-json-schema-uncompilable']);
 
     // The first react page with source pays the cost of exactly its own gate's

packages/lint/src/runtime-lazy-deps.test.ts

@@ -35,7 +35,11 @@ const distDir = join(srcDir, '..', 'dist');
 // is CLI-only (`surfaceReason: RUNTIME_OBJECT_WRITES_P2`). So the boot path must
 // not pay for it — not at import, and not while gating. Should that rule ever be
 // widened to `runtime-publish`, this assertion is what says so out loud.
-const LAZY_DEPS = ['typescript', 'sucrase', 'ajv'];
+// `ajv-formats` joined in #5029 for the same gate and the same trigger: the gate
+// compiles in the runtime's ajv ENVIRONMENT, and that environment now registers
+// the formats plugin. It carries ajv in with it, so listing it here is not
+// belt-and-braces — it is the second door onto the same load.
+const LAZY_DEPS = ['typescript', 'sucrase', 'ajv', 'ajv-formats'];
 
 const depLoaded = (cache: Record<string, unknown> | undefined, dep: string) =>
   Object.keys(cache ?? {}).some((p) => p.split(/[/\\]/).join('/').includes(`/node_modules/${dep}/`));

packages/lint/src/validate-rule-compilability.test.ts

@@ -363,10 +363,114 @@ describe('validateRuleCompilability — parity with the write path (#4762)', ()
     ]);
     expect(RUNTIME_AJV_OPTIONS).toEqual({ allErrors: true, strict: false });
 
+    // …and the PLUGINS half (#5029). Options alone stopped describing the
+    // environment the day the runtime registered `ajv-formats`: the plugin adds
+    // the `format` implementations AND the `formatMinimum`/`formatMaximum`
+    // keywords, so two instances built from identical options can still disagree
+    // about whether a schema compiles. Read the registration out of the runtime's
+    // source for the same reason the options are read out of it.
+    expect(
+      source,
+      'the runtime no longer registers ajv-formats — this gate must drop it too, or it starts disagreeing',
+    ).toContain('addFormats(ajv)');
+    // The default (full) format set, not `fast` mode: a plugin call carrying an
+    // options object is a different environment and must be mirrored, not
+    // assumed. (`addFormats(ajv, {...})` would fail this.)
+    expect(source).toMatch(/addFormats\(ajv\)\s*;/);
+
     // …and the regex half: the runtime compiles the raw source with no flags.
     expect(source).toContain('new RegExp(rule.regex)');
   });
 
+  it('compiles a `format`-bearing schema exactly as before — the gate publishes it (#5029)', () => {
+    // The #5029 change is a RUNTIME enforcement change, not an authoring
+    // restriction: `format: 'email'` is standard JSON Schema and stays
+    // publishable. Option 2 on that issue (refuse `format` at publish time) was
+    // rejected, and this is the pin that says so — if the gate ever starts
+    // rejecting these, it is rejecting legal metadata the write path enforces.
+    expect(
+      ids(
+        objectWith({
+          type: 'json_schema',
+          name: 'support_config_shape',
+          field: 'support_config',
+          message: 'm',
+          schema: {
+            type: 'object',
+            properties: {
+              email: { type: 'string', format: 'email' },
+              id: { type: 'string', format: 'uuid' },
+              at: { type: 'string', format: 'date-time' },
+              site: { type: 'string', format: 'uri' },
+            },
+            required: ['email'],
+          },
+        }),
+      ),
+    ).toEqual([]);
+  });
+
+  it('a MISSPELLED format name is published, not refused — pinned as-is, not changed (#5029)', () => {
+    // Under `strict: false` ajv logs `unknown format "emial" ignored` and drops
+    // the keyword — in the runtime AND here. So the gate must publish it: a
+    // finding would be this gate inventing a verdict the write path does not
+    // share, which is the `strict: true` mistake wearing a different hat. This
+    // records the residual gap (a typo'd format still enforces nothing) as a
+    // KNOWN, deliberate boundary of #5029 rather than an oversight; closing it
+    // is an authoring-time decision of its own.
+    expect(
+      ids(
+        objectWith({
+          type: 'json_schema',
+          name: 'typo_format',
+          field: 'support_config',
+          message: 'm',
+          schema: { type: 'object', properties: { email: { type: 'string', format: 'emial' } } },
+        }),
+      ),
+    ).toEqual([]);
+  });
+
+  it('judges `formatMinimum` the way the runtime does — the keyword only EXISTS with ajv-formats (#5029)', () => {
+    // The case that makes the plugin half of the parity load-bearing rather
+    // than tidy. `ajv-formats` registers `formatMinimum`/`formatMaximum` with a
+    // metaschema; without the plugin they are unknown keywords, and
+    // `strict: false` ignores an unknown keyword silently. So a gate lacking the
+    // plugin publishes a malformed `formatMinimum` that the runtime then refuses
+    // to compile — the rule ships, is skipped for every record, and this file's
+    // whole reason for existing is defeated. A WELL-FORMED use stays green.
+    expect(
+      ids(
+        objectWith({
+          type: 'json_schema',
+          name: 'window_ok',
+          field: 'support_config',
+          message: 'm',
+          schema: {
+            type: 'object',
+            properties: { from: { type: 'string', format: 'date', formatMinimum: '2020-01-01' } },
+          },
+        }),
+      ),
+    ).toEqual([]);
+    // …and the malformed one is refused, which is only possible because the
+    // keyword is registered.
+    expect(
+      ids(
+        objectWith({
+          type: 'json_schema',
+          name: 'window_bad',
+          field: 'support_config',
+          message: 'm',
+          schema: {
+            type: 'object',
+            properties: { from: { type: 'string', format: 'date', formatMinimum: 42 } },
+          },
+        }),
+      ),
+    ).toEqual([VALIDATION_RULE_SCHEMA_UNCOMPILABLE]);
+  });
+
   it('rejects exactly the two artifacts the runtime still skips', () => {
     // The runtime's own fail-open pin (`rule-fail-closed.test.ts` › `#4649 —
     // unchanged neighbours`) uses these two fixtures to record that the WRITE