feat(spec): $icontains(ASCII 折叠域)+ $contains 族钉死大小写敏感 + $regex 退役指引表 (#5701)

.changeset/filter-icontains-and-regex-retirement.md

@@ -0,0 +1,79 @@
+---
+"@objectstack/spec": minor
+---
+
+feat(spec): `$icontains` 入算子词表(ASCII 折叠域)、`$contains` 族钉死为大小写敏感、`$regex` 退役指引表(#5701)
+
+#4706 维护者裁决 B 案的**契约半边**。本次只改声明,不改任何运行时行为:
+五个后端今天怎么答,落地后还怎么答。驱动侧下译归 #5702,`$regex` 唯一活生产者的
+翻转归 #5710。
+
+## 1. 新增 `$icontains` —— 折叠域是 ASCII,不是 Unicode
+
+`StringOperatorSchema` / `FieldOperatorsSchema` / `Filter<T>` 新增 `$icontains`:
+忽略大小写的子串包含,**只折叠 `A-Z` 与 `a-z`**。
+
+```ts
+{ name: { $contains:  'acme' } }   // 大小写敏感:匹配 "acme corp",不匹配 "ACME Corp"
+{ name: { $icontains: 'acme' } }   // 折叠 ASCII 大小写:两者都匹配
+```
+
+**边界必须说清楚:`café` 不匹配 `CAFÉ`。** ASCII 以外一律按字面比较。
+选 ASCII 而非全 Unicode,是因为它是五个后端唯一都能真兑现的折叠域 ——
+无 ICU 的 SQLite(`driver-sqlite-wasm` / `driver-turso` 跑的就是它)的
+`LOWER()` 与 `LIKE` 只折叠 ASCII,承诺 Unicode 等于承诺三个后端做不到的事,
+那正是 #4706 用来否决「五后端真正则」的同一条判据。
+
+比较值一律**字面量**:`%` / `_` 不是 LIKE 通配符,`.` / `*` 不是正则元字符 ——
+`{ name: { $icontains: 'a.b' } }` 匹配 `a.b`,不匹配 `axb`。
+
+## 2. `$contains` / `$notContains` / `$startsWith` / `$endsWith` = 大小写敏感
+
+这条**取代**了 `filter.zod.ts` 里那句已记录的声明(Prime Directive #13,
+取代记录写在原处):
+
+> Note: Case sensitivity should be handled at backend level.
+
+那不是漏写,是写下来的「不保证」,实测代价是同一个算子三种答案:
+`driver-memory` 的参考匹配器与 `formula` 大小写敏感,`driver-mongodb` 硬编码
+`$options: 'i'` 全 Unicode 不敏感,SQL 家族看方言(SQLite 折叠 ASCII、
+Postgres 不折叠、MySQL 看 collation)。作者无法从算子名判断自己拿到哪一种。
+
+**迁移**:此前依赖某后端偶然大小写不敏感的 `$contains` 查询,应改写为
+`$icontains`。行为在 #5702 落地前不变,所以这是一次可以提前做的改写,不是断裂。
+
+## 3. `$regex` / `$options` 退役 —— 指引表 `RETIRED_FILTER_OPERATORS`
+
+`$regex` 从来不在 `FILTER_OPERATORS` 里,却有一个生产者、四个消费者,而且各读各的:
+`driver-sql` 编译成 LIKE 转义后的子串匹配(`a.b` 只匹配字面 `a.b`),
+`driver-memory` 当真正则求值(`a.b` 还匹配 `axb`;模式非法则被 `catch` 成零行,
+无声)。真正则在五后端不可实现 —— `driver-turso` 的 remote 线协议无法注册
+SQLite `REGEXP` 函数。
+
+新增 `RETIRED_FILTER_OPERATORS`(**纯数据**,不引入任何拒收行为),
+给出逐条处方,供五个既有拒收点引用同一句话:
+
+| 原写法 | 改写为 |
+|:---|:---|
+| `{ name: { $regex: 'acme' } }` | `{ name: { $icontains: 'acme' } }` |
+| `{ name: { $regex: 'acme', $options: 'i' } }` | `{ name: { $icontains: 'acme' } }` |
+| `{ name: { $regex: '^acme' } }` | `{ name: { $startsWith: 'acme' } }` |
+| `{ name: { $regex: 'acme$' } }` | `{ name: { $endsWith: 'acme' } }` |
+
+真正需要正则的查询没有 filter 层替代物:用已声明算子收窄,再在应用代码里匹配。
+
+## 4. 新姊妹 case-set `FILTER_TEXT_CASES`
+
+`filter-text-conformance.ts` —— 大小写折叠、字面比较值、`$regex` 拒收的共享标准,
+带 `expectRejection` 判别式(`FILTER_LOGIC_CASES` 刻意没长出来的那个形状,
+其表头三条章程原样保留)。五个 driver 各记一条**实测** DEBT 台账,指向 #5702。
+
+## 什么**没有**变
+
+`$icontains` 暂不进 `FILTER_OPERATORS`。那个数组不是词表而是运行时白名单 ——
+`driver-memory` 的 `SUPPORTED_FIELD_OPERATORS` 由它派生。实测:提前把
+`$icontains` 放进去,该驱动的形状门禁就不再拒收它,而匹配器没有对应分支,
+`match({name:'zzz'}, {name:{$icontains:'acme'}})` 返回 `true` —— 谓词被静默丢弃,
+全表命中。谓词被丢不是收窄而是**放大**,在 RLS 读作用域上是越权读(#3948)。
+所以它随 #5702 的实现一起入列,`filter-operator-vocabulary.test.ts` 把这处差异
+钉死为恰好 `{ $icontains }`,清偿时该断言会红,提醒作者一并删掉过渡说明。

content/docs/protocol/objectql/query-syntax.mdx

@@ -237,14 +237,82 @@ const query: QueryAST = {
 | `$lte` | Less or equal | `{ discount: { $lte: 20 } }` |
 | `$in` | In list | `{ stage: { $in: ['proposal', 'negotiation'] } }` |
 | `$nin` | Not in list | `{ status: { $nin: ['deleted', 'archived'] } }` |
-| `$contains` | String contains | `{ name: { $contains: 'Inc' } }` |
-| `$notContains` | String does not contain | `{ name: { $notContains: 'test' } }` |
-| `$startsWith` | String starts with | `{ email: { $startsWith: 'admin' } }` |
-| `$endsWith` | String ends with | `{ domain: { $endsWith: '.com' } }` |
+| `$contains` | String contains, **case-sensitive** | `{ name: { $contains: 'Inc' } }` |
+| `$icontains` | String contains, **ignoring ASCII case** | `{ name: { $icontains: 'inc' } }` |
+| `$notContains` | String does not contain, **case-sensitive** | `{ name: { $notContains: 'test' } }` |
+| `$startsWith` | String starts with, **case-sensitive** | `{ email: { $startsWith: 'admin' } }` |
+| `$endsWith` | String ends with, **case-sensitive** | `{ domain: { $endsWith: '.com' } }` |
 | `$between` | Range (inclusive) | `{ close_date: { $between: ['2024-01-01', '2024-12-31'] } }` |
 | `$null` | Null check | `{ manager_id: { $null: true } }` / `{ phone: { $null: false } }` |
 | `$exists` | Field exists (NoSQL) | `{ metadata: { $exists: true } }` |
 
+### Case Sensitivity
+
+The string operators compare **case-sensitively**. `$icontains` is the one that does
+not, and the case it ignores is **ASCII case only** — `A-Z` against `a-z`, and nothing
+else.
+
+```typescript
+// Case-sensitive: matches "acme corp", NOT "ACME Corp"
+{ name: { $contains: 'acme' } }
+
+// ASCII case-insensitive: matches BOTH "acme corp" and "ACME Corp"
+{ name: { $icontains: 'acme' } }
+```
+
+<Callout type="warn">
+  **`café` does not match `CAFÉ`.** Outside `A-Z`/`a-z`, `$icontains` compares
+  literally — accented Latin, Cyrillic, Greek and every other script are matched
+  exactly as written. If your users search non-ASCII text, `$icontains` is not an
+  accent- or case-blind search, and treating it as one will silently return fewer
+  rows than expected.
+
+  The boundary is ASCII because that is the only fold every backend can actually
+  deliver. SQLite compiled without ICU — which is what `driver-sqlite-wasm` and
+  `driver-turso` run on — folds ASCII only in both `LOWER()` and `LIKE`, so a
+  Unicode promise here would be a guarantee three of the five backends could not
+  keep. See [#4706](https://github.com/objectstack-ai/objectstack/issues/4706).
+</Callout>
+
+The comparand is always matched **literally**. `%` and `_` are ordinary characters,
+not `LIKE` wildcards, and `.` / `*` / `+` are ordinary characters, not regex
+metacharacters — `{ name: { $icontains: 'a.b' } }` matches `a.b` and not `axb`.
+
+<Callout type="info">
+  **Status:** the case rules above are the protocol's declaration as of
+  `@objectstack/spec` 18. The backend lowerings that deliver them — making SQLite's
+  and turso's `LIKE` case-exact, dropping MongoDB's hardcoded `$options: 'i'`, and
+  implementing `$icontains` everywhere — are tracked by
+  [#5702](https://github.com/objectstack-ai/objectstack/issues/5702). Until it
+  lands, a backend that has not been aligned refuses `$icontains` outright rather
+  than answering it approximately, and `$contains` still follows its dialect. The
+  shared standard both halves are measured against is `FILTER_TEXT_CASES`
+  (`@objectstack/spec/data`).
+</Callout>
+
+### `$regex` — removed
+
+`$regex` (and its `$options` companion) was never a declared operator and is
+**retired** ([#4706](https://github.com/objectstack-ai/objectstack/issues/4706)). It
+could not mean one thing across the backends: `driver-sql` compiled it to a
+LIKE-escaped substring match, so `a.b` matched only the literal `a.b`, while
+`driver-memory` evaluated it as a real `RegExp`, so the same filter also matched
+`axb` — and an invalid pattern was caught and answered zero rows, in silence. A real
+regex is not implementable on all five backends: `driver-turso`'s remote transport
+speaks a wire protocol with no way to register a SQLite `REGEXP` function.
+
+| Instead of | Write |
+|:---|:---|
+| `{ name: { $regex: 'acme' } }` | `{ name: { $icontains: 'acme' } }` |
+| `{ name: { $regex: 'acme', $options: 'i' } }` | `{ name: { $icontains: 'acme' } }` |
+| `{ name: { $regex: '^acme' } }` | `{ name: { $startsWith: 'acme' } }` |
+| `{ name: { $regex: 'acme$' } }` | `{ name: { $endsWith: 'acme' } }` |
+
+A pattern that genuinely needs a regular expression has no filter-level
+replacement — narrow the query with the declared operators and match in application
+code. The prescriptions above are declared as data in `RETIRED_FILTER_OPERATORS`
+(`@objectstack/spec/data`), so every backend's refusal quotes the same sentence.
+
 ### Multiple Conditions (Implicit AND)
 
 Multiple keys in `where` are combined with **AND** logic:
@@ -783,11 +851,24 @@ search — and over the REST/protocol ingress it is `400 INVALID_FIELD` outright
 because the engine-side intersection alone used to drop the unknown name and fall back to
 scanning the full searchable set. Internal callers reaching `engine.find()` directly keep
 the tolerant intersection. Multiple whitespace-separated terms are AND-ed and
-fields are OR-ed. Case sensitivity is the **driver's**, not the expansion's: the
-expansion emits a plain `$contains`, which `SqlDriver` compiles to a parameterised
-`LIKE '%…%'` with no case folding — so the dialect's own `LIKE`/collation rules decide —
-while the in-memory driver matches with a case-insensitive regex. Only `select` /
-`status` option *labels* are matched case-insensitively by the expansion itself.
+fields are OR-ed. Case sensitivity comes from the operator the expansion emits, not
+from the expansion: it emits a plain `$contains`, which is **case-sensitive** by the
+rule in [Case Sensitivity](#case-sensitivity) above. Note what that means for search —
+a user typing `acme` does not find `ACME Corp`. Only `select` / `status` option
+*labels* are matched case-insensitively by the expansion itself.
+
+<Callout type="warn">
+  **Measured today, and it does not match that rule yet.** The `$contains` alignment
+  is [#5702](https://github.com/objectstack-ai/objectstack/issues/5702), so until it
+  lands the answer is still the driver's: `SqlDriver` compiles a parameterised
+  `LIKE '%…%'` and the dialect decides (SQLite folds ASCII, Postgres does not),
+  `driver-mongodb` folds the full Unicode range through a hardcoded `$options: 'i'`,
+  and `driver-memory`'s query path matches with a case-insensitive regex. Which
+  driver you run therefore still changes which rows a search returns. Whether the
+  expansion should emit `$icontains` instead of `$contains` — i.e. whether search is
+  case-insensitive by definition — is a separate question that rides with that issue,
+  because it can only be answered once both operators mean one thing everywhere.
+</Callout>
 `fuzzy`, `boost`, `operator`, `minScore`, `language`, and `highlight` carry
 `[EXPERIMENTAL — not enforced]` markers (#4286): the schema accepts them, the
 expansion ignores them.

content/docs/references/data/filter.mdx

@@ -157,6 +157,7 @@ Type: `[FilterArray](#filterarray)[]`
 | **$notContains** | `string` | optional |  |
 | **$startsWith** | `string` | optional |  |
 | **$endsWith** | `string` | optional |  |
+| **$icontains** | `string` | optional | Contains substring, ignoring case — but ONLY ASCII case (A-Z against a-z). Every other character compares literally, so "café" does NOT match "CAFÉ" and "москва" does not match "МОСКВА". The domain is ASCII because that is the one fold all five backends can deliver: SQLite (and therefore turso and sqlite-wasm) folds ASCII only, so a Unicode promise here would be a guarantee three of the five could not keep. The comparand is matched LITERALLY — "%", "_" and regex metacharacters are ordinary characters, not wildcards. Case-SENSITIVE containment is $contains. [#5701: declared by the protocol; the driver lowerings land with #5702.] |
 
 
 ---

packages/spec/api-surface.json

@@ -365,6 +365,8 @@
     "FILTER_LOGIC_CASES (const)",
     "FILTER_LOGIC_ROWS (const)",
     "FILTER_OPERATORS (const)",
+    "FILTER_TEXT_CASES (const)",
+    "FILTER_TEXT_ROWS (const)",
     "FeedFilterMode (type)",
     "FeedItemType (type)",
     "Field (type)",
@@ -398,6 +400,10 @@
     "FilterLogicCase (interface)",
     "FilterLogicRow (interface)",
     "FilterOperatorKey (type)",
+    "FilterTextCase (type)",
+    "FilterTextRejectionCase (interface)",
+    "FilterTextRow (interface)",
+    "FilterTextRowsCase (interface)",
     "FormatValidation (type)",
     "FormatValidationSchema (const)",
     "FullTextSearch (type)",
@@ -532,6 +538,7 @@
     "READ_ONLY_BELONGS_ON_DATASOURCE (const)",
     "RECORD_SURFACE_PAGE_THRESHOLD (const)",
     "REFERENCE_VALUE_TYPES (const)",
+    "RETIRED_FILTER_OPERATORS (const)",
     "RPC_QUERY_ALIAS_SLOTS (const)",
     "RangeOperatorSchema (const)",
     "RecordFlow (type)",
@@ -552,6 +559,7 @@
     "ResolveApiOptions (interface)",
     "ResolveRecordDisplayNameOptions (interface)",
     "ResolvedHook (type)",
+    "RetiredFilterOperatorGuidance (interface)",
     "RowCrudActionOverride (type)",
     "RowCrudActionOverrideInput (type)",
     "RowCrudActionOverrideSchema (const)",

packages/spec/authorable-surface.json

@@ -3815,6 +3815,7 @@
     "data/StateMachineValidation:type",
     "data/StringOperator:$contains",
     "data/StringOperator:$endsWith",
+    "data/StringOperator:$icontains",
     "data/StringOperator:$notContains",
     "data/StringOperator:$startsWith",
     "data/TenancyConfig:enabled",

packages/spec/src/data/filter-operator-vocabulary.test.ts

@@ -0,0 +1,129 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * The filter operator vocabulary has TWO surfaces, and #5701 made them
+ * temporarily disagree on purpose. This file is what stops that from being
+ * silent.
+ *
+ * - **Declaration**: `FieldOperatorsSchema` / `StringOperatorSchema` /
+ *   `Filter<T>` — what an author may write and what `tsc` accepts. Nothing
+ *   derives a runtime allowlist from these (verified: `NormalizedFilterSchema`
+ *   is their only consumer and nothing parses a filter through it at runtime).
+ * - **Enforcement**: `FILTER_OPERATORS` — the array `driver-memory`'s shape
+ *   gate and `service-analytics`' coverage test DERIVE from. An entry here is a
+ *   claim that backends implement the operator.
+ *
+ * `$icontains` is declared and not yet enforced (#5701 is the contract half of
+ * the #4706 ruling; #5702 writes the lowerings). Measured on the branch that
+ * added it to `FILTER_OPERATORS` early: driver-memory's gate stopped refusing
+ * it and `match({ name: 'zzz' }, { name: { $icontains: 'acme' } })` returned
+ * `true` — the predicate silently dropped, every row matched. That is the
+ * widening #3948 is about, so the staging is not a stylistic choice.
+ *
+ * The pin below is deliberately an EQUALITY, not a subset check, so it fails in
+ * both directions: a second staged operator added without recording it fails
+ * here, and so does clearing `$icontains` in #5702 — which is the point. The
+ * failure message is the instruction.
+ */
+
+import { describe, it, expect } from 'vitest';
+import {
+  FieldOperatorsSchema,
+  StringOperatorSchema,
+  FILTER_OPERATORS,
+  LOGICAL_OPERATORS,
+  RETIRED_FILTER_OPERATORS,
+} from './filter.zod';
+
+const declaredKeys = () => Object.keys(FieldOperatorsSchema.shape).sort();
+
+describe('the declaration surface and the enforcement surface', () => {
+  it('differ by EXACTLY the operators staged ahead of their backends', () => {
+    const declared = new Set(declaredKeys());
+    const enforced = new Set<string>(FILTER_OPERATORS);
+    const stagedOnly = [...declared].filter((op) => !enforced.has(op)).sort();
+
+    expect(
+      stagedOnly,
+      'FieldOperatorsSchema and FILTER_OPERATORS differ by something other than the recorded '
+        + 'staging. If you are ADDING an operator: declare it in FieldOperatorsSchema only, and '
+        + 'add it here plus a note on FILTER_OPERATORS saying which issue implements it — an '
+        + 'operator in FILTER_OPERATORS with no backend arm makes driver-memory accept it and '
+        + "silently DROP the predicate (measured, #5701). If you are CLEARING one because you "
+        + 'just implemented it (#5702): remove it from this list AND delete the staging paragraph '
+        + 'on FILTER_OPERATORS, which is now describing something that is no longer true.',
+    ).toEqual(['$icontains']);
+  });
+
+  it('has no operator enforced that is not declared', () => {
+    const declared = new Set(declaredKeys());
+    const undeclared = FILTER_OPERATORS.filter((op) => !declared.has(op));
+    expect(
+      undeclared,
+      'FILTER_OPERATORS demands backends implement an operator FieldOperatorsSchema does not '
+        + 'declare, so an author cannot write it and `tsc` will reject it. This direction is '
+        + 'never staging — it is a drift.',
+    ).toEqual([]);
+  });
+
+  it('declares $icontains on the string operator schema too', () => {
+    expect(Object.keys(StringOperatorSchema.shape)).toContain('$icontains');
+  });
+
+  it('accepts a declared $icontains rather than stripping it', () => {
+    const parsed = FieldOperatorsSchema.parse({ $icontains: 'acme' });
+    expect(parsed).toEqual({ $icontains: 'acme' });
+  });
+
+  it('rejects a non-string $icontains comparand at the schema', () => {
+    expect(() => FieldOperatorsSchema.parse({ $icontains: 42 })).toThrow();
+  });
+});
+
+describe('RETIRED_FILTER_OPERATORS', () => {
+  const entries = Object.entries(RETIRED_FILTER_OPERATORS);
+
+  it('covers the operators #4706 retired', () => {
+    expect(Object.keys(RETIRED_FILTER_OPERATORS).sort()).toEqual(['$options', '$regex']);
+  });
+
+  it('never points at an operator the protocol no longer has', () => {
+    // The `authoring-key-lint.test.ts` rule, applied to operators: a guidance
+    // table whose prescriptions name something undeclared is advice that sends
+    // an author into a second error. Note the check is against the DECLARATION
+    // surface, because `$icontains` is deliberately not in FILTER_OPERATORS yet.
+    const declared = new Set(declaredKeys());
+    for (const [op, guidance] of entries) {
+      if (guidance.to === undefined) continue;
+      expect(declared.has(guidance.to), `${op} prescribes ${guidance.to}, which is not declared`).toBe(true);
+    }
+  });
+
+  it('states the replacement inside the prescription, not only in the `to` field', () => {
+    // A refusal prints `why`. If the replacement lives only in a sibling field
+    // the caller may not render, the error tells the author they are wrong
+    // without telling them what to write — which is the failure the tombstone
+    // convention exists to prevent (AGENTS.md, Post-Task Checklist step 3).
+    for (const [op, guidance] of entries) {
+      if (guidance.to === undefined) continue;
+      expect(guidance.why, `${op}'s prescription never names ${guidance.to}`).toContain(guidance.to);
+    }
+  });
+
+  it('names the retired operator itself, so a refusal can quote one string', () => {
+    for (const [op, guidance] of entries) {
+      expect(guidance.why, `${op}'s prescription never names ${op}`).toContain(op);
+    }
+  });
+
+  it('is not simultaneously declared anywhere — retired means gone', () => {
+    const declared = new Set([...declaredKeys(), ...FILTER_OPERATORS, ...LOGICAL_OPERATORS]);
+    for (const op of Object.keys(RETIRED_FILTER_OPERATORS)) {
+      expect(declared.has(op), `${op} is both retired and declared`).toBe(false);
+    }
+  });
+
+  it('is frozen — a consumer cannot mutate the shared prescriptions', () => {
+    expect(Object.isFrozen(RETIRED_FILTER_OPERATORS)).toBe(true);
+  });
+});