Skip to content

fix(driver-memory): analytics 面的 $notContains 编译成真正排除行的谓词,而不是不约束任何行的裸 {$not: 'x'} (#5374) - #5445

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-5374-notcontains-inert-predicate
Aug 5, 2026
Merged

fix(driver-memory): analytics 面的 $notContains 编译成真正排除行的谓词,而不是不约束任何行的裸 {$not: 'x'} (#5374)#5445
os-zhuang merged 1 commit into
mainfrom
claude/issue-5374-notcontains-inert-predicate

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes #5374

MemoryAnalyticsService 把每个 cube 算子映射成一个 mingo 算子的名字,调用点再把这个名字填成 matchStage[field] = {[name]: comparand}。这个形状只能表达「拿这个字段和这个值比」,别的都表达不了 —— 于是那两个需要包装比较数的算子被硬塞了进去。

notContains'$not' 编译出 {name: {$not: 'et'}}。mingo 的 $not 期望正则或算子表达式,给它一个裸标量时它不构成约束,所以谓词生成了、在管线里看得见、然后放过了全表:3 行,而 find() 是 2 行。对作者而言,一个「生成了但不起作用」的谓词和一个正确的谓词完全不可区分 —— 与 #3948 同一个放大方向,只是第三次从另一个地方到达。

这也是同一张面上第三个独立的层:#5345 裁的是没有映射的算子,#5373 裁的是比较数编码,这一条是有映射但指向了错的目标#5431 没有让它变小:那个调用点现在收到的是真实值而不是字符串化的值,但那与算子层做了什么正交。

采用的路线

issue 给了两条:调用点包装,或者让映射表返回一个结构而不是算子名。选了后者 —— issue 本身倾向它,代码也确实支撑得起。

CUBE_OPERATOR_TO_MONGO_PREDICATE 每个算子持有一个 builder,返回整个 {$op: …} 对象。于是 notContains 可以说 {$not: {$regex: …}},而「这个算子需要一个结构,可表里只放得下一个名字」这一问题消失了,而不只是这一个实例。调用点原本为 $in / $nin / $lte / $exists 长出来的那串 if,现在都是这张表里的普通行。

实测(issue 的 3 行固定数据,analytics vs find())

where 修复前 修复后 find()
{name: {$notContains: 'et'}} 3 2 2
{name: {$notContains: 'a'}} 3 0 0
{name: {$contains: 'a.p'}} 1 0 0
{name: {$contains: 'ALPHA'}} 0 1 1
{name: {$notContains: 'ALPHA'}} 3 2 2
{made_at: {$contains: '(完整 ISO)'}} 1 0 0
{code: {$in: []}} 3 0 0
{code: []} 3 0 0

同一调用点上另外三处,一并关掉了

不是扩范围 —— 要写出一个正确的 notContains,这三条每一条都必须先裁掉:

issue 里标为「未实测」的两条,已经裁了

  • 'inDateRange': '$gte':今天什么都编译不出来。没有任何 MONGO_TO_CUBE_OPERATOR 条目降级到这个名字,timeDimensions 走的是 Stage 2、根本不经过这个函数,而两个出口都只消费 normalizeFilters 的输出。所以它是死的 —— 并且万一被走到还是错的(用单边 >= 回答一个双端区间,这一点它自己的注释也承认了 "Will need special handling")。删掉了,同时删掉旁边同样是死的、并且方向反了的 'notSet': '$exists'
  • opMap[operator] || '$eq':出于同样的理由不可达 —— 但只是在有人拓宽词表之前,而 driver-memory 的 analytics 面静默丢弃大半个 filter:$or/$not 整条丢,$between/$startsWith/$null/$regex 因无 cube 映射而丢 —— 聚合结果被放大 #5345 恰恰刻意把拓宽做成了对 MONGO_TO_CUBE_OPERATOR 的一行编辑。所以不只是删掉:那张表现在是 as const,谓词表的键类型是从它派生出来的算子 union,于是那一行拓宽编辑编译不过,直到对应的谓词写出来为止。剩下的那个 throw 是全域性的兜底(totality floor),不是 fallback。

这一点是本 PR 长期价值的所在:兜底不再是「不可达」,而是「不必要」—— 全域性被证明了,而不是被防守。对 AI 写的元数据尤其重要:宽容的消费端正是 AI 生成的错误藏身并繁殖的地方。

测试

放在共享的 conformance 文件里,紧挨着 #5345 的 shape 表和 #5373 的比较数类型表,作为第三条轴,持有同一个不变量:与 find() 一致,或者拒收,绝不给出第三个更安静的答案。

外加「declared = enforced」那一半:ANALYTICS_FILTER_CAPABILITIES 声明的每一个算子都被驱动着走两条路并必须一致,且探针必须至少排除一行(否则「一致」什么也证明不了 —— 那正是本 bug 的形状)。往词表里加算子却没有可用的降级,现在会在这里失败,而不是发布一个安静的错数字。

只回退源码改动(保留测试),新断言失败 14 条:

 × $notContains excludes the rows that contain the comparand
 × a $notContains that matches every row selects none of them
 × $contains treats a metacharacter as a literal
 × $contains is case-insensitive, as the live path is
 × $notContains is case-insensitive, as the live path is
 × $contains matches a mixed-case comparand
 × $contains does not rewrite its pattern into a datetime storage form
 × $notContains does not rewrite its pattern either
 × $notContains over a column holding nulls
 × an empty $in selects nothing
 × an empty implicit-equality list selects nothing
 × $contains and $notContains partition the table for every comparand
 × every operator this face DECLARES compiles to a predicate that agrees with find()
 × the emitted $match wraps the negation around a pattern instead of a bare scalar
 Tests  14 failed | 74 passed (88)

各门实际输出:

pnpm --filter @objectstack/driver-memory typecheck   -> tsc --noEmit, 无输出
npx eslint packages/plugins/driver-memory/src        -> LINT OK
pnpm --filter @objectstack/driver-memory test        -> Test Files 17 passed (17) / Tests 512 passed (512)
pnpm --filter @objectstack/service-analytics test    -> Test Files 47 passed (47) / Tests 745 passed (745)

packages/runtime 的 suite 在本 worktree 里因为工作区依赖未构建而整文件 import 失败(Failed to resolve entry for package "@objectstack/objectql" 等)。把本改动 stash 掉后失败完全相同,与本 diff 无关。

范围之外,已单独立项

operatorToSqlgenerateSql 一个字节都没动 —— #5433 是同一类缺陷在另一个出口上,刻意不打包在一起。packages/specservice-analytics 同样未触碰。

🤖 Generated with Claude Code

https://claude.ai/code/session_01Pbu27iNUfQCHeuS551Rqo7


Generated by Claude Code

…dicate that excludes rows (#5374)

`MemoryAnalyticsService` mapped each cube operator to the NAME of a mingo
operator, and the call site filled that name in as
`matchStage[field] = {[name]: comparand}`. That shape can express "compare this
field to this value" and nothing else, so the two operators that need to WRAP
their comparand were pushed through it anyway.

`notContains` -> `'$not'` became `{name: {$not: 'et'}}`. mingo's `$not` takes a
regex or an operator expression; handed a bare scalar it constrains nothing, so
the predicate was emitted, appeared in the pipeline, and passed the whole table:
3 rows where `find()` returns 2. A predicate that is emitted and inert is
indistinguishable from a working one at the author's end — the same amplifying
direction as #3948, arrived at a third way. #5345 ruled on operators with NO
mapping, #5373 on the comparand ENCODING; this is a mapping pointing at the
wrong target, and #5431 did not shrink it (that call site now receives a real
value, which is orthogonal to what the operator layer does with it).

Route: let the map return a STRUCTURE rather than an operator name, which the
issue prefers and the code supports cleanly. `CUBE_OPERATOR_TO_MONGO_PREDICATE`
holds a builder per operator that returns the whole `{$op: …}` object, so
`notContains` can say `{$not: {$regex: …}}` and the CLASS of "this operator
needs a structure and the table can only hold a name" is gone rather than this
one instance. `$in`/`$nin`/`$lte`/`$exists`, which the call site had grown an
`if` chain for, are ordinary rows in that table now.

Measured on the issue's 3-row fixture, analytics vs `find()`:

| where                            | before | after | find() |
|----------------------------------|--------|-------|--------|
| {name:{$notContains:'et'}}       |      3 |     2 |      2 |
| {name:{$notContains:'a'}}        |      3 |     0 |      0 |
| {name:{$contains:'a.p'}}         |      1 |     0 |      0 |
| {name:{$contains:'ALPHA'}}       |      0 |     1 |      1 |
| {name:{$notContains:'ALPHA'}}    |      3 |     2 |      2 |
| {made_at:{$contains:'<full ISO>'}}|     1 |     0 |      0 |
| {code:{$in:[]}}                  |      3 |     0 |      0 |
| {code:[]}                        |      3 |     0 |      0 |

Three more defects at the same call site fall inside this fix and are closed
with it, because writing a correct `notContains` requires settling each:

- `contains` was the right operator with the comparand handed in RAW, so it was
  neither escaped (`.` matched any character) nor case-folded, while the live
  path escapes and matches `/…/i`. Leaving that would have made the two
  non-complementary in a new way — `alpha` would be in BOTH answers. The rule is
  now borrowed from the driver (new narrow `filterSubstringPattern`, alongside
  `filterComparandStorageForm`) rather than re-derived, per #5240.
- An operand that is NOT a comparand went through the storage-form conversion
  anyway, so on a declared `datetime` column a `$contains` PATTERN was rewritten
  into canonical form and then matched rows `find()` does not match. The builder
  input carries both lists, the same split `normalizeFieldOperators` makes
  (#4047).
- The call site's `values.length > 0` guard meant an empty `$in` emitted no
  predicate at all and widened to the whole table. A list operator taking the
  whole list has nothing to guard.

The two items the issue flagged as unmeasured, settled:

- `'inDateRange': '$gte'` compiles to NOTHING today — no `MONGO_TO_CUBE_OPERATOR`
  entry lowers to that name, `timeDimensions` never reaches this function, and
  both exits consume only `normalizeFilters` output. Dead, and wrong if it ever
  had been reached (a one-ended `>=` for a two-ended range, which its own
  comment conceded). Deleted, with the dead-and-inverted `'notSet': '$exists'`
  beside it.
- `opMap[operator] || '$eq'` is unreachable for the same reason — but only until
  someone widens the vocabulary, which #5345 deliberately made a one-line edit
  to `MONGO_TO_CUBE_OPERATOR`. So it is not merely deleted: that table is `as
  const`, the predicate table is keyed by the operator union derived from it,
  and the widening edit now FAILS TO COMPILE until the predicate exists. The
  remaining throw is a totality floor, not a fallback.

Tests go in the shared conformance file beside the #5345 shape table and the
#5373 comparand-type table, as a third axis with the same invariant: agree with
`find()`, or refuse. Plus the "declared = enforced" half — every operator
`ANALYTICS_FILTER_CAPABILITIES` declares is driven through both faces and must
agree, with a probe that must exclude at least one row, so an operator added to
the vocabulary without a working lowering fails here instead of shipping a
quietly wrong number. Reverting only the source change fails 14 of the new
assertions.

Out of scope, filed not fixed: #5440 (two operators on one field clobber each
other — the `$match` assembly layer, still broken after this), #5442
(`flattenFilterCondition` spreads an array comparand for every operator), #5444
(the `generateSql` exit emits `LIKE 'et'` with no `%` wildcards — filed as a
sub-issue of #5433, whose completion scope it falls inside). `operatorToSql` and
`generateSql` are untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pbu27iNUfQCHeuS551Rqo7
@vercel

vercel Bot commented Aug 5, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 5, 2026 11:55am

Request Review

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling size/l labels Aug 5, 2026
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/driver-memory.

9 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/data-modeling/drivers.mdx (via @objectstack/driver-memory)
  • content/docs/deployment/vercel.mdx (via @objectstack/driver-memory)
  • content/docs/getting-started/glossary.mdx (via @objectstack/driver-memory)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/driver-memory)
  • content/docs/permissions/authentication.mdx (via @objectstack/driver-memory)
  • content/docs/plugins/index.mdx (via @objectstack/driver-memory)
  • content/docs/plugins/packages.mdx (via @objectstack/driver-memory)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/driver-memory)
  • content/docs/releases/implementation-status.mdx (via @objectstack/driver-memory)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

driver-memory analytics 面的 $notContains 编译成裸 mingo {$not: 'x'},该谓词不约束任何行 —— 结果被放大到全表

2 participants