fix(driver-memory): analytics 面的 $notContains 编译成真正排除行的谓词,而不是不约束任何行的裸 {$not: 'x'} (#5374) - #5445
Merged
Merged
Conversation
…dicate that excludes rows (#5374) `MemoryAnalyticsService` mapped each cube operator to the NAME of a mingo operator, and the call site filled that name in as `matchStage[field] = {[name]: comparand}`. That shape can express "compare this field to this value" and nothing else, so the two operators that need to WRAP their comparand were pushed through it anyway. `notContains` -> `'$not'` became `{name: {$not: 'et'}}`. mingo's `$not` takes a regex or an operator expression; handed a bare scalar it constrains nothing, so the predicate was emitted, appeared in the pipeline, and passed the whole table: 3 rows where `find()` returns 2. A predicate that is emitted and inert is indistinguishable from a working one at the author's end — the same amplifying direction as #3948, arrived at a third way. #5345 ruled on operators with NO mapping, #5373 on the comparand ENCODING; this is a mapping pointing at the wrong target, and #5431 did not shrink it (that call site now receives a real value, which is orthogonal to what the operator layer does with it). Route: let the map return a STRUCTURE rather than an operator name, which the issue prefers and the code supports cleanly. `CUBE_OPERATOR_TO_MONGO_PREDICATE` holds a builder per operator that returns the whole `{$op: …}` object, so `notContains` can say `{$not: {$regex: …}}` and the CLASS of "this operator needs a structure and the table can only hold a name" is gone rather than this one instance. `$in`/`$nin`/`$lte`/`$exists`, which the call site had grown an `if` chain for, are ordinary rows in that table now. Measured on the issue's 3-row fixture, analytics vs `find()`: | where | before | after | find() | |----------------------------------|--------|-------|--------| | {name:{$notContains:'et'}} | 3 | 2 | 2 | | {name:{$notContains:'a'}} | 3 | 0 | 0 | | {name:{$contains:'a.p'}} | 1 | 0 | 0 | | {name:{$contains:'ALPHA'}} | 0 | 1 | 1 | | {name:{$notContains:'ALPHA'}} | 3 | 2 | 2 | | {made_at:{$contains:'<full ISO>'}}| 1 | 0 | 0 | | {code:{$in:[]}} | 3 | 0 | 0 | | {code:[]} | 3 | 0 | 0 | Three more defects at the same call site fall inside this fix and are closed with it, because writing a correct `notContains` requires settling each: - `contains` was the right operator with the comparand handed in RAW, so it was neither escaped (`.` matched any character) nor case-folded, while the live path escapes and matches `/…/i`. Leaving that would have made the two non-complementary in a new way — `alpha` would be in BOTH answers. The rule is now borrowed from the driver (new narrow `filterSubstringPattern`, alongside `filterComparandStorageForm`) rather than re-derived, per #5240. - An operand that is NOT a comparand went through the storage-form conversion anyway, so on a declared `datetime` column a `$contains` PATTERN was rewritten into canonical form and then matched rows `find()` does not match. The builder input carries both lists, the same split `normalizeFieldOperators` makes (#4047). - The call site's `values.length > 0` guard meant an empty `$in` emitted no predicate at all and widened to the whole table. A list operator taking the whole list has nothing to guard. The two items the issue flagged as unmeasured, settled: - `'inDateRange': '$gte'` compiles to NOTHING today — no `MONGO_TO_CUBE_OPERATOR` entry lowers to that name, `timeDimensions` never reaches this function, and both exits consume only `normalizeFilters` output. Dead, and wrong if it ever had been reached (a one-ended `>=` for a two-ended range, which its own comment conceded). Deleted, with the dead-and-inverted `'notSet': '$exists'` beside it. - `opMap[operator] || '$eq'` is unreachable for the same reason — but only until someone widens the vocabulary, which #5345 deliberately made a one-line edit to `MONGO_TO_CUBE_OPERATOR`. So it is not merely deleted: that table is `as const`, the predicate table is keyed by the operator union derived from it, and the widening edit now FAILS TO COMPILE until the predicate exists. The remaining throw is a totality floor, not a fallback. Tests go in the shared conformance file beside the #5345 shape table and the #5373 comparand-type table, as a third axis with the same invariant: agree with `find()`, or refuse. Plus the "declared = enforced" half — every operator `ANALYTICS_FILTER_CAPABILITIES` declares is driven through both faces and must agree, with a probe that must exclude at least one row, so an operator added to the vocabulary without a working lowering fails here instead of shipping a quietly wrong number. Reverting only the source change fails 14 of the new assertions. Out of scope, filed not fixed: #5440 (two operators on one field clobber each other — the `$match` assembly layer, still broken after this), #5442 (`flattenFilterCondition` spreads an array comparand for every operator), #5444 (the `generateSql` exit emits `LIKE 'et'` with no `%` wildcards — filed as a sub-issue of #5433, whose completion scope it falls inside). `operatorToSql` and `generateSql` are untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pbu27iNUfQCHeuS551Rqo7
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Contributor
📓 Docs Drift CheckThis PR changes 1 package(s): 9 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
os-zhuang
marked this pull request as ready for review
August 5, 2026 12:02
This was referenced Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5374
MemoryAnalyticsService把每个 cube 算子映射成一个 mingo 算子的名字,调用点再把这个名字填成matchStage[field] = {[name]: comparand}。这个形状只能表达「拿这个字段和这个值比」,别的都表达不了 —— 于是那两个需要包装比较数的算子被硬塞了进去。notContains→'$not'编译出{name: {$not: 'et'}}。mingo 的$not期望正则或算子表达式,给它一个裸标量时它不构成约束,所以谓词生成了、在管线里看得见、然后放过了全表:3 行,而find()是 2 行。对作者而言,一个「生成了但不起作用」的谓词和一个正确的谓词完全不可区分 —— 与 #3948 同一个放大方向,只是第三次从另一个地方到达。这也是同一张面上第三个独立的层:#5345 裁的是没有映射的算子,#5373 裁的是比较数编码,这一条是有映射但指向了错的目标。#5431 没有让它变小:那个调用点现在收到的是真实值而不是字符串化的值,但那与算子层做了什么正交。
采用的路线
issue 给了两条:调用点包装,或者让映射表返回一个结构而不是算子名。选了后者 —— issue 本身倾向它,代码也确实支撑得起。
CUBE_OPERATOR_TO_MONGO_PREDICATE每个算子持有一个 builder,返回整个{$op: …}对象。于是notContains可以说{$not: {$regex: …}},而「这个算子需要一个结构,可表里只放得下一个名字」这一类问题消失了,而不只是这一个实例。调用点原本为$in/$nin/$lte/$exists长出来的那串if,现在都是这张表里的普通行。实测(issue 的 3 行固定数据,analytics vs
find())wherefind(){name: {$notContains: 'et'}}{name: {$notContains: 'a'}}{name: {$contains: 'a.p'}}{name: {$contains: 'ALPHA'}}{name: {$notContains: 'ALPHA'}}{made_at: {$contains: '(完整 ISO)'}}{code: {$in: []}}{code: []}同一调用点上另外三处,一并关掉了
不是扩范围 —— 要写出一个正确的
notContains,这三条每一条都必须先裁掉:contains的比较数是原样进去的。 算子映射本身没错,但比较数既没转义(.变成了通配符)也没折叠大小写,而 live query path 是escapeRegex+/…/i。只修notContains而不管这个,两者会以一种新的方式不互补 ——alpha会同时出现在两边的答案里。所以子串规则改成向 driver 借(新增窄接口filterSubstringPattern,与filterComparandStorageForm并列),而不是在这里重新推导一遍({ field: {} }(零个操作符的字段约束)在同仓有三个答案:driver-sql 组合子内 TRUE、顶层抛 INVALID_FILTER、formula/driver-memory FALSE #5240)。datetime的列上,$contains的模式串本身被改写成了 canonical 形式,然后匹配上了find()匹配不到的行。builder 的入参因此带两份列表,与normalizeFieldOperators做的是同一个切分(driver-memory / driver-mongodb:Field.datetime无单一存储形态,跨类型比较恒 false —— #3912 的非 SQL 版 #4047)。values.length > 0守卫让空$in一个谓词都不生成,直接放大到全表。一个「取整个列表」的算子没有什么可守的。issue 里标为「未实测」的两条,已经裁了
'inDateRange': '$gte':今天什么都编译不出来。没有任何MONGO_TO_CUBE_OPERATOR条目降级到这个名字,timeDimensions走的是 Stage 2、根本不经过这个函数,而两个出口都只消费normalizeFilters的输出。所以它是死的 —— 并且万一被走到还是错的(用单边>=回答一个双端区间,这一点它自己的注释也承认了 "Will need special handling")。删掉了,同时删掉旁边同样是死的、并且方向反了的'notSet': '$exists'。opMap[operator] || '$eq':出于同样的理由不可达 —— 但只是在有人拓宽词表之前,而 driver-memory 的 analytics 面静默丢弃大半个 filter:$or/$not整条丢,$between/$startsWith/$null/$regex因无 cube 映射而丢 —— 聚合结果被放大 #5345 恰恰刻意把拓宽做成了对MONGO_TO_CUBE_OPERATOR的一行编辑。所以不只是删掉:那张表现在是as const,谓词表的键类型是从它派生出来的算子 union,于是那一行拓宽编辑编译不过,直到对应的谓词写出来为止。剩下的那个 throw 是全域性的兜底(totality floor),不是 fallback。这一点是本 PR 长期价值的所在:兜底不再是「不可达」,而是「不必要」—— 全域性被证明了,而不是被防守。对 AI 写的元数据尤其重要:宽容的消费端正是 AI 生成的错误藏身并繁殖的地方。
测试
放在共享的 conformance 文件里,紧挨着 #5345 的 shape 表和 #5373 的比较数类型表,作为第三条轴,持有同一个不变量:与
find()一致,或者拒收,绝不给出第三个更安静的答案。外加「declared = enforced」那一半:
ANALYTICS_FILTER_CAPABILITIES声明的每一个算子都被驱动着走两条路并必须一致,且探针必须至少排除一行(否则「一致」什么也证明不了 —— 那正是本 bug 的形状)。往词表里加算子却没有可用的降级,现在会在这里失败,而不是发布一个安静的错数字。只回退源码改动(保留测试),新断言失败 14 条:
各门实际输出:
packages/runtime的 suite 在本 worktree 里因为工作区依赖未构建而整文件 import 失败(Failed to resolve entry for package "@objectstack/objectql"等)。把本改动 stash 掉后失败完全相同,与本 diff 无关。范围之外,已单独立项
{qty: {$gte:150, $lte:250}}只剩最后一个 —— 区间塌成单边,结果被放大 #5440 —— 同一字段上的多个算子互相覆盖({qty: {$gte:150, $lte:250}}取到[1,2],find()取到[2])。在$match组装层,本 PR 之后依然完整存在;正确的修法需要一份等价于 live path_multiRegex→$and的提升,是独立的一块。flattenFilterCondition对所有算子一律摊平数组比较数,{qty: {$eq: [100]}}变成$eq: 100(find()取 0 行,analytics 取 1 行) #5442 ——flattenFilterCondition对所有算子一律摊平数组比较数,{qty: {$eq: [100]}}变成$eq: 100(find()0 行,analytics 2 行)。在更上游的降级层。generateSql把contains回显成LIKE 'et'(没有%通配符)—— 回显的是等值匹配,执行的是子串匹配 #5444 ——generateSql把contains回显成LIKE 'et'(没有%通配符),回显的是等值匹配、执行的是子串匹配。作为 driver-memory analytics 面的generateSql对in/notIn/set/notSet输出错误 SQL:回退成=且只取第一个值($in: ['100','200']→WHERE code = '100') #5433 的 sub-issue,因为修复落点完全在 driver-memory analytics 面的generateSql对in/notIn/set/notSet输出错误 SQL:回退成=且只取第一个值($in: ['100','200']→WHERE code = '100') #5433 的完成范围内(同一个operatorToSql+ 同一个 WHERE 构建循环)。operatorToSql与generateSql一个字节都没动 —— #5433 是同一类缺陷在另一个出口上,刻意不打包在一起。packages/spec、service-analytics同样未触碰。🤖 Generated with Claude Code
https://claude.ai/code/session_01Pbu27iNUfQCHeuS551Rqo7
Generated by Claude Code