Skip to content

fix(driver-memory): analytics 面拒收编译不了的过滤器,而不是静默丢掉 (#5345) - #5375

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-5345-memory-analytics-filter-refusal
Aug 5, 2026
Merged

fix(driver-memory): analytics 面拒收编译不了的过滤器,而不是静默丢掉 (#5345)#5375
os-zhuang merged 1 commit into
mainfrom
claude/issue-5345-memory-analytics-filter-refusal

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes #5345

走 issue 给的第一条路:拒收,不去补齐 cube 管线。这也是今晚另外两个面的落点(#5366 把 analytics normalizer 的拒收送到调用方,#5368 让四个 driver 在门口拒收越界输入),把 $or 补进 cube 模型是大得多的改动,而且未必表达得出来。

问题

memory-analytics.tsAnalyticsQuery.where 摊平成 cube 风格的 {member, operator, values} 列表,凡是塞不进这个形状的一律 continue:$or / $not 整条丢,以及 mongo→cube 映射表里没有的五个已声明算子($between$startsWith$endsWith$null$regex)一条条丢。注释还把它写成了一个特性 —— "ignore so a partial query still runs rather than failing entirely" —— 正是 ADR-0078 / #4286objectqlhaving 上判定为错误的那条理由。这句注释和它辩护的行为一起删掉了。

方向是放大,不是缩小。 少一个约束 = 少过滤 = 多返回行。在共享的 FILTER_LOGIC_CASES 上实测,17 条用例里有 15 条返回了比标准更宽的行集,多数是全表。$not 还不止是数字不准:cel-to-filter.ts 把 CEL !expr 的 RLS read scope 编译成的就是 {$not: {…}},丢掉的那一支就是 read scope 本身 —— 聚合里混进了调用方无权读的行。

改法

复用 #5349 的既有原语,没有新起一套。 assertFilterConditionShape 现在接受调用面的 FilterFaceCapabilities(这个面能编译什么,与协议声明了什么是两回事),差集经 unsupportedFilterError 拒收(INVALID_FILTER / 400)。query path 和 reference matcher 不传这个参数,行为逐字节不变。

两类拒收刻意分开:协议根本没声明的算子仍然是 unknownFieldOperatorError(「你拼错了」),协议声明了而本面编译不了的是新的 uncompilableFieldOperatorError。把两者合并等于告诉仪表盘作者他的 $between 是个笔误。

analytics 面的能力集从它自己的 MONGO_TO_CUBE_OPERATOR 表派生,于是「放宽本面接受的输入」和「教会它编译这个算子」变成同一处修改 —— 手抄一份词表只在敲下去那天和编译器一致。

门放在 normalizeFilters,在任何一个 key 被摊平之前,理由是那个模块自己已经写明的:摊平过程中途抛出的拒收,会因 key 顺序和先走了哪个兄弟分支而时灵时不灵。两个公开入口(query() / generateSql())都走这里。原来两处 continue 现在都改成 throw —— 其中算子那处是可达的:嵌套关系分支会用一个门没走过的合成节点 {'a.b': spec} 重新进入。

测试

把 analytics 面纳入 FILTER_LOGIC_CASES 的穿透范围 —— 这个包有三个过滤面,共享一致性表原先只盯住两个,而让这个缺陷活下来的正是一条缺失的断言。cube 管线里没有 $or,所以这个面不可能逐行通过表;它被钉住的是真正要紧的那条谓词:

每条用例,analytics 面要么给出和 find() 相同的 id 集,要么以 INVALID_FILTER 拒收。不允许有第三种、更安静的答案。

这条谓词正是两个 continue 违反的东西,而且将来 cube 管线真学会了 $or,用例只是从「拒收」栏移到「一致」栏,本文件不用改。

回归证据:只把 memory-analytics.ts 还原成修复前,新断言失败 16 条。顺带看到一个值得记下的巧合 —— a $or nested under a top-level $and 修复前也是「对」的,因为它被丢掉的 $or 恰好被一个幸存的兄弟 key 覆盖了。数字看着对,从来不是证据。

另有一个专门的 suite 断言拒收信封(code / status)本身,以及每条拒收都点名了是哪个算子/组合子被拒 —— 仪表盘作者要能看出是哪条谓词出了问题。

Test Files  17 passed (17)
     Tests  458 passed (458)
tsc --noEmit   → 0
eslint         → 0

Changeset

minor,不是 patch。这是已发布面上可观察的行为变更:一个 where 里带 $between$or 的部件,以前静默返回被放大的聚合值,现在报错。changeset 正文把每种形状的改写方式($between → 两个边界、$null$exists: false$or/$not → 走 find())都写全了,因为这段文本会作为 CHANGELOG.md 随包发出去。

#5368 用 patch 的区别在于:那一单拒收的是协议从未声明的形状(执行既有契约),这一单拒收的是协议声明了find() 今天就在跑的形状,是本面契约的收窄。

范围外 —— 已单独立单,未在本 PR 修

逐个实测「本面声明支持的 11 个算子各自编译成什么」时撞到的,都不属于 #5345 的范围面(它裁的是没有映射的算子):

两条都在 #5345 之后依然被 ANALYTICS_FILTER_CAPABILITIES 声明为「本面支持」,所以在 MONGO_TO_CUBE_OPERATOR 的 TSDoc 上留了一句指向它们的话:表里有一行只代表这个面会去尝试该算子,不代表它编译出的谓词是对的,别把这 11 个读成「11 个已知可用」。

另按范围围栏未触碰:service-analytics 的对应面(#5325 在飞)、$exists 三答案分叉(#5369)、cloud 的 RemoteTransport(cloud#1077 / cloud#1116)。本 PR 的 diff 只落在 driver-memory,packages/spec 一个字节未动。

(构建 packages/specgen:schema 会把 authorable-surface.base.jsonbaseRev 重锚到当前分支的 merge base —— 这是它在任何分支上的正常副产物,不属于本 PR,已还原。)

🤖 Generated with Claude Code

https://claude.ai/code/session_01Pbu27iNUfQCHeuS551Rqo7


Generated by Claude Code

…pile instead of dropping it (#5345)

`MemoryAnalyticsService` lowers `AnalyticsQuery.where` into a flat cube-style
`{member, operator, values}` list, and answered everything that did not fit with
`continue` — `$or` and `$not` wholesale, plus the five declared operators with no
row in the mongo→cube table (`$between`, `$startsWith`, `$endsWith`, `$null`,
`$regex`). A comment presented this as a feature: "ignore so a partial query
still runs rather than failing entirely" — the identical reasoning ADR-0078 /
#4286 judged wrong on `objectql`'s `having`. Deleted along with the behaviour it
justified.

The direction is what makes it a defect: a dropped predicate is FEWER
constraints, therefore MORE rows. Measured on `FILTER_LOGIC_CASES`, 15 of its 17
cases returned a wider row set than the standard specifies, usually every row.
`$not` makes it more than a wrong number — `cel-to-filter.ts` compiles a CEL
`!expr` RLS read scope into `{$not: {…}}`, so the dropped branch was the read
scope itself.

Route 1 of the issue's two: refuse, do not backfill the cube pipeline — where
#5366 and #5368 put the two neighbouring faces.

Implementation reuses #5349's primitives rather than adding a second validator.
`assertFilterConditionShape` now takes the calling face's `FilterFaceCapabilities`
— what that face can COMPILE, distinct from what the protocol declares — and
refuses the difference through `unsupportedFilterError` (INVALID_FILTER / 400).
The query path and the matcher pass nothing and are byte-for-byte unaffected.
The analytics face derives its capabilities from its own `MONGO_TO_CUBE_OPERATOR`
table, so widening what it accepts and teaching it to compile the operator are
now one edit. Two refusals distinct on purpose: an operator the protocol never
declared is still `unknownFieldOperatorError` ("you have a typo"), one it
declares and this face cannot lower is the new `uncompilableFieldOperatorError`.

The gate runs in `normalizeFilters`, before any lowering, for the reason that
module already documents — a refusal raised mid-lowering fires or not depending
on key order. Both public entry points (`query()`, `generateSql()`) go through
it. The two former `continue` sites now throw; the operator one is reachable via
the nested-relation branch, which re-enters with a synthesised node the gate
never walked.

Tests: `FILTER_LOGIC_CASES` now covers this third face — the package had three
filter surfaces and the shared table watched two. It cannot pass row-for-row (a
cube pipeline has no `$or`), so it is held to the predicate that actually
matters: agree with `find()`, or refuse with INVALID_FILTER, never a third
quieter answer. Reverting only `memory-analytics.ts` fails 16 of the new
assertions. Plus a dedicated suite asserting the envelope (code, status) and
that each refusal names the offending operator or combinator.

Out of scope, filed not fixed: #5373 (the cube comparand round-trip loses
booleans and `null`), #5374 (`$notContains` lowers to a bare mingo `{$not: 'x'}`
that constrains nothing).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pbu27iNUfQCHeuS551Rqo7
@vercel

vercel Bot commented Aug 5, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 5, 2026 2:42am

Request Review

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling size/l labels Aug 5, 2026
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/driver-memory.

9 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/data-modeling/drivers.mdx (via @objectstack/driver-memory)
  • content/docs/deployment/vercel.mdx (via @objectstack/driver-memory)
  • content/docs/getting-started/glossary.mdx (via @objectstack/driver-memory)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/driver-memory)
  • content/docs/permissions/authentication.mdx (via @objectstack/driver-memory)
  • content/docs/plugins/index.mdx (via @objectstack/driver-memory)
  • content/docs/plugins/packages.mdx (via @objectstack/driver-memory)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/driver-memory)
  • content/docs/releases/implementation-status.mdx (via @objectstack/driver-memory)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@os-zhuang
os-zhuang marked this pull request as ready for review August 5, 2026 10:47
@os-zhuang
os-zhuang added this pull request to the merge queue Aug 5, 2026
Merged via the queue into main with commit d085670 Aug 5, 2026
24 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-5345-memory-analytics-filter-refusal branch August 5, 2026 10:53
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Aug 6, 2026
…in 滞后、死代码删除复核 (objectstack-ai#5513) (objectstack-ai#5645)

2026-08-05 跑完一整条 filter 缺陷链(objectstack-ai#5363 / objectstack-ai#5366 / objectstack-ai#5368 / objectstack-ai#5375 / objectstack-ai#5431 / objectstack-ai#5445,
cloud#1117)后回看,六处在那一轮真实咬过人或真实救过场的规程,SKILL 里没有对应条目。
六条各落在 issue 指定的节内,**纯增补**:111 行插入、0 行删除,既有条目(objectstack-ai#5501 的接力
模式、objectstack-ai#5522 的座位模型、objectstack-ai#5630 的 assertEngineDeleteDispatch 条款)一字未动。

落点与要点:

1. **Multi-repo,rule 2 之后**「pin 滞后」——`Blocked-by:` 只保证上游已合并,姊妹仓还有
   第二个读数:本仓 pin 是否覆盖那个 commit。cloud#1116 的裁决落于 framework objectstack-ai#5368
   (`9c5abf4e9`),而 cloud 的 `.objectstack-sha` 未覆盖它,于是 `TursoDriver` 有一个
   方向反了的分叉窗口(fail-closed 一侧先到)。规程:派发前核祖先关系;未覆盖则 dev 在
   PR 正文留档窗口与方向,⛔ pin bump 不做 rider。
2. **step 3** 末「阻塞解除后重新定价」—— 前一单合入会改变后一单的成本模型,方向不止一个
   (本轮变便宜、没变、成本估计过期各有实例)。两个动作配对:派发前一单时带必答项
   「你的改动是否让 #X 变简单 / 变难 / 不必要 / 无影响」,派发被延后那单前用该回答重读
   其选项与成本估计。
3. **step 5** 派发令「多面组件的测试落点」—— 同一契约 ≥2 实现面时,新用例进共享一致性
   覆盖而非独立文件(原话照录)。附 objectstack-ai#5375 / objectstack-ai#5431 / objectstack-ai#5445 三条正交轴共用一条不变量。
4. **step 7 清单**「收益穿过它必经的那道边界之后还在吗」—— 判据是价值主张是否依赖下游
   如实转发;实例即 objectstack-ai#5423(4xx 直通曾整条替换 ≥500 字符正文,`code` 到了正文没到)。
5. **step 7 清单**「死代码删除的复核」——「这是死代码」是断言而非能从 diff 读出的事实,
   PM 在 origin/main 独立核一次引用面再 ACCEPT(查法用 Operational notes 6:notes 6 说
   怎么查不假阴性,本条说什么时候必须查)。
6. **step 8** 升级门槛之后「带前提的裁决」—— 分歧关键是可被代码证伪的事实时,第三档 =
   裁决 + 前提验证要求 + 「前提不成立报 fork,不许硬做也不许悄悄改选」禁令,三件缺一
   不可;缺第 3 条即退化为无人裁决且无读数显示。

实施时两处核实结果与 issue 正文不同,成文按核实后的事实写:

- issue 的附带论断「没有任何闸门在量这个 pin 滞后」**不成立** —— cloud 的
  `scripts/check-pin-staleness.sh`(test.yml 以 `continue-on-error` 跑)每次 CI 都报两个
  pin 各落后 main 多少 commit,advisory 是**有意设计**(`--max-behind` 需显式传)。它答
  的是「落后多少」,不是「是否覆盖我这条裁决 commit」;成文因此指向该脚本,并只把后一个
  问题留给派发前的祖先判断。据此**未**另立「无闸门」的发现单。
- 第 4 条的 rest-server 缺陷本身已由 objectstack-ai#5423 按「截断而非替换」修掉,成文改用过去时并注明,
  以免后来的读者去找一个已不存在的活 bug;该条要补的是**复核清单的缺口**,与代码是否已修
  无关。

第 1 / 3 条按 issue「未验证的部分」的克制写入适用判据(前后单共用同一契约或数据表示;
组件对同一契约有 ≥2 实现面),形态迥异的批次(纯 UI、纯文档)明确不强加。

验证:`node scripts/check-nul-bytes.mjs --self-test` + 全仓扫描绿(48 断言 / 5537 文件);
改动文件自扫 `grep -naP '[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]'` 零命中,并用邻近词反查证伪
「扫描器坏了」;`check:docs-audit-scope` 绿;markdown 结构核对(强调标记成对、代码围栏
16 个偶数、嵌套围栏缩进对齐)。

Claude-Session: https://claude.ai/code/session_01GX3sL71LFq8m2usg6VqTSE

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: os-zhuang <hr@objectstack.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

2 participants