fix(analytics,rest): analytics 的 filter 拒收带上 ADR-0112 信封,REST 面先读信封 —— 400 INVALID_FILTER 而不是 500 (#5352) - #5366
Conversation
…s 400 INVALID_FILTER (#5352) A misspelled operator in a dashboard widget's filter is refused by `filter-normalizer.ts` — correctly — but the refusal never reached the author: it landed as `500 ANALYTICS_QUERY_FAILED`, read as "the platform is broken" rather than "your filter has a typo", and counted by ops alerting as a 5xx. The identical mistake on `find()` has answered `400 INVALID_FILTER` since #3948. One defect with two halves; either alone leaves it unfixed. Producer — `filter-normalizer.ts`: seven of its nine refusals were bare `throw new Error(…)` with no `code`/`status`, so the REST face had nothing to read. All nine now go through the `invalidFilterError` helper #5334 introduced (INVALID_FILTER / 400), which becomes the module's only way to refuse. Two of the seven (`{$not: <non-object>}`, an unsupported TOP-LEVEL operator) were not among the issue's four bullets; enveloping only the listed five would have left two spellings of the same authoring mistake answering 500 next to neighbours answering 400. Consumer — `rest-server.ts`, `POST /analytics/dataset/query`: the catch discarded `error.code`/`error.status` and re-derived the classification from a hardcoded list of message substrings. It now reads the envelope first, and the substring list is demoted to a documented transitional fallback. All six of its entries were re-verified as bare `Error`s, so none could be deleted. The passthrough is 4xx-only and requires both `code` and `status`: an internal fault can never be re-labelled as the caller's fault, and this route invents no code a producer failed to supply. Which inputs are refused did not change — only the shape of the error. Pinned input-by-input, refusals and accepted inputs (with their compiled trees) alike, in `filter-refusal-envelope.test.ts`, which is green both before and after. The REST-side test drives the real `AnalyticsService` rather than a mock, because the defect lives at the seam: a mock on either side makes the other half's correctness an assumption. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pbu27iNUfQCHeuS551Rqo7
…lytics-filter-refusal-envelope
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
📓 Docs Drift CheckThis PR changes 2 package(s): 15 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
勘误 —— 正文「五、测试」里反向验证那句的计数写糊了正文写的是「8 条失败 = 7 处裸 反向验证里 service-analytics 侧 9 条拒收用例失败 8 条,这 8 条覆盖的是 7 个不同的裸 剩下的对应:
所以:9 处拒收 = 7 个裸 数字本身没错,句子没写完。代码与测试均无需改动。 Generated by Claude Code |
- filter.zod.ts:按 #5323 同步散文预留的交接("The declaration flips to stated contract with that PR"),空组合子单位元从「Deliberately NOT declared」段转为正式契约段;{field:{}} 半边保持未宣告(#5376 仍开)。 - filter-refusal-envelope.test.ts(#5366 新到):空数组两行从 REFUSALS 翻入 ACCEPTED(单位元树断言),同一守卫点的非数组拼写补位 REFUSALS,信封不变。 - filter-logic-conformance.ts:族 1 段落按分工删除(四行已进表),族 2/3 原样。 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ErbEDVAg1No9gdg1pgDAGB
…tstack-ai#5322) (objectstack-ai#5365) * fix(service-analytics): 空组合子按布尔单位元归约,两个编译器对齐五后端 (objectstack-ai#5322) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ErbEDVAg1No9gdg1pgDAGB * chore: changeset for objectstack-ai#5322 (service-analytics + spec) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ErbEDVAg1No9gdg1pgDAGB * fix(service-analytics): 空数组单位元在 objectstack-ai#5335 的 const 节点体系上重放,pin 翻向 (objectstack-ai#5322) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ErbEDVAg1No9gdg1pgDAGB * chore(objectstack-ai#5322): 收官同步 —— filter.zod 空组合子宣告转正、objectstack-ai#5366 refusal 表随裁定翻向 - filter.zod.ts:按 objectstack-ai#5323 同步散文预留的交接("The declaration flips to stated contract with that PR"),空组合子单位元从「Deliberately NOT declared」段转为正式契约段;{field:{}} 半边保持未宣告(objectstack-ai#5376 仍开)。 - filter-refusal-envelope.test.ts(objectstack-ai#5366 新到):空数组两行从 REFUSALS 翻入 ACCEPTED(单位元树断言),同一守卫点的非数组拼写补位 REFUSALS,信封不变。 - filter-logic-conformance.ts:族 1 段落按分工删除(四行已进表),族 2/3 原样。 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ErbEDVAg1No9gdg1pgDAGB * test(rest): objectstack-ai#5352 信封 suite 的空 $or 行随 objectstack-ai#5322 拍板翻向 —— 单位元 200+行数语义,非数组拼写补位 400 REST 层是 objectstack-ai#5352 refusal pin 的第三份拷贝(service-analytics 两份已翻)。 harness 的 executeAggregate 从常量改为按引擎侧 filter 求值,四条单位元 断言(空 $or 零行、空 $and 全部行、{$not:{}} 零行、{} 析取项吸收) 因此承重 —— 200 之外还钉行数,与被丢弃的 filter 可区分。 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ErbEDVAg1No9gdg1pgDAGB --------- Co-authored-by: Claude <noreply@anthropic.com>
…pile instead of dropping it (objectstack-ai#5345) (objectstack-ai#5375) `MemoryAnalyticsService` lowers `AnalyticsQuery.where` into a flat cube-style `{member, operator, values}` list, and answered everything that did not fit with `continue` — `$or` and `$not` wholesale, plus the five declared operators with no row in the mongo→cube table (`$between`, `$startsWith`, `$endsWith`, `$null`, `$regex`). A comment presented this as a feature: "ignore so a partial query still runs rather than failing entirely" — the identical reasoning ADR-0078 / objectstack-ai#4286 judged wrong on `objectql`'s `having`. Deleted along with the behaviour it justified. The direction is what makes it a defect: a dropped predicate is FEWER constraints, therefore MORE rows. Measured on `FILTER_LOGIC_CASES`, 15 of its 17 cases returned a wider row set than the standard specifies, usually every row. `$not` makes it more than a wrong number — `cel-to-filter.ts` compiles a CEL `!expr` RLS read scope into `{$not: {…}}`, so the dropped branch was the read scope itself. Route 1 of the issue's two: refuse, do not backfill the cube pipeline — where objectstack-ai#5366 and objectstack-ai#5368 put the two neighbouring faces. Implementation reuses objectstack-ai#5349's primitives rather than adding a second validator. `assertFilterConditionShape` now takes the calling face's `FilterFaceCapabilities` — what that face can COMPILE, distinct from what the protocol declares — and refuses the difference through `unsupportedFilterError` (INVALID_FILTER / 400). The query path and the matcher pass nothing and are byte-for-byte unaffected. The analytics face derives its capabilities from its own `MONGO_TO_CUBE_OPERATOR` table, so widening what it accepts and teaching it to compile the operator are now one edit. Two refusals distinct on purpose: an operator the protocol never declared is still `unknownFieldOperatorError` ("you have a typo"), one it declares and this face cannot lower is the new `uncompilableFieldOperatorError`. The gate runs in `normalizeFilters`, before any lowering, for the reason that module already documents — a refusal raised mid-lowering fires or not depending on key order. Both public entry points (`query()`, `generateSql()`) go through it. The two former `continue` sites now throw; the operator one is reachable via the nested-relation branch, which re-enters with a synthesised node the gate never walked. Tests: `FILTER_LOGIC_CASES` now covers this third face — the package had three filter surfaces and the shared table watched two. It cannot pass row-for-row (a cube pipeline has no `$or`), so it is held to the predicate that actually matters: agree with `find()`, or refuse with INVALID_FILTER, never a third quieter answer. Reverting only `memory-analytics.ts` fails 16 of the new assertions. Plus a dedicated suite asserting the envelope (code, status) and that each refusal names the offending operator or combinator. Out of scope, filed not fixed: objectstack-ai#5373 (the cube comparand round-trip loses booleans and `null`), objectstack-ai#5374 (`$notContains` lowers to a bare mingo `{$not: 'x'}` that constrains nothing). Claude-Session: https://claude.ai/code/session_01Pbu27iNUfQCHeuS551Rqo7 Co-authored-by: Claude <noreply@anthropic.com>
…in 滞后、死代码删除复核 (objectstack-ai#5513) (objectstack-ai#5645) 2026-08-05 跑完一整条 filter 缺陷链(objectstack-ai#5363 / objectstack-ai#5366 / objectstack-ai#5368 / objectstack-ai#5375 / objectstack-ai#5431 / objectstack-ai#5445, cloud#1117)后回看,六处在那一轮真实咬过人或真实救过场的规程,SKILL 里没有对应条目。 六条各落在 issue 指定的节内,**纯增补**:111 行插入、0 行删除,既有条目(objectstack-ai#5501 的接力 模式、objectstack-ai#5522 的座位模型、objectstack-ai#5630 的 assertEngineDeleteDispatch 条款)一字未动。 落点与要点: 1. **Multi-repo,rule 2 之后**「pin 滞后」——`Blocked-by:` 只保证上游已合并,姊妹仓还有 第二个读数:本仓 pin 是否覆盖那个 commit。cloud#1116 的裁决落于 framework objectstack-ai#5368 (`9c5abf4e9`),而 cloud 的 `.objectstack-sha` 未覆盖它,于是 `TursoDriver` 有一个 方向反了的分叉窗口(fail-closed 一侧先到)。规程:派发前核祖先关系;未覆盖则 dev 在 PR 正文留档窗口与方向,⛔ pin bump 不做 rider。 2. **step 3** 末「阻塞解除后重新定价」—— 前一单合入会改变后一单的成本模型,方向不止一个 (本轮变便宜、没变、成本估计过期各有实例)。两个动作配对:派发前一单时带必答项 「你的改动是否让 #X 变简单 / 变难 / 不必要 / 无影响」,派发被延后那单前用该回答重读 其选项与成本估计。 3. **step 5** 派发令「多面组件的测试落点」—— 同一契约 ≥2 实现面时,新用例进共享一致性 覆盖而非独立文件(原话照录)。附 objectstack-ai#5375 / objectstack-ai#5431 / objectstack-ai#5445 三条正交轴共用一条不变量。 4. **step 7 清单**「收益穿过它必经的那道边界之后还在吗」—— 判据是价值主张是否依赖下游 如实转发;实例即 objectstack-ai#5423(4xx 直通曾整条替换 ≥500 字符正文,`code` 到了正文没到)。 5. **step 7 清单**「死代码删除的复核」——「这是死代码」是断言而非能从 diff 读出的事实, PM 在 origin/main 独立核一次引用面再 ACCEPT(查法用 Operational notes 6:notes 6 说 怎么查不假阴性,本条说什么时候必须查)。 6. **step 8** 升级门槛之后「带前提的裁决」—— 分歧关键是可被代码证伪的事实时,第三档 = 裁决 + 前提验证要求 + 「前提不成立报 fork,不许硬做也不许悄悄改选」禁令,三件缺一 不可;缺第 3 条即退化为无人裁决且无读数显示。 实施时两处核实结果与 issue 正文不同,成文按核实后的事实写: - issue 的附带论断「没有任何闸门在量这个 pin 滞后」**不成立** —— cloud 的 `scripts/check-pin-staleness.sh`(test.yml 以 `continue-on-error` 跑)每次 CI 都报两个 pin 各落后 main 多少 commit,advisory 是**有意设计**(`--max-behind` 需显式传)。它答 的是「落后多少」,不是「是否覆盖我这条裁决 commit」;成文因此指向该脚本,并只把后一个 问题留给派发前的祖先判断。据此**未**另立「无闸门」的发现单。 - 第 4 条的 rest-server 缺陷本身已由 objectstack-ai#5423 按「截断而非替换」修掉,成文改用过去时并注明, 以免后来的读者去找一个已不存在的活 bug;该条要补的是**复核清单的缺口**,与代码是否已修 无关。 第 1 / 3 条按 issue「未验证的部分」的克制写入适用判据(前后单共用同一契约或数据表示; 组件对同一契约有 ≥2 实现面),形态迥异的批次(纯 UI、纯文档)明确不强加。 验证:`node scripts/check-nul-bytes.mjs --self-test` + 全仓扫描绿(48 断言 / 5537 文件); 改动文件自扫 `grep -naP '[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]'` 零命中,并用邻近词反查证伪 「扫描器坏了」;`check:docs-audit-scope` 绿;markdown 结构核对(强调标记成对、代码围栏 16 个偶数、嵌套围栏缩进对齐)。 Claude-Session: https://claude.ai/code/session_01GX3sL71LFq8m2usg6VqTSE Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: os-zhuang <hr@objectstack.ai>
…D/400 and shrink the route's message list to one entry (objectstack-ai#5367) (objectstack-ai#5723) `POST /analytics/dataset/query` classified six error families by matching hardcoded substrings of their message text, because all six producers were bare `throw new Error(...)`. That made their HTTP status a property of their wording: a rephrasing with no logic change moved a refusal from 400 to 500 with nothing going red. objectstack-ai#5352/objectstack-ai#5366 delivered declared/loud/tested for that accommodation; Prime Directive objectstack-ai#12 also requires removable on a schedule. Five producers now declare the verdict themselves through a new `dataset-refusal.ts` (`datasetInvalidError`, `DATASET_INVALID`/400, the same shape as `invalidFilterError`): - dataset-compiler: unsupported aggregate; undeclared relationship path - dataset-executor: unselected order key; totals grouping outside the selection - native-sql-strategy: join outside the declared allowlist Their five entries are deleted from the route's regex. `read-scope-sql` keeps its entry on purpose: its ten fail-closed refusals lower an admin-authored RLS policy, not caller input, so the right code/status is a separate judgement. `DATASET_INVALID` is registered under `@objectstack/service-analytics` in ERROR_CODE_LEDGER for provenance (the union is unchanged), and the constructor types it as `RegisteredErrorCode` so an unregistered code fails `tsc`. Co-authored-by: Claude <noreply@anthropic.com>
Fixes #5352
作者在 dashboard widget 的筛选里拼错一个算子,analytics 响亮拒收 —— 这是 #3948 / #5240 / #5325 / #5334 一路建立的姿态,正确。但这个拒收到不了作者:它以
500 ANALYTICS_QUERY_FAILED落地,读作「平台炸了」而不是「你的筛选写错了」,运维告警按 5xx 计。同一个错误走find()从 #3948 起就答400 INVALID_FILTER—— 一个作者错误,两种线上形状,取决于哪张脸接住了它。这一单是让前面那几单的拒收真正到达作者的最后一段。
一、现场核对(两次同文件 churn 之后)
worktree 基于
origin/main的e18e3da77(含 #5325/PR #5335 与 #5334/PR #5355,即b8573561e),逐条重核 issue 引用的四处拒收。结论:四条全部仍在,措辞未变,但文件里实际有 9 处拒收,不是 4 处。Unsupported filter operator "$foo" on "col"Error"col" carries a field constraint with zero operators ({})(#5240)Error"$between" on "col" needs a two-element [min, max] arrayError"$and"/"$or" requires a non-empty arrayError"$and"/"$or" branches must be filter objectsError"$not" requires a filter objectErrorUnsupported top-level filter operator "$foo"Errorreceived a 'where' array that is not a filter(#5334)isFilterAST/parseFilterAST分歧(#5334)issue 正文的四条 bullet 覆盖 1–5(第四条 bullet 一句话盖了 4 和 5),漏了 6 和 7。6 和 7 一并信封化了,理由不是顺手:半信封化的模块在 REST 边界上与完全没信封无法区分 —— 写
{$not: 5}的作者会拿到 500,写{$nott: {...}}的作者拿到 400,这正是本单要消灭的「一个条件两种形状」在文件内部复现。两处都在filter-normalizer.ts内、都只改code/status,没有越过硬禁区。invalidFilterError沿用 #5334 已落的那个,没有另造第三套;只是把它从「#5334 的 FilterArray 门」小节提到模块顶部,并写明它现在是本模块唯一的拒收方式(新增裸throw new Error就是缺陷回归)。二、半 B —— 生产面
packages/services/service-analytics/src/strategies/filter-normalizer.ts:9 处拒收全部经invalidFilterError→INVALID_FILTER/ 400。三、半 A —— 消费面
packages/rest/src/rest-server.ts,analytics dataset 路由的 catch:先读error.status/error.code,有信封按信封答;正则名单降级为兜底。两个刻意的收紧,都写进了代码注释:
status且非空code)才走信封分支。只有 status 没有 code 的话,这里就得自己发明一个 code —— 那正是 ADR-0112 要消灭的消费方宽容;半个信封是生产方的 bug,应该被发现,不该在这里被抹平。ANALYTICS_QUERY_FAILED信封(并保留logError),这样内部故障永远不会被重新贴成调用方的错、也不会绕过运维可见的那行日志。正则名单的处置:整条留着,逐条给了证据
PM 要求「若判断名单里某几条其实已带信封,逐条给证据再删」。逐条核过了 —— 六条全部仍是裸
throw new Error(...),一条都删不掉:not declared in the datasetdataset-compiler.ts:305throw new Error(not backed by a declared relationshipstrategies/native-sql-strategy.ts:222throw new Error(not supported by the v1 dataset runtimedataset-compiler.ts:137throw new Error(read-scope-sqlread-scope-sql.ts(73/107/113/144/163/171/195/200/205/215)throw new Error(not a selected dimension or measuredataset-executor.ts:436throw new Error(is not a subset of the selected dimensionsdataset-executor.ts:596throw new Error(删掉名单会让这六族从
400 DATASET_INVALID退化成 500。名单原地保留,并加了注释写明:它是过渡态,是这些家族信封化之前的占位,不是第二套分类机制;新增拒收请带code/status,信封分支免费服务它。顺带被修好的两条(不是范围外,是同一个缺陷)
信封是泛化读取的,不是
if (code === 'INVALID_FILTER')白名单 —— 那种写法就是 message 嗅探换了身衣服。于是另外两个生产方早已声明、这条路由一直在丢弃的信封也自动到位了:INVALID_FIELD/ 400 ——analytics-service.ts:1128(analytics: a measure naming a missing field 500s with SQLITE_ERROR instead of a 400 naming the field #4437)。它自己的注释写着「DATA 路由已经用400 INVALID_FIELD拒收同样的错误;这个 hook 就是为了让 ANALYTICS 路由给出同样的答案」。答案一直在,只是被这条 catch 丢了。CUBE_NOT_FOUND/ 404 ——analytics-service.ts:1170(analytics /query 未做 cube 存在性校验,未注册名直达驱动当表名;且错误路径原样回显驱动 SQL(#3770 同类,另一子系统) #3867)。它的 doc 写着「以status: 404/code: 'CUBE_NOT_FOUND'拒收,好让 HTTP 边界答『没有这个 cube』」。同样被丢弃。也就是说这条路由丢的不是一个信封,是三个。
四、只改形状,不改判定 —— 证明
没有动任何拒收的判定条件:没有输入从「通过」变成「被拒」,也没有从「被拒」变成「通过」。
证明方式是
filter-refusal-envelope.test.ts里两个互相独立的 describe:the refusal SET is unchanged—— 9 条拒收输入(逐条断言仍然抛、消息仍然匹配)+ 8 条必须继续被接受的输入,并且断言它们编译出的NormalizedFilterNode树逐字段相等(含 service-analytics 的第二个 SQL 编译器filter-normalizer.buildNode仍带着 #5297 的三条分叉:$not非 NULL-safe、{$not:{}}不加 WHERE、$or的{}析取项被丢 #5325 的{$in: []}→ FALSE 常量、{}→ TRUE、analytics 是 #5158 拍板 C 漏掉的第五道门:where为数组(FilterArray 糖)时被normalizeAnalyticsFilterTree静默丢弃,图表画全表 #5334 的[]与可下沉数组)。这一块 改前改后全绿,这就是它的全部职责。every refusal carries the ADR-0112 envelope—— 变更本身。反向验证的输出见下表,可以直接读出「只有信封断言在动」。
五、测试
反向验证(stash 掉两处源码改动,重建 dist,跑新用例)
@objectstack/service-analytics(28 例):8 条失败 = 7 处裸
Error+ ...(第 9 处isFilterAST分歧分支不可达);第 8 处(#5334 数组门)本来就带信封,所以通过。「拒收集合不变」那 20 条全绿 —— 改前就绿,改后也绿。@objectstack/rest(19 例):expected 500 to be 400—— 缺陷本身,一字不差。通过的那 10 条正是三类回归护栏(正则名单仍答DATASET_INVALID、内部错误仍 500、5xx/半信封不放行)+ 200 正控,它们改前改后都绿。端到端(本单的验收面)
packages/rest/src/analytics-filter-refusal-envelope.test.ts的 provider 是真的AnalyticsService,不是 mock —— 缺陷正在两层的接缝处,任何一侧 mock 掉都等于把另一侧的正确性当假设。为此给@objectstack/rest加了@objectstack/service-analytics的 devDependency(沿用该包已有的先例:@objectstack/objectql/@objectstack/metadata-protocol也只在集成测试里用,import-job-integration.test.ts/export-integration.test.ts)。无依赖环:service-analytics 只依赖 core + spec。打进去的是
selection.runtimeFilter—— presentation-scope 筛选,正是 dashboard widget 携带、作者会打错的那个字段。runtimeFilter: {stage: {$sortOf: 'won'}}body.code === 'INVALID_FILTER',且显式断言 不是 500 / 不是ANALYTICS_QUERY_FAILED,message 仍点名$sortOfruntimeFilter: {stage: {$eq: 'won'}}{}/$between单边 / 空$or/$or非对象分支 /$not非对象 / 顶层未知算子)INVALID_FILTERErrorDATASET_INVALIDECONNRESET: socket hang up …ANALYTICS_QUERY_FAILEDstatus: 503+code: 'WAREHOUSE_UNAVAILABLE'ANALYTICS_QUERY_FAILED(5xx 不放行)status: 400,无codeINVALID_FIELD/ 400、CUBE_NOT_FOUND/ 404连带面全量(合入
origin/main之后重跑)turbo typecheck --filter=@objectstack/rest --filter=@objectstack/service-analytics --force→Tasks: 17 successful, 17 total,0 error TS。eslint --no-inline-config packages/rest/src packages/services/service-analytics/src→ 0 problem,无as any新增(测试里的 logger 用了真实Logger类型)。check:error-code-casing✓ /check:route-envelope✓(rest-server.ts不在该 gate 的审计表内)。本地跳过 / CI 才跑的盲区
如实核查:
grep -rn "skipIf|describe.skip|it.skip|test.skip|todo(|.only("在packages/rest/src与packages/services/service-analytics/src下 零命中。两个包没有条件跳过的用例,上面的数字就是全部。六、可观察的行为变更(changeset 里也如实写了)
同一个请求,
500 ANALYTICS_QUERY_FAILED→400 INVALID_FILTER(以及400 INVALID_FIELD/404 CUBE_NOT_FOUND)。按 5xx 做告警的会看到 5xx 率下降、4xx 率出现;按 5xx 做重试的会停止重试一个重试多少次都一样的请求。 两者都是本单想要的纠正 —— 这个条件从来就是调用方的错误 —— 但它们是可见的,所以写明而不是埋掉。七、范围外
filter-normalizer.ts以外的 analytics 编译逻辑、driver-sql/**、driver-memory/**、memory-analytics.ts、packages/spec/**、content/docs/releases/**。@objectstack/spec会让gen:schema把packages/spec/authorable-surface.base.json的baseRev重锚到当前 HEAD 并删掉 ~110 个 key。每次跑完重命令都已git checkout --还原,最终 diff 不含该文件。属 AGENTS.md §9/§11 已知的生成物类别。Generated by Claude Code