Skip to content

fix(service-analytics,rest): analytics dimension 的源字段闸门 —— 不存在的 dimension 答 400 INVALID_FIELD,dataset 500 不再回显 SQL (#5520) - #5667

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-5520-dimension-field-gate
Aug 5, 2026
Merged

fix(service-analytics,rest): analytics dimension 的源字段闸门 —— 不存在的 dimension 答 400 INVALID_FIELD,dataset 500 不再回显 SQL (#5520)#5667
os-zhuang merged 1 commit into
mainfrom
claude/issue-5520-dimension-field-gate

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes #5520

前提重验(对 origin/main,含 #5567/PR #5587#5526/PR #5634 之后的最新落点)

单子的前提成立,且三条复现都在测试双上原样重现:

  • origin/mainassertDimensionFields 仍是零命中;ensureCube() 的三处校验全部只调 assertMeasureFields
  • 裸 cube 路 dimensions: ["bogus_dim"] → 生成 SELECT bogus_dim AS "bogus_dim", COUNT(*) … GROUP BY bogus_dim,驱动答 no such column,错误对象上 code/status 皆为 undefined(所以在 REST 面落到 5xx 兜底)。
  • dataset 路 → 同一条语句,错误 message 与单子里贴的一字不差:SELECT bogus_dim AS "bogus_dim", COUNT(*) AS "account_count" FROM "crm_account" GROUP BY bogus_dim - no such column: bogus_dim
  • 「真实但未声明的字段仍可分组」也确认为既有契约:dimensions: ["phone"] 在两条路上都是 200,按 phone 分组。闸门必须不误杀它。

改了什么

1. assertDimensionFields —— 对称 #4437 的 measure 闸门

ensureCube() 的三条出口(自动推断、augmented、已声明)现在都在注册 cube 之前紧跟 assertMeasureFields 调用新闸门。拒收信封与 measure 侧完全相同:INVALID_FIELD / 400,带 field / object / param,外加 dimension 字段(对称 measure 侧的 measure);message 指名字段、列出可用 dimension、附上对象已知字段清单。query / generateSql / queryDataset 三个入口都被覆盖,被拒的查询不会在 cube registry 里留下痕迹(#3867 / #4437 定的同一条规矩)。

取舍(逐条给代码证据)

  • 校验的是「承载对象有没有这个列」,不是「cube 声明过没有」。未声明的裸成员按策略自己会用的列名去核对 —— NativeSQLStrategy.resolveDimensionSql 的回退是 dim ? dim.sql : member,ObjectQLStrategy.resolveFieldName 同形。所以 phone 这类未声明真实字段照常放行(有守卫用例,两条路各一条)。
  • timeDimensions 一并覆盖。它和 dimensions 落进同一个 cube.dimensions 袋子(inferCubeFromQuery 两者都写进去)、被同一个 lookupMember 解析、并且实测同样把 date_trunc('month', bogus_at) 送到驱动 —— 是同一个缺陷的同一个 param 变体,不是新面。param 报告是哪一个请求键出错。这一点比单子正文的字面范围略宽,在此显著申报:若 PM 认为该拆单,删掉 members 数组里 timeDimensions 那一行 + 两条用例即可。
  • 成员解析镜像 lookupMember,包括它故意的最后一档:点号成员若匹配不到任何声明键,就是交给 JOIN 机器的关系穿越,闸门不判(source: null)。否则 owner.region 会被诬告成 crm_account 缺列。
  • 三档 stand-down 与 measure 闸门逐条对齐:cube.sql 不是裸对象名(表达式 cube)、getObjectFieldNames 不作答(无 registry 的宿主 / 外部数据源)、源不是裸列。id / created_at / updated_at 无条件放行,与数据面 resolveQueryFields 一致。
  • 建议列表用减法算,不照抄。自动推断路上 cube.dimensions 就是从这条查询铸出来的,照抄会把调用方的错别字当成「可用 dimension」推荐回去 —— 与 measure 闸门两趟扫描的理由相同。
  • measure 先答。两样都写错时先报 measure(既有信封不动),各自都是真实错误。

2. dataset 面回显生成 SQL —— 扩面申报

回显字符串的构造点不在本仓:knex 的错误 message 格式就是「语句 - 成因」,所以 SQL 是驱动 message 自带的。收窄只能在对外信封处做,于是本 PR 动了 packages/rest/src/rest-server.ts 一行 —— 按派发口径显著申报:

- res.status(500).json({ code: 'ANALYTICS_QUERY_FAILED', error: msg.slice(0, 500) });
+ const outward = looksLikeInternalErrorLeak(msg) ? INTERNAL_ERROR_MESSAGE : msg.slice(0, 500);
+ res.status(500).json({ code: 'ANALYTICS_QUERY_FAILED', error: outward });

不是新规则,而是补上缺失的那次应用:同一个 analytics 的兄弟面 /analytics/querydispatcher-plugin.errorResponseBase,自 #3867 起就对任何 5xx message 应用共享谓词 looksLikeInternalErrorLeak(packages/types/src/error-leak.ts)—— 这正是单子里复现 ① 读到 "Internal server error"、复现 ③ 却吐出整条语句的原因。本 PR 只补这一处应用:

测试

新增两套,共 30 例:

  • packages/services/service-analytics/src/__tests__/dimension-source-field-gate.test.ts(22 例)—— 结构与深度对称 measure-source-field-gate.test.ts:三条 describe 分别是闸门本体、dataset 面、「闸门不许做什么」。含未声明真实字段仍可分组的守卫用例(裸 cube 路 + dataset 路各一)、registry 不被污染、generateSql 同样受闸、表达式 cube / 点号关系 / 无探针 / 探针答不出四档 stand-down、以及「两样都错时先答 measure」。
  • packages/rest/src/analytics-dataset-dimension-gate.test.ts(8 例)—— 调用方视角:bogus dimension 端到端答 400 INVALID_FIELD 且响应体里没有 SQL;正向对照(声明维度 200 + 未声明真实字段 200);500 面收窄的三例(带语句的 message 被 withhold 且仍进日志、方言错误码同样 withhold、普通内部故障保留原文);以及 4xx 两条分支未被扰动。

反向验证(方向先预测再跑)

方向 A —— 拿掉 ensureCube 里三处 assertDimensionFields:预测 service 套件 10 红 / 12 绿(前两个 describe 里断言「拒收」的用例转红;第三个 describe 全绿,因为它们钉的是本改动保留的旧行为;dataset describe 里的「pre-fix 驱动错误确实带着语句」那一例是对照,前后都绿)。实测:Tests 10 failed | 12 passed (22),方向与条数完全吻合。

同一方向在 REST 套件上出现一次预测偏差,如实记录:预测 3 红,首轮只有 2 红 —— 「响应体里没有 SQL」那一例在闸门被拿掉时仍然绿,因为另一半修复(信封收窄)把驱动 message withhold 掉了,body 于是「无泄漏」却并非因为闸门。这正是「绿得不是因为逻辑对」的那类假绿:该用例已改为同时断言 400 INVALID_FIELD + 无 SQL,重跑得到预测的 3 红。偏差与修法都写进了测试文件头注,连同「本文件吃的是 service-analytics 的构建产物,不重新 build 就改 service 什么也证明不了」这条坑。

方向 B —— 把 500 分支还原成 error: msg.slice(0, 500):预测 REST 套件 2 红(两条 withhold 用例)/ 6 绿,service 套件不受影响。实测:Tests 2 failed | 6 passed (8),吻合。

门与全量

命令 结果
pnpm --filter @objectstack/service-analytics test 52 files / 927 tests passed(基线 51/905,新增 1 文件 22 例)
pnpm --filter @objectstack/rest test 50 files / 745 tests passed
pnpm --filter @objectstack/runtime test(另一条 analytics 面) 97 files / 1428 tests passed
pnpm --filter @objectstack/dogfood test(真实 plugin.ts 探针接线) 85 passed / 1 skipped, 514 tests
tsc --noEmit(两包) 各自与基线同数:service-analytics 7、rest 2 —— 新文件贡献 0(没有照抄兄弟文件 .catch 的 4 条 TS2339 债,改用带类型的 rejection() / settle() 辅助)
check:type-check-coverage / check:nul-bytes / check:route-envelope / check:error-code-casing / check:adr-anchors / check:doc-authoring / check:wildcard-fallthrough 全绿
eslint(改动文件) 无输出

已合入 origin/main(aa25a81d3 / 214f67c76,均在 cli/client,与本 diff 无重叠),合并后两包全量重跑仍绿。

消费半径

闸门只在 getObjectFieldNames 接线时生效;全仓该 hook 的接线点只有生产桥 service-analytics/src/plugin.ts 与三个测试文件(measure 闸门、本 PR 两套)。生产桥那条路由 dogfood 套件覆盖(真 kernel + 真引擎),全绿 —— 说明闸门在 showcase / CRM 的真实 dashboard、report 元数据上不误杀。compiled dataset 的 cube.sql 就是 dataset.object(裸对象名),而关系维度的 dim.sql 是点号路径(dataset-compilersql: d.field 原样透传),前者让闸门对 dataset 生效、后者被点号档跳过 —— 两者都有用例。


Generated by Claude Code

…s fields, and stop the dataset 500 echoing SQL (#5520)

#4437 gave a measure over a non-existent field a `400 INVALID_FIELD` naming the
field — a driver error class must never be the caller's `error.code` for a
caller-shaped mistake (ADR-0112). It covered the measure half only, so the
identical typo one request key over still reached the driver as a `GROUP BY`
column and came back as `500 SQLITE_ERROR`, while the measure control group on
the same route answered a clean 400.

`ensureCube` now runs `assertDimensionFields` alongside `assertMeasureFields` on
all three of its paths, so a dimension whose source column the backing object
does not have is refused before any SQL exists, with the same envelope
(`INVALID_FIELD`/400 + `field`/`object`/`param`) and a message naming the field,
the valid dimensions and the object's known fields. `query`, `generateSql` and
`queryDataset` are all covered; a rejected query leaves the cube registry as it
found it. `timeDimensions` are covered too — same `cube.dimensions` bag, same
`lookupMember`, same 500 — with `param` naming the key that carried it.

Grouping by a REAL field the cube never declared keeps working: the question is
"does the object have this field", never "did the cube declare this dimension".
Expression cubes, dotted relation dimensions and probe-less hosts stand down
exactly as the measure gate stands down.

`POST /analytics/dataset/query` composed its own 5xx body and echoed the message
verbatim, so a knex `<sql> - <cause>` error handed the caller the generated
statement with its physical table and column names. The sibling face never did:
`/analytics/query` exits through `dispatcher-plugin.errorResponseBase`, which
has applied the shared `looksLikeInternalErrorLeak` predicate to every >= 500
message since #3867. That predicate now guards this route's 500 body too —
status, code, the ADR-0112 envelope branch and the transitional message list all
unchanged, and the full text still reaches `logError`.

Fixes #5520

Co-Authored-By: Claude <noreply@anthropic.com>
@vercel

vercel Bot commented Aug 5, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 5, 2026 11:32pm

Request Review

@github-actions github-actions Bot added size/xl documentation Improvements or additions to documentation tests tooling labels Aug 5, 2026
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 2 package(s): @objectstack/rest, @objectstack/service-analytics.

15 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/connect-mcp.mdx (via @objectstack/rest)
  • content/docs/api/data-api.mdx (via @objectstack/service-analytics)
  • content/docs/api/error-handling-server.mdx (via @objectstack/rest)
  • content/docs/api/index.mdx (via @objectstack/rest, @objectstack/service-analytics)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/service-analytics)
  • content/docs/permissions/authentication.mdx (via @objectstack/rest)
  • content/docs/permissions/sharing-rules.mdx (via @objectstack/service-analytics)
  • content/docs/plugins/index.mdx (via @objectstack/rest)
  • content/docs/plugins/packages.mdx (via @objectstack/rest, @objectstack/service-analytics)
  • content/docs/protocol/kernel/http-protocol.mdx (via @objectstack/rest)
  • content/docs/protocol/kernel/i18n-standard.mdx (via packages/rest)
  • content/docs/releases/implementation-status.mdx (via @objectstack/rest, @objectstack/service-analytics)
  • content/docs/releases/v12.mdx (via @objectstack/rest)
  • content/docs/releases/v17.mdx (via @objectstack/rest, @objectstack/service-analytics)
  • content/docs/releases/v9.mdx (via @objectstack/service-analytics)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@os-zhuang
os-zhuang marked this pull request as ready for review August 5, 2026 23:43
@os-zhuang
os-zhuang added this pull request to the merge queue Aug 5, 2026
Merged via the queue into main with commit c36abfe Aug 5, 2026
24 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-5520-dimension-field-gate branch August 5, 2026 23:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/xl tests tooling

Projects

None yet

2 participants