Skip to content

fix(service-analytics): 带 measure-scoped filter / derived 度量的 dataset 查询,fields 也描述维度列 (#5537) - #5691

Merged
os-zhuang merged 1 commit into
mainfrom
claude/issue-5537-dataset-dimension-fields
Aug 6, 2026
Merged

fix(service-analytics): 带 measure-scoped filter / derived 度量的 dataset 查询,fields 也描述维度列 (#5537)#5691
os-zhuang merged 1 commit into
mainfrom
claude/issue-5537-dataset-dimension-fields

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Fixes #5537

前提重验(对 origin/main c36abfe,含 #5587 / #5634 / #5667

核心前提成立,但立单时的落点分析与影响面各有一处需要修正。

成立的部分。 A/B/C 三形态逐条复现,新测试在未改动源码时的结果就是立单描述:

A  {dimensions:['owner'], measures:['opp_count']}                        → fields ['owner','opp_count']            绿
B  {dimensions:['owner'], measures:['won_count']}                        → fields ['won_count']                    红
C  {dimensions:['lead_source'], measures:['won_count','decided_count','win_rate']}
                                                                          → fields ['won_count',…] 无维度          红

修正一 —— 触发条件比「带 filter 或 derived」更窄,而且 derived 不是独立成因。
真正的判据是「基础度量是否全部带自身 filter」。runMeasurePass 只在存在无 filter
度量时才发主查询;混合选择(['opp_count','won_count'])照旧发主查询,维度描述符一直都在
—— 该形态在修复前就是绿的,已作为守卫钉住。derived 度量本身也从不丢维度:它只在
依赖全部带 filter 时丢(C 的 win_rate),而一个依赖无 filter 度量的 ratio
(avg_deal)修复前后皆绿,也已钉住。所以立单里「一旦查询走了 measure-scoped filter
合并含 derived 度量的路径」这个 or 偏宽了;去找一条 derived 专属的缺陷路径会白跑。

修正二 —— 维度没有 format 可丢。 立单要求断言 {name,type,label,format},并把
「数值维度额外丢失 format」记为症状。DatasetDimensionSchema(packages/spec/src/ui/dataset.zod.ts)
只声明 name / label / field / type / dateGranularity —— 没有 format;
formatDatasetMeasureSchema 的键。AnalyticsResult.fields[].format 因此只由度量
富化产出,在任何路径上(含一直正常的 A)都不为维度产出。也就是说不存在「A 有、B 没有」
的维度 format,断言它等于断言一个幻影。日期维度的分桶显示是 resolveDimensionLabels
在服务端改写行值完成的,不走 field format。故本 PR 的维度描述符 = {name, type, label},
type 明确按例断言(日期轴是 time,不是渲染端默认的 string)—— 这是 A 路径实际产出的
形状,也就是收敛目标。

修正三 —— 立单指的两个可选落点里,「service 侧新建 field 条目」是错的方向。 见下。

修复位置的代码证据

产出 fields 的一共三处,且都是「维度在前,度量在后」:两个 strategy 的 buildFieldMeta
(objectql-strategy.ts L1138 / native-sql-strategy.ts L748)与草稿预览的
preview-evaluator.ts L250。投影集合的定义在 projectedDimensions(objectql-strategy.ts
L1130):每个 dimensions 条目,加上每个带 granularity 且不在其中的 timeDimensions 条目
—— #4033 的「one definition, every consumer」。

DatasetExecutor.runMeasurePass 的补充子查询已经拿到了这些描述符(sub.fields),
只是把它们扔了,只留 sub.rows 去 merge。因此选 executor 侧、并且从子结果里取:

if (!primary && i === 0) {
  for (const f of sub.fields ?? []) {
    if (!measureNames.has(f.name)) result.fields.push(f);
  }
}

为什么不在 executor 里从 compiled.cube 重算:那会成为「哪些维度被投影」这条规则的第四份
拷贝
,可以与它所标注的那些行漂移(#4033 正是这个漂移的原始伤口)。读回子结果则让两条路径
按构造收敛 —— 顺序、type、以及当前 strategy 究竟投影了什么,全都一致,包括 native
与 objectql 两条 strategy 各自的答案。

为什么不在 service 侧「维度缺失时新建 field 条目」:那一段(analytics-service.ts L915 起)
只应富化,不应造列 —— 某列是否存在是查询层的答案,在这里凭 selection.dimensions
造一条,可能描述出行里根本没有的列(反之亦然:上面 #5688 那种只在 timeDimensions 里的列,
service 侧的 selectedDims 看不见,却确实是结果列)。已在该处补注释写明这条边界与 #5537
的因果。

字段序:两条路径现在都是 [维度…, 无 filter 度量…, 带 filter 度量…, __compare…, derived…],
按 exact toEqual 钉住(不是 toContain)。

反向验证(先预测方向,再跑)

预测:还原 executor 那一处 seam → 「基础度量全部带 filter」的 8 例转红,4 例收敛目标保持绿;
方向普通,无反转、无计数移动(本改动新增原本缺失的 field 条目,不收紧任何规则、不删除
任何 ?? 分支,故下游不可能因此出一个 finding)。

实测(git stash 仅还原源码,测试文件不动):8 failed | 4 passed (12) —— 与预测逐例吻合,
红的正是 B/C 块全部,绿的正是「已正常」块全部。

范围外发现(已立单,未搭车)

#5688 —— 一个只带 dateRangetimeDimensions 条目会被 buildQuery 补上 dataset
的默认粒度,于是「窗口」变成第二层 GROUP BY:{dimensions:['owner']} + 一个日期区间筛选
回来的是按 owner × 月拆开的行,还多一个没人选过的时间列。它在本 PR 从未触及的那条
A 路径上同样复现(已在 origin/main 与本分支上各测一次,输出逐字节相同),修它会改变响应
形状(行数,不是 label),不该作为本单的搭车项。本 PR 把它当数据钉住(含行数断言)并成对
放在两个 block 里,#5688 落地那天这对用例会有意转红。


Generated by Claude Code

…the measure-filter merge path (#5537)

A dataset selection whose base measures ALL carry their own measure-scoped
`filter` (or whose selected derived measures depend only on such measures)
never issues the primary grouped query: `runMeasurePass` starts from a
synthesized `{ rows: [], fields: [] }` and each supplementary sub-query appends
exactly one MEASURE descriptor. The dimension columns are in every row — they
are the merge key — but described nowhere, so a consumer reading column
metadata had no `label` and no `type` for the grouped column and fell back to
humanizing the raw key: "owner" where the dataset declares "Owner".

The dimension descriptors are now adopted from the FIRST supplementary result,
which projects exactly the dimensions the pass groups by. Taken from the
sub-result rather than re-derived from `compiled.cube`: the projected set is
defined by each strategy's `buildFieldMeta` (#4033), so an executor-side
reconstruction would be a fourth copy of that rule, free to drift from the rows
it labels. Reading it back makes the two paths converge by construction — on
order (dimensions first), on `type`, and on whatever the active strategy
projected. `compareTo`, `totals` and derived measures all route through the
same pass, so all three are covered.

The label enrichment in `queryDataset` is left enriching-only, with a note
saying why: whether a column exists is the query layer's answer, and minting
one there would describe a column the rows may not carry.

Tests: `dataset-dimension-field-descriptors.test.ts` asserts the complete
`fields` set and order (exact `toEqual`, full descriptor) for the healthy
single-query path and for every all-filtered shape. Reverse verification —
direction predicted first — reverting the executor seam turns the 8
all-filtered cases red and leaves the 4 convergence-target cases green;
measured exactly that.

Out of scope, filed as #5688: a `timeDimensions` entry carrying only a
`dateRange` acquires the dataset's default granularity, turning a WINDOW into a
second GROUP BY. Reproduces identically on the path this fix never touched;
pinned here as data (row count included) so #5688 landing goes red on purpose.
@vercel

vercel Bot commented Aug 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 6, 2026 12:23am

Request Review

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling size/l labels Aug 6, 2026
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/service-analytics.

8 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/api/data-api.mdx (via @objectstack/service-analytics)
  • content/docs/api/index.mdx (via @objectstack/service-analytics)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/service-analytics)
  • content/docs/permissions/sharing-rules.mdx (via @objectstack/service-analytics)
  • content/docs/plugins/packages.mdx (via @objectstack/service-analytics)
  • content/docs/releases/implementation-status.mdx (via @objectstack/service-analytics)
  • content/docs/releases/v17.mdx (via @objectstack/service-analytics)
  • content/docs/releases/v9.mdx (via @objectstack/service-analytics)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

Copy link
Copy Markdown
Contributor Author

真机线上验证(matched before/after,同一 dataset、同一请求体)

showcase 自带一个可复现的全 filter 形态:showcase_invoice_metrics
paid_count(aggregate: count, filter: { status: 'paid' })—— 只选它就是「基础度量
全部带 filter」。用 pnpm dev -- --fresh -p <随机端口> 起真服务,打 POST /api/v1/analytics/dataset/query。为了让 before/after 真的成对,pre-fix 那一轮是把
dataset-executor.ts 切回 origin/main、只重建 @objectstack/service-analytics
再重启服务后打的(并核对过 dist 里确实不含本次改动)。

修复前

A) {"dimensions":["region"],   "measures":["invoice_count"]}   无 filter,对照组
   fields  [{"name":"region","type":"string","label":"Region"},{"name":"invoice_count",…}]
   rowkeys ["region","invoice_count"]

B) {"dimensions":["region"],   "measures":["paid_count"]}
   fields  [{"name":"paid_count","type":"number","label":"Paid Invoices"}]      <-- 维度没了
   rowkeys ["region","paid_count"]                                             <-- 行里一直有

D) {"dimensions":["issued_on"],"measures":["paid_count"]}       日期轴
   fields  [{"name":"paid_count","type":"number","label":"Paid Invoices"}]      <-- 维度没了
   rowkeys ["issued_on","paid_count"]

修复后

A) fields  [{"name":"region","type":"string","label":"Region"},{"name":"invoice_count",…}]   不变
B) fields  [{"name":"region","type":"string","label":"Region"},{"name":"paid_count","type":"number","label":"Paid Invoices"}]
D) fields  [{"name":"issued_on","type":"time","label":"Issued"},{"name":"paid_count","type":"number","label":"Paid Invoices"}]

三例的 fields 名字集合现在都与 rowkeys 完全一致;D 还证实日期轴拿回的是 type: "time"
(不是渲染端默认的 string)与 label: "Issued"。对照组 A 逐字节未变,说明这不是把两条路径
都改了、而是把出错的那条收敛到正常那条。

四个临时服务(39517/39631/39742/39858)均已按端口持有者 PID 关停,已确认端口全部释放。


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

2 participants