Skip to content

ci: 按 Cargo 依赖图判定平台范围,砍掉重复跑与独立胶囊 job - #1279

Merged
H-Chris233 merged 6 commits into
betafrom
ci/reduce-runner-load
Oct 28, 2026
Merged

H-Chris233 merged 6 commits into
betafrom
ci/reduce-runner-load

Conversation

@aeoform

@aeoform aeoform commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

摘要

Fixes #(无关联 issue:来自「CI 需要太多虚拟机」的排查)。

先量化,再动手。最近 42 次 CI run + 8 次 iOS run 的 job 级数据(GitHub 计价倍率 ubuntu 1x / windows 2x / macOS 10x):

job runner 平均 折算成本 每次 run 占比
Windows checks windows-latest 17.7 min 35 16%
UI motion (Chromium+WebKit) ubuntu-24.04 15.5 min 16 7%
Linux build + test ubuntu-24.04 21.0 min 21 9%
Android cargo check ubuntu-latest 9.5 min 10 4%
Harmony ubuntu-latest 7.9 min 8 4%
macOS checks macos-latest 6.2 min 62 28%
macOS Rust 1.99 MSRV macos-latest 5.9 min 59 26%
Windows frontend windows-latest 1.7 min 3 1%
Native capsule ×2 macos + windows 2.1 min 10 4%
合计 86 VM-min ≈225
iOS workflow(命中 paths) xcode-27 20 min ≈200 另加约 90%

一次 run 峰值 12 台 VM,免费额度 20 并发;而 push: ['**'] 让 feature 分支的 push 与它的 PR 各跑一遍完整矩阵——42 次 run 里 8 次是这种重复(例如 40497478537 push / 40497484491 同分支 PR)。

修复 / 新增 / 改进

  1. push 只在默认分支跑([main, beta])。feature 分支由 PR 覆盖;默认分支的 push run 仍是基线证据、也是 Rust/Playwright 缓存的唯一写入方(save-if 本来就只看 beta)。文档 §1 表格本来就写「push → main/beta」,是实现与文档不一致,这次对齐 → 去掉约 20% 的总计算。

  2. 平台范围判定改为走 Cargo 自己的包图(scripts/ci-impact.mjs,由 ci-changed-areas.sh 调用):cargo metadata --no-deps --offline 给出每个 workspace 包的目录、仓内依赖边、以及每条边上的 cfg(...) 守卫(例如 openless → openless-capsule 带 cfg(not(any(target_os = "android", target_os = "ios"))))。从各平台根包走闭包,一个文件在范围内当且仅当拥有它的包在该闭包里;非 Rust 树(bundle / ArkTS / Xcode / 工具链)由显式规则覆盖,未覆盖即 fail-open。不再靠手写 glob 猜。

    实测判定矩阵(ci-changed-areas.sh <area> <base>,真实 git diff):

    改动 tauri msrv linux harmony frontend android ios-device
    src-tauri/src/lib.rs ✅ ✅ — — — ✅ —
    crates/openless-core/src/lib.rs ✅ ✅ ✅ ✅ — ✅ —
    native-capsule/src/lib.rs ✅ ✅ — — — — —
    linux-egui/src/… — ✅ ✅ — — — —
    harmony/…/*.ets — — — ✅ — — —
    src/pages/X.tsx ✅ — — ✅ ✅ — —
    ios/** — — — — — — ✅
    docs/*.md — — — — — — —
    */Cargo.lock、.github/** ✅ ✅ ✅ ✅ ✅ ✅ ✅

    (另有 capsule 区域,控制独立的胶囊 job:只有 openless-capsule / openless-native-capsule 里的改动才为它拉起 runner。)

  3. iOS device-build 独立门控(ios-device 区域)。共享 crate 已经由 iOS 的 test job 编译 + clippy 校验;device-build 只证明 Xcode/Rime 工程层能构建,因此只跟 openless-all/ios/**。这是单 job 最贵的一项(13 min × 10x),此前被 crates/openless-core/** 的任意改动拉起。

  4. ui-motion 由新 frontend 区域门控:只跟浏览器会渲染的面(src/**、public/**、e2e/**、assets/**、样式与 vite/playwright 配置)。契约套件(*.test.mjs)只跑桌面两个 runner,不再拉起 15.5 分钟的 Playwright。

  5. android-check 独立门控:bundle-only 改动不跑(该 job 只编 Rust,npm run build 只为满足 generate_context!)。

  6. native-capsule job 按新 capsule 区域门控。(先说一个被实测否决的方案:把它们并进 cross-platform。)

    合并的原始估算——「那两个 job 只有 ~40 s 的载荷,而 cross-platform 本来就在同一 workspace 的 target 目录上,增量编译几乎免费」——是错的。本 PR 第一次跑出来后按 step 计时:

    步骤 macOS checks Windows checks
    cargo check -p openless-native-capsule 40 s 43 s
    cargo test -p openless-capsule --features ui,host 41 s 53 s
    capsule clippy -D warnings 6 s 21 s
    合计 87 s 117 s

    独立 job 原本只要 0.7 min(macOS)/ 1.4 min(Windows):它有专属的 native-capsule warm cache,而且只编 capsule 自己。合并省下 2 台 VM,却把 2 分钟加到了最慢的 job(Windows checks)上,加权分钟反而更贵。

    所以改成保留独立 job + 新增 capsule 区域:只有当改动文件的所属包是 openless-capsule / openless-native-capsule 时才跑。常见 PR 因此少 2 台 VM 且不增加任何墙钟时间;胶囊相关 PR 与今天完全一致。

兼容

  • 不包含:不改任何 job 内的检查内容(只改「什么时候跑」);Windows checks / macOS checks 的步骤顺序、缓存键、-D warnings 全部不变。
  • 对贡献者的影响:feature 分支 push 不再触发 CI,请开 PR 拿门禁;workflow_dispatch 仍是全量。
  • 对合并的影响:无。仓库没有 required status checks(只有 PR 评审/线性历史规则),Native capsule (…) 这个 check 名消失不会卡住任何已开 PR。
  • 判定收紧的前提是「够不到就不跑」必须可证明:脚本读不到差异、cargo metadata 失败、node 缺失、区域未知,全部回退 true(运行)。契约测试对两个方向都断言。

测试计划

  • 命令:node scripts/ci-changed-areas.test.mjs(真实 git 仓库 + 真实 cargo metadata,双向断言 7 个区域,含「无基线 / 空 diff / 未知 revision / 未知区域」回退)
  • 结果:passed
  • 命令:node scripts/ci-cache-sharing.test.mjs && node scripts/workflow-concurrency-contract.test.mjs
  • 结果:passed(胶囊命令断言在 cross-platform 内、无 native-capsule 缓存键;push 范围断言)
  • 命令:for t in scripts/*.test.mjs; do node "$t"; done
  • 结果:全绿
  • 命令:npx prettier --check 改动文件 + bash -n scripts/ci-changed-areas.sh
  • 结果:clean
  • 命令:cargo metadata --no-deps --offline 在空 CARGO_HOME 下跑(模拟干净 runner)
  • 结果:0.02 s / 成功,不需要 registry 缓存,changes job 不需要装 Rust
  • 证据路径:判定矩阵用真实 diff 复现(见上表);基线数据用 gh api .../runs/<id>/jobs 汇总
  • 真实 runner 上验证:Changed areas 在干净 runner 上 11 s 完成,cargo metadata --offline 无需 registry 缓存;判定结果已打进 job 日志(tauri=true / frontend=false …),事后能一眼分清「够不到」与「读不到差异」
  • 证书:本 PR 自身只触发 1 次 CI(改前 feature 分支 push + PR 会触发 2 次)
  • 未覆盖:合并后的整体降幅要等上线后测量(预期:CI 总折算计算量 −40%,每次 run 峰值 12 → 9 台 VM,iOS workflow 在非 iOS 类 PR 上从 2 job 降到 1 job)

顺手修掉一个把 UI 改动拖下水的 flake

UI motion 里的 e2e/switch-motion.spec.ts 在最近 43 次 CI run 里失败 6 次(14%),落在 5 个互不相关的分支上,包括 beta 自己的 40481488914 —— 也就是说改一点 UI 就会随机报错,而报告的信息(语音编辑 exposed its transform transition)看起来像是在指责产品代码。

失败快照把它钉死了:Expected length: 3 / Received length: 0 / Received array: [],而同 case 前面几条断言(状态翻转、滑块位移、transition 跑完)全部通过。开关每次都真的动了,只是采样钩子没挂上。

根因是测试自己的竞态:flip() 用 control.evaluate() 派发采样函数并在里面 addEventListener('transitionrun'),但没有 await 就发 mouse.down()/up()。两条独立协议消息,CI 负载高时 click 先到页面。

修法(不放宽任何断言):把"挂监听"拆成一次 await 的 evaluate,结果存页面侧 WeakMap,确认挂好后才点击;采样循环仍随点击启动、750ms 收尾,xs.at(-1) 仍是静止位置;重挂载仍会因为 xs 为 NaN 而失败。本地 webkit --repeat-each=6 跑满 30/30 通过(8.7 分钟),那条 flaky case 连跑 6 次全绿。

没有加 retries:根因修掉了,就不该用重试掩盖信号。

一次 PR 要拉起 12 台 VM,且 feature 分支的 push 和它的 PR 各跑一遍完整矩阵。
实测最近 42 次 CI run:平均每次 86 VM-min、峰值 12 台并发;42 次里 8 次是
同一 commit 的重复 push run。

1. push 只在默认分支跑(main/beta)。feature 分支由它的 PR 覆盖;默认分支的
   push run 仍保留,因为它是基线证据、也是 Rust/Playwright 缓存的唯一写入方。
   文档本来就写着「push → main/beta」,实现是 ['**'],这次对齐。

2. 平台范围判定从手写路径表换成 Cargo 自己的包图:
   scripts/ci-impact.mjs 用 cargo metadata --no-deps --offline 读出每个 workspace
   包的目录、仓内依赖边和边上的 cfg(...) 守卫,从各平台根包走闭包 —— 文件在范围
   内当且仅当拥有它的包在该闭包里。于是:
   - iOS 工程层(ios-device)只看 openless-all/ios/**:共享 crate 已由 iOS 的
     test job 编译校验,device-build(13min × 10x 计价的 macOS runner)不再被
     crates/openless-core 的改动无谓拉起;
   - ArkTS 只跑鸿蒙;linux-egui 只跑 Linux;纯前端 bundle 改动不再跑 Linux 构建
     (Linux 宿主是原生 egui,不读 dist/)也不再跑 Android cargo check;
   - 契约测试(*.test.mjs)只跑桌面两个 runner,不触发 Playwright 动效套件;
   - 新增 frontend / android / ios-device 三个区域,判定全部 fail-open。
   结果:src-tauri 改动 → 桌面+Android;core 改动 → 五个平台(真的都依赖它);
   纯 TSX → 桌面+鸿蒙+动效,Linux/iOS 跳过。

3. ui-motion(15.5min)改由 frontend 区域门控。

4. native-capsule 的 macOS/Windows 双 job(payload 只有 ~40s 的 cargo check)
   并入 cross-platform:那个 job 本来就为 generate_context! 构建同一个 app
   workspace target 目录,增量编译几乎免费,省掉两个 VM 分配和一个缓存前缀。

契约测试同步:ci-changed-areas 用真实 git diff 双向断言新区域;
ci-cache-sharing 断言胶囊命令在 cross-platform 内且不再有独立缓存键;
workflow-concurrency 断言 push 范围;docs/ci-trigger-and-cache-policy 更新判定表。
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review 🔄 Running since 2026-10-28T06:38:46.738798Z 163c837 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

跳过某平台时,事后要能一眼分清「改动够不到」和「读不到差异」。两处 detect 步
骤现在都把 `<area>=<true|false>` 打到日志,同时仍写进 GITHUB_OUTPUT。

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 161fedd525

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

// a native egui UI and does not consume `dist/` at all, and the Android job
// only compiles the Rust crate (`npm run build` there just satisfies
// `generate_context!`), so neither is in scope for a bundle-only change.
for (const tag of ['frontend', 'macos', 'windows', 'harmony']) tags.add(tag);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include native consumers when classifying shared assets

For changes confined to app/assets/** or app/public/**, this classification skips Linux and Android even when those files are native build inputs. For example, crates/openless-core/src/vocabulary.rs:327 embeds assets/vocab-presets.json, while linux-egui/src/remote_input.rs:212–217 embeds the remote-input assets and linux-egui/src/tray.rs:354 embeds public/AppIcon.png. Calling decide with these paths confirms that Linux and Android return false. These resource changes can therefore alter native behavior or break native compilation without running the affected checks. Add explicit native-consumer reachability for these resources rather than treating the entire directories as bundle-only.

Useful? React with 👍 / 👎.

device-build:
needs: test
needs: [changes, test]
if: needs.changes.outputs.device != 'false'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve iOS-target compilation for shared Rust changes

For a PR changing only shared Core sources, the new gate skips the only job that compiles them for an iOS target. The test job's Cargo commands at lines 70–72 have no --target, so they compile for the macOS runner; --all-targets selects Cargo targets, not iOS architectures. This leaves existing iOS-only branches, such as the capability implementation in openless-core/src/shared_types.rs, unchecked. Previously, device-build invoked scripts/build-core.sh device, which builds for aarch64-apple-ios. Keep shared Rust changes in this gate, or add an iOS-target compile check to test before excluding them.

Useful? React with 👍 / 👎.

run 40510306688 的真实 step 计时:
  Windows checks 里三步 capsule = 43 + 53 + 21 = 117 s(关键路径 job,直接 +2 分钟墙钟)
  macOS checks 里三步 capsule = 40 + 41 + 6 = 87 s
独立 job 原本平均 1.4 / 0.7 分钟——它有专属 warm cache,且只编 capsule 自己。
合并省了 2 台 VM,却把 2 分钟加到最慢的 job 上,加权分钟反而更贵。

改成:native-capsule job 保持原样,新增 `capsule` 区域,只有拥有改动文件的包是
openless-capsule / openless-native-capsule 时才跑。常见 PR 因此少 2 台 VM 且不增加
任何墙钟时间;胶囊相关 PR 与今天一致。

契约测试同步:胶囊 job 必须带自己的区域门控;ci-changed-areas 场景新增 capsule 列。

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 56763b82aa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread openless-all/app/scripts/ci-impact.mjs Outdated
macos: ['openless', 'openless-native-capsule'],
windows: ['openless', 'openless-native-capsule'],
android: ['openless'],
linux: ['openless-linux-egui'],

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include the Harmony host in the Linux test gate

For a PR changing only openless-all/app/harmony/rust/**, this root set makes decide('linux', ...) return false, skipping the workflow that actually runs the Harmony Rust unit tests. In the inspected .github/workflows/check-linux-egui.yml, line 120 runs cargo test --locked -p openless-harmony-host under the Linux scope gate; .github/workflows/ci.yml's harmony-check builds the HAP but does not run those tests. Consequently, changes to the host's credentials, voice configuration, or tests can merge without executing its Rust tests. Include openless-harmony-host in the Linux check's roots, or move its test command into the Harmony-gated job.

Useful? React with 👍 / 👎.

失败快照给出了确定答案:controlledXs 是空数组(Expected length 3 / Received
length 0 / Received array []),而同一 case 前面几条断言都过了——开关确实动了,
只是采样钩子没挂上。

原因:flip() 用 control.evaluate() 派发采样函数(在里面 addEventListener
'transitionrun'),但没有 await 就发 mouse.down()/up()。evaluate 和 click 是两条
独立的协议消息,CI 上负载高时 click 先到页面,监听还没挂上 → 抓不到 3 个受控
采样点。这就是最近 43 次 CI run 里失败 6 次(14%)、落在 5 个互不相关分支上
(含 beta 自身)的那条 `语音编辑 exposed its transform transition`。

改法(不放宽断言):
- 把「挂监听」拆成一次 await 的 evaluate,采样结果存进页面侧 WeakMap(按控件
  元素键),确认挂好后才点击;
- 采样循环仍在点击时启动、750ms 后收尾,所以 xs.at(-1) 依然是静止位置;
- 三条断言(状态翻转、滑块位移、transform 过渡跑完、3 个受控中间位置)全部
  保持原样,重挂载仍会因为 xs 为 NaN 而失败。

本地验证:webkit project 全文件 --repeat-each=6 → 30/30 通过(8.7 分钟),
即那条 flaky case 连跑 6 次全绿。tsc --noEmit 干净。

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e1984ab703

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +235 to +237
/^openless-all\/app\/(index\.html|package\.json|package-lock\.json|vite\.config\.[^/]+|tailwind\.[^/]+|postcss\.[^/]+|tsconfig[^/]*\.json)$/.test(
file,
) ||

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include npm manifests in the Android check's inputs

For PRs changing only package.json or package-lock.json, this classification makes decide('android', ...) return false, even though these files control Android build tooling, not just the rendered bundle. In .github/workflows/ci.yml, android-check installs the locked dependencies with npm ci and then uses the installed Tauri CLI to generate the Android project at lines 225–226 before compiling and testing its Gradle scaffolding. Consequently, a Tauri CLI dependency update skips the Android-specific validation of the project it generates. Classify npm manifests as Android tooling inputs rather than bundle-only changes.

Useful? React with 👍 / 👎.

aeoform and others added 2 commits October 27, 2026 23:10
上一版修掉了监听竞态,但采样循环仍在点击前就开始了 45 帧(~750ms);
如果 CI 把 click 延迟到 800ms,end 会取到动画前的位置。改成
挂监听(await)→ 点击 → 采样循环,45 帧仍覆盖 180ms 的滑动并收尾,
xs.at(-1) 恒为静止位置。

本地 webkit:全文件 5/5 通过;含 --repeat-each=4 的 20/20。
@H-Chris233
H-Chris233 merged commit 6d258df into beta Oct 28, 2026
16 checks passed
@appergb
appergb deleted the ci/reduce-runner-load branch October 30, 2026 05:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants