Skip to content

ci: speed up test builds, deduplicate caches, and use prebuilt cargo-ndk - #262

Open
cunninghamcard-bit wants to merge 5 commits into
masterfrom
ci/optimize-ci-speed-and-cache
Open

cunninghamcard-bit wants to merge 5 commits into
masterfrom
ci/optimize-ci-speed-and-cache

Conversation

@cunninghamcard-bit

@cunninghamcard-bit cunninghamcard-bit commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Requested by li · project thread

Before: test-only jobs built the FFI library with the shipping profile (fat LTO, one codegen unit, -Oz), the Android example compiled cargo-ndk from source on every run, every pull request run wrote its own Rust caches (12.9 GB across 99 caches, past GitHub's 10 GB limit, so caches kept evicting each other), and a run had 24 jobs, 6 of them on macOS, against a 20-job total and 5-job macOS limit: with several pull requests open, runs waited longer for runners than they spent building.

After: test-only jobs build fast, the Android example installs a prebuilt cargo-ndk, caches are written from master only and restored by pull requests, a run has 17 jobs (5 on macOS), and the uploaded node-* and ffi-* artifacts are built as before.

Changes .github/workflows/ci.yml and two lines in scripts/build-ios-xcframework.sh.

What changes

  • Test-only builds skip the shipping profile. Workflow-level CARGO_PROFILE_RELEASE_LTO=false, CODEGEN_UNITS=16, OPT_LEVEL=1 apply to the jobs that only run tests (Java, Kotlin, Python, Flutter, Android example, Swift). Not 0: aimux_stream_text drives the whole stream future on the caller's thread, and at opt-level=0 one stream needs about 530 KB of stack on the AI SDK branches, more than the 512 KB of a Swift cooperative thread (SIGBUS in the async Swift tests on [7/8] feat: the remaining upstream model kinds #255 and [11/13] refactor: provider config is plain data #259). At opt-level=1 it needs about 110 KB, the shipping profile about 160 KB. ffi-build, which uploads the ffi-* artifacts, restores lto=true, codegen-units=1, opt-level="z" in its job env and still runs cargo build, as release.yml does.
  • iOS keeps its real profile. CARGO_PROFILE_IOS_RELEASE_OPT_LEVEL=z and CODEGEN_UNITS=1 pin ios-release, so the < 64 MiB slice gate in scripts/build-ios-xcframework.sh still checks the shipping configuration.
  • No debug info for the workspace test binaries. CARGO_PROFILE_DEV_DEBUG=0 in rust-test only. node-binding uploads a dev-profile build (napi build without --release), so it keeps the default.
  • Only the library the job loads. Java, Kotlin and Flutter build with cargo rustc -p aimux-ffi --release --crate-type cdylib and skip the unused static archive. Swift keeps cargo build.
  • Fewer jobs. Java (JDK 8, 11, 17, 21) and Kotlin (JDK 17, 21) run in one job that builds the FFI library once and runs each JDK in turn. The Go binding is tested in the ffi-build legs on Linux and macOS, against the static archive that job already builds. The x86_64 macOS Node binding is cross-built on Apple Silicon and tested with x64 Node under Rosetta instead of on the slower, scarcer Intel runner.
  • iOS slices in parallel. scripts/build-ios-xcframework.sh builds both slices in one cargo build --target … --target …; the output is unchanged.
  • Caches. Swatinem/rust-cache saves only on master (save-if); pull requests restore master's cache. The Java + Kotlin and Flutter jobs share one cache (shared-key: ffi-test). The two node-binding legs per runner OS get a per-target key, so they no longer race for one key. The Node and Python jobs now cache their own build directories (bindings/node/target, bindings/python/target), which the default never covered. The Flutter SDK cache (1.5-2.2 GB per OS) is restored on every run and saved from master only, as the Gradle cache already is: flutter-action's own cache switch turns restore and save off together, so the two Flutter jobs take the key from a dry run of the action and use actions/cache/restore and actions/cache/save around it, saving at the end of the job so the engine artifacts the build downloads are included. The Flutter pub cache stays off, as on master; npm and pip caches are on.
  • Prebuilt cargo-ndk via taiki-e/install-action@v2 instead of cargo install cargo-ndk.
  • CI on every pull request. Every pull request runs the checks, whatever its base, and a new push to a pull request cancels its running checks. [1/8] chore: CI on every PR and test fixture cleanup #249 and the pull requests above it are stacked on this one, so this merges first. contract-tests and swift-contract-tests are untouched, so it merges cleanly with refactor!: remove the generated ProviderName types #246.

What is not strictly equivalent

  • Test-only jobs (Java, Kotlin, Python, Flutter, Android example, Swift) link an FFI library built without LTO at opt-level=1. The shipping profile is still exercised: the Go binding links and tests the full lto=true, opt-level="z" library in ffi-build on Linux and macOS.
  • The x86_64 macOS Node binding is cross-built on Apple Silicon; its tests run under Rosetta 2 instead of on an Intel CPU.
  • The Rust workspace test binaries carry no debug info, so a panic backtrace in rust-test has no line numbers.
  • Java and Kotlin run in one job, one JDK after another, so a failure on one JDK stops the later ones in that run.
  • Pull requests no longer save Rust or Flutter SDK caches; they restore master's. contract-tests and swift-contract-tests still save Rust caches (their steps change in [1/8] chore: CI on every PR and test fixture cleanup #249; save-if goes there once [1/8] chore: CI on every PR and test fixture cleanup #249 merges). Until this PR is on master, pull requests restore nothing and build cold.

Checks

  • node-* and ffi-* artifacts keep their size against master.
  • Flutter example (ios) passes its slice size gate.
  • All 17 jobs pass.

Generated with Devin

chenhaonan and others added 3 commits October 14, 2026 10:37
Cargo.toml configures [profile.release] with fat LTO (lto = true),
codegen-units = 1, and opt-level = "z" so published FFI binaries stay
small. In CI, 11 test-only jobs (Java, Kotlin, Go, Python, Flutter,
Android example, contract tests, Swift) were running full fat-LTO release
builds from scratch even though they upload no release artifacts, while
writing separate 275 MB cache entries per job and exceeding GitHub's 10 GB
repository cache quota.

- Skip fat LTO and -Oz in test-only jobs via top-level CARGO_PROFILE_*
  overrides, while keeping Cargo.toml's full release profile (lto = true,
  codegen-units = 1, opt-level = "z") on node-binding and ffi-build so
  uploaded release artifacts keep their exact sizes.
- Strip debug symbols from dev/test profiles (CARGO_PROFILE_DEV_DEBUG=0,
  CARGO_PROFILE_TEST_DEBUG=0) to speed up linking the 162 integration
  test binaries.
- Build only the cdylib target (cargo rustc --crate-type cdylib) in jobs
  that only link the shared library, avoiding archiving a redundant
  staticlib.
- Enable cache-all-crates on Swatinem/rust-cache and share a single
  "ffi-test" cache key across the FFI test jobs to reduce cache churn.
- Replace `cargo install cargo-ndk` (source build) with
  taiki-e/install-action@v2 and enable Flutter/Gradle/npm/pip caches.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…caches

Fixes on top of the previous commit:

- node-binding builds with `napi build` without `--release`, so its
  uploaded node-* artifacts are dev-profile builds. The workflow-level
  CARGO_PROFILE_DEV_DEBUG=0 stripped their debug info (Linux artifact
  55.3 MB -> 17.6 MB) and its CARGO_PROFILE_RELEASE_* overrides had no
  effect. Debug info is now dropped in rust-test only, where the test
  binaries are linked, and the dead overrides are gone.
- ffi-build goes back to `cargo build`, as release.yml does. Building
  only the cdylib changes the shipped library (x86_64 Linux .so
  9,446,848 -> 7,637,144 bytes, same 129 exports), so the ffi-*
  artifacts would no longer match the release build or earlier runs.
- rust-cache `cache-all-crates` keeps unused registry crates, it does
  not cache workspace crates; dropped. Caches are saved from master only
  (`save-if`); pull requests restore master's caches instead of each
  writing their own, which is what pushed the repository past 10 GB.
- The two node-binding legs on each runner OS (x64 and arm64 targets)
  shared one cache key: one leg failed to save and the other restored a
  cache built for the wrong target. The key now includes the target.
- CARGO_INCREMENTAL=0 is already set by rust-cache; dropped.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0112xDERMUyD5JALkFhnsdWS
With every stacked pull request running CI, runs wait far longer for a
runner than they spend building: each run had 6 macOS jobs against a
5-job macOS limit and 18 Linux jobs against a 20-job total limit, and
the macOS jobs of one run sat queued for 15 minutes.

- Java (JDK 8, 11, 17, 21) and Kotlin (JDK 17, 21) run in one job
  that builds the FFI library once and runs each JDK in turn: 6 jobs
  and 6 FFI builds become 1.
- The Go binding is tested in the ffi-build legs on Linux and macOS,
  against the static archive that job already builds: the 2 Go jobs
  and their FFI builds go away.
- The x86_64 macOS Node binding is cross-built on Apple Silicon and
  tested with x64 Node under Rosetta instead of on the Intel runner,
  where its build took 371 of the job's 465 seconds.
- scripts/build-ios-xcframework.sh builds both iOS slices in one cargo
  invocation, so they compile in parallel; the output is unchanged.

A run goes from 24 jobs (6 macOS) to 17 (5 macOS).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0112xDERMUyD5JALkFhnsdWS
cunninghamcard-bit pushed a commit that referenced this pull request Oct 14, 2026
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa
- subosito/flutter-action saves a 1.5-2.2 GB SDK cache per OS. A pull
  request can only restore caches saved on its own branch or on master,
  so with the action's cache on, every open pull request saved its own
  copy and the 10 GB budget filled up again. The action's `cache` input
  turns restore and save off together, so the two are split: a dry run
  of the action gives the key and path, actions/cache/restore runs on
  every branch, the action installs the SDK (a no-op when the restore
  hit), and actions/cache/save runs on master only, at the end of the
  job so the engine artifacts the build downloads are in the cache. The
  pub cache stays off, as on master. The Gradle cache already saves from
  the default branch only.
- bindings/node and bindings/python are their own Cargo workspaces and
  build into bindings/<name>/target, which rust-cache's default (the
  repository's target directory) never covered: the Node and Python jobs
  rebuilt aimux-core and aimux-providers on every run. `workspaces` now
  points rust-cache at each binding's directory.
- The ffi-build comment said the Go binding links the static archive;
  on macOS ld64 picks the .dylib next to it. Comment only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0112xDERMUyD5JALkFhnsdWS
aimux_stream_text runs the whole stream future on the caller's thread
(ffi_block_on). With the test-only override at opt-level 0, one stream
through the C ABI needs about 530 KB of stack on the AI SDK branches
(about 300 KB on master), more than the 512 KB stack of a Swift
cooperative thread: the async Swift tests died with SIGBUS on #255 and
#259 once they carried this workflow. At opt-level 1 the same stream
needs about 110 KB; the shipping profile needs about 160 KB. The
library is also 40% smaller, and the build is still far from the
shipping profile's fat LTO with one codegen unit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0112xDERMUyD5JALkFhnsdWS
cunninghamcard-bit pushed a commit that referenced this pull request Oct 14, 2026
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A3WxDBWBA4ituJjXsHaEAa

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

评审结论:rebase 后可合并(rebase 是结构性决策,不只是解冲突)

先澄清:diff 里对 master 的 2645 行"删除"是幻影(分叉点 a8e3a99 之后 RFC 批次与 #263 合入 master 的反向 diff)。真实改动仅 2 个文件:ci.yml(+186/−85)与 build-ios-xcframework.sh(双 target 合并构建)。冲突确认仅限 ci.yml 一个文件(merge-tree 验证)。

核验通过的要点:

  • 无 CI 弱化(对真实 base 逐 job 逐 step 对比):glibc 2.17 基线、Windows 静态 CRT、index.js/d.ts drift 检查、iOS 符号扫描、APK so 检查、Go vet/test/race、Java JDK 8/11/17/21、Kotlin 17/21 全部保留(JVM 合并为单 job 顺序执行,--rerun-tasks 保证逐 JDK 重跑);Go 并入 ffi-build 且链接 shipping profile 库,比 master 独立 go job 更严格;17 job(5 macOS)/24 job(6 macOS)手工核对无误;
  • 缓存 key 无串缓存路径(读了 rust-cache 上游源码验证):默认 key 含 job_id、OS/arch、CARGO* 环境值 hash——ffi-build 的 shipping profile 覆盖自动分 key;jvm/flutter 共享 key 但构建命令逐字相同;node 按 target 加 key 修正了 master 上缓存根 workspace 对 napi 实际无效的问题;
  • 上传产物不变:ffi-build job env 显式还原 Cargo.toml release profile;ios-release 显式 pin,64 MiB slice 门完整保留,release.yml 与 ci.yml 同一脚本;
  • iOS staticlib 基线节点不弱化:RFC-0036 §12.3 该行的关闭条件(strip/64 MiB 门存在)保持。

合并前置(rebase 语义决策):本分支 on.pull_request 无过滤 + 无 changes job,与 #263 的 docs-only 跳过架构冲突。两个方向:保留 #263 的 changes+ci-result(推荐,符合 RFC-0043 §9.2"必需汇总检查 always 语义"),需把 job 清单变化同步进 changes 的 run map 与 ci-result.needs(jvm-bindings 新名、go 并入 ffi-build);或明确撤销 #263。不能只解决文本冲突而丢掉 ci-result——它是 branch protection 的单一 required check。

要求与建议:

  1. tool: cargo-ndk 未固定版本(走 cargo-binstall fallback,有 SHA256 校验与 cooldown,但以固定版本为宜,如 cargo-ndk@3.5);
  2. contract/swift 两 job 无条件写缓存(每 PR 2 条目、占 10GB 限额)——已声明为 #249 收尾的过渡态,建议 rebase 时顺手补 save-if 不必等 #249;
  3. macOS x64 Node 改 Rosetta 交叉测试是已声明的可接受弱化,注意日后 Intel-only bug 的归因路径。

roadmap:非节点,但作为"逐层合入、每层 CI 全绿"策略的地基成立。

@eric8810 eric8810 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request Changes —— 合并前需完成(详细核验见上一条评审):

  1. rebase 到当前 master 并结构性调和 ci.yml 冲突:不能只解文本冲突——须保住 #263 的 changes + ci-result 机制(ci-result 是 branch protection 的单一 required check,丢失它违反 RFC-0043 §9.2 的 always 汇总语义),并把 job 清单变化同步进 changes 的 run map 与 ci-result.needs(jvm-bindings 新名、go 并入 ffi-build);若决定撤销 #263 的 docs-only 跳过,也请显式说明;
  2. 固定 cargo-ndk 版本(如 cargo-ndk@3.5):当前 tool: cargo-ndk 未固定版本,走 cargo-binstall fallback,虽有 SHA256 校验与 cooldown,未达"固定版本"标准。

建议顺手项:contract/swift 两 job 补 save-if(不必等 #249 收尾)。缓存方案、job 清单、产物 profile 已核验无问题,rebase + 版本固定后即可合并。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants