Skip to content

[PR] ci: rebalance core test matrix to /3 shards across all combos (#4351) - #4355

Merged
ranxianglei merged 1 commit into
masterfrom
2026-10-23_ci-core-matrix-rebalance
Oct 23, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-10-23_ci-core-matrix-rebalance

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Part of #4351 — the decision-free part: re-sharding the core matrix. The remaining items there (coverage downsampling, bigger runners) wait on owner calls.

What & why

Measured per-leg wall from recent runs shows the core matrix IS the bottleneck:

leg measured
test (ubuntu-latest, 24, shard 1) — whole suite on ONE leg ~990–1300s (~21 min) ← critical path
test (windows-latest, 22, *) — suite in 2 halves ~750–1050s each (~13–17 min; up to ~35 min on flake re-runs)
test (ubuntu-latest, 22, *) — already 3-shard ~400–500s
static-gates / mysql-contract / guards ~65–280s / ~50s / seconds

The shape was designed at ~656 files (#3143/#3295, Oct 15); the suite has since grown to ~860 files. Fix: shard ALL THREE core combos at /3 — partitionAll stays total for any i/N, so union of shards == every file: zero coverage loss. These are exactly the per-combo shard shapes the FULL matrix already runs every cron tick, so no new code path.

Expected wall: ~10–12 min instead of 20–25 min typical (35–70 min with tails). This PR's own CI run exercises the new matrix end-to-end as live proof.

Behavior changes (disclosed)

  • Every regular PR now spawns 9 test legs instead of 6 (+~30 billed runner-minutes, running in parallel; wall drops).
  • Check-run names unchanged (explicit name: template); all nine core-produced names already sit in master's required-checks list (which still carries 12 legacy FULL names — the win node-24 ×3 appear only in FULL scope). No protection-config change needed here (enforce_admins is off today anyway).
  • FULL matrix, static legs, mysql-contract, guards, skip/cron logic: untouched.

Verification

  • YAML parses; the plan-job shell script was extracted and executed locally against a simulated pull_request event → scope=core, testMatrix = 9 legs, valid JSON outputs.
  • Workflow-YAML-only change; no typecheck/test surface touched.

The suite grew ~399 -> ~860 files in three weeks while the core matrix
shape stayed frozen at its ~650-file design point (#3295), so its
heaviest legs became the per-PR wall-clock floor: the ubuntu-24
single-leg whole suite measured ~21 min and the windows 2-shard halves
~13-17 min (up to ~35 min tails on flake re-runs).

All three core combos now shard /3. partitionAll stays total for any
i/N, so the union of shards is still every file — zero coverage loss;
the wall drops to the slowest shard (~7-12 min instead of 20-25). The
win node-24 legs stay FULL-scope only. Check-run names are unchanged
(explicit job name template), and all nine core leg names already sit
in master's required-checks list.
@ranxianglei ranxianglei changed the title ci: rebalance core test matrix to /3 shards across all combos (#4351) [PR] ci: rebalance core test matrix to /3 shards across all combos (#4351) Oct 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-23_ci-core-matrix-rebalance (e2e26b1)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.5167-pr.4355.7fe4ae0

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4355.tgz
npm install -g package

Install instructions are refreshed on each push.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant