Skip to content

Fix historical backfill runtime compatibility and deadlines - #1149

Merged
helly25 merged 2 commits into
mainfrom
fix/backfill-runtime-compatibility
Nov 8, 2026
Merged

helly25 merged 2 commits into
mainfrom
fix/backfill-runtime-compatibility

Conversation

@helly25

@helly25 helly25 commented Nov 8, 2026

Copy link
Copy Markdown
Collaborator

Historical backfills failed on Linux because the selected LLVM linkers need libraries absent from Ubuntu 26.04, and one macOS content-search command exceeded the fixed 60-second deadline. Use a compatible Linux runner, allow longer commands in recorded backfill contracts, and retry only the platforms that still need results.

AG;DR

  • Fully restores the Linux compatibility and platform-selection changes reverted from Inventory missing merged PR benchmarks #1148. This is a separate PR based on merged main; Inventory missing merged PR benchmarks #1148 remains untouched.
  • Select Ubuntu 22.04 for historical Linux measurement jobs and explicitly install libicu70/libxml2, preserving each historical revision's compiler and LTO settings. Continue to validate earlier plans using ubuntu-latest. Ubuntu 22.04 retirement is documented; a replacement will be needed before April 17, 2027.
  • Add an optional platforms dispatch input. Record it in campaign identity, require every requested platform, and retain the default of both platforms for earlier plans. Recovery dispatches the union of missing platforms in its bounded batch; single-PR retries can preserve already completed platform results.
  • New CI backfill plans use a 300-second command deadline. Record and validate the deadline in plan, batch and report; earlier campaigns and ordinary comparison callers retain their 60-second default. Timeouts still terminate the worker process group and retain no failed sample.
  • The completed macOS Distinguish benchmark layout versions in explorer #986 artifact from run 42498912253 was downloaded and validated locally with partial-campaign retention. The required retries are Linux Explore benchmark history by measured version and platform #914/Distinguish benchmark layout versions in explorer #986, then macOS Explore benchmark history by measured version and platform #914; do not repeat macOS Distinguish benchmark layout versions in explorer #986.
  • Validation: bazel test //tools:benchmark_compare_test //tools:benchmark_backfill_test //tools:benchmark_campaign_test //tools:benchmark_recovery_test //tools:benchmark_history_test passed on macOS; Trunk and repository hooks passed. The Ubuntu 22.04 historical build and longer macOS measurement have not yet completed end to end. No retry campaign was launched from this branch.

@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown

Benchmark preview

Advisory: changes over 15% are warnings, not merge failures. Positive reference-adjusted change means slower; negative means faster.

linux

Performance overview

No compatible merged baseline comparisons are available; regression status is unknown. Advisory only.

Merged baseline: unavailable.

Compatible comparisons: 0/360; unavailable: 360.

Population Comparisons Typical change Faster Within threshold Slower
All matched cases 0 n/a 0 0 0

Typical change is an equal-weight geometric mean of reference-adjusted XFF/main factors. Negative is faster; positive is slower. These correlated cases are not a significance test.

Interactive preview | CI run

Tested merge b4268c40af; PR #1149 head 9945ed10e2.

macos

Performance overview

No compatible merged baseline comparisons are available; regression status is unknown. Advisory only.

Merged baseline: unavailable.

Compatible comparisons: 0/360; unavailable: 360.

Population Comparisons Typical change Faster Within threshold Slower
All matched cases 0 n/a 0 0 0

Typical change is an equal-weight geometric mean of reference-adjusted XFF/main factors. Negative is faster; positive is slower. These correlated cases are not a significance test.

Interactive preview | CI run

Tested merge b4268c40af; PR #1149 head 9945ed10e2.

@helly25
helly25 merged commit efffe25 into main Nov 8, 2026
34 checks passed
@helly25
helly25 deleted the fix/backfill-runtime-compatibility branch November 8, 2026 20:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant