Repository navigation
Inventory missing merged PR benchmarks - #1148
Merged
Merged
Conversation
…into docs/benchmark-merge-coverage
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Record missing merged PR benchmarks, make existing measurements identifiable by PR, and provide a targeted recovery command that preserves usable results from partially failed campaigns.
AG;DR
Inventory merged PRs from September 19, 2026 absent from the merged CI viewer catalog, including observed workflow status and known limitations. The checked-in inventory is a snapshot.
Render associated merged PR numbers in version labels, retaining commit, source, links, and backfill provenance.
Add
//tools:benchmark_recovery, which reads current merged PR metadata, the published catalog, active benchmark runs, and retained normal and backfill artifacts. It reports missing platforms, measurements still running, artifacts awaiting publication, and PRs needing backfill.--dispatch-nextdispatches a bounded batch of up to four missing PRs by default (configurable from one to eight) to the targeted main-branch backfill workflow added by PR Run benchmark measurements in parallel after building #1147. It refuses while a backfill campaign is active or queued.--prand--sinceallow focused recovery. Operators must avoid concurrent dispatches; the check and dispatch are not atomic.Reconcile completed failed or canceled backfill campaigns and retain each independently validated platform result, rather than losing every valid result when one job fails. Continue to reject incomplete or invalid records and retain immutable source and campaign provenance. The existing strict retention interface remains atomic by default.
This is an operator-driven recovery step, not an automatic durable retry scheduler. Recovery selects the union of missing platforms for its batch; a single-PR retry can preserve an already retained platform. Existing artifact presence does not guarantee validation or publication. The documented design identifies the remaining scheduler and status visibility work.
Uses the targeted revision input deployed by merged PR Run benchmark measurements in parallel after building #1147. This PR now targets main.
Validation:
bazel test //tools:benchmark_recovery_test //tools:benchmark_campaign_test //tools:benchmark_history_test //tools:benchmark_landscape_testpassed on macOS, including targeted plan selection and partial failed-campaign retention. Read-only live inventories correctly identified PR Distinguish benchmark layout versions in explorer #986 as requiring both platforms and PR Reduce basic traversal overhead and defer native-case metadata #1143 as already covered. Trunk check passed. The requested PR Explore benchmark history by measured version and platform #914/Distinguish benchmark layout versions in explorer #986 backfill was dispatched on main in run https://github.com/mboworks/xff/actions/runs/42498912253. Its end-to-end measurement and publication results remain pending.Document the continuation after the active Explore benchmark history by measured version and platform #914/Distinguish benchmark layout versions in explorer #986 and queued Reduce basic traversal overhead and defer native-case metadata #1143 backfills: verify viewer publication, refresh and recover the missing-PR inventory, resume equivalent find/rg optimization, and qualify representative combined workloads before a separately identified historical backfill.
Pin historical Linux measurement jobs to Ubuntu 22.04 and explicitly install libicu70/libxml2. The initial Distinguish benchmark layout versions in explorer #986/Explore benchmark history by measured version and platform #914 Linux jobs failed before measurement because their historical LLD executables needed libicui18n.so.70 and libxml2.so.2 on an Ubuntu 26.04 runner. Preserve historical compiler/LTO settings and accept earlier ubuntu-latest plans during validation. The Ubuntu 22.04 retirement date is documented.
Add a platforms input to the backfill workflow and bind requested platforms into campaign identity. This allows Linux-only retries without repeating the active macOS work; omitted selection keeps both platforms. Focused campaign, history and recovery tests passed, as did workflow lint. The new runner has not yet completed an end-to-end historical build.