Skip to content

Stabilize Lighthouse budgets with confirmation samples - #4

Merged
zhaowl94 merged 1 commit into
masterfrom
agent/stabilize-lighthouse-sampling
Jul 31, 2026
Merged

Stabilize Lighthouse budgets with confirmation samples#4
zhaowl94 merged 1 commit into
masterfrom
agent/stabilize-lighthouse-sampling

Conversation

@zhaowl94

Copy link
Copy Markdown
Owner

What changed

  • Keep all Lighthouse category budgets at 0.95.
  • Run a single audit when the first sample passes.
  • When the first sample misses any budget, collect two confirmation samples and evaluate the three-run median.
  • Preserve every raw HTML/JSON report and cap each audit at 90 seconds.
  • Add focused unit coverage for median calculation, thresholds, retry behavior, and persistent regressions.

Why

The master run 30602101413 reported root-page performance 0.94 because Lighthouse measured 284.5 ms total blocking time on a page with no application JavaScript. All other categories/pages and prior local/PR/master runs scored 1.00, so this was runner CPU variance rather than a product regression. Confirmation sampling filters a one-off outlier without weakening the quality budget.

Impact

Normal passing CI remains fast with one sample per page. A first miss receives two independent confirmation samples; genuine regressions still fail when the median remains below 0.95.

Validation

  • Windows: npm run check, 15/15 unit tests, build, 163/163 browser tests, Lighthouse 5/5 pages at 1.00, dependency audit (0 vulnerabilities), and 5/5 external links.
  • Isolated Ubuntu: 15/15 unit tests, 142/142 browser tests across Chromium/Firefox/WebKit/mobile/visual projects, and Lighthouse 5/5 pages at 1.00.
  • Formatting, linting, and changed-file scope checks passed.

@zhaowl94
zhaowl94 marked this pull request as ready for review July 31, 2026 06:51
@zhaowl94
zhaowl94 merged commit 12b18bf into master Jul 31, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant