Skip to content

fix(ci): preserve performance site when GPU measurements are unavailable - #642

Merged
zoeyrose merged 1 commit into
mainfrom
fix/skip-unqualified-performance-site
Oct 31, 2026
Merged

zoeyrose merged 1 commit into
mainfrom
fix/skip-unqualified-performance-site

Conversation

@zoeyrose

@zoeyrose zoeyrose commented Oct 31, 2026 •

Copy link
Copy Markdown
Member

Summary

The scheduled Classic benchmark currently emits an explicit GPU-qualification skip record. The Pages projection treated the record's existence as a complete measurement and failed schema validation.

Implementation / behavior

  • Classify only the exact current-schema gpu-hardware-qualification-required skip as an unavailable observation; all other inputs retain complete-evidence validation.
  • Preserve the published site and alert state by skipping checkpoint loading, projection, artifact publication, Pages deployment and alert reconciliation for that record.
  • Select the latest successful deployment when looking up a predecessor, excluding successful no-op runs. Preserve run/attempt/source checks and fail closed after a bounded 200-run search.
  • Document the unavailable-measurement behavior and add skip, invalid-input, complete-measurement and predecessor regression coverage.

Validation

  • 46 focused report, site, alert and workflow tests passed.
  • python3 tools/verify_import_history.py: five imported histories verified.
  • Python compilation and git diff --check passed.
  • Workflow lint passed with actionlint 1.7.12 and ShellCheck 0.11.0 in a pinned, network-disabled Linux image.
  • Fresh-context independent review found no actionable defects and independently passed the same 46 focused tests.
  • The broad local tooling run attempted 877 tests but hit 104 missing-CMake errors and one sandbox socket error; this is not a broad local pass. Current-head GitHub CI passed, including Core, Server, Client, integrated graph, native Windows tests, trusted GPU coverage and CodeQL.

Limitations / follow-up

This changes source orchestration only. It does not qualify GPU hardware, rerun the scheduled workflow, deploy Pages, change governance settings, or create new performance measurements. Existing published measurements remain unchanged when the benchmark is unavailable.

@github-actions

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

None

@zoeyrose

Copy link
Copy Markdown
Member Author

Fresh-context independent Astra review covered base 3206b6d through cc89c8a. Reviewed the complete five-file diff and affected benchmark producer, strict report validation, checkpoint construction/fetch, workflow permissions/concurrency and publication/alert failure paths, tests and docs.

No actionable P0–P3 findings; no fixes or declined findings.

Verified exact recognized GPU skip only suppresses publication, complete data retains strict validation, and successful no-op runs cannot masquerade as deployed predecessors. Predecessor lookup remains bounded and checks manifest run/attempt/SHA. Independent and implementer runs of 46 focused tests passed; import-history verification and diff check passed.

Broad local tooling suite attempted (877 tests), but 104 cases errored because CMake is absent and one socket fixture was blocked by the local sandbox; this is not a broad local pass. Current-head CI supplies broad validation. No workflow rerun, Pages deployment, or hardware qualification was performed.

@codecov

codecov Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 45.45455% with 6 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
tools/ci/daily_performance_site.py 45.45% 6 Missing ⚠️

📢 Thoughts on this report? Let us know!

@zoeyrose
zoeyrose marked this pull request as ready for review October 31, 2026 12:26
@zoeyrose zoeyrose self-assigned this Oct 31, 2026
@zoeyrose
zoeyrose requested a review from a team October 31, 2026 12:26
@zoeyrose
zoeyrose merged commit 6cacc0d into main Oct 31, 2026
33 checks passed
@zoeyrose
zoeyrose deleted the fix/skip-unqualified-performance-site branch October 31, 2026 12:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant