Repository navigation
fix(cache): complete artifact replication without blocking releases - #670
Conversation
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: castrojo The full list of commands accepted by this bot can be found here. Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
6ca1f0c
ℹ️ Sentinel notice — informational; author is trusted (owner), merge is not blockedHive flagged this PR (author @castrojo, head
Hive did not add the |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: castrojo The full list of commands accepted by this bot can be found here. Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
1 similar comment
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: castrojo The full list of commands accepted by this bot can be found here. Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
Full local just test-unit now passes on 0ce055a: 1936 pytest tests passed, 30 environment-dependent skips, all 158 Bats tests passed, coverage 84.78% (84% required). Run with isolated Git global/system configuration so workstation commit hooks do not reject temporary fixture commits; no tests or source changed to obtain this result. Requested a non-author OWNERS maintainer review. Keep held until both production seed jobs from #664 complete and a fresh-cache GCC payload pull/checkout from the project CAS succeeds. No queue entry or administrative bypass requested. The moved warmers have no changes dependency; explicit main-ref guards are retained because workflow_dispatch can target other refs. Existing actionlint and required-check policy tests cover reintroduction into the release workflow. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: castrojo The full list of commands accepted by this bot can be found here. Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@castrojo this PR is back in your court ( Review asks:
Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path. Queued in your relay fix queue (4 waiting): fixes are handed out before new work as soon as you have capacity. |
|
Cold production payload proof passed for GCC bootstrap key 1ec8073d05ef6f2bb9c15f903d266632f439e00cef81d026690ebd8c8ec4394f. Used a new empty cache with project recommendations overridden and https://cache.projectbluefin.io:11001 as the only artifact server. Native artifact pull --deps none completed successfully in 9m09s, followed by artifact checkout --deps none --no-integrate in 4s. Materialized cross/libexec/gcc/x86_64-unknown-linux-gnu/16.2.0/cc1 is a regular executable x86-64 ELF, 401875008 bytes; no build ran. The initial combined command hit its 600s limit after the pull had succeeded because checkout defaults to runtime dependencies. Finished checkout of the intended GCC-only artifact without re-pulling it. This proves actual payload bytes, not just remote metadata. Both seed-job completion logs and non-author maintainer review remain outstanding; keep held. |
0ce055a to
6bed72c
Compare
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: castrojo The full list of commands accepted by this bot can be found here. Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@Danathar please review current head Both old production seed steps failed on uncached build-only dependencies despite green job status. The native cached-parent fixture reproduces that failure and passes after materializing the full graph, including destination-only payload checkouts with upstream stopped. The full local gate on the unchanged production implementation passed: 1936 pytest, 30 environment-dependent skips, 158 Bats, 84.78% coverage. The advisory-warming decoupling is implemented and verified in this PR; landing and production verification remain outstanding. Non-author maintainer review and |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: castrojo The full list of commands accepted by this bot can be found here. Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/hold cancel Removing the extra manual hold I introduced. Head 6687800 is review-ready with green CI; the required independent review remains the existing review gate and does not need a second manual hold. I am not granting LGTM to my own PR. This cache-producer follow-up is independent of #640 and #591, which are already unheld and progressing through their native checks/queue. Both producers will need real completion verification when this PR lands. |
Image builds wait for graph classification and boot-tool preflight, not the four-hour cache seeder. Preserve all publication, boot-test, image-check and credential boundaries. A cold run may duplicate compilation while warming, but a slow producer cannot prevent release startup. Assisted-by: GPT-6.1 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: castrojo <castrojo@users.noreply.github.com>
The failing unit explicitly required the kernel-cache build dependency removed by this PR. Delete that scheduling assertion rather than re-pin its inverse; preserve release-only seed and no-signing-key security checks. The active cache repair at 47ce75e already has successful unit checks and is unchanged. Assisted-by: GPT-6.1 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: castrojo <castrojo@users.noreply.github.com>
Removing build.needs alone leaves the four-hour producers inside the workflow-wide main concurrency group, delaying later releases. Move the existing two seed jobs into a main-only cache-warm workflow with its own concurrency group; do not add producers or weaken environment restrictions. Migrate the credential, runner-disk and gate contracts and remove obsolete exceptions. Retain all image, boot and publication checks. Assisted-by: GPT-6.1 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: castrojo <castrojo@users.noreply.github.com>
…rgets Both first-main seed runs failed while pushing uncached build-only dependencies: the release certificate import and the bootstrap binary-seed element. A cached parent makes the default build plan skip these dependencies. Select --deps all before the matching all-dependencies push. Native pinned BuildStream reproduced the failed cached-parent push, then proved destination-only payload checkouts after upstream was stopped. Cache warming now lives outside release CI, so remove continue-on-error and redundant warning steps instead of turning failed seed jobs green. Preserve credential boundaries and update the failure-visibility contract. Remove an exact-argv snapshot; keep credential and failed-payload safety checks. Assisted-by: GPT-6.1 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: castrojo <castrojo@users.noreply.github.com>
Parse the emitted build selection and require dependency mode all plus the four published targets. Do not pin target ordering or long-option spelling. This guards the cached-parent regression verified by the native two-CAS proof. Production code is unchanged; the focused kernel-cache suite passed all 21 cases. Assisted-by: GPT-6.1 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: castrojo <castrojo@users.noreply.github.com>
The observed warm-kernel image build still compiled OpenZFS for 15m15s and NVIDIA modules for 7m44s. Seed their key-free compiler outputs and full dependency closures; keep signing stages private. Assisted-by: GPT-6.1 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: castrojo <castrojo@users.noreply.github.com>
6687800 to
3790eaa
Compare
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: castrojo The full list of commands accepted by this bot can be found here. Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
The native seed-graph check proved Ruby is already in the Brew closure, but systemd-base is absent. Add the measured 11m18s compiler target explicitly instead of assuming it is transitively seeded. Assisted-by: GPT-6.1 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: castrojo <castrojo@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: castrojo The full list of commands accepted by this bot can be found here. Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
The measured compiler is components/_private/systemd-base.bst, not components/systemd-base.bst. Restore the valid six-root selection and verify the actual compiler in its transitive graph. Assisted-by: GPT-6.1 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: castrojo <castrojo@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: castrojo The full list of commands accepted by this bot can be found here. Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Production failure
Both first-main seed steps in run 43639399092 failed despite green jobs: release could not push the uncached linux-module-cert import; dev could not push bootstrap/base-sdk/binary-seed-x86_64. Cached targets made the default build plan skip build-only dependencies.
continue-on-errormasked both failures.The first image in #640's merge-group run 43655195547 pulled 218 cached artifacts but still built 98 locally. Its expensive compiler artifacts included:
components/_private/systemd-base.bst)These builds overlap; the times are not additive. This is measured uncached compilation, not evidence that cache transfers are stuck.
Fix
bst build --deps allbeforebst artifact push --deps all, materializing the complete key-free dependency closure even when roots are cached.cache-warm.ymlwith separatecache-warm-*concurrency, removing both the direct image-build dependency and the workflow-wide release-queue blockage.continue-on-error/warning wrappers: independent producer failures are red without gating releases.bst-cachecredentials, both public certificate variants, private-key graph guards and every image/boot/publication gate.Verification
build --deps all. With upstream stopped, an empty consumer materialized parent and build-only dependency payloads from destination.cache.projectbluefin.io:11001, then materialized executable ELFcc1bytes. Payload proof, not metadata-only availability.just test-unitpassed 1936 Python tests with 30 environment-dependent skips and all 158 Bats tests; coverage 84.78%. Isolated Git configuration prevents workstation hooks from changing fixture behavior.just validatepassed; both release and dev certificate variants resolved every seed root plus the measured Ruby andcomponents/_private/systemd-base.bstcompiler artifacts, with the private-key import absent. The temporary proof branch is not part of this PR.Sequencing and remaining gate
#664 and #640 are merged. #591 is independently in the normal merge queue; this PR does not gate it. No administrative bypass or self-approval.
Non-author review remains required; Danathar, daegalus and hanthor are already requested. After merge, both independent main producers must genuinely succeed before claiming the new artifacts are populated or a build-time improvement is measured. A green warning wrapper is not producer success.