Repository navigation
Closed
Description
Tracking issue for follow-ups after the NVIDIA sysext work (#305, #304, #319) and the recent CI/cache work. Edit this list as ideas come up; open a separate issue or PR for each item when it's picked up.
NVIDIA sysexts
- Confirm the first main build/boot-test with the NVIDIA + ZFS sysexts (
e9b9e59, feat(nvidia): deliver the driver and toolkit sysexts to nodes #319) is green and the release set includesnvidia-open-595_<ver>.raw.zst+nvidia-container-toolkit-<ver>.raw.zst. Done: v26.09.808. - Tracking: extend
.github/scripts/track-binaries.pyto follow patch releases within each pinned NVIDIA branch (595.x). NVIDIA publishesNVIDIA-Linux-x86_64-<ver>.run.sha256sumnext to each.run. Also track nvidia-container-toolkit releases. Driver: feat(nvidia): NVIDIA sysext skill and driver release tracking #326; toolkit: ci(track-binaries): track the NVIDIA Container Toolkit by its tag's commit #333. - Toolkit version tracking: needs a verifiable source (e.g. git-kind source at the release tag commit) before track-binaries can bump it. Done in ci(track-binaries): track the NVIDIA Container Toolkit by its tag's commit #333:
git_repoat the tag's commit, written only when the GitHub API andgit ls-remoteagree. - Docs:
docs/skills/nvidia-sysext.mdcovering the flavour table, the add-a-flavour procedure, the open-modules-only (Turing+) policy with a link to common's nvidia skill, per-node selection (Bootyextensions/ sysupdate feature), and GPU Operator values (driver.enabled=false,toolkit.enabled=false,cdi.enabled=true). Add it todocs/skills/index.md. Done in feat(nvidia): NVIDIA sysext skill and driver release tracking #326. - Second flavour (optional): add another branch (e.g.
nvidia-open-615) to prove adding a flavour is purely additive. Costs ~180 MB more per release, so only when a branch is actually needed. - Hardware validation on a real Turing+ GPU node (deferred: no available test node has only Turing+ GPUs; a machine with a pre-Turing card in it can't use the open modules for that card):
nvidia-smi, GPU Operator validator, time-slicing, CUDA and NVENC/NVDEC workloads, dcgm-exporter.- GPU Operator runtime class: the operator runs its own pods (device plugin, GFD, dcgm-exporter, validator) with
runtimeClassName: nvidia, but Bluefin is CDI-only and its containerd has nonvidiahandler. Decision (revised 2026-10-02): the chart refuses the NRI plugin unless it manages its own toolkit container, and real-hardware testing (feat(nvidia): ship nvidia-container-runtime (CDI mode) so GPU Operator operands with runtimeClassName nvidia can start #350) showed the operands need annvidiaruntime handler. feat(nvidia): ship nvidia-container-runtime (CDI mode) so GPU Operator operands with runtimeClassName nvidia can start #350 adds a CDI-modenvidia-container-runtimehandler to the kubeadm containerd; the homelab GPU component (feat: Bluefin Server Homelab profile (opt-in sysexts + Ignition templates, multi-node join) #353) relies on it. During hardware validation, confirm NRI works with the kubeadm sysext containerd (2.1.5), then document the values indocs/skills/nvidia-sysext.md. Fallback if NRI doesn't work: a runc-backednvidiahandler in the kubeadm sysext containerd config.
- GPU Operator runtime class: the operator runs its own pods (device plugin, GFD, dcgm-exporter, validator) with
- Consider whether udev modalias autoload / bare
modprobefor sysext modules is worth supporting beyondbluefin-sysext-modules(today the load units cover boot; seesystemd-sysext-extensions.md). - UVM is built without HMM (FSDK kernel lacks
ZONE_DEVICE/DEVICE_PRIVATE). Revisit if a workload needs HMM.
CI speed / build cache
- ci: upload key-free BuildStream artifacts to cache.projectbluefin.io from main #299 / ci: upload key-free BuildStream artifacts to cache.projectbluefin.io from main #301: superseded by ci(kernel-cache): seed the project CAS instead of a ghcr.io tarball #365 (kernel + Go seeded into cache.projectbluefin.io over mTLS). Remaining: move CASD_CLIENT_KEY into the main-only
bst-cacheenvironment (admin). - Pull side (deferred): builds only look up FSDK elements (incl. the kernel) in FSDK's caches. Adding
cache.projectbluefin.io:11001to FSDK'sproject.confvia our patch queue is needed for uploaded FSDK artifacts to be reused. - Fixed dev module key for PR builds so the PR kernel is cacheable (maintainer decision; less urgent since ci(build): validate pull requests, build the image nightly, on main and on demand #313 stopped building images on PRs).
- Sign/compress modules in parallel in
kernel-modules.bst(~2 min per build). Done in build(kernel-modules): sign and compress modules in parallel #331 (byte-identical output).
Other
- Release race (fixed in ci(release): skip a release whose commit can no longer be tagged #332: GitHub refuses the tag only when the running workflow file differs from the default branch;
publish-release.sh taggablenow skips that version with a warning before publishing): a queued main run that releases after a later workflow-changing commit has landed failsgh release createwith HTTP 403 ("Resource not accessible by integration"). GITHUB_TOKEN may not create a tag at a commit whose.github/workflowsdiffers from the default branch; that needs theworkflowspermission, which it cannot have. Seen on run 36772602951 (1b8f3de, fix(initrd): verify the per-node Ignition config before applying it #285, tagging v26.09.803) after ci(build): run boot-test and release-dry-run when kernel-cache is skipped #328 changed build.yml; the content shipped in v26.09.805 and only the version number was skipped. Options: accept the skip (document it), skip the release when a newer main commit exists, or tag with an app token that hasworkflows. - mergeraptor is not installed on projectbluefin/server: every
track-junctions.yml/track-binaries.ymlrun fails atactions/create-github-app-tokenwithNot Found … get-a-repository-installation-for-the-authenticated-app(e.g. runs 36688254728, 36691555406). Needs an org admin to add the repo to the app installation. Once installed, the release step could mint acontents+workflowstoken so raced versions publish instead of being skipped. - fix(initrd): verify the per-node Ignition config before applying it #285 (Ignition config signature verification): changes requested; waiting on the author. Done: merged as 1b8f3de with a transitional netboot default; follow-up Signed per-node Ignition configs: dedicated config trust root + Booty signing, then remove the transitional allow-unsigned default #327.
- Countme:
os-countme.bstpins common'sd46c4df(branchfeat/first-party-countme-client); fix(countme): re-track os-countme.bst to projectbluefin/common main #274 was closed rather than sed-patch main's Dakota-scoped client. Needs a projectbluefin/common client that accepts non-bootc/DDI images (ID=bluefin-server,repo=server), then re-track. Until then, the pin breaks if that branch is deleted. - Booty:
cmd/main.go--bluefinVersionhelp text still shows the old20260927.123example; real versions look like26.09.747. Done: docs(cli): show a real Bluefin Server version in --bluefinVersion help jeefy/booty#66.
Ideas
Add here.
Metadata
Metadata
Assignees
Labels
No labels
Activity
feat(nvidia): NVIDIA sysext skill and driver release tracking (#326)
ci(track-binaries): track the NVIDIA Container Toolkit by its tag's c…
ci(release): skip a release whose commit can no longer be tagged (#332)
kylerankin commentedon Oct 4, 2026
test 2>&1
🐝 Hive Agent:
contributor| SHA:3772407kylerankin commentedon Oct 4, 2026
This tracking issue's concrete follow-ups are all done (NVIDIA sysexts #305/#304/#319, patch-release tracking #326, toolkit version tracking #333, nvidia-sysext.md docs #326, parallel module signing #331, release-race #332, Ignition sig #285, Booty version help).
Every remaining unchecked item is non-shippable from this repo today:
ID=bluefin-server) to land first (external dependency).No in-repo change is clearly actionable; the follow-ups are deferred, optional, or waiting on maintainer/external decisions.
🐝 Hive Agent:
contributor| SHA:3772407hivecommons-hive commentedon Oct 4, 2026
What to reply
/hive approve: go with the recommendation/hive decision <your instructions>: go ahead, following your instructions/hive help: show these againIf this issue is waiting only for reporter confirmation after a merged fix, the reporter or a maintainer can reply
/fixedinstead.Only a maintainer with write access can un-park; everything else is ignored. Full command reference: https://github.com/hivecommons/hive/blob/v5/src/docs/maintainer-commands.md.
jeefy commentedon Oct 6, 2026
Closing: every follow-up this repo can ship is done (see the Oct 4 summary above). What remains is an optional second NVIDIA flavour (when a branch is needed), hardware validation on a Turing+ node, and admin secret moves documented in
ci-tooling.md. Each of these can get its own issue when it's picked up.