Skip to content
projectbluefinPublic

About

Bluefin QA pipeline — Argo Workflows, KubeVirt, behave/dogtail GNOME smoke tests

Resources

Contributing

Security policy

Stars

11 stars

Watchers

0 watching

Forks

Repository files navigation

Lab

GitOps k3s cluster on ghost + exo-0, built on CNCF projects.

Purpose

Internal lab for feature development before sending PRs upstream. Three purposes, all built on the 40 Gbps point-to-point USB4 link between ghost and exo-0 (thunderbolt0, 10.99.0.0/30). Policy routing (table 40, rule 5209) sends cross-node pod traffic over the link; control-plane and DNS stay on Ethernet. Clients on the LAN submit work as Argo Workflows (http://192.168.1.102:32746).

Purpose What runs USB4 role Status
BuildStream build farm BuildGrid remote execution (buildgrid ns) with Buildbarn CAS/action cache (buildbarn ns), plus the dakota OCI pipeline: dakota-build-pipeline / bluefin-server-build-pipeline execute on BuildGrid, publish to the Zot registry (:30500), run Dakota QA and boot tests, and promote Zot candidates. Makes projectbluefin/dakota. Hard admission gate: remote-execution actions and CAS transfers run pod-to-pod over the link; no Ethernet or local fallback. Live
Local LLM inference llama.cpp on Vulkan (manifests/llm-d.yaml): exo-0 serves ornith worker and ghost serves ornith advisor; exo-0 NodePort 30800, ghost 30801, OpenAI-compatible /v1. Carries client and in-cluster traffic to the model servers. Live
Distributed ffmpeg encoding Plain Workflows from client tools (tools/farm.py in the video repos): frame-grid segments, parallel ffmpeg, concat -c copy join, ffprobe verify. Chunk exchange between the two nodes' pods. Live per node; cross-node chunking over USB4 not built yet

Stack

Layer Project Role
Kubernetes k3s Local cluster
VM workloads KubeVirt Ephemeral boot-test VMs (bluefin-server-boot-test)
CI/CD Argo Workflows DAG pipeline orchestration
GitOps Argo CD Declarative cluster state from git
Control plane KubeStellar WDS/ITS control plane for the local lab
Administration KubeStellar Console Sole private cluster-admin and single-pane UI

Cluster Topology

Host Role Specs
ghost k3s control-plane + KubeVirt compute Ryzen AI MAX+ 395, 16c/32t, 64GB RAM
exo-0 k3s worker Framework Desktop
Namespace Purpose
argo Argo Workflows
argocd ArgoCD controller
buildgrid BuildGrid remote execution (controller, bots, CAS front, Postgres, one worker per node)
buildbarn Buildbarn CAS, action cache, and remote asset (storage only)
llm-d Local LLM inference (llama.cpp on Vulkan)
local-registry Zot writable registry (30500) + pull-through cache (30501)

GitOps Model

Application Syncs path Namespace
testing-lab argo/workflow-templates/ argo
testing-lab-infra manifests/ argo (+ others)
  1. Edit argo/workflow-templates/ or manifests/ → push to main → ArgoCD reconciles.
  2. Never kubectl apply or argo create workflow-template for tracked resources — ArgoCD overwrites them.
  3. argo/bootstrap/ is not synced — run once by hand during cluster setup.

Getting Started

See docs/ops/bootstrap.md for full setup.

just setup-argocd                 # once
just argocd-sync
just run-bst-build                # dakota BST build on BuildGrid
just run-dakota-qa                # Dakota QA
just run-bluefin-server-build     # bluefin-server BST build
just run-zot-promotion ...        # promote a Zot candidate
just list-workflows

Key Design Decisions

No persistent VMs — boot-test VMs are ephemeral and destroyed via onExit; just list-vms shows zero VMs when no boot test is running.

API-only operator model — cluster reads and mutations go through the just wrappers or argo/kubectl. No SSH to cluster nodes.

One administration pane — KubeStellar Console. No Grafana or parallel dashboard framework.

WorkflowTemplate over inline DAG — reusable pipeline logic lives in argo/workflow-templates/; ArgoCD owns the template lifecycle.

Documentation Map

Doc Purpose
AGENTS.md Agent entry point
docs/reference/workflow-reference.md WorkflowTemplate submit interface and reference
docs/ops/bootstrap.md Replicate this lab from scratch
docs/ops/RUNBOOK.md Architecture + failure modes
docs/reference/agent-cheatsheet.md Command reference
docs/ops/lab-operations.md Operator procedures

About

Bluefin QA pipeline — Argo Workflows, KubeVirt, behave/dogtail GNOME smoke tests

Resources

Contributing

Security policy

Stars

11 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages