diff --git a/README.md b/README.md index adc6931c..18516e30 100644 --- a/README.md +++ b/README.md @@ -1,143 +1,275 @@ # ci-fleet -Portable, ephemeral, Dockerized GitHub Actions runner infrastructure for self-hosted environments. +[![Status: experimental](https://img.shields.io/badge/status-experimental-f59e0b)](#project-status) +[![License: Unlicense](https://img.shields.io/badge/license-Unlicense-2563eb.svg)](LICENSE) +[![Platform: Linux + Docker](https://img.shields.io/badge/platform-Linux%20%2B%20Docker-2496ed)](#requirements) -> Status: the inert prototype builds in CI and the gated live-pilot package is ready. No live runner has been registered and the system is not production-ready. +**A portable, self-hosted software delivery fleet for ephemeral GitHub Actions runners and project-owned Docker test environments.** -## Goal +Use one shared pool of disposable CI workers across multiple trusted private repositories, Docker hosts, virtual machines, bare-metal computers, home labs, remote sites, or VPS providers. Projects bring their own Dockerized build and test environment; fleet hosts stay generic. -Run identical GitHub Actions worker containers across one or many Docker hosts while allowing each project to define its own containerized test environment. +> **Status:** Experimental. The first controller host is deployed, and the first manually dispatched runner pilot is pending. The project is not production-ready. -A host may be a Proxmox VM, physical computer, remote-site machine, or VPS. Adding a host or increasing runner capacity should not require redesigning project workflows. +## What problem does this solve? + +Self-hosted CI often grows one project at a time: + +- every repository gets a different runner machine; +- language runtimes and dependencies accumulate on the host; +- idle machines cannot easily help other projects; +- persistent runners retain workspaces, containers, caches, and credentials; +- fixed ports and Docker names prevent parallel jobs; +- adding another computer means repeating undocumented setup; +- test, release, and production deployment permissions blur together. + +ci-fleet replaces that pattern with generic Docker hosts and single-job runner containers. GitHub routes work from authorized repositories to any compatible host with available capacity. The selected project then starts its own pinned Docker test environment. + +## What runs where? + +A fleet host stays generic. Its operating system provides Linux, Docker, and the fleet controller. Application runtimes and services are still required, but each project supplies them through its own Docker images and Compose configuration. + +| Layer | Examples | +| --- | --- | +| Fleet host | Linux, Docker Engine, fleet controller, monitoring and maintenance | +| Ephemeral runner | GitHub Actions agent, repository checkout, and Docker job orchestration | +| Project-owned containers | PHP, Composer, Node.js, Python, PostgreSQL, MySQL, application code, and tests | + +When a job arrives: + +1. the controller creates a fresh GitHub Actions runner container; +2. the runner checks out the selected repository; +3. the repository builds or starts its project-owned containers; +4. tests run inside the project-defined environment; +5. job-owned resources and the runner are destroyed; +6. the host returns to the shared idle pool. + +One host can run one worker. A larger host can run several. Hosts in different locations can advertise the same capability without projects knowing which machine will accept the job. + +## Is this for you? + +ci-fleet is intended for people and organizations that: + +- maintain multiple trusted private repositories or public projects with private delivery control; +- want self-hosted GitHub Actions runners without project-specific host images; +- already use Docker or want reproducible Dockerized CI; +- have spare servers, virtual machine capacity, workstations, remote-site computers, or VPS instances; +- want idle hardware shared across projects; +- need horizontal test sharding and short feedback times; +- want documented cleanup, automatic host security updates, health checks, and recovery; +- expect to add projects and hosts without rebuilding the whole fleet. + +It is probably **not** the right starting point when: + +- untrusted public pull requests must execute on the runners; +- you want fully managed CI with no infrastructure responsibility; +- containers sharing a Docker daemon must be treated as separate security boundaries; +- project tests cannot yet run reproducibly in containers; +- ordinary CI jobs require production or internal-network credentials. + +## How it works ```mermaid -flowchart TD - A["GitHub organization"] --> B["Shared CI runner group"] - B --> C["Docker host A"] - B --> D["Docker host B"] - C --> E["Ephemeral runner"] - D --> F["Ephemeral runner"] - E --> G["Project test containers"] - F --> G +flowchart LR + GH["GitHub Actions workflow"] --> RG["Organization runner group"] + RG --> CA["Controller · Docker host A"] + RG --> CB["Controller · Docker host B"] + RG --> CC["Controller · VPS or remote site"] + CA --> RA["Ephemeral runner"] + CB --> RB["Ephemeral runner"] + CC --> RC["Ephemeral runner"] + RA --> PA["Project-owned test containers"] + RB --> PB["Project-owned test containers"] + RC --> PC["Project-owned test containers"] + PA --> X["Runner and job resources destroyed"] + PB --> X + PC --> X ``` -## Core model +GitHub runner-group policy decides which repositories may schedule work. A shared routing label lets compatible repositories use any healthy host. Each controller owns a uniquely named scale set, so adding or removing one host does not require editing every project workflow. -- The fleet supplies generic ephemeral GitHub runners. -- Worker images and host configuration never contain repository allowlists or project-specific logic. -- Each runner accepts one GitHub-assigned job and is then destroyed. -- Each project supplies its own test Dockerfile, services, and commands. -- Normal CI uses a shared organization-level runner pool. -- Release, deployment, repository-writing, and internal-network jobs remain separated. -- Hosts apply automatic security maintenance and capacity-aware cleanup. -- Long-lived credentials remain in the controller or an external secret manager, never in job containers. -- Existing projects migrate through parallel validation with an explicit rollback path. +### Three responsibility layers -## Project contract +| Layer | Owns | Must not contain | +| --- | --- | --- | +| Public fleet repository | Runner image, controller, lifecycle, setup, health checks, scoped cleanup, standards, examples | Real credentials, private host inventory, production configuration | +| Project repository | Test Dockerfile, services, fixtures, migrations, test plan, `scripts/ci/run.sh` | Fleet controller credentials or host-specific setup | +| Private installation configuration | Organization settings, repository authorization, host capacity, network policy, monitoring, secrets | Project runtime dependencies or test logic | + +A public application can use the same fleet indirectly. Its public repository keeps pull-request validation unprivileged, while a separate private delivery repository checks out an approved immutable commit and performs protected CI, release, or deployment work. The public repository itself never receives privileged runner-group access or fleet credentials. See [Public projects, private delivery, and private configuration](docs/PUBLIC-PRIVATE-CONFIGURATION.md). + +## What makes it different? + +### Generic hosts + +A compatible host is enrolled once. Adding another project changes GitHub policy and that project's workflow—not every runner image or existing host. + +### Ephemeral workers + +Each runner accepts one job and is destroyed. This reduces stale workspace and process state, although Docker access still makes the job host-privileged. + +### Project-owned environments + +A PHP project can use PHP and MySQL containers while a Node project uses Node and PostgreSQL. The same runner host serves both without installing either runtime directly. + +### Horizontal test sharding -Every participating project publishes a task plan and a sharding-aware Docker entrypoint: +Projects publish independent tasks and shards. The internal target is that ordinary CI jobs complete within five minutes, with no more than about four minutes of test payload per shard. ```bash ./scripts/ci/run.sh unit --shard 1/4 ./scripts/ci/run.sh integration --shard 2/3 ``` -The fleet expands `scripts/ci/plan.json` into one GitHub job per task shard. Ordinary jobs have a hard five-minute timeout and target no more than four minutes of test payload. With sufficient independent work and available workers, total wall-clock time approaches the slowest shard rather than the sum of all tests. +With enough independent work and available workers, 45 minutes of sequential tests can become roughly nine five-minute jobs instead of one long runner reservation. -`./scripts/ci/run.sh fast` and `full` remain aggregate local commands. All application validation still executes inside project-owned containers; the fleet runner image does not carry project runtimes. +### Scoped cleanup -The mandatory rules are defined in the [Project CI Standard](docs/PROJECT-STANDARD.md). Existing projects follow [Migrating Existing CI](docs/MIGRATING-EXISTING-CI.md) and must complete the [Compliance Checklist](docs/COMPLIANCE-CHECKLIST.md). +Projects namespace resources by workflow run. Runners are destroyed after use. Host cleanup touches only expired, fleet-owned resources—never an unscoped `docker system prune`. -## Why this exists +### Separated privilege -The initial infrastructure audit found persistent repository-specific runners, inconsistent cleanup, project dependencies installed directly on some runner hosts, fixed Docker names and ports that prevent concurrency, and privileged jobs sharing labels with ordinary validation. +Read-only validation, repository-writing releases, staging, production deployment, and internal-network access belong in separate runner groups, credentials, and preferably separate hosts or security boundaries. -The first implementation will therefore be deliberately small: one experimental runner and one manual read-only smoke workflow running beside existing CI. +## Project status -## Responsibility boundaries +| Capability | Status | +| --- | --- | +| Public architecture, standards, examples, and migration rules | Available | +| Docker runner and controller prototype | Available | +| First isolated controller host | Deployed | +| First manually dispatched private-repository pilot job | Pending | +| One-command `sudo ./setup.sh` installer | Accepted design; not implemented | +| MailThisForMe migration | Planned after pilot | +| TF2 Recommendation Engine migration | Planned after pilot | +| Reusable tester and deployer components | Planned | +| Production-ready release | Not yet | + +The current live milestone is deliberately narrow: one manually triggered, read-only job on one ephemeral worker without modifying or replacing existing project CI. + +## Requirements + +A fleet installation currently assumes: + +- a Linux Docker host; +- Docker Engine and Docker Compose v2; +- a GitHub organization; +- a GitHub App with narrowly scoped self-hosted-runner permission; +- an organization runner group restricted to selected trusted private repositories; +- host-local or externally managed secrets; +- project tests that can run inside project-owned Docker containers. + +Supported deployment shapes include virtual machines, dedicated physical machines, home-lab servers, remote buildings, and VPS hosts. Kubernetes is not required. + +## Choose your path -| Location | Responsibility | +| If you want to… | Start here | | --- | --- | -| `ci-fleet` | Runner image, lifecycle controller, host bootstrap, maintenance, cleanup policy, hard CI rules, reusable workflow interfaces | -| Project repository | Test image, standard CI entrypoint, services, test commands, fixtures, migrations, project secrets, run-scoped cleanup | -| Host-local configuration | Real organization settings, capacity, credentials, network policy, monitoring and maintenance windows | +| Understand the architecture and trust boundaries | [Architecture](docs/ARCHITECTURE.md) | +| Understand how runners are created and destroyed | [Runner controller design](docs/CONTROLLER-DESIGN.md) | +| Browse every guide, concept, standard, and example | [Documentation index](docs/README.md) | +| Evaluate the current prototype safely | [Live pilot runbook](docs/LIVE-PILOT.md) | +| Add another Docker host or location | [How to add a host](docs/ADDING-A-HOST.md) | +| Add a trusted private repository | [How to add a project](docs/ADDING-A-PROJECT.md) | +| Convert existing GitHub Actions CI | [Migrating existing CI](docs/MIGRATING-EXISTING-CI.md) | +| Build a compatible project contract | [Project CI standard](docs/PROJECT-STANDARD.md) | +| Verify project compliance | [Compliance checklist](docs/COMPLIANCE-CHECKLIST.md) | +| Configure upgrades, cleanup, draining, and rebooting | [Host maintenance](docs/HOST-MAINTENANCE.md) | +| Understand secret storage and injection | [Secrets model](docs/SECRETS.md) | +| Use private fleet workers for a public project | [Public projects and private delivery](docs/PUBLIC-PRIVATE-CONFIGURATION.md) | +| See planned work | [Roadmap](docs/ROADMAP.md) | -## Migration model +## Project contract -```mermaid -flowchart LR - A["Existing CI"] --> B["Docker contract"] - B --> C["Experimental run"] - C --> D["Parallel validation"] - D --> E["Shared fleet"] +A participating repository owns its test plan and exposes one stable entrypoint: + +```text +project-repository/ +├── Dockerfile.test +├── compose.ci.yaml +└── scripts/ + └── ci/ + ├── plan.json + └── run.sh ``` -Existing required checks remain available until the new path has passed equivalent tests, cleanup checks, permission review, and rollback verification. +Aggregate commands remain useful locally: -## Security +```bash +./scripts/ci/run.sh fast +./scripts/ci/run.sh full +``` -A runner with access to a Docker daemon must be treated as host-privileged. Multiple runner containers sharing one Docker daemon increase concurrency but do not provide independent security boundaries. +The task plan expands into independent GitHub jobs and shards for fleet execution. Application validation must remain in project-owned containers; the generic runner image does not become a universal language-runtime image. -Never commit credentials or real deployment configuration. See [SECURITY.md](SECURITY.md) and [docs/SECRETS.md](docs/SECRETS.md). +Read the mandatory [Project CI standard](docs/PROJECT-STANDARD.md), then use the [migration guide](docs/MIGRATING-EXISTING-CI.md) and [compliance checklist](docs/COMPLIANCE-CHECKLIST.md). -## Documentation +## Set-and-forget growth -### Mandatory standards +```mermaid +flowchart TD + P["Add a project"] --> PP["Dockerize its tests"] + PP --> PA["Authorize it in private runner policy"] + PA --> PW["Use the shared workflow label"] -- [Project CI Standard](docs/PROJECT-STANDARD.md) -- [Migrating Existing CI](docs/MIGRATING-EXISTING-CI.md) -- [Project Compliance Checklist](docs/COMPLIANCE-CHECKLIST.md) + H["Add a host"] --> HI["Run the same host setup"] + HI --> HS["Give it a unique instance and scale-set name"] + HS --> HW["Join the shared compatible queue"] + + PW --> Q["GitHub routes work to available capacity"] + HW --> Q +``` -### Design and operations +The design goal is that future growth is routine: -- [Architecture](docs/ARCHITECTURE.md) -- [Controller decision record](docs/adr/0001-actions-scale-set-client.md) -- [Public engine / private configuration decision](docs/adr/0002-public-engine-private-configuration.md) -- [Experimental deployment prototype](docs/DEPLOYMENT-PROTOTYPE.md) -- [Live pilot runbook](docs/LIVE-PILOT.md) -- [Host maintenance standard](docs/HOST-MAINTENANCE.md) -- [How to add a fleet host](docs/ADDING-A-HOST.md) -- [How to add a project](docs/ADDING-A-PROJECT.md) -- [Sanitized discovery summary](docs/DISCOVERY-SUMMARY.md) -- [Roadmap](docs/ROADMAP.md) -- [Secrets model](docs/SECRETS.md) -- [Agent instructions](AGENTS.md) -- [Third-party notices](THIRD_PARTY_NOTICES.md) +- a new project does not require rebuilding existing hosts; +- a new host does not require changing existing projects; +- a remote location can disappear without corrupting another host's scale set; +- capacity can range from one runner to many; +- project runtimes remain independently versioned; +- updates and cleanup are observable and reversible. -### Configuration template +## Security model -- [Public configuration-repository scaffold](templates/config-repository/README.md) -- [Configuration-template milestone](https://github.com/RandomDevelopment/ci-fleet/issues/12) +A self-hosted runner with Docker socket access is effectively host-root-equivalent. Multiple runner containers sharing one Docker daemon improve utilization, not security isolation. -The scaffold is validated in this repository before being published as the standalone `ci-fleet-config-template` GitHub template. It contains fictional data only; generated organization configuration should normally be private. +Use this fleet only for explicitly trusted repositories and workflows. Keep deployment credentials out of the shared validation pool. Do not give this public repository access to a privileged private runner group. Never commit real environment files, GitHub App private keys, tokens, internal addresses, or host inventories. -### Copyable examples +Read [SECURITY.md](SECURITY.md) and the [secrets model](docs/SECRETS.md) before registration or deployment. -- [Experimental read-only workflow](examples/workflows/experimental-smoke.yml.example) -- [Five-minute parallel matrix workflow](examples/workflows/parallel-ci.yml.example) +## Installation direction + +The accepted operator experience is: + +```bash +sudo ./setup.sh +``` + +That command will eventually bootstrap or validate GitHub configuration, install the controller and maintenance services, verify health, and support safe reruns, repair, upgrades, and removal directly from the target host. Phone interaction should be limited to unavoidable, non-secret GitHub approval links. + +**That installer is not implemented yet.** For the experimental version, follow the [live pilot runbook](docs/LIVE-PILOT.md) and [deployment prototype](docs/DEPLOYMENT-PROTOTYPE.md). + +## Examples + +- [Read-only experimental workflow](examples/workflows/experimental-smoke.yml.example) +- [Private-repository live pilot](examples/workflows/live-pilot.yml.example) +- [Parallel five-minute matrix](examples/workflows/parallel-ci.yml.example) - [Project task plan](examples/project/scripts/ci/plan.json) -- [Private-repository live pilot workflow](examples/workflows/live-pilot.yml.example) -- [Standard project entrypoint](examples/project/scripts/ci/run.sh) +- [Standard CI entrypoint](examples/project/scripts/ci/run.sh) +- [Project test image](examples/project/Dockerfile.test) - [Isolated Compose project](examples/project/compose.ci.yaml) -- [Node test image](examples/project/Dockerfile.test) -Examples are starting points. Projects must replace placeholder action references and pin reviewed container images before production use. +Examples contain fictional values. Replace placeholders and pin reviewed actions and container images before production use. -## Set-and-forget growth +## Discoverability and related concepts -Enroll each Docker host once with a unique instance and scale-set name and the fleet's shared routing label. Adding a future project updates that project, private fleet policy, and GitHub runner-group access; it does not modify, rebuild, or re-register existing hosts or worker containers. Additional homes, buildings, physical machines, and VPS locations use the same host runbook and join the same compatible queue. +This project is relevant to searches for **GitHub Actions self-hosted runners**, **ephemeral Actions runners**, **autoscaling Docker runners**, **Dockerized CI**, **multi-repository CI**, **organization runner groups**, **GitHub App runner authentication**, **virtual-machine CI workers**, **home-lab CI**, **bare-metal runners**, **distributed CI workers**, **test sharding**, **parallel test execution**, **staging environments**, and **Docker production deployment**. -## First live milestone +## Contributing -After the inert prototype passes validation, the first separately authorized live proof of concept must: +The project is public so other operators can study, reuse, and improve the system. Open an issue for a use case, deployment shape, documentation gap, or safely redacted failure report. Never include credentials, private network details, or active vulnerability information in a public issue. -1. register one runner under a new experimental label; -2. accept exactly one manually triggered read-only job; -3. use `permissions: contents: read`; -4. leave no job-owned container, network, volume, or workspace residue; -5. keep all existing project CI unchanged; -6. demonstrate a rollback that removes only the experimental path. +Infrastructure and operational changes should include matching documentation and preserve the boundary between public reusable code and private installation configuration. See [AGENTS.md](AGENTS.md) for repository-specific engineering rules. ## License -Original work in this repository is released under [the Unlicense](LICENSE). +Original work is released into the public domain under [the Unlicense](LICENSE). Third-party components retain their own terms; see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). diff --git a/docs/CONTROLLER-DESIGN.md b/docs/CONTROLLER-DESIGN.md new file mode 100644 index 00000000..72078a80 --- /dev/null +++ b/docs/CONTROLLER-DESIGN.md @@ -0,0 +1,69 @@ +# Runner controller design + +## Purpose + +The controller turns GitHub job demand into disposable Dockerized GitHub Actions runners. It lets trusted private repositories share idle capacity without installing project runtimes directly on fleet hosts. + +A persistent runner service per VM fragments capacity and retains job state. This controller instead creates one runner for one job and destroys it afterward. + +## How it works + +```mermaid +flowchart TD + A[GitHub App private key] -->|read-only mounted secret| C[Controller] + G[GitHub job queue] --> C + C -->|short-lived registration| R[Ephemeral runner] + C -->|create, observe, log, destroy| R + R -->|Docker socket| D[Host Docker Engine] + D --> P[Project-owned test containers] + P -. no fleet credential .-> X[No controller credential access] +``` + +The controller: + +1. authenticates to GitHub with a GitHub App; +2. watches demand for its uniquely named scale set; +3. generates short-lived just-in-time runner configuration; +4. creates a runner container with resource limits, rotated logs, ownership labels, and the Docker socket; +5. observes one assigned job; +6. retains final diagnostics outside the disposable runner; +7. destroys the runner and its writable state; +8. reconciles capacity with current demand. + +On restart, it recovers only stale runners carrying the same fleet-instance label. It does not manage unrelated Docker workloads or another host's scale set. + +## Security boundary + +Docker socket access is host-root-equivalent. This pool is therefore limited to trusted repositories and trusted workflow revisions. Containers provide repeatability and cleanup; they do not make hostile workflow code safe. + +The GitHub App private key exists only as a file-mounted controller secret. A runner receives only its short-lived registration configuration. Project secrets remain in GitHub repository or environment settings and are supplied only to jobs that explicitly require them. + +## Choosing a runner model + +ci-fleet is optimized for organizations that already operate Linux and Docker hosts, trust the repositories using the pool, and want idle capacity shared across projects. Other runner models may fit different requirements. + +| Runner model | Good fit | Important tradeoff | +| --- | --- | --- | +| Persistent self-hosted runner | A small setup with one trusted repository and minimal orchestration | Workspaces and process state can survive between jobs, and idle capacity stays tied to that runner | +| Repository-specific runners sharing one host | Projects that require distinct GitHub registration or routing | Multiple services may compete for the same CPU, memory, disk, ports, and Docker daemon | +| Ephemeral Docker runners with ci-fleet | Multiple trusted repositories sharing Linux or Docker capacity across one or many locations | Runners are disposable, but jobs sharing a Docker daemon remain inside one host security boundary | +| Actions Runner Controller on Kubernetes | Organizations that already operate Kubernetes and want Kubernetes-native scaling | Requires a Kubernetes control plane and its associated operations | +| Disposable VM per job | Higher-isolation or higher-risk workloads | Stronger separation costs more startup time, storage, and provisioning infrastructure | +| GitHub-hosted runners | Teams that prefer managed capacity and do not require self-hosted resources or networks | Provides less control over hardware, locality, caching, and private infrastructure access | + +The current controller uses the official `actions/scaleset` client at a pinned revision. Updates require a reviewed build and live validation before rollout. + +## Operations and rollback + +Hosts must be treated as disposable infrastructure, patched automatically, monitored for disk pressure, and restricted to trusted jobs. + +To roll back an experimental controller: + +1. stop new capacity; +2. confirm no managed job is active; +3. stop the controller; +4. run scoped cleanup for that fleet instance; +5. verify that host's scale set is absent from GitHub; +6. restore the last reviewed image and configuration if needed. + +Existing project-specific CI remains available until migration validation is complete. diff --git a/docs/PUBLIC-PRIVATE-CONFIGURATION.md b/docs/PUBLIC-PRIVATE-CONFIGURATION.md new file mode 100644 index 00000000..12af79fd --- /dev/null +++ b/docs/PUBLIC-PRIVATE-CONFIGURATION.md @@ -0,0 +1,91 @@ +# Public projects, private delivery, and private configuration + +ci-fleet is public so operators can inspect, reuse, review, and improve the engine. A real installation contains organization-specific topology and policy that should not be published. Secret values do not belong in either public or private Git history. + +The fleet supports both private application repositories and public applications that use a separate private repository for privileged CI, release, and deployment. + +## Supported repository models + +```mermaid +flowchart LR + F[Public ci-fleet engine] --> T[Public configuration template] + T --> C[Private organization configuration] + + subgraph PublicProject[Public application] + PS[Public source repository] --> GH[GitHub-hosted or unprivileged PR checks] + PS -->|approved commit or tag| PD[Private delivery repository] + end + + subgraph PrivateProject[Private application] + PR[Authorized private source repository] + end + + C --> PD + C --> PR + PD --> RG[Restricted private runner group] + PR --> RG + RG --> W[Ephemeral fleet workers] + W --> TEST[Dockerized tests] + W --> ENV[Protected staging or production environment] +``` + +A private application repository may use the restricted runner group directly. A public application repository does not. Its private delivery repository selects an approved public commit, runs the privileged work, and owns the deployment policy and secrets. + +## What belongs where? + +| Location | Contains | +| --- | --- | +| Public fleet repository | Schemas, generic controller and deployment code, validation, examples, standards, and reusable interfaces | +| Public configuration template | Fictional examples and the expected private-repository structure | +| Public application repository | Source code, public tests, public Dockerfiles, and unprivileged workflows | +| Private application repository | Source code and workflows authorized to use an appropriate private runner group | +| Private delivery repository for a public project | Approved source mappings, CI and deployment workflows, environment policy, promotion rules, and required secret names | +| Private organization configuration | Repository mappings, logical host groups, environment policy, capacity, image names, and internal operating notes | +| Secret manager, GitHub environment, or host-local protected file | Actual keys, tokens, passwords, and deployment credentials | + +Each organization generates its own private installation configuration from the public [configuration template](../templates/config-repository/README.md). The resulting private repository is an implementation detail of that organization and is not required to be accessible to users of the public project. + +## Public source with private CI and deployment + +A public project can use fleet workers without granting the public repository access to them: + +1. Pull requests run on GitHub-hosted runners or another unprivileged environment. +2. A maintainer, protected workflow, release tag, or trusted automation selects an approved commit. +3. The private delivery repository receives or records the exact public repository and full commit SHA. +4. It verifies that the repository, revision, branch or tag policy, and requested operation are allowed. +5. Its private workflow checks out that exact revision and runs the project's Dockerized tests on the fleet. +6. Deployment runs only through a protected private environment with the minimum required secrets and approvals. +7. Results and release metadata may be published back without exposing runner or deployment credentials. + +The private delivery repository is the authorized GitHub Actions identity. Fleet hosts and worker images remain generic and do not contain the public project's name or credentials. + +## Security requirements for public projects + +- Never authorize the public source repository or its forks for a privileged self-hosted runner group. +- Never allow pull-request code to choose an arbitrary repository URL, Git ref, shell command, runner label, environment, or deployment target. +- Use a full immutable commit SHA and verify that it belongs to the approved public repository. +- Treat artifacts produced by an untrusted public workflow as untrusted; rebuild them privately or verify their provenance before signing or deployment. +- Keep the private delivery workflow, repository allowlist, promotion policy, and environment protections outside the public repository's control. +- Keep validation runners and deployment runners separated when deployment credentials or internal network access are involved. +- Give workflows and GitHub Apps only the permissions required for the selected operation. +- Ensure logs and published results cannot reveal private configuration or credentials. + +## General rules + +- Public repositories never receive direct access to privileged self-hosted runner groups. +- Reusable public workflow references are pinned to reviewed immutable commits. +- Public examples use fictional organizations, domains, repositories, hosts, and addresses. +- Private configuration may declare required secret names but never their values. +- GitHub App keys, tokens, project secrets, and deployment credentials stay outside Git history. +- Real private configuration is validated against the same public schema used by examples. +- A host remains generic; adding a repository changes policy and project configuration, not every worker image. + +## Secret locations + +Use one of these mechanisms according to the credential's scope: + +- GitHub repository or environment secrets for job-specific credentials; +- root-owned host-local files for a small single-host controller; +- an external secret manager for a distributed or higher-assurance fleet. + +See the [secrets model](SECRETS.md) and [security policy](../SECURITY.md) before configuring a live installation. diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000..31a77eaf --- /dev/null +++ b/docs/README.md @@ -0,0 +1,86 @@ +# Documentation + +Use this index to find ci-fleet concepts, requirements, examples, and step-by-step procedures. Repository Markdown is the authoritative documentation and is versioned with the implementation it describes. + +## Start here + +| I want to… | Read | +| --- | --- | +| Understand the whole system | [Architecture](ARCHITECTURE.md) | +| Understand how ephemeral runners are created | [Runner controller design](CONTROLLER-DESIGN.md) | +| Decide whether it fits my infrastructure | [Architecture](ARCHITECTURE.md) and the root [README](../README.md) | +| Try the experimental implementation safely | [Live pilot runbook](LIVE-PILOT.md) | +| Add another Docker host, VM, computer, or VPS | [Adding a host](ADDING-A-HOST.md) | +| Add a private project to the shared runner pool | [Adding a project](ADDING-A-PROJECT.md) | +| Convert an existing GitHub Actions workflow | [Migrating existing CI](MIGRATING-EXISTING-CI.md) | +| Make a project compliant | [Project CI standard](PROJECT-STANDARD.md) and [compliance checklist](COMPLIANCE-CHECKLIST.md) | +| Split tests across parallel workers | [Project CI standard](PROJECT-STANDARD.md) and the [parallel workflow example](../examples/workflows/parallel-ci.yml.example) | +| Configure automatic updates and cleanup | [Host maintenance](HOST-MAINTENANCE.md) | +| Handle GitHub App, workflow, or deployment secrets | [Secrets model](SECRETS.md) and [security policy](../SECURITY.md) | +| Run private CI or deployment for a public project | [Public projects, private delivery, and private configuration](PUBLIC-PRIVATE-CONFIGURATION.md) | +| Review current priorities | [Roadmap](ROADMAP.md) | +| See what informed the design | [Discovery summary](DISCOVERY-SUMMARY.md) | + +## Concepts + +| Term | Meaning | +| --- | --- | +| Fleet host | A generic Linux machine, VM, or VPS running Docker and a controller. It contains no project runtime. | +| Controller | Host-side service that watches GitHub demand and creates or removes ephemeral runners. | +| Ephemeral runner | A disposable GitHub Actions runner container that accepts one job and is destroyed. | +| Project test container | The project-owned Docker image containing its Node, PHP, Python, database, or other test environment. | +| Runner group | GitHub organization policy that controls which repositories may use a runner pool. | +| Scale set | One controller's uniquely named runner capacity advertised to GitHub. | +| Shared routing label | The stable `runs-on` capability used by compatible projects, regardless of which physical host accepts the job. | +| Test shard | One bounded slice of a larger test suite, designed to run independently and usually finish within five minutes. | +| Private delivery configuration | Organization names, repository allowlists, host inventory, capacity, network policy, and credentials kept outside this public repository. | + +## Design and boundaries + +- [System architecture](ARCHITECTURE.md) +- [Runner controller design](CONTROLLER-DESIGN.md) +- [Public projects, private delivery, and private configuration](PUBLIC-PRIVATE-CONFIGURATION.md) +- [Secrets model](SECRETS.md) +- [Security policy](../SECURITY.md) + +## Standards and contracts + +These pages are normative for compatible projects and hosts: + +- [Project CI standard](PROJECT-STANDARD.md) +- [Migration procedure](MIGRATING-EXISTING-CI.md) +- [Compliance checklist](COMPLIANCE-CHECKLIST.md) +- [Host maintenance standard](HOST-MAINTENANCE.md) +- [Secrets model](SECRETS.md) +- [Security policy](../SECURITY.md) + +## Operator how-tos + +- [Run the live pilot](LIVE-PILOT.md) +- [Add a host](ADDING-A-HOST.md) +- [Add a project](ADDING-A-PROJECT.md) +- [Deploy the current experimental prototype](DEPLOYMENT-PROTOTYPE.md) +- [Maintain, drain, clean, update, and reboot hosts](HOST-MAINTENANCE.md) +- [Use the public configuration-repository scaffold](../templates/config-repository/README.md) + +The accepted target is a single target-host command, `sudo ./setup.sh`. Until that installer is implemented and released, follow the versioned pilot and deployment procedures above rather than assuming the future interface exists. + +## Project integration examples + +- [Read-only experimental workflow](../examples/workflows/experimental-smoke.yml.example) +- [Private-repository live pilot](../examples/workflows/live-pilot.yml.example) +- [Parallel five-minute job matrix](../examples/workflows/parallel-ci.yml.example) +- [Project task plan](../examples/project/scripts/ci/plan.json) +- [Standard project CI entrypoint](../examples/project/scripts/ci/run.sh) +- [Project test Dockerfile](../examples/project/Dockerfile.test) +- [Isolated Docker Compose configuration](../examples/project/compose.ci.yaml) + +Examples use fictional values and are starting points. Pin reviewed actions and images before production use. + +## Documentation rules + +- Mandatory instructions live in this repository. +- Implementation changes update affected documentation in the same pull request. +- Real secrets, hostnames, internal addresses, and private repository inventories never appear in public examples. +- The GitHub Wiki is not an independent source of operational truth. +- A future generated documentation site may improve browsing, but repository Markdown remains its source. diff --git a/docs/adr/0001-actions-scale-set-client.md b/docs/adr/0001-actions-scale-set-client.md deleted file mode 100644 index 254cc7d1..00000000 --- a/docs/adr/0001-actions-scale-set-client.md +++ /dev/null @@ -1,53 +0,0 @@ -# ADR 0001: Official scale-set client with a ci-fleet Docker controller - -- Status: Accepted for an isolated proof of concept -- Date: 2026-07-12 -- Tracks: [#1](https://github.com/RandomDevelopment/ci-fleet/issues/1) -- Upstream revision: `actions/scaleset@9b2803251ede77816509dcd90aadc0690dd80763` - -## Context - -The fleet needs to share idle capacity across trusted repositories while keeping every project runtime in project-owned test images. A persistent runner service per VM wastes capacity and retains job state. GitHub's `actions/scaleset` project provides the official APIs and listener behavior needed to create just-in-time ephemeral runners. Its Docker example is explicitly simplified and not intended to be a production deployment. - -## Decision - -Use the official Go client at an immutable revision. Maintain a small controller here that adds the operational boundaries missing from the example: - -```mermaid -flowchart TD - A[GitHub App PEM] -->|read-only Compose secret| C[controller] - G[GitHub job queue] --> C - C -->|single-use JIT config| R[ephemeral runner] - C -->|create, log, destroy| R - R -->|Docker socket| D[host Docker Engine] - D --> P[project test containers] - P -. no fleet credential .-> X[no access] -``` - -The controller accepts GitHub App authentication only. It creates runner containers with CPU/memory limits, rotated logs, ownership/expiry labels, and the Docker socket. It emits the runner's final logs and destroys the container after one job. On restart it recovers only stale runners bearing the same fleet-instance label. - -## Security boundary - -Docker socket access is root-equivalent on the host. This pool is therefore only for trusted repositories and trusted workflow revisions. Containerization supplies repeatability and cleanup, not a security boundary against hostile workflow code. - -The GitHub App private key exists only as a file-mounted controller secret. A runner receives a short-lived JIT configuration for its one registration. Project secrets remain in GitHub environments/repository settings and are supplied only to jobs that explicitly need them. - -## Alternatives considered - -- Persistent runner services on each VM: rejected as the target because state persists and idle capacity is fragmented. -- Multiple repository-scoped runner services per host: rejected because scheduling and resource contention become ambiguous. -- Actions Runner Controller on Kubernetes: deferred; Kubernetes would add an unnecessary control plane for the current one-server estate. -- Third-party Docker runner managers: not selected; the official client reduces protocol drift and avoids adopting a differently licensed codebase. -- One VM per job: strongest isolation, retained as a future tier for untrusted or high-risk workloads. - -## Consequences - -The controller depends on a public-preview API and requires deliberate upstream review before updates. Docker hosts must be treated as disposable infrastructure, patched automatically, monitored for disk pressure, and restricted to trusted jobs. Project workflows must satisfy `docs/PROJECT-STANDARD.md` before migration. - -## Rollback - -Stop the Compose controller, verify no managed runner containers remain, and remove the experimental scale set if GitHub still shows it. Existing repository-specific CI remains unchanged until a later migration phase, so rollback does not require restoring those workflows. - -## Review cadence - -Review the pinned upstream revision and GitHub runner release monthly and before every fleet image release. Update the pin only through a reviewed pull request with a successful inert validation build. diff --git a/docs/adr/0002-public-engine-private-configuration.md b/docs/adr/0002-public-engine-private-configuration.md deleted file mode 100644 index 7faf4a62..00000000 --- a/docs/adr/0002-public-engine-private-configuration.md +++ /dev/null @@ -1,38 +0,0 @@ -# ADR 0002: Public delivery engine with private organization configuration - -- Status: Accepted -- Date: 2026-07-13 -- Tracks: [#12](https://github.com/RandomDevelopment/ci-fleet/issues/12) - -## Context - -ci-fleet is expanding from runner lifecycle management into a container delivery standard. The reusable engine benefits from public documentation, review, examples, and an Unlicense release. Real deployments contain organization-specific topology and policy that should not be published. Neither public nor private Git repositories are appropriate secret stores. - -## Decision - -Keep ci-fleet public. Publish a separate public `ci-fleet-config-template` that users generate into private configuration repositories. Random Development will generate `rd-delivery-config` from the same public template used by everyone else. - -```mermaid -flowchart LR - F[public ci-fleet engine] --> T[public config template] - T --> R[private RD configuration] - T --> U[another user's private configuration] - R --> P[private project repositories] - U --> X[their private project repositories] -``` - -The public engine owns schemas, reusable workflows, generic controller/deployment code, validation, examples, and policy. A private configuration repository owns real repository mappings, logical host groups, environment policies, capacity, image names, and internal operating notes. - -Secret values remain in GitHub Environments, host-local root-owned files, or an external secret manager. Configuration may declare required secret names but never their values. - -## Consequences - -- Public reusable workflows must be called at immutable commit SHAs. -- Public repositories never receive access to self-hosted runners. -- Examples use fictional organizations, domains, repositories, and hosts. -- Real private configuration is validated against the same public schema. -- Publishing the controller does not weaken its security boundary; credentials, runner-group policy, environment protection, and host isolation remain authoritative. - -## Rollback - -The configuration template is additive. Organizations can stop consuming it without changing runner lifecycle code. Random Development can keep its configuration private or migrate it to another configuration system while retaining the public delivery standards.