diff --git a/docs/GITHUB-APP-SETUP.md b/docs/GITHUB-APP-SETUP.md index 7ada9610..d32990bb 100644 --- a/docs/GITHUB-APP-SETUP.md +++ b/docs/GITHUB-APP-SETUP.md @@ -44,11 +44,49 @@ Nothing else. No `write` on contents, no actions, no administration. If the controller ever needs more, that is a reviewed design change, not a settings tweak. -## 3. Generate the private key +## 3. Generate and transfer the private key On the app page: Private keys → Generate a private key. GitHub downloads one -PEM. Store it only on the controller host, root-owned `0600`. It is never -committed, printed, or copied elsewhere — see [SECRETS.md](SECRETS.md). +PEM. The normal manual workflow in this guide stores it on the controller's +local filesystem at `/etc/ci-fleet/secrets/github-app.pem`, owned by root with +mode `0600`. + +GitHub's browser download is necessarily present briefly on a controlled +management workstation. Choose a fresh temporary directory outside synchronized, +indexed, and backed-up locations, restrict the downloaded file to the operator +immediately, and transfer it at once. This is transient handling, not approved +long-term credential storage; never claim that the key was absent from the +workstation. + +Before any key bytes arrive, create the controller directory as root-owned mode +`0700` and pre-create the destination as a root-owned regular file with mode +`0600`. Initial setup uses only the active path: + +```bash +sudo install -d -o root -g root -m 0700 /etc/ci-fleet/secrets +sudo install -o root -g root -m 0600 /dev/null \ + /etc/ci-fleet/secrets/github-app.pem +``` + +Verify the directory and destination ownership, type, and mode without reading +the content. Then use an authenticated encrypted channel to stream into that +already secured file; the transfer must not replace it with a default-mode node. +Keep key bytes out of tracing, logs, stdout, process arguments, Git, issues, and +PRs. Compare a SHA-256 digest at both ends without printing file content, then +verify the destination again. + +Delete the workstation copy immediately after authenticated transfer and those +transfer checks succeed, before token, reconciliation, health, or convergence +checks. Stop if transfer verification or local deletion fails. The PEM must +never be committed or printed; see [SECRETS.md](SECRETS.md). + +This manual workflow permits exactly two host-local files: the active path above +and `/etc/ci-fleet/secrets/github-app.next.pem` while rotating. It does not cover +other custom paths, symlinks, or an external secret manager's import, rotation, +or deletion lifecycle. Those cases require provider-specific tested automation. +Do not improvise them from these Markdown examples; +[issue #27](https://github.com/RandomDevelopment/ci-fleet/issues/27) tracks that +automation. ## 4. Install the app @@ -68,7 +106,7 @@ Record from the installation page URL and app page: ```bash CI_FLEET_GITHUB_APP_CLIENT_ID= CI_FLEET_GITHUB_APP_INSTALLATION_ID= -CI_FLEET_GITHUB_APP_PRIVATE_KEY_FILE= +CI_FLEET_GITHUB_APP_PRIVATE_KEY_FILE=/etc/ci-fleet/secrets/github-app.pem ``` The controller exchanges a short-lived JWT signed with the PEM for an @@ -77,18 +115,26 @@ stored. ## 6. Verify -On the controller host: +The token helper writes a token to stdout. Every verification invocation must +redirect stdout to `/dev/null`; exit status alone is the result. + +Before installation, use only a reviewed checkout at the intended immutable +commit. Confirm `git rev-parse HEAD` is that commit, ensure the checkout is +clean, and run the checkout's helper: ```bash -sudo /opt/ci-fleet/manager/current/scripts/github-app-token.sh \ - --env-file /etc/ci-fleet/host.env +sudo ./scripts/github-app-token.sh \ + --env-file /etc/ci-fleet/host.env >/dev/null ``` -Prints nothing secret; exit 0 means JWT signing and token exchange work. -Then a check-only reconcile validates the full fetch path without applying -anything: +**Stop** if the checkout or commit is not the reviewed source, the checkout is +dirty, or the command fails. Do not substitute a downloaded standalone script. + +After installation, use the installed manager rather than a working tree: ```bash +sudo /opt/ci-fleet/manager/current/scripts/github-app-token.sh \ + --env-file /etc/ci-fleet/host.env >/dev/null sudo /opt/ci-fleet/manager/current/scripts/remote-reconcile.sh --check-only ``` @@ -98,15 +144,59 @@ sudo /opt/ci-fleet/manager/current/scripts/remote-reconcile.sh --check-only ## Troubleshooting -| Symptom | Cause | -| --- | --- | -| `Repository not found` on fetch | repository not in the installation's selected list | -| 403 on content API after granting permission | permission change saved on the app but not yet accepted on the installation — reopen the installation page and approve the pending permission request | -| 401 on token exchange | wrong client ID, installation ID, or PEM path in `host.env` | - -## Rotation and removal - -- New controller: new app. Do not share one app across controllers. -- Rotate: generate a new key, update the PEM path, delete the old key. -- Retire: uninstall the app from the organization. The host keeps no usable - credential. +- `Repository not found` on fetch: the repository is not in the installation's + selected list. +- 403 on the content API after granting permission: the installation has not + accepted the app's pending permission change. Reopen the installation page + and approve it. +- 401 on token exchange: `host.env` has the wrong client ID, installation ID, + or PEM path. + +## Rotation + +Use this ordered safety checklist for the normal host-local root-owned PEM +workflow. It is a set of gates, not a copy-and-paste shell program. + +1. Generate a new GitHub key and transfer, verify, and install it as described + above at the one approved replacement path, + `/etc/ci-fleet/secrets/github-app.next.pem`. Pre-create it as root-owned + `0600` inside the root-owned `0700` directory before transfer. Keep the old + key and `/etc/ci-fleet/secrets/github-app.pem` active for rollback, and delete + the workstation copy before continuing. +2. Update the protected controller identity configuration to select the new PEM. +3. Activate the new key and require the installed manager's token verification, + reconciliation, health check, and installed-state convergence check all to + succeed with the new key. +4. Confirm the controller remains healthy and converged after a fresh check. +5. Only then revoke the old key in GitHub. Remove its exact old controller PEM + only after revocation is confirmed. Retain the now-active replacement PEM; + never use a wildcard in the shared secrets directory. + +**Stop before old-key revocation or deletion** if new-key activation, +reconciliation, health, or convergence is incomplete or fails. Restore the old +configuration while its key and old PEM remain valid. The approved replacement +file is not cleanup residue after successful activation; it is the selected +active key. Further rotation, canonical-path normalization, custom paths, +symlinks, and secret managers require the tested automation tracked by +[issue #27](https://github.com/RandomDevelopment/ci-fleet/issues/27); do not +adapt this checklist into ad hoc shell. + +## Retirement + +1. In the reviewed private schema-v3 desired state, drain the controller and + converge that state; verify zero managed runners and zero effective capacity. +2. Revoke every key for this controller in GitHub and uninstall its GitHub App. +3. Confirm the controller can no longer authenticate. +4. Remove the controller's PEM and its local GitHub App identity state, including + the protected client ID, installation ID, and PEM-path configuration. +5. Through review, permanently disable or remove the controller declaration, + update lifecycle and capacity inventory as appropriate, and converge the + reviewed private desired state. Keep private identifiers out of this public + repository. +6. Verify that no usable controller credential or identity state remains and + that the authoritative desired state no longer declares active capacity. + +**Stop and preserve evidence** if revocation cannot be confirmed or if the +normal host-local files cannot be identified safely. Arbitrary paths, symlinks, +and secret-manager lifecycle operations require the tested automation tracked +by issue #27; operators must not improvise removal from Markdown examples. diff --git a/scripts/test_quickstart.py b/scripts/test_quickstart.py index 55a82a24..a008a268 100644 --- a/scripts/test_quickstart.py +++ b/scripts/test_quickstart.py @@ -16,4 +16,28 @@ assert quickstart.index("Cancel every queued job") < quickstart.index("3. Authorize the repository") assert "PROJECT_PREFIX=" not in raw_quickstart assert "managed controller managed controller" not in quickstart + +raw_app_setup = (Path(__file__).resolve().parents[1] / "docs" / "GITHUB-APP-SETUP.md").read_text() +app_setup = " ".join(raw_app_setup.split()) +app_safety_contract = ( + "sudo ./scripts/github-app-token.sh", + "sudo /opt/ci-fleet/manager/current/scripts/github-app-token.sh", + "Every verification invocation must redirect stdout to `/dev/null`", + "/etc/ci-fleet/secrets/github-app.next.pem", + "controlled management workstation", + "Before any key bytes arrive", + "Delete the workstation copy immediately after", + "before token, reconciliation, health, or convergence checks", + "new-key activation, reconciliation, health, or convergence", + "Revoke every key for this controller in GitHub", + "disable or remove the controller declaration", + "Remove the controller's PEM and its local GitHub App identity state", + "operators must not improvise removal from Markdown examples", + "issue #27", +) +for text in app_safety_contract: + assert text in app_setup, f"GitHub App safety contract missing: {text}" +assert raw_app_setup.count("--env-file /etc/ci-fleet/host.env >/dev/null") == 2 +assert "LOCAL_PEMS=" not in raw_app_setup +assert "declare -a" not in raw_app_setup print("quickstart_contract=PASS")