Skip to content
JorisJonkers-devPublic

About

Estate delivery state: Platform document, composition lock, ClusterState snapshot and one pin per Project. Never a rendered tree.

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

720 Commits

Folders and files

Repository files navigation

estate

The estate's delivery state: what composition reads and what it records, and nothing rendered. The model and its rules live in JorisJonkers-dev/deploy-kit; this repository's shape is deploy-kit's spec/v1/60-setup.md#the-estate-repository.

path what it is who writes it
platform.intent.yml the Platform document: what the estate offers, and the handover ledger a human, by pull request
platform/ the platform's own project files, <project>.project.yml: secrets in its own directory beside Vault's configuration and env file, delivered by its pin since its handover (docs/runbooks/secrets-handover.md); edge, in its own directory beside external-dns's env file; observability in its own directory beside its config, rules, dashboards and env files; data, in its own directory beside its config, still delivered by fleet-infra; mail, in its own directory beside its config and env file; and outline and utility, third-party applications, each in its own directory beside its env files and config a human, by pull request
node-contract.yml what each node offers a placement, which the Platform document pins by digest copied from JorisJonkers-dev/nix-config, which generates it from its node files
images.lock.yml every image the platform's own projects and policies name, by digest and user a human, by pull request
participants.yml every Project composition expects a fragment from, and how stale each may be a human, by pull request
toolkit.yml the one deploy-kit release every workflow on the estate path runs: composition here, and the shared Publish Fragment workflow every repository calls
cluster-state.yml the ClusterState snapshot the Collector
the composition lock the inputs the last composition rendered from the composition workflow, from its first run
projects/<project>/source.yaml one pin per Project on the estate path the composition workflow
bootstrap/ the scripts that create the GitHub Apps, put every secret a person writes in Vault, and write a cluster's credentials: what git cannot hold a human, by pull request
drills/ the live drills that settle a claim deploy-kit's decisions rest on: each makes and checks its own changes in a throwaway namespace, removes them, and posts its result on its issue a human, by pull request
HANDOVER.md what each Project's handover changes here, what it waits on, and what is left to delete by hand a human, by pull request
docs/runbooks/ the steps of a live change a person runs, such as handing a Project over a human, by pull request

A rendered tree is never committed here. Each Project's render is a signed OCI artifact at ghcr.io/jorisjonkers-dev/render/<project>, and a deploy is a commit that moves the digest its pin names.

The Platform publishes like any Project. Each release of this repository publishes the Platform document's Intent Fragment, intent-jorisjonkers.dev, and one for each Project in platform/, through JorisJonkers-dev/github-workflows' publish-fragment (release.yml). A change to the platform reaches composition only by a release.

Reading a delivery

compose.yml shows in the Actions tab as Deliver to production, after the cluster the Platform document names. Its file keeps the name compose.yml because that path on main is the identity every render is signed as and Flux verifies against; renaming it is a signer migration, not a rename.

  • The run title says why the run started: the release that published, a rollback, a resume, or the nightly safety net.
  • The job summary has one row per Project: the release it ran before the run and the one it runs after, whether its pin moved, and, for a Project composition refused, the E_ codes that say what to fix. A composition that refused the estate as a whole says what it refused on.
  • The pin commit names each Project it moves and the release it delivers, chore(deploy): delivery 0.4.0, with the repository and commit that released it in the body.
  • A Deployment opens on the repository that released each moved Project: environment production, at the release's commit, with the release's URL where the Project is exposed, and first status " pinned". An earlier Deployment of the same Project still in progress is marked superseded. One opens only where the fragment was signed by a run of that repository at that commit (deployments.sh; its fixtures run with bash .github/scripts/test-deployments.sh).
  • A run fails when composition isolated any Project (a refused, missing or stale fragment), on its last step, so everything else was delivered first. Each repository's commit then carries the estate/production status, and the condition's issue is open here.

Replaying a render

replay.yml, Replay production renders, proves every pinned render reproduces from the inputs its artifact names (deploy-kit spec/v1/55-delivery.md#rendered-artifacts-and-pins). It runs nightly after the delivery safety net, by hand from the Actions tab, and on a pull request that changes it. It only reads: contents: read and packages: read, no environment, no App key, and nothing signed, pushed, tagged or committed.

For each pin under projects/, the artifact's dev.jorisjonkers.lock-digest names the composed artifact holding the lock of the composition that rendered it, and the estate commit it ran at. Each such lock is replayed once, as compose.yml composed (replay.sh):

  • every fragment the lock records, by digest, with the push time the registry recorded for it;
  • participants.yml, cluster-state.yml and the pin sources at that commit, and the pins composition built from them;
  • the toolkit release the lock records, not the one toolkit.yml names now. A pin a later composition left in place still names a render an earlier release made: on 2026-11-01 the pins named renders by 1.4.0, 1.5.0 and 1.6.0, and 1.6.0 rendering a 1.4.0 lock's inputs gave 15 of its 21 artifacts another content hash;
  • the clock stopped at the lock's generatedAt (compose-at.mjs): a participant's age is measured against it, so a fragment fresh then may be stale now. From deploy-kit 1.7.0 the replay passes it as deploy-kit compose --now; a lock an older release wrote is replayed through that release's own entry point;
  • the lock itself as the previous lock, so each Project composes at the fragment the lock records, as the one isolated at an earlier fragment did.

Then the artifact the pin names is pulled and compared with the replay's share for its Project, byte for byte (compare-render.sh; its fixtures run with bash .github/scripts/test-compare-render.sh, and in the job before the replay). The run fails naming every differing path, and its job summary has one row per Project. Beside the files, the replay must give the lock's composedDigest and clusterStateDigest, and each artifact's dev.jorisjonkers.content-hash and dev.jorisjonkers.render-hash.

What is not compared, and why:

  • The artifact's digest. oras push packs the tree into a gzipped tar whose headers carry each file's time, owner and mode, and the manifest carries the push time (org.opencontainers.image.created), the estate commit and the lock digest. None of that is the render, and none of it can come out the same twice, so the unpacked files are compared instead, by their bytes alone: no mode, owner or time, and no empty directory.
  • The signature. A keyless signature holds a certificate issued for that run and a transparency log entry made at that moment; it says who published the artifact, never what it holds.
  • The rest of the lock. Replayed against itself, the lock's previousLockDigest and lockChain differ by construction, and a Project isolated in the recorded run is isolated, if at all, on the fragment it composed at rather than the one it refused. What the lock says of the render, composedDigest and clusterStateDigest, is compared.

How Flux reads it

Flux's controllers are still owned by JorisJonkers-dev/fleet-infra, which delivers every Project not yet handed over. Its flux-system declares one more source, GitRepository flux-system/platform, reading this repository's main with a read-only deploy key, and a Kustomization, estate-pins, applying projects/ and nothing else. Each pin is an OCIRepository that verifies its artifact keyless against compose.yml on main before anything applies, pulled with the flux-system/render-pull credential.

Who can write

Two GitHub Apps reach this repository, each through short-lived installation tokens, and they are the only actors that bypass main's ruleset:

App permissions used by
estate-dispatch actions: write, contents: read, deployments: write, statuses: write, on every repository an application repository's publish workflow, to start composition; composition, to report a commit status and open each moved Project's Deployment
estate-collector contents: write, on this repository only the Collector, to commit cluster-state.yml; composition, to commit the moved pins and the pin sources it rewrites; and the pause, resume and rollback workflows, to commit a hold. Its key is in Vault, and also the secret ESTATE_COLLECTOR_APP_PRIVATE_KEY of this repository's production environment, which only main deploys to; its id is the variable ESTATE_COLLECTOR_APP_ID

Everything else reaches main by pull request. Both Apps, and every credential a cluster needs to read this repository, are created by bootstrap/, which is also what a cluster move reruns.

Holding a Project

Three workflows, run by hand from the Actions tab on main, hold one Project still, or take it back to an earlier release, without touching its repository (deploy-kit chapter 55). Pause and resume are one commit each to projects/<project>/source.yaml; a rollback is two.

  • pause.yml records who paused the Project, when and why. Composition still checks its newest fragment and reports what it finds, and moves no pin for it.
  • resume.yml removes the record, a Rollback's with it, and starts a delivery, which moves the pin to the Project's newest fragment.
  • rollback.yml takes a version, and refuses it before anything runs unless it is earlier than the pinned release and, where that release carries a Migration Proof, the one the proof names (found by walking every composition's lock). Its first commit pauses the Project and asks for a backup; composition renders one-shot backup Jobs, and the run waits up to 45 minutes for the Collector to report in cluster-state.yml that every Job the render holds succeeded. Only then does its second commit record the release, and composition moves the pin there. A failed, unexpected or late Job stops it with the pin where it was. Either way the Project stays paused until a resume. The dump of the Project's own database is not rendered yet (JorisJonkers-dev/deploy-kit#423), so a Project with no backed-up volume rolls back with no backup at all.

Naming a Project

Our own apps take their repository's name: home-portal, tribelt, grimoire. A third-party app takes the software's name: outline, overleaf, n8n. A Project that bundles several apps keeps the name of what it is for: data, edge, observability. The namespace is always <project>-system, and public hosts stay as they are.

Handover

platform.intent.yml's handover block is the ledger of which Projects this repository delivers (estate) and which fleet-infra still does (legacy). A Project moves in three changes, in order: fleet-infra orphans it, fleet-infra drops it, and only then does the ledger move it here (deploy-kit spec/v1/60-setup.md#handing-over-one-project-at-a-time). The first to move is home-portal, in the app Project.

About

Estate delivery state: Platform document, composition lock, ClusterState snapshot and one pin per Project. Never a rendered tree.

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages