From 719d8fff650be8ba1a5e49484628110d8fc55d1e Mon Sep 17 00:00:00 2001 From: Daniel JB Clark Date: Fri, 13 Nov 2026 06:04:04 -0500 Subject: [PATCH 1/9] docs: record the monorepo plugin-type layout (D46): compiler folds into tendcf MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit djbclark, 2026-11-13: frdminc/nix2cf folds into tendcf under compiler/; the compiler becomes `tendcf compile`, the name nix2cf moves to the Nix frontend plugin, plugins are found through the tendcf.frontends, tendcf.supervisors and tendcf.importers entry-point groups and run at compile time only. Docs only: no files move, no code or schema changes. - guide §2-§4, §18: layer diagram, lockfile, compiler name and home; buildfile recorded as built (verified: nix2cf master 572b0b7 CI green) - map: §1, §3, §5, Step 3, D12, D21 amended; new D46; document map - change-process: one repo, no tendcf.lock, two-step bump; site.lock kept - linux-targets step 6 and §5; GLOSSARY nix2cf entry; architecture README - new docs/architecture/plugin-layout.md: tree, today vs planned, entry-point groups, consent rule, contrib/out of tree, move order Co-Authored-By: Claude Opus 5.5 --- docs/architecture/GLOSSARY.md | 16 ++ docs/architecture/README.md | 3 + .../architecture-DEFINITIVE-v3.md | 72 ++++++--- docs/architecture/change-process.md | 101 +++++++----- docs/architecture/linux-targets.md | 11 +- docs/architecture/plugin-layout.md | 152 ++++++++++++++++++ docs/paper/tendcf-architecture-guide.md | 88 ++++++---- 7 files changed, 348 insertions(+), 95 deletions(-) create mode 100644 docs/architecture/plugin-layout.md diff --git a/docs/architecture/GLOSSARY.md b/docs/architecture/GLOSSARY.md index 419a55b..aba04e7 100644 --- a/docs/architecture/GLOSSARY.md +++ b/docs/architecture/GLOSSARY.md @@ -124,6 +124,22 @@ and may change without reopening the ruling. Authority: `architecture-DEFINITIVE-v3.md` D45, with its open points in §12. +### nix2cf + +**A referent that moved.** + +- **Current — the Nix module frontend.** The compiler plugin + `plugins/frontends/nix/` that renders a Nix-authored Site Model to the + same JSON as the YAML frontend (map D12). Not built yet. +- **Before 2026-11-13 — the whole compiler**, and the separate repository + `frdminc/nix2cf` that held it (`nix2cf buildfile`). That compiler is + being folded into tendcf under `compiler/`, its command becomes + `tendcf compile`, and the repository is archived once the move is done. + Older documents and handoffs use the old sense and are not rewritten. + +Authority: `architecture-DEFINITIVE-v3.md` D46 (amending D12 and D21); +layout and order of the move in `plugin-layout.md`. + ### coverage **One enum, three values**: `comprehensive`, `not-yet-migrated`, diff --git a/docs/architecture/README.md b/docs/architecture/README.md index 777ffb0..62ae43a 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -23,6 +23,9 @@ gates — issue #2) and [`upstream-overlay.md`](upstream-overlay.md) (the patch overlay on libntech / cfengine/core in [`../../overlay/`](../../overlay/) — issue #3). [`upstream-register.md`](upstream-register.md) is the living record of every CFEngine/libntech defect and contribution. +[`plugin-layout.md`](plugin-layout.md) is the repository layout and the +compiler's plugin kinds, decided 2026-11-13 together with moving the +compiler into tendcf (map D46). Site Model contract (JSON Schema, fixtures, lint): [`../../schema/`](../../schema/), [`../../examples/`](../../examples/), diff --git a/docs/architecture/architecture-DEFINITIVE-v3.md b/docs/architecture/architecture-DEFINITIVE-v3.md index 1c219bd..794573c 100644 --- a/docs/architecture/architecture-DEFINITIVE-v3.md +++ b/docs/architecture/architecture-DEFINITIVE-v3.md @@ -54,7 +54,8 @@ cannot start its own privileged helper”). ``` tendcf engine: schemas/types, generic adapters, - (public) default advisor prompt, compiler interface + (public) default advisor prompt, and the compiler + with its plugins (one repository, D46) site-shared reusable recipes (a Caddy role, an apt pattern). (optional, public) Not live inventory. @@ -66,14 +67,15 @@ cannot start its own privileged helper”). (never an input) secret names, trust policy, extra advisor prompt. Holds the lockfile that pins everything else. - tool forks (optional) nix2cf, sudo-secretspec, Shizuku, … only if + tool forks (optional) sudo-secretspec, Shizuku, … only if patching them. │ Site Model instances (private) + recipes (shared) ▼ - nix2cf compiler: merge → conflict check → infer - (tool) ordering → CFEngine Augments JSON. - Not the home of schemas or facts. + tendcf compile compiler, in tendcf: frontend (YAML, or Nix + (tool) via the nix2cf plugin) → merge → conflict + check → infer ordering → per-host goal file. + Schemas stay in `schema/`; holds no facts. ▼ Signed release manifest + per-host canonical goal file @@ -128,23 +130,31 @@ error**. Only site-private may bind a winner or a short name. Git-repo count is not the composition mechanism. Layers are **roles**. Site-private holds a **lockfile** (flake-style inputs) that pins tendcf, -nix2cf, site-shared, foreign shared sites, and optional tool forks. The -signed release is the deploy artifact. There is no lockstep of sibling +site-shared, foreign shared sites, and optional tool forks. **Amended +2026-11-13 (D46):** the compiler is part of tendcf, so one tendcf pin +covers schemas and compiler together (~~pins tendcf, nix2cf, +site-shared, …~~); until the move lands it is still the separate +`frdminc/nix2cf` repository. The repository's plugin-type layout is +[`plugin-layout.md`](plugin-layout.md). The signed release is the +deploy artifact. There is no lockstep of sibling checkouts that must share one tag. | Layer | Holds | Visibility | | --- | --- | --- | -| **tendcf** | Engine, **schemas/types**, generic adapters, default advisor prompt | Public product | -| **nix2cf** | Compiler tool only (Site Model → CFEngine JSON) | Public tool; fork only to patch | +| **tendcf** | Engine, **schemas/types**, generic adapters, default advisor prompt, the compiler (`compiler/`) and its plugins (`plugins/`, D46) | Public product | +| ~~**nix2cf**~~ | ~~Compiler tool only (Site Model → CFEngine JSON)~~ Folded into tendcf (D46). `nix2cf` now names the Nix frontend plugin | Repository archived once the move is done | | **site-shared** | Reusable recipes | Public or shared | | **site-private** | Inventory, allocations, secret *names*, trust policy, extra prompt, lockfile | Private | | **foreign shared** | Other sites' recipes | Read-only inputs | | **tool forks** | sudo-secretspec, Shizuku, … | Optional | -**Schemas live in tendcf**, not in nix2cf. A schema change is a tendcf -interface change. An instance change is site data. The JSON Schema files, -fixtures, and lint are `schema/`, `examples/`, and `bin/schema_lint.py` -in this repository. +**Schemas live in tendcf**, in `schema/`, not in the compiler's code +(~~not in nix2cf~~; amended 2026-11-13, D46: the compiler is moving into +this repository, so schema and compiler change in the same commit). A +schema change is a tendcf interface change. An instance change is site +data. The JSON Schema files, fixtures, and lint are `schema/`, +`examples/`, and `bin/schema_lint.py` in this repository (`bin/` becomes +`tools/` in a later change, [`plugin-layout.md`](plugin-layout.md) §5). **Inventory is private by default (D35).** Site-shared ships: @@ -258,11 +268,25 @@ graph,” not “don’t require names.” --- -## 5. The compiler (`nix2cf`) +## 5. The compiler (`tendcf compile`) A **tool**: Site Model (YAML/JSON, or Nix frontend → JSON) → merge → -conflict check → inference → [CFEngine Augments](https://docs.cfengine.com/docs/3.21/reference-language-concepts-augments.html) -(`def.json` / `host_specific.json`). +conflict check → inference → render of the per-host goal file (§9), +which the device projects to [CFEngine Augments](https://docs.cfengine.com/docs/3.21/reference-language-concepts-augments.html) +(`host_specific.json`) after approval. (~~… → inference → CFEngine +Augments (`def.json` / `host_specific.json`)~~: superseded by §9's +render-stage decision; corrected here on 2026-11-13.) + +**Amended 2026-11-13 (D46).** The compiler was built as the separate +repository `frdminc/nix2cf` (`nix2cf buildfile`, Step 3). It moves into +tendcf with its history, first under `compiler/`, and its command +becomes `tendcf compile`; `nix2cf buildfile` stays as a short-lived +alias. The name `nix2cf` moves to the Nix module frontend (D12). +Frontends, supervisor renderers and importers are plugins found through +Python entry points; they run at compile time only, and their output +passes the same schema lint and projector, so a plugin cannot bypass +consent. Layout, plugin kinds and the order of the move: +[`plugin-layout.md`](plugin-layout.md). CFEngine’s [Masterfiles Policy Framework](https://github.com/cfengine/masterfiles) already consumes that data layer. Common case: emit data, not `.cf` @@ -284,7 +308,9 @@ here is Site Model → CFEngine, because of multi-owner operation — not because the compiler mechanism is new. **Where it runs:** operator machine or CI. Consented devices receive a -signed artifact. They do not run Nix or nix2cf. +signed artifact. They do not run the compiler, any of its plugins, or +Nix (except where Nix already is the host's configuration system: D12, +passthrough). --- @@ -1115,7 +1141,7 @@ Each step leaves a coherent system. Not a schedule. | 0 | Schemas in **tendcf** (`schema/`, `examples/`, `bin/schema_lint.py` — provides/requires, interlocks, comprehensive, report-row). Remaining: peer_actions, trust-policy shape, generic unit-writers, lookup stub, YAML canonicalize. Transcribe reality (`not-yet-migrated` is the correct day-one state). | | 1 | macOS services adapter. Dry-run default. No nix-darwin. **Started** (`policy/tendcf_services.cf`, `222da45`: launchd present/absent and interlock refusal, verified against a real `cf-agent`; package/file promises, environment rendering, the unit-writer detector still open). 2026-10-29: the satisfied-interlock path had never opened (verdict class fixed in pass 1 before the check ran); the check moved to `tendcf_interlocks`, sequenced before `tendcf_services`, and `tests/policy/run.sh` proves both paths on Linux CI and the Mac. The CFEngine it was verified against is upstream plus the patch overlay (`overlay/`, [`upstream-overlay.md`](upstream-overlay.md)); Linux CI builds the full fork ([`linux-targets.md`](linux-targets.md)). | | 2 | Android under the Site Model; Termux types; agent owns JSONL+SQLite. **Deferred** (operator, 2026-10-29): Linux comes first, see [`linux-targets.md`](linux-targets.md). Not started as code: the schema already carries a `termux` supervisor flavour (`services.schema.json`) and the `android-termux-services` domain fixture, but no adapter bundle, no agent, no JSONL/SQLite ownership. `frdminc/ShizukuTendCF` is the R4 helper (a tool fork), not this step. | -| 3 | nix2cf: `buildfile` first, conflict, extra-entry, then inference (needs step 1 and a Linux systemd adapter; Android is deferred). **Unblocked** on 2026-10-29 by the operator's decision recorded on [#4](https://github.com/frdminc/tendcf/issues/4); `frdminc/nix2cf` has `buildfile` only. Its `tendcf.lock` is specified in [`change-process.md`](change-process.md) §2. | +| 3 | Compiler (`tendcf compile`; ~~nix2cf~~): `buildfile` first, conflict, extra-entry, then inference (needs step 1 and a Linux systemd adapter; Android is deferred). **Unblocked** on 2026-10-29 by the operator's decision recorded on [#4](https://github.com/frdminc/tendcf/issues/4); `frdminc/nix2cf` has `buildfile` only. **Amended 2026-11-13 (D46):** the compiler moves into tendcf under `compiler/` and is split into plugins ([`plugin-layout.md`](plugin-layout.md)); ~~its `tendcf.lock` is specified in [`change-process.md`](change-process.md) §2~~ with schema and compiler in one repository there is no `tendcf.lock` (it was specified, never built). | | 4 | Linux reference path on a stock distro. | | 5 | First real Linux host; prove roles are data. | | 6 | Signed releases, push-only; goal-file render + diff + on-device validator (§9). Operator hosts. The lockfile, release, rollback and CI-gate design is [`change-process.md`](change-process.md) (#2); the code is not written. On Linux and macOS the on-device validator is `tendcf-validate`, built after schema v5 lands and before any further host is enforced; how much of this step's validator its first build carries is open (D45, §12). | @@ -1165,7 +1191,7 @@ Older “superseded by” archaeology lives in | D6 | Nix on bare metal | No. Builds / dev shells / authoring language / passthrough where a host already runs NixOS or nix-darwin. Never the OS a stranger must install. | | D8 | Trust layer | Build it, gated. Sovereignty is the point. | | D10 | Task runner | `just` for humans; mise for toolchains. | -| D12 | Authoring frontend | **Amended 2026-10-29.** Two equal frontends, Nix modules and YAML, rendering to byte-identical goal files; a Nix-authored site runs on hosts without Nix. Existing NixOS/nix-darwin configurations are used verbatim (passthrough). JSON remains the wire format. [`linux-targets.md`](linux-targets.md) §5. | +| D12 | Authoring frontend | **Amended 2026-10-29.** Two equal frontends, Nix modules and YAML, rendering to byte-identical goal files; a Nix-authored site runs on hosts without Nix. Existing NixOS/nix-darwin configurations are used verbatim (passthrough). JSON remains the wire format. [`linux-targets.md`](linux-targets.md) §5. **Amended 2026-11-13 (D46):** the Nix frontend is the compiler plugin `plugins/frontends/nix/` and carries the name `nix2cf`; the YAML frontend is `plugins/frontends/yaml/` (built in); a Guix frontend may follow. Not built yet ([`plugin-layout.md`](plugin-layout.md)). | | D13 | Service owner | CFEngine promises everywhere; mise toolchains only. | | D14 | Deployment shape | Git-synced policy; `cf-serverd` on every host; push and pull are one mechanism. | | D15 | Compile target | CFEngine Augments, not freehand `.cf`. | @@ -1174,7 +1200,7 @@ Older “superseded by” archaeology lives in | D18 | Local record | ~~JSONL is the durable capture; SQLite in tendcf-agent is the index. On Android the agent owns both in app-private storage.~~ **Amended 2026-11-02 by D45:** JSONL is the durable capture; SQLite is the index, rebuildable from it. On Android tendcf-agent owns both in app-private storage. On Linux and macOS there is no agent, and the index's owner there is open (§8, §12). | | D19 | Composition | Private-site lockfile pins inputs (DAG). Signed release is the deploy artifact. No sibling-repo tag lockstep. | | D20 | Nix store locality | Single writer per host. | -| D21 | Schema home | **tendcf** owns schemas/types. nix2cf is the compiler. | +| D21 | Schema home | **tendcf** owns schemas/types. ~~nix2cf is the compiler.~~ **Amended 2026-11-13 (D46):** the compiler also lives in tendcf (`compiler/`, command `tendcf compile`); schemas stay in `schema/`, outside the compiler's code, and the two change in the same commit. `nix2cf` names the Nix frontend plugin. | | D22 | Platform sequence | macOS, Linux. Android deferred by the operator on 2026-10-29 ([`linux-targets.md`](linux-targets.md)). | | D23 | YAML canonicalize | Parse, re-serialize, diff on the YAML path. | | D24 / D40 | Lookup + auto-provide | Registry/token CLI; default `service:`; compile errors teach names. | @@ -1198,6 +1224,7 @@ Older “superseded by” archaeology lives in | D43 | ChangePlan model | ~~Model A: a closed `capability` vocabulary per operation, executor refuses effects outside the declared set.~~ **Superseded 2026-08-15 by Model B** (synthesis finding E1, adjudicated in `e1-adjudication-xhigh-2026-08-15.md`, which is final and supersedes `e1-adjudication-2026-08-15.md`): the wire artifact is the complete canonical per-host **goal file**; the ChangePlan is the **diff** against the device’s approved baseline; the executor is a **validator** over that diff, not an interpreter. Rationale: CFEngine has no runtime capability confinement, so Model A’s executor requires an interpreter plus a vocabulary-to-policy correspondence proof that does not exist and is not in budget; and coverage closes by construction when compiler and validator share one schema. The vocabulary, its versioning, and its skew policy are dropped. See §9. | | D44 | Goal-file schema family | The consent artifact is three paired schemas (goal file, goal diff, approval record) with canonical-byte fixtures. Binding sub-decisions, all from E1 §5 and normative in §9: diff computed **device-side**, full file on the wire, compiler diff is preview (§9.1 / E1 §5.1); RFC 8785 canonical JSON, no defaults, no empty collections, refuse-never-normalize, entry-granular structural hunks (§9.2 / E1 §5.2); accept is all-or-nothing, refusal annotated, no dependency groups (§9.3 / E1 §5.3); first adoption and schema migration are bounded **baseline ceremonies**, migration must diff to empty (§9.4 / E1 §5.4); hunk attribution is **excluded from the format** — impossible device-side — and replaced by an `explain-hunk` query (§9.5 / E1 §5.5); `schema_version` fails closed on unknown kinds, strandedness prevented at the compiler, two-phase bumps (§9.6 / E1 §5.6); per-domain coverage travels **in** the goal file (§9.7 / E1 §5.7). §14.2’s pre-build review applies to this family. | | D45 | Device-side component per platform | **Operator, 2026-11-02.** (1) tendcf-agent is **Android-only**, written in Kotlin. A Linux or macOS host runs CFEngine plus "a small validate-and-project command" (the ruling's words; the name **`tendcf-validate`** was chosen by the amending session on 2026-11-02 and may change without reopening the ruling), run before cf-agent reads `host_specific.json`; no daemon. (2) `tendcf-validate` is Python, the promotion of `bin/projector.py`: one implementation, so no byte-identity drift on those platforms. It needs `python3` on each host (Debian ships it, NixOS through the derivation, macOS bundled or the Command Line Tools). The Android agent's Kotlin projector is a port held to the same golden bytes in CI (residue R22), not a second normative source. (3) Built after schema v5 lands, before any further host is enforced. **Supersedes** every reading that ran the validator, the projector or the SQLite index inside tendcf-agent on every host: the §1 diagram's "CFEngine + tendcf-agent" per device, §8's index owner and D18 as written, §9's "run inside tendcf-agent" and "the agent binary" as the projector's call site, `goal-file-schema-reconciliation-2026-08-15.md` §0 item 7, §9 and §13 ("CI invokes the agent's own projector"), and `projector-reconciliation-2026-08-16.md` §9 and R22 ("tendcf-agent must reproduce" the reference). Not decided by it, and open for the operator: §12's list. | +| D46 | Repository layout and compiler plugins | **Operator, 2026-11-13.** (1) `frdminc/nix2cf` folds into tendcf with full history, first under `compiler/`, and is archived once the move is done. (2) The repository takes the plugin-type layout of [`plugin-layout.md`](plugin-layout.md): `schema/`, `examples/`, `policy/`, `overlay/`, `deploy/` unchanged; `tools/` (was `bin/`); `compiler/` (merge, conflict check, render); `plugins/frontends/` (`yaml`, `nix`, later `guix`), `plugins/supervisors/` (`launchd`, `systemd`, `runit`), `plugins/importers/` (`ansible`, `salt`); `contrib/` for in-tree third-party plugins. One Python package. (3) Plugins are found through the entry-point groups `tendcf.frontends`, `tendcf.supervisors` and `tendcf.importers`, so a third-party plugin can also live out of tree as a pip package. (4) Plugins are compile-time only: none runs on a device, and everything one produces passes the schema lint and the projector, so a plugin cannot bypass consent. A frontend is a language a site keeps authoring in; an importer is a one-shot migration into the Site Model. (5) The compiler's command becomes `tendcf compile` (`nix2cf buildfile` a short-lived alias); `nix2cf` names the Nix frontend (D12). (6) The CFEngine data keys `nix2cf_services` and `nix2cf_edges` are renamed `tendcf_*` in a later schema bump (v6): planned, not done. Today those names occur only in guide §16.A's illustrative preview-channel example; no schema, fixture, projector or bundle carries them, and the projection's keys are already `tendcf_service` and `tendcf_interlock` ([`plugin-layout.md`](plugin-layout.md) §6). (7) Consequence for [`change-process.md`](change-process.md): the compiler is no longer a separately pinned input, the two-repository bump order and the compiler's `tendcf.lock` go away; site inputs keep `site.lock`. Moving `bin/` to `tools/` and the plugin split wait until open PRs #41 and #42 merge. Amends D12, D21, §1, §3, §5 and Step 3. | Silence = proceed from Step 0. Objections amend this register. @@ -1289,7 +1316,10 @@ with full mesh) is **D38**. - `docs/paper/tendcf-architecture-paper.md` — technical paper; the guide wins on conflict. - `schema/`, `examples/`, `bin/schema_lint.py` — Site Model contract. -- `frdminc/nix2cf` — compiler tool (Step 3). Consumes this contract. +- `frdminc/nix2cf` — compiler tool (Step 3), until it is folded into this + repository (D46) and archived. Consumes this contract. +- `plugin-layout.md` — the repository's plugin-type layout, the three + plugin entry-point groups, and the order of the move (D46). - `tendcf-validate-proposal-2026-11-03.md` — a proposal, deciding nothing: options and a recommendation for each of §12's open D45 items, a minimal first build, and an operator picklist. diff --git a/docs/architecture/change-process.md b/docs/architecture/change-process.md index 57c6e24..6708e58 100644 --- a/docs/architecture/change-process.md +++ b/docs/architecture/change-process.md @@ -9,6 +9,17 @@ on Step 6 until the guide or the map says otherwise. On conflict about the leaves as "a lockfile (flake-style inputs)" and "the signed release is the deploy artifact" with a mechanism concrete enough to build. +**Amended 2026-11-13 (map D46).** The compiler, built as the separate +repository `frdminc/nix2cf`, is folded into tendcf with its history and +`frdminc/nix2cf` is archived ([`plugin-layout.md`](plugin-layout.md)). +The compiler is therefore no longer a separately pinned input: schema and +compiler change in the same tendcf commit, and the two-repository bump +order and the compiler's own `tendcf.lock` that this document used to +specify are gone. Site inputs (site-shared, foreign sites, tool forks) +keep `site.lock` exactly as below. Until the move lands, the compiler's +CI still checks out tendcf's default branch beside it (it never had the +`tendcf.lock` specified here). + > The requirement, verbatim from #2: the Site Model → signed-release → > deploy path must be **much faster and much more atomic** than cutting a > coordinated tag across several sibling repositories. It does not relax @@ -51,8 +62,7 @@ device seven releases behind still computes one true diff (guide §7). | Layer (map §3) | Repository today | What a release takes from it | | --- | --- | --- | -| tendcf | `frdminc/tendcf` | schemas, `bin/projector.py`, generic bundles under `policy/`, the upstream overlay (`overlay/`, see [`upstream-overlay.md`](upstream-overlay.md)) | -| nix2cf | `frdminc/nix2cf` | the compiler (Step 3; today a README) | +| tendcf | `frdminc/tendcf` | schemas, `bin/projector.py`, generic bundles under `policy/`, the upstream overlay (`overlay/`, see [`upstream-overlay.md`](upstream-overlay.md)), and the compiler with its plugins (Step 3; `compiler/` and `plugins/` once the move of map D46 lands, today still `frdminc/nix2cf` with `buildfile` only) | | site-shared | not yet created | recipes | | foreign site-shared | none yet | recipes, namespaced | | site-private | `~/ops/site-private` (today: ops repo; the tendcf inventory does not exist yet) | inventory, allocations, secret *names*, trust policy, the lockfile | @@ -73,13 +83,6 @@ version is chosen. Everything else is content. "type": "git", "url": "https://github.com/alice/site-shared" }, - "nix2cf": { - "ref": "master", - "rev": "582a66123df2919a21279381f47b2d9c6132ee74", - "tree": "8ee45f66743fe1549c162bd9921cf0c222871a50", - "type": "git", - "url": "https://github.com/frdminc/nix2cf" - }, "tendcf": { "ref": "master", "rev": "af0e0c5781ba6eb0123a9fe65db37e6a0ea74905", @@ -114,10 +117,11 @@ Rules for the file, each with the reason it is a rule: (map §9.8, DC-11). The lockfile pins sources, not binaries. -**nix2cf has a one-input lock of the same shape** (`tendcf.lock`) pinning -the tendcf rev its golden tests run against. That is what lets a schema -change land in tendcf before the compiler supports it without breaking the -compiler's CI (§5). +~~**nix2cf has a one-input lock of the same shape** (`tendcf.lock`) +pinning the tendcf rev its golden tests run against.~~ **Amended +2026-11-13:** there is no compiler lock. The compiler is in tendcf, so its +goldens run against the schemas of the same commit, and a schema change +lands together with the compiler support it needs (§4, §5). A `flake.lock` was considered as *the* format and rejected as the requirement (§8.5); a converter from `site.lock` to flake inputs is a @@ -168,24 +172,28 @@ The ordinary case, one site-private commit: by pull (Step 8); consented devices take it when their advisor and their person say yes (Step 9). None of that waits on any other repository. -The cross-repo case — say a new entry kind, which needs a schema (tendcf), -projector support (tendcf), a generic bundle promise (tendcf), compiler -support (nix2cf), and site data (site-private): +The cross-repo case — say a new entry kind, which needs a schema, +projector support, a generic bundle promise and compiler support (all +tendcf), and site data (site-private): 1. **tendcf lands first**, as one commit: schema, fixture, projector, - bundle, and the broken fixtures that prove the lint catches the new - kind's failure modes. tendcf CI is green on its own. No device notices. -2. **nix2cf bumps `tendcf.lock` to that rev and lands compiler support**, in - one commit, green against the new fixtures. No device notices. -3. **site-private bumps both pins and adds the data in one commit.** The - release pipeline runs once. That release is the first thing any device - sees, and it contains all three changes or none of them. - -If step 3 is split (pins first, data later) nothing breaks: a release whose + bundle, compiler support, and the broken fixtures that prove the lint + catches the new kind's failure modes. tendcf CI, compiler goldens + included, is green on its own. No device notices. +2. **site-private bumps the tendcf pin and adds the data in one commit.** + The release pipeline runs once. That release is the first thing any + device sees, and it contains both changes or neither. + +(~~Before 2026-11-13 this was three steps: tendcf, then nix2cf bumping its +`tendcf.lock` and landing compiler support, then site-private bumping +both pins.~~ Map D46 put the compiler in tendcf, so the middle step is +part of the first.) + +If step 2 is split (pin first, data later) nothing breaks: a release whose inputs changed but whose data did not renders the same goal files, and every host computes an empty diff. That is the test that the pin bump was pure. Doing it that way on purpose is cheap and is recommended for any -tendcf or nix2cf bump that touches the projector or the compiler. +tendcf bump that touches the projector or the compiler. If the schema change is **not** backward compatible, map §9.6 governs: it ships as two releases, the validator update first as an ordinary diff, the @@ -199,16 +207,20 @@ Each repository's CI runs against pinned inputs, never against a sibling's `master`: - **tendcf** has no inputs. Its CI is the schema lint, the cross-reference - lint, and the overlay self-test. -- **nix2cf** pins tendcf (`tendcf.lock`). A schema change in tendcf cannot - break nix2cf's CI until nix2cf chooses to bump. -- **site-private** pins both. A change in either cannot reach a release - until site-private chooses to bump. + lint, the overlay self-test and, once the compiler has moved in (map + D46), the compiler's goldens against the same commit's schemas. A schema + change that breaks the compiler fails tendcf's own CI, in the commit + that makes it. +- ~~**nix2cf** pins tendcf (`tendcf.lock`). A schema change in tendcf + cannot break nix2cf's CI until nix2cf chooses to bump.~~ Gone with map + D46. +- **site-private** pins tendcf and the other site inputs. A change in any + of them cannot reach a release until site-private chooses to bump. So the only ordering constraint is the one the data imposes (a compiler must understand a kind before a release uses it), and it is enforced by the -release pipeline refusing, not by anyone cutting tags in sequence. The -sibling repositories can be hours, days, or a month apart; the device sees +release pipeline refusing, not by anyone cutting tags in sequence. tendcf +and the site inputs can be hours, days, or a month apart; the device sees one consistent set. ## 6. CI gates, in the order they run @@ -221,7 +233,7 @@ one consistent set. | tendcf | `bin/xref_lint.py` | yes | | tendcf | `bin/overlay.py self-test` and `lint` (patch overlay applies; drift fails loudly) | yes, this change | | tendcf | overlay drift against upstream (`check --against upstream/master`) | scheduled / manual job, this change | -| nix2cf | compile the tendcf fixtures at the pinned rev; goldens byte-identical | Step 3 | +| tendcf (compiler) | compile the tendcf fixtures at the same commit; goldens byte-identical | in `frdminc/nix2cf` CI today, against tendcf's default branch; moves into tendcf CI with the compiler (map D46) | **Release gates** (site-private, every commit; a pull request gets the same run with signing and publishing skipped, and the per-host diffs attached as @@ -262,7 +274,7 @@ ship. | Measure | Target | Why this number | | --- | --- | --- | -| Site Model edit pushed → signed release exists | **≤ 5 minutes p95** | one CI run: fetch three small inputs, compile a fleet of under fifty hosts, lint, sign. Nothing in the pipeline is heavier than the schema lint, which runs in seconds today. | +| Site Model edit pushed → signed release exists | **≤ 5 minutes p95** | one CI run: fetch a few small inputs, compile a fleet of under fifty hosts, lint, sign. Nothing in the pipeline is heavier than the schema lint, which runs in seconds today. | | Signed release → converged on every *reachable* operator host | **≤ 10 minutes p95** | push (Step 6) triggers an immediate `cf-agent` run; the default 5-minute schedule bounds the pull path (Step 8). | | Devices never reached | reported per release, by trust tier | guide §17 names this as the measurement that says when the release-as-artifact model stops being adequate. | @@ -292,9 +304,10 @@ moved on since. 4. **A bad release that must be stopped before devices apply it** uses the emergency role (map D42): revoke that release's targets without consent, then cut M. Tightening never needs a local yes; the rollback content does. -5. **A bad tendcf or nix2cf commit** is reverted in its own repository on - its own schedule, or not at all; the fleet is fixed by the site-private - pin going back, which is step 1. This is the whole point: the sibling +5. **A bad tendcf commit** (schema, projector, bundle or compiler; ~~a bad + tendcf or nix2cf commit~~ before map D46) is reverted in tendcf on its + own schedule, or not at all; the fleet is fixed by the site-private + pin going back, which is step 1. This is the whole point: the input repositories' history is never load-bearing for what devices run. ## 9. Rejected alternatives @@ -307,6 +320,9 @@ moved on since. require the public engine, the optional public recipes, and the private inventory to be separable, and foreign sites' recipes are by definition other people's repositories. A monorepo also cannot hold the tool forks. + (Map D46, 2026-11-13, does merge two *public engine* repositories, tendcf + and the compiler, which share an owner, a licence and reviewers; the + layers this item keeps apart stay apart.) 3. **Git submodules in site-private.** A submodule pointer is a lockfile with worse properties: it records only `rev` (no `tree`, so a rewritten fork is undetectable until the fetch fails), it is bound to a checkout @@ -315,7 +331,8 @@ moved on since. namespacing or private binds, which the composition rules need anyway. Everything a submodule gives, `site.lock` gives with one JSON file and one script. -4. **Vendoring copies of tendcf/nix2cf into site-private.** Atomic, but it +4. **Vendoring copies of tendcf (and, before map D46, nix2cf) into + site-private.** Atomic, but it duplicates review (every tendcf change is re-reviewed as a vendored diff), loses provenance (`rev`/`tree` of a copy is whatever the copier says), and makes the public engine's history irrelevant to its users. @@ -346,7 +363,9 @@ moved on since. - **The privileged-region ceremony marker** (§6 gate 5) is a site-private commit convention to be chosen with the TUF ceremony (map §14.4). - **The inventory does not exist yet**, so `site.lock` has nothing to pin - for a real fleet; the format is fixed now so that nix2cf's `tendcf.lock` - (Step 3) and the real one (Step 6) are the same thing. + for a real fleet; the format is fixed now so that the real one (Step 6) + needs no redesign. (~~so that nix2cf's `tendcf.lock` (Step 3) and the + real one (Step 6) are the same thing~~: there is no compiler lock since + map D46.) - **The latency numbers in §7 are a proposal.** They become the requirement when the operator confirms them on #2. diff --git a/docs/architecture/linux-targets.md b/docs/architecture/linux-targets.md index daab6d1..5783f5c 100644 --- a/docs/architecture/linux-targets.md +++ b/docs/architecture/linux-targets.md @@ -35,7 +35,7 @@ is a derivation kept in the host flake, not in this repo. | 3 | Fail-closed guard in `policy/tendcf_services.cf`: refuse any unit flavour the bundle does not implement, with a negative test. **Done 2026-10-29:** a present entry is actuated only when its `unit` flavour is `launchd` AND the host is `darwin`; otherwise it is refused and reported by name and flavour. `tests/policy/run.sh` checks 4 (systemd entry, every host) and 5 (launchd entries on a non-Darwin host, CI only). Before this, a systemd entry would have been rendered as a `LaunchDaemons` plist and a launchd entry on Linux handed to a `launchctl` that is not there. | | 4 | One `schema_version` bump from an adapter prototype: richer goal-file systemd flavour (scope, restart, install target, externally-written unit), writer vocabulary with `dpkg`, `nixos` and `nix-darwin`, and an inventory schema carrying `supervisor` and distro family. **Slice A done 2026-10-31:** the family went to `schema_version: 2` (goal-file, goal-diff and approval-record; every fixture and digest regenerated; case 13 now claims 3). `unit.systemd` gained `restart` (`no`/`on-failure`/`always`) and `wanted_by` (a `*.target`), the two knobs the renderer writes, both explicit. A systemd entry's argv and `working_dir` are held to a safe alphabet (no `"` `\` `$` `%` or control character; `working_dir` limited to `[A-Za-z0-9._+@:-]` segments): refused structurally, not escaped, so a value systemd would re-read never reaches the unit file (`service_present` `allOf`; negative cases 70-72). **Slice B, writers, done 2026-10-31 (`schema_version: 3`):** the writer vocabulary gained `nixos` and `dpkg` (`nix-darwin` was already there) in both the goal file and `launchd-writers`; case 13 claims 4; negative case 73 (unknown writer); `deploy/linux/passthrough/goal-file.json` was the first `nixos`-writer goal file, projecting to `{"vars":{}}`. **Schema v4 (2026-11-01, [schema-v4-reconciliation-2026-10-31.md](schema-v4-reconciliation-2026-10-31.md)):** `supervisor` is a required `device-trust` fact (launchd, systemd, runit, nixos); a `nixos` host carries no service, interlock or unit-writer anywhere, and `nixos` left the writer vocabulary; the passthrough fixture says `supervisor: nixos`; cases 93-112. **Schema v5 (2026-11-02, [schema-v5-dpkg-reconciliation-2026-11-02.md](schema-v5-dpkg-reconciliation-2026-11-02.md), slice S2):** Debian package claims: the `unit-package` kind (binary package names, systemd hosts only), `dpkg` left the `unit-writer` writer vocabulary (a package's units are claimed by a `unit-package` entry, never a writer value) and `tendcf-install` joined it (one unit, `tendcf-execd.service`); the `unit-writer` key grammar is exact names, template keys and prefixes (a systemd prefix belongs to cfengine alone; launchd and runit keep the v4 language); lint overlap, cross-domain and package-uniqueness rules; `report-row` gained `extra_scan`, `two_writers` and `unmet_absence`; cases 113-155. Report-only: v5 prevents no competing write, and the detector itself (S5) is not built. Still deferred: `scope`, externally-written unit, distro family, an inventory schema (spec Q8). | | 5 | systemd branch of the generic bundle, enforce tests on T1. **Done 2026-10-31 (PR #11):** a present `systemd` entry on a host booted by systemd (`/run/systemd/system`) renders `/etc/systemd/system/` from `policy/templates/systemd.service.mustache`, then `daemon-reload`, `enable`/`disable` per `enabled`, `start` (or `restart` when a running unit changed); a tombstone with a regular unit file present (`isplain`) runs `disable --now`, deletes the file, reloads; a tombstone on an alias link, mask or dangling link is reported (`tombstone not actuated`) and nothing runs (v5 spec S0). State probes are run-cached snapshots, so one run converges and the next issues no `systemctl`. `tests/policy/run.sh` check 6 runs that lifecycle against the real manager on the three ubuntu VM jobs (`TENDCF_TEST_SYSTEMD=1`, passwordless sudo; unit passes `systemd-analyze verify`) and the refusal path on `debian:13` and Darwin; check 4 now uses a runit entry. The prototype's gaps (no restart policy, unescaped argv) closed in step 4 slice A. | -| 6 | nix2cf: systemd renderer (supervisor from inventory), the Linux-relevant `TODO(operator)` markers, the `tendcf.lock` consumer. nix2cf emits only the goal file; Augments stay on the device (`projector.py`). | +| 6 | Compiler (`tendcf compile`, was nix2cf; map D46): systemd renderer (supervisor from inventory; the `plugins/supervisors/systemd` plugin once the split of [`plugin-layout.md`](plugin-layout.md) lands), the Linux-relevant `TODO(operator)` markers. (~~The `tendcf.lock` consumer~~ dropped 2026-11-13: with the compiler in tendcf there is no `tendcf.lock`.) The compiler emits only the goal file; Augments stay on the device (`projector.py`). | | 7 | T3 dry-run, then one enforced service. **Dry-run done 2026-10-29:** the `debian:13` CI artifact (`cfengine-fork-debian13`, fork `7363abf43`) installed under `~/cf` on pastebin as the unprivileged user; `tests/policy/run.sh` passes all five checks there, and `cf-agent -n` on `examples/host_specific.json` refuses both entries by name (interlock unmet: the example's caddy path is a Mac path; service: `launchd` flavour on `debian_13`) and touches nothing. Found on the way: the artifact needed `librsync`/`libb2` bundled (now done) and the harness had to own its umask (pastebin is 002). **Enforced service, plan decided 2026-10-31:** the canonical CFEngine shape, not a sudoers rule. pastebin's sudo asks for a password and `~/cf` is user-writable, so a passwordless root `cf-agent` there would be an escalation path; CFEngine's own model is a root-owned install and a root `cf-execd` (book ch. 19, 21, 23; [cf-execd](https://docs.cfengine.com/docs/lts/reference/components/cf-execd), [MPF](https://docs.cfengine.com/docs/lts/reference/masterfiles-policy-framework); an unprivileged `cf-agent` cannot do root work, [CFEngine blog](https://cfengine.com/blog/2024/feature-friday-25-unprivileged-execution)), and our v3 design already self-bootstraps each host. Stage A, one password-sudo session: install `/opt/tendcf` root-owned, place the policy and the canary data, one `cf-agent -K -I` run, inspect. Stage B: enable a root `cf-execd`. The canary is `tendcf.canary.service` (`sleep infinity` as `nobody`), its CMDB data a projection of a schema-valid goal file. No masterfiles framework: the installer copies the policy in as root. The CI build now uses `--prefix=/opt/tendcf` so the binaries' library path is root-owned; `tests/deploy/linux-install.sh` runs the full sequence on the ubuntu VMs. Runbook and files: [`deploy/linux/`](../../deploy/linux/README.md). **Done on pastebin 2026-10-31 (tendcf `8f63225`, fork `b8abf2840`):** operator granted root SSH, so the installer ran as root with no sudoers rule left behind. Stage A: canary rendered, enabled, active as `nobody` with `Restart=on-failure`; a second run issued no `systemctl` command. Stage B: a root `cf-execd` (`tendcf-execd.service`) restored the canary within 3 minutes of a manual stop. Found live, invisible to CI (apt installs librsync there): `RUNPATH` on the `cf-*` binaries is not inherited by `libpromises.so`'s own dependencies, so the bundled `librsync`/`libb2` were not found; the installer and the unit now set `LD_LIBRARY_PATH=/opt/tendcf/lib`. The canary is left running under `cf-execd`; its tombstone is the removal path. | | 8 | T4: the NixOS passthrough adapter (§5), dry-run first. Then T5/T6/T7. **Probe done 2026-10-31, plan proposed:** [`nixos-passthrough-plan.md`](nixos-passthrough-plan.md). vps-primary's `/etc/systemd/system` is a read-only store symlink, so the generic systemd adapter must refuse there (guard first), and the CFEngine build is a derivation in the host flake. | @@ -80,10 +80,11 @@ both first-class: 2. **Nix as an authoring language.** The Site Model's types are also Nix module options (`mkOption`, `mkIf`, `mkDefault`). A site authored in Nix evaluates (`nix eval --json --store dummy://`) to the same - schema-validated goal file as one authored in YAML, and nix2cf renders it - for a launchd or systemd host with no Nix installed. The person's Nix - knowledge carries over; Nix the runtime is not needed on the target (D6 - unchanged). + schema-validated goal file as one authored in YAML, and the compiler + renders it for a launchd or systemd host with no Nix installed. The Nix + frontend is the compiler plugin named `nix2cf` (map D12, D46). The + person's Nix knowledge carries over; Nix the runtime is not needed on + the target (D6 unchanged). YAML and Nix are equal frontends: each fixture authored in Nix has a YAML twin, and the two must render to byte-identical goal files. Nobody has to diff --git a/docs/architecture/plugin-layout.md b/docs/architecture/plugin-layout.md new file mode 100644 index 0000000..9ef3bd8 --- /dev/null +++ b/docs/architecture/plugin-layout.md @@ -0,0 +1,152 @@ +# Repository layout and compiler plugins (decided 2026-11-13) + +> Operator ruling, 2026-11-13; recorded as map D46 +> ([`architecture-DEFINITIVE-v3.md`](architecture-DEFINITIVE-v3.md)). +> Status: decided, **almost none of it built**. §2 marks each part as +> exists today or planned. Where this file and the guide disagree on the +> current design, the guide wins. + +## 1. The decision + +1. `frdminc/nix2cf`, the compiler, folds into `frdminc/tendcf` with its + full git history, first under `compiler/`. A separate pull request does + the import; this document moves no files. +2. The repository takes the plugin-type layout of §2. It stays one Python + package. +3. Plugins are found through Python entry-point groups (§3), so a + third-party plugin can live outside this repository as an ordinary + pip package. +4. Plugins run at compile time only (§4). +5. The compiler's command becomes `tendcf compile`. The name `nix2cf` + moves to the Nix frontend (§6). +6. `frdminc/nix2cf` is archived once the move is done. + +The consequence for the change process: the compiler is no longer a +separately pinned input. Schema and compiler change in the same commit, +so the two-repository bump order and the compiler's `tendcf.lock` go +away. Site inputs (site-shared, foreign sites, tool forks) keep +`site.lock` ([`change-process.md`](change-process.md) §2, §4). + +## 2. The tree + +``` +tendcf/ + schema/ examples/ the contract; paths unchanged + policy/ overlay/ deploy/ device side; unchanged + tools/ was bin/ (schema_lint, projector, xref_lint, overlay, …) + compiler/ was the nix2cf repository: merge, conflict check, render + plugins/ + frontends/yaml/ the YAML Site Model loader (built in) + frontends/nix/ the Nix module frontend (map D12); named nix2cf + frontends/guix/ later: Guix System, Guile configuration → JSON via `guix repl` + supervisors/launchd/ unit renderers, one per supervisor + supervisors/systemd/ + supervisors/runit/ + importers/ansible/ optional one-shot migration into the Site Model + importers/salt/ + contrib/ in-tree third-party plugins, lower bar + docs/ tests/ +``` + +| Path | Today (2026-11-13) | Planned | +| --- | --- | --- | +| `schema/`, `examples/` | exist in tendcf | unchanged | +| `policy/`, `overlay/`, `deploy/` | exist in tendcf | unchanged | +| `tools/` | is `bin/`: standalone `uv run --script` files; tendcf has no `pyproject.toml` | `bin/` renamed `tools/`, after #41 and #42 merge | +| `compiler/` | `frdminc/nix2cf`: `nix2cf/cli.py`, `compile.py`, `sitemodel.py`; one command, `nix2cf buildfile` | imported with history, then `tendcf compile` | +| `plugins/frontends/yaml/` | `nix2cf/sitemodel.py` (YAML loading with duplicate-key refusal and line numbers) | split out of the compiler, still built in | +| `plugins/frontends/nix/` | only the hook: `nix2cf eval-nix FILE` evaluates a plain Nix expression with `nix-instantiate --eval --strict --json`; no module frontend | the D12 Nix module frontend, named `nix2cf` | +| `plugins/frontends/guix/` | nothing | later | +| `plugins/supervisors/launchd/` | inside `nix2cf/compile.py`, the only supervisor whose goal-file `unit` entry is rendered | split out | +| `plugins/supervisors/systemd/`, `runit/` | nothing: a service selected for a systemd or runit host is a compile error (`RENDERED_SUPERVISORS` in `compile.py`) | written as plugins ([`linux-targets.md`](linux-targets.md) §3, step 6) | +| `plugins/importers/` | nothing | optional | +| `contrib/` | nothing | as third parties contribute | +| entry-point groups | none; nix2cf's `pyproject.toml` declares only the `nix2cf` script | the three groups of §3 | + +A supervisor plugin renders a service record into the goal file's `unit` +entry for that supervisor and resolves that supervisor's defaults (for +example `working_dir`). It does not write the unit file on the host; the +generic bundle under `policy/` does that, after approval and projection. + +## 3. Plugin kinds and entry-point groups + +| Kind | Entry-point group | What it is | +| --- | --- | --- | +| Frontend | `tendcf.frontends` | A language a site keeps authoring in. It turns its source into the Site Model's JSON, which the compiler then merges, checks and renders like any other. YAML and Nix are the two equal frontends of map D12; Guix may follow. | +| Supervisor | `tendcf.supervisors` | A unit renderer for one supervisor (launchd, systemd, runit, …): service record in, goal-file `unit` entry out (§2). | +| Importer | `tendcf.importers` | An optional one-shot migration from another tool's configuration (Ansible, Salt) into Site Model files, which the site then keeps and edits as its own. Not run on every compile. | + +The difference between a frontend and an importer is what happens next. A +site authored through a frontend keeps its source in that language. A site +migrated by an importer keeps the Site Model files the importer wrote, and +the importer is not needed again. + +Built-in plugins are found through the same groups as third-party ones. +The Python interface a plugin implements is not specified yet; it is to +be settled when the split is done (§5, step 3). + +## 4. Rules for every plugin + +**Compile time only.** No plugin runs on a device. A device receives a +signed release and runs CFEngine with `tendcf-validate` or tendcf-agent +(map D45), never the compiler. + +**No way around consent.** Everything a plugin produces passes through the +same checks as the rest of the compiler's output: the Site Model and the +goal file are validated against `schema/`, and the goal file is projected +by the reference projector. A goal file the schema lint or the projector +rejects stops the release ([`change-process.md`](change-process.md) §6, +release gates 3 and 4). One that passes still reaches a device only as a +diff the device validates against its approved baseline (guide §7). A +plugin has no other path to a device. + +**Where a plugin lives.** + +1. **In tree, `plugins/`.** The YAML frontend, the Nix frontend, the + supervisor renderers the project supports, and any importer the + project maintains. Held to this repository's full review rule. +2. **In tree, `contrib/`.** Third-party plugins accepted into the + repository with a lower bar. They pass the same schema lint and + projector as everything else (the rule above does not relax). +3. **Out of tree.** A pip package that registers under one of the three + entry-point groups, the way Salt extensions and Ansible collections are + shipped apart from Salt and Ansible. tendcf does not review it; its + output is checked like any other plugin's. + +## 5. Order of the move + +| # | Change | State | +| --- | --- | --- | +| 1 | This documentation: the guide, the map (D46), the change process, the Linux targets plan, the glossary, this file. Moves no files and changes no code. | this change | +| 2 | Import `frdminc/nix2cf` with full history under `compiler/`; the root `README.md` follows. | separate pull request | +| 3 | `bin/` → `tools/`; split the compiler into `compiler/` and `plugins/` with the three entry-point groups; the command becomes `tendcf compile`, `nix2cf buildfile` kept as a short-lived alias. Waits until open PRs #41 and #42 merge. | planned | +| 4 | Schema v6: rename the `nix2cf_*` data keys to `tendcf_*` (§6). | planned | +| 5 | Archive `frdminc/nix2cf`. | planned, once the move is done | + +## 6. Names + +1. **`tendcf compile`** is the compiler. Today the same code is + `nix2cf buildfile`; that spelling stays as a short-lived alias. +2. **`nix2cf`** is the Nix module frontend, `plugins/frontends/nix/`. Up + to 2026-11-13 the name meant the whole compiler; older documents and + handoffs use it that way and are not rewritten. +3. **`guix`** is the planned Guix frontend, `plugins/frontends/guix/`. +4. **The `nix2cf_*` data keys.** The decision renames `nix2cf_services` + and `nix2cf_edges` to `tendcf_*` in schema v6 (planned; today's goal + file is `schema_version` 5). Checked on 2026-11-13: those two names + occur only in the illustrative, preview-channel `host_specific.json` + of guide §16.A (and its copy in the paper) and in review notes about + it; the guide itself says that file does not load as Augments. No + schema, fixture, projector or bundle carries them; the + projection's `vars` keys are already `tendcf_service` and + `tendcf_interlock`. What v6 renames therefore depends on what carries + those names by then. + +## 7. Not decided here + +- The Python interface each plugin kind implements (§3). +- The review bar for `contrib/` beyond the schema lint and the projector. +- Which importers, if any, get built. +- How out-of-tree plugins are pinned for a release. `site.lock` pins + sources ([`change-process.md`](change-process.md) §2, rule 5); a plugin + installed from pip is not yet covered by it. diff --git a/docs/paper/tendcf-architecture-guide.md b/docs/paper/tendcf-architecture-guide.md index 6f72548..ca6d608 100644 --- a/docs/paper/tendcf-architecture-guide.md +++ b/docs/paper/tendcf-architecture-guide.md @@ -26,9 +26,9 @@ open questions in research form, is **Nothing described here is deployed.** The data formats exist and are checked, a first generic policy bundle runs, and the projection step has a -reference implementation. The compiler, the on-device executor, and the -screens a person would use to accept or refuse a change are still to be -built. No device has been set up from factory reset by this automation. +reference implementation. The compiler has its first piece (`buildfile`, +§4); the rest of it, the on-device executor, and the screens a person +would use to accept or refuse a change are still to be built. No device has been set up from factory reset by this automation. One thing that *was* built, and was not planned: §19 describes the defects found underneath, in CFEngine itself, by leaning on the parts of it this @@ -70,7 +70,8 @@ Two other goals sit alongside that: ``` tendcf engine: schemas/types, generic adapters, - (public) default advisor prompt, compiler interface + (public) default advisor prompt, and the compiler + with its plugins (one repository, §4) site-shared reusable recipes (a Caddy role, an apt pattern). (optional, public) Not live inventory. @@ -82,14 +83,15 @@ Two other goals sit alongside that: (never an input) secret names, trust policy, extra advisor prompt. Holds the lockfile that pins everything else. - tool forks (optional) nix2cf, sudo-secretspec, Shizuku, … only if + tool forks (optional) sudo-secretspec, Shizuku, … only if patching them. │ Site Model instances (private) + recipes (shared) ▼ - nix2cf compiler: merge → conflict check → infer - (tool) ordering → CFEngine Augments JSON. - Not the home of schemas or facts. + tendcf compile compiler, in tendcf: frontend (YAML, or Nix + (tool) via the nix2cf plugin) → merge → conflict + check → infer ordering → per-host goal file. + Schemas stay in `schema/`; holds no facts. ▼ Signed release manifest + per-host canonical goal file @@ -110,7 +112,11 @@ Two other goals sit alongside that: Git-repo count is not the composition mechanism. Layers are roles. Site-private holds a lockfile (flake-style inputs) that pins tendcf, -nix2cf, site-shared, foreign shared sites, and optional tool forks. The +site-shared, foreign shared sites, and optional tool forks. The compiler +is part of tendcf, so one tendcf pin covers schemas and compiler together +(decided 2026-11-13; until the move lands the compiler is still the +separate `frdminc/nix2cf` repository, see +[`plugin-layout.md`](../architecture/plugin-layout.md)). The signed release is the deploy artifact. There is no lockstep of sibling checkouts that must share one tag. @@ -136,8 +142,9 @@ those facts. Adapters translate. CFEngine, the toolchain bootstrapper, and Nix-for-builds are *consumers*. Any one of them can be replaced without moving the facts. -**Schemas and types live in tendcf**, not in the compiler. A schema change -is an interface change. A hostname change is site data. They do not move +**Schemas and types live in tendcf's `schema/`**, not in the compiler's +code, although the compiler is moving into the same repository (decided +2026-11-13, §4). A schema change is an interface change. A hostname change is site data. They do not move together. The files are [`schema/`](../../schema/) and [`examples/`](../../examples/) in this repository. @@ -184,7 +191,8 @@ Debian box that has no Nix at all; and a host that already runs NixOS or nix-darwin keeps its configuration verbatim, tendcf adding only distribution, consent and reporting around it (decided 2026-10-29, [`linux-targets.md`](../architecture/linux-targets.md) §5). The Nix -frontend is not built yet; until it is, the files are written as YAML. +frontend, which will carry the name `nix2cf` (§4), is not built yet; +until it is, the files are written as YAML. Every schema is paired with a concrete example file, except schemas that hold only shared definitions and describe no document of their own; those @@ -199,17 +207,32 @@ reading the whole registry. ## 4. A compiler turns facts into CFEngine data -The compiler (working name `nix2cf`) reads the Site Model and writes -[CFEngine Augments](https://docs.cfengine.com/docs/3.21/reference-language-concepts-augments.html) +The compiler reads the Site Model and, by way of the per-host goal file +and the device-side projection of §7, feeds the `host_specific.json` half +of [CFEngine Augments](https://docs.cfengine.com/docs/3.21/reference-language-concepts-augments.html) — JSON files (`def.json`, `host_specific.json`) CFEngine reads natively as a data-injection layer. The version history is on that page: 3.7.0 put augments into the Masterfiles Policy Framework and 3.7.3 back-ported `def.json` parsing into the core agent, but `$(sys.workdir)/data/host_specific.json` has only been parsed **since 3.18.0**. Using both files therefore sets the -floor at CFEngine 3.18, not 3.7. The name is historical, and despite it -**YAML is not a valid Augments input** — CFEngine 3.27.1 reads JSON only. -YAML remains an authoring format for the Site Model (§7) and never reaches -the agent. The compiler is a tool, not the home of schemas. +floor at CFEngine 3.18, not 3.7. **YAML is not a valid Augments input** +— CFEngine 3.27.1 reads JSON only. YAML remains an authoring format for +the Site Model (§7) and never reaches the agent. The compiler is a tool, +not the home of schemas. + +**Name and home (decided 2026-11-13).** The compiler was built as the +separate repository `frdminc/nix2cf`, whose `nix2cf buildfile` command is +the first piece described below. It is being folded into tendcf with its +history, under `compiler/`, and its command becomes `tendcf compile` +(`nix2cf buildfile` stays for a while as an alias). The name `nix2cf` +moves to what it describes: the Nix module frontend (§3), one of the +compiler's plugins. Frontends, supervisor renderers and one-shot +importers from other tools are plugins found through Python entry points, +so a third party can ship one as an ordinary package. Plugins run only at +compile time, never on a device, and everything they produce still passes +the schema lint and the projector, so no plugin can get around consent. +The layout and the order of the move are in +[`plugin-layout.md`](../architecture/plugin-layout.md). **The two files are not read by the same code, and they do not land in the same place.** `def.json` goes through the augments loader and its keys @@ -242,15 +265,17 @@ The pipeline has four stages: the resource, every writer that declared it and where, the conflicting values, and what a resolution would look like. 3. **Dependency inference** — see §9. -4. **Render** the Augments JSON. +4. **Render** the per-host goal file (§7). The device, not the compiler, + projects it to Augments JSON after approval. Merge happens once, in the compiler, before render. CFEngine’s own `mergedata()` is not used for this. Because the render is a pure function of the Site Model — same input, same output — “show me exactly what device X would receive, without touching -device X” is almost free. That command is planned as the first piece of -the compiler. It is how an agent checks its own work, how the compiler +device X” is almost free. That command, `buildfile`, was the first piece +of the compiler built: merge, conflict check and goal-file render, with +service units for launchd only and no inference yet (§18). It is how an agent checks its own work, how the compiler regression-tests itself, and how a human sees a proposed change before it lands. The shape is [Bcfg2 `buildfile`](https://docs.bcfg2.org/). @@ -263,8 +288,9 @@ The pairing here is Site Model into CFEngine Augments, because of multi-owner operation — not because the compiler mechanism is new. **Where it runs:** operator machine or CI. Consented devices receive a -signed artifact. They do not run nix2cf; Nix runs on a device only where -it already is that host's configuration system (NixOS, nix-darwin). +signed artifact. They do not run the compiler or any of its plugins; Nix +runs on a device only where it already is that host's configuration +system (NixOS, nix-darwin). --- @@ -1269,13 +1295,19 @@ observe themselves crossing is a disclaimer rather than a ceiling. - **A design for the change process** — lockfile, release, rollback, CI gates — in `docs/architecture/change-process.md`. Design only: the resolver and the release pipeline are Step 6 code. +- **The compiler's first piece**, `nix2cf buildfile` (§4): merge, + conflict check and render of one host's goal file, with service units + for launchd only. Its tests run in CI against tendcf's schemas and + reference projector. It lives in the separate `frdminc/nix2cf` + repository until the move into tendcf decided on 2026-11-13 lands. **Not built — this is still most of it** -The compiler `nix2cf` itself (merge, conflict check, inference, render), -every supervisor adapter except launchd, the signed release path, the -on-device validator, the consent surface, and the peer-action runtime. No -operational numbers of any kind. +The rest of the compiler (inference, extra-entry reporting, service +units for any supervisor but launchd, the Nix module frontend and the +plugin layout, §4), every supervisor adapter except launchd, the signed +release path, the on-device validator, the consent surface, and the +peer-action runtime. No operational numbers of any kind. **Build order from here** (each step is meant to leave the system in a coherent, describable state): @@ -1285,7 +1317,7 @@ coherent, describable state): | 0 | Schemas in tendcf | Existing contract is `schema/` + `examples/` + `bin/schema_lint.py`. Remaining: peer_actions, trust-policy shape, generic unit-writers, lookup stub, YAML canonicalize. Transcribe reality (`not-yet-migrated` is the correct day-one state). | | 1 | macOS services adapter | **Started.** `policy/tendcf_services.cf` renders and loads launchd services from projected data, with interlocks enforced (§16.C; the interlock verdict was split into its own bundle on 2026-10-29 after the pass path was found never to open). Still open in this step: package and file promises, environment rendering, the unit-writer detector. Dry-run default. Not in this step: nix-darwin. | | 2 | Android under the Site Model | Same vocabulary. Agent owns JSONL+SQLite in app-private storage. | -| 3 | `nix2cf` compiler | “What would device X receive?” first, then conflict check, extra-entry reporting, then inference. Inference waits until types exist on two platforms. | +| 3 | Compiler (`tendcf compile`, was `nix2cf`) | “What would device X receive?” first, then conflict check, extra-entry reporting, then inference. Inference waits until types exist on two platforms. **Started:** `buildfile` (merge, conflict check, render) exists in `frdminc/nix2cf`; it moves into tendcf under `compiler/` and is split into plugins ([`plugin-layout.md`](../architecture/plugin-layout.md)). | | 4 | Linux reference path | A stock distro, not NixOS. Distro choice is open until this step; Ubuntu Server is the working default. | | 5 | First real Linux host | A second node that can hold backup roles. Proves that roles are data. | | 6 | Signed releases, push-only | TUF-subset ceremony; goal-file render and diff; the on-device validator. Operator hosts only. | From 1ce21a8fd9510faff5b11cb9dacf5da2dc42bd0c Mon Sep 17 00:00:00 2001 From: Daniel JB Clark Date: Fri, 13 Nov 2026 06:09:01 -0500 Subject: [PATCH 2/9] docs: rename illustrative nix2cf_* keys now; bring the paper in line with D46 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The nix2cf_services / nix2cf_edges keys only ever appeared in guide §16.A's illustrative, non-loading preview example and its copy in the paper; no schema, fixture, projector or bundle carried them. Rename them to tendcf_services / tendcf_edges there now, note that the projection's real keys are tendcf_service / tendcf_interlock (and that tendcf_services is the real bundle's name), and drop the planned schema-v6 rename from D46 and plugin-layout.md: no schema bump is involved. Paper §2.1, §2.2, §2.6, §6.2, §8: compiler is tendcf compile, folded into tendcf (decided 2026-11-13); nix2cf names the Nix frontend; one tendcf pin covers schemas and compiler; buildfile recorded as built. Co-Authored-By: Claude Opus 5.5 --- .../architecture-DEFINITIVE-v3.md | 2 +- docs/architecture/plugin-layout.md | 28 ++--- docs/paper/tendcf-architecture-guide.md | 17 +-- docs/paper/tendcf-architecture-paper.md | 100 +++++++++++------- 4 files changed, 90 insertions(+), 57 deletions(-) diff --git a/docs/architecture/architecture-DEFINITIVE-v3.md b/docs/architecture/architecture-DEFINITIVE-v3.md index 794573c..ff4e6eb 100644 --- a/docs/architecture/architecture-DEFINITIVE-v3.md +++ b/docs/architecture/architecture-DEFINITIVE-v3.md @@ -1224,7 +1224,7 @@ Older “superseded by” archaeology lives in | D43 | ChangePlan model | ~~Model A: a closed `capability` vocabulary per operation, executor refuses effects outside the declared set.~~ **Superseded 2026-08-15 by Model B** (synthesis finding E1, adjudicated in `e1-adjudication-xhigh-2026-08-15.md`, which is final and supersedes `e1-adjudication-2026-08-15.md`): the wire artifact is the complete canonical per-host **goal file**; the ChangePlan is the **diff** against the device’s approved baseline; the executor is a **validator** over that diff, not an interpreter. Rationale: CFEngine has no runtime capability confinement, so Model A’s executor requires an interpreter plus a vocabulary-to-policy correspondence proof that does not exist and is not in budget; and coverage closes by construction when compiler and validator share one schema. The vocabulary, its versioning, and its skew policy are dropped. See §9. | | D44 | Goal-file schema family | The consent artifact is three paired schemas (goal file, goal diff, approval record) with canonical-byte fixtures. Binding sub-decisions, all from E1 §5 and normative in §9: diff computed **device-side**, full file on the wire, compiler diff is preview (§9.1 / E1 §5.1); RFC 8785 canonical JSON, no defaults, no empty collections, refuse-never-normalize, entry-granular structural hunks (§9.2 / E1 §5.2); accept is all-or-nothing, refusal annotated, no dependency groups (§9.3 / E1 §5.3); first adoption and schema migration are bounded **baseline ceremonies**, migration must diff to empty (§9.4 / E1 §5.4); hunk attribution is **excluded from the format** — impossible device-side — and replaced by an `explain-hunk` query (§9.5 / E1 §5.5); `schema_version` fails closed on unknown kinds, strandedness prevented at the compiler, two-phase bumps (§9.6 / E1 §5.6); per-domain coverage travels **in** the goal file (§9.7 / E1 §5.7). §14.2’s pre-build review applies to this family. | | D45 | Device-side component per platform | **Operator, 2026-11-02.** (1) tendcf-agent is **Android-only**, written in Kotlin. A Linux or macOS host runs CFEngine plus "a small validate-and-project command" (the ruling's words; the name **`tendcf-validate`** was chosen by the amending session on 2026-11-02 and may change without reopening the ruling), run before cf-agent reads `host_specific.json`; no daemon. (2) `tendcf-validate` is Python, the promotion of `bin/projector.py`: one implementation, so no byte-identity drift on those platforms. It needs `python3` on each host (Debian ships it, NixOS through the derivation, macOS bundled or the Command Line Tools). The Android agent's Kotlin projector is a port held to the same golden bytes in CI (residue R22), not a second normative source. (3) Built after schema v5 lands, before any further host is enforced. **Supersedes** every reading that ran the validator, the projector or the SQLite index inside tendcf-agent on every host: the §1 diagram's "CFEngine + tendcf-agent" per device, §8's index owner and D18 as written, §9's "run inside tendcf-agent" and "the agent binary" as the projector's call site, `goal-file-schema-reconciliation-2026-08-15.md` §0 item 7, §9 and §13 ("CI invokes the agent's own projector"), and `projector-reconciliation-2026-08-16.md` §9 and R22 ("tendcf-agent must reproduce" the reference). Not decided by it, and open for the operator: §12's list. | -| D46 | Repository layout and compiler plugins | **Operator, 2026-11-13.** (1) `frdminc/nix2cf` folds into tendcf with full history, first under `compiler/`, and is archived once the move is done. (2) The repository takes the plugin-type layout of [`plugin-layout.md`](plugin-layout.md): `schema/`, `examples/`, `policy/`, `overlay/`, `deploy/` unchanged; `tools/` (was `bin/`); `compiler/` (merge, conflict check, render); `plugins/frontends/` (`yaml`, `nix`, later `guix`), `plugins/supervisors/` (`launchd`, `systemd`, `runit`), `plugins/importers/` (`ansible`, `salt`); `contrib/` for in-tree third-party plugins. One Python package. (3) Plugins are found through the entry-point groups `tendcf.frontends`, `tendcf.supervisors` and `tendcf.importers`, so a third-party plugin can also live out of tree as a pip package. (4) Plugins are compile-time only: none runs on a device, and everything one produces passes the schema lint and the projector, so a plugin cannot bypass consent. A frontend is a language a site keeps authoring in; an importer is a one-shot migration into the Site Model. (5) The compiler's command becomes `tendcf compile` (`nix2cf buildfile` a short-lived alias); `nix2cf` names the Nix frontend (D12). (6) The CFEngine data keys `nix2cf_services` and `nix2cf_edges` are renamed `tendcf_*` in a later schema bump (v6): planned, not done. Today those names occur only in guide §16.A's illustrative preview-channel example; no schema, fixture, projector or bundle carries them, and the projection's keys are already `tendcf_service` and `tendcf_interlock` ([`plugin-layout.md`](plugin-layout.md) §6). (7) Consequence for [`change-process.md`](change-process.md): the compiler is no longer a separately pinned input, the two-repository bump order and the compiler's `tendcf.lock` go away; site inputs keep `site.lock`. Moving `bin/` to `tools/` and the plugin split wait until open PRs #41 and #42 merge. Amends D12, D21, §1, §3, §5 and Step 3. | +| D46 | Repository layout and compiler plugins | **Operator, 2026-11-13.** (1) `frdminc/nix2cf` folds into tendcf with full history, first under `compiler/`, and is archived once the move is done. (2) The repository takes the plugin-type layout of [`plugin-layout.md`](plugin-layout.md): `schema/`, `examples/`, `policy/`, `overlay/`, `deploy/` unchanged; `tools/` (was `bin/`); `compiler/` (merge, conflict check, render); `plugins/frontends/` (`yaml`, `nix`, later `guix`), `plugins/supervisors/` (`launchd`, `systemd`, `runit`), `plugins/importers/` (`ansible`, `salt`); `contrib/` for in-tree third-party plugins. One Python package. (3) Plugins are found through the entry-point groups `tendcf.frontends`, `tendcf.supervisors` and `tendcf.importers`, so a third-party plugin can also live out of tree as a pip package. (4) Plugins are compile-time only: none runs on a device, and everything one produces passes the schema lint and the projector, so a plugin cannot bypass consent. A frontend is a language a site keeps authoring in; an importer is a one-shot migration into the Site Model. (5) The compiler's command becomes `tendcf compile` (`nix2cf buildfile` a short-lived alias); `nix2cf` names the Nix frontend (D12). (6) The keys `nix2cf_services` and `nix2cf_edges` were only ever illustrative: they occurred in guide §16.A's preview-channel example and its copy in the paper, and no schema, fixture, projector or bundle carried them (the projection's keys are `tendcf_service` and `tendcf_interlock`). They were renamed `tendcf_services` and `tendcf_edges` in those illustrations in the same change that recorded this decision. No schema bump is involved ([`plugin-layout.md`](plugin-layout.md) §6). (7) Consequence for [`change-process.md`](change-process.md): the compiler is no longer a separately pinned input, the two-repository bump order and the compiler's `tendcf.lock` go away; site inputs keep `site.lock`. Moving `bin/` to `tools/` and the plugin split wait until open PRs #41 and #42 merge. Amends D12, D21, §1, §3, §5 and Step 3. | Silence = proceed from Step 0. Objections amend this register. diff --git a/docs/architecture/plugin-layout.md b/docs/architecture/plugin-layout.md index 9ef3bd8..f2e8e87 100644 --- a/docs/architecture/plugin-layout.md +++ b/docs/architecture/plugin-layout.md @@ -117,11 +117,10 @@ plugin has no other path to a device. | # | Change | State | | --- | --- | --- | -| 1 | This documentation: the guide, the map (D46), the change process, the Linux targets plan, the glossary, this file. Moves no files and changes no code. | this change | +| 1 | This documentation: the guide, the paper, the map (D46), the change process, the Linux targets plan, the glossary, this file; and the rename of the illustrative `nix2cf_*` keys (§6). Moves no files and changes no code or schema. | this change | | 2 | Import `frdminc/nix2cf` with full history under `compiler/`; the root `README.md` follows. | separate pull request | | 3 | `bin/` → `tools/`; split the compiler into `compiler/` and `plugins/` with the three entry-point groups; the command becomes `tendcf compile`, `nix2cf buildfile` kept as a short-lived alias. Waits until open PRs #41 and #42 merge. | planned | -| 4 | Schema v6: rename the `nix2cf_*` data keys to `tendcf_*` (§6). | planned | -| 5 | Archive `frdminc/nix2cf`. | planned, once the move is done | +| 4 | Archive `frdminc/nix2cf`. | planned, once the move is done | ## 6. Names @@ -131,16 +130,19 @@ plugin has no other path to a device. to 2026-11-13 the name meant the whole compiler; older documents and handoffs use it that way and are not rewritten. 3. **`guix`** is the planned Guix frontend, `plugins/frontends/guix/`. -4. **The `nix2cf_*` data keys.** The decision renames `nix2cf_services` - and `nix2cf_edges` to `tendcf_*` in schema v6 (planned; today's goal - file is `schema_version` 5). Checked on 2026-11-13: those two names - occur only in the illustrative, preview-channel `host_specific.json` - of guide §16.A (and its copy in the paper) and in review notes about - it; the guide itself says that file does not load as Augments. No - schema, fixture, projector or bundle carries them; the - projection's `vars` keys are already `tendcf_service` and - `tendcf_interlock`. What v6 renames therefore depends on what carries - those names by then. +4. **The illustrative `tendcf_services` / `tendcf_edges` keys.** Until + 2026-11-13 these were spelled `nix2cf_services` and `nix2cf_edges`. + Checked that day: they occurred only in the illustrative, + preview-channel `host_specific.json` of guide §16.A, its copy in the + paper, and review notes about it; the guide itself says that file does + not load as Augments. No schema, fixture, projector or bundle carried + them. They were renamed in those two illustrations in the same change + as this file, and no schema bump is involved (the goal file stays at + `schema_version` 5). They are still not the projection's keys, which + are `tendcf_service` and `tendcf_interlock`; `tendcf_services` is also + the name of the real generic bundle (`policy/tendcf_services.cf`), + which reads `tendcf_service`. Dated review and reconciliation notes + keep the old spelling. ## 7. Not decided here diff --git a/docs/paper/tendcf-architecture-guide.md b/docs/paper/tendcf-architecture-guide.md index ca6d608..6de2953 100644 --- a/docs/paper/tendcf-architecture-guide.md +++ b/docs/paper/tendcf-architecture-guide.md @@ -912,19 +912,23 @@ host. edge from that match: This illustration is **preview-channel material and does not load as -Augments.** Both of its top-level keys — `data` and `nix2cf_edges` — are +Augments.** Both of its top-level keys — `data` and `tendcf_edges` — are skipped by CFEngine 3.27.1, so as written this file injects nothing. The shape the agent actually consumes is `{"vars": {…}}` and nothing else; the projection onto it runs device-side, after approval. The nesting below is kept because it shows the *derivation* legibly, which is what this walkthrough is for — do not copy it onto a device and expect an effect. +Its key names `tendcf_services` and `tendcf_edges` are illustrative too +(renamed from `nix2cf_*` on 2026-11-13; no schema change): the +projection's real `vars` keys are `tendcf_service` and +`tendcf_interlock` (§16.C), and the goal file carries no edges. ```jsonc // host_specific.json — ILLUSTRATIVE ONLY. Preview channel; does NOT load -// as Augments (top-level `data` and `nix2cf_edges` are both skipped). +// as Augments (top-level `data` and `tendcf_edges` are both skipped). { "data": { - "nix2cf_services": { + "tendcf_services": { "caddy": { "service_policy": "start", "launchd_label": "com.djbclark.caddy", @@ -943,7 +947,7 @@ walkthrough is for — do not copy it onto a device and expect an effect. } } }, - "nix2cf_edges": [ + "tendcf_edges": [ { "from": "litellm-proxy", "to": "caddy", @@ -974,8 +978,9 @@ anyone writing that down either. question is “does this rule apply to this pair?” — a lookup — not “where did this ordering come from?” — a search. -The generic bundle behind `nix2cf_services` is what materializes the -on-device artifact. For `caddy` on macOS, that is a launchd plist +The generic bundle `tendcf_services`, which reads the projected +`tendcf_service` entries, is what materializes the on-device artifact. +For `caddy` on macOS, that is a launchd plist CFEngine keeps present and loaded: ```xml diff --git a/docs/paper/tendcf-architecture-paper.md b/docs/paper/tendcf-architecture-paper.md index 69a2468..508680d 100644 --- a/docs/paper/tendcf-architecture-paper.md +++ b/docs/paper/tendcf-architecture-paper.md @@ -188,18 +188,21 @@ attacked. Git-repo count is not the composition mechanism. Layers are roles: - **tendcf** (public) — engine, schemas/types, generic adapters, default - advisor prompt, compiler interface. + advisor prompt, and the compiler with its plugins (§2.2). - **site-shared** (optional, public) — reusable recipes, not live inventory. - **foreign site-shared** — other people's recipes, listed as read-only inputs. - **site-private** (never an input) — this site's facts: inventory, allocations, secret *names*, trust policy, extra advisor prompt. Holds the lockfile that pins everything else. -- **tool forks** (optional) — nix2cf, sudo-secretspec, Shizuku, only if - patching them. - -Site-private holds a lockfile (flake-style inputs) that pins tendcf, nix2cf, -site-shared, foreign shared sites, and optional tool forks. The signed -release is the deploy artifact. There is no lockstep of sibling checkouts +- **tool forks** (optional) — sudo-secretspec, Shizuku, only if patching + them. + +Site-private holds a lockfile (flake-style inputs) that pins tendcf, +site-shared, foreign shared sites, and optional tool forks. The compiler is +part of tendcf, so one tendcf pin covers schemas and compiler together +(decided 2026-11-13; until the move lands the compiler is still the +separate `frdminc/nix2cf` repository). The signed release is the deploy +artifact. There is no lockstep of sibling checkouts that must share one tag. Collision of the same identity from two peer inputs is a **compile error**. Only site-private may bind a winner (`caddy: from: alice`) or a short name. Never silent last-wins. Foreign @@ -230,8 +233,9 @@ unit-name prefix for every supervisor, killing a two-writers hazard at the source. launchd is one adapter, not the model. Schemas and types belong in tendcf (`schema/`, `examples/`, -`bin/schema_lint.py`). The compiler (`nix2cf`) is a tool; it is not the -home of the contract. A schema change is a tendcf interface change; a +`bin/schema_lint.py`). The compiler (`tendcf compile`, §2.2) is a tool; +its code is not the home of the contract, even though the two are moving +into one repository. A schema change is a tendcf interface change; a hostname change is site data. The Site Model *may* be authored in the Nix module system — `mkOption`, @@ -241,23 +245,36 @@ JSON is what is schema-validated, signed, and read downstream, and a person adopting the project is never required to know Nix to read or fork their own site data. We use the Nix *language* and not the Nix *build system*: a service that should be running is typed data, and nothing about it needs to -become a derivation. +become a derivation. That frontend, not yet built, will carry the name +`nix2cf` (§2.2). ### 2.2 The compiler: a pure function into CFEngine's own data layer -A compiler (working name `nix2cf`) reads the Site Model and renders CFEngine -augments — `def.json` / `host_specific.json` — CFEngine's native JSON -data-injection layer. The name `nix2cf` is historical; an optional Nix module -frontend may author the same JSON later. The compiler is a tool, not the home -of schemas. +A compiler, `tendcf compile`, reads the Site Model and renders a per-host +goal file, which the device projects, after approval, into the +`host_specific.json` half of CFEngine augments — `def.json` / +`host_specific.json` — CFEngine's native JSON data-injection layer (§2.5). +The compiler is a tool, not the home of schemas. + +It was built as the separate repository `frdminc/nix2cf`, whose +`nix2cf buildfile` command is its first piece. On 2026-11-13 it was decided +to fold that repository into tendcf with its history, under `compiler/`, to +rename the command `tendcf compile` (keeping `nix2cf buildfile` for a while +as an alias), and to give the name `nix2cf` to what it describes: the +optional Nix module frontend of §2.1. Frontends, supervisor renderers and +one-shot importers from other tools become plugins found through Python +entry points. They run at compile time only, and everything they produce +still passes the schema lint and the projector, so no plugin can get around +consent. The layout is in +[`plugin-layout.md`](../architecture/plugin-layout.md). Two details of that layer are load-bearing and neither is obvious from the documentation; we state them because we got both wrong first. **The version floor is 3.18, not 3.7.** 3.7.0 put augments into the Masterfiles Policy Framework and 3.7.3 back-ported `def.json` parsing into the core agent, but `$(sys.workdir)/data/host_specific.json` has only been parsed since 3.18.0, -so a design using both files inherits the later floor. **And despite the -name, YAML is not a valid augments input** — the agent reads JSON only. +so a design using both files inherits the later floor. **And YAML is not a +valid augments input** — the agent reads JSON only. YAML remains an authoring format for the Site Model and never reaches a device. @@ -295,12 +312,14 @@ and not a schema redesign. Because the render is a pure function of the Site Model — the same purity Nix's own build model is named for [27] — "show me exactly what device X -would receive, without touching device X" is nearly free. We -plan to build that affordance *first*, before the pipeline is finished, and -§6.2 explains why it earns its place three separate times over. +would receive, without touching device X" is nearly free. We built that +affordance *first*, before the pipeline is finished — `buildfile` merges, +conflict-checks and renders one host's goal file, with service units for +launchd only and no inference yet — and §6.2 explains why it earns its place +three separate times over. **Where it runs:** operator machine or CI. Consented devices receive a -signed artifact. They do not run Nix or nix2cf. +signed artifact. They do not run Nix, the compiler, or any of its plugins. **This compile-to-native-format shape is not new.** A typed or module-based authoring language compiling to an existing execution engine's native format @@ -486,21 +505,25 @@ showing the same interlock as running code rather than as intent. systemd unit or a runit service on another host. The *outputs* — the CFEngine augments and the rendered promise text — are hand-authored by us to show the target shape, as is the Nix rendering of the same input shown -partway through Example A. `nix2cf`'s render stage and its Nix authoring -frontend do not exist yet (§6.5, §8), so nothing in Example A except the -YAML was produced mechanically. +partway through Example A. The compiler's inference stage, which Example A +needs, and its Nix authoring frontend do not exist yet (§6.5, §8), so +nothing in Example A except the YAML was produced mechanically. **One further warning, and it is the kind of thing this paper exists to surface.** The augments document shown in Example A is *pedagogically* right and *mechanically* wrong: its two top-level keys, `data` and -`nix2cf_edges`, are both keys the agent silently skips. As written it +`tendcf_edges`, are both keys the agent silently skips. As written it injects nothing. We keep it because it shows the derivation legibly, which is what the example is for, and we flag it because an illustration that would not run is exactly the sort of claim a reader should be able to catch us on. The shape the agent actually consumes is `{"vars": {…}}` and nothing else, reached by the device-side projection of §2.5; that shape, and a generic bundle that reads it, both exist and are shown at the end of -this section. +this section. The illustration's key names, `tendcf_services` and +`tendcf_edges`, are illustrative as well (renamed from `nix2cf_*` on +2026-11-13, with no schema change); the projection's real keys are +`tendcf_service` and `tendcf_interlock`, and the goal file carries no +edges. **Example A — an edge nobody wrote.** `caddy` and `litellm-proxy` are two records in the same `edge-http` bundle. `litellm-proxy` states only what it @@ -610,11 +633,11 @@ edge to carry its own provenance rather than appear as a bare ordering fact: ```jsonc // host_specific.json (host: mac) — ILLUSTRATIVE ONLY, hand-authored. -// Does NOT load as Augments: top-level `data` and `nix2cf_edges` are both +// Does NOT load as Augments: top-level `data` and `tendcf_edges` are both // skipped by the agent. Shown for the derivation, not to be copied. { "data": { - "nix2cf_services": { + "tendcf_services": { "caddy": { "service_policy": "start", "launchd_label": "com.djbclark.caddy", @@ -630,7 +653,7 @@ edge to carry its own provenance rather than appear as a bare ordering fact: } } }, - "nix2cf_edges": [ + "tendcf_edges": [ { "from": "litellm-proxy", "to": "caddy", @@ -643,9 +666,10 @@ edge to carry its own provenance rather than appear as a bare ordering fact: } ``` -The generic bundle behind `nix2cf_services` is what materializes the actual -on-device artifact — for `caddy`, the launchd plist CFEngine keeps present -and loaded: +The generic bundle `tendcf_services`, which reads the projected +`tendcf_service` entries, is what materializes the actual on-device +artifact — for `caddy`, the launchd plist CFEngine keeps present and +loaded: ```xml @@ -672,7 +696,7 @@ and loaded: ``` -`nix2cf_edges[0]` is what §4.1 means by mandatory attribution: `origin` says +`tendcf_edges[0]` is what §4.1 means by mandatory attribution: `origin` says this edge was never authored, `rule` names the mechanism that produced it, and `source` points at the exact `requires` entry responsible. If this edge turns out to be spurious, the question it presents is "does `rule` correctly @@ -1402,8 +1426,8 @@ been demonstrated rather than argued. `bcfg2-info buildfile` renders exactly the artifact a named client would receive, server-side, without touching the client; the `buildall` variant diffs every client's output across server upgrades. For a pure compiler this -is nearly free, and we plan it first because three separate justifications -converge on it: +is nearly free, and we built it first (`nix2cf buildfile`, §2.2) because +three separate justifications converge on it: - It is the **agent self-check loop.** An agent that can ask "what does my change actually do to device X" verifies its own work locally instead of @@ -1706,8 +1730,10 @@ Two of the original twelve fixtures exposed error messages that were useless failure §3 rules out for the compiler, which we had not applied to the compiler's own tooling. -**Not implemented, and this is still most of it:** the compiler `nix2cf` -itself — merge, conflict check, inference, render — the signed release path, +**Not implemented, and this is still most of it:** the rest of the compiler +— inference, extra-entry reporting, service units for any supervisor but +launchd, the Nix module frontend and the plugin layout (its first piece, +`buildfile`, exists, §2.2) — the signed release path, the on-device validator, the consent surface, peer-action runtime, and every supervisor adapter except launchd. Remaining Step 0 work: `peer_actions`, trust-policy shape, generic unit-writers, lookup stub, YAML canonicalize, From e99db2891b7600540c34cc658b869499b21d02ec Mon Sep 17 00:00:00 2001 From: Daniel JB Clark Date: Fri, 13 Nov 2026 06:11:27 -0500 Subject: [PATCH 3/9] docs: name the illustrative keys preview_services / preview_edges MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Lead call on the follow-up: tendcf_services matched the real generic bundle's name and was one letter off the projected key tendcf_service. Rename the guide §16.A / paper §2.6 illustration keys to preview_* and reword the note, D46 (6) and plugin-layout.md §6 item 4 accordingly. No schema change. Co-Authored-By: Claude Opus 5.5 --- .../architecture-DEFINITIVE-v3.md | 2 +- docs/architecture/plugin-layout.md | 25 +++++++++---------- docs/paper/tendcf-architecture-guide.md | 17 +++++++------ docs/paper/tendcf-architecture-paper.md | 21 ++++++++-------- 4 files changed, 33 insertions(+), 32 deletions(-) diff --git a/docs/architecture/architecture-DEFINITIVE-v3.md b/docs/architecture/architecture-DEFINITIVE-v3.md index ff4e6eb..6ec4fe1 100644 --- a/docs/architecture/architecture-DEFINITIVE-v3.md +++ b/docs/architecture/architecture-DEFINITIVE-v3.md @@ -1224,7 +1224,7 @@ Older “superseded by” archaeology lives in | D43 | ChangePlan model | ~~Model A: a closed `capability` vocabulary per operation, executor refuses effects outside the declared set.~~ **Superseded 2026-08-15 by Model B** (synthesis finding E1, adjudicated in `e1-adjudication-xhigh-2026-08-15.md`, which is final and supersedes `e1-adjudication-2026-08-15.md`): the wire artifact is the complete canonical per-host **goal file**; the ChangePlan is the **diff** against the device’s approved baseline; the executor is a **validator** over that diff, not an interpreter. Rationale: CFEngine has no runtime capability confinement, so Model A’s executor requires an interpreter plus a vocabulary-to-policy correspondence proof that does not exist and is not in budget; and coverage closes by construction when compiler and validator share one schema. The vocabulary, its versioning, and its skew policy are dropped. See §9. | | D44 | Goal-file schema family | The consent artifact is three paired schemas (goal file, goal diff, approval record) with canonical-byte fixtures. Binding sub-decisions, all from E1 §5 and normative in §9: diff computed **device-side**, full file on the wire, compiler diff is preview (§9.1 / E1 §5.1); RFC 8785 canonical JSON, no defaults, no empty collections, refuse-never-normalize, entry-granular structural hunks (§9.2 / E1 §5.2); accept is all-or-nothing, refusal annotated, no dependency groups (§9.3 / E1 §5.3); first adoption and schema migration are bounded **baseline ceremonies**, migration must diff to empty (§9.4 / E1 §5.4); hunk attribution is **excluded from the format** — impossible device-side — and replaced by an `explain-hunk` query (§9.5 / E1 §5.5); `schema_version` fails closed on unknown kinds, strandedness prevented at the compiler, two-phase bumps (§9.6 / E1 §5.6); per-domain coverage travels **in** the goal file (§9.7 / E1 §5.7). §14.2’s pre-build review applies to this family. | | D45 | Device-side component per platform | **Operator, 2026-11-02.** (1) tendcf-agent is **Android-only**, written in Kotlin. A Linux or macOS host runs CFEngine plus "a small validate-and-project command" (the ruling's words; the name **`tendcf-validate`** was chosen by the amending session on 2026-11-02 and may change without reopening the ruling), run before cf-agent reads `host_specific.json`; no daemon. (2) `tendcf-validate` is Python, the promotion of `bin/projector.py`: one implementation, so no byte-identity drift on those platforms. It needs `python3` on each host (Debian ships it, NixOS through the derivation, macOS bundled or the Command Line Tools). The Android agent's Kotlin projector is a port held to the same golden bytes in CI (residue R22), not a second normative source. (3) Built after schema v5 lands, before any further host is enforced. **Supersedes** every reading that ran the validator, the projector or the SQLite index inside tendcf-agent on every host: the §1 diagram's "CFEngine + tendcf-agent" per device, §8's index owner and D18 as written, §9's "run inside tendcf-agent" and "the agent binary" as the projector's call site, `goal-file-schema-reconciliation-2026-08-15.md` §0 item 7, §9 and §13 ("CI invokes the agent's own projector"), and `projector-reconciliation-2026-08-16.md` §9 and R22 ("tendcf-agent must reproduce" the reference). Not decided by it, and open for the operator: §12's list. | -| D46 | Repository layout and compiler plugins | **Operator, 2026-11-13.** (1) `frdminc/nix2cf` folds into tendcf with full history, first under `compiler/`, and is archived once the move is done. (2) The repository takes the plugin-type layout of [`plugin-layout.md`](plugin-layout.md): `schema/`, `examples/`, `policy/`, `overlay/`, `deploy/` unchanged; `tools/` (was `bin/`); `compiler/` (merge, conflict check, render); `plugins/frontends/` (`yaml`, `nix`, later `guix`), `plugins/supervisors/` (`launchd`, `systemd`, `runit`), `plugins/importers/` (`ansible`, `salt`); `contrib/` for in-tree third-party plugins. One Python package. (3) Plugins are found through the entry-point groups `tendcf.frontends`, `tendcf.supervisors` and `tendcf.importers`, so a third-party plugin can also live out of tree as a pip package. (4) Plugins are compile-time only: none runs on a device, and everything one produces passes the schema lint and the projector, so a plugin cannot bypass consent. A frontend is a language a site keeps authoring in; an importer is a one-shot migration into the Site Model. (5) The compiler's command becomes `tendcf compile` (`nix2cf buildfile` a short-lived alias); `nix2cf` names the Nix frontend (D12). (6) The keys `nix2cf_services` and `nix2cf_edges` were only ever illustrative: they occurred in guide §16.A's preview-channel example and its copy in the paper, and no schema, fixture, projector or bundle carried them (the projection's keys are `tendcf_service` and `tendcf_interlock`). They were renamed `tendcf_services` and `tendcf_edges` in those illustrations in the same change that recorded this decision. No schema bump is involved ([`plugin-layout.md`](plugin-layout.md) §6). (7) Consequence for [`change-process.md`](change-process.md): the compiler is no longer a separately pinned input, the two-repository bump order and the compiler's `tendcf.lock` go away; site inputs keep `site.lock`. Moving `bin/` to `tools/` and the plugin split wait until open PRs #41 and #42 merge. Amends D12, D21, §1, §3, §5 and Step 3. | +| D46 | Repository layout and compiler plugins | **Operator, 2026-11-13.** (1) `frdminc/nix2cf` folds into tendcf with full history, first under `compiler/`, and is archived once the move is done. (2) The repository takes the plugin-type layout of [`plugin-layout.md`](plugin-layout.md): `schema/`, `examples/`, `policy/`, `overlay/`, `deploy/` unchanged; `tools/` (was `bin/`); `compiler/` (merge, conflict check, render); `plugins/frontends/` (`yaml`, `nix`, later `guix`), `plugins/supervisors/` (`launchd`, `systemd`, `runit`), `plugins/importers/` (`ansible`, `salt`); `contrib/` for in-tree third-party plugins. One Python package. (3) Plugins are found through the entry-point groups `tendcf.frontends`, `tendcf.supervisors` and `tendcf.importers`, so a third-party plugin can also live out of tree as a pip package. (4) Plugins are compile-time only: none runs on a device, and everything one produces passes the schema lint and the projector, so a plugin cannot bypass consent. A frontend is a language a site keeps authoring in; an importer is a one-shot migration into the Site Model. (5) The compiler's command becomes `tendcf compile` (`nix2cf buildfile` a short-lived alias); `nix2cf` names the Nix frontend (D12). (6) The keys `nix2cf_services` and `nix2cf_edges` were only ever illustrative: they occurred in guide §16.A's preview-channel example and its copy in the paper, and no schema, fixture, projector or bundle carried them (the projection's keys are `tendcf_service` and `tendcf_interlock`). In the same change that recorded this decision they were renamed `preview_services` and `preview_edges` (illustrative only, chosen so they cannot be mistaken for the `tendcf_services` bundle or the real projected keys `tendcf_service`/`tendcf_interlock`). No schema change is involved ([`plugin-layout.md`](plugin-layout.md) §6). (7) Consequence for [`change-process.md`](change-process.md): the compiler is no longer a separately pinned input, the two-repository bump order and the compiler's `tendcf.lock` go away; site inputs keep `site.lock`. Moving `bin/` to `tools/` and the plugin split wait until open PRs #41 and #42 merge. Amends D12, D21, §1, §3, §5 and Step 3. | Silence = proceed from Step 0. Objections amend this register. diff --git a/docs/architecture/plugin-layout.md b/docs/architecture/plugin-layout.md index f2e8e87..f70d202 100644 --- a/docs/architecture/plugin-layout.md +++ b/docs/architecture/plugin-layout.md @@ -130,19 +130,18 @@ plugin has no other path to a device. to 2026-11-13 the name meant the whole compiler; older documents and handoffs use it that way and are not rewritten. 3. **`guix`** is the planned Guix frontend, `plugins/frontends/guix/`. -4. **The illustrative `tendcf_services` / `tendcf_edges` keys.** Until - 2026-11-13 these were spelled `nix2cf_services` and `nix2cf_edges`. - Checked that day: they occurred only in the illustrative, - preview-channel `host_specific.json` of guide §16.A, its copy in the - paper, and review notes about it; the guide itself says that file does - not load as Augments. No schema, fixture, projector or bundle carried - them. They were renamed in those two illustrations in the same change - as this file, and no schema bump is involved (the goal file stays at - `schema_version` 5). They are still not the projection's keys, which - are `tendcf_service` and `tendcf_interlock`; `tendcf_services` is also - the name of the real generic bundle (`policy/tendcf_services.cf`), - which reads `tendcf_service`. Dated review and reconciliation notes - keep the old spelling. +4. **The illustrative `preview_services` / `preview_edges` keys.** The + example's keys were `nix2cf_services` and `nix2cf_edges`. Checked on + 2026-11-13: they occurred only in the illustrative, preview-channel + `host_specific.json` of guide §16.A, its copy in the paper, and review + notes about it; the guide itself says that file does not load as + Augments. No schema, fixture, projector or bundle carried them. On + 2026-11-13 they were renamed `preview_services` and `preview_edges`, in + the same change as this file: illustrative only, chosen so they cannot + be mistaken for the `tendcf_services` bundle or the real projected + keys `tendcf_service` and `tendcf_interlock`. No schema change is + involved (the goal file stays at `schema_version` 5). Dated review and + reconciliation notes keep the old spelling. ## 7. Not decided here diff --git a/docs/paper/tendcf-architecture-guide.md b/docs/paper/tendcf-architecture-guide.md index 6de2953..775a26c 100644 --- a/docs/paper/tendcf-architecture-guide.md +++ b/docs/paper/tendcf-architecture-guide.md @@ -912,23 +912,24 @@ host. edge from that match: This illustration is **preview-channel material and does not load as -Augments.** Both of its top-level keys — `data` and `tendcf_edges` — are +Augments.** Both of its top-level keys — `data` and `preview_edges` — are skipped by CFEngine 3.27.1, so as written this file injects nothing. The shape the agent actually consumes is `{"vars": {…}}` and nothing else; the projection onto it runs device-side, after approval. The nesting below is kept because it shows the *derivation* legibly, which is what this walkthrough is for — do not copy it onto a device and expect an effect. -Its key names `tendcf_services` and `tendcf_edges` are illustrative too -(renamed from `nix2cf_*` on 2026-11-13; no schema change): the -projection's real `vars` keys are `tendcf_service` and -`tendcf_interlock` (§16.C), and the goal file carries no edges. +Its keys were `nix2cf_*` until 2026-11-13, when they were renamed +`preview_services` and `preview_edges`: illustrative only, chosen so they +cannot be mistaken for the `tendcf_services` bundle or the projection's +real `vars` keys `tendcf_service` and `tendcf_interlock` (§16.C). No +schema change was involved, and the goal file carries no edges. ```jsonc // host_specific.json — ILLUSTRATIVE ONLY. Preview channel; does NOT load -// as Augments (top-level `data` and `tendcf_edges` are both skipped). +// as Augments (top-level `data` and `preview_edges` are both skipped). { "data": { - "tendcf_services": { + "preview_services": { "caddy": { "service_policy": "start", "launchd_label": "com.djbclark.caddy", @@ -947,7 +948,7 @@ projection's real `vars` keys are `tendcf_service` and } } }, - "tendcf_edges": [ + "preview_edges": [ { "from": "litellm-proxy", "to": "caddy", diff --git a/docs/paper/tendcf-architecture-paper.md b/docs/paper/tendcf-architecture-paper.md index 508680d..6f81cdf 100644 --- a/docs/paper/tendcf-architecture-paper.md +++ b/docs/paper/tendcf-architecture-paper.md @@ -512,18 +512,19 @@ nothing in Example A except the YAML was produced mechanically. **One further warning, and it is the kind of thing this paper exists to surface.** The augments document shown in Example A is *pedagogically* right and *mechanically* wrong: its two top-level keys, `data` and -`tendcf_edges`, are both keys the agent silently skips. As written it +`preview_edges`, are both keys the agent silently skips. As written it injects nothing. We keep it because it shows the derivation legibly, which is what the example is for, and we flag it because an illustration that would not run is exactly the sort of claim a reader should be able to catch us on. The shape the agent actually consumes is `{"vars": {…}}` and nothing else, reached by the device-side projection of §2.5; that shape, and a generic bundle that reads it, both exist and are shown at the end of -this section. The illustration's key names, `tendcf_services` and -`tendcf_edges`, are illustrative as well (renamed from `nix2cf_*` on -2026-11-13, with no schema change); the projection's real keys are -`tendcf_service` and `tendcf_interlock`, and the goal file carries no -edges. +this section. The illustration's keys were `nix2cf_*` until 2026-11-13, +when they were renamed `preview_services` and `preview_edges`: +illustrative only, chosen so they cannot be mistaken for the +`tendcf_services` bundle or the projection's real keys `tendcf_service` +and `tendcf_interlock`. No schema change was involved, and the goal file +carries no edges. **Example A — an edge nobody wrote.** `caddy` and `litellm-proxy` are two records in the same `edge-http` bundle. `litellm-proxy` states only what it @@ -633,11 +634,11 @@ edge to carry its own provenance rather than appear as a bare ordering fact: ```jsonc // host_specific.json (host: mac) — ILLUSTRATIVE ONLY, hand-authored. -// Does NOT load as Augments: top-level `data` and `tendcf_edges` are both +// Does NOT load as Augments: top-level `data` and `preview_edges` are both // skipped by the agent. Shown for the derivation, not to be copied. { "data": { - "tendcf_services": { + "preview_services": { "caddy": { "service_policy": "start", "launchd_label": "com.djbclark.caddy", @@ -653,7 +654,7 @@ edge to carry its own provenance rather than appear as a bare ordering fact: } } }, - "tendcf_edges": [ + "preview_edges": [ { "from": "litellm-proxy", "to": "caddy", @@ -696,7 +697,7 @@ loaded: ``` -`tendcf_edges[0]` is what §4.1 means by mandatory attribution: `origin` says +`preview_edges[0]` is what §4.1 means by mandatory attribution: `origin` says this edge was never authored, `rule` names the mechanism that produced it, and `source` points at the exact `requires` entry responsible. If this edge turns out to be spurious, the question it presents is "does `rule` correctly From b47960b854f75b7229c88749f457d6115e94d890 Mon Sep 17 00:00:00 2001 From: Daniel JB Clark Date: Fri, 13 Nov 2026 06:36:40 -0500 Subject: [PATCH 4/9] docs: apply D46 review fixes and djbclark's 2026-11-13 decisions Both frontier reviews (Astra gpt-6-astra high, Fable claude-fable-5-2 high) said merge after fixes. This records djbclark's decisions on them: plugins are trusted build-time code the operator allows, with gates and signing run apart from plugins; a plugin loads only if site-private names it, in-tree only for releases until #47; supervisor renderers are in tree only; one tendcf rev is the pinned unit; no shims (tendcf compile with no alias, package renamed tendcf, bin/ -> tools/ with no wrappers); current schema_version only until a non-operator device enrolls; nix2cf archived as soon as #45 merges. Also: device-boundary import rules (M1), site.lock wording (M6), the compile/--host mapping (L2), the living-file list for the restructure (L4), D15 pointer (L5), the paper's Nix passthrough exception and the systemd adapter status (A4, A5), the device receives the full goal file (A3), fixture counts 159 (125/6/28). Co-Authored-By: Claude Opus 5.5 --- docs/architecture/GLOSSARY.md | 14 +- .../architecture-DEFINITIVE-v3.md | 39 ++- docs/architecture/change-process.md | 82 +++-- docs/architecture/linux-targets.md | 9 +- docs/architecture/plugin-layout.md | 284 ++++++++++++++---- docs/paper/tendcf-architecture-guide.md | 82 +++-- docs/paper/tendcf-architecture-paper.md | 50 +-- 7 files changed, 420 insertions(+), 140 deletions(-) diff --git a/docs/architecture/GLOSSARY.md b/docs/architecture/GLOSSARY.md index aba04e7..e06c15e 100644 --- a/docs/architecture/GLOSSARY.md +++ b/docs/architecture/GLOSSARY.md @@ -133,9 +133,17 @@ points in §12. same JSON as the YAML frontend (map D12). Not built yet. - **Before 2026-11-13 — the whole compiler**, and the separate repository `frdminc/nix2cf` that held it (`nix2cf buildfile`). That compiler is - being folded into tendcf under `compiler/`, its command becomes - `tendcf compile`, and the repository is archived once the move is done. - Older documents and handoffs use the old sense and are not rewritten. + being folded into tendcf under `compiler/`, and the repository is + archived as soon as that import merges. Older documents and handoffs use + the old sense and are not rewritten. +- **Which sense holds when.** Until the restructure + (`plugin-layout.md` §5, step 4) the only thing that ships under the name + is still the old compiler: the package `nix2cf` and its command + `nix2cf buildfile`, first in `frdminc/nix2cf`, then under `compiler/`. + The restructure replaces that command with `tendcf compile`, with no + alias, and renames the package `tendcf`; from then on `nix2cf` means + only the Nix frontend. The "Current" sense above is the decided + meaning, for a plugin not yet built. Authority: `architecture-DEFINITIVE-v3.md` D46 (amending D12 and D21); layout and order of the move in `plugin-layout.md`. diff --git a/docs/architecture/architecture-DEFINITIVE-v3.md b/docs/architecture/architecture-DEFINITIVE-v3.md index 6ec4fe1..76dc648 100644 --- a/docs/architecture/architecture-DEFINITIVE-v3.md +++ b/docs/architecture/architecture-DEFINITIVE-v3.md @@ -142,7 +142,7 @@ checkouts that must share one tag. | Layer | Holds | Visibility | | --- | --- | --- | | **tendcf** | Engine, **schemas/types**, generic adapters, default advisor prompt, the compiler (`compiler/`) and its plugins (`plugins/`, D46) | Public product | -| ~~**nix2cf**~~ | ~~Compiler tool only (Site Model → CFEngine JSON)~~ Folded into tendcf (D46). `nix2cf` now names the Nix frontend plugin | Repository archived once the move is done | +| ~~**nix2cf**~~ | ~~Compiler tool only (Site Model → CFEngine JSON)~~ Folded into tendcf (D46). `nix2cf` now names the Nix frontend plugin | Repository archived as soon as the import ([#45](https://github.com/frdminc/tendcf/pull/45)) merges | | **site-shared** | Reusable recipes | Public or shared | | **site-private** | Inventory, allocations, secret *names*, trust policy, extra prompt, lockfile | Private | | **foreign shared** | Other sites' recipes | Read-only inputs | @@ -280,12 +280,18 @@ render-stage decision; corrected here on 2026-11-13.) **Amended 2026-11-13 (D46).** The compiler was built as the separate repository `frdminc/nix2cf` (`nix2cf buildfile`, Step 3). It moves into tendcf with its history, first under `compiler/`, and its command -becomes `tendcf compile`; `nix2cf buildfile` stays as a short-lived +becomes `tendcf compile`, which replaces `nix2cf buildfile` with no alias. The name `nix2cf` moves to the Nix module frontend (D12). -Frontends, supervisor renderers and importers are plugins found through -Python entry points; they run at compile time only, and their output -passes the same schema lint and projector, so a plugin cannot bypass -consent. Layout, plugin kinds and the order of the move: +Frontends, supervisor renderers and importers are plugins, all compile +time only; frontends and importers are found through the public Python +entry-point groups `tendcf.frontends` and `tendcf.importers`, supervisor +renderers are in tree only. Plugins are trusted build-time code the +operator explicitly allows, not sandboxed: a plugin cannot make a device +accept anything that device's own diff does not show, and cannot emit a +shape the schema forbids, but it can make the goal file say something +other than what the Site Model says. Release gates and signing therefore +run apart from plugin code. Layout, plugin kinds, the trust and +activation rules and the order of the move: [`plugin-layout.md`](plugin-layout.md). CFEngine’s [Masterfiles Policy Framework](https://github.com/cfengine/masterfiles) @@ -716,6 +722,13 @@ Cost, accepted: the compiler carries multi-version render ability for a window and tracks per-host versions. The price is paid in one place rather than as device-side leniency on every device. +**Amended 2026-11-13 (D46).** There are no devices to strand yet, so the +compiler renders only the current `schema_version`, and its older-version +rendering is removed when the compiler is restructured +([`plugin-layout.md`](plugin-layout.md) §4, §5). This section's design +stands and applies once the first device that is not an operator host +enrolls; the multi-version render returns with it. + ### 9.7 Coverage travels in the goal file (E1 §5.7) The goal-file schema **MUST include the per-domain coverage @@ -1141,7 +1154,7 @@ Each step leaves a coherent system. Not a schedule. | 0 | Schemas in **tendcf** (`schema/`, `examples/`, `bin/schema_lint.py` — provides/requires, interlocks, comprehensive, report-row). Remaining: peer_actions, trust-policy shape, generic unit-writers, lookup stub, YAML canonicalize. Transcribe reality (`not-yet-migrated` is the correct day-one state). | | 1 | macOS services adapter. Dry-run default. No nix-darwin. **Started** (`policy/tendcf_services.cf`, `222da45`: launchd present/absent and interlock refusal, verified against a real `cf-agent`; package/file promises, environment rendering, the unit-writer detector still open). 2026-10-29: the satisfied-interlock path had never opened (verdict class fixed in pass 1 before the check ran); the check moved to `tendcf_interlocks`, sequenced before `tendcf_services`, and `tests/policy/run.sh` proves both paths on Linux CI and the Mac. The CFEngine it was verified against is upstream plus the patch overlay (`overlay/`, [`upstream-overlay.md`](upstream-overlay.md)); Linux CI builds the full fork ([`linux-targets.md`](linux-targets.md)). | | 2 | Android under the Site Model; Termux types; agent owns JSONL+SQLite. **Deferred** (operator, 2026-10-29): Linux comes first, see [`linux-targets.md`](linux-targets.md). Not started as code: the schema already carries a `termux` supervisor flavour (`services.schema.json`) and the `android-termux-services` domain fixture, but no adapter bundle, no agent, no JSONL/SQLite ownership. `frdminc/ShizukuTendCF` is the R4 helper (a tool fork), not this step. | -| 3 | Compiler (`tendcf compile`; ~~nix2cf~~): `buildfile` first, conflict, extra-entry, then inference (needs step 1 and a Linux systemd adapter; Android is deferred). **Unblocked** on 2026-10-29 by the operator's decision recorded on [#4](https://github.com/frdminc/tendcf/issues/4); `frdminc/nix2cf` has `buildfile` only. **Amended 2026-11-13 (D46):** the compiler moves into tendcf under `compiler/` and is split into plugins ([`plugin-layout.md`](plugin-layout.md)); ~~its `tendcf.lock` is specified in [`change-process.md`](change-process.md) §2~~ with schema and compiler in one repository there is no `tendcf.lock` (it was specified, never built). | +| 3 | Compiler (`tendcf compile`; ~~nix2cf~~): `buildfile` first, conflict, extra-entry, then inference (needs step 1 and a Linux systemd adapter; Android is deferred). **Unblocked** on 2026-10-29 by the operator's decision recorded on [#4](https://github.com/frdminc/tendcf/issues/4); `frdminc/nix2cf` has one compilation command, `buildfile`, plus the `eval-nix` evaluation hook. **Amended 2026-11-13 (D46):** the compiler moves into tendcf under `compiler/` and is split into plugins ([`plugin-layout.md`](plugin-layout.md)); ~~its `tendcf.lock` is specified in [`change-process.md`](change-process.md) §2~~ with schema and compiler in one repository there is no `tendcf.lock` (it was specified, never built). | | 4 | Linux reference path on a stock distro. | | 5 | First real Linux host; prove roles are data. | | 6 | Signed releases, push-only; goal-file render + diff + on-device validator (§9). Operator hosts. The lockfile, release, rollback and CI-gate design is [`change-process.md`](change-process.md) (#2); the code is not written. On Linux and macOS the on-device validator is `tendcf-validate`, built after schema v5 lands and before any further host is enforced; how much of this step's validator its first build carries is open (D45, §12). | @@ -1194,7 +1207,7 @@ Older “superseded by” archaeology lives in | D12 | Authoring frontend | **Amended 2026-10-29.** Two equal frontends, Nix modules and YAML, rendering to byte-identical goal files; a Nix-authored site runs on hosts without Nix. Existing NixOS/nix-darwin configurations are used verbatim (passthrough). JSON remains the wire format. [`linux-targets.md`](linux-targets.md) §5. **Amended 2026-11-13 (D46):** the Nix frontend is the compiler plugin `plugins/frontends/nix/` and carries the name `nix2cf`; the YAML frontend is `plugins/frontends/yaml/` (built in); a Guix frontend may follow. Not built yet ([`plugin-layout.md`](plugin-layout.md)). | | D13 | Service owner | CFEngine promises everywhere; mise toolchains only. | | D14 | Deployment shape | Git-synced policy; `cf-serverd` on every host; push and pull are one mechanism. | -| D15 | Compile target | CFEngine Augments, not freehand `.cf`. | +| D15 | Compile target | CFEngine Augments, not freehand `.cf`. (By way of the per-host goal file, which the device projects to Augments after approval: D43, D44, §5, §9.) | | D16 | Ordering | No Puppet catalog. Inference + fixpoint + authored `depends_on`. Interlocks. Default-on comprehensiveness. | | D17 | ncf | Vendor generic methods as a reference corpus; strip Rudder reporting. | | D18 | Local record | ~~JSONL is the durable capture; SQLite in tendcf-agent is the index. On Android the agent owns both in app-private storage.~~ **Amended 2026-11-02 by D45:** JSONL is the durable capture; SQLite is the index, rebuildable from it. On Android tendcf-agent owns both in app-private storage. On Linux and macOS there is no agent, and the index's owner there is open (§8, §12). | @@ -1224,7 +1237,7 @@ Older “superseded by” archaeology lives in | D43 | ChangePlan model | ~~Model A: a closed `capability` vocabulary per operation, executor refuses effects outside the declared set.~~ **Superseded 2026-08-15 by Model B** (synthesis finding E1, adjudicated in `e1-adjudication-xhigh-2026-08-15.md`, which is final and supersedes `e1-adjudication-2026-08-15.md`): the wire artifact is the complete canonical per-host **goal file**; the ChangePlan is the **diff** against the device’s approved baseline; the executor is a **validator** over that diff, not an interpreter. Rationale: CFEngine has no runtime capability confinement, so Model A’s executor requires an interpreter plus a vocabulary-to-policy correspondence proof that does not exist and is not in budget; and coverage closes by construction when compiler and validator share one schema. The vocabulary, its versioning, and its skew policy are dropped. See §9. | | D44 | Goal-file schema family | The consent artifact is three paired schemas (goal file, goal diff, approval record) with canonical-byte fixtures. Binding sub-decisions, all from E1 §5 and normative in §9: diff computed **device-side**, full file on the wire, compiler diff is preview (§9.1 / E1 §5.1); RFC 8785 canonical JSON, no defaults, no empty collections, refuse-never-normalize, entry-granular structural hunks (§9.2 / E1 §5.2); accept is all-or-nothing, refusal annotated, no dependency groups (§9.3 / E1 §5.3); first adoption and schema migration are bounded **baseline ceremonies**, migration must diff to empty (§9.4 / E1 §5.4); hunk attribution is **excluded from the format** — impossible device-side — and replaced by an `explain-hunk` query (§9.5 / E1 §5.5); `schema_version` fails closed on unknown kinds, strandedness prevented at the compiler, two-phase bumps (§9.6 / E1 §5.6); per-domain coverage travels **in** the goal file (§9.7 / E1 §5.7). §14.2’s pre-build review applies to this family. | | D45 | Device-side component per platform | **Operator, 2026-11-02.** (1) tendcf-agent is **Android-only**, written in Kotlin. A Linux or macOS host runs CFEngine plus "a small validate-and-project command" (the ruling's words; the name **`tendcf-validate`** was chosen by the amending session on 2026-11-02 and may change without reopening the ruling), run before cf-agent reads `host_specific.json`; no daemon. (2) `tendcf-validate` is Python, the promotion of `bin/projector.py`: one implementation, so no byte-identity drift on those platforms. It needs `python3` on each host (Debian ships it, NixOS through the derivation, macOS bundled or the Command Line Tools). The Android agent's Kotlin projector is a port held to the same golden bytes in CI (residue R22), not a second normative source. (3) Built after schema v5 lands, before any further host is enforced. **Supersedes** every reading that ran the validator, the projector or the SQLite index inside tendcf-agent on every host: the §1 diagram's "CFEngine + tendcf-agent" per device, §8's index owner and D18 as written, §9's "run inside tendcf-agent" and "the agent binary" as the projector's call site, `goal-file-schema-reconciliation-2026-08-15.md` §0 item 7, §9 and §13 ("CI invokes the agent's own projector"), and `projector-reconciliation-2026-08-16.md` §9 and R22 ("tendcf-agent must reproduce" the reference). Not decided by it, and open for the operator: §12's list. | -| D46 | Repository layout and compiler plugins | **Operator, 2026-11-13.** (1) `frdminc/nix2cf` folds into tendcf with full history, first under `compiler/`, and is archived once the move is done. (2) The repository takes the plugin-type layout of [`plugin-layout.md`](plugin-layout.md): `schema/`, `examples/`, `policy/`, `overlay/`, `deploy/` unchanged; `tools/` (was `bin/`); `compiler/` (merge, conflict check, render); `plugins/frontends/` (`yaml`, `nix`, later `guix`), `plugins/supervisors/` (`launchd`, `systemd`, `runit`), `plugins/importers/` (`ansible`, `salt`); `contrib/` for in-tree third-party plugins. One Python package. (3) Plugins are found through the entry-point groups `tendcf.frontends`, `tendcf.supervisors` and `tendcf.importers`, so a third-party plugin can also live out of tree as a pip package. (4) Plugins are compile-time only: none runs on a device, and everything one produces passes the schema lint and the projector, so a plugin cannot bypass consent. A frontend is a language a site keeps authoring in; an importer is a one-shot migration into the Site Model. (5) The compiler's command becomes `tendcf compile` (`nix2cf buildfile` a short-lived alias); `nix2cf` names the Nix frontend (D12). (6) The keys `nix2cf_services` and `nix2cf_edges` were only ever illustrative: they occurred in guide §16.A's preview-channel example and its copy in the paper, and no schema, fixture, projector or bundle carried them (the projection's keys are `tendcf_service` and `tendcf_interlock`). In the same change that recorded this decision they were renamed `preview_services` and `preview_edges` (illustrative only, chosen so they cannot be mistaken for the `tendcf_services` bundle or the real projected keys `tendcf_service`/`tendcf_interlock`). No schema change is involved ([`plugin-layout.md`](plugin-layout.md) §6). (7) Consequence for [`change-process.md`](change-process.md): the compiler is no longer a separately pinned input, the two-repository bump order and the compiler's `tendcf.lock` go away; site inputs keep `site.lock`. Moving `bin/` to `tools/` and the plugin split wait until open PRs #41 and #42 merge. Amends D12, D21, §1, §3, §5 and Step 3. | +| D46 | Repository layout and compiler plugins | **Operator, 2026-11-13.** (1) `frdminc/nix2cf` folds into tendcf with full history, first under `compiler/` ([#45](https://github.com/frdminc/tendcf/pull/45)), and is archived as soon as that import merges. (2) The repository takes the plugin-type layout of [`plugin-layout.md`](plugin-layout.md): `schema/`, `examples/`, `policy/`, `overlay/`, `deploy/` unchanged; `tools/` (was `bin/`); `compiler/` (merge, conflict check, render); `plugins/frontends/` (`yaml`, `nix`, later `guix`, an authoring frontend), `plugins/supervisors/` (`launchd`, `systemd`, `runit`), `plugins/importers/` (`ansible`, `salt`); `contrib/` for in-tree third-party frontends and importers. It becomes one Python package, import name `tendcf`. (3) Frontends and importers are found through the public entry-point groups `tendcf.frontends` and `tendcf.importers`, so a third-party one can also live out of tree as a pip package. Supervisor renderers are in tree only; there is no public `tendcf.supervisors` group, because the goal file's `unit` is a closed `oneOf` and an out-of-tree renderer could only shadow a built-in. (4) Plugins are compile-time only: none runs on a device. The device artifact (`tendcf-validate`, D45) is built from the validator's own source alone, never imports `compiler/`, `plugins/` or `contrib/` and never calls entry-point discovery; the schema lint and the projector never import the compiler; a CI import check is to enforce both (planned). Plugins are trusted build-time code that the operator explicitly allows, not sandboxed: a plugin cannot make a device accept anything that device's own diff does not show, and cannot emit a shape the schema forbids, but it can make the goal file say something other than what the Site Model says, and it sees the site's private data. Release gates (schema lint, projector, preview diff) and signing run in a separate environment with no plugin installed; the compile step holds no signing key. A compile-side preview diff or `explain-hunk` (D44) is compiler output and inherits the plugins' trust. `contrib/`'s lower bar waives support and maintenance promises, never the standing review rule. Out-of-tree trust establishment and plugin management: [#47](https://github.com/frdminc/tendcf/issues/47). (5) Activation: a plugin loads only if site-private names it (name → distribution, version, hash); an installed but unnamed plugin is ignored; a duplicate name in a group is a compile error; built-in names are reserved; until out-of-tree pinning exists (#47) a release build loads in-tree plugins only; the release's `targets.json` `custom` records the plugin set. (6) No users and no backward compatibility yet, so no shims: the compiler's command becomes `tendcf compile`, which replaces `nix2cf buildfile` with no alias (it compiles every host; `--host` renders one); `eval-nix` moves to the Nix frontend; `bin/` → `tools/` leaves no symlink or wrapper and rewrites every reference in living files; `nix2cf` names the Nix frontend (D12). The compiler renders only the current `schema_version`; older-version rendering is removed in the restructure, and §9.6's ceiling design applies once the first device that is not an operator host enrolls. (7) The keys `nix2cf_services` and `nix2cf_edges` were only ever illustrative: they occurred in guide §16.A's preview-channel example and its copy in the paper, and no schema, fixture, projector or bundle carried them (the projection's keys are `tendcf_service` and `tendcf_interlock`). In the same change that recorded this decision they were renamed `preview_services` and `preview_edges` (illustrative only, chosen so they cannot be mistaken for the `tendcf_services` bundle or the real projected keys `tendcf_service`/`tendcf_interlock`). No schema change is involved ([`plugin-layout.md`](plugin-layout.md) §6). (8) Consequence for [`change-process.md`](change-process.md): the compiler is no longer a separately pinned input, the two-repository bump order and the compiler's `tendcf.lock` go away; `site.lock` loses only its `nix2cf` entry. One tendcf rev is the unit a site pins, so a compiler fix brings the `schema/` and `policy/` commits before it; accepted, with no maintenance-branch machinery, revisited when the first device that is not an operator host enrolls. Moving `bin/` to `tools/` and the plugin split wait until open PRs #41 and #42 merge. Amends D12, D15 (a pointer), D21, §1, §3, §5, §9.6 (when it applies) and Step 3. | Silence = proceed from Step 0. Objections amend this register. @@ -1317,9 +1330,11 @@ with full mesh) is **D38**. wins on conflict. - `schema/`, `examples/`, `bin/schema_lint.py` — Site Model contract. - `frdminc/nix2cf` — compiler tool (Step 3), until it is folded into this - repository (D46) and archived. Consumes this contract. -- `plugin-layout.md` — the repository's plugin-type layout, the three - plugin entry-point groups, and the order of the move (D46). + repository (D46); archived as soon as that import (#45) merges. + Consumes this contract. +- `plugin-layout.md` — the repository's plugin-type layout, the plugin + kinds and their two public entry-point groups, the plugin trust and + activation rules, and the order of the move (D46). - `tendcf-validate-proposal-2026-11-03.md` — a proposal, deciding nothing: options and a recommendation for each of §12's open D45 items, a minimal first build, and an operator picklist. diff --git a/docs/architecture/change-process.md b/docs/architecture/change-process.md index 6708e58..6186a20 100644 --- a/docs/architecture/change-process.md +++ b/docs/architecture/change-process.md @@ -10,15 +10,20 @@ leaves as "a lockfile (flake-style inputs)" and "the signed release is the deploy artifact" with a mechanism concrete enough to build. **Amended 2026-11-13 (map D46).** The compiler, built as the separate -repository `frdminc/nix2cf`, is folded into tendcf with its history and -`frdminc/nix2cf` is archived ([`plugin-layout.md`](plugin-layout.md)). -The compiler is therefore no longer a separately pinned input: schema and -compiler change in the same tendcf commit, and the two-repository bump -order and the compiler's own `tendcf.lock` that this document used to -specify are gone. Site inputs (site-shared, foreign sites, tool forks) -keep `site.lock` exactly as below. Until the move lands, the compiler's -CI still checks out tendcf's default branch beside it (it never had the -`tendcf.lock` specified here). +repository `frdminc/nix2cf`, is folded into tendcf with its history +([#45](https://github.com/frdminc/tendcf/pull/45)), and `frdminc/nix2cf` +is archived as soon as that import merges +([`plugin-layout.md`](plugin-layout.md)). The compiler is therefore no +longer a separately pinned input: schema and compiler change in the same +tendcf commit, and the two-repository bump order and the compiler's own +`tendcf.lock` that this document used to specify are gone. `site.lock` +loses only its `nix2cf` entry: it still pins tendcf, site-shared and +foreign sites, and tool forks stay as rule 5 has them. One tendcf rev is +now the unit a site pins (§8 item 5). Compiler plugins are trusted +build-time code; which ones a release loads, and the rule that the gates +and signing run apart from them, are in §3 and §6. Until the move lands, +the compiler's CI still checks out tendcf's default branch beside it (it +never had the `tendcf.lock` specified here). > The requirement, verbatim from #2: the Site Model → signed-release → > deploy path must be **much faster and much more atomic** than cutting a @@ -62,7 +67,7 @@ device seven releases behind still computes one true diff (guide §7). | Layer (map §3) | Repository today | What a release takes from it | | --- | --- | --- | -| tendcf | `frdminc/tendcf` | schemas, `bin/projector.py`, generic bundles under `policy/`, the upstream overlay (`overlay/`, see [`upstream-overlay.md`](upstream-overlay.md)), and the compiler with its plugins (Step 3; `compiler/` and `plugins/` once the move of map D46 lands, today still `frdminc/nix2cf` with `buildfile` only) | +| tendcf | `frdminc/tendcf` | schemas, `bin/projector.py`, generic bundles under `policy/`, the upstream overlay (`overlay/`, see [`upstream-overlay.md`](upstream-overlay.md)), and the compiler with its plugins (Step 3; `compiler/` and `plugins/` once the move of map D46 lands, today still `frdminc/nix2cf`, whose one compilation command is `buildfile`) | | site-shared | not yet created | recipes | | foreign site-shared | none yet | recipes, namespaced | | site-private | `~/ops/site-private` (today: ops repo; the tendcf inventory does not exist yet) | inventory, allocations, secret *names*, trust policy, the lockfile | @@ -121,7 +126,13 @@ Rules for the file, each with the reason it is a rule: pinning the tendcf rev its golden tests run against.~~ **Amended 2026-11-13:** there is no compiler lock. The compiler is in tendcf, so its goldens run against the schemas of the same commit, and a schema change -lands together with the compiler support it needs (§4, §5). +lands together with the compiler support it needs (§4, §5). This holds +for the current `schema_version`, the only one the compiler renders (map +D46): older-version rendering is removed in the restructure. Map §9.6's +ceiling design, which needs the compiler to render older versions for +hosts below the ceiling, applies once the first device that is not an +operator host enrolls; how that older output is validated when schema and +compiler share a commit is decided then. A `flake.lock` was considered as *the* format and rejected as the requirement (§8.5); a converter from `site.lock` to flake inputs is a @@ -135,7 +146,7 @@ A release is one immutable, numbered artifact set: | --- | --- | --- | | `targets/.goal.json` | that host's complete canonical goal file (map §9) | targets role | | `targets/policy-tree.tar` | the generic bundles and templates at the tendcf rev, whose digest is the privileged region map §9.8 names | targets role | -| `targets.json` | the digest of every target, plus `custom`: release number, `site.lock` SHA-256, the site-private commit, the compiler and projector versions | targets role | +| `targets.json` | the digest of every target, plus `custom`: release number, `site.lock` SHA-256, the site-private commit, the tendcf rev (compiler and projector; one rev since map D46) and the compiler plugin set the compile loaded (name, distribution, version, hash; map D46) | targets role | | `snapshot.json`, `timestamp.json` | TUF metadata binding the set and its freshness | snapshot / timestamp roles | Three properties follow: @@ -144,7 +155,12 @@ Three properties follow: `version`; it only goes up (high-water mark on every client, map §9.11). The content is reproducible from the site-private commit, and the lockfile hash in `targets.json` is how a reader gets from a release - back to its inputs without trusting a branch name. + back to its inputs without trusting a branch name. That needs every + compiler plugin to come from a pinned input: until out-of-tree pinning + exists ([#47](https://github.com/frdminc/tendcf/issues/47)), a release + build loads only in-tree plugins named by site-private, so the tendcf + pin covers them, and plugins installed on the builder but not named + are ignored (map D46). 2. **Per-host targets, one release.** One release carries every host. Hosts that did not change get a byte-identical goal file and compute an empty diff; nothing is applied. This is why "a release" and "a change" @@ -210,7 +226,9 @@ Each repository's CI runs against pinned inputs, never against a sibling's lint, the overlay self-test and, once the compiler has moved in (map D46), the compiler's goldens against the same commit's schemas. A schema change that breaks the compiler fails tendcf's own CI, in the commit - that makes it. + that makes it. That is a statement about the current `schema_version`, + the only one the compiler renders until map §9.6's ceiling design + applies (§2). - ~~**nix2cf** pins tendcf (`tendcf.lock`). A schema change in tendcf cannot break nix2cf's CI until nix2cf chooses to bump.~~ Gone with map D46. @@ -229,7 +247,7 @@ one consistent set. | Repo | Gate | Exists today | | --- | --- | --- | -| tendcf | `bin/schema_lint.py` incl. the 92 broken fixtures and projector goldens | yes | +| tendcf | `bin/schema_lint.py` incl. the 159 broken fixtures (125 schema-level, 6 byte-class, 28 projection; counted 2026-11-13) and projector goldens | yes | | tendcf | `bin/xref_lint.py` | yes | | tendcf | `bin/overlay.py self-test` and `lint` (patch overlay applies; drift fails loudly) | yes, this change | | tendcf | overlay drift against upstream (`check --against upstream/master`) | scheduled / manual job, this change | @@ -242,8 +260,10 @@ the review artifact): 1. `site.lock` is canonical bytes; every input's `rev` is reachable and its fetched `tree` matches. Refuse otherwise: a pin that cannot be verified is not a pin. -2. Compile every host. Conflict-as-error (map §3): the same identity from - two peer inputs without a private bind refuses the whole release. +2. Compile every host, loading only the compiler plugins site-private + names (in-tree only until #47; map D46). Conflict-as-error (map §3): + the same identity from two peer inputs without a private bind refuses + the whole release, and so does a duplicate plugin name within a group. 3. Every goal file passes `bin/schema_lint.py` at the pinned tendcf rev and is canonical bytes. 4. Project every goal file with the pinned projector; the projection passes @@ -268,6 +288,16 @@ host; `root` stays offline (map §9.11, residue §14.4). A pull request can run every gate but 6, so review happens over the exact diffs a merge would ship. +**Gates 3 to 6 run apart from plugin code** (map D46). Compiler plugins +are trusted build-time code, not sandboxed, so the schema lint, the +projector, the preview diff and signing run in a separate environment in +which no plugin distribution is installed, and the compile step (gate 2) +holds no signing key. The gates bound the shape of what a plugin renders, +not its faithfulness to the Site Model: gate 5's preview diff shows what +the plugins rendered, and a preview diff or `explain-hunk` attribution +produced on the compile side (map D44) is compiler output that inherits +the plugins' trust ([`plugin-layout.md`](plugin-layout.md) §4). + ## 7. Latency target (proposed; needs the operator) #2 says no number has been picked. Proposal, to be confirmed or replaced: @@ -296,9 +326,10 @@ moved on since. Removals revert as `absent → present` replace hunks, which is why map §9.8 insists removals are states, not events: a rollback is just another diff. -2. **Partial rollback is the same mechanism**: revert one input's pin or one - host's entries, cut a release. There is no "roll back repo X" because no - device ever depended on repo X directly. +2. **Partial rollback is the same mechanism**: move one input's pin back + (the tendcf pin moves schemas, policy tree and compiler together, item + 5) or revert one host's entries, cut a release. There is no "roll back + repo X" because no device ever depended on repo X directly. 3. **A device that never took the bad release** computes the diff from its own baseline to M and finds it empty or small. Nothing special happens. 4. **A bad release that must be stopped before devices apply it** uses the @@ -309,6 +340,13 @@ moved on since. own schedule, or not at all; the fleet is fixed by the site-private pin going back, which is step 1. This is the whole point: the input repositories' history is never load-bearing for what devices run. + Since map D46 one tendcf rev is the unit a site pins: taking or + reverting a compiler fix also takes or reverts every `schema/` and + `policy/` commit before it, and on a consented device a policy-tree + change is a privileged-region hunk (map §9.8). That coupling is + accepted. There is no maintenance-branch machinery for carrying a + compiler fix alone; revisit when the first device that is not an + operator host enrolls. ## 9. Rejected alternatives @@ -322,7 +360,9 @@ moved on since. other people's repositories. A monorepo also cannot hold the tool forks. (Map D46, 2026-11-13, does merge two *public engine* repositories, tendcf and the compiler, which share an owner, a licence and reviewers; the - layers this item keeps apart stay apart.) + layers this item keeps apart stay apart. The cost is rollback + granularity: one tendcf rev pins compiler, schemas and policy tree + together, accepted in §8 item 5.) 3. **Git submodules in site-private.** A submodule pointer is a lockfile with worse properties: it records only `rev` (no `tree`, so a rewritten fork is undetectable until the fetch fails), it is bound to a checkout diff --git a/docs/architecture/linux-targets.md b/docs/architecture/linux-targets.md index 5783f5c..fa7349f 100644 --- a/docs/architecture/linux-targets.md +++ b/docs/architecture/linux-targets.md @@ -79,9 +79,12 @@ both first-class: working configuration is ever required. 2. **Nix as an authoring language.** The Site Model's types are also Nix module options (`mkOption`, `mkIf`, `mkDefault`). A site authored in Nix - evaluates (`nix eval --json --store dummy://`) to the same - schema-validated goal file as one authored in YAML, and the compiler - renders it for a launchd or systemd host with no Nix installed. The Nix + evaluates (`nix eval --json --store dummy://` in this plan; the + `eval-nix` hook that exists today runs + `nix-instantiate --eval --strict --json`) to Site Model JSON that + compiles to the same schema-validated goal file as one authored in + YAML, and the compiler renders it for a launchd or systemd host with no + Nix installed. The Nix frontend is the compiler plugin named `nix2cf` (map D12, D46). The person's Nix knowledge carries over; Nix the runtime is not needed on the target (D6 unchanged). diff --git a/docs/architecture/plugin-layout.md b/docs/architecture/plugin-layout.md index f70d202..54167a5 100644 --- a/docs/architecture/plugin-layout.md +++ b/docs/architecture/plugin-layout.md @@ -9,23 +9,45 @@ ## 1. The decision 1. `frdminc/nix2cf`, the compiler, folds into `frdminc/tendcf` with its - full git history, first under `compiler/`. A separate pull request does - the import; this document moves no files. -2. The repository takes the plugin-type layout of §2. It stays one Python - package. -3. Plugins are found through Python entry-point groups (§3), so a - third-party plugin can live outside this repository as an ordinary - pip package. -4. Plugins run at compile time only (§4). -5. The compiler's command becomes `tendcf compile`. The name `nix2cf` - moves to the Nix frontend (§6). -6. `frdminc/nix2cf` is archived once the move is done. + full git history, first under `compiler/`. A separate pull request + ([#45](https://github.com/frdminc/tendcf/pull/45)) does the import; + this document moves no files. +2. The repository takes the plugin-type layout of §2. It becomes one + Python package, import name `tendcf` (today's `nix2cf` package is + renamed; how the directories map onto it is open, §7). +3. Frontends and importers are found through the public Python + entry-point groups `tendcf.frontends` and `tendcf.importers` (§3), so a + third-party frontend or importer can live outside this repository as + an ordinary pip package. Supervisor renderers are in tree only; there + is no public `tendcf.supervisors` group (§3). +4. Plugins run at compile time only, and they are trusted build-time code + that the operator explicitly allows, not sandboxed code (§4). +5. A plugin loads only if site-private names it (§4, activation). +6. The compiler's command becomes `tendcf compile`, which replaces + `nix2cf buildfile` outright: no alias (§6). The name `nix2cf` moves to + the Nix frontend. +7. The compiler renders only the current `schema_version` (§4). +8. `frdminc/nix2cf` is archived as soon as the import (#45) merges (§5). The consequence for the change process: the compiler is no longer a separately pinned input. Schema and compiler change in the same commit, so the two-repository bump order and the compiler's `tendcf.lock` go -away. Site inputs (site-shared, foreign sites, tool forks) keep -`site.lock` ([`change-process.md`](change-process.md) §2, §4). +away. `site.lock` loses only its `nix2cf` entry: it still pins tendcf, +site-shared and foreign sites, and tool forks stay as its rule 5 has +them, never an input of their own +([`change-process.md`](change-process.md) §2, §4). + +The price: one tendcf rev is the unit a site pins. A compiler fix brings +with it every `schema/` and `policy/` commit that landed before it, and a +change to the policy tree is a privileged-region hunk on a consented +device (map §9.8). The coupling is accepted. There is no +maintenance-branch machinery for carrying a compiler fix on its own; the +question is revisited when the first device that is not an operator host +enrolls ([`change-process.md`](change-process.md) §8). + +No compatibility layer is kept, because the project has no users yet: no +command alias, no `bin/` symlinks or wrappers, no rendering of older +schema versions (§4, §5, §6). ## 2. The tree @@ -38,13 +60,13 @@ tendcf/ plugins/ frontends/yaml/ the YAML Site Model loader (built in) frontends/nix/ the Nix module frontend (map D12); named nix2cf - frontends/guix/ later: Guix System, Guile configuration → JSON via `guix repl` - supervisors/launchd/ unit renderers, one per supervisor + frontends/guix/ later: an authoring frontend, Guile records → Site Model JSON + supervisors/launchd/ unit renderers, one per supervisor; in tree only supervisors/systemd/ supervisors/runit/ importers/ansible/ optional one-shot migration into the Site Model importers/salt/ - contrib/ in-tree third-party plugins, lower bar + contrib/ in-tree third-party frontends and importers docs/ tests/ ``` @@ -52,38 +74,61 @@ tendcf/ | --- | --- | --- | | `schema/`, `examples/` | exist in tendcf | unchanged | | `policy/`, `overlay/`, `deploy/` | exist in tendcf | unchanged | -| `tools/` | is `bin/`: standalone `uv run --script` files; tendcf has no `pyproject.toml` | `bin/` renamed `tools/`, after #41 and #42 merge | -| `compiler/` | `frdminc/nix2cf`: `nix2cf/cli.py`, `compile.py`, `sitemodel.py`; one command, `nix2cf buildfile` | imported with history, then `tendcf compile` | +| `tools/` | is `bin/`: standalone `uv run --script` files; tendcf has no `pyproject.toml` | `bin/` renamed `tools/` after #41 and #42 merge, with no symlink or wrapper left behind and every reference rewritten (§5, step 4) | +| `compiler/` | `frdminc/nix2cf` (#45 imports it as `compiler/`): `nix2cf/cli.py`, `compile.py`, `sitemodel.py`. Two subcommands: `buildfile`, the implemented compilation command, which renders one host; and `eval-nix`, a plain-expression evaluation hook, not the D12 module frontend | the compiler of the `tendcf` package; command `tendcf compile` (§6) | | `plugins/frontends/yaml/` | `nix2cf/sitemodel.py` (YAML loading with duplicate-key refusal and line numbers) | split out of the compiler, still built in | -| `plugins/frontends/nix/` | only the hook: `nix2cf eval-nix FILE` evaluates a plain Nix expression with `nix-instantiate --eval --strict --json`; no module frontend | the D12 Nix module frontend, named `nix2cf` | +| `plugins/frontends/nix/` | only the hook: `nix2cf eval-nix FILE` evaluates a plain Nix expression with `nix-instantiate --eval --strict --json`; no module frontend | the D12 Nix module frontend, named `nix2cf`; `eval-nix` moves here | | `plugins/frontends/guix/` | nothing | later | | `plugins/supervisors/launchd/` | inside `nix2cf/compile.py`, the only supervisor whose goal-file `unit` entry is rendered | split out | -| `plugins/supervisors/systemd/`, `runit/` | nothing: a service selected for a systemd or runit host is a compile error (`RENDERED_SUPERVISORS` in `compile.py`) | written as plugins ([`linux-targets.md`](linux-targets.md) §3, step 6) | +| `plugins/supervisors/systemd/`, `runit/` | nothing: a service selected for a systemd or runit host is a compile error (`RENDERED_SUPERVISORS` in `compile.py`) | written in tree ([`linux-targets.md`](linux-targets.md) §3, step 6) | | `plugins/importers/` | nothing | optional | | `contrib/` | nothing | as third parties contribute | -| entry-point groups | none; nix2cf's `pyproject.toml` declares only the `nix2cf` script | the three groups of §3 | +| entry-point groups | none; nix2cf's `pyproject.toml` declares only the `nix2cf` script | the two public groups of §3 | A supervisor plugin renders a service record into the goal file's `unit` entry for that supervisor and resolves that supervisor's defaults (for example `working_dir`). It does not write the unit file on the host; the generic bundle under `policy/` does that, after approval and projection. +`tendcf-validate`, the device-side command that D45 makes out of the +reference projector, has no directory of its own in this tree yet (§7). +Wherever it lands, the device boundary rules of §4 apply to it. + ## 3. Plugin kinds and entry-point groups | Kind | Entry-point group | What it is | | --- | --- | --- | -| Frontend | `tendcf.frontends` | A language a site keeps authoring in. It turns its source into the Site Model's JSON, which the compiler then merges, checks and renders like any other. YAML and Nix are the two equal frontends of map D12; Guix may follow. | -| Supervisor | `tendcf.supervisors` | A unit renderer for one supervisor (launchd, systemd, runit, …): service record in, goal-file `unit` entry out (§2). | -| Importer | `tendcf.importers` | An optional one-shot migration from another tool's configuration (Ansible, Salt) into Site Model files, which the site then keeps and edits as its own. Not run on every compile. | +| Frontend | `tendcf.frontends` (public) | A language a site keeps authoring in. It turns its source into the Site Model's JSON, which the compiler then merges, checks and renders like any other. YAML and Nix are the two equal frontends of map D12; Guix may follow as an authoring frontend (Guile records → Site Model JSON). | +| Importer | `tendcf.importers` (public) | An optional one-shot migration from another tool's configuration (Ansible, Salt) into Site Model files, which the site then keeps and edits as its own. Not run on every compile. | +| Supervisor renderer | none: in tree only | A unit renderer for one supervisor (launchd, systemd, runit): service record in, goal-file `unit` entry out (§2). | The difference between a frontend and an importer is what happens next. A site authored through a frontend keeps its source in that language. A site migrated by an importer keeps the Site Model files the importer wrote, and the importer is not needed again. -Built-in plugins are found through the same groups as third-party ones. -The Python interface a plugin implements is not specified yet; it is to -be settled when the split is done (§5, step 3). +**Why supervisor renderers are not a public group.** The goal file's +`unit` is a closed `oneOf` over `launchd`, `systemd` and `runit`, each with +`additionalProperties: false` (`schema/goal-file.schema.json`, the `unit` +definition), and the generic bundle refuses any flavour it does not +implement ([`linux-targets.md`](linux-targets.md) §3, step 3). An +out-of-tree plugin therefore cannot add a supervisor: that needs a +`schema_version` bump, a bundle branch and projector support in tendcf +first. All an out-of-tree supervisor plugin could do is replace the +renderer of one of the three existing supervisors, which is exactly the +shadowing of a built-in that §4 forbids. Frontends and importers are +genuinely open, because their output is Site Model data that the compiler +checks like any other. + +**Not Guix System.** The Guix frontend is an authoring language for the +Site Model, not a host type. A Guix System host, whose init is Shepherd +(not one of the supervisors above), is a separate question and is not +decided (§7). + +Built-in frontends and importers register under the same groups as +third-party ones, and their names are reserved (§4). The Python interface +a plugin implements is not specified yet; it is to be settled when the +split is done (§5, step 4). ## 4. Rules for every plugin @@ -91,45 +136,157 @@ be settled when the split is done (§5, step 3). signed release and runs CFEngine with `tendcf-validate` or tendcf-agent (map D45), never the compiler. -**No way around consent.** Everything a plugin produces passes through the -same checks as the rest of the compiler's output: the Site Model and the -goal file are validated against `schema/`, and the goal file is projected -by the reference projector. A goal file the schema lint or the projector -rejects stops the release ([`change-process.md`](change-process.md) §6, -release gates 3 and 4). One that passes still reaches a device only as a -diff the device validates against its approved baseline (guide §7). A -plugin has no other path to a device. +**The device boundary is structural, not a convention** (planned with the +restructure, §5 step 4): + +1. The device artifact (`tendcf-validate`, D45) is built from the + validator's own source alone. It imports nothing from `compiler/`, + `plugins/` or `contrib/` and never calls entry-point discovery. +2. Dependency direction: the schema lint and the projector never import + the compiler. The contract stays outside the compiler's code even + though both live in one repository. +3. A CI import check enforces both. It does not exist yet. + +**Plugins are trusted build-time code.** A plugin is Python that runs in +the compile step with the compiler's own access: the merged Site Model, +including inventory that is private by default (map D35), allocations and +secret names; the build host's files and processes. A pip distribution +can also run code at import time in every Python process of the +environment it is installed in. Nothing sandboxes it. Allowing a plugin +(by naming it, below) is trusting it as much as the compiler itself, with +the site's private data and, on operator hosts, with root. What the +release checks do and do not give, stated exactly: + +1. A plugin cannot make a device accept anything that device's own diff + does not show. A goal file that passes reaches the device as a + complete signed canonical goal file; the device computes and validates + the diff against its own approved baseline before approval and + projection (guide §7). +2. A plugin cannot emit a shape the schema forbids. The schema lint and + the projector reject it, and a goal file they reject stops the release + ([`change-process.md`](change-process.md) §6, release gates 3 and 4). +3. A plugin **can** make the goal file say something other than what the + Site Model says. Valid is not faithful: a buggy or malicious renderer + or frontend can emit a schema-valid entry with a different argv, a + different `run_as`, an extra service or a tombstone. On a consented + device the check is the person's yes over the device-side diff. An + operator host applies without a local yes (change-process §8), so + there the only human check is the pull request's preview diff. + +**Release gates and signing run apart from plugins.** Release gates 3 to 5 +(schema lint, projector, preview diff) and signing (gate 6) run in a +separate environment in which no plugin distribution is installed, and +the compile step holds no signing key +([`change-process.md`](change-process.md) §6). The gates' preview diff is +computed apart from the plugins, but it diffs their output, so it shows +what they rendered. A preview diff or an `explain-hunk` attribution +produced on the compile side (D44, map §9.5) is compiler output and +inherits the plugins' trust. + +**Activation: a plugin loads only if site-private names it.** + +1. Site-private names every plugin a compile uses: name → distribution, + version and hash. An installed plugin that site-private does not name + is ignored; installing a package activates nothing. +2. A duplicate name within a group is a compile error, never a silent + winner (the same rule map §3 applies to identities from two peer + inputs). +3. Built-in names are reserved: no other distribution may register them. +4. Until out-of-tree pinning exists + ([#47](https://github.com/frdminc/tendcf/issues/47)), a release build + loads in-tree plugins only (`plugins/` and `contrib/`, at the pinned + tendcf rev). That way the tendcf pin covers the plugin set, and a + release stays reproducible from its site-private commit. +5. The release's `targets.json` `custom` records the plugin set + ([`change-process.md`](change-process.md) §3). + +How an out-of-tree plugin earns trust (an OpenPGP web of trust) and how a +site manages its plugins (a `tendcf plugin` command) is +[#47](https://github.com/frdminc/tendcf/issues/47). + +**Current schema version only.** The compiler renders only the current +`schema_version`. Its ability to render older versions (today nix2cf's +`SUPPORTED_SCHEMA_VERSIONS`, 3 to 5) is removed in the restructure (§5, +step 4). Map §9.6's ceiling design, under which the compiler renders each +host at the highest version that host last reported, stays the design; +it applies once the first device that is not an operator host enrolls, +and the multi-version render returns with it. **Where a plugin lives.** -1. **In tree, `plugins/`.** The YAML frontend, the Nix frontend, the - supervisor renderers the project supports, and any importer the - project maintains. Held to this repository's full review rule. -2. **In tree, `contrib/`.** Third-party plugins accepted into the - repository with a lower bar. They pass the same schema lint and - projector as everything else (the rule above does not relax). -3. **Out of tree.** A pip package that registers under one of the three - entry-point groups, the way Salt extensions and Ansible collections are - shipped apart from Salt and Ansible. tendcf does not review it; its - output is checked like any other plugin's. +1. **In tree, `plugins/`.** The YAML frontend, the Nix frontend, every + supervisor renderer, and any importer the project maintains. Held to + this repository's full review rule. +2. **In tree, `contrib/`.** Third-party frontends and importers accepted + into the repository with a lower bar. The lower bar waives support and + maintenance promises only: the project does not promise to fix, keep + or port a contrib plugin. It never waives the standing review rule + (root `AGENTS.md`: every pull request gets a cross-model frontier + review), because contrib code runs in the release pipeline with the + same privilege as everything else. A contrib plugin is off unless + site-private names it, like any other. +3. **Out of tree.** A pip package that registers under `tendcf.frontends` + or `tendcf.importers`, the way Salt extensions and Ansible collections + are shipped apart from Salt and Ansible. tendcf does not review it. + Its output passes the same release gates as any plugin's, which bound + its shape, not its faithfulness (above). A release build does not load + one until #47 exists. ## 5. Order of the move | # | Change | State | | --- | --- | --- | | 1 | This documentation: the guide, the paper, the map (D46), the change process, the Linux targets plan, the glossary, this file; and the rename of the illustrative `nix2cf_*` keys (§6). Moves no files and changes no code or schema. | this change | -| 2 | Import `frdminc/nix2cf` with full history under `compiler/`; the root `README.md` follows. | separate pull request | -| 3 | `bin/` → `tools/`; split the compiler into `compiler/` and `plugins/` with the three entry-point groups; the command becomes `tendcf compile`, `nix2cf buildfile` kept as a short-lived alias. Waits until open PRs #41 and #42 merge. | planned | -| 4 | Archive `frdminc/nix2cf`. | planned, once the move is done | +| 2 | Import `frdminc/nix2cf` with full history under `compiler/`; the root `README.md` follows. | [#45](https://github.com/frdminc/tendcf/pull/45) | +| 3 | Archive `frdminc/nix2cf`. | as soon as #45 merges | +| 4 | The restructure, one pull request: `bin/` → `tools/`; the Python package renamed `tendcf`; the compiler split into `compiler/` and `plugins/` with the two public entry-point groups; `tendcf compile` replaces `nix2cf buildfile`; older-version rendering removed; the device-boundary import check (§4). No alias, symlink or wrapper. Waits until open PRs #41 and #42 merge. | planned | + +Step 4 rewrites every reference in the living files. Found on 2026-11-13 +(`rg 'nix2cf'` and `rg 'bin/(schema_lint|projector|xref_lint|overlay|flag_coverage|unit_classifier)'` +outside dated records); rerun both before the change, because the list +moves: + +1. **The compiler's in-tree check** (from #45): + `compiler/nix2cf/cli.py` accepts an enclosing tendcf checkout only if + it has `bin/projector.py` (`_in_tree_tendcf`), loads the projector from + that path (`load_projector`) and names it in an error message; `compiler/nix2cf/__init__.py` and `compile.py` cite + `bin/projector.py` and `bin/schema_lint.py` in their docstrings. +2. **Schema description text**: `schema/common.schema.json` (three + descriptions say nix2cf infers edges or fills in a field; under the new + names that is the compiler), `schema/services.schema.json` and + `schema/launchd-writers.schema.json` (`bin/schema_lint.py`). +3. **Examples**: `examples/services.yml`, `examples/launchd-writers.yml`, + and the broken fixtures `04-nested-writer-prefix`, + `107-launchd-writers-nixos-writer` and `123-launchd-writers-dpkg-writer` + (their `launchd-writers.yml` copies the note) plus + `examples/broken/README.md`. +4. **Repository files**: `AGENTS.md` (the standing rule names nix2cf as a + repository, and its description of `check.yml` names the `bin/` + tools), + `README.md`, `overlay/overlay.json`, `overlay/README.md`, + `.github/workflows/check.yml`, `.githooks/pre-commit`, + `deploy/linux/canary/check.py`, `tests/unit_classifier/test_logic.py`, + and the tools' own cross-references inside `bin/`. +5. **Documents**: map D45 (`bin/projector.py` as the source of + `tendcf-validate`), and every living document `bin/xref_lint.py` lints. + Dated records (handoffs, reviews, reconciliations, opinions) keep the + old paths as history and are not rewritten. ## 6. Names -1. **`tendcf compile`** is the compiler. Today the same code is - `nix2cf buildfile`; that spelling stays as a short-lived alias. +1. **`tendcf compile`** is the compiler. By default it compiles every + host; `tendcf compile --host NAME` renders one host, which is what + `nix2cf buildfile` does today. It replaces `nix2cf buildfile` in the + restructure (§5, step 4) with no alias. The Bcfg2 `buildfile` lineage + (guide §4) is kept in prose only; no subcommand keeps the name. 2. **`nix2cf`** is the Nix module frontend, `plugins/frontends/nix/`. Up to 2026-11-13 the name meant the whole compiler; older documents and - handoffs use it that way and are not rewritten. -3. **`guix`** is the planned Guix frontend, `plugins/frontends/guix/`. + handoffs use it that way and are not rewritten. Until the restructure + the only thing that ships under the name is still the old compiler + (`frdminc/nix2cf`, then `compiler/` after #45, command + `nix2cf buildfile`). +3. **`guix`** is the planned Guix authoring frontend, + `plugins/frontends/guix/`. 4. **The illustrative `preview_services` / `preview_edges` keys.** The example's keys were `nix2cf_services` and `nix2cf_edges`. Checked on 2026-11-13: they occurred only in the illustrative, preview-channel @@ -145,9 +302,22 @@ plugin has no other path to a device. ## 7. Not decided here -- The Python interface each plugin kind implements (§3). -- The review bar for `contrib/` beyond the schema lint and the projector. +- The Python interface each plugin kind implements (§3), and how the + compiler finds its in-tree supervisor renderers without a public group. +- How the directories (`compiler/`, `plugins/`, `tools/`, `contrib/`) map + onto the `tendcf` import package, and whether `schema/` ships as package + data (an out-of-tree plugin that depends on tendcf needs the contract + from the installed package). +- Where `tendcf-validate`'s source lives (§2), within the device boundary + rules of §4. +- How site-private names its plugins: the file and its encoding, and how + an in-tree plugin is named in it (§4, activation). +- Out-of-tree pinning and trust establishment, after which a release + build may load an out-of-tree plugin + ([#47](https://github.com/frdminc/tendcf/issues/47)). +- Whether the compile step has network access. The guide calls the render + a pure function of the Site Model (guide §4); with plugins that holds + only for what the release environment controls. - Which importers, if any, get built. -- How out-of-tree plugins are pinned for a release. `site.lock` pins - sources ([`change-process.md`](change-process.md) §2, rule 5); a plugin - installed from pip is not yet covered by it. +- Guix System hosts with Shepherd as their supervisor (§3). +- Whether the PyPI names `tendcf` and `nix2cf` are free; unchecked. diff --git a/docs/paper/tendcf-architecture-guide.md b/docs/paper/tendcf-architecture-guide.md index 775a26c..85bcde0 100644 --- a/docs/paper/tendcf-architecture-guide.md +++ b/docs/paper/tendcf-architecture-guide.md @@ -28,7 +28,8 @@ open questions in research form, is checked, a first generic policy bundle runs, and the projection step has a reference implementation. The compiler has its first piece (`buildfile`, §4); the rest of it, the on-device executor, and the screens a person -would use to accept or refuse a change are still to be built. No device has been set up from factory reset by this automation. +would use to accept or refuse a change are still to be built. No device +has been set up from factory reset by this automation. One thing that *was* built, and was not planned: §19 describes the defects found underneath, in CFEngine itself, by leaning on the parts of it this @@ -144,8 +145,8 @@ moving the facts. **Schemas and types live in tendcf's `schema/`**, not in the compiler's code, although the compiler is moving into the same repository (decided -2026-11-13, §4). A schema change is an interface change. A hostname change is site data. They do not move -together. The files are [`schema/`](../../schema/) and +2026-11-13, §4). A schema change is an interface change. A hostname +change is site data. They do not move together. The files are [`schema/`](../../schema/) and [`examples/`](../../examples/) in this repository. **Inventory is private by default.** Site-shared may ship device *kinds*, @@ -223,16 +224,28 @@ not the home of schemas. **Name and home (decided 2026-11-13).** The compiler was built as the separate repository `frdminc/nix2cf`, whose `nix2cf buildfile` command is the first piece described below. It is being folded into tendcf with its -history, under `compiler/`, and its command becomes `tendcf compile` -(`nix2cf buildfile` stays for a while as an alias). The name `nix2cf` -moves to what it describes: the Nix module frontend (§3), one of the -compiler's plugins. Frontends, supervisor renderers and one-shot -importers from other tools are plugins found through Python entry points, -so a third party can ship one as an ordinary package. Plugins run only at -compile time, never on a device, and everything they produce still passes -the schema lint and the projector, so no plugin can get around consent. -The layout and the order of the move are in -[`plugin-layout.md`](../architecture/plugin-layout.md). +history, under `compiler/`, and that repository is archived as soon as +the import merges. The command becomes `tendcf compile`, which replaces +`nix2cf buildfile` outright, with no alias: it compiles every host, and +`--host` renders one. The name `nix2cf` moves to what it describes: the +Nix module frontend (§3), one of the compiler's plugins. Frontends, +supervisor renderers and one-shot importers from other tools are +plugins. Frontends and importers are found through Python entry points, +so a third party can ship one as an ordinary package; supervisor +renderers are in tree only. Plugins run only at compile time, never on a +device. + +A plugin is trusted build-time code that the operator explicitly allows, +not sandboxed code, and the claim it supports is narrow. A plugin cannot +make a device accept anything that device's own diff does not show (§7), +and it cannot emit a shape the schema forbids. It can make the goal file +say something other than what the Site Model says, and it sees the +site's private data. So the release gates (schema lint, projector, +preview diff) and signing run in a separate environment with no plugin +installed, and the compile step holds no signing key; a release loads +only the plugins site-private names, and only in-tree ones until +out-of-tree plugins can be pinned. The rules, the layout and the order +of the move are in [`plugin-layout.md`](../architecture/plugin-layout.md). **The two files are not read by the same code, and they do not land in the same place.** `def.json` goes through the augments loader and its keys @@ -253,9 +266,14 @@ directories exist, this service is loaded” for any entry in the data. A first version of that bundle exists and runs: [`policy/tendcf_services.cf`](../../policy/tendcf_services.cf). It handles -launchd only — systemd, runit and Jobber are unwritten — so “supervisors -are adapters” (§5) is still a claim about a design, not about running code. -§16.C shows it working, including an interlock that genuinely refuses. +launchd and, since 2026-10-31, systemd +([`linux-targets.md`](../architecture/linux-targets.md) §3, step 5; a +systemd canary runs under it on a live Linux host). runit and Jobber are +unwritten, and the compiler still renders service units for launchd only +(§18), so “supervisors are adapters” (§5) holds in running code for two +supervisors on the device side and is still a design claim for the rest. +§16.C shows the launchd branch working, including an interlock that +genuinely refuses. The pipeline has four stages: @@ -275,9 +293,13 @@ Because the render is a pure function of the Site Model — same input, same output — “show me exactly what device X would receive, without touching device X” is almost free. That command, `buildfile`, was the first piece of the compiler built: merge, conflict check and goal-file render, with -service units for launchd only and no inference yet (§18). It is how an agent checks its own work, how the compiler -regression-tests itself, and how a human sees a proposed change before it -lands. The shape is [Bcfg2 `buildfile`](https://docs.bcfg2.org/). +service units for launchd only and no inference yet (§18). It is how an +agent checks its own work, how the compiler regression-tests itself, and +how a human sees a proposed change before it lands. The shape is +[Bcfg2 `buildfile`](https://docs.bcfg2.org/). That lineage survives in +prose only: from the restructure on, the command is +`tendcf compile --host NAME`, and no subcommand keeps the name +`buildfile`. Compiling a typed description into an existing engine’s native format is a known pattern. [NixOS](https://nixos.org/) does it into systemd units; @@ -1272,15 +1294,17 @@ observe themselves crossing is a disclaimer rather than a ceiling. alone: reference resolution, launchd labels checked against declared writer prefixes, no prefix nested inside another. A second lint checks that every cross-reference in the documentation resolves. -- **Ninety-two deliberately broken fixtures**, every one of which the lint +- **One hundred and fifty-nine deliberately broken fixtures** (counted + 2026-11-13), every one of which the lint must catch, and it fails if any is accepted. A passing lint on correct - input is not a check. They come in three kinds: fifty-nine schema-level + input is not a check. They come in three kinds: a hundred and + twenty-five schema-level (an opt-out with no reason, a rogue launchd label, a literal secret where a key name belongs, a proposer setting a privileged flag, a migration that is not a no-op); six byte-level, which exist because a canonicalization fault is invisible once the file has been parsed (a pretty-printed twin, duplicate keys, a non-NFC path, `15.0` where `15` - belongs); and twenty-seven projection-level (a resolved secret value + belongs); and twenty-eight projection-level (a resolved secret value where a name belongs, a goal file smuggled in under `vars`). Each rule in the projector’s checker has a fixture that trips **that rule alone** — a suite where one fixture trips three rules cannot tell you the other two @@ -1290,8 +1314,10 @@ observe themselves crossing is a disclaimer rather than a ceiling. `tendcf-validate`, the command Linux and macOS hosts run, so that CI tests the device code itself; the Android agent’s Kotlin port must agree with the same goldens (implementer map D45). -- **A first generic bundle** (`policy/tendcf_services.cf`), launchd only — - the first thing here that CFEngine actually executes (§16.C). +- **A first generic bundle** (`policy/tendcf_services.cf`), with a + launchd branch and, since 2026-10-31, a systemd branch — the first + thing here that CFEngine actually executes (§16.C; + [`linux-targets.md`](../architecture/linux-targets.md) §3). - **A patch overlay on libntech and cfengine/core** (`overlay/`, `bin/overlay.py`): the six fork commits that bundle depends on (§19), pinned to one upstream commit per component, with a lint, a self-test, a @@ -1311,9 +1337,11 @@ observe themselves crossing is a disclaimer rather than a ceiling. The rest of the compiler (inference, extra-entry reporting, service units for any supervisor but launchd, the Nix module frontend and the -plugin layout, §4), every supervisor adapter except launchd, the signed -release path, the on-device validator, the consent surface, and the -peer-action runtime. No operational numbers of any kind. +plugin layout, §4); device-side supervisor adapters other than launchd +and systemd (runit and the rest), and the open parts of those two (Step +1 below; [`linux-targets.md`](../architecture/linux-targets.md) §3); +the signed release path, the on-device validator, the consent surface, +and the peer-action runtime. No operational numbers of any kind. **Build order from here** (each step is meant to leave the system in a coherent, describable state): diff --git a/docs/paper/tendcf-architecture-paper.md b/docs/paper/tendcf-architecture-paper.md index 6f81cdf..2710127 100644 --- a/docs/paper/tendcf-architecture-paper.md +++ b/docs/paper/tendcf-architecture-paper.md @@ -258,14 +258,25 @@ The compiler is a tool, not the home of schemas. It was built as the separate repository `frdminc/nix2cf`, whose `nix2cf buildfile` command is its first piece. On 2026-11-13 it was decided -to fold that repository into tendcf with its history, under `compiler/`, to -rename the command `tendcf compile` (keeping `nix2cf buildfile` for a while -as an alias), and to give the name `nix2cf` to what it describes: the -optional Nix module frontend of §2.1. Frontends, supervisor renderers and -one-shot importers from other tools become plugins found through Python -entry points. They run at compile time only, and everything they produce -still passes the schema lint and the projector, so no plugin can get around -consent. The layout is in +to fold that repository into tendcf with its history, under `compiler/` +(archiving it as soon as the import merges), to replace `nix2cf buildfile` +with `tendcf compile` outright, with no alias, and to give the name +`nix2cf` to what it describes: the optional Nix module frontend of §2.1. +Frontends, supervisor renderers and one-shot importers from other tools +become plugins; frontends and importers are found through Python entry +points, supervisor renderers stay in tree. Plugins run at compile time +only. + +A plugin is trusted build-time code that the operator explicitly allows, +not sandboxed code, and the guarantee is correspondingly narrow. A plugin +cannot make a device accept anything that device's own diff does not +show (§2.5), and it cannot emit a shape the schema forbids. It can make +the goal file say something other than what the Site Model says, and it +sees the site's private data. The release gates (schema lint, projector, +preview diff) and signing therefore run in a separate environment with no +plugin installed, the compile step holds no signing key, and a release +loads only the plugins site-private names, in-tree ones only until +out-of-tree plugins can be pinned. The rules and the layout are in [`plugin-layout.md`](../architecture/plugin-layout.md). Two details of that layer are load-bearing and neither is obvious from the @@ -319,7 +330,9 @@ launchd only and no inference yet — and §6.2 explains why it earns its place three separate times over. **Where it runs:** operator machine or CI. Consented devices receive a -signed artifact. They do not run Nix, the compiler, or any of its plugins. +signed artifact. They do not run the compiler or any of its plugins; Nix +runs on a device only where it already is that host's configuration +system (NixOS, nix-darwin). **This compile-to-native-format shape is not new.** A typed or module-based authoring language compiling to an existing execution engine's native format @@ -1705,21 +1718,23 @@ Beyond the schemas, two pieces of running code: the **reference projector** and is gated in CI against golden bytes; it is to be promoted to the `tendcf-validate` command Linux and macOS hosts run, and the Android agent's Kotlin port must agree with the same goldens; and the **generic bundle** -(`policy/tendcf_services.cf`, v1, launchd only), which is the first thing in -this project that CFEngine actually executes. +(`policy/tendcf_services.cf`; a launchd branch and, since 2026-10-31, a +systemd branch), which is the first thing in this project that CFEngine +actually executes. **Validated in the following narrow but no longer trivial sense.** A lint that passes on correct input demonstrates nothing about whether it catches -incorrect input, so the contracts are tested against **ninety-two -deliberately broken fixtures**, each of which must be caught, and the lint -fails if any is accepted: fifty-nine schema-level negatives (an opt-out with +incorrect input, so the contracts are tested against **one hundred and +fifty-nine deliberately broken fixtures** (counted 2026-11-13), each of +which must be caught, and the lint fails if any is accepted: a hundred and +twenty-five schema-level negatives (an opt-out with no reason, a rogue launchd label, a nested writer prefix, a literal secret where a key name belongs, a typo'd token kind, an enforce-mode row carrying an audit-mode outcome, a proposer setting a privileged flag, a migration that is not a no-op), six byte-level negatives that exist because canonicalization violations are invisible after a parse (a pretty-printed twin, duplicate keys, a non-NFC path, `15.0` where `15` belongs), and -twenty-seven projection negatives (a resolved secret value where a name +twenty-eight projection negatives (a resolved secret value where a name belongs, a canonified interlock id, a goal file smuggled under `vars`, an `env` that is not an object). Each rule in the projector's checker has a fixture that trips **it alone**, which is a stronger property than @@ -1735,8 +1750,9 @@ compiler's own tooling. — inference, extra-entry reporting, service units for any supervisor but launchd, the Nix module frontend and the plugin layout (its first piece, `buildfile`, exists, §2.2) — the signed release path, -the on-device validator, the consent surface, peer-action runtime, and every -supervisor adapter except launchd. Remaining Step 0 work: `peer_actions`, +the on-device validator, the consent surface, peer-action runtime, and the +device-side supervisor adapters other than launchd and systemd (runit and +the rest), plus the open parts of those two. Remaining Step 0 work: `peer_actions`, trust-policy shape, generic unit-writers, lookup stub, YAML canonicalize, and transcribing existing reality as `not-yet-migrated`. From 74967fcb3584aae768c7c7a305dcb495c152fe3b Mon Sep 17 00:00:00 2001 From: Daniel JB Clark Date: Fri, 13 Nov 2026 06:37:36 -0500 Subject: [PATCH 5/9] docs: rewrap long lines left by the D46 review fixes Co-Authored-By: Claude Opus 5.5 --- docs/architecture/plugin-layout.md | 7 ++++--- docs/paper/tendcf-architecture-guide.md | 5 +++-- docs/paper/tendcf-architecture-paper.md | 6 +++--- 3 files changed, 10 insertions(+), 8 deletions(-) diff --git a/docs/architecture/plugin-layout.md b/docs/architecture/plugin-layout.md index 54167a5..865eade 100644 --- a/docs/architecture/plugin-layout.md +++ b/docs/architecture/plugin-layout.md @@ -60,7 +60,7 @@ tendcf/ plugins/ frontends/yaml/ the YAML Site Model loader (built in) frontends/nix/ the Nix module frontend (map D12); named nix2cf - frontends/guix/ later: an authoring frontend, Guile records → Site Model JSON + frontends/guix/ later: Guile records → Site Model JSON supervisors/launchd/ unit renderers, one per supervisor; in tree only supervisors/systemd/ supervisors/runit/ @@ -249,8 +249,9 @@ moves: 1. **The compiler's in-tree check** (from #45): `compiler/nix2cf/cli.py` accepts an enclosing tendcf checkout only if it has `bin/projector.py` (`_in_tree_tendcf`), loads the projector from - that path (`load_projector`) and names it in an error message; `compiler/nix2cf/__init__.py` and `compile.py` cite - `bin/projector.py` and `bin/schema_lint.py` in their docstrings. + that path (`load_projector`) and names it in an error message; + `compiler/nix2cf/__init__.py` and `compile.py` cite `bin/projector.py` + and `bin/schema_lint.py` in their docstrings. 2. **Schema description text**: `schema/common.schema.json` (three descriptions say nix2cf infers edges or fills in a field; under the new names that is the compiler), `schema/services.schema.json` and diff --git a/docs/paper/tendcf-architecture-guide.md b/docs/paper/tendcf-architecture-guide.md index 85bcde0..458d9ab 100644 --- a/docs/paper/tendcf-architecture-guide.md +++ b/docs/paper/tendcf-architecture-guide.md @@ -146,8 +146,9 @@ moving the facts. **Schemas and types live in tendcf's `schema/`**, not in the compiler's code, although the compiler is moving into the same repository (decided 2026-11-13, §4). A schema change is an interface change. A hostname -change is site data. They do not move together. The files are [`schema/`](../../schema/) and -[`examples/`](../../examples/) in this repository. +change is site data. They do not move together. The files are +[`schema/`](../../schema/) and [`examples/`](../../examples/) in this +repository. **Inventory is private by default.** Site-shared may ship device *kinds*, example inventories, and **explicit exports** (public endpoints, role diff --git a/docs/paper/tendcf-architecture-paper.md b/docs/paper/tendcf-architecture-paper.md index 2710127..ee691b4 100644 --- a/docs/paper/tendcf-architecture-paper.md +++ b/docs/paper/tendcf-architecture-paper.md @@ -1752,9 +1752,9 @@ launchd, the Nix module frontend and the plugin layout (its first piece, `buildfile`, exists, §2.2) — the signed release path, the on-device validator, the consent surface, peer-action runtime, and the device-side supervisor adapters other than launchd and systemd (runit and -the rest), plus the open parts of those two. Remaining Step 0 work: `peer_actions`, -trust-policy shape, generic unit-writers, lookup stub, YAML canonicalize, -and transcribing existing reality as `not-yet-migrated`. +the rest), plus the open parts of those two. Remaining Step 0 work: +`peer_actions`, trust-policy shape, generic unit-writers, lookup stub, YAML +canonicalize, and transcribing existing reality as `not-yet-migrated`. **Nothing is deployed.** No device has been provisioned from factory reset by this automation. We have no deployment time, no effort figure, no From 64994e3fe87cefa5553efd0f83de0b67ee422d32 Mon Sep 17 00:00:00 2001 From: Daniel JB Clark Date: Fri, 13 Nov 2026 06:39:51 -0500 Subject: [PATCH 6/9] plugin-layout: the path rewrite covers every file, dated records included (djbclark 2026-11-13) Co-Authored-By: Claude Opus 5.5 --- docs/architecture/plugin-layout.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/docs/architecture/plugin-layout.md b/docs/architecture/plugin-layout.md index 865eade..1e67aba 100644 --- a/docs/architecture/plugin-layout.md +++ b/docs/architecture/plugin-layout.md @@ -241,10 +241,16 @@ and the multi-version render returns with it. | 3 | Archive `frdminc/nix2cf`. | as soon as #45 merges | | 4 | The restructure, one pull request: `bin/` → `tools/`; the Python package renamed `tendcf`; the compiler split into `compiler/` and `plugins/` with the two public entry-point groups; `tendcf compile` replaces `nix2cf buildfile`; older-version rendering removed; the device-boundary import check (§4). No alias, symlink or wrapper. Waits until open PRs #41 and #42 merge. | planned | -Step 4 rewrites every reference in the living files. Found on 2026-11-13 -(`rg 'nix2cf'` and `rg 'bin/(schema_lint|projector|xref_lint|overlay|flag_coverage|unit_classifier)'` -outside dated records); rerun both before the change, because the list -moves: +Step 4 rewrites every path reference in every file, dated records +included (handoffs, reviews, reconciliations, opinions, `deprecated/`): +djbclark, 2026-11-13, a one-time exception to their append-only +convention for this move, so that no path in the repository points at +a location that no longer exists. Prose that names nix2cf as the old +repository stays as history; only paths change. The living files found +on 2026-11-13 are listed below (`rg 'nix2cf'` and +`rg 'bin/(schema_lint|projector|xref_lint|overlay|flag_coverage|unit_classifier)'` +outside dated records). Rerun both over the whole tree before the +change, because the list moves: 1. **The compiler's in-tree check** (from #45): `compiler/nix2cf/cli.py` accepts an enclosing tendcf checkout only if From b315772b3419af07450548594f54eacca8344f49 Mon Sep 17 00:00:00 2001 From: Daniel JB Clark Date: Fri, 13 Nov 2026 06:50:41 -0500 Subject: [PATCH 7/9] docs: D46 fix round 2: path-rewrite scope, reproducibility, #45 merged Re-reviews (Astra gpt-6-astra high, Fable claude-fable-5-2 high) both said merge after fixes. - Rewrite scope matches djbclark's ruling everywhere: every path in every file, dated records included; prose naming nix2cf stays (A1/F1). - Reproducibility qualified: the pin identifies compiler and plugin source; byte reproduction also needs a fixed dependency/toolchain closure and controlled external inputs, not yet specified (A2). - Guide section 7 carries the current-schema-version-only note (F2). - PR 45 merged (ca4fa90) and frdminc/nix2cf is archived: compiler/ exists and its tests run in tendcf CI (F3). Branch rebased on master. - N1-N4: operator-host wording, deployment caveat, operator-host check in the guide, the rest of compiler/ in the restructure list. Co-Authored-By: Claude Opus 5.5 --- docs/architecture/GLOSSARY.md | 11 +-- .../architecture-DEFINITIVE-v3.md | 32 +++++---- docs/architecture/change-process.md | 27 +++++--- docs/architecture/plugin-layout.md | 67 ++++++++++++------- docs/paper/tendcf-architecture-guide.md | 52 ++++++++------ docs/paper/tendcf-architecture-paper.md | 10 +-- 6 files changed, 121 insertions(+), 78 deletions(-) diff --git a/docs/architecture/GLOSSARY.md b/docs/architecture/GLOSSARY.md index e06c15e..ba976e7 100644 --- a/docs/architecture/GLOSSARY.md +++ b/docs/architecture/GLOSSARY.md @@ -132,14 +132,15 @@ points in §12. `plugins/frontends/nix/` that renders a Nix-authored Site Model to the same JSON as the YAML frontend (map D12). Not built yet. - **Before 2026-11-13 — the whole compiler**, and the separate repository - `frdminc/nix2cf` that held it (`nix2cf buildfile`). That compiler is - being folded into tendcf under `compiler/`, and the repository is - archived as soon as that import merges. Older documents and handoffs use - the old sense and are not rewritten. + `frdminc/nix2cf` that held it (`nix2cf buildfile`). That compiler was + folded into tendcf under `compiler/` on 2026-11-13 (#45), and the + repository was archived the same day. Older documents and handoffs use + the old sense, and that prose is not rewritten (only their paths change + in the restructure, `plugin-layout.md` §5). - **Which sense holds when.** Until the restructure (`plugin-layout.md` §5, step 4) the only thing that ships under the name is still the old compiler: the package `nix2cf` and its command - `nix2cf buildfile`, first in `frdminc/nix2cf`, then under `compiler/`. + `nix2cf buildfile`, first in `frdminc/nix2cf`, now under `compiler/`. The restructure replaces that command with `tendcf compile`, with no alias, and renames the package `tendcf`; from then on `nix2cf` means only the Nix frontend. The "Current" sense above is the decided diff --git a/docs/architecture/architecture-DEFINITIVE-v3.md b/docs/architecture/architecture-DEFINITIVE-v3.md index 76dc648..96d12ae 100644 --- a/docs/architecture/architecture-DEFINITIVE-v3.md +++ b/docs/architecture/architecture-DEFINITIVE-v3.md @@ -133,8 +133,9 @@ Site-private holds a **lockfile** (flake-style inputs) that pins tendcf, site-shared, foreign shared sites, and optional tool forks. **Amended 2026-11-13 (D46):** the compiler is part of tendcf, so one tendcf pin covers schemas and compiler together (~~pins tendcf, nix2cf, -site-shared, …~~); until the move lands it is still the separate -`frdminc/nix2cf` repository. The repository's plugin-type layout is +site-shared, …~~); the compiler is under `compiler/` since #45 +(2026-11-13), and the separate `frdminc/nix2cf` repository is archived. +The repository's plugin-type layout is [`plugin-layout.md`](plugin-layout.md). The signed release is the deploy artifact. There is no lockstep of sibling checkouts that must share one tag. @@ -142,15 +143,16 @@ checkouts that must share one tag. | Layer | Holds | Visibility | | --- | --- | --- | | **tendcf** | Engine, **schemas/types**, generic adapters, default advisor prompt, the compiler (`compiler/`) and its plugins (`plugins/`, D46) | Public product | -| ~~**nix2cf**~~ | ~~Compiler tool only (Site Model → CFEngine JSON)~~ Folded into tendcf (D46). `nix2cf` now names the Nix frontend plugin | Repository archived as soon as the import ([#45](https://github.com/frdminc/tendcf/pull/45)) merges | +| ~~**nix2cf**~~ | ~~Compiler tool only (Site Model → CFEngine JSON)~~ Folded into tendcf (D46). `nix2cf` now names the Nix frontend plugin | Repository archived 2026-11-13, right after the import ([#45](https://github.com/frdminc/tendcf/pull/45)) merged | | **site-shared** | Reusable recipes | Public or shared | | **site-private** | Inventory, allocations, secret *names*, trust policy, extra prompt, lockfile | Private | | **foreign shared** | Other sites' recipes | Read-only inputs | | **tool forks** | sudo-secretspec, Shizuku, … | Optional | **Schemas live in tendcf**, in `schema/`, not in the compiler's code -(~~not in nix2cf~~; amended 2026-11-13, D46: the compiler is moving into -this repository, so schema and compiler change in the same commit). A +(~~not in nix2cf~~; amended 2026-11-13, D46: the compiler moved into +this repository, `compiler/`, so schema and compiler change in the same +commit). A schema change is a tendcf interface change. An instance change is site data. The JSON Schema files, fixtures, and lint are `schema/`, `examples/`, and `bin/schema_lint.py` in this repository (`bin/` becomes @@ -278,9 +280,10 @@ Augments (`def.json` / `host_specific.json`)~~: superseded by §9's render-stage decision; corrected here on 2026-11-13.) **Amended 2026-11-13 (D46).** The compiler was built as the separate -repository `frdminc/nix2cf` (`nix2cf buildfile`, Step 3). It moves into -tendcf with its history, first under `compiler/`, and its command -becomes `tendcf compile`, which replaces `nix2cf buildfile` with no +repository `frdminc/nix2cf` (`nix2cf buildfile`, Step 3). It moved into +tendcf with its history under `compiler/` (#45, 2026-11-13; the old +repository is archived), and in the restructure its command becomes +`tendcf compile`, which replaces `nix2cf buildfile` with no alias. The name `nix2cf` moves to the Nix module frontend (D12). Frontends, supervisor renderers and importers are plugins, all compile time only; frontends and importers are found through the public Python @@ -722,7 +725,8 @@ Cost, accepted: the compiler carries multi-version render ability for a window and tracks per-host versions. The price is paid in one place rather than as device-side leniency on every device. -**Amended 2026-11-13 (D46).** There are no devices to strand yet, so the +**Amended 2026-11-13 (D46).** There is no device outside the operator's +own hosts to strand yet, so the compiler renders only the current `schema_version`, and its older-version rendering is removed when the compiler is restructured ([`plugin-layout.md`](plugin-layout.md) §4, §5). This section's design @@ -1154,7 +1158,7 @@ Each step leaves a coherent system. Not a schedule. | 0 | Schemas in **tendcf** (`schema/`, `examples/`, `bin/schema_lint.py` — provides/requires, interlocks, comprehensive, report-row). Remaining: peer_actions, trust-policy shape, generic unit-writers, lookup stub, YAML canonicalize. Transcribe reality (`not-yet-migrated` is the correct day-one state). | | 1 | macOS services adapter. Dry-run default. No nix-darwin. **Started** (`policy/tendcf_services.cf`, `222da45`: launchd present/absent and interlock refusal, verified against a real `cf-agent`; package/file promises, environment rendering, the unit-writer detector still open). 2026-10-29: the satisfied-interlock path had never opened (verdict class fixed in pass 1 before the check ran); the check moved to `tendcf_interlocks`, sequenced before `tendcf_services`, and `tests/policy/run.sh` proves both paths on Linux CI and the Mac. The CFEngine it was verified against is upstream plus the patch overlay (`overlay/`, [`upstream-overlay.md`](upstream-overlay.md)); Linux CI builds the full fork ([`linux-targets.md`](linux-targets.md)). | | 2 | Android under the Site Model; Termux types; agent owns JSONL+SQLite. **Deferred** (operator, 2026-10-29): Linux comes first, see [`linux-targets.md`](linux-targets.md). Not started as code: the schema already carries a `termux` supervisor flavour (`services.schema.json`) and the `android-termux-services` domain fixture, but no adapter bundle, no agent, no JSONL/SQLite ownership. `frdminc/ShizukuTendCF` is the R4 helper (a tool fork), not this step. | -| 3 | Compiler (`tendcf compile`; ~~nix2cf~~): `buildfile` first, conflict, extra-entry, then inference (needs step 1 and a Linux systemd adapter; Android is deferred). **Unblocked** on 2026-10-29 by the operator's decision recorded on [#4](https://github.com/frdminc/tendcf/issues/4); `frdminc/nix2cf` has one compilation command, `buildfile`, plus the `eval-nix` evaluation hook. **Amended 2026-11-13 (D46):** the compiler moves into tendcf under `compiler/` and is split into plugins ([`plugin-layout.md`](plugin-layout.md)); ~~its `tendcf.lock` is specified in [`change-process.md`](change-process.md) §2~~ with schema and compiler in one repository there is no `tendcf.lock` (it was specified, never built). | +| 3 | Compiler (`tendcf compile`; ~~nix2cf~~): `buildfile` first, conflict, extra-entry, then inference (needs step 1 and a Linux systemd adapter; Android is deferred). **Unblocked** on 2026-10-29 by the operator's decision recorded on [#4](https://github.com/frdminc/tendcf/issues/4); `frdminc/nix2cf` had one compilation command, `buildfile`, plus the `eval-nix` evaluation hook. **Amended 2026-11-13 (D46):** the compiler moved into tendcf under `compiler/` (#45; `frdminc/nix2cf` archived) and the restructure splits it into plugins ([`plugin-layout.md`](plugin-layout.md)); ~~its `tendcf.lock` is specified in [`change-process.md`](change-process.md) §2~~ with schema and compiler in one repository there is no `tendcf.lock` (it was specified, never built). | | 4 | Linux reference path on a stock distro. | | 5 | First real Linux host; prove roles are data. | | 6 | Signed releases, push-only; goal-file render + diff + on-device validator (§9). Operator hosts. The lockfile, release, rollback and CI-gate design is [`change-process.md`](change-process.md) (#2); the code is not written. On Linux and macOS the on-device validator is `tendcf-validate`, built after schema v5 lands and before any further host is enforced; how much of this step's validator its first build carries is open (D45, §12). | @@ -1237,7 +1241,7 @@ Older “superseded by” archaeology lives in | D43 | ChangePlan model | ~~Model A: a closed `capability` vocabulary per operation, executor refuses effects outside the declared set.~~ **Superseded 2026-08-15 by Model B** (synthesis finding E1, adjudicated in `e1-adjudication-xhigh-2026-08-15.md`, which is final and supersedes `e1-adjudication-2026-08-15.md`): the wire artifact is the complete canonical per-host **goal file**; the ChangePlan is the **diff** against the device’s approved baseline; the executor is a **validator** over that diff, not an interpreter. Rationale: CFEngine has no runtime capability confinement, so Model A’s executor requires an interpreter plus a vocabulary-to-policy correspondence proof that does not exist and is not in budget; and coverage closes by construction when compiler and validator share one schema. The vocabulary, its versioning, and its skew policy are dropped. See §9. | | D44 | Goal-file schema family | The consent artifact is three paired schemas (goal file, goal diff, approval record) with canonical-byte fixtures. Binding sub-decisions, all from E1 §5 and normative in §9: diff computed **device-side**, full file on the wire, compiler diff is preview (§9.1 / E1 §5.1); RFC 8785 canonical JSON, no defaults, no empty collections, refuse-never-normalize, entry-granular structural hunks (§9.2 / E1 §5.2); accept is all-or-nothing, refusal annotated, no dependency groups (§9.3 / E1 §5.3); first adoption and schema migration are bounded **baseline ceremonies**, migration must diff to empty (§9.4 / E1 §5.4); hunk attribution is **excluded from the format** — impossible device-side — and replaced by an `explain-hunk` query (§9.5 / E1 §5.5); `schema_version` fails closed on unknown kinds, strandedness prevented at the compiler, two-phase bumps (§9.6 / E1 §5.6); per-domain coverage travels **in** the goal file (§9.7 / E1 §5.7). §14.2’s pre-build review applies to this family. | | D45 | Device-side component per platform | **Operator, 2026-11-02.** (1) tendcf-agent is **Android-only**, written in Kotlin. A Linux or macOS host runs CFEngine plus "a small validate-and-project command" (the ruling's words; the name **`tendcf-validate`** was chosen by the amending session on 2026-11-02 and may change without reopening the ruling), run before cf-agent reads `host_specific.json`; no daemon. (2) `tendcf-validate` is Python, the promotion of `bin/projector.py`: one implementation, so no byte-identity drift on those platforms. It needs `python3` on each host (Debian ships it, NixOS through the derivation, macOS bundled or the Command Line Tools). The Android agent's Kotlin projector is a port held to the same golden bytes in CI (residue R22), not a second normative source. (3) Built after schema v5 lands, before any further host is enforced. **Supersedes** every reading that ran the validator, the projector or the SQLite index inside tendcf-agent on every host: the §1 diagram's "CFEngine + tendcf-agent" per device, §8's index owner and D18 as written, §9's "run inside tendcf-agent" and "the agent binary" as the projector's call site, `goal-file-schema-reconciliation-2026-08-15.md` §0 item 7, §9 and §13 ("CI invokes the agent's own projector"), and `projector-reconciliation-2026-08-16.md` §9 and R22 ("tendcf-agent must reproduce" the reference). Not decided by it, and open for the operator: §12's list. | -| D46 | Repository layout and compiler plugins | **Operator, 2026-11-13.** (1) `frdminc/nix2cf` folds into tendcf with full history, first under `compiler/` ([#45](https://github.com/frdminc/tendcf/pull/45)), and is archived as soon as that import merges. (2) The repository takes the plugin-type layout of [`plugin-layout.md`](plugin-layout.md): `schema/`, `examples/`, `policy/`, `overlay/`, `deploy/` unchanged; `tools/` (was `bin/`); `compiler/` (merge, conflict check, render); `plugins/frontends/` (`yaml`, `nix`, later `guix`, an authoring frontend), `plugins/supervisors/` (`launchd`, `systemd`, `runit`), `plugins/importers/` (`ansible`, `salt`); `contrib/` for in-tree third-party frontends and importers. It becomes one Python package, import name `tendcf`. (3) Frontends and importers are found through the public entry-point groups `tendcf.frontends` and `tendcf.importers`, so a third-party one can also live out of tree as a pip package. Supervisor renderers are in tree only; there is no public `tendcf.supervisors` group, because the goal file's `unit` is a closed `oneOf` and an out-of-tree renderer could only shadow a built-in. (4) Plugins are compile-time only: none runs on a device. The device artifact (`tendcf-validate`, D45) is built from the validator's own source alone, never imports `compiler/`, `plugins/` or `contrib/` and never calls entry-point discovery; the schema lint and the projector never import the compiler; a CI import check is to enforce both (planned). Plugins are trusted build-time code that the operator explicitly allows, not sandboxed: a plugin cannot make a device accept anything that device's own diff does not show, and cannot emit a shape the schema forbids, but it can make the goal file say something other than what the Site Model says, and it sees the site's private data. Release gates (schema lint, projector, preview diff) and signing run in a separate environment with no plugin installed; the compile step holds no signing key. A compile-side preview diff or `explain-hunk` (D44) is compiler output and inherits the plugins' trust. `contrib/`'s lower bar waives support and maintenance promises, never the standing review rule. Out-of-tree trust establishment and plugin management: [#47](https://github.com/frdminc/tendcf/issues/47). (5) Activation: a plugin loads only if site-private names it (name → distribution, version, hash); an installed but unnamed plugin is ignored; a duplicate name in a group is a compile error; built-in names are reserved; until out-of-tree pinning exists (#47) a release build loads in-tree plugins only; the release's `targets.json` `custom` records the plugin set. (6) No users and no backward compatibility yet, so no shims: the compiler's command becomes `tendcf compile`, which replaces `nix2cf buildfile` with no alias (it compiles every host; `--host` renders one); `eval-nix` moves to the Nix frontend; `bin/` → `tools/` leaves no symlink or wrapper and rewrites every reference in living files; `nix2cf` names the Nix frontend (D12). The compiler renders only the current `schema_version`; older-version rendering is removed in the restructure, and §9.6's ceiling design applies once the first device that is not an operator host enrolls. (7) The keys `nix2cf_services` and `nix2cf_edges` were only ever illustrative: they occurred in guide §16.A's preview-channel example and its copy in the paper, and no schema, fixture, projector or bundle carried them (the projection's keys are `tendcf_service` and `tendcf_interlock`). In the same change that recorded this decision they were renamed `preview_services` and `preview_edges` (illustrative only, chosen so they cannot be mistaken for the `tendcf_services` bundle or the real projected keys `tendcf_service`/`tendcf_interlock`). No schema change is involved ([`plugin-layout.md`](plugin-layout.md) §6). (8) Consequence for [`change-process.md`](change-process.md): the compiler is no longer a separately pinned input, the two-repository bump order and the compiler's `tendcf.lock` go away; `site.lock` loses only its `nix2cf` entry. One tendcf rev is the unit a site pins, so a compiler fix brings the `schema/` and `policy/` commits before it; accepted, with no maintenance-branch machinery, revisited when the first device that is not an operator host enrolls. Moving `bin/` to `tools/` and the plugin split wait until open PRs #41 and #42 merge. Amends D12, D15 (a pointer), D21, §1, §3, §5, §9.6 (when it applies) and Step 3. | +| D46 | Repository layout and compiler plugins | **Operator, 2026-11-13.** (1) `frdminc/nix2cf` folds into tendcf with full history, first under `compiler/` ([#45](https://github.com/frdminc/tendcf/pull/45)), and is archived as soon as that import merges. Both done 2026-11-13 (merge commit `ca4fa90`; the repository archived the same day). (2) The repository takes the plugin-type layout of [`plugin-layout.md`](plugin-layout.md): `schema/`, `examples/`, `policy/`, `overlay/`, `deploy/` unchanged; `tools/` (was `bin/`); `compiler/` (merge, conflict check, render); `plugins/frontends/` (`yaml`, `nix`, later `guix`, an authoring frontend), `plugins/supervisors/` (`launchd`, `systemd`, `runit`), `plugins/importers/` (`ansible`, `salt`); `contrib/` for in-tree third-party frontends and importers. It becomes one Python package, import name `tendcf`. (3) Frontends and importers are found through the public entry-point groups `tendcf.frontends` and `tendcf.importers`, so a third-party one can also live out of tree as a pip package. Supervisor renderers are in tree only; there is no public `tendcf.supervisors` group, because the goal file's `unit` is a closed `oneOf` and an out-of-tree renderer could only shadow a built-in. (4) Plugins are compile-time only: none runs on a device. The device artifact (`tendcf-validate`, D45) is built from the validator's own source alone, never imports `compiler/`, `plugins/` or `contrib/` and never calls entry-point discovery; the schema lint and the projector never import the compiler; a CI import check is to enforce both (planned). Plugins are trusted build-time code that the operator explicitly allows, not sandboxed: a plugin cannot make a device accept anything that device's own diff does not show, and cannot emit a shape the schema forbids, but it can make the goal file say something other than what the Site Model says, and it sees the site's private data. Release gates (schema lint, projector, preview diff) and signing run in a separate environment with no plugin installed; the compile step holds no signing key. A compile-side preview diff or `explain-hunk` (D44) is compiler output and inherits the plugins' trust. `contrib/`'s lower bar waives support and maintenance promises, never the standing review rule. Out-of-tree trust establishment and plugin management: [#47](https://github.com/frdminc/tendcf/issues/47). (5) Activation: a plugin loads only if site-private names it (name → distribution, version, hash); an installed but unnamed plugin is ignored; a duplicate name in a group is a compile error; built-in names are reserved; until out-of-tree pinning exists (#47) a release build loads in-tree plugins only; the release's `targets.json` `custom` records the plugin set. (6) No users and no backward compatibility yet, so no shims: the compiler's command becomes `tendcf compile`, which replaces `nix2cf buildfile` with no alias (it compiles every host; `--host` renders one); `eval-nix` moves to the Nix frontend; `bin/` → `tools/` leaves no symlink or wrapper and rewrites every path reference in every file, dated records included (djbclark, 2026-11-13, a one-time exception to their append-only convention; only paths change, and prose naming nix2cf as the old repository stays as history); `nix2cf` names the Nix frontend (D12). The compiler renders only the current `schema_version`; older-version rendering is removed in the restructure, and §9.6's ceiling design applies once the first device that is not an operator host enrolls. (7) The keys `nix2cf_services` and `nix2cf_edges` were only ever illustrative: they occurred in guide §16.A's preview-channel example and its copy in the paper, and no schema, fixture, projector or bundle carried them (the projection's keys are `tendcf_service` and `tendcf_interlock`). In the same change that recorded this decision they were renamed `preview_services` and `preview_edges` (illustrative only, chosen so they cannot be mistaken for the `tendcf_services` bundle or the real projected keys `tendcf_service`/`tendcf_interlock`). No schema change is involved ([`plugin-layout.md`](plugin-layout.md) §6). (8) Consequence for [`change-process.md`](change-process.md): the compiler is no longer a separately pinned input, the two-repository bump order and the compiler's `tendcf.lock` go away; `site.lock` loses only its `nix2cf` entry. One tendcf rev is the unit a site pins, so a compiler fix brings the `schema/` and `policy/` commits before it; accepted, with no maintenance-branch machinery, revisited when the first device that is not an operator host enrolls. Moving `bin/` to `tools/` and the plugin split wait until open PRs #41 and #42 merge. Amends D12, D15 (a pointer), D21, §1, §3, §5, §9.6 (when it applies) and Step 3. | Silence = proceed from Step 0. Objections amend this register. @@ -1329,9 +1333,9 @@ with full mesh) is **D38**. - `docs/paper/tendcf-architecture-paper.md` — technical paper; the guide wins on conflict. - `schema/`, `examples/`, `bin/schema_lint.py` — Site Model contract. -- `frdminc/nix2cf` — compiler tool (Step 3), until it is folded into this - repository (D46); archived as soon as that import (#45) merges. - Consumes this contract. +- `frdminc/nix2cf` — compiler tool (Step 3) until 2026-11-13, when it + was folded into this repository as `compiler/` (D46, #45) and archived. +- `compiler/` — the compiler; consumes this contract. - `plugin-layout.md` — the repository's plugin-type layout, the plugin kinds and their two public entry-point groups, the plugin trust and activation rules, and the order of the move (D46). diff --git a/docs/architecture/change-process.md b/docs/architecture/change-process.md index 6186a20..470b55c 100644 --- a/docs/architecture/change-process.md +++ b/docs/architecture/change-process.md @@ -11,8 +11,8 @@ deploy artifact" with a mechanism concrete enough to build. **Amended 2026-11-13 (map D46).** The compiler, built as the separate repository `frdminc/nix2cf`, is folded into tendcf with its history -([#45](https://github.com/frdminc/tendcf/pull/45)), and `frdminc/nix2cf` -is archived as soon as that import merges +under `compiler/` ([#45](https://github.com/frdminc/tendcf/pull/45), +merged 2026-11-13), and `frdminc/nix2cf` was archived the same day ([`plugin-layout.md`](plugin-layout.md)). The compiler is therefore no longer a separately pinned input: schema and compiler change in the same tendcf commit, and the two-repository bump order and the compiler's own @@ -21,9 +21,11 @@ loses only its `nix2cf` entry: it still pins tendcf, site-shared and foreign sites, and tool forks stay as rule 5 has them. One tendcf rev is now the unit a site pins (§8 item 5). Compiler plugins are trusted build-time code; which ones a release loads, and the rule that the gates -and signing run apart from them, are in §3 and §6. Until the move lands, -the compiler's CI still checks out tendcf's default branch beside it (it -never had the `tendcf.lock` specified here). +and signing run apart from them, are in §3 and §6. Since #45 the +compiler's tests run in tendcf's own CI against the same commit (the +`compiler` job of `.github/workflows/check.yml`); before that, nix2cf's +CI checked out tendcf's default branch beside it and never had the +`tendcf.lock` specified here. > The requirement, verbatim from #2: the Site Model → signed-release → > deploy path must be **much faster and much more atomic** than cutting a @@ -67,7 +69,7 @@ device seven releases behind still computes one true diff (guide §7). | Layer (map §3) | Repository today | What a release takes from it | | --- | --- | --- | -| tendcf | `frdminc/tendcf` | schemas, `bin/projector.py`, generic bundles under `policy/`, the upstream overlay (`overlay/`, see [`upstream-overlay.md`](upstream-overlay.md)), and the compiler with its plugins (Step 3; `compiler/` and `plugins/` once the move of map D46 lands, today still `frdminc/nix2cf`, whose one compilation command is `buildfile`) | +| tendcf | `frdminc/tendcf` | schemas, `bin/projector.py`, generic bundles under `policy/`, the upstream overlay (`overlay/`, see [`upstream-overlay.md`](upstream-overlay.md)), and the compiler with its plugins (Step 3; in `compiler/` since map D46's import, #45, whose one compilation command is `buildfile`; `plugins/` after the restructure, [`plugin-layout.md`](plugin-layout.md) §5 step 4) | | site-shared | not yet created | recipes | | foreign site-shared | none yet | recipes, namespaced | | site-private | `~/ops/site-private` (today: ops repo; the tendcf inventory does not exist yet) | inventory, allocations, secret *names*, trust policy, the lockfile | @@ -153,14 +155,17 @@ Three properties follow: 1. **Immutable and content-addressed.** The release number is the TUF `version`; it only goes up (high-water mark on every client, map - §9.11). The content is reproducible from the site-private commit, and + §9.11). The site-private commit identifies the release's sources, and the lockfile hash in `targets.json` is how a reader gets from a release back to its inputs without trusting a branch name. That needs every compiler plugin to come from a pinned input: until out-of-tree pinning exists ([#47](https://github.com/frdminc/tendcf/issues/47)), a release build loads only in-tree plugins named by site-private, so the tendcf pin covers them, and plugins installed on the builder but not named - are ignored (map D46). + are ignored (map D46). Reproducing the release byte for byte also + needs a fixed dependency and toolchain closure for the build and + controlled or recorded external inputs (environment, network); those + are not specified yet. 2. **Per-host targets, one release.** One release carries every host. Hosts that did not change get a byte-identical goal file and compute an empty diff; nothing is applied. This is why "a release" and "a change" @@ -223,8 +228,8 @@ Each repository's CI runs against pinned inputs, never against a sibling's `master`: - **tendcf** has no inputs. Its CI is the schema lint, the cross-reference - lint, the overlay self-test and, once the compiler has moved in (map - D46), the compiler's goldens against the same commit's schemas. A schema + lint, the overlay self-test and, since the compiler moved in (map D46, + #45), the compiler's goldens against the same commit's schemas. A schema change that breaks the compiler fails tendcf's own CI, in the commit that makes it. That is a statement about the current `schema_version`, the only one the compiler renders until map §9.6's ceiling design @@ -251,7 +256,7 @@ one consistent set. | tendcf | `bin/xref_lint.py` | yes | | tendcf | `bin/overlay.py self-test` and `lint` (patch overlay applies; drift fails loudly) | yes, this change | | tendcf | overlay drift against upstream (`check --against upstream/master`) | scheduled / manual job, this change | -| tendcf (compiler) | compile the tendcf fixtures at the same commit; goldens byte-identical | in `frdminc/nix2cf` CI today, against tendcf's default branch; moves into tendcf CI with the compiler (map D46) | +| tendcf (compiler) | compile the tendcf fixtures at the same commit; goldens byte-identical | yes: the `compiler` job of `.github/workflows/check.yml`, since #45 (map D46); before that in `frdminc/nix2cf` CI, against tendcf's default branch | **Release gates** (site-private, every commit; a pull request gets the same run with signing and publishing skipped, and the per-host diffs attached as diff --git a/docs/architecture/plugin-layout.md b/docs/architecture/plugin-layout.md index 1e67aba..fbda0e2 100644 --- a/docs/architecture/plugin-layout.md +++ b/docs/architecture/plugin-layout.md @@ -10,8 +10,9 @@ 1. `frdminc/nix2cf`, the compiler, folds into `frdminc/tendcf` with its full git history, first under `compiler/`. A separate pull request - ([#45](https://github.com/frdminc/tendcf/pull/45)) does the import; - this document moves no files. + ([#45](https://github.com/frdminc/tendcf/pull/45)) did the import; it + merged on 2026-11-13 (merge commit `ca4fa90`). This document moves no + files. 2. The repository takes the plugin-type layout of §2. It becomes one Python package, import name `tendcf` (today's `nix2cf` package is renamed; how the directories map onto it is open, §7). @@ -28,6 +29,8 @@ the Nix frontend. 7. The compiler renders only the current `schema_version` (§4). 8. `frdminc/nix2cf` is archived as soon as the import (#45) merges (§5). + Done 2026-11-13: archived the same day #45 merged, its README pointing + here. The consequence for the change process: the compiler is no longer a separately pinned input. Schema and compiler change in the same commit, @@ -75,7 +78,7 @@ tendcf/ | `schema/`, `examples/` | exist in tendcf | unchanged | | `policy/`, `overlay/`, `deploy/` | exist in tendcf | unchanged | | `tools/` | is `bin/`: standalone `uv run --script` files; tendcf has no `pyproject.toml` | `bin/` renamed `tools/` after #41 and #42 merge, with no symlink or wrapper left behind and every reference rewritten (§5, step 4) | -| `compiler/` | `frdminc/nix2cf` (#45 imports it as `compiler/`): `nix2cf/cli.py`, `compile.py`, `sitemodel.py`. Two subcommands: `buildfile`, the implemented compilation command, which renders one host; and `eval-nix`, a plain-expression evaluation hook, not the D12 module frontend | the compiler of the `tendcf` package; command `tendcf compile` (§6) | +| `compiler/` | exists since #45 (2026-11-13), imported from the now archived `frdminc/nix2cf`: package `nix2cf` (`nix2cf/cli.py`, `compile.py`, `sitemodel.py`), its tests run by the `compiler` job of `.github/workflows/check.yml`. Two subcommands: `buildfile`, the implemented compilation command, which renders one host; and `eval-nix`, a plain-expression evaluation hook, not the D12 module frontend | the compiler of the `tendcf` package; command `tendcf compile` (§6) | | `plugins/frontends/yaml/` | `nix2cf/sitemodel.py` (YAML loading with duplicate-key refusal and line numbers) | split out of the compiler, still built in | | `plugins/frontends/nix/` | only the hook: `nix2cf eval-nix FILE` evaluates a plain Nix expression with `nix-instantiate --eval --strict --json`; no module frontend | the D12 Nix module frontend, named `nix2cf`; `eval-nix` moves here | | `plugins/frontends/guix/` | nothing | later | @@ -195,8 +198,11 @@ inherits the plugins' trust. 4. Until out-of-tree pinning exists ([#47](https://github.com/frdminc/tendcf/issues/47)), a release build loads in-tree plugins only (`plugins/` and `contrib/`, at the pinned - tendcf rev). That way the tendcf pin covers the plugin set, and a - release stays reproducible from its site-private commit. + tendcf rev). That way the tendcf pin identifies the source of the + compiler and of every plugin a release loads. Reproducing a release + byte for byte also needs a fixed dependency and toolchain closure and + controlled or recorded external inputs (environment, network); those + controls are not specified yet (§7). 5. The release's `targets.json` `custom` records the plugin set ([`change-process.md`](change-process.md) §3). @@ -237,8 +243,8 @@ and the multi-version render returns with it. | # | Change | State | | --- | --- | --- | | 1 | This documentation: the guide, the paper, the map (D46), the change process, the Linux targets plan, the glossary, this file; and the rename of the illustrative `nix2cf_*` keys (§6). Moves no files and changes no code or schema. | this change | -| 2 | Import `frdminc/nix2cf` with full history under `compiler/`; the root `README.md` follows. | [#45](https://github.com/frdminc/tendcf/pull/45) | -| 3 | Archive `frdminc/nix2cf`. | as soon as #45 merges | +| 2 | Import `frdminc/nix2cf` with full history under `compiler/`; the root `README.md` follows. | done: [#45](https://github.com/frdminc/tendcf/pull/45), merged 2026-11-13 (`ca4fa90`) | +| 3 | Archive `frdminc/nix2cf`. | done 2026-11-13, right after #45 merged (README points here) | | 4 | The restructure, one pull request: `bin/` → `tools/`; the Python package renamed `tendcf`; the compiler split into `compiler/` and `plugins/` with the two public entry-point groups; `tendcf compile` replaces `nix2cf buildfile`; older-version rendering removed; the device-boundary import check (§4). No alias, symlink or wrapper. Waits until open PRs #41 and #42 merge. | planned | Step 4 rewrites every path reference in every file, dated records @@ -252,12 +258,19 @@ on 2026-11-13 are listed below (`rg 'nix2cf'` and outside dated records). Rerun both over the whole tree before the change, because the list moves: -1. **The compiler's in-tree check** (from #45): - `compiler/nix2cf/cli.py` accepts an enclosing tendcf checkout only if - it has `bin/projector.py` (`_in_tree_tendcf`), loads the projector from - that path (`load_projector`) and names it in an error message; - `compiler/nix2cf/__init__.py` and `compile.py` cite `bin/projector.py` - and `bin/schema_lint.py` in their docstrings. +1. **The compiler's in-tree check, and the rest of `compiler/`** (from + #45): `compiler/nix2cf/cli.py` accepts an enclosing tendcf checkout + only if it has `bin/projector.py` (`_in_tree_tendcf`), loads the + projector from that path (`load_projector`) and names it in an error + message; `compiler/nix2cf/__init__.py`, `compile.py` and + `sitemodel.py` cite `bin/projector.py` or `bin/schema_lint.py` in + docstrings and comments; so do `compiler/README.md` (links to + `../bin/…`), `compiler/docs/DESIGN.md`, `compiler/pyproject.toml`, + `compiler/examples/site/launchd-writers.yml` and + `compiler/tests/test_compile.py`. The package name `nix2cf` itself + (`compiler/pyproject.toml`, `compiler/AGENTS.md`, + `compiler/scripts/check_schema_version.py`, the tests and examples) + changes with the rename to `tendcf`. 2. **Schema description text**: `schema/common.schema.json` (three descriptions say nix2cf infers edges or fills in a field; under the new names that is the compiler), `schema/services.schema.json` and @@ -269,15 +282,18 @@ change, because the list moves: `examples/broken/README.md`. 4. **Repository files**: `AGENTS.md` (the standing rule names nix2cf as a repository, and its description of `check.yml` names the `bin/` - tools), + tools; its rule that `docs/handoffs/` is append-only and never + rewritten yields to the one-time path exception above, for paths + only), `README.md`, `overlay/overlay.json`, `overlay/README.md`, `.github/workflows/check.yml`, `.githooks/pre-commit`, `deploy/linux/canary/check.py`, `tests/unit_classifier/test_logic.py`, and the tools' own cross-references inside `bin/`. 5. **Documents**: map D45 (`bin/projector.py` as the source of - `tendcf-validate`), and every living document `bin/xref_lint.py` lints. - Dated records (handoffs, reviews, reconciliations, opinions) keep the - old paths as history and are not rewritten. + `tendcf-validate`), every living document `bin/xref_lint.py` lints, + and, under the exception above, every dated record (handoffs, reviews, + reconciliations, opinions, `deprecated/`): their paths are rewritten, + their prose is not. ## 6. Names @@ -288,10 +304,10 @@ change, because the list moves: (guide §4) is kept in prose only; no subcommand keeps the name. 2. **`nix2cf`** is the Nix module frontend, `plugins/frontends/nix/`. Up to 2026-11-13 the name meant the whole compiler; older documents and - handoffs use it that way and are not rewritten. Until the restructure - the only thing that ships under the name is still the old compiler - (`frdminc/nix2cf`, then `compiler/` after #45, command - `nix2cf buildfile`). + handoffs use it that way, and that prose is not rewritten (only their + paths change, §5). Until the restructure the only thing that ships + under the name is still the old compiler (`frdminc/nix2cf`, since #45 + `compiler/`, command `nix2cf buildfile`). 3. **`guix`** is the planned Guix authoring frontend, `plugins/frontends/guix/`. 4. **The illustrative `preview_services` / `preview_edges` keys.** The @@ -322,9 +338,12 @@ change, because the list moves: - Out-of-tree pinning and trust establishment, after which a release build may load an out-of-tree plugin ([#47](https://github.com/frdminc/tendcf/issues/47)). -- Whether the compile step has network access. The guide calls the render - a pure function of the Site Model (guide §4); with plugins that holds - only for what the release environment controls. +- Whether the compile step has network access, the dependency and + toolchain closure a release build runs in, and how external inputs are + controlled or recorded. The guide calls the render a pure function of + the Site Model (guide §4); until these are specified, a pinned + site-private commit identifies a release's sources but does not by + itself reproduce its bytes. - Which importers, if any, get built. - Guix System hosts with Shepherd as their supervisor (§3). - Whether the PyPI names `tendcf` and `nix2cf` are free; unchecked. diff --git a/docs/paper/tendcf-architecture-guide.md b/docs/paper/tendcf-architecture-guide.md index 458d9ab..19fd0bc 100644 --- a/docs/paper/tendcf-architecture-guide.md +++ b/docs/paper/tendcf-architecture-guide.md @@ -24,7 +24,8 @@ and must agree with this guide. The technical paper, with citations and open questions in research form, is [`tendcf-architecture-paper.md`](tendcf-architecture-paper.md). -**Nothing described here is deployed.** The data formats exist and are +**Nothing described here is deployed**, apart from a canary service on +one operator host (§4). The data formats exist and are checked, a first generic policy bundle runs, and the projection step has a reference implementation. The compiler has its first piece (`buildfile`, §4); the rest of it, the on-device executor, and the screens a person @@ -115,9 +116,9 @@ Git-repo count is not the composition mechanism. Layers are roles. Site-private holds a lockfile (flake-style inputs) that pins tendcf, site-shared, foreign shared sites, and optional tool forks. The compiler is part of tendcf, so one tendcf pin covers schemas and compiler together -(decided 2026-11-13; until the move lands the compiler is still the -separate `frdminc/nix2cf` repository, see -[`plugin-layout.md`](../architecture/plugin-layout.md)). The +(decided 2026-11-13 and done the same day: the compiler is under +`compiler/`, and the separate `frdminc/nix2cf` repository is archived; +see [`plugin-layout.md`](../architecture/plugin-layout.md)). The signed release is the deploy artifact. There is no lockstep of sibling checkouts that must share one tag. @@ -144,8 +145,8 @@ Nix-for-builds are *consumers*. Any one of them can be replaced without moving the facts. **Schemas and types live in tendcf's `schema/`**, not in the compiler's -code, although the compiler is moving into the same repository (decided -2026-11-13, §4). A schema change is an interface change. A hostname +code, although the compiler now lives in the same repository (`compiler/`, +since 2026-11-13, §4). A schema change is an interface change. A hostname change is site data. They do not move together. The files are [`schema/`](../../schema/) and [`examples/`](../../examples/) in this repository. @@ -224,9 +225,10 @@ not the home of schemas. **Name and home (decided 2026-11-13).** The compiler was built as the separate repository `frdminc/nix2cf`, whose `nix2cf buildfile` command is -the first piece described below. It is being folded into tendcf with its -history, under `compiler/`, and that repository is archived as soon as -the import merges. The command becomes `tendcf compile`, which replaces +the first piece described below. It was folded into tendcf with its +history, under `compiler/`, on 2026-11-13, and that repository is +archived. In the restructure still to come the command becomes +`tendcf compile`, which replaces `nix2cf buildfile` outright, with no alias: it compiles every host, and `--host` renders one. The name `nix2cf` moves to what it describes: the Nix module frontend (§3), one of the compiler's plugins. Frontends, @@ -239,7 +241,9 @@ device. A plugin is trusted build-time code that the operator explicitly allows, not sandboxed code, and the claim it supports is narrow. A plugin cannot make a device accept anything that device's own diff does not show (§7), -and it cannot emit a shape the schema forbids. It can make the goal file +and it cannot emit a shape the schema forbids. (An operator host applies +without a local yes, so there the pull request's preview diff is the only +human check.) It can make the goal file say something other than what the Site Model says, and it sees the site's private data. So the release gates (schema lint, projector, preview diff) and signing run in a separate environment with no plugin @@ -292,11 +296,15 @@ Merge happens once, in the compiler, before render. CFEngine’s own Because the render is a pure function of the Site Model — same input, same output — “show me exactly what device X would receive, without touching -device X” is almost free. That command, `buildfile`, was the first piece -of the compiler built: merge, conflict check and goal-file render, with -service units for launchd only and no inference yet (§18). It is how an -agent checks its own work, how the compiler regression-tests itself, and -how a human sees a proposed change before it lands. The shape is +device X” is almost free. (That purity is a design goal: with plugins it +also needs a fixed dependency and toolchain closure and controlled +external inputs, which are not specified yet; +[`plugin-layout.md`](../architecture/plugin-layout.md) §7.) That command, +`buildfile`, was the first piece of the compiler built: merge, conflict +check and goal-file render, with service units for launchd only and no +inference yet (§18). It is how an agent checks its own work, how the +compiler regression-tests itself, and how a human sees a proposed change +before it lands. The shape is [Bcfg2 `buildfile`](https://docs.bcfg2.org/). That lineage survives in prose only: from the restructure on, the command is `tendcf compile --host NAME`, and no subcommand keeps the name @@ -486,7 +494,10 @@ reviewed diff. Devices are not stranded by that rule, because goal files are per-host: the compiler renders each host at the highest schema version that host last reported it can read (§6 is where that report comes from), and a schema change ships as two releases — the validator update first, as an -ordinary diff, and the migration behind it. +ordinary diff, and the migration behind it. Decided 2026-11-13: until the +first device that is not an operator host enrolls, the compiler renders +only the current schema version; the per-host render described here +returns then (map §9.6). **Accept is all or nothing.** A refusal may name the entries that caused it; it never applies the remainder. The proposer withdraws the corresponding @@ -1330,9 +1341,10 @@ observe themselves crossing is a disclaimer rather than a ceiling. resolver and the release pipeline are Step 6 code. - **The compiler's first piece**, `nix2cf buildfile` (§4): merge, conflict check and render of one host's goal file, with service units - for launchd only. Its tests run in CI against tendcf's schemas and - reference projector. It lives in the separate `frdminc/nix2cf` - repository until the move into tendcf decided on 2026-11-13 lands. + for launchd only, in [`compiler/`](../../compiler/) (imported from the + now archived `frdminc/nix2cf` on 2026-11-13). Its tests run in + tendcf's own CI against the same commit's schemas and reference + projector. **Not built — this is still most of it** @@ -1352,7 +1364,7 @@ coherent, describable state): | 0 | Schemas in tendcf | Existing contract is `schema/` + `examples/` + `bin/schema_lint.py`. Remaining: peer_actions, trust-policy shape, generic unit-writers, lookup stub, YAML canonicalize. Transcribe reality (`not-yet-migrated` is the correct day-one state). | | 1 | macOS services adapter | **Started.** `policy/tendcf_services.cf` renders and loads launchd services from projected data, with interlocks enforced (§16.C; the interlock verdict was split into its own bundle on 2026-10-29 after the pass path was found never to open). Still open in this step: package and file promises, environment rendering, the unit-writer detector. Dry-run default. Not in this step: nix-darwin. | | 2 | Android under the Site Model | Same vocabulary. Agent owns JSONL+SQLite in app-private storage. | -| 3 | Compiler (`tendcf compile`, was `nix2cf`) | “What would device X receive?” first, then conflict check, extra-entry reporting, then inference. Inference waits until types exist on two platforms. **Started:** `buildfile` (merge, conflict check, render) exists in `frdminc/nix2cf`; it moves into tendcf under `compiler/` and is split into plugins ([`plugin-layout.md`](../architecture/plugin-layout.md)). | +| 3 | Compiler (`tendcf compile`, was `nix2cf`) | “What would device X receive?” first, then conflict check, extra-entry reporting, then inference. Inference waits until types exist on two platforms. **Started:** `buildfile` (merge, conflict check, render) exists in `compiler/` (imported from `frdminc/nix2cf` on 2026-11-13); the restructure splits it into plugins ([`plugin-layout.md`](../architecture/plugin-layout.md)). | | 4 | Linux reference path | A stock distro, not NixOS. Distro choice is open until this step; Ubuntu Server is the working default. | | 5 | First real Linux host | A second node that can hold backup roles. Proves that roles are data. | | 6 | Signed releases, push-only | TUF-subset ceremony; goal-file render and diff; the on-device validator. Operator hosts only. | diff --git a/docs/paper/tendcf-architecture-paper.md b/docs/paper/tendcf-architecture-paper.md index ee691b4..75433f0 100644 --- a/docs/paper/tendcf-architecture-paper.md +++ b/docs/paper/tendcf-architecture-paper.md @@ -200,8 +200,9 @@ Git-repo count is not the composition mechanism. Layers are roles: Site-private holds a lockfile (flake-style inputs) that pins tendcf, site-shared, foreign shared sites, and optional tool forks. The compiler is part of tendcf, so one tendcf pin covers schemas and compiler together -(decided 2026-11-13; until the move lands the compiler is still the -separate `frdminc/nix2cf` repository). The signed release is the deploy +(decided and done 2026-11-13: the compiler is under `compiler/`, and the +separate `frdminc/nix2cf` repository is archived). The signed release is +the deploy artifact. There is no lockstep of sibling checkouts that must share one tag. Collision of the same identity from two peer inputs is a **compile error**. Only site-private may bind a winner @@ -259,8 +260,9 @@ The compiler is a tool, not the home of schemas. It was built as the separate repository `frdminc/nix2cf`, whose `nix2cf buildfile` command is its first piece. On 2026-11-13 it was decided to fold that repository into tendcf with its history, under `compiler/` -(archiving it as soon as the import merges), to replace `nix2cf buildfile` -with `tendcf compile` outright, with no alias, and to give the name +(done the same day, and the old repository archived), to replace +`nix2cf buildfile` with `tendcf compile` outright in a later +restructure, with no alias, and to give the name `nix2cf` to what it describes: the optional Nix module frontend of §2.1. Frontends, supervisor renderers and one-shot importers from other tools become plugins; frontends and importers are found through Python entry From fd436798f1e8b0e4d3dbb4c5d02b83d31d8d8713 Mon Sep 17 00:00:00 2001 From: Daniel JB Clark Date: Fri, 13 Nov 2026 06:52:11 -0500 Subject: [PATCH 8/9] =?UTF-8?q?paper=20=C2=A72.2:=20qualify=20render=20pur?= =?UTF-8?q?ity=20to=20match=20the=20guide=20(Astra=20A2)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 --- docs/paper/tendcf-architecture-paper.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/paper/tendcf-architecture-paper.md b/docs/paper/tendcf-architecture-paper.md index 75433f0..33f38f0 100644 --- a/docs/paper/tendcf-architecture-paper.md +++ b/docs/paper/tendcf-architecture-paper.md @@ -325,7 +325,10 @@ and not a schema redesign. Because the render is a pure function of the Site Model — the same purity Nix's own build model is named for [27] — "show me exactly what device X -would receive, without touching device X" is nearly free. We built that +would receive, without touching device X" is nearly free. (That purity is +a design goal: with plugins it also needs a fixed dependency and toolchain +closure and controlled external inputs, which are not specified yet; +[`plugin-layout.md`](../architecture/plugin-layout.md) §7.) We built that affordance *first*, before the pipeline is finished — `buildfile` merges, conflict-checks and renders one host's goal file, with service units for launchd only and no inference yet — and §6.2 explains why it earns its place From a13e892eafc9695ba02ac7ef7a786d3e3bded886 Mon Sep 17 00:00:00 2001 From: Daniel JB Clark Date: Fri, 13 Nov 2026 06:57:10 -0500 Subject: [PATCH 9/9] docs: Fable final-check leftovers X1-X3 (paper canary clause, purity caveat, two sentences) Co-Authored-By: Claude Opus 5.5 --- docs/architecture/change-process.md | 2 +- docs/paper/tendcf-architecture-guide.md | 4 ++-- docs/paper/tendcf-architecture-paper.md | 7 ++++--- 3 files changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/architecture/change-process.md b/docs/architecture/change-process.md index 470b55c..d85638f 100644 --- a/docs/architecture/change-process.md +++ b/docs/architecture/change-process.md @@ -69,7 +69,7 @@ device seven releases behind still computes one true diff (guide §7). | Layer (map §3) | Repository today | What a release takes from it | | --- | --- | --- | -| tendcf | `frdminc/tendcf` | schemas, `bin/projector.py`, generic bundles under `policy/`, the upstream overlay (`overlay/`, see [`upstream-overlay.md`](upstream-overlay.md)), and the compiler with its plugins (Step 3; in `compiler/` since map D46's import, #45, whose one compilation command is `buildfile`; `plugins/` after the restructure, [`plugin-layout.md`](plugin-layout.md) §5 step 4) | +| tendcf | `frdminc/tendcf` | schemas, `bin/projector.py`, generic bundles under `policy/`, the upstream overlay (`overlay/`, see [`upstream-overlay.md`](upstream-overlay.md)), and the compiler with its plugins (Step 3; in `compiler/` since map D46's import (#45); its one compilation command is `buildfile`; `plugins/` after the restructure, [`plugin-layout.md`](plugin-layout.md) §5 step 4) | | site-shared | not yet created | recipes | | foreign site-shared | none yet | recipes, namespaced | | site-private | `~/ops/site-private` (today: ops repo; the tendcf inventory does not exist yet) | inventory, allocations, secret *names*, trust policy, the lockfile | diff --git a/docs/paper/tendcf-architecture-guide.md b/docs/paper/tendcf-architecture-guide.md index 19fd0bc..4be4c25 100644 --- a/docs/paper/tendcf-architecture-guide.md +++ b/docs/paper/tendcf-architecture-guide.md @@ -296,10 +296,10 @@ Merge happens once, in the compiler, before render. CFEngine’s own Because the render is a pure function of the Site Model — same input, same output — “show me exactly what device X would receive, without touching -device X” is almost free. (That purity is a design goal: with plugins it +device X” is almost free. (That purity is a design goal: it also needs a fixed dependency and toolchain closure and controlled external inputs, which are not specified yet; -[`plugin-layout.md`](../architecture/plugin-layout.md) §7.) That command, +[`plugin-layout.md`](../architecture/plugin-layout.md) §7.) The command for it, `buildfile`, was the first piece of the compiler built: merge, conflict check and goal-file render, with service units for launchd only and no inference yet (§18). It is how an agent checks its own work, how the diff --git a/docs/paper/tendcf-architecture-paper.md b/docs/paper/tendcf-architecture-paper.md index 33f38f0..b2eeab3 100644 --- a/docs/paper/tendcf-architecture-paper.md +++ b/docs/paper/tendcf-architecture-paper.md @@ -38,7 +38,8 @@ signed release carrying a typed operation plan that the on-device executor mechanically refuses to exceed. Bcfg2 supplies four of its load-bearing ideas, credited in §6. -**Nothing described here is deployed.** No device has been provisioned from +**Nothing described here is deployed**, apart from a canary service on +one operator host. No device has been provisioned from factory reset by this automation; we have no operational numbers of any kind, and one of our central negative results (§5.4) rests on reasoning about a code path that has never been executed. What we do have is an @@ -326,7 +327,7 @@ and not a schema redesign. Because the render is a pure function of the Site Model — the same purity Nix's own build model is named for [27] — "show me exactly what device X would receive, without touching device X" is nearly free. (That purity is -a design goal: with plugins it also needs a fixed dependency and toolchain +a design goal: it also needs a fixed dependency and toolchain closure and controlled external inputs, which are not specified yet; [`plugin-layout.md`](../architecture/plugin-layout.md) §7.) We built that affordance *first*, before the pipeline is finished — `buildfile` merges, @@ -1761,7 +1762,7 @@ the rest), plus the open parts of those two. Remaining Step 0 work: `peer_actions`, trust-policy shape, generic unit-writers, lookup stub, YAML canonicalize, and transcribing existing reality as `not-yet-migrated`. -**Nothing is deployed.** No device has been provisioned from factory reset by +**Nothing is deployed** beyond that one operator-host canary. No device has been provisioned from factory reset by this automation. We have no deployment time, no effort figure, no managed/unmanaged ratio, and no failure data. Examples A and B in §2.6 remain hand-authored to show the target rendering; only Example C is real.