Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
197 changes: 182 additions & 15 deletions AGENTS.md

Large diffs are not rendered by default.

480 changes: 422 additions & 58 deletions docs/architecture/architecture-DEFINITIVE-v3.md

Large diffs are not rendered by default.

25 changes: 18 additions & 7 deletions docs/architecture/change-process.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,8 +112,12 @@ Rules for the file, each with the reason it is a rule:
commit, reachable from what branch); `tree` is what the build verifies
after fetching, because a rewritten branch can move `rev` out from under
a `ref` while `tree` still proves the content. Both are plain git
objects: no Nix is needed to compute or check them, which R5 and D6
require (Nix is an optional builder, never a dependency of the path).
objects: no Nix is needed to compute or check them. Nix may be required
to build a release; devices never need Nix at run time (operator ruling,
2026-11-14, B4 rider "Nix on release path": "2. Allow on release path",
scope "1. Build-time only"). This amends the earlier reading of R5/D6
(Nix an optional builder, never a dependency of the release path); the
map's R5/D6 text is amended to match there.
3. **A build never follows `ref`, but `ref` is verified.** The resolver
updates `rev`/`tree` from `ref` on request, and a build never replaces
the pinned `rev` with the tip of the moving `ref`. Gate 1 does fetch
Expand All @@ -127,7 +131,11 @@ Rules for the file, each with the reason it is a rule:
is content inside the tendcf input; a built binary or the Android
agent's APK enters a release only as a digest-bound fetched artifact
(map §9.8, DC-11). The lockfile
pins sources, not binaries.
pins sources, not binaries. A person's own fork (G9, later) is not a
tool fork in this sense. Its build, like a tool fork's, reaches a device
only as a digest-bound artifact; its upstream pin is in site.lock, and
its patches are site content. (Operator ruling, 2026-11-14, Smaller 4,
X-forks-05: "1. Yes, a site.lock source input".)
6. **A changed `url` is a privileged change** (decision D-d, issue #48;
planned, Step 6). Gate 1 below verifies that `rev` belongs to `ref` at
`url` and that `tree` matches, so a pull request that repoints a pin
Expand Down Expand Up @@ -199,7 +207,7 @@ Three properties follow:
exists ([#47](https://github.com/frdminc/tendcf/issues/47)), a release
build loads only in-tree plugins named by site-private, so the tendcf
pin covers them, and plugins installed on the builder but not named
are not selected (map D46; installing one can still run code, plugin-layout §4). Reproducing the release byte for byte also
are not selected (map D46; installing one can still run code, plugin-layout §4). Out-of-tree discovery is itself deferred until a user needs it and out-of-tree plugins can be pinned (operator ruling, 2026-11-14, Smaller 5). Reproducing the release byte for byte also
needs a fixed dependency and toolchain closure for the build and
controlled or recorded external inputs (environment, network); those
are not specified yet.
Expand Down Expand Up @@ -427,7 +435,7 @@ the review artifact; a diff computed in the compile job never is):
verifies the pins itself; jobs B′ and B take nothing from job A but
goal-file bytes and the digest manifest (plugin-layout §4).
2. Compile every host, loading only the compiler plugins site-private
names (in-tree only until #47; map D46; the loader is planned,
names (in-tree only; out-of-tree discovery is deferred, with #47, until a user needs it and plugins can be pinned: operator ruling, 2026-11-14, Smaller 5; map D46; the loader is planned,
[`plugin-layout.md`](plugin-layout.md) §4). Conflict-as-error (map §3):
the same identity from two peer inputs without a private bind refuses
the whole release, and so does a duplicate plugin name within a group.
Expand Down Expand Up @@ -711,8 +719,11 @@ moved on since.
says), and makes the public engine's history irrelevant to its users.
5. **Nix `flake.lock` as the mandatory lock.** The shape is right and the
resolver is battle-tested, but it needs Nix to compute `narHash`, which
makes Nix a dependency of the release path on the builder; R5/D6 keep Nix
optional. `site.lock` keeps the two git hashes Nix does not need and can
makes Nix a dependency of the release path on the builder. The operator
has since allowed Nix on the release path, build-time only (§2 rule 2;
operator ruling, 2026-11-14, B4 rider "Nix on release path"), so Nix is
no longer excluded on principle; this choice stands on the other
grounds: `site.lock` keeps the two git hashes Nix does not need and can
be converted into flake inputs when a Nix builder exists (Step 7/10+).
6. **Devices reading git directly** (a device fetches the policy tree from
a branch). This is what "git-synced policy" is sometimes taken to mean and
Expand Down
92 changes: 67 additions & 25 deletions docs/architecture/plugin-layout.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,23 @@
> code enforces yet is tagged "planned", with its step.** Today there is
> no plugin loader, no release pipeline, no gate separation, no import
> step and no device-side validation command.
>
> **Amended 2026-11-14 (operator ruling, 2026-11-14, Smaller 5 (D46
> out-of-tree plugin entry points, G-§4-01 / map Step 0, v2 C3), "1. Defer
> out-of-tree discovery"; map D46, amendment at the end of the row).** All
> plugins live in tree, run at compile time only and never on a device.
> Python entry-point discovery of out-of-tree plugins is **deferred until a
> user needs one and out-of-tree plugins can be pinned** (both, not either;
> pinning is [#47](https://github.com/frdminc/tendcf/issues/47)). The
> deliverable of restructure step 4 (§5) is the **static in-tree import
> table**; the public entry-point groups `tendcf.frontends` and
> `tendcf.importers` are not built, and no release build reads an entry
> point. Every passage below that describes third-party entry-point
> discovery (§1 item 3, §2's entry-point row, §3's table, §4's activation
> steps 2 and 5 and the "Out of tree" item, §5 step 4, §7's #47 item) is
> now **deferred design, kept as history** and marked "deferred" where it
> stands; it takes effect only if both conditions are met (a user needs
> one, and out-of-tree plugins can be pinned).

## 1. The decision

Expand All @@ -26,14 +43,19 @@
2. The repository takes the plugin-type layout of §2. It becomes one
Python package, import name `tendcf` (today's `nix2cf` package is
renamed; how the directories map onto it is open, §7).
3. A third-party frontend or importer can live outside this repository as
3. ~~A third-party frontend or importer can live outside this repository as
an ordinary pip package, found through the public Python entry-point
groups `tendcf.frontends` and `tendcf.importers` (§3). Every plugin in
this repository (frontends, supervisor renderers and importers) is
found through a static import table, never through entry points (§4,
activation; planned, restructure step 4: no table exists today).
Supervisor renderers are in tree only; there is no public
`tendcf.supervisors` group (§3).
groups `tendcf.frontends` and `tendcf.importers` (§3).~~ **Amended
2026-11-14 (Smaller 5, "1. Defer out-of-tree discovery"):** all plugins
live in tree and run only at compile time; entry-point discovery of
out-of-tree plugins is deferred until a user needs one and out-of-tree
plugins can be pinned, and the public groups are not built until then
(the deferred design stays in §3 and §4). Every plugin in this
repository (frontends, supervisor renderers and importers) is found
through a static import table, never through entry points (§4,
activation; planned, restructure step 4: no table exists today; this
table is the deliverable). Supervisor renderers are in tree only; there
is no public `tendcf.supervisors` group (§3).
4. Plugins run at compile time only, and they are trusted build-time code
that the operator explicitly allows, not sandboxed code (§4).
5. A plugin is selected only if site-private names it (§4, activation;
Expand Down Expand Up @@ -102,7 +124,7 @@ tendcf/
| `plugins/supervisors/systemd/`, `runit/` | nothing: a service selected for a systemd or runit host is a compile error (`RENDERED_SUPERVISORS` in `compile.py`) | written in tree ([`linux-targets.md`](linux-targets.md) §3, step 6) |
| `plugins/importers/` | nothing | optional |
| `contrib/` | nothing | as third parties contribute |
| entry-point groups | none; nix2cf's `pyproject.toml` declares only the `nix2cf` script | the two public groups of §3, read for third-party plugins only; in-tree plugins are found through a static import table (§4) |
| entry-point groups | none; nix2cf's `pyproject.toml` declares only the `nix2cf` script | none for now: the two public groups of §3 are deferred (Smaller 5, 2026-11-14); in-tree plugins are found through a static import table (§4) |

A supervisor plugin renders a service record into the goal file's `unit`
entry for that supervisor and resolves that supervisor's defaults (for
Expand All @@ -115,10 +137,19 @@ Wherever it lands, the device boundary rules of §4 apply to it.

## 3. Plugin kinds and entry-point groups

> **Amended 2026-11-14 (operator ruling, 2026-11-14, Smaller 5, "1. Defer
> out-of-tree discovery"):** the two public groups in this table are the
> **deferred design**, not a deliverable. All plugins live in tree and are
> found through a static import table; discovery of out-of-tree plugins by
> Python entry points waits until a user needs one **and** out-of-tree
> plugins can be pinned ([#47](https://github.com/frdminc/tendcf/issues/47)).
> The table below keeps the old design as history, so a later ruling can
> revive it without rewriting it.

| Kind | Entry-point group | What it is |
| --- | --- | --- |
| Frontend | `tendcf.frontends` (public; third-party plugins only, §4) | A language a site keeps authoring in. It turns its source into the Site Model's JSON, which the compiler then merges, checks and renders like any other. YAML and Nix are the two equal frontends of map D12; Guix may follow as an authoring frontend (Guile records → Site Model JSON) once it has a sandbox design of its own (§4). |
| Importer | `tendcf.importers` (public; third-party plugins only, §4) | An optional one-shot migration from another tool's configuration (Ansible, Salt) into Site Model files, which the site then keeps and edits as its own. Not run on every compile. |
| Frontend | `tendcf.frontends` (deferred: public group for third-party plugins only, §4) | A language a site keeps authoring in. It turns its source into the Site Model's JSON, which the compiler then merges, checks and renders like any other. YAML and Nix are the two equal frontends of map D12; Guix may follow as an authoring frontend (Guile records → Site Model JSON) once it has a sandbox design of its own (§4). |
| Importer | `tendcf.importers` (deferred: public group for third-party plugins only, §4) | An optional one-shot migration from another tool's configuration (Ansible, Salt) into Site Model files, which the site then keeps and edits as its own. Not run on every compile. |
| Supervisor renderer | none: in tree only | A unit renderer for one supervisor (launchd, systemd, runit): service record in, goal-file `unit` entry out (§2). |

The difference between a frontend and an importer is what happens next. A
Expand Down Expand Up @@ -147,7 +178,7 @@ decided (§7).
Built-in frontends, supervisor renderers and importers are in tree, are
to be found through a static import table and never through entry points
(planned, §4: no table exists today), and their names are reserved
against third-party entry points (§4). The
against third-party entry points should those be built (deferred, §4). The
Python interface a plugin implements, and the table's format, are not
specified yet; they are to be settled when the split is done (§5, step 4).

Expand Down Expand Up @@ -787,7 +818,13 @@ the preview diff is **advisory**. The rule:

**Activation: a plugin is selected only if site-private names it** (the
loader is **planned**, restructure step 4, and Step 6 for the release
checks; no loader exists today). The steps run in this order:
checks; no loader exists today). The steps run in this order.
**Deferral (operator ruling, 2026-11-14, Smaller 5, "1. Defer out-of-tree
discovery"):** the deliverable is steps 1, 3 (for in-tree plugins), 4, 6, 7
and 8 over the static import table. Step 2 and the third-party parts of
steps 3 and 5 describe out-of-tree discovery, which is deferred until a
user needs it and out-of-tree plugins can be pinned (#47); they are kept
as the deferred design.

1. **In-tree plugins come from a static import table.** A plugin is
in-tree when its module's realpath, with symlinks resolved, sits under
Expand All @@ -807,8 +844,10 @@ checks; no loader exists today). The steps run in this order:
resolves each table entry to a file, checks that file's realpath, and
only then executes it by file location, the way `load_projector` in
`compiler/nix2cf/cli.py` already loads the projector.
2. **Entry points are read only for third-party plugins**, and only after
[#47](https://github.com/frdminc/tendcf/issues/47) exists. The loader
2. **Deferred: entry points are read only for third-party plugins**, and
only after [#47](https://github.com/frdminc/tendcf/issues/47) exists
(until then they are not read at all, so there is no loader code for
this step). The loader
lists every entry point, with no deduplication by distribution name
(`importlib.metadata.entry_points()` drops a second distribution with
the same name, so "a duplicate name is a compile error" could never
Expand All @@ -835,9 +874,9 @@ checks; no loader exists today). The steps run in this order:
4. **Importers follow the same rules as frontends**: an importer runs
only if it is named and pinned. Which importers an operator may run is
exactly the set site-private names.
5. **Until out-of-tree pinning exists**
([#47](https://github.com/frdminc/tendcf/issues/47)), a release build
loads in-tree plugins only (`plugins/` and `contrib/`, at the pinned
5. **Until out-of-tree pinning exists** (and entry-point discovery, which
is deferred with it; [#47](https://github.com/frdminc/tendcf/issues/47)),
a release build loads in-tree plugins only (`plugins/` and `contrib/`, at the pinned
tendcf rev). That way the tendcf pin identifies the source of the
compiler and of every plugin a release loads. Reproducing a release
byte for byte also needs a fixed dependency and toolchain closure and
Expand Down Expand Up @@ -865,7 +904,8 @@ checks; no loader exists today). The steps run in this order:

How an out-of-tree plugin earns trust (an OpenPGP web of trust) and how a
site manages its plugins (a `tendcf plugin` command) is
[#47](https://github.com/frdminc/tendcf/issues/47).
[#47](https://github.com/frdminc/tendcf/issues/47); it is deferred with
out-of-tree discovery (Smaller 5, 2026-11-14).

**Current schema version only.** The compiler renders only the current
`schema_version`. Its ability to render older versions (today nix2cf's
Expand All @@ -884,12 +924,13 @@ and the multi-version render returns with it.
into the repository with a lower bar. The lower bar waives support and
maintenance promises only: the project does not promise to fix, keep
or port a contrib plugin. It never waives the standing review rule
(root `AGENTS.md`: every pull request gets a cross-model frontier
review), because contrib code runs in the release pipeline with the
(root `AGENTS.md` rule 3: contrib code is neither docs, a spike note nor
device code, so it keeps at least one cross-model frontier review), because contrib code runs in the release pipeline with the
same privilege as everything else. A contrib plugin is off unless
site-private names it, like any other.
3. **Out of tree.** A pip package that registers under `tendcf.frontends`
or `tendcf.importers`, the way Salt extensions and Ansible collections
3. **Out of tree (deferred, Smaller 5, 2026-11-14: not built until a user
needs it and out-of-tree plugins can be pinned).** A pip package that
registers under `tendcf.frontends` or `tendcf.importers`, the way Salt extensions and Ansible collections
are shipped apart from Salt and Ansible. tendcf does not review it.
Its output passes the same release gates as any plugin's, which bound
its shape, not its faithfulness (above). A release build does not load
Expand All @@ -903,7 +944,7 @@ and the multi-version render returns with it.
| 1 | This documentation: the guide, the paper, the map (D46), the change process, the Linux targets plan, the glossary, this file; and the rename of the illustrative `nix2cf_*` keys (§6). Moves no files and changes no code or schema. | this change |
| 2 | Import `frdminc/nix2cf` with full history under `compiler/`; the root `README.md` follows. | done: [#45](https://github.com/frdminc/tendcf/pull/45), merged 2026-11-13 (`ca4fa90`) |
| 3 | Archive `frdminc/nix2cf`. | done 2026-11-13, right after #45 merged (README points here) |
| 4 | The restructure, one pull request: `bin/` → `tools/`; the Python package renamed `tendcf`; the compiler split into `compiler/` and `plugins/` with the static import table for in-tree plugins (§4) and the two public entry-point groups, which are not read until #47; `tendcf compile` replaces `nix2cf buildfile`; older-version rendering removed; the device-boundary import check (§4). No alias, symlink or wrapper. Waits until open PRs #41 and #42 merge. | planned |
| 4 | The restructure, one pull request: `bin/` → `tools/`; the Python package renamed `tendcf`; the compiler split into `compiler/` and `plugins/` with the static import table for in-tree plugins (§4), which is the deliverable (~~and the two public entry-point groups, which are not read until #47~~: the groups are deferred, not part of this step, until a user needs out-of-tree plugins and they can be pinned; operator ruling, 2026-11-14, Smaller 5); `tendcf compile` replaces `nix2cf buildfile`; older-version rendering removed; the device-boundary import check (§4). No alias, symlink or wrapper. PRs #41 and #42 are merged. | planned |
| 5 (map's Step 6) | The release pipeline (Step 6, [`change-process.md`](change-process.md) §10) and the §4 properties that depend on it: the plugin loader's activation steps; the two-job split and its signer backends; promotion of the pull request's artifacts and branch protection on site-private; the privileged-change checks of gate 5; the import step for external recipes with the Nix limits; the projection lint's cross-host check. None is built. The two numberings differ: this table's step 5 is the map's Step 6. | planned |

Step 4 rewrites every path reference in every file, dated records
Expand Down Expand Up @@ -998,8 +1039,9 @@ change, because the list moves:
`external_import.without_ai` (§4, frontend inputs) lives in this file,
so the file's format decides how that setting is written; TOML unless
decided otherwise.
- Out-of-tree pinning and trust establishment, after which a release
build may load an out-of-tree plugin
- Out-of-tree pinning and trust establishment, after which (and only when
a user needs one: Smaller 5, 2026-11-14) entry-point discovery of an
out-of-tree plugin may be built and a release build may load one
([#47](https://github.com/frdminc/tendcf/issues/47)).
- Compile-step network access: job A's egress is unrestricted for now
(operator ruling, §4), and a future restriction stays open below. Still
Expand Down
3 changes: 2 additions & 1 deletion docs/architecture/tendcf-validate-proposal-2026-11-03.md
Original file line number Diff line number Diff line change
Expand Up @@ -1225,7 +1225,8 @@ The fork-side all-or-nothing loader (PR #41) and this check hold each without
the other: with the loader failing closed, the file's keys are not loaded and
the check refuses and reports it, which is correct and louder. The independence
is limited, not complete: the loader's strict parse (a repeated member or
trailing input refuses the file; fork `4443504`) belongs to the CMDB loader
trailing input refuses the file; fork `4443504`, the pin moved by PR #61, merged
2026-11-15T00:12:15Z, `401e0f6`) belongs to the CMDB loader
only, while this check's `readjson` stays lenient and sees the last of a
repeated member or `vars` section, so an earlier container the loader refused
is not named by the check. The check reports a refusal when that lenient view
Expand Down
Loading
Loading