diff --git a/bin/xref_lint.py b/bin/xref_lint.py index 194ec1f..c54e9b3 100755 --- a/bin/xref_lint.py +++ b/bin/xref_lint.py @@ -130,6 +130,14 @@ "corrections register rather than in place, and verified " "byte-untouched by the same scope guarantee.", ), + ( + "docs/architecture/schema-v4-opinion-*.md", + "frozen inputs to schema-v4-reconciliation-2026-10-31.md: the Opus " + "draft and the two independent frontier reviews (Fable, Argon), each " + "one model's unedited final message. The reconciliation's §8 records " + "where each was wrong; correcting them in place would destroy that " + "evidence.", + ), ) # `§5.x` is a literal placeholder ("references of the form E1 §5.x"), not a diff --git a/docs/architecture/nixos-passthrough-plan.md b/docs/architecture/nixos-passthrough-plan.md index f5d3b89..0cc81b8 100644 --- a/docs/architecture/nixos-passthrough-plan.md +++ b/docs/architecture/nixos-passthrough-plan.md @@ -12,7 +12,7 @@ runs as an unprivileged, store-only build. | No CFEngine anywhere in the running system closure | `nix-store -qR /run/current-system` has no `cfengine` path | nothing to conflict with; the first install is ours | | `/etc/systemd/system` is a symlink into `/etc/static` and the store; a write fails with `Read-only file system`; `/nix/store` is mounted read-only | `touch /etc/systemd/system/.probe` | **the generic systemd adapter cannot work here**, by design: on NixOS a unit exists only if the system configuration declares it | | `/opt` does not exist; `/var/lib` is writable | `ls` | CFEngine's work directory can live under `/var/lib`; the binaries come from the store, not `/opt/tendcf` | -| 21 running services; root's SSH keys are declared in the configuration | `systemctl`, `configuration.nix` | tendcf must not touch any of them (writer is `nixos`) | +| 21 running services; root's SSH keys are declared in the configuration | `systemctl`, `configuration.nix` | tendcf must not touch any of them (supervisor is `nixos`) | | The host configuration lives in `site-public-djbclark/nixos/` (flake, `vps-primary/configuration.nix`); deployed from the pushed branch head by `deploy-vps-primary` | repo layout, AGENTS.md rule 36 | the CFEngine derivation and module belong in that flake (linux-targets.md §2), not in this repo | ## 2. Design consequence: tendcf must refuse, not attempt @@ -83,35 +83,40 @@ host until stage C. directory only. Then, as root over SSH, one `cf-agent -n -K -f ...` against the passthrough goal file (below). `-n` makes no change. Output: the refusal lines and the report rows. -4. **D. Stop and read.** No `switch`, no enforcement. Enforcement on this host - is a separate, later ruling, because the only things tendcf could enforce - here are files and the report channel, never services. +4. **D. Stop and read.** **Ruled 2026-10-31:** tendcf enforces nothing on a Nix host + for now. It only reports and refuses, and `cf-execd` stays off. The only things it + could ever enforce here are files and the report channel, never services. Allowing + either one needs a later ruling. The schema and consent-diff consequences are in + [schema-v4-reconciliation-2026-10-31.md](schema-v4-reconciliation-2026-10-31.md). ## 4. The passthrough goal file for this host -Open question from linux-targets.md §5, to be settled by stage C's output rather -than argued in advance: how the Nix side appears in the goal file so the consent -diff stays in ordinary language. The two candidates are the evaluated -configuration's JSON and the flake lock plus a Nix-side diff. Stage B already -produces both (`nix eval --json` of the options tendcf cares about; the -`diff-closures` text), so stage C can render each next to the real host and the -choice can be made on an actual diff. The goal file for stage C declares the -`unit-writer` for the host's unit namespace as `nixos` and carries no -`service` entries, so `cf-agent -n` has nothing to actuate and should report -nothing but its own health. First cut: `deploy/linux/passthrough/goal-file.json` -(`schema_version: 3`, writer `nixos` on `org.nixos.*`), projecting to -`{"vars":{}}`. Open, for stage C: the unit-writer prefix pattern is reverse-DNS -with two labels or more, and NixOS unit names are not (`afsd.service`, -`sshd.service`); a passthrough host needs either a whole-namespace prefix or a -host-level `supervisor: nixos` fact (the deferred inventory field), decided -against the real `systemctl list-unit-files` output. +**Settled 2026-10-31** by +[schema-v4-reconciliation-2026-10-31.md](schema-v4-reconciliation-2026-10-31.md) +§1 and §5. Stage C's output (`systemctl list-unit-files` on get) showed 116 service units, +and none of them fits the reverse-DNS unit-writer prefix. So the passthrough host is +described by a required host-level `supervisor: nixos` fact (`schema_version: 4`), not +by a prefix. On such a host, `supervision` is `deliberately-unmanaged`, and no domain +may carry a service, interlock or unit-writer entry. + +The consent diff for the host flake works as follows: +- The authority is the PR's own diff (with `flake.lock` reduced to + `(input, type, rev, narHash)` per changed node), plus the `.nix` diff of each moved + input. +- The evaluated options JSON is a readable summary under it. +- `diff-closures` shows the effect. +- djbclark approves it on the host-flake PR, behind a required CI job. + +`deploy/linux/passthrough/goal-file.json` is a fixture with the example key, not get's +goal file. It projects to `{"vars":{}}`. ## 5. Work this unlocks, in order 1. Bundle guard from §2 plus negative test (this repo, small PR, CI on the VM and Debian jobs). -2. Step 4 slice B's `nixos` and `nix-darwin` writer vocabulary and the inventory - `supervisor`/distro family, needed by the goal file in §4 (schema bump). +2. Step 4 slice B's `nixos` and `nix-darwin` writer vocabulary (`schema_version` 3, done) + and the `supervisor` fact (`schema_version` 4, slice S1 of the reconciliation; distro + family stays deferred). 3. Stages A to D in the host flake. ## 6. Decisions for the operator (taken 2026-10-31) @@ -125,6 +130,10 @@ against the real `systemctl list-unit-files` output. `get.sipb.cloud`. 3. The writer vocabulary landed as `schema_version: 3` (this PR); inventory fields stay deferred. +4. Stage D (2026-10-31): consent diff = authority plus summary; `supervisor: nixos` + with `schema_version: 4`; enforce nothing on a Nix host. The follow-on rulings + (Q1-Q10) are in + [schema-v4-reconciliation-2026-10-31.md](schema-v4-reconciliation-2026-10-31.md) §1. Original questions: diff --git a/docs/architecture/schema-v4-opinion-argon.md b/docs/architecture/schema-v4-opinion-argon.md new file mode 100644 index 0000000..a54742c --- /dev/null +++ b/docs/architecture/schema-v4-opinion-argon.md @@ -0,0 +1,278 @@ +### Overall Verdict: **SOUND WITH CHANGES** + +The core architecture of the Stage D design (`schema_version: 4`, placing `supervisor` as a required singleton `device-trust` kind, using `deliberately-unmanaged` on `supervision` for `nixos` hosts, and generating the D1 consent diff in the host-flake CI) is structurally sound and respects D1–D3. However, empirical execution of the proposed JSON Schema against `jsonschema` (Draft 2020-12) and inspection of the codebase, `nix2cf`, and fixture corpus reveal **one high-severity schema loophole** in the `allOf`/`if`/`then` rule, **two medium-severity projector/migration gaps**, and several concrete test-harness and fixture-enumeration corrections needed before S0–S4 can land cleanly. + +--- + +### Check-by-Check Analysis (1–6) + +#### 1. Correctness of the JSON-Schema Deltas (B.2) +I ran the exact `B.2(a)–(e)` schema delta in memory against `Draft202012Validator` (`bin/schema_lint.py` harness) and tested cases 93–100 plus adversarial variants: +- **Does `allOf`/`if`/`then` work under Draft 2020-12?** Mostly yes, **except for a bypass in `if` when `"device-trust"` is omitted** (see **Finding 1** below). In JSON Schema Draft 2020-12, `properties` only evaluates present keys, so `"required": ["domains"]` without `"required": ["device-trust"]` on the `domains` subschema inside `if` evaluates `if` to **true** when `domains` is `{}`. And in `then`, `additionalProperties` under Draft 2020-12 evaluates every property not listed in the *same subschema's* `properties` keyword. +- **Is `"device-trust": true` needed and sufficient?** Yes. Because `then.properties.domains` is a separate subschema from the top-level `properties.domains`, omitting `"device-trust": true` from `then.properties.domains.properties` causes `then.properties.domains.additionalProperties` (`{"properties": {"entries": false}}`) to apply to `device-trust`, rejecting `device-trust.entries` and making every `nixos` goal file invalid. With `"device-trust": true`, `device-trust` is exempted from `then`'s `additionalProperties`. +- **Does `additionalProperties: {"properties": {"entries": false}}` inside `then` constrain what it claims?** No—it is **necessary but not sufficient** to enforce B.2(c)'s claim that *"no domain carries a state entry"* and leaves a contradiction hole on additional domains (see **Finding 2**). While it blocks `entries` on any domain other than `device-trust` and `supervision`, it still permits a `nixos` goal file to declare `"domains": {"apps": {"coverage": "comprehensive"}}` (or `"not-yet-migrated"`) with no entries—claiming comprehensive tendcf management of another domain on a host where D3 says tendcf enforces nothing. Furthermore, because `jsonschema.descend()` short-circuits on a `False` boolean subschema before appending the property name (`bin/schema_lint.py:1141-1146`), the error path reported for cases 97 and 98 is `domains/apps` and `domains/packaging`, **not** `domains/apps/entries` as F's table claims (see **Finding 5**). +- **Can a `nixos`-supervisor file still carry a `unit-writer`, `service`, or tombstone entry?** No. On `supervision`, `"coverage": {"const": "deliberately-unmanaged"}` combines with `$defs.state_domain`'s existing `if/then` ([`schema/goal-file.schema.json`](file:///Users/djbclark/src/tendcf-review-policy-wt/schema/goal-file.schema.json#L47-L51)) to forbid `entries`. On all other state domains, `additionalProperties: {"properties": {"entries": false}}` forbids `entries`. Since `service` (present or absent tombstone), `interlock`, and `unit-writer` can only live inside `entries`, none of them can appear anywhere in a `nixos`-supervisor goal file. + +#### 2. Soundness of `supervisor` as a `device-trust` Kind (B.1) vs. D2 +- **Soundness against diff/approval/ceremony:** Placing `supervisor` under `device-trust.entries.supervisor.host` fits the existing ceremony architecture cleanly: + - [`derive_ceremony_class()`](file:///Users/djbclark/src/tendcf-review-policy-wt/bin/schema_lint.py#L1498-L1499) classifies any hunk under `device-trust` as `privileged`, so changing `supervisor` (e.g. `nixos` $\to$ `systemd`) can never pass as an `ordinary` ceremony. + - [`apply_diff()`](file:///Users/djbclark/src/tendcf-review-policy-wt/bin/schema_lint.py#L1509-L1562) works without change because `supervisor` is a standard `(domain, kind, id)` map entry. + - [`project()`](file:///Users/djbclark/src/tendcf-review-policy-wt/bin/projector.py#L120-L121) ignores `device-trust`, preserving `{"vars":{}}` on passthrough hosts. + - [`check_trust_inexpressible()`](file:///Users/djbclark/src/tendcf-review-policy-wt/bin/schema_lint.py#L1787-L1856) passes (verified by running it: 0 findings with 5 trust kinds). +- **Strongest case against B.1:** + 1. **Category error (Hardware/OS taxonomy vs. Gate configuration):** [`schema/goal-file.schema.json`](file:///Users/djbclark/src/tendcf-review-policy-wt/schema/goal-file.schema.json#L263-L270) defines `device-trust` as *"the region the validator and agent read their own configuration ONLY from here"*. A host's service manager (`launchd`, `systemd`, `runit`, `nixos`) is a target platform/OS fact (like `arch` or `platform` in [`docs/architecture/architecture-DEFINITIVE-v3.md`](file:///Users/djbclark/src/tendcf-review-policy-wt/docs/architecture/architecture-DEFINITIVE-v3.md#L178-L180)), not a cryptographic root, consent policy, or agent binary pin. Putting target OS taxonomy into `device-trust` turns `device-trust` into a catch-all for any host-wide fact because the header is locked. + 2. **Ceremony class mismatch (`privileged` vs. `baseline`):** Can a live host ever legitimately transition from `launchd` to `systemd` or `nixos` to `systemd` in-place via a `privileged` diff? No—changing a host's OS/supervisor is a wipe-and-reinstall (a `baseline` re-adoption ceremony, like re-keying `host`), not an in-place `privileged` hunk. A top-level header field (`supervisor` alongside `schema_version` and `host`, with no diff hunk support) would force a `baseline` ceremony if the OS ever changed, which is arguably *more* faithful to reality than allowing an in-place hunk! + 3. **Projector N-5 blind spot (see Finding 3):** Because `supervisor`'s body (`{"name": "launchd", "state": "present"}`) contains no `sha256:` or `ed25519:` digest, [`bin/projector.py`](file:///Users/djbclark/src/tendcf-review-policy-wt/bin/projector.py#L72-L81)'s `validate_projection()` N-5 check does **not** catch it if smuggled into a service body, lowering N-5's coverage of `device-trust` bodies from 3/4 to 3/5. +- **Why B.1 still wins (with Finding 3 documented):** Adding a top-level header field breaks [`goal-file-schema-reconciliation-2026-08-15.md` §7 (G-3)](file:///Users/djbclark/src/tendcf-review-policy-wt/docs/architecture/goal-file-schema-reconciliation-2026-08-15.md#L374-L383) (*"The header is `schema_version` + `host`, nothing else"*), and a `systemd` $\leftrightarrow$ `nixos` transition on a dual-boot or migrated host (or `systemd` $\leftrightarrow$ `runit`) *can* share a host key and benefit from a reviewable `privileged` diff rather than an un-diffed baseline reset. And D2's phrase *"host-level `supervisor: nixos` inventory fact"* comes directly from [`docs/architecture/nixos-passthrough-plan.md`](file:///Users/djbclark/src/tendcf-review-policy-wt/docs/architecture/nixos-passthrough-plan.md#L105-L106), where "host-level" contrasts with *"per-unit prefix"* (`unit-writer`), just as `trust_tier` is a host-level inventory fact that compiles to `device-trust.entries.trust-policy.consent`. + +#### 3. Fail-Closed Analysis (C) +- **Old validators (C.1):** Verified. A v3 validator rejects a v4 file both at `schema_version` (`const: 3`) and at `device-trust.entries` (`additionalProperties: false` rejects `"supervisor"`). The author rightly noted (`C.1`, line 136) that nothing in [`bin/schema_lint.py`](file:///Users/djbclark/src/tendcf-review-policy-wt/bin/schema_lint.py) implements distinct `version-above-ceiling` reporting today. +- **D44 render rule (C.2):** Verified against [`schema/report-row.schema.json`](file:///Users/djbclark/src/tendcf-review-policy-wt/schema/report-row.schema.json#L118-L122) (`schema_ceiling` is `integer`, `minimum: 1`, no maximum) and [`docs/architecture/goal-file-schema-reconciliation-2026-08-15.md`](file:///Users/djbclark/src/tendcf-review-policy-wt/docs/architecture/goal-file-schema-reconciliation-2026-08-15.md#L309-L318). Enrolling `get.sipb.cloud` at v4 on first adoption is sound. +- **Downgrade & §9.4 migration rule (C.3):** Verified. A `nixos` host has no valid v3 representation (since `org.nixos.*` matched 0 of 116 units), so the compiler must refuse to render below v4 for `supervisor: nixos`. The author also correctly spotted that [`schema/goal-diff.schema.json`](file:///Users/djbclark/src/tendcf-review-policy-wt/schema/goal-diff.schema.json#L99-L115)'s `version_bump` rule (`if: required: [version_bump] then: not: anyOf: [required: [hunks], required: [coverage_changes]]`) cannot express a v3 $\to$ v4 migration if v4 adds a required entry (`supervisor`). Moot while no device holds an enrolled v3 baseline, but worth recording as a known property of `goal-diff.schema.json`. +- **On-device tombstone gap (C.5):** I verified all cited lines in [`policy/tendcf_services.cf`](file:///Users/djbclark/src/tendcf-review-policy-wt/policy/tendcf_services.cf): + - Lines 172–185 define `tendcf_systemd_units_writable`, `tendcf_systemd_units_link`, `tendcf_systemd_units_elsewhere` (`linux.tendcf_systemd_booted.!tendcf_systemd_units_writable`), and `tendcf_systemd_host` (`linux.tendcf_systemd_booted.tendcf_systemd_units_writable`). + - Line 200 defines `service_$(service_canon[$(service_ids)])_refused` as `service_..._present.!service_..._supported`. + - Lines 225–232 gate `service_..._unit_present` and `service_..._teardown` on `tendcf_systemd_host.!service_..._present`. + - Lines 314–318 gate the refusal `reports:` promises on `service_..._refused`. + - **Nuance missed in C.5 / S0 (see Finding 6):** Why was `_refused` gated on `_present` at line 200 in the first place? Because a tombstone (`{"state": "absent"}`) has **no `unit` field** ([`policy/tendcf_services.cf`](file:///Users/djbclark/src/tendcf-review-policy-wt/policy/tendcf_services.cf#L140-L142): `unit_flavour` is `""` for a tombstone)! On a normal Debian (`tendcf_systemd_host`) or macOS (`darwin`) host, a tombstone carries no flavour: on Darwin it cleans `/Library/LaunchDaemons/$(id).plist` (line 260), and on `tendcf_systemd_host` it checks `fileexists("/etc/systemd/system/$(id)")` (line 227) and tears down only if the file exists. On a NixOS-shaped host (`tendcf_systemd_units_elsewhere`), `/etc/systemd/system` is a symlink into the store where NixOS's own units live (`sshd.service`, `afsd.service`, etc.). If CMDB data on a NixOS host contains a tombstone, should it report a refusal unconditionally for every absent entry, or specifically under `tendcf_systemd_units_elsewhere.!service_..._present`? See **Finding 6** for the exact CFEngine class and `reports:` promise needed for S0. + +#### 4. Consent-Diff Design (D) Against D1 and D3 +- **Against D1:** D1 requires *(1) `flake.lock` + Nix-side diff as authority, (2) evaluated-options JSON as readable summary*. Look closely at `D` (lines 158–186): the author's three steps and rendered example include `flake.lock` diff, `services.cfengine-tendcf` options JSON, and `nix store diff-closures`, **but omit the Nix-side source diff** (`.nix` files changed in the commit range, or changing a module in `site-djbclark` / `djbclark/get` when `flake.lock` is untouched!). If a PR in `djbclark/get` edits `configuration.nix` directly (for example, setting `services.cfengine-tendcf.execd.enable = true` or adding a systemd service in `get`'s own `flake.nix`), **`flake.lock` does not change at all**! Claiming `AUTHORITY: flake.lock` alone violates D1 ("flake.lock + Nix-side diff as authority") and misses direct `.nix` edits in the host repo. See **Finding 4**. +- **Verified ``:** As the lead verified, `get`'s attribute is `nixosConfigurations.get` (`flake.nix:55` in `djbclark/get`), so the command in D.2 is `nix eval --json .#nixosConfigurations.get.config.services.cfengine-tendcf ...`. +- **Note on `c.package` string interpolation in D.2:** In Nix, evaluating `"${c.package}"` inside `nix eval --json` forces instantiation (and if dependent on IFD or store access, requires a store) and includes string context; [`nixos-cfengine-build.yml`](file:///Users/djbclark/ops/site-public-djbclark/.github/workflows/nixos-cfengine-build.yml#L148) uses `builtins.unsafeDiscardStringContext` or `c.package.version` / `c.package.outPath`. Using `builtins.unsafeDiscardStringContext "${c.package}"` (or `{ enable = c.enable; execd.enable = c.execd.enable; package = builtins.unsafeDiscardStringContext "${c.package}"; }`) is safer and cleaner. +- **The two proposed hard CI checks (D lines 192–197):** + 1. *"`execd.enable` becomes `true` while D3 stands"*: **Strong yes.** In [`nixos-cfengine-build.yml`](file:///Users/djbclark/ops/site-public-djbclark/.github/workflows/nixos-cfengine-build.yml#L101-L115), `vps-primary` already has a hard step checking `services.cfengine-tendcf.enable == false` and `!(services ? "cfengine-tendcf-execd")`. On `get` (where `enable = true` and `execd.enable = false`), asserting `execd.enable == false` and `!(config.systemd.services ? "cfengine-tendcf-execd")` in CI mechanically enforces D3. + 2. *"Any options key outside the summary appears"*: **Good, with a specific implementation.** In `nix eval --json #nixosConfigurations.get.config.services.cfengine-tendcf --apply 'builtins.attrNames'`, asserting the sorted attribute list equals `["enable", "execd", "package"]` and `builtins.attrNames c.execd == ["enable"]` guarantees the module didn't grow a new unreviewed option behind the scenes without updating the summary projector. + +#### 5. Migration and Test Plan (E, F) +- **Fixture counts and missed files (see Finding 7):** + - The author claims in E.4 (line 206): *"All 63 `examples/broken/*` goal-family overlays are full alternate files... Each needs `schema_version` 4 and the supervisor entry"*. **False on two counts:** + 1. Only **51** of the 63 directories in `examples/broken/` are goal-family cases (cases 1–12 are YAML Site Model overlays: `services.yml`, `roles.yml`, `launchd-writers.yml`, `report-rows.yml`). + 2. Even among those 51 goal-family cases, **not all overlay `goal-file.json`**! Cases `49`, `50`, `51`, `54`, `55`, `56`, `57`, `59`, `60` overlay only `goal-diff.json` and/or `approval-record*.json` (and `52`, `53` overlay `goal-file-baseline.json`, `goal-diff.json`, and `approval-record*.json`, not `goal-file.json`). Plus case `18-missing-device-trust` intentionally omits `device-trust`. + 3. Conversely, the author **completely missed** the **5 byte-class fixtures** in `examples/broken-bytes/` (`44`, `45`, `46`, `47`, `48`), **1 projection fixture** in `examples/broken-projection/75-goal-file-under-vars.json`, and **`README.md:22`** (`Ninety-six deliberately broken fixtures (63 + 6 + 27)`). I verified this against commit `f814f1f` (the v3 bump), which had to update all of those! +- **Numbering from 93:** Verified. `70–73` were accidentally reused between `examples/broken/` and `examples/broken-projection/`, and `broken-projection/` runs up to `92`. Starting at `93` is right. And 3-digit directory names (`100-...`, `101-...`): I checked [`bin/schema_lint.py`](file:///Users/djbclark/src/tendcf-review-policy-wt/bin/schema_lint.py#L415) (`_README_ROW = re.compile(r"^\|\s*\d+\s*\|...")`) and `sorted(p for p in BROKEN_DIR.iterdir() if p.is_dir())`—both handle 3-digit names without issue (note lexicographic sort puts `100-...` between `10-...` and `11-...` in iteration order, which is harmless because `check_negative_fixtures` only counts `caught` and looks up `declared[case.name]` by dict key). +- **`nix2cf` is at v1 and already failing against tendcf v3 (see Finding 8):** Running `pytest` in `/Users/djbclark/src/nix2cf` today fails 10 of 24 tests with `rendered goal file fails goal-file.schema.json: /schema_version: 3 was expected`. Slice S4 cannot just "consume `supervisor`"; it must bring `nix2cf` from v1 $\to$ v4 (`GOAL_FILE_SCHEMA_VERSION = 4`, `HostSpec` / `device-trust.json`, `unit.systemd` `restart`/`wanted_by` from v2, and golden regeneration). +- **The `dpkg`/Debian unit-name problem in Risk 4:** Confirmed. In [`schema/goal-file.schema.json`](file:///Users/djbclark/src/tendcf-review-policy-wt/schema/goal-file.schema.json#L243), `unit_writer_map.propertyNames.pattern` is `^[a-z][a-z0-9]*(\.[A-Za-z0-9][A-Za-z0-9_-]*)+\.\*$`, which requires at least two dot-separated labels before `.*` (e.g. `tendcf.canary.*`). Standard Debian `dpkg` unit names (`ssh.service`, `cron.service`, `nginx.service`) have only one label before `.service`, so no `unit-writer` entry with `writer: "dpkg"` can ever match a real Debian package unit unless that pattern is widened or redesigned! The author is right not to bundle an unruled fix into v4, and this should be surfaced to the operator as **Q8**. + +--- + +### Numbered Findings + +#### Finding 1 [HIGH] — `B.2(d)`'s top-level `if` fires on `domains: {}` (`18-missing-device-trust`), causing spurious `if/then` findings when `device-trust` or its keys are absent +- **Evidence:** [`stage-d-design-report.md` lines 94–96](file:///Users/djbclark/.local/state/bigteam/tendcf-stage-d-design-20261031/stage-d-design-report.md#L94-L96); [`examples/broken/18-missing-device-trust/goal-file.json`](file:///Users/djbclark/src/tendcf-review-policy-wt/examples/broken/18-missing-device-trust/goal-file.json). +- **Problem:** In `B.2(d)`, the outermost object of `if` has `"required": ["domains"]`, and inside `"domains"` it has `"properties": {"device-trust": {..., "required": ["device-trust"]}}`—Notice that `"required": ["device-trust"]` is placed **inside** the `"device-trust"` subschema (where it asks whether the `device-trust` object has a child property named `"device-trust"`, which is **false** on every valid goal file!), instead of on the `"domains"` object! + Let's trace what happens on **every** goal file with `B.2(d)` as written: + Look at line 95 of `stage-d-design-report.md`: + ```json + "if": { "properties": { "domains": { "properties": { "device-trust": { "properties": { "entries": { "properties": { "supervisor": { "properties": { "host": { "properties": { "name": { "const": "nixos" } }, "required": ["name"] } }, "required": ["host"] } }, "required": ["supervisor"] } }, "required": ["entries"] } }, "required": ["device-trust"] } }, "required": ["domains"] } + ``` + Look at where each `"required"` is attached in line 95: + - `"required": ["name"]` is inside `"host"` (correct: `host` has property `name`). + - `"required": ["host"]` is inside `"supervisor"` (correct: `supervisor` has property `host`). + - `"required": ["supervisor"]` is inside `"entries"` (correct: `entries` has property `supervisor`). + - `"required": ["entries"]` is inside `"device-trust"` (correct: `device-trust` has property `entries`). + - `"required": ["device-trust"]` is inside `"domains"` (correct: `domains` has property `device-trust`). + - `"required": ["domains"]` is at the root (correct: root has property `domains`). + Wait: why did `18-missing-device-trust` not fail in my test earlier? Because `domains` had `"required": ["device-trust"]`. What if the root object omits `"domains"`? Root has `"required": ["domains"]`. What if `"domains"` is `null` or an array (wrong type)? In JSON Schema 2020-12, `properties` and `required` ignore non-object instances (`"domains": []` satisfies `"properties": ...` and `"required": ["device-trust"]`!). Adding `"type": "object"` at each level (or relying on the top-level schema) prevents `if` from matching non-objects. + +#### Finding 2 [HIGH] — `B.2(d)` allows non-`supervision` state domains to be declared on a `nixos` host with `"coverage": "comprehensive"` or `"not-yet-migrated"` (and no entries) +- **Evidence:** [`stage-d-design-report.md` lines 75, 96–103](file:///Users/djbclark/.local/state/bigteam/tendcf-stage-d-design-20261031/stage-d-design-report.md#L96-L103); [`schema/goal-file.schema.json` lines 39–52](file:///Users/djbclark/src/tendcf-review-policy-wt/schema/goal-file.schema.json#L39-L52). +- **Problem:** Under `B.2(d)`, `then.properties.domains.additionalProperties` is `{"properties": {"entries": false}}`. Because `$defs.state_domain` only requires `"coverage"` (`"entries"` is optional), a `nixos` goal file with: + ```json + "domains": { + "device-trust": { ... }, + "supervision": { "coverage": "deliberately-unmanaged" }, + "packaging": { "coverage": "comprehensive" } + } + ``` + **validates with zero findings** (verified empirically against `Draft202012Validator`). Yet on a `nixos` host under D3, tendcf enforces nothing and has no `packaging` or `files` adapter; claiming `"coverage": "comprehensive"` on another domain is either a false coverage claim or an empty domain (`bin/schema_lint.py:1600`: *"a declared domain with no entries is not a document the schema admits"*—a claim `apply_diff` makes in comments that `state_domain` only enforces via `not-yet-migrated` / `deliberately-unmanaged` convention!). Even worse, if `then` wants a `nixos` passthrough host in v4 to declare *only* `device-trust` and `supervision` (or wants any additional domain on a `nixos` host to also be `deliberately-unmanaged`), `additionalProperties: {"properties": {"entries": false}}` does not enforce that. +- **Concrete fix:** Either: + - If a `nixos` host in v4 can only carry `device-trust` and `supervision` (until another domain exists), or if any other domain must also forbid `entries`, keep `additionalProperties: {"properties": {"entries": false}}` (so cases 97 and 98 test `entries: false` on arbitrary domains) **and** require `"coverage": {"const": "deliberately-unmanaged"}` (or document explicitly why non-`supervision` empty domains are permitted for negative testing in cases 97–98). Note that cases 97 and 98 specifically put `entries` under `domains/apps` (`not-yet-migrated`) and `domains/packaging` (`comprehensive`), so `additionalProperties: {"properties": {"entries": false}}` is intentionally designed to let cases 97 and 98 fail on `entries: false` rather than on domain name or coverage! That design is fine **provided** `state_domain` itself is understood to allow entry-less domains, or `supervision` also gets `"entries": false` directly in `then` so a `supervision` domain with `"coverage": "comprehensive"` AND `"entries": {...}` reports `entries: false` too instead of only the coverage `const`. + - Specifically, in `then.properties.domains.properties.supervision`, add `"entries": false`: + ```json + "supervision": { + "properties": { + "coverage": { "const": "deliberately-unmanaged" }, + "entries": false + } + } + ``` + Why? Because if a `nixos` goal file has `"supervision": {"coverage": "comprehensive", "entries": {"service": {"foo.service": {"state": "absent"}}}}`, `state_domain`'s own `if/then` (`coverage == "deliberately-unmanaged" -> not required [entries]`) **does not fire** (since `coverage` is `"comprehensive"`), and `then`'s `additionalProperties` **does not apply to `supervision`** (since `supervision` is in `then.properties.domains.properties`)! Without `"entries": false` inside `supervision`, a `nixos` file with `"supervision": {"coverage": "comprehensive", "entries": ...}` only reports that `coverage` wasn't `deliberately-unmanaged` and **never flags the forbidden `entries` under `supervision`**! Adding `"entries": false` to `supervision` in `then` closes that gap and gives a direct `False schema does not allow ...` error on `domains/supervision` whenever `supervision` carries `entries` on a `nixos` host. + +#### Finding 3 [MEDIUM] — `bin/projector.py` docstring and ` bin/schema_lint.py` comments hard-code "four device-trust bodies" (F7), and `supervisor`'s body is the second trust body without a digest +- **Evidence:** [`bin/projector.py` lines 73–81, 172–173](file:///Users/djbclark/src/tendcf-review-policy-wt/bin/projector.py#L73-L81); [`bin/schema_lint.py` lines 1795–1798, 1810](file:///Users/djbclark/src/tendcf-review-policy-wt/bin/schema_lint.py#L1795-L1810). +- **Problem:** `bin/projector.py:73-81` states: + > *"Measured against the corpus's own four device-trust bodies, that catches three: `advisor-key` (digest in the key), `agent` and `policy-tree` (digest in the value). It does NOT catch `trust-policy`... Three of four, stated as a number because the earlier wording said only 'trust in a value is not caught' and that understated it to one of four (F7)."* + With `supervisor` (`{"name": "launchd", "state": "present"}`) added as the fifth `device-trust` kind, there are now **five** `device-trust` bodies, and `validate_projection()`'s N-5 digest regexes catch **three of five** (missing both `trust-policy` and `supervisor`, which I verified by running `validate_projection(b'{"vars":{"tendcf_service":{"com.djbclark.caddy":{"name":"launchd","state":"present"}}}}')` $\to$ `[]`). + Leaving "four device-trust bodies" and "Three of four" in `bin/projector.py` (and "four trust bodies... a fifth trust kind that does not exist yet" in `bin/schema_lint.py:1795,1810`) violates the F7 precision rule that comment was written to uphold. +- **Concrete fix:** Update [`bin/projector.py`](file:///Users/djbclark/src/tendcf-review-policy-wt/bin/projector.py#L73-L81) and [`bin/schema_lint.py`](file:///Users/djbclark/src/tendcf-review-policy-wt/bin/schema_lint.py#L1795-L1810) in S1 to state **five** device-trust bodies and **three of five** caught by digest shape (`trust-policy` and `supervisor` carry no digest). + +#### Finding 4 [MEDIUM] — Section D's consent diff omits the Nix-side source diff when `flake.lock` is unchanged (incomplete implementation of D1) +- **Evidence:** [`stage-d-design-report.md` lines 18–21, 158–186](file:///Users/djbclark/.local/state/bigteam/tendcf-stage-d-design-20261031/stage-d-design-report.md#L158-L186). +- **Problem:** Operator ruling D1 states: *"consent diff for a Nix host = flake.lock + Nix-side diff as authority, evaluated-options JSON as readable summary."* In Section D (lines 158–186), the author defines Step 1 as **`Lock diff.` `git diff` of `flake.lock`** and titles the authority block `AUTHORITY: flake.lock`. + If a PR on `djbclark/get` (or `site-public-djbclark/nixos`) edits `flake.nix`, `configuration.nix`, or `modules/cfengine-tendcf.nix` directly without bumping `flake.lock`, the `flake.lock` diff is **empty**! Furthermore, when `flake.lock` *does* bump `site-djbclark` from `` to ``, D1's *"flake.lock + Nix-side diff as authority"* means showing both the `flake.lock` pin change **and** the Nix source diff (`git diff base..head -- '*.nix' 'flake.lock'` in the host repo, plus `git -C diff .. -- nixos/` when the `site-djbclark` input moves). +- **Concrete fix:** Amend Section D Step 1 so the `AUTHORITY` section includes: + 1. `flake.lock` node changes `(input, type, rev, narHash)`; + 2. `git diff base..head -- '*.nix'` in the host flake repository; and + 3. for any local/pinned our-own input that moved (e.g. `site-djbclark`), the `.nix` diff across that input's `oldrev..newrev`. + +#### Finding 5 [MEDIUM] — Section F's table (`stage-d-design-report.md:231-248`) has inaccurate instance paths for cases 97 and 98 and omits verified `schema_evidence` tokens +- **Evidence:** [`stage-d-design-report.md` lines 231–248](file:///Users/djbclark/.local/state/bigteam/tendcf-stage-d-design-20261031/stage-d-design-report.md#L231-L248); [`bin/schema_lint.py` lines 1117–1154](file:///Users/djbclark/src/tendcf-review-policy-wt/bin/schema_lint.py#L1117-L1154). +- **Problem:** Because the author had no shell, they marked the error strings `[mem]` and left `schema_evidence()` unverified (line 247). Running `bin/schema_lint.py`'s actual `validate()` and `schema_evidence()` functions on cases 93–100 yields the exact outputs: + - **Cases 97 & 98 instance path:** Because `entries: false` is a boolean subschema, `jsonschema.descend()` yields the error **before** pushing `"entries"` onto `error.path` (`bin/schema_lint.py:1135-1138`). Thus `error_location(error)` returns **`domains/apps`** (case 97) and **`domains/packaging`** (case 98), **not** `domains/apps/entries` or `domains/packaging/entries`. And it is **not** `` (resolving Risk 5 in G, line 268: `error.path` is `['domains', 'apps']`, which is non-empty!). + - **Verified `schema_evidence()` tokens for `examples/broken/README.md`:** + - Case 93 (`93-unknown-supervisor`): `['enum', 'supervisor_map']` $\to$ `schema (\`enum\`)` or `schema (\`supervisor_map\` \`enum\`)` works. + - Case 94 (`94-missing-supervisor`): `['required', 'trust_domain']` $\to$ `schema (\`required\`)` works. + - Case 95 (`95-supervisor-not-singleton-id`): `['const', 'propertyNames', 'supervisor_map']`, message is `'host' was expected` $\to$ `schema (\`propertyNames\`)` works. + - Case 96 (`96-nixos-supervision-comprehensive`): `['const', 'if/then']` $\to$ `schema (\`if/then\`)` works! (Confirms line 247's unverified question: `BRANCH_LOCATORS` in `bin/schema_lint.py:1049` maps `"then"` anywhere in `full` schema_path to `"if/then"`, even inside top-level `allOf`.) + - Case 97 (`97-nixos-tombstone-elsewhere`): `['if/then']` $\to$ `schema (\`if/then\`)` works. + - Case 98 (`98-nixos-unit-writer-elsewhere`): `['if/then']` $\to$ `schema (\`if/then\`)` works. + - Case 99 (`99-nixos-no-supervision-domain`): `['if/then', 'required']` $\to$ `schema (\`if/then\`)` or `schema (\`required\`)` works. + - Case 100 (`100-nixos-writer-retired`): `['enum', 'unit_writer_entry']` $\to$ `schema (\`enum\`)` works. + - **Important interaction between Case 97 and B.3's new lint rule:** In Case 97 (`97-nixos-tombstone-elsewhere`), `supervision` has no entries and `apps` has a tombstone `{"state": "absent"}` (which is not `state == "present"`, so B.3's flavour-vs-supervisor check does not fire). In Case 98 (`98-nixos-unit-writer-elsewhere`), `packaging` has a `unit-writer` entry; `check_goal_file_cross_file` only checks `unit-writer` for duplicate/nested prefixes, so it does not fire. However, remember that `validate_loaded()` runs `check_goal_file_family()` on every `examples/broken/` case (`bin/schema_lint.py:1214`), which will emit `family (hash)` and `family (apply)` findings because `goal-file.json` was overlaid. Because `check_declared_class()` checks `want in fired` (`bin/schema_lint.py:487`), `schema (`if/then`)` will pass as long as `schema` fires with `if/then` in `evidence`. + +#### Finding 6 [MEDIUM] — Slice S0 (`C.5` tombstone refusal on symlinked-units hosts) needs exact CFEngine class scoping so it does not misfire on Darwin or normal systemd hosts +- **Evidence:** [`stage-d-design-report.md` lines 147, 258, 283](file:///Users/djbclark/.local/state/bigteam/tendcf-stage-d-design-20261031/stage-d-design-report.md#L147); [`policy/tendcf_services.cf` lines 182–201, 225–232, 314–318](file:///Users/djbclark/src/tendcf-review-policy-wt/policy/tendcf_services.cf#L182-L201). +- **Problem:** `stage-d-design-report.md` proposes Slice S0: *"In `policy/tendcf_services.cf`, report a tombstone on a symlinked-units host by name (the D3 'report'), and add a test check"* (`check 7b`), but never gives the `.cf` policy diff! + Care is required here: a tombstone in `host_specific.json` (`{"state": "absent"}`) carries **no `unit` object**, so `unit_flavour[$(service_ids)]` is `""` ([`policy/tendcf_services.cf:140-147`](file:///Users/djbclark/src/tendcf-review-policy-wt/policy/tendcf_services.cf#L140-L147)). + - If you simply removed `service_..._present` from `service_..._refused` at line 200, then **every tombstone on every host** (including Darwin and normal Debian systemd hosts!) would define `service_..._refused` (because `unit_flavour` is `""`, so `service_..._supported` is false!) and print a bogus refusal report! + - Even on a normal Linux container not booted by systemd (`check 4` / `check 6` non-systemd path), or on `tendcf_systemd_units_elsewhere` (NixOS shape, `check 7`), a tombstone has no `service_..._systemd` class set (because `unit_flavour` is `""`). +- **Concrete proposed policy change for Slice S0 (`policy/tendcf_services.cf`):** + Keep `service_..._refused` for present entries (line 200), or add a dedicated class for absent entries on a symlinked-units host (`tendcf_systemd_units_elsewhere`): + ```cfengine + # On a NixOS-shaped host (/etc/systemd/system is not a writable directory), + # a tombstone cannot be actuated either: units belong to the Nix + # configuration (D3). Report the refusal rather than silently skipping. + "service_$(service_canon[$(service_ids)])_absent_refused_elsewhere" + expression => "!service_$(service_canon[$(service_ids)])_present.tendcf_systemd_units_elsewhere"; + ``` + And in `reports:` (after line 318): + ```cfengine + "service_$(service_canon[$(service_ids)])_absent_refused_elsewhere":: + "tendcf: service '$(service_ids)' (absent) refused: /etc/systemd/system is not a writable directory on this host ($(sys.flavour)); its units are written by the system configuration (unit writer nixos), not by cfengine"; + ``` + And in [`tests/policy/run.sh`](file:///Users/djbclark/src/tendcf-review-policy-wt/tests/policy/run.sh#L239-L260) (Check 7), run a second pass (`7b`) with `tests/policy/fixtures/systemd-absent.json` and assert: + - `grep -q "tendcf: service '$unit' (absent) refused: /etc/systemd/system is not a writable directory on this host .*unit writer nixos"` + - `no_systemctl` + +#### Finding 7 [MEDIUM] — Section E.4 miscounts the `examples/broken/*` goal-file overlays and omits `examples/broken-bytes/*`, `examples/broken-projection/75-goal-file-under-vars.json`, and `README.md` +- **Evidence:** [`stage-d-design-report.md` lines 206–210](file:///Users/djbclark/.local/state/bigteam/tendcf-stage-d-design-20261031/stage-d-design-report.md#L206-L210); commit `f814f1f` diff; [`README.md` line 22](file:///Users/djbclark/src/tendcf-review-policy-wt/README.md#L22). +- **Problem:** Section E.4 misses several files that `bin/schema_lint.py` checks on every run and that had to be updated in commit `f814f1f` (the v3 bump): + 1. **`README.md:22`** (root README): Currently reads `Ninety-six deliberately broken fixtures (63 + 6 + 27) the lint must catch`. With 9 new cases (93–101), this must be updated to `One hundred and five deliberately broken fixtures (72 + 6 + 27) the lint must catch` (or `72` in the breakdown). + 2. **`examples/broken-bytes/*.json` (5 files):** Cases `44-pretty-printed-twin.json`, `45-trailing-newline.json`, `46-duplicate-keys.json`, `47-non-nfc-path.json`, and `48-float-spelling-of-integer.json` are full v3 `goal-file.json` copies (`58-not-json.json` is a 43-byte stub `{"schema_version": 3, "host": "ed25519:a1",}`). As noted in `examples/broken/README.md:260-264`, cases 44, 45, 46, and 48 are supposed to parse to **the exact document the happy path parses to** so they produce zero findings in downstream layers. They must be regenerated from the v4 `examples/goal-file.json` (and `58-not-json.json` bumped to `4`). + 3. **`examples/broken-projection/75-goal-file-under-vars.json` (1 file):** Embeds the entire `examples/goal-file.json` under `vars` (`f814f1f` updated it to v3). Update to v4. + 4. **Exact breakdown of `examples/broken/` (63 directories today):** + - Cases `01`–`12` (12 dirs): YAML Site Model overlays (`services.yml`, `roles.yml`, `launchd-writers.yml`, `report-rows.yml`) — **untouched**. + - **39 directories** overlay `goal-file.json`: `13`–`43`, `61`–`65`, `70`–`73`. All 39 need `schema_version: 4`, and **38 of the 39** (all except `18-missing-device-trust`, which has `"domains": {}`) need `supervisor: {"host": {"name": "launchd", "state": "present"}}` in `device-trust.entries` (with case `13` claiming `schema_version: 5`). + - **2 directories** (`52-ordinary-over-privileged-hunk`, `53-coverage-retreat-ordinary-class`) overlay `goal-file-baseline.json`, `goal-diff.json`, `approval-record.json`, and `approval-record-reject.json` — all 4 files in each of those 2 directories must be bumped to v4, `supervisor` added to `goal-file-baseline.json`, and their `baseline_sha256` / `proposed_sha256` recomputed. + - **10 directories** (`49`, `50`, `51`, `54`, `55`, `56`, `57`, `59`, `60` — wait: that's 9 directories; total 12 + 39 + 2 + 9 = 62? Let's check: 13..43 is 31 dirs; 61..65 is 5 dirs; 70..73 is 4 dirs; 31 + 5 + 4 = **40 directories** overlaying `goal-file.json`! And 9 directories overlaying `goal-diff.json` and/or `approval-record*.json`: `49`, `50`, `51`, `54`, `55`, `56`, `57`, `59`, `60`. 12 + 40 + 2 + 9 = **63 directories**). Every overlaid `goal-diff.json` and `approval-record*.json` in those 9 directories must have `schema_version: 4` and updated `baseline_sha256` / `proposed_sha256` hashes matching the v4 happy `goal-file.json` and `goal-file-baseline.json`. + 5. ** Text updates in `examples/broken/README.md` and `bin/schema_lint.py`:** + - `bin/schema_lint.py:36` and `examples/broken/README.md:3`: `"Sixty-three"` $\to$ `"Seventy-two"` (if 9 new cases 93–101). + - `bin/schema_lint.py:87`: `EXPECTED_BROKEN = 63` $\to$ `72`. + - `examples/broken/README.md:9`: `"cases 13-73"` $\to$ `"cases 13-73, 93-101"` (since 74–92 are in `broken-projection/`!). + - `examples/broken/README.md:57` and `bin/schema_lint.py:415` comment: `schema_version: 4` $\to$ `schema_version: 5`. + +#### Finding 8 [MEDIUM] — `nix2cf` is pinned to `schema_version: 1` (`compile.py:40`), lacks v2/v3 changes, and already fails 10/24 tests against tendcf `HEAD` +- **Evidence:** [`/Users/djbclark/src/nix2cf/nix2cf/compile.py` lines 40, 60, 104–113, 328–340, 386–390](file:///Users/djbclark/src/nix2cf/nix2cf/compile.py#L40); [`/Users/djbclark/src/nix2cf/tests/test_compile.py` line 84](file:///Users/djbclark/src/nix2cf/tests/test_compile.py#L84). +- **Problem:** In Risk 2 (`stage-d-design-report.md:265`), the author noted *"I did not check the nix2cf repo"*, and in E.6 / S4 assumed `nix2cf` only needs to map inventory `supervisor` $\to$ `device-trust.supervisor` and select the renderer. In reality: + 1. `nix2cf/compile.py:40` has `GOAL_FILE_SCHEMA_VERSION = 1` and its test suite (`pytest`) currently fails 10 tests against tendcf v3 (`rendered goal file fails goal-file.schema.json: /schema_version: 3 was expected`). + 2. `nix2cf` does not yet construct `device-trust` entries itself; [`HostSpec.device_trust`](file:///Users/djbclark/src/nix2cf/nix2cf/compile.py#L107-L112) takes a pre-authored `device-trust.json` dict (`examples/site/device-trust.json` and `tests/fixtures/tendcf-device-trust.json`) and copies it verbatim (`compile.py:335`)! + 3. Therefore, in `nix2cf`, if `supervisor` lives inside `device-trust.entries.supervisor`, `nix2cf` must either inject `domains["device-trust"]["entries"]["supervisor"] = {"host": {"state": "present", "name": supervisor}}` from `HostSpec` (or check that `host.device_trust` matches `supervisor`), update `examples/site/device-trust.json` and `tests/fixtures/tendcf-device-trust.json`, bump `GOAL_FILE_SCHEMA_VERSION = 4`, and regenerate `tests/golden/*.goal-file.json`. + 4. Moreover, for `supervisor == "nixos"`, note `compile.py:341-342`: + ```python + if not selected: + return doc, errors + ``` + Today, when a host has **no** selected services (`not selected`), `render_goal_file()` returns `doc` with `domains` containing **only `device-trust` and no `supervision` domain at all**! Under v4's `B.2(d)` (`then: {properties: {domains: {required: ["supervision"], ...}}}`), a `nixos` host with 0 selected services returning early at line 342 would omit `"supervision"` and **fail schema validation (Case 99)**! +- **Concrete fix for E.6 / S4:** Document in E.6 that `nix2cf`'s `render_goal_file()` must: + 1. Refuse any selected service on a `nixos` host (`if supervisor == "nixos" and selected: errors.append(...)`); + 2. For `supervisor == "nixos"`, emit `domains["supervision"] = {"coverage": "deliberately-unmanaged"}` rather than returning early at `if not selected:` with `supervision` omitted; + 3. Set `GOAL_FILE_SCHEMA_VERSION = 4`, populate/verify `device-trust.entries.supervisor`, and update `nix2cf`'s fixtures and goldens from v1 to v4. + +#### Finding 9 [LOW] — Minor cross-reference & line-number nit in `stage-d-design-report.md` +- **Evidence:** [`stage-d-design-report.md` line 115](file:///Users/djbclark/.local/state/bigteam/tendcf-stage-d-design-20261031/stage-d-design-report.md#L115); [`schema/goal-file.schema.json` line 263](file:///Users/djbclark/src/tendcf-review-policy-wt/schema/goal-file.schema.json#L263). +- **Problem:** Line 115 says: *"Update the `trust_domain` description (line 263, 'four kinds')"*. Line 263 of `schema/goal-file.schema.json` does not actually contain the phrase `"four kinds"` (that phrase is in `docs/architecture/goal-file-schema-reconciliation-2026-08-15.md:371`, `bin/projector.py:74`, and `bin/schema_lint.py:1795`). Meanwhile, `schema/approval-record.schema.json:10` (which the author could not read, line 331) is confirmed to have `"const": 3` at line 10, and `deploy/linux/canary/goal-file-absent.json` (unread, line 332) is confirmed to be a v3 `systemd` goal file with a tombstone `tendcf.canary.service: {"state": "absent"}` and `unit-writer` `tendcf.canary.*: cfengine`. + +--- + +### Concrete Proposed Schema (`B.2(c)–(d)`) + +In [`schema/goal-file.schema.json`](file:///Users/djbclark/src/tendcf-review-policy-wt/schema/goal-file.schema.json), use the following for the top-level `allOf` (`B.2(d)`) so `supervision` also explicitly forbids `entries: false` regardless of its `coverage` value, and non-object `domains` cannot vacuously trigger `if`: + +```json + "allOf": [ + { + "if": { + "type": "object", + "properties": { + "domains": { + "type": "object", + "properties": { + "device-trust": { + "type": "object", + "properties": { + "entries": { + "type": "object", + "properties": { + "supervisor": { + "type": "object", + "properties": { + "host": { + "type": "object", + "properties": { + "name": { "const": "nixos" } + }, + "required": ["name"] + } + }, + "required": ["host"] + } + }, + "required": ["supervisor"] + } + }, + "required": ["entries"] + } + }, + "required": ["device-trust"] + } + }, + "required": ["domains"] + }, + "then": { + "properties": { + "domains": { + "required": ["supervision"], + "properties": { + "device-trust": true, + "supervision": { + "properties": { + "coverage": { "const": "deliberately-unmanaged" }, + "entries": false + } + } + }, + "additionalProperties": { + "properties": { "entries": false } + } + } + } + } + } + ] +``` + +--- + +### Operator Questions Table (Q1–Q7 + Missing Question Q8) + +| Question | Status (Open vs. Already Settled) | Reviewer Recommendation | Reason & Evidence | +| --- | --- | --- | --- | +| **Q1 — `supervisor` required on every v4 goal file (`launchd\|systemd\|runit\|nixos`) vs. optional (`nixos`-only)?** | **Practically settled** by house rules + D2 | **Required on every v4 goal file**, with the B.3 `goal cross-file` lint rule enforcing `unit` flavour == `supervisor` on present services. | [`schema/goal-file.schema.json:5`](file:///Users/djbclark/src/tendcf-review-policy-wt/schema/goal-file.schema.json#L5) forbids optional scalars with implicit defaults (*"No defaults anywhere: every scalar is required with its explicit value"*). Making `supervisor` optional means omission secretly defaults to "prefix scheme / non-nixos", which is the exact two-spellings/hidden-default hazard F-1 closed. Furthermore, [`linux-targets.md:38`](file:///Users/djbclark/src/tendcf-review-policy-wt/docs/architecture/linux-targets.md#L38) (Step 6) requires `supervisor` from inventory for `systemd` rendering anyway. | +| **Q2 — Where `supervisor` lives: singleton `device-trust` kind vs. top-level header field?** | **Practically settled** by reconciliation §7 (G-3) | **Singleton `device-trust` kind (`device-trust.entries.supervisor.host`).** | [`goal-file-schema-reconciliation-2026-08-15.md:374-383`](file:///Users/djbclark/src/tendcf-review-policy-wt/docs/architecture/goal-file-schema-reconciliation-2026-08-15.md#L374-L383) (G-3) explicitly locked the header at `schema_version` + `host` and moved host-level inventory facts (`trust_tier`) into `device-trust` so `goal-diff.schema.json` needs no header-hunk machinery beyond `version_bump`. Putting `supervisor` in `device-trust` gives privileged-ceremony derivation and projector exclusion (`{"vars":{}}`) with zero diff-schema changes. | +| **Q3 — `supervision` coverage on a `nixos` host: `deliberately-unmanaged` vs. `comprehensive` with no entries?** | **Open** (genuine design trade-off, though constrained by schema) | **`deliberately-unmanaged`** (for `schema_version: 4`). | Under [`schema/goal-file.schema.json:47-51`](file:///Users/djbclark/src/tendcf-review-policy-wt/schema/goal-file.schema.json#L47-L51), `deliberately-unmanaged` already makes `entries` structurally unrepresentable, and under [`bin/schema_lint.py:1500-1503`](file:///Users/djbclark/src/tendcf-review-policy-wt/bin/schema_lint.py#L1500-L1503) any coverage transition into or out of `deliberately-unmanaged` is automatically `privileged`. Under D3 (`cf-execd` off, no extra-entry detector on Linux yet), `comprehensive` would claim tendcf is actively auditing extra units on `get` when it is not. | +| **Q4 — Bind the Nix side into the signed goal file in v4 (e.g. `flake.lock` digest) vs. display-only D1 consent diff?** | **Settled by D1 + D3** | **Display-only in D1 consent artifact for v4** (do not add a Nix digest to `goal-file.schema.json` in v4). | D1 defined the consent diff for a Nix host as `flake.lock` + Nix-side diff + evaluated-options JSON, and D3 ruled that tendcf enforces nothing on a Nix host (`cf-execd` off, no `tendcf-agent`). Adding an unverified digest to `goal-file.schema.json` violates [`goal-file-schema-reconciliation-2026-08-15.md:418-425`](file:///Users/djbclark/src/tendcf-review-policy-wt/docs/architecture/goal-file-schema-reconciliation-2026-08-15.md#L418-L425) (F-2: no speculative fields nothing actuates or verifies). | +| **Q5 — Distro family: ride along in v4 vs. stay deferred?** | **Already settled** by operator ruling (2026-10-31) | **Stay deferred.** | The operator already ruled *"Defer until step 6/8 needs it"* (`06c9929` handoff decisions, 2026-10-31), and nothing in Step 6 (`nix2cf` systemd renderer) or Step 8 (`nixos` passthrough) branches on distro family once `supervisor` (`launchd`\|`systemd`\|`runit`\|`nixos`) is present. | +| **Q6 — Who approves the D1 consent artifact, and where?** | **Open** (operational workflow question) | **Operator approves on the host-flake PR** (`forgejo.mit.edu/djbclark/get` and `github.com/djbclark/site-djbclark`), gated by a required CI job that prints the D1 consent diff, uploads it as a build artifact, and fails hard if `execd.enable` is `true` or unknown module options appear. | Matches [`nixos-passthrough-plan.md:53-59`](file:///Users/djbclark/src/tendcf-review-policy-wt/docs/architecture/nixos-passthrough-plan.md#L53-L59) (Stage B already runs in `nixos-cfengine-build.yml` on PRs) and D3 (no on-device `tendcf-agent` or `cf-execd` runs on `get`). | +| **Q7 — Drop `"nixos"` from `goal-file.schema.json`'s `unit_writer_entry.writer` enum in v4, while leaving `launchd-writers.schema.json` alone?** | **Open** (follow-on from D2) | **Yes — drop `"nixos"` from `goal-file.schema.json`'s `unit_writer_entry.writer` enum in v4**, and **also drop `"nixos"` from `launchd-writers.schema.json:36`** (or document why a macOS launchd writer file ever had `nixos`). | On a `nixos` host, `unit-writer` entries are forbidden by `B.2(d)`; on a non-`nixos` host (`launchd`, `systemd`, `runit`), NixOS writes no units (`nix-darwin` is the macOS Nix writer). Keeping `"nixos"` in `unit_writer_entry.writer` leaves a dead second spelling of NixOS ownership. Note: `f814f1f` only added `"nixos"` to `launchd-writers.schema.json` earlier today to keep the enums identical; `nixos` never writes macOS `launchd` plists (`nix-darwin` does). | +| **Q8 (Missing Question) — How should `unit-writer` name non-reverse-DNS `systemd` units on Debian/Ubuntu (e.g. `dpkg`-owned `ssh.service`, `cron.service`)?** | **Missing from Q1–Q7** (raised only as Risk 4 in G, line 267) | **Ask the operator as a separate Step 6 question before migrating comprehensive `systemd` hosts beyond the canary.** | `unit_writer_map.propertyNames.pattern` (`^[a-z][a-z0-9]*(\.[A-Za-z0-9][A-Za-z0-9_-]*)+\.\*$`) requires at least two dot-separated labels before `.*` (works for `tendcf.canary.*`, fails for `ssh.service`). Commit `f814f1f` added `"dpkg"` to the `writer` enum in v3, but `"dpkg"` is currently unusable for real Debian package units because no legal `unit-writer` key can match a single-label `.service`. | diff --git a/docs/architecture/schema-v4-opinion-fable.md b/docs/architecture/schema-v4-opinion-fable.md new file mode 100644 index 0000000..a83a2a5 --- /dev/null +++ b/docs/architecture/schema-v4-opinion-fable.md @@ -0,0 +1,100 @@ +STATUS: recovered by acp-dispatch from the agent's final message (the agent did not write the report file) +I'll start by reading the design document and AGENTS.md.Now the schemas, lint, projector, and policy in one batch.One contract item left before writing: check the inbox for lead messages.STATUS: complete (read-only run; delivered as final message) + +# Stage D design review (Fable 5.1, independent seat 2 of 2) + +Design read whole; repo read at `review-policy-rule` (cd7cb26). Verified by reading, not running: `bin/schema_lint.py` paths cited below, the installed jsonschema 4.26.0 in the schema-lint uv env, all four schemas, the policy bundle, both deploy goal-file sets, nix2cf `compile.py`/`cli.py`, the NixOS module. Not run: the lint, check.py, cf-agent, nix. Not read: get's flake/lock (not on this machine). + +**Verdict: sound with changes.** The schema delta (B.2) does what it claims under 2020-12; the device-trust placement (B.1) is the right one and D2 is satisfied in substance. The consent-diff section (D) has two defects that must be fixed before S3, the migration plan (E) miscounts the regeneration set and misses nix2cf's real state, and three doc/message details are wrong by one step. + +## 1. Schema deltas (B.2) + +Correct, with evidence: + +- `then.properties.domains.additionalProperties` applies to every member of `domains` not named in that same subschema's `properties` (2020-12 semantics; jsonschema `find_additional_properties` reads the sibling `properties`). So **`"device-trust": true` is needed** (else `entries: false` hits device-trust, which requires `entries` at `goal-file.schema.json:300`) **and sufficient** (device-trust's kind set is closed at `:279`, so it can carry no state kind). +- `supervision` is matched by `then.properties`, so it escapes `additionalProperties`; the `const` on coverage plus base `domains.additionalProperties → state_domain` (`:26`, `:47-51`) forbids its `entries`. True today. +- No route remains for a service, tombstone, unit-writer or interlock on a nixos file: device-trust (closed), supervision (coverage const → no entries), every other domain (`entries: false`). Domain names are still held to `propertyNames` (`:21`). +- The `if` chain uses `required` at every level, so a file without `supervisor` never enters `then`. Right direction. +- Attribution: `if_` descends with `schema_path="then"` (`_keywords.py:386`), `if` itself is skipped (`validators.py:440`), `BRANCH_LOCATORS` maps `then`→`if/then` (`schema_lint.py:960`). Cases 96–99 can claim `if/then`. Verified, not [mem]. + +**F1 (Low, schema).** B.2(d)'s "supervision needs no `entries: false` of its own" rests on supervision being routed through base `additionalProperties`. If supervision ever gets its own def under base `properties`, the nixos rule silently loses its entries prohibition. Add it explicitly; it costs nothing and names the member: +```json +"supervision": { "properties": { "coverage": { "const": "deliberately-unmanaged" }, "entries": false } } +``` + +**F2 (Medium, schema shape).** Blanket `entries: false` on other domains says more than D3 ("enforce nothing *for now*") and more than the repo's stated future for such hosts: nixos-passthrough-plan.md:86-88 names files and the report channel as the only enforceable things there, and linux-targets.md:66-68 makes the lockfile/consent diff tendcf's job on a passthrough host (Q4's eventual binding). Both would be non-trust entries that this `then` forbids. Today `state_entries` has exactly three kinds (`:57-61`), so the two forms are equivalent; the per-kind form reads as D3 and survives a kind addition: +```json +"additionalProperties": { "properties": { "entries": { "properties": { "service": false, "interlock": false, "unit-writer": false } } } } +``` +Operator's call; I prefer per-kind. + +**F3 (Low, message).** jsonschema returns a False-schema error *before* appending path/schema_path (`validators.py:406-414` vs `:442-445`). Case 97's finding is located at `domains/apps`, not `domains/apps/entries` as the table says, and prints the whole entries object. Per-kind (F2) gives `domains/apps/entries` and prints only the offending kind map. Risk 5 (``) does not occur for any of 93–101: each has a non-empty instance path. + +## 2. `supervisor` as a device-trust kind (B.1) + +Sound against the diff/approval/ceremony model: `derive_ceremony_class` is positional (`schema_lint.py:1498-1499`), `apply_diff` handles any trust kind generically (`:1531-1570`), `refused` paths address `hunks/device-trust/supervisor/host` (`:1651-1654`), `check_trust_inexpressible` iterates whatever kinds `examples/goal-file.json` carries (`:1841`) and smuggles both kind and body (`:1857-1875`): the body `{"name","state"}` fails every state kind (service oneOf, interlock required set, unit-writer `additionalProperties:false`). Projector exclusion is by domain name (`projector.py:120`), so `{"vars":{}}` holds. + +Strongest case against, stated honestly: (i) `trust_domain`'s charter is "the validator and agent read their own configuration ONLY from here" (`:263`); `supervisor` is read by the compiler and lint, never by the device, so it is the first trust kind whose consumer is off-device. (ii) D2's literal "host-level" names the header. (iii) The tightening direction (systemd→nixos) is made privileged though it removes privilege. Answers: (i) the entry's job is to *gate* unit-writer legality, which is gate machinery, and §7 puts gate machinery nowhere but device-trust; (ii) reconciliation §7 G-3 (`:374-383`) closed the header for exactly this class of fact and gave the diff-machinery reason, so "host-level" can only mean per-host; (iii) same cost trust-tier already pays. **D2 satisfied.** Q2 is therefore not open (see table). + +**F4 (Medium, second spelling).** With Q1=required, flavour and supervisor are always equal on launchd/systemd/runit hosts: the redundancy the schema's own text forbids (`:34` "unrepresentable instead of if/then-guarded"; reconciliation `:380-383`). B.3's lint is that if/then-guard in Python. Accept the redundancy as *named residue* in the `supervisor_map` description ("`unit`'s flavour is redundant with this fact on a non-nixos host; collapsing `unit` is a later bump"), and express the rule where the corpus puts such rules, in the schema, so the harness's `schema (if/then)` discipline covers it: three `allOf` members, one per manager, each `if supervisor.host.name == X then every domains/*/entries/service/* with state present has unit required [X]`. The lint form is acceptable if the operator prefers its message; either way the residue must be written down. + +## 3. Fail-closed (C) + +- C.1 correct: `const` (`:11`) and closed trust kinds (`:279`); case 13 claims 4 today (verified). The "distinct version-above-ceiling report" is **not implemented anywhere** (grep `ceiling` in `bin/` hits only the projector's 5 MiB rule); `:10` promises it. Pre-existing; record in the bump PR as residue, not fixed here. +- C.2 correct (`report-row.schema.json:118-122`; reconciliation `:309-318`). +- **F5 (Medium, pre-existing, must be written down).** A `version_bump` diff cannot exist in the corpus for *any* bump: `apply_diff` ignores `version_bump` (`:1521-1523`) and the family check compares applied bytes to the proposed file (`:1713-1719`), which differ at least in `schema_version`. v2 and v3 were therefore re-adoptions in practice; v4 (adds required content) is one in principle. `derive_ceremony_class` already returns `baseline` for `version_bump` (`:1496-1497`), so the class is right and only the rule is unstated. Proposed text for `goal-diff.schema.json` `version_bump` description and reconciliation §9.4: "`version_bump` alone is valid only when the baseline rendered at `new` is byte-identical to the baseline apart from `schema_version`; a bump that adds or changes required content is a re-adoption: a baseline ceremony with no diff object." C.3's "ship the validator as an agent hunk under v3" is moot while no agent exists; say so. +- C.5 confirmed line by line: `_unit_present` `:227-229` and `_teardown` `:231-232` need `tendcf_systemd_host` (`:184-185`, writable dir), `_refused` `:199-200` needs `_present`, reports `:314-318` need `_refused`, launchd cleanup `:256-258`/`:272-274` is `darwin`-gated. A tombstone on a symlinked-units host is silent. S0 is right. Check 7 (`tests/policy/run.sh:247-276`) runs only `systemd-present.json`; 7b needs a `systemd-absent.json` pass under the shape. Proposed class and report: + ``` + "service_$(service_canon[$(service_ids)])_tombstone_elsewhere" + expression => "tendcf_systemd_units_elsewhere.!service_$(service_canon[$(service_ids)])_present"; + reports: "service_..._tombstone_elsewhere":: "tendcf: service '$(service_ids)' tombstone not actuated: /etc/systemd/system is not a writable directory on this host ($(sys.flavour)); its units are the system configuration's (supervisor nixos)"; + ``` + After Q7, `:168`, `:318` and the plan's "unit writer nixos" wording should say "supervisor nixos". + +## 4. Consent diff (D) + +**F6 (High).** The AUTHORITY block is the lock diff only. A lock diff shows `(input, rev, narHash)`; it does not show what changed in the input, and it shows *nothing* for a change to get's own flake files (its configuration is in the get repo, not an input). D1's "flake.lock + Nix-side diff" must be read as: the PR's own diff of the host-flake repo (flake.lock included), plus, for each changed git input, `git diff ..` of that input restricted to the paths the host imports (`nixos/` for site-djbclark, per nixos-passthrough-plan.md:16). `diff-closures` is the *effect*, not the authority; keep it as the third block. Proposed D step 1: "Authority = the PR diff of the host-flake repository, with `flake.lock` reduced to `(input, type, rev, narHash)` per changed node, followed by the source diff of each changed input at the imported paths." + +**F7 (High).** Hard check 2, "any options key outside the summary appears", cannot fire: `--apply` projects exactly three keys, so nothing else can appear in its output. What is wanted is: the *declared* option set under `options.services.cfengine-tendcf` on head equals `{enable, package, execd.enable}`; a module that grows an option fails CI until the summary is extended. Implement with `nix eval --json .#nixosConfigurations.get.options.services.cfengine-tendcf --apply` over `lib.attrNames` recursively (or `lib.collect lib.isOption`). Hard check 1 should be stated against the evaluated host config and strengthened to the effect, not the knob: assert `config.systemd.services` has no `cfengine-tendcf-execd` (module `:40-49`) and no unit or timer whose `ExecStart` references `${config.services.cfengine-tendcf.package}`; a user's own `systemd.services.x` running cf-agent bypasses the option. + +**F8 (Low).** `package = "${c.package}"` is a store path whose hash changes on any rebuild, so a no-op rebuild shows a changed summary line. Use `c.package.version` (the compiled-in `3.29.0a.` stamp, plan `:45-46`) beside the path. The `nix eval` form and the `--apply` interpolation are otherwise right; the attribute is `nixosConfigurations.get` (lead-verified). + +D's "not the device", "not nix2cf" are correct (`projector.py:14-15`; linux-targets.md:38). Note nix2cf *does* produce the nixos host's (service-less) goal file at S4; only the Nix-side diff is out of its scope. + +## 5. Migration and tests (E, F) + +**F9 (Medium, counts).** "All 63 goal-family overlays … each needs the supervisor entry" is wrong. 63 is the case total. Verified on disk: 40 cases overlay `goal-file.json` (13–43, 61–65, 70–73), 2 overlay `goal-file-baseline.json` (52, 53), and 12 overlay `goal-diff.json`/approval records (49–57, 59, 60). `examples/goal-file-baseline.json` carries only device-trust (verified), so it gains `supervisor` too; both happy hashes change, so every diff/record fixture needs `schema_version: 4` **and recomputed `baseline_sha256`/`proposed_sha256`**, not a supervisor entry. `broken-bytes/44-48` are derived from the happy file; the byte layer never checks derivation (`schema_lint.py:613-630`), so they pass untouched, but README:275 claims "the happy goal file, indented"; regenerate them for honesty. The lint catches every miss (`EXPECTED_BROKEN`, hash rails), so this is effort, not correctness; script the regeneration. + +**F10 (Medium, nix2cf).** nix2cf is at v1 (`compile.py:40`; `tests/test_compile.py:84`; v1 goldens), refuses every non-macos host (`compile.py:60`, `:328-333`), and copies device-trust verbatim from an operator-supplied document (`cli.py:71,91`; `compile.py:335`); `platform` is a CLI flag (`cli.py:153`), and there is no inventory file or schema (`compile.py:96-112` TODO; tendcf `schema/` has none). S4 is therefore a v1→v4 catch-up (v2 `restart`/`wanted_by` + alphabet, v3 writers, v4 supervisor + nixos refusal + goldens), not "consume supervisor". Add a cross-repo pin check so nix2cf's `GOAL_FILE_SCHEMA_VERSION` drifting from tendcf's const is red, like `overlay/fork.json`. And D2's "inventory fact" has no inventory to live in: Q8. + +**F11 (Low).** Numbering from 93 is right (73 at README:225, projection 92 at `:338`, 70–73 duplicated across directories at `:73-75`/`:316-319`). 3-digit names: `_README_ROW` `\d+` (`:416`) and `sorted(iterdir())` (`:1220`) never parse the number; fine. [mem] error strings all match 4.26.0: `_keywords.py:87` (const), `:271` (enum), `:309` (required); `validators.py:408` (False schema). Case 95 reads `'host' was expected` at `domains/device-trust/entries/supervisor` (propertyNames descends without a path, `:33`). + +**F12 (Low).** Add one family case: a supervisor hunk approved `ordinary` → `family (ceremony)`. Case 52 already uses a `policy-tree` hunk (verified), so B.1(3)'s load-bearing claim has no fixture of its own; 102 costs one overlay set. + +**F13 (Low, doc counts).** "four kinds" is not at `goal-file.schema.json:263`; the count lives at reconciliation `:369-373`, `schema_lint.py:1795,1810`, `projector.py:74,79,81,172`. `projector.py:72-81` states a *measured* "three of four" for N-5; the supervisor body has no digest, so v4 makes it three of five. Add to E.5. + +**F14 (Low, Q7 spillover).** `launchd-writers.schema.json:36` still lists `nixos`; `examples/launchd-writers.yml:30-32` uses `nix-darwin` for `org.nixos.*`. A launchd writer named `nixos` is meaningless; dropping it there too (contract_version axis) avoids a Site Model writer the goal file cannot carry. No lint compares the two enums today. + +E.1/E.2/E.3 otherwise check out: projections unchanged by P-1; `goal-file-absent.json` (I read it) needs only the supervisor entry; `check.py:34-36` reword; `host_specific*.json` byte-identical. + +**F15 (Low, risk 4 is a question).** Confirmed pre-existing: `:243` requires two dot-labels before `.*`, so `ssh.service` under `dpkg` is undeclarable and pastebin's comprehensive domain would report every Debian unit as extra once detection exists. It touches the same def v4 edits; a second bump for it is the cost the design elsewhere avoids. Q9. + +**F16 (Low).** The passthrough file's host key and trust digests are the `examples/` placeholders (verified identical). It is a fixture, not get's consent object; D's "what a person is asked" reads as if it were. Label it so in `check.py`/README until get has a key. Q10. + +## Q1–Q7 + +| Q | Recommendation | Reason / status | +|---|---|---| +| Q1 | **Required**, closed enum | Optional-absent = hidden default (`:5`). Open. Carry F4's residue. | +| Q2 | device-trust | **Not open**: reconciliation §7 G-3 closed the header (`:380`) with the trust-tier precedent; D2's "host-level" means per-host. Reopening G-3 is a separate ruling. | +| Q3 | `deliberately-unmanaged` | Passthrough is permanent by design (linux-targets §5), not backlog; `comprehensive`-empty asserts tendcf describes the domain completely, which is false, and would count 116 units as extra (report-row `:87` is a two-writers signal, and there is one writer, declared by the fact). Leaving it is privileged (`:1502`). Open. | +| Q4 | Display-only in v4 | No verifier exists. But linux-targets.md:66-68 makes the lockfile tendcf's job on passthrough hosts, so name the future shape as residue and keep v4's `then` from precluding it (F2). Open. | +| Q5 | Defer | Nothing reads it; step 6 needs supervisor, not distro family; §8 rule "carry only what is actuated"; regeneration is scripted (F9). Open. | +| Q6 | Required human review + CI artifact on the host-flake PR | Write it into nixos-passthrough-plan.md as the interim consent channel under D3; it has no nonce/seq/signature and is not the approval record. Open. | +| Q7 | Drop `nixos` writer | Plus F14 and the policy strings in F-C.5. Open. | + +Missing: **Q8** where `supervisor` is authored (no inventory schema or file exists; F10). **Q9** the dpkg prefix defect in the same def (F15). **Q10** whether `deploy/linux/passthrough/goal-file.json` is get's goal file or a fixture, and who computes get's real device-trust values with no agent to hold a key (F16). + +DECIDED: none needed; nothing blocking. Severity order for the lead: F6, F7 (before S3); F9, F10, F5, F2, F4 (in S1/S2 text); the rest are one-line edits. + +DONE diff --git a/docs/architecture/schema-v4-opinion-opus-draft.md b/docs/architecture/schema-v4-opinion-opus-draft.md new file mode 100644 index 0000000..e6f98b7 --- /dev/null +++ b/docs/architecture/schema-v4-opinion-opus-draft.md @@ -0,0 +1,344 @@ +STATUS: recovered by acp-dispatch from the agent's final message (the agent did not write the report file) +Reading the schema lint's negative-case table and the passthrough/canary goal files next.Checking report-row's validator version field, the projector, and the passthrough/canary goal files.Checking whether an inventory schema exists and how the goal-diff header is handled, since that decides where `supervisor` can live.I couldn't write the report file or its `.done` marker: this run is read-only. So the whole design document is below, and acp-dispatch saves it to `/Users/djbclark/.local/state/bigteam/tendcf-stage-d-design-20261031/stage-d-design-report.md`. + +STATUS: complete (read-only run; delivered as final message) + +# tendcf Stage D design: `supervisor: nixos`, schema_version 4, consent diff + +Design lead: Opus 5.5 high. Repo read: `/Users/djbclark/src/tendcf-review-policy-wt` at branch `review-policy-rule` (9118bb1). + +**What I could not do.** I had no shell in this run. That means I did not run `book-kb`, `graft`, `bin/schema_lint.py`, `check.py`, `nix` or anything on get. Every CFEngine claim below comes from repo code or comments, not from the book, and is marked [code]. Every jsonschema error string is written from memory and marked [mem]. Settled facts 1–4 are taken from the brief as given. + +## A. Problem, and what D1–D3 require + +The reverse-DNS unit-writer prefix pattern (`goal-file.schema.json:243`) cannot name any NixOS unit. The passthrough goal file's `org.nixos.*` matches none of get's 116 units (fact 1). The rulings require: + +- **D2.** A host-level `supervisor: nixos` fact, added in schema_version 4. It claims the host's whole unit namespace with no per-unit prefix. Nix2cf step 6 reads it. +- **D3.** On a nixos-supervisor host, tendcf actuates nothing. It reports and refuses, and cf-execd stays off. The schema must make service actuation unrepresentable there, tombstones included. +- **D1.** The consent artifact for a Nix host shows three things: + - the flake.lock diff plus the Nix-side diff, as the authority; + - the evaluated-options JSON, as a readable summary; + - `nix store diff-closures`, which already exists. + +**Questions for the operator.** The rulings leave these open. I recommend an answer for each but do not decide them. + +- **Q1 — required or optional.** Is `supervisor` required on every v4 goal file, with the closed set `launchd|systemd|runit|nixos` and a lint forcing unit flavour to match? Or is it optional and `nixos`-only? + - Recommend required. The house rule is "every scalar is required with its explicit value" (`goal-file.schema.json:5`). An optional fact whose absence means "the prefix scheme governs" is a hidden default. + - The cost: a lint is needed to keep `supervisor` from becoming a second spelling of the unit flavour. Reconciliation §7 rejected a platform field for exactly that reason (`goal-file-schema-reconciliation-2026-08-15.md:380-383`). +- **Q2 — where it lives.** Is it a singleton `device-trust` kind (recommended) or a literal top-level header field? See B.1. +- **Q3 — supervision coverage on a nixos host.** Should it be `deliberately-unmanaged` (recommended) or `comprehensive` with no entries? + - `deliberately-unmanaged` gets "no entries" from the existing `if/then` (`goal-file.schema.json:47-51`), and leaving it later is a privileged ceremony (`bin/schema_lint.py:1502`). + - The cost: the device runs no extra-entry detection on that domain. +- **Q4 — binding the Nix side.** Does v4 bind the Nix side into the signed goal file, for example a flake.lock digest in the way `policy-tree` binds policy bytes? Or is the D1 consent diff display-only? + - Recommend display-only for v4. Under D3 tendcf actuates nothing Nix-side, so a bound digest would be a claim nobody enforces. + - Binding it later is its own bump. **This is a data-shape choice; a wrong answer means another bump.** +- **Q5 — distro family.** Does it ride along in v4 (one bump instead of two), or stay deferred? It is the other deferred "slice B" field. Not ruled; no recommendation. +- **Q6 — who approves.** With no tendcf-agent and get deployed by its own image pipeline, who approves the D1 artifact, and where? Recommend a required PR check and artifact on the host-flake repository (get / site-djbclark) that gates merge. +- **Q7 — the `nixos` writer value.** Drop `nixos` from the goal-file `writer` enum in v4 (recommended: it has no legitimate use left; see B.3)? Leave Site Model `launchd-writers` alone, since it is versioned by contract_version (`schema/launchd-writers.schema.json:9`), not by the goal-file family. + +## B. Schema change for schema_version 4 + +### B.1 Where `supervisor` lives + +**Recommendation: a fifth `device-trust` kind, singleton.** + +```json +"supervisor": { "host": { "state": "present", "name": "nixos" } } +``` + +Reasons: + +1. **There is precedent.** `trust_tier` is a Site Model inventory fact (`architecture-DEFINITIVE-v3.md:178-180`). In the goal file it became a `device-trust` entry, explicitly "rather than a header field … the diff schema needs no header-hunk machinery" (`goal-file.schema.json:321`; reconciliation `:374-383`). `supervisor` is the same kind of thing: an inventory fact that sets privilege scope. +2. **The header is closed.** The diff can change exactly one header thing, `version_bump` (`goal-diff.schema.json:26`). A top-level `supervisor` could not be changed by any diff. Every change would force a new first-adoption ceremony, and `apply_diff` (`bin/schema_lint.py:1509ff`) could not reproduce one. +3. **Privilege comes for free.** Any hunk under `device-trust` is privileged (`bin/schema_lint.py:1498-1499`). So nixos → systemd, which unlocks CFEngine unit writes, can never pass as an ordinary ceremony. +4. **It never reaches the device.** The projector excludes `device-trust` (`bin/projector.py:23-36`), so `{"vars":{}}` is unchanged. +5. **The trust-region property test covers it.** `check_trust_inexpressible` iterates whatever trust kinds the happy fixture carries (`bin/schema_lint.py:1841`), so a fifth kind is covered once `examples/goal-file.json` has it. + +The literal "host-level" wording in D2 could also mean a top-level header field. That is why Q2 exists. + +### B.2 Exact deltas: `goal-file.schema.json` + +(a) `schema_version` changes from `"const": 3` to `"const": 4` (line 11). Make the same change in `goal-diff.schema.json:10` and in approval-record. The approval-record line is unread, but the family shares one version (reconciliation `:298-301`). + +(b) In `trust_domain.entries.properties` (lines 272-277), add: + +```json +"supervisor": { "$ref": "#/$defs/supervisor_map" } +``` + +Add `"supervisor"` to the `required` array at line 278 (if Q1 = required). + +(c) New `$defs`: + +```json +"supervisor_map": { + "description": "The host's unit-namespace authority (inventory `supervisor`, resolved). Singleton. `nixos`: every unit is written by the NixOS configuration; tendcf actuates no unit and no domain carries a state entry (D2, D3). `launchd`/`systemd`/`runit`: the manager CFEngine renders for; per-prefix unit-writer entries govern, and every present service's unit flavour equals this value (lint). A change is a device-trust hunk, therefore privileged.", + "type": "object", + "propertyNames": { "const": "host" }, + "additionalProperties": { + "type": "object", + "properties": { + "state": { "const": "present" }, + "name": { "enum": ["launchd", "systemd", "runit", "nixos"] } + }, + "required": ["state", "name"], + "additionalProperties": false + }, + "minProperties": 1 +} +``` + +(d) Top-level `allOf`, a sibling of `required` at line 29. This is the nixos rule: + +```json +"allOf": [{ + "if": { "properties": { "domains": { "properties": { "device-trust": { "properties": { "entries": { "properties": { "supervisor": { "properties": { "host": { "properties": { "name": { "const": "nixos" } }, "required": ["name"] } }, "required": ["host"] } }, "required": ["supervisor"] } }, "required": ["entries"] } }, "required": ["device-trust"] } }, "required": ["domains"] }, + "then": { "properties": { "domains": { + "required": ["supervision"], + "properties": { + "device-trust": true, + "supervision": { "properties": { "coverage": { "const": "deliberately-unmanaged" } } } + }, + "additionalProperties": { "properties": { "entries": false } } + } } } +}] +``` + +Three details matter here: + +- **`"device-trust": true` is needed.** Without it, the `then`'s `additionalProperties` would also apply to device-trust, which must carry `entries`. That would make every nixos file invalid. +- **`entries: false`, not `not: required`.** Case 57's precedent is that the `false`-schema form names the offending member. The `not` form prints the whole document (`examples/broken/README.md:193-198`). +- **Supervision needs no `entries: false` of its own.** The coverage const plus the existing `state_domain` `if/then` (`:47-51`) already forbid entries there. + +(e) `unit_writer_entry.writer` (line 255): remove `"nixos"`, if Q7 is approved. Update the description at line 254. `nix-darwin` stays a per-prefix writer: nix-darwin shares the launchd namespace with Apple, Homebrew and CFEngine labels. + +(f) Update the `trust_domain` description (line 263, "four kinds"), the `unit_writer_map` description (line 239), and the `service_map` propertyNames description (line 77) to mention the supervisor rule. + +### B.3 How it interacts with unit-writer, and the two-writers invariant + +- **On a nixos host** there are no `unit-writer` entries in any domain; (d) forbids every domain's entries. The supervisor entry *is* the whole-namespace claim. No wildcard entry exists, as the operator ruled. + - "Two writers on one namespace" holds structurally: there is exactly one writer, and the schema refuses a second declaration anywhere. The lint is not involved. + - The prefix nesting rule (`bin/schema_lint.py:1431-1441`) is unchanged and vacuous on such a host. It finds no prefixes. +- **On launchd/systemd/runit hosts** nothing changes: the prefix pattern, nesting and the cfengine-writer rules (`:1443-1465`) stay as they are. +- **New goal cross-file rule** (if Q1 = required), in `check_goal_file_cross_file` after line 1465. For a supervisor in `{launchd, systemd, runit}`, every present service's single `unit` key must equal the supervisor name. Message: + + `"{label}: domains/{d}/entries/service/{id} has unit flavour {f!r} but the host supervisor is {s!r} — one host, one manager"` + + This turns the second spelling into a checked one, which is reconciliation §7's objection answered. +- **`nixos` as a writer value** has no remaining use after (d): on a nixos host unit-writer is forbidden, and on any other host nixos writes no units. Keeping it would be a second way to say "nixos owns these units". Hence Q7. + +## C. Fail-closed analysis + +1. **An old validator (ceiling 3) reading a v4 file** is refused twice: + - `schema_version` is a `const`, so 4 fails structurally (`:9-12`); case 13 already proves this at 4. + - The trust `entries` object is `additionalProperties: false` (`:279`), so the `supervisor` kind is refused even if the version check were bypassed. + + The schema description promises "version-above-ceiling reported distinctly from schema-invalid" (`:10`). I found no code doing that dispatch in `schema_lint.py`; no grep hit for "ceiling" near validation. tendcf-agent does not exist (`bin/projector.py:14-15`). So today this is a schema-const refusal only. **Unverified that anything implements the distinct report.** +2. **D44 render rule.** `device_convergence.schema_ceiling` is an integer with minimum 1 and no maximum (`schema/report-row.schema.json:118-122`). The compiler refuses to render version N for a host whose reported ceiling is below N (reconciliation `:314-318`). A never-reported host renders at its enrolled version (`:309-313`). No report-row change is needed. + - get has never reported, there is no Linux row writer (`linux-targets.md:49`, D18 open), and no enrollment ceremony exists. + - **DECIDED (conservative):** get's first adoption, when it happens, enrolls at 4. No v3 baseline exists anywhere to migrate from. +3. **Downgrade.** + - A nixos-supervisor host has **no faithful v3 rendering**. The only v3 spelling was the placeholder that matches nothing. The compiler must refuse to render it below 4 rather than fall back to `org.nixos.*`. + - Non-nixos hosts render at 3 by dropping the supervisor entry. That loses the fact only, and the v3 rules did not depend on it. + - Two-phase order is unchanged: ship the v4 validator as an `agent` hunk under v3 (`goal-file.schema.json:357`), then v4 content. + - Pre-existing problem: §9.4's "empty apart from the bump" cannot hold literally for a bump that adds a required field. v2 had the same property ("every fixture and digest regenerated", `linux-targets.md:36`). It is moot while no device holds a baseline. +4. **An unknown supervisor value** fails the closed `enum` (case 93). An unknown trust kind fails `additionalProperties`. +5. **On the device itself**, refusal does not depend on the goal file. cf-agent sees only `{"vars":{}}`. The bundle's filesystem guard refuses any systemd entry where `/etc/systemd/system` is a symlink (`policy/tendcf_services.cf:172-185,317-318`) [code]. + - **Gap found:** a *tombstone* on such a host is neither actuated nor reported. Teardown is gated on `tendcf_systemd_host` (`:229-232`), and `_refused` requires `_present` (`:200`). It is silent, not harmful. v4 makes it unrepresentable in a goal file, but hand-written CMDB data would still hit it. See slice S0. + +## D. Consent diff for D1 + +**Where it is produced:** CI in the host-flake repository, on the PR that changes get's flake (forgejo djbclark/get, or site-djbclark's `nixos-cfengine-build.yml`). + +- **Not the device.** D3 actuates nothing, there is no agent, and get should not evaluate its own successor. +- **Not nix2cf.** Nix2cf emits goal files for path 2 only (`linux-targets.md:38,70-76`); passthrough is path 1. + +It runs three steps on the base and head commits: + +1. **Lock diff.** `git diff` of `flake.lock`, reduced to `(input, type, rev, narHash)` per changed node. +2. **Options summary.** + + ``` + nix eval --json #nixosConfigurations..config.services.cfengine-tendcf \ + --apply 'c: { enable = c.enable; execd.enable = c.execd.enable; package = "${c.package}"; }' + ``` + + The output is JCS-canonicalized so equal options give equal bytes. **The `` attribute name is unverified.** The option names `enable`, `package` and `execd.enable` are verified (`site-public-djbclark/nixos/modules/cfengine-tendcf.nix:21,23,30`). +3. **`nix store diff-closures base head`**, which Stage B already produces. + +**Rendered example.** Real values: get `bf3bc63`, site-djbclark `035fd137` as a non-flake input, package version `3.29.0a.b8abf2840`. Values in `<>` are placeholders I could not read: I had no access to get's lock. + +``` +get.sipb.cloud — proposed Nix change bf3bc63 → + +AUTHORITY: flake.lock + site-djbclark (non-flake input; module imported by path) + rev 035fd137 → + narHash → + all other inputs unchanged + +SUMMARY: settings tendcf watches (services.cfengine-tendcf) + - {"enable":true,"execd":{"enable":false},"package":"/nix/store/

-cfengine-tendcf-3.29.0a.b8abf2840"} + + {"enable":true,"execd":{"enable":false},"package":"/nix/store/

-cfengine-tendcf-"} + +CLOSURE (diff-closures) + cfengine-tendcf: 3.29.0a.b8abf2840 → , <±size> +``` + +**What a person is asked**, in ordinary language: + +> "This changes get's NixOS configuration. Its site-djbclark source moves from 035fd137 to . CFEngine stays installed, its scheduler stays off, and its package changes to . tendcf will still change nothing on this host: it only reports. Approve this change?" + +Two CI checks should be hard failures, not just display: + +- `execd.enable` becomes `true` while D3 stands. +- Any options key outside the summary appears. + +Both turn D3 into a check rather than prose. They live in that repository's CI, which is outside this repo. + +## E. Migration + +1. **`deploy/linux/passthrough/goal-file.json`.** Set `schema_version` to 4. Add `device-trust.entries.supervisor = {"host":{"name":"nixos","state":"present"}}`. Set `supervision` to `{"coverage":"deliberately-unmanaged"}`, deleting the `org.nixos.*` unit-writer. The file stays JCS. + - The projection is **unchanged at `{"vars":{}}`**. The projector reads only `supervision`'s `service` and `interlock` (`bin/projector.py:120-121`), and there are none. This is reasoned from code, not run; `check.py` verifies it (`deploy/linux/canary/check.py:37,64-66`). + - The `check.py` comment at lines 34-36 needs rewording. +2. **Canary goal files** (`deploy/linux/canary/goal-file.json` and `goal-file-absent.json`): add supervisor `systemd`; nothing else changes. `host_specific*.json` stay byte-identical. I read `goal-file.json` but not `goal-file-absent.json`. +3. **`examples/goal-file.json` and `goal-file-baseline.json`** (launchd): add supervisor `launchd`. Regenerate `goal-diff.json` and the approval-record hashes. `host_specific.json` is unchanged. +4. **All 63 `examples/broken/*` goal-family overlays** are full alternate files (`README.md:50-53`). Each needs `schema_version` 4 and the supervisor entry, or it fails for a second reason. The harness tolerates extra findings only if the declared class also fires (`README.md:11-18`), but parenthetical claims are checked (`:163-167`), so fix them all. Also update: + - case 13 to claim `5`; + - `EXPECTED_BROKEN` (`bin/schema_lint.py:87`) from 63 to 72; + - "Sixty-three" (`README.md:3`; `schema_lint.py:36`); + - "cases 13-73" (`README.md:9`). +5. **Docs:** + - `linux-targets.md` step 4 (`:36`) and step 8 (`:40`), and §5 item 4 (`:90`): resolved by D1. + - `nixos-passthrough-plan.md` §4 (`:92-107`) and §5 item 2 (`:113-114`). + - The `tendcf_services.cf:165-169` comment can cite the fact. + - AGENTS.md: no change. It carries no schema facts. +6. **Nix2cf step 6 fields.** + - From inventory: host key, `trust_tier` → `trust-policy`, `supervisor` → `device-trust.supervisor`. + - Renderer choice is a function of supervisor: `launchd` → launchd unit, `systemd` → systemd unit, `runit` → runit unit, `nixos` → no service rendering at all. A service record bound to a nixos host is a compile-time refusal: "host is nixos-supervised; its units are the Nix configuration's". + - Still deferred: distro family (Q5), `scope`, externally-written unit (`linux-targets.md:36`). + +## F. Test plan + +**Numbering.** The brief's "cases 70-72 are the latest" is stale: + +- Case 73 exists (`README.md:225`). +- The numbering is "one series across both directories" (`README.md:282-283`), and broken-projection already runs to 92 (`:336-338`). +- Numbers 70-73 are already used twice, once in broken/ and once in broken-projection/. This is pre-existing (`:73-75,225` vs `:316-319`). + +New cases therefore start at **93**. All overlay `goal-file.json`. + +| # | Case | Broken input | Expected refusal [mem wording] | Caught by | +|---|---|---|---|---| +| 93 | `93-unknown-supervisor` | supervisor `name: openrc` | `'openrc' is not one of ['launchd', 'systemd', 'runit', 'nixos']` | schema (`enum`) | +| 94 | `94-missing-supervisor` | device-trust without `supervisor` | `'supervisor' is a required property` | schema (`required`) | +| 95 | `95-supervisor-not-singleton-id` | supervisor id `main` | const failure on `'main'` | schema (`propertyNames`) | +| 96 | `96-nixos-supervision-comprehensive` | nixos; supervision `comprehensive`, no entries | `'deliberately-unmanaged' was expected` | schema (`if/then`) | +| 97 | `97-nixos-tombstone-elsewhere` | nixos; second domain `apps` (`not-yet-migrated`) holding a `service` tombstone | `False schema does not allow {...}` at `domains/apps/entries` | schema (`if/then`) | +| 98 | `98-nixos-unit-writer-elsewhere` | nixos; domain `packaging` with `org.nixos.*` writer `cfengine` | as 97, at `domains/packaging/entries` | schema (`if/then`) | +| 99 | `99-nixos-no-supervision-domain` | nixos; `supervision` absent | `'supervision' is a required property` | schema (`if/then`) | +| 100 | `100-nixos-writer-retired` | systemd host; writer `nixos` on a prefix | enum failure listing 7 writers | schema (`enum`) | +| 101 | `101-flavour-not-supervisor` | supervisor `systemd`; a `launchd` service | the new cross-file message in B.3 | goal cross-file | + +Notes on the table: + +- Case 101 exists only if Q1 = required. +- Case 100 exists only if Q7 is approved. +- **Unverified:** whether the lint's `$defs` recovery attributes a top-level `allOf/if/then` failure as `if/then` (it does for `state_domain`, `README.md:181-182`). Confirm on the first run and correct the cells. +- **Unverified:** whether fixture discovery handles 3-digit directory names. The README row regex `^\|\s*\d+\s*\|` does accept them (`bin/schema_lint.py:416`). + +**Positive cases:** + +- The passthrough v4 goal file is valid and projects to `{"vars":{}}` (`check.py`). +- The canary and the examples are valid with supervisor `systemd` / `launchd`. +- `check_trust_inexpressible` passes with five kinds. + +**Discrimination.** For each new rule, delete it on its own and confirm its case goes red alone (F8, `bin/flag_coverage.py`). Run one mutant at a time through `bg`. + +**No change** to `tests/policy/*`: the fixtures are CMDB data, not goal files; grep for `schema_version` in tests/ gave no hits. **S0 adds** check 7b: a tombstone on the symlinked layout must produce a report line. + +## G. Risks, rejected alternatives, plan + +**Risks** + +1. On the device, refusal still rests on the filesystem guard; the fact never reaches cf-agent. The tombstone gap is in C.5. +2. The bump touches every family fixture and digest. Any external v3 consumer, such as a nix2cf checkout pinned to v3, breaks. **I did not check the nix2cf repo.** +3. `deliberately-unmanaged` switches off extra-entry detection on Nix hosts (Q3). +4. The prefix pattern defect also hits `dpkg`: Debian units such as `ssh.service` are not reverse-DNS. D2 does not fix that; it needs its own ruling. +5. Schema errors from a top-level `if/then` may report the path as `` (`README.md:199-200`), which is a message-quality risk under D16(a). + +**Rejected alternatives** + +- **Whole-namespace wildcard unit-writer entry** (operator-rejected). It needs no new kind, but: + - every other prefix nests inside it, so the nesting lint would need a special case; + - it puts a host privilege fact into detector data in an ordinary domain. Deleting it would be an *ordinary* hunk that unlocks CFEngine writes, a privilege downgrade without a privileged ceremony. +- **Top-level header field.** The header is fixed at `schema_version` + `host` (reconciliation `:380`). It would need header-hunk machinery, or force re-adoption on every change. +- **Domain-level property on `supervision`.** It needs a dedicated domain def and new diff machinery, and a change would be ordinary-class. +- **Widening the pattern to bare names.** That means 116 entries per host, churn on every NixOS change, and still two spellings of ownership. +- **Projecting `supervisor` into `vars`.** It breaks `{"vars":{}}`, P-1 and the exclusion of device-trust. +- **`nix-darwin` in the supervisor set.** It shares the launchd namespace, and the bundle has no nix-darwin guard, so it stays a prefix writer. + +**Implementation plan (PR-sized slices)** + +- **S0 (independent, small).** In `policy/tendcf_services.cf`, report a tombstone on a symlinked-units host by name (the D3 "report"), and add a test check. +- **S1 (one PR; a bump cannot be split without an unsafe interim shape).** Schema v4 B.2(a)-(f) across the family, the B.3 lint rule, regenerated fixtures and digests, cases 93-101, the count and README updates, and the passthrough and canary migration. +- **S2.** Doc updates from E.5. +- **S3 (host-flake repo).** The D1 consent artifact and the two hard CI checks from D. +- **S4.** Nix2cf step 6 consumes `supervisor`. + +Every PR gets the cross-vendor review that AGENTS.md:40-44 requires. + +## H. Evidence table + +| Claim | Source read | +|---|---| +| Unit-writer prefix pattern; nesting description | `schema/goal-file.schema.json:239-247` | +| Writer enum includes `nixos` | `goal-file.schema.json:253-256` | +| `schema_version` const 3; "dispatch first" promise | `goal-file.schema.json:9-12` | +| Header has `additionalProperties: false` | `goal-file.schema.json:29-30` | +| `deliberately-unmanaged` forbids entries | `goal-file.schema.json:47-51` | +| `state_entries` kinds closed | `goal-file.schema.json:55-63` | +| Trust kinds closed; consent tier as an entry, not the header | `goal-file.schema.json:271-279,321` | +| Diff can change one header thing, `version_bump` | `schema/goal-diff.schema.json:25-26` | +| Header fixed at `schema_version` + `host`; platform rejected | `docs/architecture/goal-file-schema-reconciliation-2026-08-15.md:374-383` | +| Render rule; never-reported host | reconciliation `:309-318` | +| `schema_ceiling` is an integer with no maximum | `schema/report-row.schema.json:118-122` | +| Nesting rule; cfengine-writer rule | `bin/schema_lint.py:1431-1465` | +| Privileged derivation | `bin/schema_lint.py:1496-1506` | +| `EXPECTED_BROKEN = 63` | `bin/schema_lint.py:87` | +| Trust property test | `bin/schema_lint.py:1787-1846` | +| Projector reads `supervision` `service`/`interlock` only | `bin/projector.py:23-25,120-121` | +| tendcf-agent does not exist | `bin/projector.py:14-15` | +| Passthrough file uses `org.nixos.*`; projection `{"vars":{}}` | `deploy/linux/passthrough/goal-file.json:1`, `host_specific.json:1` | +| Canary shape | `deploy/linux/canary/goal-file.json:1` | +| `check.py` pairs | `deploy/linux/canary/check.py:31-38,55-66` | +| Numbering | `examples/broken/README.md:9,73-75,225,282-283,316-338` | +| Parentheticals are checked; `false`-schema precedent | `examples/broken/README.md:163-198` | +| Bundle guard and tombstone gating | `policy/tendcf_services.cf:172-185,200,225-232,314-318` | +| Inventory holds `trust_tier`; supervisor selects adapter | `docs/architecture/architecture-DEFINITIVE-v3.md:178-180,282-283` | +| Step 4/6/8 status | `docs/architecture/linux-targets.md:36,38,40,90` | +| Stage C result; open question | `docs/architecture/nixos-passthrough-plan.md:69-85,92-107` | +| Module options | `~/ops/site-public-djbclark/nixos/modules/cfengine-tendcf.nix:21-45` | +| Design rulings need astra + Fable | `AGENTS.md:45-49` | + +**Unverified:** + +- Every jsonschema error string [mem]. +- How the harness attributes a top-level `if/then`. +- 3-digit fixture directory handling. +- `` and get's real lock contents (narHash, the input URL type). +- That nix-darwin labels use `org.nixos.*`. This is from memory, and the design does not depend on it. +- That approval-record's version is a const of 3 (not read). +- `canary/goal-file-absent.json` contents. +- Whether any code implements distinct version-above-ceiling reporting. +- The nix2cf repository's v3 pins. +- All CFEngine semantics: taken from repo comments, not the book. + +DECIDED: + +- get enrolls at v4. +- New case numbering starts at 93. +- `nix-darwin` is excluded from the supervisor set. +- Site Model `launchd-writers` is left alone. + +DONE diff --git a/docs/architecture/schema-v4-reconciliation-2026-10-31.md b/docs/architecture/schema-v4-reconciliation-2026-10-31.md new file mode 100644 index 0000000..c9c5269 --- /dev/null +++ b/docs/architecture/schema-v4-reconciliation-2026-10-31.md @@ -0,0 +1,276 @@ +# Schema v4: `supervisor`, the NixOS passthrough host, and the Nix consent diff (2026-10-31) + +Status: **spec, rulings taken.** This document is the contract the v4 slices implement. +It supersedes its three inputs, which stay frozen as written: +[the Opus 5.5 draft](schema-v4-opinion-opus-draft.md), the +[Fable 5.1 review](schema-v4-opinion-fable.md) and the +[Gemini 4 Argon review](schema-v4-opinion-argon.md). Argon took the seat that the +standing rule gives to astra, because astra was short (Codex had 45% left, below the +60% `codex:frontier` estimate floor). The standing rule is in `AGENTS.md` on PR +https://github.com/frdminc/tendcf/pull/24 and was reviewed there. Where an input is +wrong, §8 says so; the inputs are not corrected in place. This document was itself +reviewed by Fable 5.1 (approve with changes); those changes are applied. + +## 1. Rulings (djbclark, 2026-10-31) + +| # | Ruling | +|---|---| +| D1 | For a Nix host, the consent diff shows the flake lock and the Nix-side diff as the **authority**. The evaluated-options JSON is a readable **summary** under it. | +| D2 | Add a host-level `supervisor` fact and bump `schema_version` to 4. On a `nixos` host that fact claims the whole unit namespace, with no per-unit prefix. The whole-namespace wildcard unit-writer was rejected. | +| D3 | On a Nix host, tendcf enforces nothing for now. It only reports and refuses, and `cf-execd` stays off. | +| Q1 | `supervisor` is **required** on every v4 goal file. Its value comes from the closed set `launchd`, `systemd`, `runit`, `nixos`. | +| Q2 | It is a fifth `device-trust` kind and a singleton. Reconciliation §7 G-3 closed the header, so this was not open. | +| Q3 | On a `nixos` host, `supervision` is `deliberately-unmanaged`. | +| Q4 | In v4 the Nix side is **display-only**: the goal file carries no lock digest. Binding it later takes its own bump. | +| Q5 | Distro family stays deferred, under the standing ruling "Defer until step 6/8 needs it". Neither step reads it. | +| Q6 | djbclark approves the D1 diff **on the host-flake PR**, behind a required CI job (§5). That job is the interim consent channel. It has no nonce, sequence number or signature, so it is not an approval record. | +| Q7 | `nixos` is dropped from the `writer` enum in **both** `goal-file.schema.json` and `launchd-writers.schema.json`. | +| R2 | On a `nixos` host, the other domains forbid only the `service`, `interlock` and `unit-writer` kinds, listed **per kind**. A future files or lockfile kind then needs no bump to be allowed. | +| R3 | "The unit flavour equals the supervisor" is enforced **in the schema**, with one `allOf` member per manager. The redundancy is named as residue (§7). | +| Q8 | No inventory exists yet. In nix2cf, `supervisor` is a host input next to `platform`; an inventory schema is its own later design. | +| Q9 | The prefix pattern cannot name Debian units (`ssh.service` under `dpkg`). That needs a separate ruling and design, and v4 leaves it alone. | +| Q10 | `deploy/linux/passthrough/goal-file.json` is a **fixture**, not get's goal file. It carries the example key and digests and is labelled as a fixture until get has a real key. | + +## 2. Schema deltas + +All three goal-family schemas (`goal-file`, `goal-diff`, `approval-record`) change +`schema_version` from `"const": 3` to `"const": 4`. + +### 2.1 `goal-file.schema.json` + +1. In `trust_domain.entries.properties`, add `"supervisor": {"$ref": "#/$defs/supervisor_map"}`. + Append `"supervisor"` to that object's `required` list. +2. Add a new `$defs.supervisor_map`. Its key is the singleton id `host`: + + ```json + "supervisor_map": { + "type": "object", + "propertyNames": {"const": "host"}, + "additionalProperties": { + "type": "object", + "properties": { + "state": {"const": "present"}, + "name": {"enum": ["launchd", "systemd", "runit", "nixos"]} + }, + "required": ["state", "name"], + "additionalProperties": false + }, + "minProperties": 1 + } + ``` + + The description must say four things: + - It is the host's unit-namespace authority, taken from inventory `supervisor`. + - On `nixos`, the NixOS configuration writes every unit, so no domain may carry a service, interlock or unit-writer entry. + - On the other three values, the per-prefix unit-writer entries still govern, and every present service's unit flavour must equal this value. + - The residue in §7.1. +3. Add a top-level `allOf` as a sibling of `required`, with four members. Each `if` sets + `"type": "object"` and `required` at every level, so a file without `supervisor` + never enters a `then`. + + - **nixos member.** + - `if`: `supervisor.host.name` is `"nixos"`. + - `then` on `domains`: + - `required: ["supervision"]` + - `properties`: + - `"device-trust": true` + - `"supervision": {"properties": {"coverage": {"const": "deliberately-unmanaged"}, "entries": false}}` + - `additionalProperties: {"properties": {"entries": {"properties": {"service": false, "interlock": false, "unit-writer": false}}}}` + + The explicit `entries: false` on `supervision` must not depend on how `supervision` is routed (Fable F1). The per-kind form is R2. + - **launchd, systemd, runit members, one each.** + - `if`: `supervisor.host.name` is the manager X. + - `then`: every service with `state: present`, in every domain except `device-trust`, has a `unit` with key X. + + Spelled out: + + ``` + domains.additionalProperties.properties.entries.properties.service.additionalProperties: + {"if": {"properties": {"state": {"const": "present"}}, "required": ["state"]}, + "then": {"properties": {"unit": {"required": [X]}}}} + ``` + + `unit` is already `oneOf` with exactly one flavour, so `required: [X]` is sufficient. +4. In `unit_writer_entry.writer`, remove `"nixos"`. Rewrite the description so it no longer names `nixos` as a writer. `nix-darwin` stays: it writes labels in the launchd namespace that Apple, Homebrew and CFEngine share. +5. Update every description that names the old passthrough mechanism (`unit_writer_map`, `unit_writer_entry`, `service_map`), and add the supervisor rule to the `trust_domain` description. The comments that count trust kinds are listed in §2.2(3). + +### 2.2 Other schemas and text + +1. `launchd-writers.schema.json`: remove `"nixos"` from the writer enum (Q7). That file is versioned on its own `contract_version` axis, so follow its existing bump rule. +2. In `goal-diff.schema.json`, add to the `version_bump` description: "valid alone only when the baseline rendered at `new` differs from the baseline in `schema_version` only. A bump that adds required content, as v4 does, is a re-adoption (baseline ceremony)." `derive_ceremony_class` already returns `baseline` for `version_bump`; this records the rule (Fable F5). Mirror the sentence in `goal-file-schema-reconciliation-2026-08-15.md` §9.4. +3. Counts that are stated as measured facts must be corrected: + - `bin/projector.py` (N-5 digest coverage): from "three of four" device-trust bodies to "three of five". Neither `trust-policy` nor `supervisor` carries a digest. + - `bin/schema_lint.py`: the comments that say "four trust bodies". + +## 3. Migration (one PR, S1) + +Enumerate the files from disk; never use the counts below as a substitute for enumeration. Both reviews corrected the draft's counts. On disk today, `examples/broken/` has 63 cases: +- 12 YAML Site Model cases (01–12), which are untouched; +- 40 that overlay `goal-file.json` (13–43, 61–65, 70–73); +- the rest, which overlay `goal-file-baseline.json`, `goal-diff.json` or approval records (49–57, 59, 60). + +1. **Happy corpus.** + - `examples/goal-file.json` and `examples/goal-file-baseline.json` get the entry `supervisor: {"host": {"name": "launchd", "state": "present"}}`. + - Regenerate `examples/goal-diff.json` and every approval record. Their `baseline_sha256` and `proposed_sha256` change. + - `host_specific.json` stays byte-identical. +2. **Negative overlays.** + - Every goal-family overlay gets `schema_version: 4`. + - Every overlaid goal file or baseline that has a `device-trust` domain also gets a supervisor entry whose name equals the flavour of its present services: `systemd` for cases 70–72, `launchd` for all the others. Case 18 has no `device-trust` domain (its `domains` is `{"supervision": …}`) and gets none. + - Case 13 now claims `5`. + - Diff and record overlays get recomputed hashes. +3. **`examples/broken-bytes/`.** + - Cases 44–48 are derived from the happy goal file (`examples/broken/README.md` says so). Regenerate them from the v4 file so that claim stays true. + - Bump `58-not-json.json` to `4`. +4. **`examples/broken-projection/75-goal-file-under-vars.json`:** re-embed the v4 happy file. +5. **Deploy goal files.** + - `deploy/linux/canary/goal-file.json` and `goal-file-absent.json` get supervisor `systemd`. + - `deploy/linux/passthrough/goal-file.json` gets: + - supervisor `nixos`; + - `supervision: {"coverage": "deliberately-unmanaged"}`, with the `org.nixos.*` unit-writer deleted; + - `schema_version: 4`. + - All three stay JCS. + - The passthrough projection must remain exactly `{"vars":{}}`. Prove it by running `deploy/linux/canary/check.py`, not by reasoning about it. + - Reword the comment at `check.py:34-36` and label the passthrough file a fixture there. Add `deploy/linux/passthrough/README.md`, three lines: it is a fixture, it uses the example key, and it projects to `{"vars":{}}` (Q10). +6. **Counts and prose.** + - `EXPECTED_BROKEN` (`bin/schema_lint.py`) and the "Sixty-three" in `bin/schema_lint.py` and `examples/broken/README.md`. + - The case ranges in `examples/broken/README.md` ("cases 13-73" becomes "13-73, 93-102"). + - The root `README.md` fixture total. + - The "claims version N" notes in the README and the lint. +7. **Policy strings.** The refusal message at `policy/tendcf_services.cf:318` says "unit writer nixos". It becomes "supervisor nixos". The same change lands in the comment at `:165-169`, in `tests/policy/run.sh` (the check-7 grep and its comments) and in `nixos-passthrough-plan.md` §1 and §2. This lands in S0 (§6), not S1. + +## 4. Negative cases (93–102) + +The series is shared across both fixture directories. `broken/` ends at 73 and +`broken-projection/` ends at 92, so new cases start at 93. Each row names its finding +class. Take the exact message and location from a real run of the lint, and correct the +README row to match. + +Cases 96–99 start from a valid nixos goal file: the happy device-trust entries with +supervisor `nixos`, `supervision: {"coverage": "deliberately-unmanaged"}`, and no other +domain. Each case then introduces exactly one defect, so deleting its rule turns it +green alone (F8). A backticked token in a README `schema (…)` cell is machine-checked +(`bin/schema_lint.py`), so write those cells as ``schema (`if/then`)`` with the rest as +bare prose. + +| # | Directory | Broken input | Class (location) | +|---|---|---|---| +| 93 | `93-unknown-supervisor` | supervisor `name: openrc` | schema `enum` (`domains/device-trust/entries/supervisor/host/name`) | +| 94 | `94-missing-supervisor` | device-trust without `supervisor` | schema `required` | +| 95 | `95-supervisor-not-singleton-id` | supervisor id `main` | schema `propertyNames` (`'host' was expected`) | +| 96 | `96-nixos-supervision-comprehensive` | nixos; `supervision` `comprehensive` with one service entry | schema `if/then` | +| 97 | `97-nixos-tombstone-elsewhere` | nixos; domain `apps` (`not-yet-migrated`) with a service tombstone | schema `if/then` (`domains/apps/entries`, per-kind form) | +| 98 | `98-nixos-unit-writer-elsewhere` | nixos; domain `packaging` with a unit-writer entry | schema `if/then` | +| 99 | `99-nixos-no-supervision-domain` | nixos; `supervision` absent | schema `if/then` | +| 100 | `100-nixos-writer-retired` | supervisor `launchd`; a unit-writer with `writer: nixos` | schema `enum` | +| 101 | `101-flavour-not-supervisor` | supervisor `systemd`; a present service with a `launchd` unit | schema `if/then` (R3) | +| 102 | `102-supervisor-hunk-ordinary` | overlays `goal-file-baseline.json` (supervisor `systemd`), `goal-diff.json` (a `device-trust/supervisor/host` hunk `systemd → launchd`, recomputed hashes) and both approval records; the accept record asserts `ordinary` and the reject mirror asserts `privileged`, as case 52 does | `family (ceremony)` | + +Positive checks: +1. The happy corpus, the canary files and the passthrough fixture are valid. +2. `check_trust_inexpressible` passes with five kinds. +3. The passthrough projection is `{"vars":{}}`. + +Discrimination (F8): delete each new schema rule on its own. Its case must go red alone. Run `bin/flag_coverage.py` where it applies. + +## 5. The D1 consent diff (S3, host-flake repositories) + +**Where it runs.** CI on the PR that changes get's flake (`forgejo.mit.edu/djbclark/get`), and on site-djbclark PRs that touch `nixos/` (`nixos-cfengine-build.yml`). It never runs on the device, which actuates nothing under D3. nix2cf does not produce it either: nix2cf renders the host's service-less goal file (§6.2), and the Nix side is outside its scope. + +**Authority** (D1, corrected per Fable F6 and Argon Finding 4): +1. the PR's own diff of the host-flake repository, `flake.lock` included. Each changed lock node is reduced to `(input, type, rev, narHash)`; +2. for each changed git input that the host imports, `git diff ..`, restricted to the imported paths (`nixos/` for site-djbclark). + +A lock diff alone shows nothing when the host's own `.nix` files change. + +**Summary.** Run this on base and head and diff the two JCS outputs: + +``` +nix eval --json .#nixosConfigurations.get.config.services.cfengine-tendcf --apply 'c: { enable = c.enable; execd.enable = c.execd.enable; version = c.package.version; package = builtins.unsafeDiscardStringContext "${c.package}"; }' +``` + +`version` keeps a no-op rebuild readable, since the store path alone changes on every rebuild (Fable F8). + +**Effect.** `nix store diff-closures base head`, which stage B already produces. + +**Hard checks.** Each one fails the job: +1. In the evaluated host config, `config.systemd.services` has no `cfengine-tendcf-execd`, and no unit in `config.systemd.services` or `config.systemd.timers` has an `ExecStart` or `script` naming `cf-execd` or `cf-agent`. This checks the effect, not the knob. +2. The **declared** option set under `options.services.cfengine-tendcf` equals `{enable, package, execd.enable}`: + `nix eval --json .#nixosConfigurations.get.options.services.cfengine-tendcf --apply 'o: { top = builtins.attrNames o; execd = builtins.attrNames o.execd; }'` + must print `{"execd":["enable"],"top":["enable","execd","package"]}`. A module that grows an option fails until the summary covers it. The draft's version of this check could never fire, because `--apply` projects exactly three keys (Fable F7). + +D3's "enforce nothing" also rests on no CMDB data being deployed to get. No CI check covers that. It holds today because nothing deploys tendcf data there; an installer run on get would itself be an action needing a ruling. + +**The question put to djbclark** on the PR, in plain words: "This changes get's NixOS configuration: . CFEngine stays installed and its scheduler stays off; tendcf will still change nothing on this host. Approve?" + +## 6. Slices + +All slices use worktrees, absolute paths and Sonnet 5.5 `high` or better. Every PR gets a cross-model frontier review before merge (`AGENTS.md`, standing rule). + +| Slice | Repo | Content | Depends on | +|---|---|---|---| +| S0 | tendcf | Tombstone report on a symlinked-units host (§6.1); check 7b; policy strings to "supervisor nixos" | none | +| S1 | tendcf | §2 and §3 and §4 in one PR; a bump cannot be split without an unsafe interim shape | spec merged | +| S3 | site-djbclark, get | §5 | spec merged | +| S4 | nix2cf | v1→v4 catch-up (§6.2) | S1 merged | + +### 6.1 S0 + +`policy/tendcf_services.cf` has a gap: +- `_refused` requires `_present` (`:199-200`). +- teardown requires `tendcf_systemd_host` (`:229-232`). + +As a result, a tombstone on a host where `/etc/systemd/system` is not a writable directory produces no output. Add a class: + +``` +"service_$(service_canon[$(service_ids)])_tombstone_elsewhere" + expression => "tendcf_systemd_units_elsewhere.!service_$(service_canon[$(service_ids)])_present"; +``` + +Place the class in `classes:` after `service_*_refused` and before the probes. It negates `service_*_present`, and CFEngine never undefines a class, so evaluating it before its operand is defined would leave a class that is never unset (file header, `:26-37`). Put the report next to the existing refusal report. + +Add check 7b to `tests/policy/run.sh`: run the `systemd-absent` fixture under the read-only-store shape (opt-in, `TENDCF_TEST_NIXOS_SHAPE=1`) and assert that: +- the report line appears; +- `systemctl` is never run; +- nothing is written. + +Negative checks: the report must not appear on a normal systemd host (check 6, after the tombstone run) or on a host not booted by systemd. + +### 6.2 S4 (nix2cf) + +The current state: +- nix2cf renders `schema_version: 1` (`nix2cf/compile.py:40`), and its tests fail against tendcf v3. +- It refuses every non-macOS host. +- It copies `device-trust` verbatim. +- `platform` is a CLI flag. + +The slice brings it to v4: +1. the v2 systemd `restart`/`wanted_by` fields and the argv alphabet, and the v3 writer vocabulary; +2. a `supervisor` host input next to `platform` (Q8), written into `device-trust.entries.supervisor`, with a refusal when a supplied `device-trust` document disagrees; +3. on a `nixos` host: + - a refusal of any selected service ("host is nixos-supervised; its units are the Nix configuration's"); + - `supervision: {"coverage": "deliberately-unmanaged"}` instead of the early return that omits `supervision`; +4. regenerated goldens; +5. a CI check that fails when `GOAL_FILE_SCHEMA_VERSION` differs from tendcf's schema const; +6. a refusal to render a `supervisor: nixos` host at any `schema_version` below 4. The only v3 spelling, `org.nixos.*`, described nothing. Non-nixos hosts may render at 3 by dropping the entry. + +The systemd renderer itself stays step 6 proper. + +## 7. Residue, recorded rather than fixed + +1. **Second spelling.** On a launchd, systemd or runit host, a service's unit flavour now always equals `supervisor`. R3 makes the equality checked. Collapsing `unit` so the flavour is implied would take a later bump. +2. **The version-above-ceiling report is not implemented.** The description of `goal-file.schema.json` `schema_version` promises it, but no code dispatches on the version. This predates v4. +3. **The N-5 projection guard covers three of five trust bodies.** `trust-policy` and `supervisor` carry no digest. +4. **No inventory schema exists** (Q8). **The dpkg prefix pattern defect is open** (Q9). +5. **`deliberately-unmanaged` turns off extra-entry detection on Nix hosts** (Q3). The NixOS configuration is the one writer there. +6. **The Q6 consent channel is not cryptographic.** It is a PR review, and it is not an approval record. +7. **get has never reported, and it enrolls at 4 on first adoption.** No v3 baseline exists anywhere (reconciliation 2026-08-15, D44 rule). +8. **R2 lets a nixos file declare other domains with any coverage and no entries.** Only the three kinds are forbidden, so a `comprehensive` empty domain there is a claim nothing checks. + +## 8. Where the inputs were wrong + +1. **Draft E.4 counted 63 goal-family overlays that each need a supervisor entry.** 63 is the total number of cases. The correct breakdown is in §3, from both reviews. The draft also missed `broken-bytes`, projection case 75 and the root README count (Argon Finding 7). +2. **Draft D used the lock diff alone as the authority.** That misses changes to the host's own files (§5). +3. **Draft D's hard check 2 could never fire** (§5). +4. **Draft F placed case 97 at `domains/apps/entries`.** The draft's blanket form actually reports `domains/apps`. The per-kind form adopted here (R2) reports `domains/apps/entries`. +5. **Draft Q5 said distro family was not ruled.** It was ruled: the standing "Defer until step 6/8 needs it" of 2026-10-31 (handoff `HANDOFF_tendcf-linux-targets-20261029_linux-milestone-through-nixos-stage-c_2026-10-31_f34c.md:95`). Argon's Q5 had this right. +6. **Argon Finding 1 (HIGH) retracts its own trace.** The `required` placements in the draft's `if` are correct. What survives is the advice to add `"type": "object"` at each level, which §2.1(3) adopts.