Skip to content

Machine-readable (JSON) output for cf-agent --simulate #1

Open

Description

@djbclark

Status: working document. This issue lives on a personal fork and is not an
upstream submission. It is where the feature request is drafted and revised as
investigation proceeds. When it is solid enough to be worth other people's time,
it goes upstream — see "Route upstream" below, which is not GitHub Issues.

What is wanted

A machine-readable rendering of what cf-agent --simulate computed: the set of
files that would change (with enough per-file detail to reconstruct the diff),
renames, and package operations, as JSON on stdout or to a named file.

Proposed surface, for discussion: --simulate-output=json, orthogonal to the
existing --simulate=<manifest|manifest-full|diff> mode selector.

Why

I am building a personal fleet configuration system (tendcf) in which a human
being reviews and approves a change to their own machines before it is applied.
The artifact the person approves has to be the same artifact the machine
enforces, and the approval has to bind to it. That requires the simulated change
set as data — something a program can hash, diff, attribute, and present — not
prose intended for a human reading a terminal.

Current simulate output is prose. simulate_mode.c renders with printf()
throughout: '%s' is a %s, '%s' no longer exists, Package '%s-%s [%s]' would be %s, and for --simulate=diff it shells out to diff -u and passes the
result through. Parsing that back into structure is possible and unwise —
it is a stat(1)-shaped text format with no stability contract, and any consumer
doing it is one wording change away from silently misreading a security-relevant
artifact.

This is a general want, not a tendcf-specific one. Anything that wants to gate,
audit, or report on a simulated run — CI, a review UI, a policy-diff bot,
Enterprise reporting — hits the same wall.

Current state, as of 17eb78e

CFEngine already computes and persists exactly the structured data needed. The
gap is entirely in rendering and lifetime.

The records exist. During a simulate run, four files are written inside the
changes chroot (libpromises/changes_chroot.h:28-31):

File Written by Format
/changed_files RecordFileChangedInChroot() length-prefixed strings
/renamed_files RecordFileRenamedInChroot() length-prefixed pairs
/kept_files RecordFileEvaluatedInChroot() length-prefixed strings
/pkgs_ops RecordPkgOperationInChroot() CSV: op, name, arch, version

The package record is already fully structured — a four-field CSV row with an
operation code from a defined enum (ChrootPkgOperationCode, one of
present/absent/install/remove in cf-agent/simulate_mode.h:38-43).

The renderer already runs before cleanup. This is the important structural
fact, and it corrects a reading I had earlier. cf-agent.c:392-428 dispatches on
EVAL_MODE and calls ManifestChangedFiles() / ManifestAllFiles() /
ManifestPkgOperations() / DiffChangedFiles() / DiffPkgOperations(). Only
afterwards, at cf-agent.c:435, does CallCleanupFunctions() run, which invokes
the DeleteChangesChroot handler registered at generic_agent.c:1634 and
deletes the tree at generic_agent.c:1793.

So the chroot's teardown is not a barrier to an in-process JSON renderer. It is
a barrier only to external consumers, who currently have no way to read the
records because the tree is gone by the time the process exits. A JSON renderer
slots in beside the existing prose renderers at cf-agent.c:392-428, with all
the same data in hand and the chroot still present.

The JSON library is already in-tree. libntech/libutils/json.h provides
JsonObjectCreate(), JsonArrayCreate(), JsonObjectAppendString(),
JsonWrite(), and the rest, and libntech is already a submodule dependency. This
means a single-repo change — no coordinated cfengine/core + NorthernTechHQ/libntech
pull request, which the multi-repo section of CONTRIBUTING.md would otherwise
require.

Taken together: this is a rendering change against data the agent already
computes, using a serializer already linked in, at a call site that already has
the data live. That is the cheap end of the difficulty range.

What this does not get, and should not claim

Worth stating plainly so the request is not oversold, and so I do not build on
something that was never there:

  • Files and packages only. Simulate mode covers files promises and packages
    promises. Service activation is not simulated. commands promises are not
    executed, which is the subject of the still-open CFE-3548 (--simulate does
    not execute commands promises tagged simulate_safe, 2021, open).
  • No runtime confinement. Simulate mode is a chroot for computing a
    would-be change set; it is not a capability sandbox, and nothing here makes it
    one.
  • --simulate=manifest is stat(1) prose. ManifestStatInfo() prints
    size/mode/uid/gid/atime/mtime/ctime. Structuring that is straightforward but is
    a distinct piece of work from structuring the change list.

Upstream context

Assembled from the git history, the public Jira, and GitHub. Recording it here so
the eventual upstream ask is informed rather than naive.

Simulate mode was built in a burst and then went quiet. Audit mode was renamed
to simulate mode in Nov 2020 (cfengine#4396, ENT-6398); acceptance tests followed
(cfengine#4397/cfengine#4407, ENT-5301/5302); manifest-full arrived Jan 2021 (cfengine#4469); package
support Jan-Mar 2021 (CFE-3554, CFE-3596, cfengine#4537). After mid-2021 there is
essentially nothing.

Three weeks ago upstream deleted the aspirational parts. cfengine#6242 (ENT-3787,
merged 2026-07-23) removed the 28_inform_testing/02_soft_fail_files acceptance
suite, whose stated reason was that it "encoded aspirational inform messages for
the audit / simulate feature... Those messages were never implemented." I read
this two ways at once and do not think either can be dismissed: it is evidence
that structured/consolidated simulate output was wanted, specified as tests, and
abandoned; and it is evidence that the current maintainer disposition is to trim
the feature's unbuilt ambitions rather than build them. A working patch changes
that calculus in a way a feature request does not.

Nobody has asked for this. A search of the public CFE Jira for simulate,
machine-readable, and JSON output turns up no ticket requesting structured
simulate output. Nearest neighbours: CFE-1929 "Console output greppability"
(open since 2015), CFE-3505 "Simulate mode should define hard class(es)"
(open, 2020), CFE-3548 above. The absence cuts both ways — no duplicate to
defer to, and no existing constituency to point at.

Most simulate work carries ENT- tickets, not CFE-. The feature's history is
largely Enterprise-driven, which is worth knowing when guessing who owns the
roadmap decision and what argument lands.

Route upstream

Not GitHub Issues: cfengine/core has issues disabled (has_issues: false).
Per CONTRIBUTING.md the path for a feature is a ticket in the public
CFE Jira project plus a message to the dev-cfengine mailing list before
contributing code, "in order to get discussion going." GitHub Discussions, the
help-cfengine list, #CFEngine:matrix.org, and #cfengine@Libera.Chat also
exist.

That ordering suits the goal here. The aim is not to carry a fork — it is to do
enough of the work that upstream will take and maintain it. So: ask first with a
concrete design and a demonstrated prototype, and let them state a preference on
surface and scope before the PR is polished.

Constraints to have understood before submitting anything:

  • Contributor statement (AUTHORS): joint copyright assignment to
    Northern.tech, with sublicensing rights, plus a patent grant. Not a bare DCO.
    This needs a deliberate decision, not a checkbox.
  • AI policy (CONTRIBUTING.md): AI-assisted contribution is explicitly
    allowed, with the human who reviews, corrects, and submits considered the
    author, held to the same quality bar. Worth honouring visibly.
  • Release timing: features land in the next minor (x.y.0) and must be
    merge-ready two months before the scheduled release date.
  • PR shape: one concern, usually one commit, CFE-1234: Title format, unit
    tests for C functions and acceptance tests for user-visible behaviour, plus a
    changelog entry.

Open questions

  1. Surface. --simulate-output=json orthogonal to --simulate=<mode>, or a
    new mode value such as --simulate=json? Orthogonal is cleaner and composes
    with manifest / manifest-full / diff, but adds a flag.
  2. Destination. stdout, mixed with logging? A --simulate-output-file=PATH?
    Anything on stdout has to survive Log() output interleaving to be parseable.
  3. Schema. What does a changed file entry contain — path, change kind, old
    and new stat, content hashes, the unified diff as a string, or a structured
    hunk list? The security-relevant question for my use case is whether content
    is committed to by digest.
  4. Diff representation. --simulate=diff currently shells out to diff -u
    (RunDiff(), simulate_mode.c:360). Keep that and embed the text, or
    restructure? Embedding is far cheaper and probably the right first cut.
  5. Stability contract. Is the JSON a supported interface with a version
    field, or explicitly unstable? Upstream will care about this more than about
    the shape.
  6. Retaining the chroot. Separately useful and much smaller: a flag to skip
    DeleteChangesChroot so external tooling can inspect the records and the
    simulated tree. Possibly a better first contribution than the full JSON
    renderer — smaller, obviously correct, and independently useful. Its downside
    is that it exports the on-disk record formats as a de facto interface.

Next steps

  • Prototype the JSON renderer on a branch of this fork, against 17eb78e
  • Build and run it locally to confirm the change set round-trips (a real
    cf-agent run, not source reasoning — an earlier note in tendcf reasoned
    from source alone and got the chroot lifetime wrong)
  • Decide the schema questions above with a working prototype in hand
  • Decide on the copyright-assignment question
  • Open the upstream conversation (Jira ticket + dev-cfengine), with the
    prototype as evidence, asking for their preference on surface and scope
  • Only then: a PR shaped to CONTRIBUTING.md

Activity

  1. djbclark commented on Aug 15, 2026

    @djbclark
    OwnerAuthor

    Decisions taken (2026-08-15)

    The Northern.tech contributor statement is accepted. The joint copyright
    assignment and patent grant in AUTHORS are fine for this work. That removes the
    one blocking non-technical question.

    Two separate PRs, not one. They are complementary rather than redundant, and
    the distinction needs to be made explicitly upstream because the obvious reviewer
    question is "if you have JSON, why do you need the chroot?":

    • The chroot is the would-be bytes. PrepareChangesChroot() mirrors the
      real tree and promises then act on the chrooted paths, so the retained chroot
      contains the files as they would be after the run. No JSON document can carry
      that without embedding whole file contents.
    • The JSON is the would-be change set. Which paths, which kinds of change,
      which package operations — structured, versioned, and cheap to consume.

    A consumer that wants to hash the resulting artifacts needs the first. A consumer
    that wants to gate, brief, or report on the change needs the second. Neither
    substitutes for the other.

    PR 1 — retain the changes chroot

    Change. The chroot is created in generic_agent.c:1629-1637, which registers
    DeleteChangesChroot as a cleanup handler; cf-agent.c:435 runs the handlers and
    generic_agent.c:1793 deletes the tree. Retention is: skip that registration when
    the flag is set, and report the path.

    Path discovery is already solved — generic_agent.c:1635 already logs the chroot
    path at WARNING level on every simulate run.

    Design points that need deciding before writing it:

    1. The chroot is $statedir/<pid>.changes (generic_agent.c:1785). PID-named,
      so retained chroots accumulate in the state dir across runs. Options: accept it
      and document; add a --simulate-chroot=PATH to name the destination; or move
      the retained tree to a stable path on exit. This is the main thing to get a
      maintainer opinion on, because it is a housekeeping burden they will own.
    2. Retention has a security cost that deletion currently prevents. The chroot is
      a permission-mirrored copy of real files, potentially including sensitive ones,
      left on disk after exit. Expect pushback, and design for it up front: opt-in
      only, never a default, loud log line on retention, path reported explicitly.
      Anticipating this objection in the ticket is likely to matter more than the
      implementation.
    3. Does retention export the record-file formats as a de facto interface? The
      four files in changes_chroot.h:28-31 become readable by anyone. Preferred
      framing: the retained artifact of record is the tree; the record files stay
      explicitly internal and unstable, with JSON (PR 2) as the supported interface
      for the change set. Worth stating in the PR so it is not inferred otherwise.

    Size. Small — the skip, a flag, a log line, an acceptance test, changelog.

    PR 2 — JSON rendering of the change set

    Change. New render functions beside the existing prose ones at the
    cf-agent.c:392-428 dispatch, emitting JsonElement trees via
    libntech/libutils/json.h. The prose renderers stay untouched and remain the
    default.

    Surface. --simulate-output=json, orthogonal to --simulate=<mode> — to be
    confirmed with maintainers before implementation, per the open questions above.

    Schema questions stay open until a prototype exists. Deciding them on paper is
    how the first pass at this got the chroot lifetime wrong; they get decided against
    something that runs.

    Size. Meaningfully larger — schema design, unit tests for the C functions,
    acceptance tests, and a stability-contract decision maintainers will care about
    more than the shape.

    Sequencing, and why

    PR 1 first, deliberately, as a probe. It is small enough to be judged on its
    merits in one pass, and putting it through the whole pipeline first — Jira ticket,
    CLA, PR shape, CI, review cadence — surfaces what upstream is actually like at low
    stakes. The response tells us what appetite exists for the larger change before we
    invest in it.

    The risk of that ordering, stated so it can be managed: landing chroot
    retention first invites "you can read the tree, JSON is unnecessary." The
    mitigation is to open a single Jira ticket describing both pieces and their
    division of labour, then reference it from both PRs — so the second is visibly
    half of a plan that was disclosed up front, not scope creep discovered later.

    Practical constraints found

    • Target release is 3.28.0. CHANGELOG.md shows master accumulating 3.28.0
      entries; the installed binary here is 3.27.1. Per CONTRIBUTING.md, features
      land in the next minor and must be merge-ready two months before its
      scheduled release date.
    • A Jira ticket is needed for the PR title format (CFE-1234: Title), and
      commit messages carry Changelog: and Ticket: trailers — trailers must be in
      the commit message, not the PR description.
    • Tests have a home to imitate: tests/acceptance/29_simulate_mode/ from the
      original ENT-5301/5302 work. Note it contains normalize_agent_output.cf.sub —
      the existing tests need output normalisation to be comparable at all, which is
      itself an argument for structured output.
    • Local build toolchain is incomplete. autoconf and automake are missing
      (and macOS /usr/bin/libtool is Apple's, not GNU libtool) so autogen.sh will
      not run yet. Dependencies are present: openssl@3, pcre2, lmdb, libyaml.

    House style is a first-class requirement

    The bar for this code is not "it works." It is code a CFEngine maintainer reads
    and recognises as belonging to the project — which is also the cheapest way to
    get a small contribution from an unknown contributor merged. CONTRIBUTING.md
    is unusually specific about this and repays reading in full before writing a
    line:

    • Formatting: Allman-4, 4 spaces and no tabs, fold at 78 columns, pointer
      star bound to the name (const char *const d), braces on their own lines
      (except do-while and struct typedef tails), single spaces around operators,
      one space after a cast. A .clang-format is checked in, so the mechanical
      part is settled — use it.
    • Sections that constrain design, not just layout: Readability /
      Maintainability
      , Safety / Correctness, Log levels, and Logging
      Guidelines
      . Log level choice matters here specifically — retention (PR 1)
      needs a deliberate level, and the existing chroot notice at
      generic_agent.c:1635 is LOG_LEVEL_WARNING, which is the precedent to weigh.
    • Per-directory guidance exists for libutils, libpromises, and
      cf-agent — all three are in scope for this work.
    • Commit hygiene: one concern per PR, usually one commit, CFE-1234: Title,
      Changelog: and Ticket: trailers in the commit message.

    Match the surrounding code over any general preference: the existing
    simulate_mode.c and changes_chroot.c are the local idiom, and the new code
    should be indistinguishable from them in shape.

    Next steps

    • Install the missing build toolchain and get cf-agent building from source
    • Confirm the built binary reproduces the documented simulate behaviour
      before changing anything
    • Prototype PR 1 on a branch
    • Register on the CFEngine Jira and open one ticket covering both pieces
    • Post to dev-cfengine referencing the ticket, asking for preferences on the
      retention path question and the JSON surface
    • PR 1, shaped to CONTRIBUTING.md
    • Prototype PR 2 once the schema questions have maintainer input
  2. djbclark commented on Aug 15, 2026

    @djbclark
    OwnerAuthor

    Split into two tickets

    The work is now tracked per-PR, each ticket self-contained and each opening with a plain-language section explaining what the feature is good for:

    This issue stays as the umbrella: the investigation trail, the upstream context, and the shared constraints (routing, contributor statement, release timing, house style) that apply to both.

    Upstream, both still go under a single Jira ticket describing the pair — so the second PR reads as half of a plan disclosed up front rather than scope creep discovered later.

  3. djbclark commented on Aug 15, 2026

    @djbclark
    OwnerAuthor

    Built from source and ran it — 2026-08-15

    Toolchain installed (autoconf, automake; GNU libtool was already present as glibtool), autogen.sh + configure + make clean, make install into a local prefix. Everything below is observed from a running binary, not read from source.

    ./configure --prefix=$HOME/opt/cfengine-dev \
      --with-openssl=/opt/homebrew/opt/openssl@3 \
      --with-pcre2=/opt/homebrew/opt/pcre2 \
      --with-lmdb=/opt/homebrew/opt/lmdb \
      --with-libyaml=/opt/homebrew/opt/libyaml \
      --enable-maintainer-mode
    

    Correction to this issue's release-timing note. It said the target release is 3.28.0, inferred from the top section of CHANGELOG.md. Wrong: the built binary reports CFEngine Core 3.29.0a.17eb78e6d and CFVERSION says 3.29.0a. 3.28.0 is the section being closed; master is 3.29.0 alpha. The target for new features is 3.29.0, and the two-month-before-release rule applies to that date.

    Simulate mode verified working, against a policy touching two files (one modified, one created):

    • Real files were untouched — the modified file kept its original contents and the created file never appeared outside the chroot. Isolation is genuine.
    • The chroot path is logged at LOG_LEVEL_WARNING on every run, as expected.
    • The chroot is gone after exit — nothing matching *.changes remains in the state dir. PR 1: Retain the changes chroot after a --simulate run #2's premise confirmed by observation.

    Running it locally needs two things worth writing down, both of which cost time to discover:

    1. cf-promises must be in $WORKDIR/bin or the agent falls back to failsafe policy. Copying it from the build tree gets you a libtool wrapper script, not the binary — use make install and the installed tree, or .libs/cf-promises.
    2. CFENGINE_TEST_OVERRIDE_WORKDIR gives a rootless workdir, but chown failures on the mirrored tree are normal and noisy on macOS; they do not stop the run.
  4. added a commit that references this issue on Aug 15, 2026
  5. djbclark commented on Oct 9, 2026

    @djbclark
    OwnerAuthor

    First upstream submission of the simulate-mode batch: cfengine#6389 (swap fix, fork #39). One PR at a time; the rest wait for a go per PR. Upstream texts no longer carry the ELI5 blocks.

  6. djbclark commented on Oct 15, 2026

    @djbclark
    OwnerAuthor

    B (fork PR #44) is upstream as cfengine#6412 (2026-10-15), after cfengine#6389 merged 2026-10-13 (approved by larsewi). Next in the queue: C #45, D #40, E #41, F libntech#9; each waits for upstream to act on cfengine#6412 and your go.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions