Skip to content

Added --simulate-json option to write the changed files of a simulate run as JSON - #93

Open
djbclark wants to merge 100 commits into
masterfrom
simjson-1-files
Open

djbclark wants to merge 100 commits into
masterfrom
simjson-1-files

Conversation

@djbclark

@djbclark djbclark commented Nov 14, 2026 •

Copy link
Copy Markdown
Owner

PR 1/4 of the --simulate-json stack (fork review vehicle; nothing sent upstream). Supersedes #92 in smaller steps. Base: master.

Rebuilt 2026-11-14 (comment-only): PR 1 no longer mentions digests (PR 3 adds that comment), PR 1's .sub notes renames are not in the output yet (PR 2 drops the note). Tip tree still equals #92's 799bd0d8c. Adversary verdicts below are for the pre-rebuild heads with identical code (df7373eac); not re-run, no code changed.

--simulate-json=FILE: a --simulate run also writes the changed files (path, created/modified/deleted, type) to FILE as one JSON document, via an O_EXCL temp file + rename(). Renames are not covered yet (PR 2).

Size: +182 non-test C, +234 tests.

Linux gate (Ubuntu 24.04 aarch64, root, --enable-debug), head bfc2e1d06: unit All 79 tests passed; simulate_mode_test 15/15; 29_simulate_mode/ acceptance 8/8 (incl. simulate_json.cf); no warnings in touched files.
Negative control: master + only the acceptance files: simulate_json.cf FAIL, 7/8.
Adversary verdict (head df7373eac): "APPROVE df7373e: no blockers or should-fix, three notes." Notes: a race on rename() if the operator picks a non-sticky directory that another user can write; the cf-agent.c comment mentions digests, which arrive in PR 3; the .sub renames a file that the expected output does not list yet (both fixed by the rebuild).

Prior art / docs checked: same as #92 (no upstream machine-readable simulate output; cfengine#6294 closed, CFE-4716 rejected).

🤖 Generated with Claude Code

Explain it like I'm 5

You ask a careful friend to check what some changes would do to your computer, without actually making them. The friend comes back with a note written for a person: "I'd change /etc/motd and create /etc/app.conf." That works for you, but a program reading the note has to guess at the wording, and the wording can change.

This PR asks the friend to also fill in a simple form: one row per file, with three boxes. Which file, what would happen to it (created, modified or deleted), and what kind of thing it is (a regular file, a directory, a link). A program can read that form reliably, for example to make CI fail when a change touches a file it shouldn't.

This first part only lists which files would change. The next three PRs add renames; owner, permissions, size and checksum; and where links point. Until then, a file that would only be renamed doesn't show up.

Dhanu-dynamic and others added 30 commits October 18, 2026 12:37
FnCallFormat() hands each format piece straight to snprintf() or
BufferAppendF() with exactly one argument. The modifier denylist
("hLqjzt") lets through '*', which takes a width or precision from the
argument list, and '$', which selects a positional argument, so
format("%*d", 7) prints whatever is in the next vararg slot and
format("%2$s", "a") or format("%*s", "a") dereference a slot that was
never passed as a char *. 'l' on %s/%S makes printf read the char *
argument as a wchar_t *, 4 bytes per character past the end of the
string.

The format argument is expanded before the function runs, so a
variable interpolated into it (file contents, module output, data
container values) can carry such a specifier.

Changelog: Title
Signed-off-by: Dhanraj E <dhanraj@labs.digiscrypt.com>
GetServerLiteral() zeroed and formatted CF_BUFSIZE bytes into the
sendbuffer it was handed, but BusyWithNewProtocol() allocates that
buffer as CF_BUFSIZE - CF_INBAND_OFFSET bytes (the most a single
transaction can carry). Any VAR request admitted by the literals ACL
therefore wrote 8 bytes past the end of the handler's stack buffer.

Pass the buffer size into GetServerLiteral(), as DoExec2() already
does, and bound the writes to it. Both callers pass sizeof(sendbuffer).

Changelog: Title
Signed-off-by: Dhanraj E <dhanraj@labs.digiscrypt.com>
The "xattr" branch of filestat() reads each extended attribute value
into an uninitialized stack buffer and then uses it as a C string:

    char data[CF_BUFSIZE];
    int datasize = lgetxattr(path, current, data, sizeof(data));
    ...
    if (!StringIsPrintable(data))
    ...
    BufferPrintf(printattr, "%s=%s", current, data);

lgetxattr() returns raw bytes and never adds a terminating NUL. An
unprivileged user can set user.* attributes on files they own, so when
a policy run as root calls filestat() on such a file, a printable value
of exactly 4096 bytes makes StringIsPrintable() read past the end of
data. Shorter values are followed by whatever stale stack bytes come
before the next NUL, which either makes a valid attribute get skipped
as non-printable or copies stack contents into the returned string.

Read at most sizeof(data) - 1 bytes and terminate the value at
datasize. CopyFileExtendedAttributesDisk() in files_copy.c, the other
lgetxattr() caller, handles values by length and is not affected.

Changelog: Title
Signed-off-by: Dhanraj E <dhanraj@labs.digiscrypt.com>
Ticket: CFE-3898
Changelog: None
Signed-off-by: Nick Anderson <nick@cmdln.org>
Added the reactor tests to jenkins, to be able to run them in jenkins. This
means we can test cf-reactor on all the different platforms.

Signed-off-by: Victor Moene <victor.moene@northern.tech>
Made the logs prettier and added a log file where the xtrace is
printed

Signed-off-by: Victor Moene <victor.moene@northern.tech>
The guest environment cannot contain "-". On hosts containing "-", the
promise fails. The host name is now canonofied.

Signed-off-by: Victor Moene <victor.moene@northern.tech>
X509_get_subject_name() now returns a const name. Build a new
X509_NAME and set it as both subject and issuer instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Tests that print Pass message but have logs errors will fail with `FAIL (Error printed)` message.
Tests which are expected to log errors should use `.error.cf` or `.error.pass.cf` suffix. see: `tests/acceptance/README`

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
These tests read files created by their own init bundle, but pre-evaluation calls the read functions before the files exist, logging "Could not examine file" errors which now fail regular tests.

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
Errors printed by the command were reported on lines starting with "error:", so testall mistook them for errors of the test itself.

Ticket: CFE-3041

Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
These tests read files and directories created by their own policy, but pre-evaluation calls the functions before they exist, logging errors which now fail regular tests.

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
The agent loads def.json from the directory of the input policy file, so tests in 00_basics/ tried to read this directory as their augments file and logged "Could not open JSON file" errors.

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
The test used the 'immediate' body from plucked.sub.cf before it was
loaded during pre-evaluation, so the promise is now skipped until the
test framework is loaded.

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
The test referred to a promise module which never existed and did not run its init and cleanup bundles.

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
The check bundle counted lines in files which do not exist until the test bundle has run, but the agent evaluates the functions of all promises before any bundle runs.

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
wrong-bundle-type.cf printed the sub-agent output in its DEBUG report, so the sub-agent errors were mistaken for errors of the test, and ifelse.cf used invalid class expressions as false conditions.

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
These tests intentionally exercise error paths, the reason is documented in the header of each test.

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
Ticket: CFE-3041
Signed-off-by: Nick Anderson <nick@cmdln.org>
Ticket: CFE-3041
Signed-off-by: Nick Anderson <nick@cmdln.org>
The errorpass branch only checked for logged errors and the Pass
message, so a test marked test_soft_fail or test_skip_unsupported was
reported as FAIL (e.g. process_select_fncalls.error.pass.cf on Windows,
which soft fails due to ENT-10217 and logs no errors there). Errorpass
tests now go through the regular outcome handling and only differ in
requiring at least one logged error on Pass.
Ticket: CFE-3041
Signed-off-by: Nick Anderson <nick@cmdln.org>
Signed-off-by: Dhanraj E <dhanraj@labs.digiscrypt.com>
Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
…ptions() public

Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
larsewi and others added 24 commits November 5, 2026 11:57
…assed-args

Made format() reject specifiers that read unpassed printf arguments
Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Co-authored-by: Lars Erik Wik <53906608+larsewi@users.noreply.github.com>
Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
…ror-messages

cf-reactor: fix watchers on windows
ENT-14717: Made the backport action a reusable workflow
…changelog-master

docs: update changelog for master
cf-agent reconstructs the absolute target of a relative symbolic link by
appending the readlink() result onto the link's directory name in a
CF_BUFSIZE stack buffer. KillGhostLink() in files_links.c did this with an
unbounded strcat(linkpath, linkbuf), and LinkCopy() in verify_files_utils.c
with strncat(vbuff, linkbuf, CF_BUFSIZE - 1), whose third argument is the
maximum number of bytes to take from the source, not the space left in the
destination. readlink() returns up to CF_BUFSIZE - 1 bytes, so a symlink
whose relative target is near CF_BUFSIZE, left by an unprivileged user in a
directory that a root files promise walks (depth_search, perms, link copy),
overflows the stack of the agent. KillGhostLink() runs on every symlink met
during a recursive descent, before the dead-link stat().

Bound both appends with strlcat(..., sizeof(buf)), the same way ExpandLinks()
in files_links.c already does. KillGhostLink() bails out with an "Internal
limit reached" message and LinkCopy() fails the promise when the target does
not fit, rather than truncating it into a link pointing somewhere else.
Normal links are unchanged.

The regression test is skipped when symlink() rejects the long target with
ENAMETOOLONG, as macOS does.

Changelog: Title
Signed-off-by: Dhanraj E <dhanraj@labs.digiscrypt.com>
…th-overflow

fix stack overflow building absolute path from relative symlink target
Ticket: ENT-14717
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
Ticket: CFE-3478
Signed-off-by: Victor Moene <victor.moene@northern.tech>
ENT-14717: Now removes the cherry-pick? label after successful backports
ENT-14927: Fixed cf-serverd crash when policy reload races with a new connection
…talls

DiffPkgOperations() compared the versions of two recorded installations
of the same package without checking that both were set, which
PkgVersionIsGreater() asserts. The diff report now keeps the record the
way the manifest report does: a versioned record over an unversioned
one, otherwise the higher version, otherwise the first one recorded.

Ticket: none
Changelog: Title
Signed-off-by: Daniel JB Clark <djbclark@gmail.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CFE-3478: Added isvalidip: function for validating IP addresses
…sioned-install-assert

Fixed simulate diff mode aborting on repeated unversioned package installs
Branch protection patterns like "*" do not match "/", so the old backport-<PR>-<branch> name was blocked in repos protecting all branches.

Changelog: None
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
Changed backport branch name to backport/<PR>-<branch>
…check

Checked RAND_bytes() result when creating bootstrap_id.dat
Signed-off-by: Ole Herman Schumacher Elgesem <ole@northern.tech>
@greptile-apps

greptile-apps Bot commented Nov 14, 2026

Copy link
Copy Markdown

Automatic reviews are disabled for this repository. Configure in Code Review Settings.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-14T16:02:07.127947Z df7373e PR opened
🔒 Security Review ✅ Completed 2026-11-14T16:08:54.848269Z df7373e PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: df7373eacb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

}
}

return PackageWatcherStateNewFromModule(name, module);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include resolved package-module settings in watcher identity

When a policy reload changes a package module's path, interpreter, or options without changing the events promise, the watcher continues using the old module configuration. WatcherPromiseHash() hashes only the expanded events promise, so WatcherRegister() reuses the previous state and destroys the newly constructed state, including its newly resolved listing. Changing the default package module in common control has the same problem. Include the resolved module configuration in the watcher identity, or check its compatibility before retaining the previous state, so configuration updates take effect without restarting cf-reactor.

Useful? React with 👍 / 👎.

… run as JSON

With --simulate-json=FILE, cf-agent also writes the changed files of a
--simulate run to FILE as one JSON document; renames are not covered yet.

Ticket: none
Changelog: Title
Signed-off-by: Daniel JB Clark <djbclark@gmail.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants