Skip to content

Added --simulate-json option to write the simulated change set as JSON - #92

Draft
djbclark wants to merge 100 commits into
masterfrom
simulate-json-upstream
Draft

djbclark wants to merge 100 commits into
masterfrom
simulate-json-upstream

Conversation

@djbclark

@djbclark djbclark commented Nov 14, 2026 •

Copy link
Copy Markdown
Owner

Review vehicle for the upstream-ready --simulate-json PR (not sent upstream). One commit on upstream master a8ac0dc7e; replaces the closed cfengine#6294 in smaller form. Tracking: #3.

With --simulate-json=FILE, a --simulate run also writes the changed files (path, change, type, uid, gid, permissions, size, sha256, symlink target) and renames to FILE as one JSON document.

Trimmed from d5cc2c4 / 94d325e: failsafe_fallback and aborted fields, the generic_agent.c/h config field (a static in cf-agent.c instead), the MinGW mask (now upstream), the UTF-8 workaround (libntech still escapes per byte; known limitation), previous_type. Non-test C 315 → 259 lines; tests 401 → 280.

Security review (adversary): round 1 on 2ffde2299: BLOCK. Writing in place through FileWriteOver() let a local user hardlink a root-owned file at a /tmp output path. Fixed by writing an O_EXCL temp file and rename()ing it into place, with a hardlink regression test. Round 2 on 799bd0d8c: "Verdict: APPROVE, B1 fixed, no new findings, 3 notes." The notes: an operator-chosen, world-writable directory without the sticky bit can still be raced; no fsync; a stale file is left behind on an early exit.

Linux gate: head 799bd0d8c: unit 79/79, simulate_mode_test 16/16, 29_simulate_mode/ acceptance 8/8 (includes simulate_json.cf), no warnings in touched files.

Prior art / docs checked: gh pr list -R cfengine/core --search (simulate json, simulate in:title, machine-readable simulate) found only cfengine#6294; CFE Jira JQL text ~ simulate AND text ~ json found only CFE-4716 (Rejected); firecrawl web search for cf-agent simulate JSON output found none. Upstream libntech master 3c21479 still writes \u00XX per byte, and libntech#293 is closed.

🤖 Generated with Claude Code

Dhanu-dynamic and others added 30 commits October 18, 2026 12:37
FnCallFormat() hands each format piece straight to snprintf() or
BufferAppendF() with exactly one argument. The modifier denylist
("hLqjzt") lets through '*', which takes a width or precision from the
argument list, and '$', which selects a positional argument, so
format("%*d", 7) prints whatever is in the next vararg slot and
format("%2$s", "a") or format("%*s", "a") dereference a slot that was
never passed as a char *. 'l' on %s/%S makes printf read the char *
argument as a wchar_t *, 4 bytes per character past the end of the
string.

The format argument is expanded before the function runs, so a
variable interpolated into it (file contents, module output, data
container values) can carry such a specifier.

Changelog: Title
Signed-off-by: Dhanraj E <dhanraj@labs.digiscrypt.com>
GetServerLiteral() zeroed and formatted CF_BUFSIZE bytes into the
sendbuffer it was handed, but BusyWithNewProtocol() allocates that
buffer as CF_BUFSIZE - CF_INBAND_OFFSET bytes (the most a single
transaction can carry). Any VAR request admitted by the literals ACL
therefore wrote 8 bytes past the end of the handler's stack buffer.

Pass the buffer size into GetServerLiteral(), as DoExec2() already
does, and bound the writes to it. Both callers pass sizeof(sendbuffer).

Changelog: Title
Signed-off-by: Dhanraj E <dhanraj@labs.digiscrypt.com>
The "xattr" branch of filestat() reads each extended attribute value
into an uninitialized stack buffer and then uses it as a C string:

    char data[CF_BUFSIZE];
    int datasize = lgetxattr(path, current, data, sizeof(data));
    ...
    if (!StringIsPrintable(data))
    ...
    BufferPrintf(printattr, "%s=%s", current, data);

lgetxattr() returns raw bytes and never adds a terminating NUL. An
unprivileged user can set user.* attributes on files they own, so when
a policy run as root calls filestat() on such a file, a printable value
of exactly 4096 bytes makes StringIsPrintable() read past the end of
data. Shorter values are followed by whatever stale stack bytes come
before the next NUL, which either makes a valid attribute get skipped
as non-printable or copies stack contents into the returned string.

Read at most sizeof(data) - 1 bytes and terminate the value at
datasize. CopyFileExtendedAttributesDisk() in files_copy.c, the other
lgetxattr() caller, handles values by length and is not affected.

Changelog: Title
Signed-off-by: Dhanraj E <dhanraj@labs.digiscrypt.com>
Ticket: CFE-3898
Changelog: None
Signed-off-by: Nick Anderson <nick@cmdln.org>
Added the reactor tests to jenkins, to be able to run them in jenkins. This
means we can test cf-reactor on all the different platforms.

Signed-off-by: Victor Moene <victor.moene@northern.tech>
Made the logs prettier and added a log file where the xtrace is
printed

Signed-off-by: Victor Moene <victor.moene@northern.tech>
The guest environment cannot contain "-". On hosts containing "-", the
promise fails. The host name is now canonofied.

Signed-off-by: Victor Moene <victor.moene@northern.tech>
X509_get_subject_name() now returns a const name. Build a new
X509_NAME and set it as both subject and issuer instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Tests that print Pass message but have logs errors will fail with `FAIL (Error printed)` message.
Tests which are expected to log errors should use `.error.cf` or `.error.pass.cf` suffix. see: `tests/acceptance/README`

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
These tests read files created by their own init bundle, but pre-evaluation calls the read functions before the files exist, logging "Could not examine file" errors which now fail regular tests.

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
Errors printed by the command were reported on lines starting with "error:", so testall mistook them for errors of the test itself.

Ticket: CFE-3041

Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
These tests read files and directories created by their own policy, but pre-evaluation calls the functions before they exist, logging errors which now fail regular tests.

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
The agent loads def.json from the directory of the input policy file, so tests in 00_basics/ tried to read this directory as their augments file and logged "Could not open JSON file" errors.

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
The test used the 'immediate' body from plucked.sub.cf before it was
loaded during pre-evaluation, so the promise is now skipped until the
test framework is loaded.

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
The test referred to a promise module which never existed and did not run its init and cleanup bundles.

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
The check bundle counted lines in files which do not exist until the test bundle has run, but the agent evaluates the functions of all promises before any bundle runs.

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
wrong-bundle-type.cf printed the sub-agent output in its DEBUG report, so the sub-agent errors were mistaken for errors of the test, and ifelse.cf used invalid class expressions as false conditions.

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
These tests intentionally exercise error paths, the reason is documented in the header of each test.

Ticket: CFE-3041
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
Ticket: CFE-3041
Signed-off-by: Nick Anderson <nick@cmdln.org>
Ticket: CFE-3041
Signed-off-by: Nick Anderson <nick@cmdln.org>
The errorpass branch only checked for logged errors and the Pass
message, so a test marked test_soft_fail or test_skip_unsupported was
reported as FAIL (e.g. process_select_fncalls.error.pass.cf on Windows,
which soft fails due to ENT-10217 and logs no errors there). Errorpass
tests now go through the regular outcome handling and only differ in
requiring at least one logged error on Pass.
Ticket: CFE-3041
Signed-off-by: Nick Anderson <nick@cmdln.org>
Signed-off-by: Dhanraj E <dhanraj@labs.digiscrypt.com>
Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
…ptions() public

Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
larsewi and others added 21 commits November 5, 2026 15:56
…ror-messages

cf-reactor: fix watchers on windows
ENT-14717: Made the backport action a reusable workflow
…changelog-master

docs: update changelog for master
cf-agent reconstructs the absolute target of a relative symbolic link by
appending the readlink() result onto the link's directory name in a
CF_BUFSIZE stack buffer. KillGhostLink() in files_links.c did this with an
unbounded strcat(linkpath, linkbuf), and LinkCopy() in verify_files_utils.c
with strncat(vbuff, linkbuf, CF_BUFSIZE - 1), whose third argument is the
maximum number of bytes to take from the source, not the space left in the
destination. readlink() returns up to CF_BUFSIZE - 1 bytes, so a symlink
whose relative target is near CF_BUFSIZE, left by an unprivileged user in a
directory that a root files promise walks (depth_search, perms, link copy),
overflows the stack of the agent. KillGhostLink() runs on every symlink met
during a recursive descent, before the dead-link stat().

Bound both appends with strlcat(..., sizeof(buf)), the same way ExpandLinks()
in files_links.c already does. KillGhostLink() bails out with an "Internal
limit reached" message and LinkCopy() fails the promise when the target does
not fit, rather than truncating it into a link pointing somewhere else.
Normal links are unchanged.

The regression test is skipped when symlink() rejects the long target with
ENAMETOOLONG, as macOS does.

Changelog: Title
Signed-off-by: Dhanraj E <dhanraj@labs.digiscrypt.com>
…th-overflow

fix stack overflow building absolute path from relative symlink target
Ticket: ENT-14717
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
Ticket: CFE-3478
Signed-off-by: Victor Moene <victor.moene@northern.tech>
ENT-14717: Now removes the cherry-pick? label after successful backports
ENT-14927: Fixed cf-serverd crash when policy reload races with a new connection
…talls

DiffPkgOperations() compared the versions of two recorded installations
of the same package without checking that both were set, which
PkgVersionIsGreater() asserts. The diff report now keeps the record the
way the manifest report does: a versioned record over an unversioned
one, otherwise the higher version, otherwise the first one recorded.

Ticket: none
Changelog: Title
Signed-off-by: Daniel JB Clark <djbclark@gmail.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CFE-3478: Added isvalidip: function for validating IP addresses
…sioned-install-assert

Fixed simulate diff mode aborting on repeated unversioned package installs
Branch protection patterns like "*" do not match "/", so the old backport-<PR>-<branch> name was blocked in repos protecting all branches.

Changelog: None
Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
Changed backport branch name to backport/<PR>-<branch>
…check

Checked RAND_bytes() result when creating bootstrap_id.dat
Signed-off-by: Ole Herman Schumacher Elgesem <ole@northern.tech>
With --simulate-json=FILE, cf-agent also writes the files and renames of
a --simulate run to FILE as one JSON document.

Ticket: none
Changelog: Title
Signed-off-by: Daniel JB Clark <djbclark@gmail.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Nov 14, 2026

Copy link
Copy Markdown

Automatic reviews are disabled for this repository. Configure in Code Review Settings.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-14T14:50:35.826007Z 799bd0d PR opened
🔒 Security Review ✅ Completed 2026-11-14T14:53:54.321504Z 799bd0d PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 799bd0d8c6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +473 to +475
PackageWatcherState *state = xcalloc(1, sizeof(PackageWatcherState));
state->name = xstrdup(name);
state->listing = listing;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Refresh package watchers when their module configuration changes

When a policy reload changes a package module's options, interpreter, or path—or changes the default module—without changing the events promise itself, the watcher continues using the old module configuration. WatcherRegister() in cf-reactor/watcher.c identifies watchers solely by the expanded promise hash, takes over the previous state, and destroys this newly constructed state. Consequently, the old PackageListing and its copied module remain active, potentially detecting package events against the wrong inventory until cf-reactor restarts. Include the resolved module configuration in watcher identity, or invalidate the previous state when that configuration changes.

Useful? React with 👍 / 👎.

@djbclark

Copy link
Copy Markdown
Owner Author

Superseded by the 4-PR stack #93 → #94 → #95 → #96 (same code split into smaller steps; the tip tree of #96 equals this PR's 799bd0d8c). Kept open as a draft for reference.

@djbclark
djbclark marked this pull request as draft November 14, 2026 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants