Repository navigation
Conversation
Signed-off-by: Victor Moene <victor.moene@northern.tech>
This was referenced Sep 30, 2026
djbclark
force-pushed
the
simulate-json-files
branch
from
September 30, 2026 18:43
8182ea9 to
d5cc2c4
Compare
GetProcessStat() asserted on unexpected open() errors, e.g. when the process exited in the meantime. Release builds would loop forever instead. Now logs the error and returns false. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Ticket: CFE-4285 Changelog: Title Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
CFE-4285: Fixed cf-agent assertion failure when reading /proc/<pid>/stat
…ions.c Linked TODOs to new tickets to fix old bugs/TODOs that were already in cf-agent before moving the code. Signed-off-by: Victor Moene <victor.moene@northern.tech>
Better concurrency for cf-reactor static state - No need for map anymore, since watcher keeps track of the promise - No need to keep key as variable anymore, since we do not use a map - No need to run bundle inside watcher.c - No risk of running a bundle from a wrong key between policy reads Signed-off-by: Victor Moene <victor.moene@northern.tech>
Promises are skipped to prevent running them several times. However, on event, we want to run them every single time: - we clear the promise lock cache - we set the default if_elapsed time for bundles run from an events promise to be 0, so it doesn't skip the promises. Fixed also connection cache and custom promise prologue and epilogue. Clear function cache before "then" bundle run Signed-off-by: Victor Moene <victor.moene@northern.tech>
Signed-off-by: Victor Moene <victor.moene@northern.tech>
- Moved mod_methods.c's MethodsParseTreeCheck implementation to PromiseCheckBundleCallArity in policy.c, and made it more generic - Now, MethodsParseTreeCheck and EventsParseTreeCheck both call this new function when calling bundles Signed-off-by: Victor Moene <victor.moene@northern.tech>
ENT-14194: Run policy on event
With --simulate-json=FILE, cf-agent also writes the files and renames of a --simulate run to FILE as one JSON document, together with whether the run fell back to the failsafe policy or was aborted. Under manifest-full the JSON lists only the changed files, not the kept ones. Ticket: none Changelog: Title Signed-off-by: Daniel JB Clark <djbclark@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
djbclark
force-pushed
the
simulate-json-files
branch
from
October 9, 2026 03:08
d5cc2c4 to
f8342de
Compare
This was referenced Oct 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Fork-internal PR for CI; not an upstream submission. PR 1 of the plan on #3:
--simulate-jsonfor files and renames. Packages follow in PR 2 (#41).Checked on macOS: unit tests 14/14, and
simulate_json.cfpasses in the29_simulate_modeacceptance directory (the three pre-existing prose tests fail there for environmental reasons: non-root uid and nodiffin bindir). On Ubuntu 24.04 as root the whole directory passes, 5/5, with 75/75 unit tests. Reviewed by Claude and by a Fable workflow with adversarial verification, then a final review, with every finding fixed or tracked in #28–#38 and #43; the 2026-10-08 rework applied the last review's findings (aborted,previous_type, config plumbing, extra tests).ELI5 — auditable without C
You know shell and a little Python. Four facts carry the whole argument, and a fifth note covers non-ASCII file names. Each fact can be checked on its own. If any one is wrong, the argument collapses, so that's where to attack it.
What the code is for.
cf-agent --simulateruns the policy against a scratch copy of the files it touches (the "changes chroot"), not the real system. It keeps a log of what it did and, at the end, prints a report for humans. This PR adds--simulate-json=FILE, which writes the files and renames of that change set as one JSON document for programs to read. Packages follow in a separate PR.Fact 1 — the JSON reads only what the text report reads. Simulate mode already leaves logs in its scratch area, including the paths it changed and the renames it did (old/new pairs). The JSON is built from those logs plus the scratch copies:
pathchangedeletedif the scratch copy is gone and the real path exists (a file both created and deleted in the run is left out). Otherwisecreatedormodified, depending on whether the real path exists.type,permissions,uid,gid,size,sha256,targetstat/readlinkwould, without following linksprevious_typerenamesfailsafe_fallbackfailsafe_fallbackclass, i.e. ranfailsafe.cfinstead of your policyabortedabortclassesclass was defined, the same test that makes cf-agent exit 6So the JSON can't know anything the text report doesn't. It also inherits the report's blind spots (see the end). The document is the same in all three
--simulatemodes.Fact 2 — FILE only ever changes through
rename(). In Python terms, the writer does this:Three properties follow. Each comes from the standard meaning of these system calls, not from anything CFEngine-specific.
O_EXCLmakes the create fail if anything already has that name, including a symlink. So the temp file is always a new file that cf-agent made, never something planted.rename(). A failed or killed write leaves FILE exactly as it was, at worst with a stray*.cfnewfile next to it.rename()replaces the name FILE in one step. Readers see either the old file or the complete new one, never half of one. If FILE was a symlink, the link itself is replaced and its target is never opened.The 16 hex characters come from OpenSSL's
RAND_bytes(), CFEngine's source of unpredictable bytes. That matters: with a predictable name, another user could create that file in advance and make the run fail. Even then,O_EXCLmeans a collision only makes the run fail; it never overwrites.Fact 3 — any failure to write the whole document makes cf-agent exit non-zero, with one deliberate exception. The document is built in memory first. If the logs can't be read, nothing is written. Failing to create, write, close or rename the temp file also counts as a failure, and each is reported with its own message. In every one of those cases cf-agent exits 1, unless it was already going to exit with some other non-zero code, which it keeps.
The deliberate exception is a file whose SHA-256 can't be computed. It is written without a
sha256key, an error is logged, and the exit code doesn't change. Leaving the key out is safer than writing zeros. The hashing function has no way to report failure; it just leaves its output all zeros. Written out,"sha256": "0000…0000"looks exactly like a real digest, so a program comparing digests would get a wrong answer with full confidence. A missing key can't be mistaken for anything else. (The code treats an all-zero digest as failure. The chance of a real file hashing to all zeros is 2⁻²⁵⁶.)The document is written just before cf-agent's shutdown routine, because that routine shuts down the OpenSSL library that computes SHA-256.
Fact 4 — a symlink's permission bits carry no information. What you can do through a symlink depends on the target's permissions, not the link's. Linux reports every symlink as
0777. macOS reports whatever the umask gave it. Including them would make the same change look different on two platforms while meaning nothing on either. So symlinks gettargetinstead ofpermissions.Fact 5 (note) — non-ASCII names come out escaped byte by byte because of libntech, not this change. A file name is bytes, but a JSON string is characters:
libntech's
JsonWrite()escapes each byte outside printable ASCII on its own, as\u00XX. In JSON, though,Ãmeans "the character U+00C3" (Ã), not "the byte 0xC3". Socaféis written ascafé, and a standard parser such as Python's reads backcafé. The existingstorejson()policy function produces the same escape, which shows the cause isn't this change. CFEngine's ownreadjson()reverses the escape and gets the right bytes back. The document is pure ASCII, so it's always valid JSON; only non-CFEngine readers misread non-ASCII names. A separate fix to libntech's encoder is being prepared.What is not guaranteed.
fsync, so the all-or-nothing replacement in Fact 2 protects against other programs and a killed cf-agent, not against a power loss.failsafe.cfand still exit 0, as it always has; from a terminal it skipsfailsafe.cfinstead. So check"failsafe_fallback": falsein the document, not the exit code. It comes from the same class cf-agent already sets for this case. Likewise an aborted run (abortclasses) does exit 6, but the document is still written and says"aborted": true, so a reader that only has the file can tell a complete change set from a cut-short one.rel -> modify-meis reported, in both the text and the JSON, as target/modify-me, while a real run createsmodify-me.uid/gidshow the user who ran it, because cf-agent can'tchownits scratch copies.manifest-full. That mode's listing of unchanged files is not in the JSON.How you'd catch me lying, without reading C.
a. The 14 unit tests (
tests/unit/simulate_mode_test) include a created-then-deleted file (not reported), a directory that became a regular file (previous_type), both flags set and unset, and these three cases:The acceptance test
29_simulate_mode/simulate_json.cfruns a real cf-agent and compares the whole document with an expected file. It also checks that the option is refused without--simulateand with a relative path, and that an output path in a missing directory gives exit 1.b. Point FILE into a read-only directory that already holds an old FILE. You get exit 1, the old contents are intact, and no
.cfnewfile is left.c. Make FILE a symlink to a victim file. You get exit 0, the victim is unchanged, and FILE is now a regular 0600 file.
d. Without root, have a policy create a file and then set it to mode
000, so cf-agent can't read its own scratch copy. The entry has nosha256, the log says "Failed to compute the SHA-256 digest", and the exit code is 0.e. Run
printf created | shasum -a 256. It matches thesha256ofcreate-mein the acceptance test's expected output.f. Create a file named
caféand read the JSON with Python'sjson. You getcafé.storejson()on the same string gives the sameé.g. Break your policy's syntax and run non-interactively (
< /dev/null). The log shows the failsafe bundles running, and the document says"failsafe_fallback": true. With a working policy it saysfalse.Checks b through g were run against an installed build. If Fact 2 were wrong, b or c would show a changed victim or a clobbered FILE. If Fact 3 were wrong, d would show zeros or a wrong exit code.
🤖 Generated with Claude Code