v0.4.6: review the surfaces nobody had reviewed — 20 findings in the collectors, render and ignore paths - #4
Merged
Merged
Conversation
Round 4, my pass — I attacked the property tests rather than the code, by asking the strongest available question: can an OBVIOUSLY wrong redact() still pass all 176 property cases? Three could: - "never mask ':' assignments" passed, because _credential_value() only ever generates values with >=2 character classes — which the whitespace-separator branch masks anyway. A LOW-entropy secret under `key: value` was generated by nothing, so that whole separator distinction was untested. - "give up on lines longer than 200 chars" passed: no property put a secret in a long line. The bounded-cost test uses long lines but asserts nothing about masking. - "delete every shape pass" (PEM, bearer/basic/token, URL credentials, standalone AKIA/ghp_/sk_live/xoxb/JWT shapes, command-flag credentials) passed — the properties never exercised that layer at all; only example tests did. Added: low-entropy-value-under-assignment, secret-far-into-a-long-line, a SHAPE_SECRETS table driven both bare and under noise, and whole-line key material on both diff markers. All six wrong implementations (plus three finer-grained ones that drop a single shape regex) are now caught by the properties ALONE. One expectation of mine was wrong and is now encoded rather than "fixed": a raw base64 key body is matched only when it IS the whole line. Widening that would redact any 40-char hex string — a git SHA in a comment or config — so the residual (a commented-out key body, `# AAAA…`, is shown) is accepted and documented. Suite 463 -> 526. Co-Authored-By: Claude <noreply@anthropic.com>
…ouched
Round 4 pointed two reviewers at (a) the restructured redact() and (b) the
surfaces five prior rounds had never examined. The second was by far the most
productive of the project: the collectors, render and ignore paths had never been
adversarially reviewed, and they held the worst defects found so far.
WORST — a broken helper silently emptied its category. `lsof`, `osascript`,
`scutil`, `kextstat`, `systemextensionsctl` all returned "" on a timeout or a
non-zero exit, so the collector reported an EMPTY category; the diff reads that as
"every listener/login item/DNS entry was removed", the next day as "all of them
are new" with the attacker's port among them, and if the tool stays broken both
snapshots are empty and the report says "Nothing changed" while a backdoor
listens. SECURITY.md promises a skip WITH A NOTE; only a raised error delivers it.
All of them now use need()+run_checked, and coverage_lost no longer requires a
tool stamp — which only 4 of 13 categories have, so the nine loudest categories
could never reach the ORANGE "LOST VISIBILITY" finding at all.
WORST (mine, from round 3) — the interpreter caveat CLEARED `suspicious`. An
unsigned binary named `sh`/`node`/`python_helper` — a name the attacker picks —
dropped from RED to YELLOW, stopped firing --notify, and had its "unsigned" label
replaced by a reassuring explanation. `cp miner ~/Library/.../sh` was a one-line
evasion of the launch-item RED path. The caveat is now appended, never
substituted. Related: an Apple-signed binary COPIED outside the system paths (the
`cp /bin/sh` laundering trick) is now suspicious in its own right.
Collectors that could not see the attack at all:
- browser extensions were fingerprinted by DISPLAY NAME, so overwriting
background.js, adding <all_urls>/cookies/webRequest, or swapping the .xpi were
invisible in a user-writable directory. Now version + manifest hash (+ size/mtime
for Firefox .xpi), and version dirs sort numerically (1.10 > 1.9, which was
reading the OLD manifest).
- `startswith("com.apple")` is a string test, not provenance: naming a rootkit
`com.apple.driver.AudioHelper` removed it from the report entirely, and real
third-party prefixes (co., dev., me.) were never collected.
- the listening anti-churn rule ("no overlap => churn") silently dropped a
single-port rebind (8080 -> 4444) and a backdoor sharing a churny process name.
Only an all-ephemeral multi-port set is churn now.
- Linux .socket and .path units — standard user-level persistence — were never
collected.
- creating a tracked config file was YELLOW while editing one was ORANGE, so
planting ~/.zshenv (sourced by every zsh) was the quieter attack. Now ORANGE.
- the app trust check rebuilt the bundle path from the KEY, so `Calculator
(cask).app` printed another app's signature and `Evil (snap).app` pointed
nowhere; the collector now stores the real path.
redact() leaks closed: an unlisted auth scheme absorbed the mask and printed the
credential (`Authorization: SSWS <token>` in a tracked .curlrc); a quoted value
past the old 512 bound leaked its tail; a separator RUN (`:=`, `=>`, `==`, `=""`)
left the secret as the next token.
Reverted two of my own attempted fixes after measuring them: a whitespace-scheme
rule masked `pam_deny.so` and — worse — masked the wrong token while leaving real
secrets (19 property cases), and a mask-tail pass broke compositionality by eating
shell separators (25 cases). Both are documented residuals with the reasoning.
A third, the H2 path exemption, leaked base64 tokens (base64 uses '/', so they
match "looks like a path") until gated on entropy.
Also: find's partial output is kept on timeout instead of discarded (a false
all-clear on a large HOME); a corrupt labels.json no longer unprotects checkpoints
from pruning.
Suite 526 -> 651. Mutations K1-K11 all caught, 83/83 cumulative.
Co-Authored-By: Claude <noreply@anthropic.com>
… silence a RED
My call on the two items I had left open, both the right trade for a security tool.
Login items were keyed AND fingerprinted by display name with no path collected,
so three attacks were invisible: planting an app named after an existing item,
retargeting an existing item at another binary, and — with no path — no signature
check at all, so login items could never reach RED while the equivalent LaunchAgent
did. Now keyed on the path (via `path of every login item`), trust-checked, with
the name kept as the value for the undo hint. One-time effect: existing login items
appear once as removed+added as the keys change, the same trade as the ` (system)`
autostart tag.
Ignore rules can no longer silence a CRITICAL finding, and suppressions are
disclosed ("N finding(s) hidden by M ignore rule(s)"). A user's own broad rule —
the README suggests `listening:com.docker*` — can be matched by an attacker-chosen
process name, and nothing in either output said rules were active or that anything
had been hidden. A matched RED is kept and annotated instead.
Suite 651 -> 656; mutations L1-L4 caught, 87/87 cumulative.
Co-Authored-By: Claude <noreply@anthropic.com>
CI caught it; my local gate structurally cannot. trust_of() short-circuits on non-macOS, so the Apple-signed-copy assertion fails on ubuntu. Skipped unless codesign is present. Second time this exact class has slipped through: a macOS-only test looks green locally because the local gate IS macOS. I audited the rest of the suite for unmarked platform dependencies — the other candidates all either mock the subprocess layer or degrade gracefully, and CI confirms them green. The durable rule is simply that CI is the only cross-platform gate; a green local run says nothing about the other platform. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follows #3. A fourth adversarial round, pointed for the first time at the code the previous five rounds never examined. 20 findings, and the worst were not in the heavily-audited
redact(). Suite 526 → 656, 87/87 mutations caught.The lesson of this round
Five rounds hammered
redact()and the capability guard. Round 4 asked instead: what has nobody attacked? The answer — collectors, render, ignore rules — held the most serious defects in the project's history.A broken helper silently emptied its category
lsof,osascript,scutil,kextstat,systemextensionsctlall returned""on a timeout or non-zero exit, so the collector reported an empty category. The diff reads that as "every listener was removed"; the next day, "all of them are new" with the attacker's port among them; and if the tool stays broken, both snapshots are empty and the report reads:SECURITY.mdpromises a skip with a note. Only a raised error delivers that. Compounding it,coverage_lostrequired a tool stamp — which only 4 of 13 categories have — so the nine loudest categories could never reach the ORANGELOST VISIBILITYfinding at all.An unsigned binary named
shfell out of RED (regression from v0.4.5)The interpreter caveat I added in 0.4.5 cleared the
suspiciousflag, and the basename is attacker-chosen.cp miner ~/Library/.../shdropped an unsigned payload from RED to YELLOW, stopped--notifyfiring, and replaced "unsigned" with a reassuring explanation. Now appended, never substituted — and an Apple-signed binary copied outside the system paths is suspicious in its own right.Collectors that could not see the attack
background.js, adding<all_urls>/cookies/webRequest, swapping the.xpiall invisible1.10was reading as older than1.9)startswith("com.apple")— naming a rootkitcom.apple.driver.AudioHelperremoved it from the report8080 → 4444silently.socket/.pathpersistence never collectedSeverity and disclosure
Creating a tracked config file was YELLOW while editing one was ORANGE — so planting
~/.zshenv(sourced by every zsh) was the quieter attack. And an ignore rule can no longer silence a critical finding; suppressions are now disclosed, because the README's own example rule can be matched by an attacker-chosen process name.redact()leaks closedAn unlisted auth scheme absorbed the mask and printed the credential (
Authorization: SSWS <token>— a real.curlrcline); a quoted value past the 512-char bound leaked its tail; a separator run (:=,=>,==) left the secret as the next token.Three of my own fixes were reverted after measuring them
A whitespace-scheme rule masked
pam_deny.soand, worse, masked the wrong token while leaving real secrets (19 property cases). A mask-tail pass broke compositionality by eating shell separators (25 cases). A payload-path exemption leaked base64 tokens — base64 uses/, so they match "looks like a path". All three are documented residuals, not silent reverts.Upgrade note
Login-item keys change from display name to path, so existing items appear once as removed+added — the same one-time transition as the
(system)autostart tag.🤖 Generated with Claude Code