From edf3d4a9dbc9b80c231796f24c6f830d5756bfa6 Mon Sep 17 00:00:00 2001 From: u00dxk2 Date: Sat, 29 Aug 2026 13:53:54 -0600 Subject: [PATCH] =?UTF-8?q?0.5.0:=20three=20ports=20=E2=80=94=20presence?= =?UTF-8?q?=20guards=20that=20strip=20comments,=20CLIs=20that=20refuse=20u?= =?UTF-8?q?nknown=20flags,=20verdicts=20a=20pipe=20cannot=20eat?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three artifacts, each generalizing a failure that recurred across an agent fleet, each carrying the house contract: one pure file, declared failure posture, limits pinned as tested expectations, a where-it-stops section. lib/strip-comments.mjs — ANY "string X still appears in file Y" check strips comments first, because a drift guard grepping raw source was satisfied by the JSX comment EXPLAINING the removal it should have caught. A presence guard has both failure directions silent (needle survives in a comment = false green; needle eaten from code = false red), so this is a scanner, not two regexes: strings, template text, ${} substitutions, and regex literals are tracked. The header keeps the correction where the first cut had the exact bug it prevents — a quote in a regex character class desynchronised it and 215 comments survived stripping. Shell (#) variant included; heredocs scanned like code, erring toward FOUND, pinned. lib/known-flags.mjs — an unrecognized option is a usage error, not a default. A dropped flag returns ok:true with the flag ignored, which is how a documented-but-unimplemented flag "worked" three times and a typo'd scope flag shrank a 1,192-file sweep to 2 and printed PASS. Asserts over raw argv (deliberately not a parser); prints every offender; exit accepts a function OR a number, because two authors read the docs as a number and the function-only contract crashed on the one path whose job is a clean refusal. lib/result-line.mjs — every verdict-emitting check prints one terminal greppable RESULT: line, because `cmd | tail; echo exit:$?` prints tail's exit, and that class hit four independent lanes in one day against a correct, starred rule saying not to do it. Documentation is not the remedy; it is what already existed. Unmapped exits render UNKNOWN rather than a guessed verdict; fd:2 is mandatory on --json scripts (a verdict line after a JSON document breaks the parser — pinned); nesting is declared by the parent, sanitized and bounded. 26 new tests, three of which pin LIMITS rather than features. 214 pass. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01CkTF1JryMkGt4Qx2A2vHEh --- README.md | 22 ++ lib/known-flags.mjs | 138 ++++++++++++ lib/result-line.mjs | 181 ++++++++++++++++ lib/strip-comments.mjs | 405 +++++++++++++++++++++++++++++++++++ package.json | 9 +- test/known-flags.test.mjs | 92 ++++++++ test/result-line.test.mjs | 125 +++++++++++ test/strip-comments.test.mjs | 114 ++++++++++ 8 files changed, 1083 insertions(+), 3 deletions(-) create mode 100644 lib/known-flags.mjs create mode 100644 lib/result-line.mjs create mode 100644 lib/strip-comments.mjs create mode 100644 test/known-flags.test.mjs create mode 100644 test/result-line.test.mjs create mode 100644 test/strip-comments.test.mjs diff --git a/README.md b/README.md index af92441..c349f5e 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,9 @@ both fixes are in the history. | [`lib/memory-usage-ledger.mjs`](./lib/memory-usage-ledger.mjs) | Usage evidence for agent-memory eviction: session close records which memory files carried load, the tally reports counts / last touch / never-touched over a window. The tally is EVIDENCE for an eviction pass, never a verdict — and an empty ledger reports itself as absence of evidence, not rot. | **Use as-is** — vendor the one file; wire a thin touch/tally CLI (the lib is pure) | | [`lib/scannable-command.mjs`](./lib/scannable-command.mjs) | De-obfuscates command text before policy regexes scan it (quote-split tokens, ANSI-C `$'…'` including hex/octal/unicode spellings, backslash escapes; `C:\` and `\\server` prefixes survive but interior path separators do not, so match path rules against the raw text), under the raw-OR-normalized doctrine: a hit on either form counts, so normalization can only widen detection, never narrow it. A speed bump against mistakes and injection, not a sandbox boundary. | **Use as-is** — vendor the one file | | [`lib/shadow-screen.mjs`](./lib/shadow-screen.mjs) | The four-state gate vocabulary (`would_block`/`shadow_allow` watching; `block`/`allow` enforcing) plus the explicit `unscreened` verdict when the screener never ran — a check that never ran must never read as a pass. Fail-closed on unscreened in enforce mode; fail-open is a declared edit, never a default. | **Use as-is** — vendor the one file; the pattern doc below says how to read the log | +| [`lib/strip-comments.mjs`](./lib/strip-comments.mjs) | Removes JS/TS/JSX or POSIX-shell comments before a string-PRESENCE check reads the source — a guard that greps raw source is satisfied by the comment explaining a removal, which is a false green with a production outage under it. Tracks strings, template-literal text, `${}` substitutions and regex literals, so neither silent failure direction (needle survives in a comment / needle eaten out of code) occurs on those shapes. | **Use as-is** — vendor the one file (zero deps, pure functions); `includesOutsideComments(src, needle)` is the whole adoption | +| [`lib/known-flags.mjs`](./lib/known-flags.mjs) | Refuses any unrecognized `--flag` before a CLI writes anything, printing every offender. A dropped flag returns `ok:true` with the flag ignored — how a documented-but-never-implemented flag "works" repeatedly, and how a typo'd scope flag shrinks a 1,192-file sweep to 2 files and prints PASS. Deliberately not a parser: it asserts over raw argv, so adoption is one call and no parsing behavior moves. | **Use as-is** — vendor the one file (node: builtins only); pass `entry: import.meta.url` from any script that is also imported by another | +| [`lib/result-line.mjs`](./lib/result-line.mjs) | Prints one terminal, greppable `RESULT: ` line on every path a check process can exit by — because `\| tail`, `\| head`, a captured file, and PowerShell `-Last N` all report the pipe's exit, not the command's. The verdict rides the TEXT as a second carrier; the exit code stays authoritative. An unmapped exit renders `UNKNOWN` rather than a guessed verdict. | **Use as-is** — vendor the one file; pass `{ fd: 2 }` on any `--json` script or the verdict line corrupts the document | | [`patterns/shadow-screen-states.md`](./patterns/shadow-screen-states.md) | Ship gates watching before enforcing, with vocabulary that keeps the log decidable: reviewing `would_block` rows before promotion, refusing to count `unscreened` as a pass, and treating a rising unscreened rate as the outage it is. | **Read and apply** — `shadow-screen` is the state helper | | [`patterns/fail-soft-detectors.md`](./patterns/fail-soft-detectors.md) | The discipline for zero-LLM health detectors around agent fleets: PRESENCE-not-judgment, declared failure postures, structural no-LLM enforcement, alert dedup, no watchdog stacks, resurrect-else-reap, kill-switches. | **Read and apply** — protocol, with `snippet-redact` + `memory-integrity` as reference implementations | | [`patterns/checks-that-cant-fail.md`](./patterns/checks-that-cant-fail.md) | Why a green check nobody has seen go red is not evidence, and the guard for four ways a check silently stops running while still reporting "clean": the never-red monitor, the dead-instrument zero (positive controls), the sweep that reached nothing (NOTHING SWEPT), and the config-absent silent disable. | **Read and apply** — protocol; the operations analog of mutation testing | @@ -175,6 +178,25 @@ Honest framing first, quoted from its upstream (qm): **a speed bump against mist - **It cannot judge screener quality** — a screener that never flags yields wall-to-wall allows. Prove the screener can fire before trusting its quiet ([checks-that-cant-fail](./patterns/checks-that-cant-fail.md)). - **`isPass` refuses `unscreened` and `would_block`** (tested), so a pass-rate dashboard cannot absorb a dark screener into its green number. +### strip-comments + +- **A lexer heuristic, not a parser.** Division-vs-regex is decided by the previous significant token plus a well-formedness lookahead, and the four shapes it misreads (ASI continuation, `/` after `}`, `/` after `)`, JSX text) are named in the header — every one copies verbatim, so a misread can leave a comment start unrecognized inside a regex body but can never strip real code. The ASI case is pinned in [`test/strip-comments.test.mjs`](./test/strip-comments.test.mjs) as a tested LIMIT, not just prose. +- **Template-literal text is opaque on purpose.** A needle between backticks is output the program emits, so a presence guard should find it. If your needle is in a comment inside a `${...}` substitution, that is code and it strips. +- **Two languages.** JS/TS/JSX and POSIX-shell `#`. No HTML comments; shell heredoc bodies are scanned like code, which for a presence guard errs toward FOUND (pinned) — a false absence is the failure this lib exists to prevent, a false presence just makes you look. + +### known-flags + +- **Long flags only.** Single-dash short options, bare positionals, and everything after a literal `--` pass untouched; validating those stays with the caller's own parser. +- **It checks recognition, not implementation.** A flag in the known set that the CLI never wired up sails through (pinned as a tested LIMIT) — and that is the incident's worst case. Keeping the known set equal to the *implemented* set is the caller's half of the contract. +- **`entry` gating fails closed to a no-op** on loaders where `import.meta.url` is not a file URL — the guard goes quiet rather than refusing an importer's valid flags. + +### result-line + +- **Nothing in-process survives `kill -9`.** The line fires on `process.exit()`, natural drain, and uncaught throws; on SIGKILL the exit status (or its absence) is still the only contract. +- **`{ fd: 2 }` is mandatory for `--json` scripts** — a verdict line appended after a JSON document makes the document unparseable, and the first ship of the origin broke a consumer exactly that way (pinned: stdout stays parseable, the verdict moves to stderr). +- **One line per process.** A gate that runs inside another script prints a line that reads as the outer script's verdict; the parent declares nesting via `RESULT_LINE_NEST` because only the parent knows it is one (pinned, including the env value being sanitized and bounded — it reaches a terminal and is not a trusted format string). +- **The vocabulary is closed.** A label outside it renders `UNKNOWN` at exit and warns loudly at arm time (pinned) — a deliberate label must fail visibly on every invocation, never quietly on the one exit that matters. + ## Evidence and lineage These shipped inside a production system, not as a framework exercise: the silent-merge check found a live instance in our own memory index the day it was written; the redaction lib guards our cross-session search path. diff --git a/lib/known-flags.mjs b/lib/known-flags.mjs new file mode 100644 index 0000000..7a017e6 --- /dev/null +++ b/lib/known-flags.mjs @@ -0,0 +1,138 @@ +// @ts-check +// known-flags.mjs — an unrecognized option is a usage error, not a default. +// +// The incident this generalizes: four CLI writers parsed argv permissively — +// an unknown `--flag` landed in the args bag and was never read, so the call +// ran with the flag silently dropped and returned `{ok:true}`. One documented +// flag was never implemented at all and "worked" three times (the CLI fell +// through to its default action and returned that action's shape); a typo'd +// scope flag made a fleet-wide scan cover 2 files instead of 1,192 and print +// PASS. A typo'd or aspirational flag must FAIL, not be ignored. +// +// Deliberately not a parser: callers keep their own parseArgs (they differ — +// repeated flags, stray positional collection). This asserts over raw argv, +// so adoption is one call and no parsing behavior moves. +// +// FAIL-CLOSED at the gate: any unknown `--flag` refuses the whole run, before +// anything is written. The refusal prints EVERY offender, so one re-run fixes +// the whole command line. +// +// WHERE IT STOPS: +// - Only `--long` flags are checked. Single-dash short options, bare +// positionals, and anything after a literal `--` pass untouched — those +// are the caller's to validate. +// - It cannot know a KNOWN flag is unimplemented. The incident's worst case +// (a documented flag that fell through to the default action) is only +// caught here once the flag is removed from the known set; keeping the +// known set equal to the IMPLEMENTED set is the caller's contract. +// - `entry` gating compares resolved file URLs; on exotic loaders where +// `import.meta.url` is not a file URL, the comparison fails closed to +// "not the entry" and the guard becomes a no-op for that module. + +import { resolve } from "node:path"; +import { pathToFileURL } from "node:url"; + +/** Is `moduleUrl` (a caller's import.meta.url) the process entry script? */ +function isEntryModule(moduleUrl) { + const entry = process.argv[1]; + if (!entry) return false; + try { return pathToFileURL(resolve(entry)).href === moduleUrl; } catch { return false; } +} + +/** Levenshtein distance, capped work — flag names are short. */ +function editDistance(a, b) { + const prev = Array.from({ length: b.length + 1 }, (_, i) => i); + for (let i = 1; i <= a.length; i++) { + let diag = prev[0]; + prev[0] = i; + for (let j = 1; j <= b.length; j++) { + const tmp = prev[j]; + prev[j] = Math.min( + prev[j] + 1, + prev[j - 1] + 1, + diag + (a[i - 1] === b[j - 1] ? 0 : 1), + ); + diag = tmp; + } + } + return prev[b.length]; +} + +/** Nearest known flag for a bad token, or null. Distance ≤ 3 and ≤ ~⅓ of the + * flag's length, so an unrelated token gets no misleading suggestion. */ +function suggest(bad, known) { + const lower = bad.toLowerCase(); + let best = null; + let bestD = Infinity; + for (const k of known) { + const d = editDistance(lower, k.toLowerCase()); + if (d < bestD) { bestD = d; best = k; } + } + if (best === null) return null; + return bestD <= Math.min(3, Math.max(1, Math.ceil(best.length / 3))) ? best : null; +} + +/** + * Collect unknown `--flags` from raw argv. + * @param {string[]} argv - process.argv.slice(2) + * @param {Iterable} knownFlags - flag names WITHOUT the leading `--` + * @returns {string[]} human-readable messages, one per unknown flag (empty = clean) + */ +export function findUnknownFlags(argv, knownFlags) { + const known = new Set(knownFlags); + const msgs = []; + for (const tok of Array.isArray(argv) ? argv : []) { + if (typeof tok !== "string" || !tok.startsWith("--")) continue; + if (tok === "--") break; // conventional end-of-flags + const key = tok.slice(2).split("=")[0]; + if (key === "" || known.has(key)) continue; + const near = suggest(key, known); + msgs.push(`unknown flag --${key}${near ? ` (did you mean --${near}?)` : ""}`); + } + return msgs; +} + +/** + * Hard-error on any unknown flag. Prints every offender (not just the first) + * so one re-run fixes the whole command line. + * @param {string[]} argv + * @param {Iterable} knownFlags + * @param {string} scriptName - prefix for the error lines + * @param {{ exit?: ((code: number) => void) | number, log?: (msg: string) => void, entry?: string }} [io] + * `entry: import.meta.url` — enforce ONLY when the calling module is the + * process entry script. REQUIRED for any script that is also IMPORTED: a + * module-scope call without it scans the IMPORTER's argv, and the importer's + * perfectly valid flags get refused by a library it never called. + * + * `exit` accepts EITHER a function OR a plain exit-code NUMBER. The number + * form exists because prose documentation described this option as "the + * exit override carries a non-2 usage code," and two independent authors + * read that as a number and passed one. Under the old function-only + * contract that crashed with `exit is not a function` — a TypeError stack + * trace on the ONE path whose whole job is turning a typo into a clean + * refusal. Found by an adversarial review. The doc was not wrong about + * intent; the lib was too narrow: accept both rather than fix two call + * sites and leave the next author the same trap. + * @returns {boolean} true when clean (only reachable if `exit` does not exit) + */ +export function assertKnownFlags(argv, knownFlags, scriptName, io = {}) { + if (typeof io.entry === "string" && !isEntryModule(io.entry)) return true; + const msgs = findUnknownFlags(argv, knownFlags); + if (msgs.length === 0) return true; + const log = io.log ?? ((m) => console.error(m)); + const exit = + typeof io.exit === "function" + ? io.exit + : Number.isInteger(io.exit) + ? () => process.exit(io.exit) + : (c) => process.exit(c); + for (const m of msgs) log(`${scriptName}: ${m}`); + log( + `${scriptName}: ${msgs.length} unknown flag(s) — nothing was written. ` + + `A dropped flag returns ok:true with the flag ignored, which is how a ` + + `documented-but-unimplemented flag can "work" repeatedly without existing. ` + + `Run --help for the flag list.`, + ); + exit(2); + return false; +} diff --git a/lib/result-line.mjs b/lib/result-line.mjs new file mode 100644 index 0000000..14a5d0b --- /dev/null +++ b/lib/result-line.mjs @@ -0,0 +1,181 @@ +// @ts-check +/** + * result-line.mjs — a terminal, greppable verdict line so a pipe cannot eat + * the answer. + * + * The incident this generalizes recurred on four independent agent lanes in + * one day, against a correct, indexed, starred standing rule that said not to + * do it: `cmd | tail -N; echo exit:$?` prints TAIL's exit, not the command's. + * The same class arrives as `| head -5` printing EXIT=0 over an exit 1, as + * `npm test > out; grep '^# tests' out` reporting FAILED over a fully passing + * suite, as a backgrounded compound reporting 0 over a 127, and as PowerShell + * `Select-Object -Last 4` hiding the failing lines. Documentation is not the + * remedy; documentation is what already existed. + * + * The remedy: every verdict-emitting check prints ONE line as its LAST stdout + * line, carrying the verdict in the TEXT rather than only in the exit code: + * + * RESULT: PASS (exit 0) + * RESULT: FAIL — 3 findings (exit 3) + * RESULT: NOTHING-SWEPT — 0 files matched the scope (exit 2) + * + * A truncating `tail`, a captured file, a backgrounded compound, or a + * PowerShell `-Last N` all still carry it, and `grep '^RESULT:'` finds it + * anywhere. The exit code stays authoritative for CI — this is a second, + * pipe-proof carrier of the same fact, never a replacement. + * + * Usage — one call near the top of the script, before any exit path: + * + * import { armResultLine } from "./lib/result-line.mjs"; + * armResultLine({ 0: "PASS", 2: "NOTHING-SWEPT", 3: "FAIL" }); + * + * It rides `process.on("exit")` and writes with fs.writeSync, so it fires on + * `process.exit()`, on `process.exitCode` + natural drain, and on an uncaught + * throw — every path a check script can leave by. Call `setResultDetail()` at + * any point to attach the count/reason the verdict is about. + * + * FAIL-SAFE by construction: an unmapped exit code renders the code itself + * rather than guessing a verdict, and the emitter swallows its own errors — a + * bug in the verdict line must never change a check's exit code or crash it. + * + * WHERE IT STOPS: + * - It cannot survive `kill -9` / abrupt process death — nothing in-process + * can. The exit code (or its absence) is still the contract there. + * - `fd: 2` is MANDATORY for a run producing machine-readable stdout + * (`--json`): a verdict line appended after a JSON document makes the + * document unparseable — caught by a downstream consumer's test the first + * time the origin shipped this, which is exactly the "a fix that breaks a + * consumer far from its cause" shape the motivating retro was about. + * Compute the JSON flag BEFORE arming and pass `{ fd: 2 }`. + * - One line per PROCESS. A nested gate's verdict line reads as the OUTER + * script's verdict unless the parent declares the nesting (see + * RESULT_LINE_NEST below); the parent must do it, because only the parent + * knows it is one. + */ + +import { writeSync } from "node:fs"; + +// Controlled vocabulary. The trap this guards is LOUD on purpose: three +// origin scripts once used DELIBERATE labels outside the list, and every one +// silently rendered UNKNOWN at the moment it mattered. The vocabulary stays +// controlled, but armResultLine warns at ARM time about any mapped label +// outside it, so the next custom label fails visibly on every invocation +// instead of quietly printing UNKNOWN on the one exit that counts. +const KNOWN_VERDICTS = new Set(["PASS", "FAIL", "NOTHING-SWEPT", "ERROR", "REVIEW", "FINDINGS", "REAPED"]); + +/** + * Labels in a caller's map that the vocabulary would discard. Exported so the + * arm-time warning is unit-testable. + * @param {Record} map + * @returns {string[]} + */ +export function unknownLabelsInMap(map) { + const m = map && typeof map === "object" ? map : {}; + return [...new Set(Object.values(m).filter((v) => typeof v !== "string" || !KNOWN_VERDICTS.has(v)).map(String))]; +} + +let detail = ""; +let armed = false; + +/** + * Attach the count / reason the verdict is about. Last call wins. Safe to + * call from anywhere, including after the verdict is decided. + * @param {unknown} text + */ +export function setResultDetail(text) { + detail = typeof text === "string" ? text : text == null ? "" : String(text); +} + +/** + * Render the line. Pure — exported for tests and for callers that want to + * place the line themselves rather than on process exit. + * + * @param {{ verdict?: unknown, detail?: unknown, exitCode?: unknown }} opts + * @returns {string} + */ +export function formatResultLine({ verdict, detail: d = "", exitCode } = {}) { + const v = typeof verdict === "string" && KNOWN_VERDICTS.has(verdict) ? verdict : "UNKNOWN"; + const text = String(d ?? "").replace(/\s+/g, " ").trim(); + const code = Number.isInteger(exitCode) ? ` (exit ${exitCode})` : ""; + return `RESULT: ${v}${text ? ` — ${text}` : ""}${code}`; +} + +/** + * Map an exit code to a verdict. An unmapped code is NOT guessed — it renders + * as UNKNOWN with the raw code, because a wrong verdict is worse than an + * illegible one (this whole lib exists because a confident wrong answer + * travelled further than a missing one). + * + * @param {number} code + * @param {Record} map + * @returns {string} + */ +export function verdictForExitCode(code, map) { + const m = map && typeof map === "object" ? map : {}; + const hit = m[code]; + return typeof hit === "string" && KNOWN_VERDICTS.has(hit) ? hit : "UNKNOWN"; +} + +/** + * Arm the terminal RESULT line for this process. Idempotent — the first call + * wins, so an imported helper arming again cannot double-print. + * + * Nesting: a gate that runs INSIDE another script prints a line that reads as + * the outer script's verdict, with nothing saying whose it was — a line whose + * whole job is to survive truncation, surviving and naming the wrong subject. + * The parent declares it, because only the parent knows: spawn the child with + * `RESULT_LINE_NEST=