diff --git a/CHANGELOG.md b/CHANGELOG.md index eb94cc4d2..2be82049a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 right after the read under a per-user lock so parallel runs cannot store one copy twice, reports partial stores, and never prints value characters. Piped answers without `--yes` and `--yes` without `--clipboard` are refused. + The sentinel carries the date, time and zone, and the error adds its age; + if the value was copied before the stamp, re-collect both names. + A run that stored but could not clear the clipboard exits 3 instead of 0. ### Fixed diff --git a/skills/sudo-secretspec/SKILL.md b/skills/sudo-secretspec/SKILL.md index aaee9ad43..3c3d7ec8f 100644 --- a/skills/sudo-secretspec/SKILL.md +++ b/skills/sudo-secretspec/SKILL.md @@ -189,11 +189,18 @@ run, on a pty or not: two harnesses share it) is held from before `pbpaste` until after the clear. A second run waits about 10 s for it, then exits 1 ("another secrets-collect run holds the clipboard lock"); one that gets in after the clear reads the sentinel, - which names the run that consumed the copy (`... cleared by the run for NAME at - HH:MM:SS ...`), and exits 1. **A parallel loser is not harmless:** the winner may - have taken a copy meant for the loser's name, so **both names must be - re-collected, one at a time**, each after a fresh copy. Re-running only the - loser leaves the winner holding the wrong value (its length would still pass). + which names the run that consumed the copy and carries its date, time, zone and age + (`... cleared by the run for NAME at 2026-10-26T01:40:59-0400 ...`, "N s ago"/"N min ago"), + and exits 1. **The age never proves the loser is harmless: the message is always + conditional.** A copy made after the stamp would have replaced the sentinel, so while the + sentinel is on the clipboard every copy the operator made was before it. **If the operator told + you they copied the value for the loser's name** (whenever they said so: their message + arrives after the copy, often after the stamp, so never compare the stamp with the message), + the named run may have taken that copy, so **both names must be re-collected, one at a time**, + each after a fresh copy (re-running only the loser leaves the named run holding the wrong + value, and its length would still pass). **Only if the operator confirms they had not copied + yet:** copy the value again. **Unsure: re-collect both.** Never read an old sentinel as "no + other run is involved". 3. **The spec MUST carry the expected length** (`NAME=desc:32`), else exit 2. A regex does not count: the spec cannot carry both, and a regex can match stale prose (`.+`); if you need a pattern, pin the length in it and use it @@ -213,8 +220,12 @@ than retrying, and **overwrites the clipboard itself right after the read** SIGTERM and SIGHUP are blocked while the overwrite runs, so one that arrives then is delivered after it. A SIGKILL or power loss between the read and the clear cannot be caught, and neither can a `pbcopy` that fails). The -next run sees a fresh copy or the sentinel, never the previous secret. A failed -overwrite prints `WARNING: ... NOT cleared`. The rejection text gives a count +next run sees a fresh copy or the sentinel, never the previous secret. **A failed +overwrite prints `WARNING: ... NOT cleared` and the run exits 3 instead of 0** (still +stored and notified, `--dry-run` included; its last line says "...the clipboard was NOT +cleared: overwrite it yourself, exit 3"): treat 3 as "done, but the secret may still be on the +pasteboard", never as a clean success. A run that fails otherwise keeps its code (1, 2, 130). +The rejection text gives a count and character classes, never the characters. A clipboard that was stale **before the first run** and has the same shape and length cannot be detected: have the operator copy, say so in chat, then run. SIGTERM/SIGHUP mid-store gets @@ -230,14 +241,21 @@ since a regex does not waive it. For those, and only those, use `--literal` with ```bash python3 $S --secret 'NAME=desc:32' --clipboard --yes -printf 'cleared' | ~/ops/site-private/bin/clip # regardless of the exit status: `;`, never `&&` +# ONLY if it exited 3, its stdout or stderr says `NOT cleared`, or it gave no report at all +# (killed or timed out: exit 124/137, or no output). After a run that cleared, do NOT run this line. +# A run that never read the clipboard (exit 2, `bad --secret`/`bad secret name`/`bad regex`, +# `could not open the lock file`, `could not read the clipboard`): fix the call and re-run it, +# that run reads, clears and reports any sentinel; run this line only if you give up on the name. +printf 'cleared' | ~/ops/site-private/bin/clip sudo-secretspec run -- sh -c 'echo ${#NAME}' ``` -The script clears the clipboard itself; the second line is the belt for the -runs where it died before the read (or printed the WARNING). Run it whether or -not the first line succeeded, so a secret never stays on the pasteboard (clipboard -managers, Universal Clipboard). +The script clears the clipboard itself, so the `printf` line is only for the runs where +the value can still be on the pasteboard (a failed clear, or a run that died before it reported), +or for giving up on a name after a run that never read the clipboard. Do not run it before +re-running: such a run never looked at the clipboard, which may hold another run's sentinel. +After a run that cleared it would overwrite the sentinel, which names the run that consumed the +copy, and the next run could no longer tell whose copy it was. Verified 2026-10-26 (stub `sudo-secretspec`/`pbpaste`/`pbcopy`; no live vault). @@ -488,7 +506,10 @@ listener cannot reach private data through it. an expiry you write down. A fine-grained token is `github_pat_` plus 82 characters (93 in all). 2. **Declare:** `sudo-secretspec add NAME --description "..." --optional --reason "..."` (`--optional` so `check` stays green on hosts that never set it). -3. **Store:** the clipboard route above; then overwrite the clipboard. +3. **Store:** the clipboard route above. With `secrets-collect.py` the script clears the clipboard + itself; overwrite it by hand only on exit 3, `NOT cleared` in stdout or stderr, or no report at all, + or to give up on the name after a run that never read the clipboard (the rule under "An agent + with no terminal"); otherwise you would erase the sentinel's consumer name. 4. **Prove its reach without printing it.** Run a throwaway script under `run` that prints only the length, the prefix and HTTP statuses, then delete the script: ```bash diff --git a/skills/sudo-secretspec/scripts/secrets-collect.py b/skills/sudo-secretspec/scripts/secrets-collect.py index 46c44fe43..88a1bc2b9 100755 --- a/skills/sudo-secretspec/scripts/secrets-collect.py +++ b/skills/sudo-secretspec/scripts/secrets-collect.py @@ -50,12 +50,17 @@ arrives then is delivered after the clear), and (d) holds an exclusive per-user lock (~/Library/Caches/secrets-collect.lock, not under TMPDIR) from before pbpaste until after that overwrite. A second run waits about 10 s, then exits 1; one that gets in after the - clear reads the sentinel and fails, naming the run that consumed the copy: that run may have - taken a copy meant for the loser's name, so BOTH names must be re-collected. A SIGKILL or - power loss between the read and the clear, or a failing pbcopy (a WARNING is printed), leaves - the value on the pasteboard. A clipboard that was stale before the FIRST run and happens to - have the same shape and length is not detectable; --dry-run reads and clears too, so it - consumes the operator's copy. + clear reads the sentinel (it carries the date, time and zone; the error adds the age) and + fails, naming the run that consumed the copy. The age never proves that run is unrelated: + a copy made after the stamp would have replaced it, so if the operator copied the value for + this name at all, that run may have taken it and BOTH names must be re-collected, one at a + time; only if the operator had not copied yet: copy the value again; unsure: both. A same-name sentinel only needs a new copy. A SIGKILL or power + loss between the read and the clear leaves the value on the + pasteboard. A failing pbcopy prints a WARNING and the run (still storing and notifying, also + under --dry-run) exits 3 instead of 0, its last line saying the clipboard was NOT cleared; + a run that fails otherwise keeps its own code (1, 2, 130). A clipboard that was stale + before the FIRST run and happens to have the same shape and length is not detectable; + --dry-run reads and clears too, so it consumes the operator's copy. --clipboard with a non-tty stdin and no --yes is refused: piped answers are not the operator. 8. A clipboard value identical to one already collected in the same run is refused (the clipboard did not change between secrets), so a pty cannot read one value into N names. @@ -64,6 +69,7 @@ import argparse import contextlib +import datetime import fcntl import getpass import os @@ -80,27 +86,46 @@ CLEARED_PREFIX = "secrets-collect: clipboard cleared" # every sentinel this script ever wrote CLEARED_WORD = "cleared" LOCK_WAIT = 10.0 # seconds a second run waits for the clipboard lock before giving up -_CONSUMER = re.compile(r"cleared by the run for (\S+) at (\d\d:\d\d:\d\d)") +AGE_IN_MINUTES_FROM = 60 # seconds: the age is shown as "N s ago" below this, "N min ago" from it +_CONSUMER = re.compile(r"cleared by the run for (\S+) at " + r"(\d{4}-\d\d-\d\dT\d\d:\d\d:\d\d[+-]\d{4}|\d\d:\d\d:\d\d)") # 2nd: old form +_clear_failed = False # set by clear_clipboard when pbcopy failed; main() resets it and exits 3 def is_sentinel(value: str) -> bool: return value == CLEARED_WORD or value.startswith(CLEARED_PREFIX) +def _sentinel_age(at: str) -> float | None: + """Seconds since the sentinel's stamp, or None when it has no date (the old HH:MM:SS form), + cannot be parsed or lies in the future.""" + try: + age = time.time() - datetime.datetime.strptime(at, "%Y-%m-%dT%H:%M:%S%z").timestamp() + except ValueError: + return None + return age if age >= 0 else None + + def sentinel_error(value: str, name: str) -> str: - """Why a sentinel was read. A run that lost the lock race read the winner's sentinel, and the - winner may have taken a copy meant for THIS name, so say who consumed it.""" + """Why a sentinel was read. Another name's sentinel is ALWAYS conditional, whatever its age: + the run that consumed the copy may have taken a copy meant for THIS name, and the age cannot + say whether the operator copied for this name before that run read the clipboard.""" m = _CONSUMER.search(value) if not m: return ("the clipboard holds a cleared-clipboard sentinel: no fresh copy was made since " "the last clear") who, at = m.groups() + age = _sentinel_age(at) + ago = "" + if age is not None: + ago = (f" ({int(age)} s ago)" if age < AGE_IN_MINUTES_FROM + else f" ({int(age // 60)} min ago)") if who == name: return (f"the clipboard holds a cleared-clipboard sentinel: an earlier run for {name} " - f"consumed the copy at {at}; copy the value again") - return (f"the clipboard holds a cleared-clipboard sentinel: the run for {who} consumed this " - f"copy at {at}; if the copy was for {name}, {who} now holds that value: " - f"re-collect {who} too") + f"consumed the copy at {at}{ago}; copy the value again") + return (f"the clipboard holds a cleared-clipboard sentinel: the run for {who} consumed a copy " + f"at {at}{ago}; if the operator copied the value for {name} before {at}, {who} now " + f"holds it: re-collect {who} too; if nothing was copied since, only copy again") class StoreError(Exception): @@ -160,6 +185,8 @@ def _ignore_cancel_signals() -> None: def _not_cleared(e: BaseException) -> None: + global _clear_failed + _clear_failed = True print(f" WARNING: the clipboard was NOT cleared ({type(e).__name__}); " "overwrite it yourself", file=sys.stderr) @@ -167,8 +194,8 @@ def _not_cleared(e: BaseException) -> None: def clear_clipboard(name: str) -> None: """Overwrite the pasteboard with a sentinel naming the run that consumed the copy, so a later run cannot read this secret again and a loser of a lock race can say whom to re-collect.""" - mark = (f"{CLEARED_PREFIX} by the run for {name} at {time.strftime('%H:%M:%S')}, " - "copy the value again") + mark = (f"{CLEARED_PREFIX} by the run for {name} at " + f"{time.strftime('%Y-%m-%dT%H:%M:%S%z')}, copy the value again") try: subprocess.run(["pbcopy"], input=mark, text=True, check=True, capture_output=True, env=_pb_env(), timeout=15) @@ -377,6 +404,8 @@ def notify(role: str, names: list[tuple[str, int]], reason: str) -> None: def main() -> int: + global _clear_failed + _clear_failed = False # in-process callers (tests) run main more than once ap = argparse.ArgumentParser(description="Collect and store secret values (see the module docstring).") ap.add_argument("--secret", action="append", required=True, metavar="NAME=DESC[:LEN]", help="repeatable; LEN is an expected character count (warns on mismatch; " @@ -396,11 +425,25 @@ def main() -> int: ap.add_argument("--yes", action="store_true", help="skip the final 'Store these?' prompt. With --clipboard it also skips the " "Enter wait (tty or not): ONE secret per run, LEN required, the clipboard is " - "cleared after the read, and a lock stops two runs at once. Needed for any " - "--clipboard run with a non-tty stdin. Without --clipboard it needs a " - "terminal (hidden prompt)") + "cleared after the read, and a lock stops two runs at once. A run that " + "finds another name's sentinel prints its date, time, zone and age: compare " + "the stamp with when the value for your name was copied (before it: " + "re-collect both names one at a time; nothing copied since: copy again). " + "Exit 3 = done, but the clipboard could not be cleared: overwrite it " + "yourself. Needed for any --clipboard run with a non-tty stdin. Without " + "--clipboard it needs a terminal (hidden prompt)") a = ap.parse_args() + rc = _run(a) + if rc == 0 and _clear_failed: + what = "dry run, nothing stored" if a.dry_run else "stored" + print(f"{what}, but the clipboard was NOT cleared: overwrite it yourself, exit 3", + flush=True) + return 3 + return rc + + +def _run(a: argparse.Namespace) -> int: tty = sys.stdin.isatty() if a.yes and not a.clipboard and not tty: print("refusing: --yes without --clipboard needs a terminal; a value piped on stdin " diff --git a/tests/sudo_postinstall/test_secrets_collect.py b/tests/sudo_postinstall/test_secrets_collect.py index 7406141c6..08dee54fe 100644 --- a/tests/sudo_postinstall/test_secrets_collect.py +++ b/tests/sudo_postinstall/test_secrets_collect.py @@ -2,6 +2,7 @@ Every subprocess has a hard timeout: a mutant that loops must fail, not hang CI. """ +import calendar import importlib.util import os import pty @@ -11,6 +12,7 @@ import subprocess import sys import time +import types from pathlib import Path import pytest @@ -20,7 +22,7 @@ LEN = len(VALUE) OLD_MARK = "secrets-collect: clipboard cleared, copy the value again" # round 1/2 sentinel CLEARED = re.compile(r"secrets-collect: clipboard cleared by the run for (\w+) at " - r"\d\d:\d\d:\d\d, copy the value again") + r"\d{4}-\d\d-\d\dT\d\d:\d\d:\d\d[+-]\d{4}, copy the value again") TIMEOUT = 10 @@ -141,9 +143,11 @@ def test_second_run_without_a_fresh_copy_stores_nothing(env): assert calls(log) == "" assert "sentinel" in out(r) # it names the run that consumed the copy, since that run may have taken B's value - assert re.search(r"the run for A consumed this copy at \d\d:\d\d:\d\d", out(r)) + assert re.search(r"the run for A consumed a copy at " + r"\d{4}-\d\d-\d\dT\d\d:\d\d:\d\d[+-]\d{4} \(\d+ s ago\)", out(r)) assert "re-collect A too" in out(r) - assert "nothing was copied" not in out(r) + assert "if nothing was copied since, only copy again" in out(r) + assert "leftover" not in out(r) def test_second_run_for_the_same_name_says_so(env): @@ -157,7 +161,9 @@ def test_second_run_for_the_same_name_says_so(env): @pytest.mark.parametrize("stale", [ "cleared", OLD_MARK, - "secrets-collect: clipboard cleared by the run for Z at 01:02:03, copy the value again"]) + "secrets-collect: clipboard cleared by the run for Z at 01:02:03, copy the value again", + "secrets-collect: clipboard cleared by the run for Z at 2020-01-02T03:04:05+0000, " + "copy the value again"]) def test_sentinels_refused_even_when_length_and_literal_fit(env, stale): _, log, clip, _ = env clip.write_text(stale) @@ -236,6 +242,142 @@ def test_pbcopy_failure_is_loud(env): assert clip.read_text() == VALUE +# --- the sentinel carries a date and an age: the other-name text stays conditional at every age ------------------------- + +def _stamp(epoch): + return time.strftime("%Y-%m-%dT%H:%M:%S%z", time.localtime(epoch)) + + +def _mark(who, stamp): + return f"secrets-collect: clipboard cleared by the run for {who} at {stamp}, copy the value again" + + +@pytest.mark.parametrize("who", ["A", "Z"]) +def test_an_old_sentinel_for_another_name_is_still_conditional(env, who): + _, log, clip, _ = env + mark = _mark(who, _stamp(time.time() - 600)) + clip.write_text(mark) + r = run(env, yes_args(f"B=b:{len(mark)}", "--literal")) + o = out(r) + assert r.returncode == 1 and calls(log) == "" + assert "10 min ago" in o + assert f"re-collect {who} too" in o and "if nothing was copied since, only copy again" in o + for verdict in ("no re-collection", "leftover", "not a race", "no other run"): + assert verdict not in o + assert "Traceback" not in o + + +@pytest.mark.parametrize("tz", ["UTC", "America/New_York"]) +def test_sentinel_error_age_and_fallbacks(monkeypatch, tz): + monkeypatch.setenv("TZ", tz) + time.tzset() + try: + _check_sentinel_error_age_and_fallbacks(monkeypatch) + finally: + monkeypatch.undo() + time.tzset() + + +def _check_sentinel_error_age_and_fallbacks(monkeypatch): + m = _load_module() + base = calendar.timegm((2026, 10, 26, 12, 0, 0)) + + def err(stamp, delta, name="A"): + monkeypatch.setattr(m, "time", types.SimpleNamespace(time=lambda: base + delta)) + return m.sentinel_error(_mark("Z", stamp), name) + + def conditional(e): + return ("re-collect Z too" in e and "if nothing was copied since, only copy again" in e + and "if the operator copied the value for A before" in e + and not any(v in e for v in ("no re-collection", "leftover", "not a race", + "no other run"))) + + dated = "2026-10-26T12:00:00+0000" + e = err(dated, 59) + assert "(59 s ago)" in e and conditional(e) + e = err(dated, 60) # the unit switches at 60 s, nothing else does + assert "(1 min ago)" in e and conditional(e) + e = err(dated, 61) + assert "(1 min ago)" in e and conditional(e) + e = err(dated, 7200) + assert "(120 min ago)" in e and conditional(e) + # a stamp with a non-zero offset is the same instant as 12:00 UTC; a parse that ignores + # the zone (reads it as local time) is off by hours at TZ=UTC or New_York alike + e = err("2026-10-26T08:00:00-0400", 30) + assert "(30 s ago)" in e and conditional(e) + e = err("2026-10-26T17:30:00+0530", 90) # 12:00 UTC + assert "(1 min ago)" in e and conditional(e) + e = err(dated, 7200, name="Z") # the same name: only a new copy, never "re-collect" + assert "an earlier run for Z consumed the copy at" in e and "(120 min ago)" in e + assert "re-collect" not in e + e = err(dated, 30, name="Z") + assert "an earlier run for Z consumed the copy at" in e and "(30 s ago)" in e + # not usable as an age: the conditional text without an age, no crash + for stamp, delta in [(dated, -30), ("2026-13-45T99:99:99+0000", 100), ("01:02:03", 100000)]: + e = err(stamp, delta) + assert conditional(e) and "ago" not in e, stamp + + +# --- a failed clear is loud: exit 3 -------------------------------------------------------- + +def _break_pbcopy(bindir): + (bindir / "pbcopy").write_text("#!/bin/sh\nexit 1\n") + + +def test_failed_clear_after_a_store_exits_3_and_says_so(env): + _, log, clip, bindir = env + _break_pbcopy(bindir) + r = run(env, yes_args()) + assert r.returncode == 3, out(r) + assert VALUE in calls(log) # stored all the same + assert "NOT cleared" in r.stderr + assert r.stdout.splitlines()[-1] == ("stored, but the clipboard was NOT cleared: " + "overwrite it yourself, exit 3") + assert clip.read_text() == VALUE + + +def test_failed_clear_on_a_dry_run_exits_3_with_nothing_stored(env): + _, log, clip, bindir = env + _break_pbcopy(bindir) + r = run(env, yes_args(f"A=a:{LEN}", "--dry-run")) + assert r.returncode == 3, out(r) + assert calls(log) == "" + assert r.stdout.splitlines()[-1] == ("dry run, nothing stored, but the clipboard was NOT " + "cleared: overwrite it yourself, exit 3") + + +@pytest.mark.parametrize("spec, extra_env", [(f"A=a:{LEN + 1}", None), (f"A=a:{LEN}", {"FAIL_NAME": "A"})]) +def test_failed_clear_does_not_change_another_failures_code(env, spec, extra_env): + _, log, _, bindir = env + _break_pbcopy(bindir) + r = run(env, yes_args(spec), extra_env=extra_env) # a rejected value / a failed store + assert r.returncode == 1, out(r) + assert "NOT cleared" in r.stderr and "exit 3" not in out(r) + + +def test_a_working_clear_exits_0_and_the_failed_flag_does_not_leak_between_calls( + env, monkeypatch, capsys): + e, log, clip, bindir = env + m = _load_module() + for k in ("PATH", "TMPDIR", "SECRETS_COLLECT_LOCK_DIR"): + monkeypatch.setenv(k, e[k]) + monkeypatch.setattr(sys, "argv", ["secrets-collect", *yes_args()]) + # main() installs SIGTERM/SIGHUP handlers on the pytest process: put them back afterwards + old = {sig: signal.getsignal(sig) for sig in (signal.SIGTERM, signal.SIGHUP)} + try: + good = (bindir / "pbcopy").read_text() + _break_pbcopy(bindir) + assert m.main() == 3 + (bindir / "pbcopy").write_text(good) + clip.write_text(VALUE) + assert m.main() == 0 # a fresh call starts clean + assert cleared_by(clip.read_text()) == "A" + finally: + for sig, h in old.items(): + signal.signal(sig, h) + capsys.readouterr() + + # --- findings 2/12a/3: piped answers are not the operator ------------------------------------ def test_clipboard_with_piped_stdin_and_no_yes_is_refused(env): @@ -305,7 +447,7 @@ def test_parallel_runs_store_the_copy_at_most_once(env, tmp_path, own_tmpdirs): winner = names[[rc for _, rc in res].index(0)] loser_out = res[1 - names.index(winner)][0] assert cleared_by(clip.read_text()) == winner - assert f"the run for {winner} consumed this copy" in "".join(loser_out) + assert f"the run for {winner} consumed a copy" in "".join(loser_out) assert f"re-collect {winner} too" in "".join(loser_out) @@ -372,6 +514,20 @@ def test_unopenable_lock_file_is_reported_as_the_lock(env, tmp_path): assert clip.read_text() == VALUE and calls(log) == "" +def test_default_lock_path_is_under_the_users_home_not_tmpdir(tmp_path, monkeypatch): + m = _load_module() + other = tmp_path / "other_tmpdir" + other.mkdir() + monkeypatch.delenv("SECRETS_COLLECT_LOCK_DIR", raising=False) + monkeypatch.setenv("TMPDIR", str(other)) + monkeypatch.setattr(m, "pwd", types.SimpleNamespace( + getpwuid=lambda uid: types.SimpleNamespace(pw_dir=str(tmp_path)))) + caches = tmp_path / "Library" / "Caches" + assert m.lock_path() == str(caches / "secrets-collect.lock") + assert stat.S_IMODE(caches.stat().st_mode) == 0o700 + assert list(other.iterdir()) == [] # nothing went under TMPDIR + + # --- findings 3, 4: partial stores --------------------------------------------------------- TWO = ["--secret", "A=a", "--secret", "B=b"]