From 140c60fd7de7f5ca351ad375e467f26b22debbaa Mon Sep 17 00:00:00 2001 From: itswl Date: Mon, 12 Oct 2026 19:23:58 +0800 Subject: [PATCH 1/5] stack: close the two open doors, and make the headline numbers reproducible MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two boundaries the containment page had named and left open are closed, and the two things that kept a newcomer out — a front page that was a catalogue, and numbers nobody could check — are fixed. Security (a reversal of a shipped default; see the note in .agents/notes): * the bridge refuses to start with an empty BRIDGE_INBOUND_SECRET. The loud opt-out is BRIDGE_ALLOW_UNSIGNED=1, and the two deploy composes now require the secret at compose time. * the pipe claims each inbound signature once (security.replay_token + Store.claim_signature), so a capture replayed inside the 300-second skew is folded as a duplicate BEFORE the fuse and the pipeline — outside the 120s content window, that used to be a fresh judge call. HOOKRELAY_SIGNATURE_REPLAY_SECONDS, default 300; 0 restores the old bound. * HOOKPROBE_POSTURE_CHECK=enforce is spelled out in the deploy composes. The code already defaulted to enforce; only the documents disagreed. Onboarding: * README opens on Start here (30 seconds / 10 minutes / half a day); the feature list moved to docs/what-you-get.md; docs/quickstart.md is the single entry point. * gen_front_pages rewrites body links to repo-root files into absolute URLs — without it, the new README put a dead relative link on the site front page, which check-docs.py then found. Evidence: * docs/evidence.md + scripts/reproduce_claims.py split every headline number into reproducible-here (the script runs it and fails if it stopped holding) and reported (measured on the retired deployments, source data gitignored). Wired into the root gate and ci-stack, and enumerated by the gate contract test so it cannot drift. Ware: * pytest --cov-fail-under per service (90 / 90 / 85), with ci.yml, ci-hookjudge.yml, ci-hookprobe.yml and the three test_gate_matches_ci contracts moving in lockstep. * assert_weight.py prints the heaviest modules on the PASSING line, not only on failure. The ceiling moves 6,300 -> 6,400 for the replay cache; the doctrine answer is beside it, and both READMEs state the new number. Release: * docs/releasing.md, and the quickstart pins its images by digest as well as tag, so `docker compose up` is reproducible. Verified: `bash scripts/gate.sh` green — all three component gates with their new coverage floors, every stack check, and the docker image builds for hookrelay and hookjudge. NOT run here: `scripts/stack-smoke.sh`, which needs `compose up` and a live stack; CI's stack job covers it. --- ...open-bridge-and-the-replay-window-close.md | 65 ++++++++ .github/workflows/ci-hookjudge.yml | 3 +- .github/workflows/ci-hookprobe.yml | 3 +- .github/workflows/ci-stack.yml | 8 + .github/workflows/ci.yml | 3 +- .gitignore | 4 + CONTRIBUTING.md | 8 + README.md | 40 ++--- README.zh-CN.md | 40 ++--- deploy/docker-compose.shadow.yml | 11 +- deploy/docker-compose.work.yml | 6 +- deploy/lark-bridge/bridge.py | 35 +++- deploy/lark-bridge/tests/test_fail_closed.py | 65 ++++++++ docker-compose.quickstart.yml | 16 +- docker-compose.yml | 1 + docs/blog/proving-the-boundaries-run.md | 60 +++++++ docs/containment.md | 10 +- docs/evidence.md | 47 ++++++ docs/index.md | 36 +---- docs/quickstart.md | 84 ++++++++++ docs/releasing.md | 57 +++++++ docs/what-you-get.md | 41 +++++ docs/zh/index.md | 36 +---- hookjudge/requirements.lock | 2 + hookjudge/requirements.txt | 1 + hookjudge/scripts/gate.sh | 8 +- hookjudge/tests/test_app.py | 2 +- hookprobe/deploy/docker-compose.prod.yml | 4 +- hookprobe/requirements.lock | 2 + hookprobe/requirements.txt | 1 + hookprobe/scripts/gate.sh | 8 +- hookprobe/tests/test_ops.py | 2 +- hookrelay/README.md | 4 +- hookrelay/deploy/docker-compose.yml | 1 + hookrelay/docs/reference.md | 3 +- hookrelay/hookrelay/app.py | 15 +- hookrelay/hookrelay/security.py | 53 ++++-- hookrelay/hookrelay/settings.py | 7 + hookrelay/hookrelay/store.py | 40 +++++ hookrelay/requirements.lock | 2 + hookrelay/requirements.txt | 1 + hookrelay/scripts/gate.sh | 8 +- hookrelay/tests/test_extensibility.py | 4 +- hookrelay/tests/test_signature_replay.py | 130 +++++++++++++++ scripts/assert_weight.py | 25 ++- scripts/gate.sh | 5 + scripts/gen_front_pages.py | 10 +- scripts/reproduce_claims.py | 152 ++++++++++++++++++ 48 files changed, 996 insertions(+), 173 deletions(-) create mode 100644 .agents/notes/implemented/2026-10-12-the-open-bridge-and-the-replay-window-close.md create mode 100644 deploy/lark-bridge/tests/test_fail_closed.py create mode 100644 docs/blog/proving-the-boundaries-run.md create mode 100644 docs/evidence.md create mode 100644 docs/quickstart.md create mode 100644 docs/releasing.md create mode 100644 docs/what-you-get.md create mode 100644 hookrelay/tests/test_signature_replay.py create mode 100755 scripts/reproduce_claims.py diff --git a/.agents/notes/implemented/2026-10-12-the-open-bridge-and-the-replay-window-close.md b/.agents/notes/implemented/2026-10-12-the-open-bridge-and-the-replay-window-close.md new file mode 100644 index 00000000..c9c42094 --- /dev/null +++ b/.agents/notes/implemented/2026-10-12-the-open-bridge-and-the-replay-window-close.md @@ -0,0 +1,65 @@ +--- +title: The open bridge and the replay window close, both fail-closed +status: implemented +date: 2026-10-12 +scope: stack +--- + +## Decision + +Two boundaries that were shaped so an operator had to remember something are now +shaped so forgetting fails closed. + +1. **The bridge refuses to start with an empty `BRIDGE_INBOUND_SECRET`.** It used + to accept every card, and `docs/containment.md` recorded that as the *shipped + default* — the reason both deployments set the secret. A deployment that + genuinely has nobody untrusted on the wire now says `BRIDGE_ALLOW_UNSIGNED=1` + and accepts everything on purpose, out loud. +2. **The pipe claims each inbound signature once.** `security.replay_token` + + `Store.claim_signature` answer the second copy of a signed request as a + duplicate *before* the fuse and the pipeline, so a capture replayed inside the + 300-second skew cannot buy a second verdict. `HOOKRELAY_SIGNATURE_REPLAY_SECONDS` + is the window (default 300; 0 turns the cache off and leaves the old behaviour). + +The `HOOKPROBE_POSTURE_CHECK=enforce` default, which the containment page used to +read as an opt-in, is now spelled out in the deploy composes. The code already +defaulted to `enforce`; only the documents disagreed. + +## Why + +Both rows in `docs/containment.md` named the gap themselves, and a gap that is +safe only while an operator remembers is one the next deployment will not have. +That page's own thesis is that a boundary with limits written down is worth more +than one trusted for more than it holds — but writing the limit down is not the +same as closing it, and two of these were closable in the schema and a startup +check. + +The replay cache closes the last hole in the old argument. `verify_signature` +said a replay "mostly arrives as a duplicate and is skipped as one" — true only +inside the 120-second content-dedup window. An alert replayed at t=200s is inside +the 300-second skew and *outside* the content window, and was a fresh judge call. +The second identity (`replayed_signature`, keyed on the signature) folds exactly +that case without touching the first (`duplicate`, keyed on content). + +The bridge change is a reversal of a shipped default, which is why it has this +note rather than a comment: the old behaviour was documented, deliberate, and +wrong for anyone but the one operator who knew to set the secret. + +## Consequences + +- Two knobs to know: `HOOKRELAY_SIGNATURE_REPLAY_SECONDS` and + `BRIDGE_ALLOW_UNSIGNED`. Both are reachable from a compose and documented in + the reference tables the gate generates. +- A legitimate sender that retries an identical signed request inside the skew + now gets `{"outcome": "skipped", "skip_code": "replayed_signature"}` instead of + a recorded `duplicate` row. Same HTTP shape, one fewer ledger row; the sender's + view is unchanged. +- Plugin adapters that speak their own header (`X-Hub-Signature-256`, Stripe) are + **not** tracked — `replay_token` says so rather than guessing, so those doors + keep the bounded window. Closing them means teaching the adapter its own replay + key, not widening this one. +- The bridge's in-body legacy sign still covers only the timestamp, so its + five-minute window stays bounded, not closed. The containment page says so. +- The pipe grew past 6,300 source lines, so `CEILINGS['hookrelay']` moved to + 6,400 with the doctrine answer recorded beside it: the door is the pipe's job, + it reads the pipe's own header and never a payload. diff --git a/.github/workflows/ci-hookjudge.yml b/.github/workflows/ci-hookjudge.yml index 835e5bed..fdbac09f 100644 --- a/.github/workflows/ci-hookjudge.yml +++ b/.github/workflows/ci-hookjudge.yml @@ -54,7 +54,8 @@ jobs: JSON.parse(fs.readFileSync("hookjudge/static/manifest.webmanifest", "utf8")); console.log("status.html inline JS, sw.js and the manifest: OK"); ' - - run: pytest -q + # Coverage floor, matching gate.sh: a regression catch, not a target. + - run: pytest -q --cov=hookjudge --cov-fail-under=90 - run: python -m pip_audit --progress-spinner off docker: diff --git a/.github/workflows/ci-hookprobe.yml b/.github/workflows/ci-hookprobe.yml index b3062d91..142b6c2c 100644 --- a/.github/workflows/ci-hookprobe.yml +++ b/.github/workflows/ci-hookprobe.yml @@ -60,7 +60,8 @@ jobs: ' - name: shell scripts parse run: find examples scripts -name '*.sh' -print0 | xargs -0 -r -n1 bash -n - - run: pytest -q + # Coverage floor, matching gate.sh: a regression catch, not a target. + - run: pytest -q --cov=hookprobe --cov-fail-under=85 - run: python -m pip_audit --progress-spinner off docker: diff --git a/.github/workflows/ci-stack.yml b/.github/workflows/ci-stack.yml index cad9df64..67fc4c91 100644 --- a/.github/workflows/ci-stack.yml +++ b/.github/workflows/ci-stack.yml @@ -133,6 +133,14 @@ jobs: # in is a comment. - name: the pipe and the brain are inside their stated ceilings run: python3 scripts/assert_weight.py + # Every headline number, in the two classes it can be in: reproducible here + # (run now, fail if it has stopped holding) or measured on the retired + # deployments and named as such. The two pytest-backed claims are UNRUN in + # this job (no service venvs) and reported as UNRUN, never as OK; the + # whole-repository gate, where the component gates just built the venvs, + # runs them for real. + - name: every headline number is reproduced here or named as reported + run: python3 scripts/reproduce_claims.py # Three services build one image tag, and the tag is whichever build ran # last: a build arg one of them does not pass is an image the others did # not mean. The work image shipped without kubectl this way (2026-09-29). diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 13188c16..a8d32ebd 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -94,7 +94,8 @@ jobs: cfg = Config.from_file("config.example.yaml") print(f"{len(cfg.sources)} sources, {len(cfg.channels)} channels, {len(cfg.routes)} routes") PYEOF - - run: pytest -q + # Coverage floor, matching gate.sh: a regression catch, not a target. + - run: pytest -q --cov=hookrelay --cov-fail-under=90 - run: python -m pip_audit --progress-spinner off docker: diff --git a/.gitignore b/.gitignore index 39eaedf0..cdb6a984 100644 --- a/.gitignore +++ b/.gitignore @@ -3,6 +3,10 @@ __pycache__/ .pytest_cache/ .ruff_cache/ +# Coverage data from `pytest --cov`, which every service's gate now runs. +.coverage +.coverage.* +htmlcov/ *.db .env # Rotation backups hold the same credentials as .env. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c703b2f0..acd09b16 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -80,6 +80,14 @@ Four workflows, and between them nothing at the root is uncovered: `ci` (pipe) · `ci-hookjudge` (brain) · `ci-hookprobe` (investigator) · `ci-stack` (the demo pair together, plus the docs and the compose files). +## Evidence and releases + +- [docs/evidence.md](docs/evidence.md) and `scripts/reproduce_claims.py`: every + headline number, split into one this repository can reproduce (the script runs + it) and one measured on the retired deployments and named as such. +- [docs/releasing.md](docs/releasing.md): a release is one tag and four + digest-pinned images, and the notes are the decision records between two tags. + ## The one rule that is not about layout Run `bash scripts/gate.sh` as its own step and READ the verdict before any diff --git a/README.md b/README.md index dab072d7..25ce93c4 100644 --- a/README.md +++ b/README.md @@ -18,38 +18,16 @@ A signal comes in: an alert, a chat message, a ticket, a timer. A pipe signs, ro The investigator is the product. The pipe and the judge are the smallest alerting front end for a team that has none. One deployment is one team: this is not a work-management platform, a multi-tenant SaaS or an agent framework. -![hookprobe's console: a real investigation, conclusion first, the root cause it could not establish said so, and the bill under it](docs/img/hookprobe-sessions.png) - -## Try it: ten minutes, no keys, no bill - -```bash -curl -fsSLO https://raw.githubusercontent.com/itswl/hookstack/main/docker-compose.quickstart.yml -docker compose -f docker-compose.quickstart.yml up -d # pipe + judge + stub model + investigator on a rehearsal + readable sink -bash <(curl -fsSL https://raw.githubusercontent.com/itswl/hookstack/main/scripts/demo.sh) -``` - -The demo runs the whole loop. An alert is judged, a recorded investigation replays through the real read-only gate, the report arrives as a card, an approve press runs two allowlisted commands, and the alert resolves. To use a real model, put `HOOKPROBE_RUNTIME=claude`, `HOOKPROBE_MODEL` and a key in `.env`. +## Start here -## What you get +- **30 seconds:** [docs/quickstart.md](docs/quickstart.md) opens with the shape of it — three services, one diagram. +- **10 minutes, no keys, no bill:** the same page's second section runs the whole loop on a rehearsal: an alert judged, an investigation replayed through the real read-only gate, a card, an approve press, a resolved alert. +- **Half a day, for real:** [STACK.md](STACK.md) runs all three from source; [docs/containment.md](docs/containment.md) is the boundary map you read *before* pointing it at a cluster. +- **What it does, in full:** [docs/what-you-get.md](docs/what-you-get.md). -- **Only the work worth doing.** A storm, a restatement or a recovery never buys a second verdict. On 795 production alerts, 28 of 29 rules answered the same every time, so `rule-reuse` answers them for free. -- **A card a person can rule on.** Every button is signed before the card leaves, and every press is recorded. -- **Remediation with a person in the loop.** An approved procedure runs step by step against an allowlist, as argv, never through a shell. A press approves the version that was read: the card and the console both name the procedure's digest, and one that changed since is refused. A plan a person hands off is carried out by a separate work node, with a write credential of its own and a guard that refuses destructive commands; it made its first real change on 2026-09-30 ([how it went](docs/deployments.md#the-first-real-write)). -- **Investigations that leave something behind.** A finished run distills a runbook, and the next occurrence starts from it. -- **Agents you can contain.** Read-only by default and measured at startup, budget ceilings that refuse out loud, and the structural boundaries, each written up with what it does **not** stop ([containment](docs/containment.md)). -- **Boards that install on your phone.** All three boards are web apps you add to the home screen, the investigator's under its node's own name; each keeps its tokens in that browser and nothing else, and offline it shows the page and says its service is out of reach. On a phone the header runs under the status bar in its own colour and follows the theme, the tabs fit one row, and wide tables become one block per row. -- **One audit page per event.** Every hop, digest, decision and human action, a flight recorder of every tool call, and a timing waterfall per run. -- **Your model, your chat.** Any Anthropic-dialect endpoint for the investigator, any OpenAI-compatible one for the judge, local models included. Feishu/Lark through a bridge ([protocol](docs/bridge-protocol.md)), DingTalk and WeCom as a plugin, or signed JSON to any webhook. - -![hookrelay's board: one sentence on what waits on you, four numbers, and one row per alert with its seven stages](docs/img/hookrelay-timeline.png) - -![hookjudge's board: verdicts with their routes and what each cost](docs/img/hookjudge-status.png) - -![hookrelay's ledger: every event with its decision chain, its deliveries, and what came back](docs/img/hookrelay-ledger.png) - -![The three boards on a phone: the pipe's attention entry, the investigator's procedure executed after a named person approved it, and the judge's board](docs/img/hookstack-phones.png) +![hookprobe's console: a real investigation, conclusion first, the root cause it could not establish said so, and the bill under it](docs/img/hookprobe-sessions.png) -The three boards read in Chinese or English and follow the system's light or dark. The screenshots come from local runs, not mockups. +The three boards read in Chinese or English and follow the system's light or dark; [the full tour](OVERVIEW.md) has every screenshot, from local runs and not mockups. ## How it fits together @@ -64,6 +42,8 @@ The same pipe also carries an operator's own work signals to a planner and, on a ## Read more +- [docs/quickstart.md](docs/quickstart.md): the one page to start from +- [docs/what-you-get.md](docs/what-you-get.md): the feature list, in full - [OVERVIEW.md](OVERVIEW.md): the tour, with every screenshot - [STACK.md](STACK.md): run the whole stack locally, step by step - [hookrelay](hookrelay/README.md), [hookjudge](hookjudge/README.md), [hookprobe](hookprobe/README.md): each service in depth @@ -73,6 +53,6 @@ The same pipe also carries an operator's own work signals to a planner and, on a ## Developing -`bash scripts/gate.sh` runs every service's gate plus the stack checks. Read its verdict before you commit. The pipe caps itself at 6,300 source lines and the judge at 3,400; the investigator is uncapped. +`bash scripts/gate.sh` runs every service's gate plus the stack checks. Read its verdict before you commit. The pipe caps itself at 6,400 source lines and the judge at 3,400; the investigator is uncapped. MIT licensed. diff --git a/README.zh-CN.md b/README.zh-CN.md index 1f0c98a6..12e85bbf 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -14,38 +14,16 @@ 调查员是产品本身，管道和判官是给没有告警平台的团队准备的最小前端。一套部署就是一个团队：它不是工作管理平台，不是多租户 SaaS，也不是 agent 框架。 -![hookprobe 的控制台：一次真实的调查，结论在最前面，查不到的根因明说查不到，下面是这次的花费](docs/img/zh/hookprobe-sessions.png) - -## 试一下：十分钟，不要 key，不花钱 - -```bash -curl -fsSLO https://raw.githubusercontent.com/itswl/hookstack/main/docker-compose.quickstart.yml -docker compose -f docker-compose.quickstart.yml up -d # 管道 + 判官 + 桩模型 + 跑排练的调查员 + 可读的接收端 -bash <(curl -fsSL https://raw.githubusercontent.com/itswl/hookstack/main/scripts/demo.sh) -``` - -演示会跑完整条闭环：告警被判定，一次录好的调查经过真实的只读门回放，报告变成卡片，按下批准后跑两条允许清单里的命令，告警恢复。要换成真实模型，在 `.env` 里设置 `HOOKPROBE_RUNTIME=claude`、`HOOKPROBE_MODEL` 和模型 key。 +## 从这里开始 -## 你能得到什么 +- **30 秒：**[docs/quickstart.md](docs/quickstart.md) 开头就讲清了它的形状 —— 三个服务，一张图。 +- **10 分钟，不要 key，不花钱：** 同一页的第二段会在排练上跑完整条闭环：告警被判定，调查经过真实的只读门回放，卡片，按下批准，告警恢复。 +- **半天，真上：**[STACK.md](STACK.md) 从源码一步步跑起三个服务；[docs/containment.md](docs/containment.md) 是指向真实集群**之前**要读的边界地图。 +- **完整功能清单：**[docs/what-you-get.md](docs/what-you-get.md)（英文）。 -- **只做值得做的事。** 告警风暴、重复的同一条件、恢复事件都不会再花第二次判定的钱。在 795 条生产告警上实测，29 条规则里有 28 条每次答案都一样，所以 `rule-reuse` 不调模型就能回答。 -- **一张人可以裁定的卡片。** 每个按钮在卡片发出前就签了名，每次按下都有记录。 -- **修复有人把关。** 批准后的流程逐步对照允许清单执行，以 argv 运行，从不经过 shell。批准的是读到的那个版本：卡片和控制台都会带上流程的摘要，之后变过的流程会被拒绝。人交接的计划由单独的执行节点用它自己的写凭证去做，危险命令会被拒绝；2026-09-30 它第一次真实改动了系统（[经过](docs/deployments.md#the-first-real-write)）。 -- **调查会留下东西。** 跑完的调查会提炼成手册，同一条件下次出现时从手册开始。 -- **关得住的 agent。** 默认只读并在启动时实测，预算用完会明确拒绝，另有一组结构性边界，每一条都写明它**挡不住**什么（[containment](docs/containment.md)）。 -- **装到手机上的看板。** 三个看板都是可以添加到主屏幕的 web 应用，调查员的以节点自己的名字出现；口令只留在那个浏览器里，离线时显示页面并告诉你连不上服务。在手机上，页头延伸到状态栏下方、颜色随主题切换，页签一行放下，宽表格按行变成小块。 -- **每个事件一页审计。** 每一跳、摘要、决策和人的操作，每次工具调用的飞行记录，以及每次运行的耗时瀑布图。 -- **模型和聊天工具由你选。** 调查员接任意 Anthropic 方言的端点，判官接任意 OpenAI 兼容的端点，本地模型也可以。飞书经桥接送达（[协议](docs/bridge-protocol.md)），钉钉和企微有插件，也可以把签名 JSON 发到任意 webhook。 - -![hookrelay 的看板：一句话说有几件事等你处理，四个数字，每条告警一行，各带七个阶段](docs/img/zh/hookrelay-timeline.png) - -![hookjudge 的看板：每条判定走了哪条路径、花了多少](docs/img/zh/hookjudge-status.png) - -![hookrelay 的流水：每个事件的决策、投递去向和回流](docs/img/zh/hookrelay-ledger.png) - -![三块看板在手机上：管道的待处理入口，调查员那条经有名字的人批准后执行的处置流程，以及判官的看板](docs/img/zh/hookstack-phones.png) +![hookprobe 的控制台：一次真实的调查，结论在最前面，查不到的根因明说查不到，下面是这次的花费](docs/img/zh/hookprobe-sessions.png) -三块看板都能切换中英文，并跟随系统的浅色或深色。截图取自本地真实运行，不是效果图。 +三块看板都能切换中英文，并跟随系统的浅色或深色；[完整介绍](OVERVIEW.md)（英文）里有全部截图，取自本地真实运行，不是效果图。 ## 怎么串起来 @@ -62,6 +40,8 @@ bash <(curl -fsSL https://raw.githubusercontent.com/itswl/hookstack/main/scripts 以下文档都是英文。 +- [docs/quickstart.md](docs/quickstart.md)：应该从这里读的第一页 +- [docs/what-you-get.md](docs/what-you-get.md)：完整功能清单 - [OVERVIEW.md](OVERVIEW.md)：完整介绍，含全部截图 - [STACK.md](STACK.md)：在本地一步步跑起整个栈 - [hookrelay](hookrelay/README.md)、[hookjudge](hookjudge/README.md)、[hookprobe](hookprobe/README.md)：每个服务的详细说明 @@ -71,6 +51,6 @@ bash <(curl -fsSL https://raw.githubusercontent.com/itswl/hookstack/main/scripts ## 开发 -`bash scripts/gate.sh` 会跑每个服务的 gate 和整个栈的检查。提交前读它的结论。管道的源码上限是 6,300 行，判官 3,400 行，调查员不设上限。 +`bash scripts/gate.sh` 会跑每个服务的 gate 和整个栈的检查。提交前读它的结论。管道的源码上限是 6,400 行，判官 3,400 行，调查员不设上限。 MIT 协议。 diff --git a/deploy/docker-compose.shadow.yml b/deploy/docker-compose.shadow.yml index 4c3c679d..96ca0b9c 100644 --- a/deploy/docker-compose.shadow.yml +++ b/deploy/docker-compose.shadow.yml @@ -40,6 +40,7 @@ services: HOOKRELAY_DB: /data/hookrelay.db HOOKRELAY_CONFIG: /etc/hookrelay/shadow.yaml HOOKRELAY_WORKER_INTERVAL: "0.5" + HOOKRELAY_SIGNATURE_REPLAY_SECONDS: ${HOOKRELAY_SIGNATURE_REPLAY_SECONDS:-300} # closes the captured-request replay window (0 = bounded only) WW_RELAY_SECRET: ${WW_RELAY_SECRET:?mirror FORWARD_RELAY_SECRET into .env} # The platform's report intake (its /v1/deep-analyses/intake), and the # secret it verifies — mirror of its DEEP_ANALYSIS_REPORT_INTAKE_SECRET. @@ -239,10 +240,10 @@ services: LARK_CHAT_ID: ${LARK_CHAT_ID:?set LARK_CHAT_ID in .env} RELAY_ACTION_URL: http://hookrelay:8100/card-action # Inbound card authentication. Same value as hookrelay's LARK_BRIDGE_SECRET - # above. Empty (default) = accept any inbound card (previous behaviour); - # set both to close the shared-network hole where a neighbour could post a - # card into the operator's chat as the app. - BRIDGE_INBOUND_SECRET: ${LARK_BRIDGE_SECRET:-} + # above. REQUIRED since 2026-10-12: the bridge refuses to start unsigned, so + # a neighbour on the shared network cannot post a card into the operator's + # chat as the app. `:?` makes compose say so before the container is built. + BRIDGE_INBOUND_SECRET: ${LARK_BRIDGE_SECRET:?set LARK_BRIDGE_SECRET in .env (the bridge refuses to start unsigned)} # A person's reply under one of the pipe's cards goes to this door, signed # with LARK_THREAD_SECRET (the value the door in shadow.yaml verifies). # Needs the app to receive group messages (im.message.receive_v1) — see @@ -266,7 +267,7 @@ services: mem_limit: 128m environment: LARK_WEBHOOK_URL: ${WATCH_FEISHU_WEBHOOK:?set WATCH_FEISHU_WEBHOOK in .env} - BRIDGE_INBOUND_SECRET: ${LARK_BRIDGE_SECRET:-} + BRIDGE_INBOUND_SECRET: ${LARK_BRIDGE_SECRET:?set LARK_BRIDGE_SECRET in .env (the bridge refuses to start unsigned)} depends_on: - hookrelay diff --git a/deploy/docker-compose.work.yml b/deploy/docker-compose.work.yml index c9d3beb5..e9859a0e 100644 --- a/deploy/docker-compose.work.yml +++ b/deploy/docker-compose.work.yml @@ -60,6 +60,7 @@ services: HOOKRELAY_DB: /data/hookrelay.db HOOKRELAY_CONFIG: /etc/hookrelay/work.yaml HOOKRELAY_WORKER_INTERVAL: "0.5" + HOOKRELAY_SIGNATURE_REPLAY_SECONDS: ${HOOKRELAY_SIGNATURE_REPLAY_SECONDS:-300} # closes the captured-request replay window (0 = bounded only) # work.yaml's ${...} refs are expanded by hookrelay from its OWN # environment, not by compose — every name used there has to be passed # through here or it silently resolves to empty. An empty URL crashes at @@ -742,6 +743,9 @@ services: HOOKPROBE_RETURN_SECRET: ${WORK_RETURN_SECRET:?set WORK_RETURN_SECRET in .env} # The whole reason this container exists as a separate service. HOOKPROBE_BASH_GUARD: danger-only + # The code default is already `enforce`; named here because this is the + # node to which it matters (a credential that grows past BLAST_RADIUS refuses). + HOOKPROBE_POSTURE_CHECK: ${HOOKPROBE_POSTURE_CHECK:-enforce} # The blast radius this node declares about itself: one measured line per # entry, exactly as /v1/posture reports it. Empty means the posture check # records what the credential can do and judges none of it — which is @@ -899,7 +903,7 @@ services: # The default chat (the watcher's) and the other chat it may serve (the planner's). LARK_CHAT_ID: ${LARK_WATCH_CHAT_ID:-} BRIDGE_CHAT_IDS: ${LARK_PLAN_CHAT_ID:-} - BRIDGE_INBOUND_SECRET: ${LARK_BRIDGE_SECRET:-} + BRIDGE_INBOUND_SECRET: ${LARK_BRIDGE_SECRET:?set LARK_BRIDGE_SECRET in .env (the bridge refuses to start unsigned)} RELAY_ACTION_URL: http://hookrelay:8100/card-action RELAY_THREAD_URL: ${RELAY_THREAD_URL:-http://hookrelay:8100/hook/lark-thread} THREAD_SECRET: ${LARK_THREAD_SECRET:-} diff --git a/deploy/lark-bridge/bridge.py b/deploy/lark-bridge/bridge.py index 6feb64a5..363b2afe 100644 --- a/deploy/lark-bridge/bridge.py +++ b/deploy/lark-bridge/bridge.py @@ -80,10 +80,17 @@ # app. When BRIDGE_INBOUND_SECRET is set, require the pipe's Feishu-style sign: # hookrelay's feishu channel, given a channel secret, adds {timestamp, sign} to # the body, sign = base64(HMAC-SHA256(key="{ts}\n{secret}", msg="")). Set the -# same value as the `to-me` channel's secret. Unset = accept everything, which -# is the previous behaviour, so turning this on is a two-line .env change with no -# code risk: an unset bridge never rejects a card. +# same value as the `to-me` channel's secret. +# +# FAIL-CLOSED since 2026-10-12. Until then an unset secret accepted every card, +# and `docs/containment.md` named that — not as a caveat but as the shipped +# default, "which is why both deployments set it". A boundary that is safe only +# while an operator remembers is one the next deployment will not have, so an +# empty secret now refuses to START (main() below). A deployment that genuinely +# has nobody untrusted on the wire says BRIDGE_ALLOW_UNSIGNED=1 and accepts +# every card on purpose, out loud. INBOUND_SECRET = os.environ.get("BRIDGE_INBOUND_SECRET", "") +ALLOW_UNSIGNED = os.environ.get("BRIDGE_ALLOW_UNSIGNED", "").strip().lower() in ("1", "true", "yes", "on") _SIGN_SKEW_SECONDS = 300 # Preventive reconnect, and the honest name for it is a RECYCLE. @@ -177,10 +184,11 @@ def is_authentic_protocol(headers: Any, body: bytes) -> bool: X-Hook-Timestamp and X-Hook-Signature = hex HMAC-SHA256(secret, "{ts}.{body}") — the same scheme this bridge uses when it signs a message for the pipe's thread door, so both directions are one function. The body is covered, which - the legacy in-body sign below never managed. Unset secret = always True. + the legacy in-body sign below never managed. An empty secret answers + ALLOW_UNSIGNED, which main() refuses to reach with at startup. """ if not INBOUND_SECRET: - return True + return ALLOW_UNSIGNED timestamp = str(headers.get("x-hook-timestamp") or "") provided = str(headers.get("x-hook-signature") or "") if not timestamp or not provided: @@ -198,10 +206,11 @@ def is_authentic(payload: dict) -> bool: The sign covers the timestamp, not the card body, so a captured card can be replayed inside the window — bounded, and the card is not attacker-chosen. - Forging a NEW card still needs the secret. Unset secret = always True. + Forging a NEW card still needs the secret. An empty secret answers + ALLOW_UNSIGNED, which main() refuses to reach with at startup. """ if not INBOUND_SECRET: - return True + return ALLOW_UNSIGNED timestamp = str(payload.get("timestamp") or "") provided = str(payload.get("sign") or "") if not timestamp or not provided: @@ -859,6 +868,18 @@ def consume(event_key: str, handler, attach_only: bool = False) -> None: def main() -> None: + if not INBOUND_SECRET and not ALLOW_UNSIGNED: + raise SystemExit( + "bridge: BRIDGE_INBOUND_SECRET is empty. This port shares a container network with the " + "platform, so an unsigned card from any neighbour is a card in the operator's chat. Set " + "BRIDGE_INBOUND_SECRET to the `to-me` channel's secret, or set BRIDGE_ALLOW_UNSIGNED=1 to " + "accept every card on purpose (docs/containment.md)." + ) + if not INBOUND_SECRET: + logger.warning( + "BRIDGE_ALLOW_UNSIGNED=1: accepting unsigned cards — anyone who can reach port %s can post as the app", + LISTEN_PORT, + ) if WEBHOOK_MODE: # Nothing to consume: a custom bot has no events. Cards in, webhook out. logger.info("bridge up (webhook mode): port=%s", LISTEN_PORT) diff --git a/deploy/lark-bridge/tests/test_fail_closed.py b/deploy/lark-bridge/tests/test_fail_closed.py new file mode 100644 index 00000000..9c9d493e --- /dev/null +++ b/deploy/lark-bridge/tests/test_fail_closed.py @@ -0,0 +1,65 @@ +"""An empty inbound secret refuses to start, instead of accepting every card. + +`docs/containment.md` used to name the open bridge as the shipped default: with +BRIDGE_INBOUND_SECRET unset the port took a card from anything on the shared +network, and the only thing standing between that and the operator's chat was an +operator remembering to set it. These tests pin the fail-closed replacement — an +empty secret is a refusal to run, and BRIDGE_ALLOW_UNSIGNED=1 is the loud, +deliberate way to say a deployment has nobody untrusted on the wire. + +The two `is_authentic*` halves are checked as well as `main()`, because the door +and the guard are separate code paths and only one of them used to be reachable +through `main()`. +""" + +from __future__ import annotations + +import bridge +import pytest + + +def test_protocol_door_closed_when_secret_empty_and_not_allowed(monkeypatch): + monkeypatch.setattr(bridge, "INBOUND_SECRET", "") + monkeypatch.setattr(bridge, "ALLOW_UNSIGNED", False) + assert bridge.is_authentic_protocol({"x-hook-timestamp": "1", "x-hook-signature": "x"}, b"{}") is False + + +def test_legacy_door_closed_when_secret_empty_and_not_allowed(monkeypatch): + monkeypatch.setattr(bridge, "INBOUND_SECRET", "") + monkeypatch.setattr(bridge, "ALLOW_UNSIGNED", False) + assert bridge.is_authentic({"timestamp": "1", "sign": "x"}) is False + + +def test_everything_is_accepted_only_when_allowed_out_loud(monkeypatch): + monkeypatch.setattr(bridge, "INBOUND_SECRET", "") + monkeypatch.setattr(bridge, "ALLOW_UNSIGNED", True) + # The deliberate posture: no secret AND the opt-in. Nothing is checked. + assert bridge.is_authentic_protocol({"x-hook-timestamp": "1", "x-hook-signature": "x"}, b"{}") is True + assert bridge.is_authentic({"timestamp": "1", "sign": "x"}) is True + + +def test_main_refuses_to_start_without_a_secret(monkeypatch): + monkeypatch.setattr(bridge, "INBOUND_SECRET", "") + monkeypatch.setattr(bridge, "ALLOW_UNSIGNED", False) + with pytest.raises(SystemExit, match="BRIDGE_INBOUND_SECRET is empty"): + bridge.main() + + +def test_main_starts_when_the_opt_in_is_set(monkeypatch): + monkeypatch.setattr(bridge, "INBOUND_SECRET", "") + monkeypatch.setattr(bridge, "ALLOW_UNSIGNED", True) + monkeypatch.setattr(bridge, "WEBHOOK_MODE", True) + + class FakeServer: + def __init__(self, address, handler): + self.address = address + + def serve_forever(self, **kwargs): + return None + + def shutdown(self): + return None + + monkeypatch.setattr(bridge, "ThreadingHTTPServer", FakeServer) + # Returns rather than raising: the opt-in is the only way past the guard. + assert bridge.main() is None diff --git a/docker-compose.quickstart.yml b/docker-compose.quickstart.yml index afe6feee..76bc01e6 100644 --- a/docker-compose.quickstart.yml +++ b/docker-compose.quickstart.yml @@ -44,7 +44,7 @@ name: hookstack services: hookrelay: - image: ghcr.io/itswl/hookrelay:0.6.0 + image: ghcr.io/itswl/hookrelay:0.6.0@sha256:2489b6e89626cfe85b26db49d96221b0be481a4bb1510d18755744be2fba6ed1 container_name: hs-hookrelay restart: unless-stopped environment: @@ -65,6 +65,10 @@ services: HOOKRELAY_ACTION_SECRET: demo-card-secret HOOKRELAY_PUBLIC_URL: ${HOOKRELAY_PUBLIC_URL:-http://127.0.0.1:8100} HOOKRELAY_WORKER_INTERVAL: "0.5" + # A captured request replayed inside the signature's freshness window is + # answered as a duplicate instead of buying a second verdict. 0 leaves the + # window bounded rather than closed (security.verify_signature). + HOOKRELAY_SIGNATURE_REPLAY_SECONDS: ${HOOKRELAY_SIGNATURE_REPLAY_SECONDS:-300} # Where the escalation route delivers: the investigator's door. Point it # at the sink's /probe-standin instead to see the escalation shape alone. HOOKPROBE_EVENT_URL: ${HOOKPROBE_EVENT_URL:-http://hookprobe:8088/hooks/event} @@ -82,7 +86,7 @@ services: - sink hookjudge: - image: ghcr.io/itswl/hookjudge:0.6.0 + image: ghcr.io/itswl/hookjudge:0.6.0@sha256:9b6c537c82e8b4ead1f8d68029889caca642f653a8fbb419b7f5173707204e30 container_name: hs-hookjudge restart: unless-stopped environment: @@ -112,7 +116,7 @@ services: # operator would actually have received. It runs from the pipe's own image # because that image now carries it — one fewer thing to pull, and no mount. sink: - image: ghcr.io/itswl/hookrelay:0.6.0 + image: ghcr.io/itswl/hookrelay:0.6.0@sha256:2489b6e89626cfe85b26db49d96221b0be481a4bb1510d18755744be2fba6ed1 container_name: hs-sink restart: unless-stopped command: ["python", "-u", "/app/examples/sink.py"] @@ -125,7 +129,7 @@ services: # exactly the hop a real deployment makes to a custom bot's incoming URL. # `ops-feishu` in the baked config is `type: bridge` and points here. lark-bridge: - image: ghcr.io/itswl/lark-bridge:0.6.0 + image: ghcr.io/itswl/lark-bridge:0.6.0@sha256:781fcd8166a266903e266ed11e0990f6235c7c4ef71b46ce66450561c2159afa container_name: hs-lark-bridge restart: unless-stopped environment: @@ -143,7 +147,7 @@ services: # not a model: it reports itself as `stub-4o-mini`. Point the judge's AI vars # at a real provider and this stops being used. stub-ai: - image: ghcr.io/itswl/hookjudge:0.6.0 + image: ghcr.io/itswl/hookjudge:0.6.0@sha256:9b6c537c82e8b4ead1f8d68029889caca642f653a8fbb419b7f5173707204e30 container_name: hs-stub-ai restart: unless-stopped command: ["python", "-u", "/app/examples/stub_ai.py"] @@ -155,7 +159,7 @@ services: # pipe to the same channels; the disk alert in demo.sh also proposes a # two-step procedure the approve button runs against the allowlist below. hookprobe: - image: ghcr.io/itswl/hookprobe:0.6.0 + image: ghcr.io/itswl/hookprobe:0.6.0@sha256:019b5a3ff36bee4c857badf912e24b476b4fde8ff9b93259e3140c3583354223 container_name: hs-hookprobe restart: unless-stopped # The agent executes arbitrary bash, so the container is bounded even in a diff --git a/docker-compose.yml b/docker-compose.yml index 7726f1af..a1af38b3 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -28,6 +28,7 @@ services: HOOKRELAY_DB: /data/hookrelay.db HOOKRELAY_CONFIG: /etc/hookrelay/config.yaml HOOKRELAY_WORKER_INTERVAL: "0.5" + HOOKRELAY_SIGNATURE_REPLAY_SECONDS: ${HOOKRELAY_SIGNATURE_REPLAY_SECONDS:-300} # closes the captured-request replay window (0 = bounded only) # The shipped example plugins, mounted read-only below: the DingTalk # dialect the stack config uses is one of them (chat_markdown_channels.py). HOOKRELAY_PLUGINS: /etc/hookrelay/plugins diff --git a/docs/blog/proving-the-boundaries-run.md b/docs/blog/proving-the-boundaries-run.md new file mode 100644 index 00000000..0f417503 --- /dev/null +++ b/docs/blog/proving-the-boundaries-run.md @@ -0,0 +1,60 @@ +# DRAFT — Proving the boundaries actually run + +*An outline, not a finished post. The repository already holds the material +(`.agents/notes/`, `docs/containment.md`, `scripts/assert_guards_are_tested.py`); +this is the shape a public write-up would take, and the reason it is worth +writing: it is the one thing here that is hard to copy.* + +## The thesis + +Most security documentation describes a boundary. This one makes a second, +uncomfortable claim: **that the thing enforcing the boundary is still running**. +The two are different, and the gap between them is where incidents live. A regex +over model output *described* in a README and a regex *executing* on every tool +call produce identical documentation and different worlds. + +## The failure this is about + +A guard that has quietly stopped catching anything looks exactly like a guard with +nothing to catch. State it once, early; everything else is an instance of it. + +## Three instances, from this codebase + +1. **The read-only bash guard skipped a whole class of commands.** The AWS rule + was written as a list of mutating verbs; the guard matched `aws` inside a + dotted hostname, so a documentation URL was refused and a real mutation was + missed. A unit test proved the *shape* check refused a string a developer + typed — which is not the same as proving a real model cannot be steered. + +2. **A tool allowlist that said nothing about sockets.** An agent's MCP allowlist + is enforced where the agent asks for a *tool*. Measured: one plain HTTP POST + from inside a probe reached the chat client's MCP port and called a tool the + list did not name. The fix was not a better list; it was a network with no + route and a gate in another container. Same lesson at a different layer. + +3. **A posture check that had been silent for ten days.** The read-only claim is + measured against a kubeconfig at startup. When `kubectl` was missing from one + image, the measurement degraded to `unverifiable` — and `unverifiable` read + like `recorded`, so ten days passed with every gate green and nothing measured. + +## The four mechanisms, and what each is for + +| mechanism | catches | does not catch | +| --- | --- | --- | +| `assert_guards_are_tested.py` | a boundary whose tests are decorative — it breaks each claim and requires a test to fail | a boundary with no test yet | +| the inverted checker (`assert_watch_signed`) | a checker that has stopped catching anything | a checker pointed at the wrong thing | +| `gen_reference.py` | a knob documented but not generated from code | prose about behaviour | +| `docs/containment.md`'s four lists | a boundary whose limits are unwritten | an operator who overrides a limit on purpose | + +## The honest limit + +None of this is containment against a genuinely adversarial model — the +read-only credential is, and the page says so in those words. Heuristics catch +the over-eager model, which is the case that happens. Say that plainly; a security +post that oversells loses the reader it was written for. + +## Closing + +The ask is small and specific: for every guarantee you write down, name the thing +that would tell you it stopped. If there is no such thing, you have a preference, +not a boundary. diff --git a/docs/containment.md b/docs/containment.md index e721ae25..ac3f78b4 100644 --- a/docs/containment.md +++ b/docs/containment.md @@ -15,7 +15,8 @@ a real model cannot be steered into emitting a harmful one. > **Measured, not only declared (2026-09-07).** The read-only boundary is the one boundary whose truth lives outside this > repository — in a kubeconfig and an AWS profile an operator mounted. Every runner now checks them at startup > (`hookprobe/posture.py`): what `kubectl auth can-i` and `iam simulate-principal-policy` say the identity may do, against -> the posture it declares. Declared `readonly` with credentials that can mutate refuses to start (`HOOKPROBE_POSTURE_CHECK=enforce`); +> the posture it declares. Declared `readonly` with credentials that can mutate refuses to start — `HOOKPROBE_POSTURE_CHECK` +> defaults to `enforce` in the code, so the deploy composes now spell it out rather than leaving the reader to find it; > a `danger-only` runner records its blast radius; an identity that cannot be asked is reported *unverifiable*, never confirmed. > A credential whose measuring tool is not in the image is *unmeasurable* (2026-10-12): a readonly runner refuses to start > under `enforce`, a writing runner starts and pages, and the hourly selftest fails until the tool is back — ten days of @@ -111,8 +112,9 @@ design and what is the design working as documented. **Outside the supported posture:** - A node declaring `readonly` with wider credentials and `HOOKPROBE_POSTURE_CHECK=off`. -- The bridge on a shared container network with `BRIDGE_INBOUND_SECRET` empty — - the shipped default, and why both deployments set it. +- The bridge on a shared container network with `BRIDGE_INBOUND_SECRET` empty AND + `BRIDGE_ALLOW_UNSIGNED=1` set — no longer the shipped default (2026-10-12), and + now something an operator has to type on purpose. - Treating anything in the second list as the thing that holds when the input surface is untrusted. @@ -123,7 +125,7 @@ Every row here is a claim, and a claim is worth what its test is worth. `scripts | Boundary | Stops | Does NOT stop | Enforced in | |---|---|---|---| -| **Signatures at every door** — every pipe door verifies `sha256(secret, "{ts}.{body}")`; a card's button carries a signed single-use token; the bridge accepts a card only with `X-Hook-Timestamp` + `X-Hook-Signature` over the exact bytes, within five minutes | An unsigned or forged event reaching a door; anyone in a group chat pressing on your behalf; a neighbour on the shared container network posting a card into the operator's chat AS the application — the bridge's port has no other guard | A replay inside the window (`max_skew_seconds`, 300; five minutes at the bridge): **limited, not closed**, and there is no nonce cache. Nothing when a secret is empty: with `HOOKRELAY_ACTION_SECRET` unset the buttons render and do nothing, and with `BRIDGE_INBOUND_SECRET` unset — the shipped default — the bridge's port is open, which is why both deployments set it | `hookrelay/security.py`, `hookprobe/wire.py`, `hookrelay/actions.py`, `app.py`, `deploy/lark-bridge/bridge.py` (`is_authentic_protocol`) | +| **Signatures at every door** — every pipe door verifies `sha256(secret, "{ts}.{body}")`; a card's button carries a signed single-use token; the bridge accepts a card only with `X-Hook-Timestamp` + `X-Hook-Signature` over the exact bytes, within five minutes | An unsigned or forged event reaching a door; anyone in a group chat pressing on your behalf; a neighbour on the shared container network posting a card into the operator's chat AS the application — the bridge's port has no other guard, and since 2026-10-12 it refuses to start without a secret | A replay inside the skew, which the pipe CLOSES and the bridge only BOUNDS: the pipe claims each signature once (`HOOKRELAY_SIGNATURE_REPLAY_SECONDS`, 300; 0 leaves the window bounded) and answers the second copy as a duplicate, while the bridge's five-minute window is still bounded, not closed, and its legacy in-body sign covers only the timestamp. Nothing when a secret is empty: with `HOOKRELAY_ACTION_SECRET` unset the buttons render and do nothing, and an unset `BRIDGE_INBOUND_SECRET` is now a refusal to start (`BRIDGE_ALLOW_UNSIGNED=1` is the loud opt-out) | `hookrelay/security.py` (`replay_token`), `hookrelay/store.py` (`claim_signature`), `hookprobe/wire.py`, `hookrelay/actions.py`, `app.py`, `deploy/lark-bridge/bridge.py` (`is_authentic_protocol`) | | **Per-node credentials, and the agent's bearer reads only** — each process-external node gets its own secrets, budget and guards; inside a node the console's `HOOKPROBE_TOKEN` is blanked out of the agent's subprocess and replaced with `HOOKPROBE_AGENT_TOKEN`, which the API refuses on every method but `GET` | One node's compromise becoming every node's; an injected instruction reaching a Bash step and using the service's own write surface: `PUT /v1/memory` (no shape check, unlike the suggestion path), `PUT /v1/skills`, `/v1/system-prompt`, approving a remediation, and `POST /hooks/agent`, which spends money. The regret route *claimed* this for its whole life while the token was inherited | Anything inside a node once it is compromised; everything a `GET` exposes, which is most of a node's own state; and anything the agent can do without this API, which is the whole shell. The real boundary is the credential each node is given, and that lives outside this repository | the compose files; `hookprobe/settings.py` (`agent_token`), `app.py` (`require_token`), `gate.py` (`agent_credentials`) | | **Closed sets at the doors** — config names the channel set a payload may pick from (the bypass lanes); `HOOKPROBE_VERDICTS` and `HOOKPROBE_MCP_TOOLS` are closed sets, and empty denies all | An injected message routing itself around a gatekeeper; an investigator's free text choosing which lane spends money; a chat message asking the agent to use a write tool on a mounted server | A wrong choice among lanes the operator already declared. **A socket.** The MCP allowlist is enforced where the agent asks for a TOOL and says nothing about a request the agent makes itself: measured on 2026-09-30, one plain HTTP POST from inside the watcher reached the chat client's own MCP port, with no credential, and called a tool this list does not name. The row below is the answer; this one alone was advice | `hookrelay/config.py`, `routing.py`; `hookprobe/reports.py` (`verdict`), `engine.py` (`mcp_deny_reason`) | | **The chat's tools are on a list in another container** — the probes sit on a docker network with `internal: true`, so they have no route to the host or the internet at all; `mcp-gate` holds the only reach to the chat client and forwards a `tools/call` only when the tool's name is on the calling node's list, recording both halves. Several MCP servers can share its one port as routes: each keeps its own credential, which never reaches another, and a node's list names tools per route | An agent that goes around its own MCP allowlist with an HTTP request, which is what the row above could not stop. The writes are not refused, they are unreachable: the watcher sees 8 read tools where the server offers 34, 20 of them writes | Anything a PERMITTED read does: an argument is not checked, and what a read returns still goes to the model provider. The server-to-client stream is declined rather than relayed (405), so a chat server that pushed notifications would not reach the agent through this. A client token lives in the agent's own MCP config by construction — it admits exactly the tools that agent already had, which is the point of the list being here rather than there | `deploy/sidecars/gate.py`, `deploy/docker-compose.work.yml` (`probe_net`) | diff --git a/docs/evidence.md b/docs/evidence.md new file mode 100644 index 00000000..d9f38d5f --- /dev/null +++ b/docs/evidence.md @@ -0,0 +1,47 @@ +# Evidence + +Every number the README leans on, and whether a reader can reproduce it here. + +The repository ran two deployments, both of somebody's real infrastructure. The +identifiers are scrubbed on purpose (`scripts/assert_no_estate_identifiers.py`), +and the ledgers those numbers were measured from are not in the tree — they are +runtime data, and `dataset.jsonl` is gitignored. So the honest split is not +"verified / unverified" but **two classes**: + +- **Reproducible here** — a command in this repository produces the number, and + `scripts/reproduce_claims.py` runs it. +- **Reported** — the number was measured on the retired deployments and the + source data is not in this repository. The row names the artefact that WOULD + back it, so a reader knows exactly what to ask for and what to distrust. + +Run `python3 scripts/reproduce_claims.py` for the machine-readable version of +this table. It fails on a *reproducible* claim whose command has stopped +agreeing; it never fails on a *reported* one, because there is nothing here to +run. + +| claim | stated in | artefact that backs it | class | +| --- | --- | --- | --- | +| hookrelay stays at or under 6,400 source lines | `hookrelay/README.md` | `scripts/assert_weight.py` | Reproducible | +| hookjudge stays at or under 3,400 source lines | `hookjudge/README.md` | `scripts/assert_weight.py` | Reproducible | +| every route, knob and containment row is written up or named as deliberately not | `docs/containment.md`, `*/docs/reference.md` | `scripts/assert_docs.py` | Reproducible | +| the three boards share one design language | the service pages | `scripts/assert_design.py` | Reproducible | +| a replayed signed request cannot buy a second verdict | `docs/containment.md` | `hookrelay/tests/test_signature_replay.py` | Reproducible | +| an empty `BRIDGE_INBOUND_SECRET` refuses to start | `docs/containment.md` | `deploy/lark-bridge/tests/test_fail_closed.py` | Reproducible | +| **795 production alerts; 28 of 29 rules answered identically** | `README.md`, `hookjudge/eval/README.md` | the retired deployment's `webhook_events` table — `hookjudge/eval/dataset.jsonl`, gitignored | Reported | +| **the fold sends 55% fewer cards than the rule as shipped** | `docs/deployments.md` | a replay against the retired judge ledger (`hookjudge/scripts/replay_ledger.py`), ledger not in tree | Reported | +| **the first real write happened 2026-09-30** | `README.md`, `docs/deployments.md#the-first-real-write` | the work deployment's run records and audit log | Reported | +| **`kubectl` was absent from the work image for ten days** | `docs/containment.md`, `.agents/notes/` | the release job history and the two nodes' posture records | Reported | +| **one HTTP POST from inside a probe reached the chat MCP port** | `deploy/sidecars/gate.py` docstring | the measurement on the work deployment, 2026-09-30 | Reported | + +## Why the split is the point + +A project whose most persuasive numbers all come from a private deployment has +two ways to be honest about it: say nothing, or say which ones a reader cannot +check. This page is the second. The alternative — presenting every number as if +it were in the tree — is the failure `docs/containment.md` names when it says a +boundary trusted for something it never claimed is the way a unit test gets read +as proof about a live model. + +If you are evaluating this for real, the Reproducible rows are what you can hold +it to today. For the Reported ones, the ask is concrete: the ledger and the run +records exist, off this host, and a maintainer can export them. diff --git a/docs/index.md b/docs/index.md index 6f6ee74a..02488fc9 100644 --- a/docs/index.md +++ b/docs/index.md @@ -17,38 +17,16 @@ A signal comes in: an alert, a chat message, a ticket, a timer. A pipe signs, ro The investigator is the product. The pipe and the judge are the smallest alerting front end for a team that has none. One deployment is one team: this is not a work-management platform, a multi-tenant SaaS or an agent framework. -![hookprobe's console: a real investigation, conclusion first, the root cause it could not establish said so, and the bill under it](img/hookprobe-sessions.png) - -## Try it: ten minutes, no keys, no bill - -```bash -curl -fsSLO https://raw.githubusercontent.com/itswl/hookstack/main/docker-compose.quickstart.yml -docker compose -f docker-compose.quickstart.yml up -d # pipe + judge + stub model + investigator on a rehearsal + readable sink -bash <(curl -fsSL https://raw.githubusercontent.com/itswl/hookstack/main/scripts/demo.sh) -``` - -The demo runs the whole loop. An alert is judged, a recorded investigation replays through the real read-only gate, the report arrives as a card, an approve press runs two allowlisted commands, and the alert resolves. To use a real model, put `HOOKPROBE_RUNTIME=claude`, `HOOKPROBE_MODEL` and a key in `.env`. +## Start here -## What you get +- **30 seconds:** [docs/quickstart.md](quickstart.md) opens with the shape of it — three services, one diagram. +- **10 minutes, no keys, no bill:** the same page's second section runs the whole loop on a rehearsal: an alert judged, an investigation replayed through the real read-only gate, a card, an approve press, a resolved alert. +- **Half a day, for real:** [STACK.md](https://github.com/itswl/hookstack/blob/main/STACK.md) runs all three from source; [docs/containment.md](containment.md) is the boundary map you read *before* pointing it at a cluster. +- **What it does, in full:** [docs/what-you-get.md](what-you-get.md). -- **Only the work worth doing.** A storm, a restatement or a recovery never buys a second verdict. On 795 production alerts, 28 of 29 rules answered the same every time, so `rule-reuse` answers them for free. -- **A card a person can rule on.** Every button is signed before the card leaves, and every press is recorded. -- **Remediation with a person in the loop.** An approved procedure runs step by step against an allowlist, as argv, never through a shell. A press approves the version that was read: the card and the console both name the procedure's digest, and one that changed since is refused. A plan a person hands off is carried out by a separate work node, with a write credential of its own and a guard that refuses destructive commands; it made its first real change on 2026-09-30 ([how it went](deployments.md#the-first-real-write)). -- **Investigations that leave something behind.** A finished run distills a runbook, and the next occurrence starts from it. -- **Agents you can contain.** Read-only by default and measured at startup, budget ceilings that refuse out loud, and the structural boundaries, each written up with what it does **not** stop ([containment](containment.md)). -- **Boards that install on your phone.** All three boards are web apps you add to the home screen, the investigator's under its node's own name; each keeps its tokens in that browser and nothing else, and offline it shows the page and says its service is out of reach. On a phone the header runs under the status bar in its own colour and follows the theme, the tabs fit one row, and wide tables become one block per row. -- **One audit page per event.** Every hop, digest, decision and human action, a flight recorder of every tool call, and a timing waterfall per run. -- **Your model, your chat.** Any Anthropic-dialect endpoint for the investigator, any OpenAI-compatible one for the judge, local models included. Feishu/Lark through a bridge ([protocol](bridge-protocol.md)), DingTalk and WeCom as a plugin, or signed JSON to any webhook. - -![hookrelay's board: one sentence on what waits on you, four numbers, and one row per alert with its seven stages](img/hookrelay-timeline.png) - -![hookjudge's board: verdicts with their routes and what each cost](img/hookjudge-status.png) - -![hookrelay's ledger: every event with its decision chain, its deliveries, and what came back](img/hookrelay-ledger.png) - -![The three boards on a phone: the pipe's attention entry, the investigator's procedure executed after a named person approved it, and the judge's board](img/hookstack-phones.png) +![hookprobe's console: a real investigation, conclusion first, the root cause it could not establish said so, and the bill under it](img/hookprobe-sessions.png) -The three boards read in Chinese or English and follow the system's light or dark. The screenshots come from local runs, not mockups. +The three boards read in Chinese or English and follow the system's light or dark; [the full tour](https://github.com/itswl/hookstack/blob/main/OVERVIEW.md) has every screenshot, from local runs and not mockups. ## How it fits together diff --git a/docs/quickstart.md b/docs/quickstart.md new file mode 100644 index 00000000..20dc20b9 --- /dev/null +++ b/docs/quickstart.md @@ -0,0 +1,84 @@ +# Start here + +Three doors, depending on how much time you have. None of them needs an account +or a key, and the first two do not need a checkout. + +## Thirty seconds — what it is + +A signal arrives — an alert, a chat message, a ticket, a timer. Three small +services handle it: + +``` +upstreams ──► hookrelay ──► hookjudge ──► hookrelay ──► chat bridge / webhook + │ + └──► hookprobe ──► hookrelay ──► the same channels + (critical/high; a rehearsal until it has a model) +``` + +- **hookrelay** is the pipe. It signs, routes and prices every signal and keeps + one ledger of every hop. +- **hookjudge** decides cheaply whether a person must act now. Most events never + pay for a model. +- **hookprobe** is the investigator. One read-only agent run per event, and it + proves it is read-only at startup. The report reaches a person as a card. + +Nothing touches production without a signed press. Read +[the overview](../OVERVIEW.md) for the tour, or +[what you get](what-you-get.md) for the feature list. + +## Ten minutes — the whole loop, no keys, no bill + +This runs the pipe, the judge, a canned model, the investigator on a recorded +rehearsal, and a sink you can read. Docker is the only prerequisite. + +```bash +curl -fsSLO https://raw.githubusercontent.com/itswl/hookstack/main/docker-compose.quickstart.yml +docker compose -f docker-compose.quickstart.yml up -d +bash <(curl -fsSL https://raw.githubusercontent.com/itswl/hookstack/main/scripts/demo.sh) +``` + +What you should see: an alert is judged, a recorded investigation replays through +the **real** read-only gate and the real audit log, the report arrives as a card, +an approve press runs two allowlisted commands, and the alert resolves. The +demo's own last section says every number in a rehearsal was written in advance. + +To investigate with a real model instead of the rehearsal, hand the node a model +and restart: + +```bash +printf 'HOOKPROBE_RUNTIME=claude\nHOOKPROBE_MODEL=claude-opus-5\nANTHROPIC_API_KEY=sk-ant-...\n' > .env +docker compose --env-file .env -f docker-compose.quickstart.yml up -d +``` + +Then open the three boards: the pipe at , the judge at +, the investigator's console at . + +## Half a day — run it for real + +The quickstart is a demo on published images. A real deployment is the same code +with your own credentials, a chat channel and an allowlist of fixes a person may +approve. That path, step by step: + +1. [STACK.md](../STACK.md) — run all three from source, in order, and watch the + gate green as you go. +2. [Two deployments, one codebase](deployments.md) — the two deployment shapes + (alerts, and an operator's own work signals), the compose files under + [`deploy/`](../deploy), and what each one needs in `.env`. +3. [docs/containment.md](containment.md) — **read this before you point it at a + real cluster.** It is the map of which boundaries hold against a hostile + model, which only reduce accidents, and what each one does not stop. The + read-only claim is measured against your kubeconfig and AWS profile at + startup. +4. The service in depth, when you are ready to change it: + [hookrelay](../hookrelay/README.md) · + [hookjudge](../hookjudge/README.md) · + [hookprobe](../hookprobe/README.md). + +## If something looks wrong + +- The gate: `bash scripts/gate.sh` runs every service's gate plus the stack + checks. Run it as its own step and read the verdict — see + [CONTRIBUTING.md](../CONTRIBUTING.md). +- The boards need a token in the browser and keep it there; offline they show + the page and say their service is out of reach. +- Containment question, or a boundary that does not hold: [SECURITY.md](../SECURITY.md). diff --git a/docs/releasing.md b/docs/releasing.md new file mode 100644 index 00000000..16e596ac --- /dev/null +++ b/docs/releasing.md @@ -0,0 +1,57 @@ +# Releasing + +A release is one tag and four images. There is no package to publish and no +registry but the container one. + +## What a release is + +| artefact | where | named by | +| --- | --- | --- | +| `hookrelay`, `hookjudge`, `hookprobe`, `lark-bridge` images | `ghcr.io/itswl/` | `` tag | +| the quickstart | `docker-compose.quickstart.yml` | the same tags, **digest-pinned** | +| the notes | `.agents/notes/implemented/` | the git log between two tags | + +The release notes are not a separate document. They are the implemented decision +records that landed since the last tag — that is what the bucket is for, and a +second place to write them is the duplicate-source mistake this repository keeps +finding its way back to. + +## The steps + +1. **The gate, read, not chained.** `bash scripts/gate.sh` as its own step; read + the verdict. Never `gate.sh && docker push` — an exit code is not a report + (`AGENTS.md` has the incident that taught this). +2. **Tag.** `git tag -a vX.Y.Z -m "…" && git push origin vX.Y.Z`. The release + workflow builds and pushes the four images at that tag. +3. **Pin the quickstart to the digest, not the tag alone.** A tag can be moved; a + digest cannot. For each image: + + ```bash + tok=$(curl -fsS "https://ghcr.io/token?scope=repository:itswl/hookrelay:pull&service=ghcr.io" \ + | python3 -c "import sys,json;print(json.load(sys.stdin)['token'])") + curl -fsSI -H "Authorization: Bearer $tok" \ + -H "Accept: application/vnd.oci.image.index.v1+json,application/vnd.docker.distribution.manifest.list.v2+json" \ + "https://ghcr.io/v2/itswl/hookrelay/manifests/X.Y.Z" | tr -d '\r' | grep -i docker-content-digest + ``` + + Write the result as `ghcr.io/itswl/hookrelay:X.Y.Z@sha256:…` in + `docker-compose.quickstart.yml`. This is why the file cannot point at an older + tag: it needs whichever release carries everything the demo shows. +4. **Re-run the quickstart the way a stranger does** — `curl` the published + compose, `docker compose up`, `bash scripts/demo.sh`. The file's own comment + records three releases that were wrong until somebody actually ran it. +5. **A GitHub Release** whose body is the notes since the last tag, linking the + tag. The images already exist; this is the human-facing index of them. + +## A note on tags vs. digests + +The tag is for humans ("0.6.0"), the digest is for the container runtime ("the +bytes we tested"). Pinning both means `docker compose up` is reproducible *and* +readable. Tests may reference the tag; the shipped quickstart references the +digest. + +## Rollback + +Images are immutable, so a rollback is re-pinning the quickstart to the previous +release's digest and re-running it. The ledger drives the behaviour, so state is +unchanged by a rollback; only the code moves. diff --git a/docs/what-you-get.md b/docs/what-you-get.md new file mode 100644 index 00000000..8df48521 --- /dev/null +++ b/docs/what-you-get.md @@ -0,0 +1,41 @@ +# What you get + +The feature list long enough to live off the front page. Each line names the +thing it is, and the page that shows it working. + +- **Only the work worth doing.** A storm, a restatement or a recovery never buys + a second verdict. On 795 production alerts, 28 of 29 rules answered the same + every time, so `rule-reuse` answers them for free + ([the measurement](../hookjudge/eval/README.md), [the judge](../hookjudge/README.md)). +- **A card a person can rule on.** Every button is signed before the card leaves, + and every press is recorded ([the token](../hookrelay/hookrelay/actions.py)). +- **Remediation with a person in the loop.** An approved procedure runs step by + step against an allowlist, as argv, never through a shell. A press approves the + version that was read: the card and the console both name the procedure's + digest, and one that changed since is refused. A plan a person hands off is + carried out by a separate work node, with a write credential of its own and a + guard that refuses destructive commands; it made its first real change on + 2026-09-30 ([how it went](deployments.md#the-first-real-write)). +- **Investigations that leave something behind.** A finished run distills a + runbook, and the next occurrence starts from it + ([learning](../hookprobe/docs/learning.md)). +- **Agents you can contain.** Read-only by default and measured at startup, + budget ceilings that refuse out loud, and the structural boundaries, each + written up with what it does **not** stop ([containment](containment.md)). +- **Boards that install on your phone.** All three boards are web apps you add to + the home screen, the investigator's under its node's own name; each keeps its + tokens in that browser and nothing else, and offline it shows the page and says + its service is out of reach. On a phone the header runs under the status bar in + its own colour and follows the theme, the tabs fit one row, and wide tables + become one block per row. +- **One audit page per event.** Every hop, digest, decision and human action, a + flight recorder of every tool call, and a timing waterfall per run + ([the pipe's audit](../hookrelay/README.md), [the investigator's](../hookprobe/README.md)). +- **Your model, your chat.** Any Anthropic-dialect endpoint for the investigator, + any OpenAI-compatible one for the judge, local models included. Feishu/Lark + through a bridge ([protocol](bridge-protocol.md)), DingTalk and WeCom as a + plugin, or signed JSON to any webhook. + +The three boards read in Chinese or English and follow the system's light or +dark. The screenshots in the [overview](../OVERVIEW.md) come from local runs, not +mockups. diff --git a/docs/zh/index.md b/docs/zh/index.md index 30ca8697..da76c5f7 100644 --- a/docs/zh/index.md +++ b/docs/zh/index.md @@ -17,38 +17,16 @@ description: 让 agent 值班，但不把钥匙交出去 —— 管道给每条 调查员是产品本身，管道和判官是给没有告警平台的团队准备的最小前端。一套部署就是一个团队：它不是工作管理平台，不是多租户 SaaS，也不是 agent 框架。 -![hookprobe 的控制台：一次真实的调查，结论在最前面，查不到的根因明说查不到，下面是这次的花费](../img/zh/hookprobe-sessions.png) - -## 试一下：十分钟，不要 key，不花钱 - -```bash -curl -fsSLO https://raw.githubusercontent.com/itswl/hookstack/main/docker-compose.quickstart.yml -docker compose -f docker-compose.quickstart.yml up -d # 管道 + 判官 + 桩模型 + 跑排练的调查员 + 可读的接收端 -bash <(curl -fsSL https://raw.githubusercontent.com/itswl/hookstack/main/scripts/demo.sh) -``` - -演示会跑完整条闭环：告警被判定，一次录好的调查经过真实的只读门回放，报告变成卡片，按下批准后跑两条允许清单里的命令，告警恢复。要换成真实模型，在 `.env` 里设置 `HOOKPROBE_RUNTIME=claude`、`HOOKPROBE_MODEL` 和模型 key。 +## 从这里开始 -## 你能得到什么 +- **30 秒：**[docs/quickstart.md](../quickstart.md) 开头就讲清了它的形状 —— 三个服务，一张图。 +- **10 分钟，不要 key，不花钱：** 同一页的第二段会在排练上跑完整条闭环：告警被判定，调查经过真实的只读门回放，卡片，按下批准，告警恢复。 +- **半天，真上：**[STACK.md](https://github.com/itswl/hookstack/blob/main/STACK.md) 从源码一步步跑起三个服务；[docs/containment.md](../containment.md) 是指向真实集群**之前**要读的边界地图。 +- **完整功能清单：**[docs/what-you-get.md](../what-you-get.md)（英文）。 -- **只做值得做的事。** 告警风暴、重复的同一条件、恢复事件都不会再花第二次判定的钱。在 795 条生产告警上实测，29 条规则里有 28 条每次答案都一样，所以 `rule-reuse` 不调模型就能回答。 -- **一张人可以裁定的卡片。** 每个按钮在卡片发出前就签了名，每次按下都有记录。 -- **修复有人把关。** 批准后的流程逐步对照允许清单执行，以 argv 运行，从不经过 shell。批准的是读到的那个版本：卡片和控制台都会带上流程的摘要，之后变过的流程会被拒绝。人交接的计划由单独的执行节点用它自己的写凭证去做，危险命令会被拒绝；2026-09-30 它第一次真实改动了系统（[经过](../deployments.md#the-first-real-write)）。 -- **调查会留下东西。** 跑完的调查会提炼成手册，同一条件下次出现时从手册开始。 -- **关得住的 agent。** 默认只读并在启动时实测，预算用完会明确拒绝，另有一组结构性边界，每一条都写明它**挡不住**什么（[containment](../containment.md)）。 -- **装到手机上的看板。** 三个看板都是可以添加到主屏幕的 web 应用，调查员的以节点自己的名字出现；口令只留在那个浏览器里，离线时显示页面并告诉你连不上服务。在手机上，页头延伸到状态栏下方、颜色随主题切换，页签一行放下，宽表格按行变成小块。 -- **每个事件一页审计。** 每一跳、摘要、决策和人的操作，每次工具调用的飞行记录，以及每次运行的耗时瀑布图。 -- **模型和聊天工具由你选。** 调查员接任意 Anthropic 方言的端点，判官接任意 OpenAI 兼容的端点，本地模型也可以。飞书经桥接送达（[协议](../bridge-protocol.md)），钉钉和企微有插件，也可以把签名 JSON 发到任意 webhook。 - -![hookrelay 的看板：一句话说有几件事等你处理，四个数字，每条告警一行，各带七个阶段](../img/zh/hookrelay-timeline.png) - -![hookjudge 的看板：每条判定走了哪条路径、花了多少](../img/zh/hookjudge-status.png) - -![hookrelay 的流水：每个事件的决策、投递去向和回流](../img/zh/hookrelay-ledger.png) - -![三块看板在手机上：管道的待处理入口，调查员那条经有名字的人批准后执行的处置流程，以及判官的看板](../img/zh/hookstack-phones.png) +![hookprobe 的控制台：一次真实的调查，结论在最前面，查不到的根因明说查不到，下面是这次的花费](../img/zh/hookprobe-sessions.png) -三块看板都能切换中英文，并跟随系统的浅色或深色。截图取自本地真实运行，不是效果图。 +三块看板都能切换中英文，并跟随系统的浅色或深色；[完整介绍](https://github.com/itswl/hookstack/blob/main/OVERVIEW.md)（英文）里有全部截图，取自本地真实运行，不是效果图。 ## 怎么串起来 diff --git a/hookjudge/requirements.lock b/hookjudge/requirements.lock index f49f7a3a..db89f966 100644 --- a/hookjudge/requirements.lock +++ b/hookjudge/requirements.lock @@ -9,6 +9,7 @@ CacheControl==0.14.4 certifi==2026.7.22 charset-normalizer==3.5.1 click==8.4.2 +coverage==7.16.2 cyclonedx-python-lib==11.12.0 defusedxml==0.7.1 fastapi==0.142.2 @@ -41,6 +42,7 @@ pydantic_core==2.46.4 Pygments==2.21.0 pyparsing==3.3.2 pytest==9.1.1 +pytest-cov==7.1.0 pytest-asyncio==1.4.0 python-dotenv==1.2.3 PyYAML==6.0.3 diff --git a/hookjudge/requirements.txt b/hookjudge/requirements.txt index 2a1e466e..97befd2c 100644 --- a/hookjudge/requirements.txt +++ b/hookjudge/requirements.txt @@ -7,6 +7,7 @@ httpx>=0.28.1 # Test + lint pytest>=9.1.1 pytest-asyncio>=1.4.0 +pytest-cov>=7.0 ruff==0.16.10 bandit>=1.9.4 pip-audit>=2.10.1 diff --git a/hookjudge/scripts/gate.sh b/hookjudge/scripts/gate.sh index f72a62ba..cc60ec06 100755 --- a/hookjudge/scripts/gate.sh +++ b/hookjudge/scripts/gate.sh @@ -45,8 +45,12 @@ JSON.parse(fs.readFileSync("hookjudge/static/manifest.webmanifest", "utf8")); console.log("status.html inline JS, sw.js and the manifest: OK"); ' -step "pytest" -$PY -m pytest -q +step "pytest (with a coverage floor)" +# The floor is a REGRESSION catch, not a target: set just below what the suite +# covers today, so a change that deletes tests or leaves a branch unreached fails +# the gate instead of quietly lowering the number. hookjudge measured 92% when +# this step was added; `--cov-fail-under` is the part that must not move. +$PY -m pytest -q --cov=hookjudge --cov-fail-under=90 step "pip-audit (known-vulnerable dependencies)" $PY -m pip_audit --progress-spinner off diff --git a/hookjudge/tests/test_app.py b/hookjudge/tests/test_app.py index fd1351d2..6b284314 100644 --- a/hookjudge/tests/test_app.py +++ b/hookjudge/tests/test_app.py @@ -352,7 +352,7 @@ def runs(text: str, command: str) -> bool: "ruff format --check hookjudge tests scripts", "mypy hookjudge", "bandit -q -r hookjudge", - "pytest -q", + "pytest -q --cov=hookjudge --cov-fail-under=90", "pip_audit --progress-spinner off", ): assert runs(gate, command), f"gate.sh does not run {command!r}" diff --git a/hookprobe/deploy/docker-compose.prod.yml b/hookprobe/deploy/docker-compose.prod.yml index 74f838ea..93eca401 100644 --- a/hookprobe/deploy/docker-compose.prod.yml +++ b/hookprobe/deploy/docker-compose.prod.yml @@ -314,7 +314,9 @@ services: HOOKPROBE_MAX_TIMEOUT_SECONDS: ${HOOKPROBE_MAX_TIMEOUT_SECONDS:-} HOOKPROBE_MAX_TURNS: ${HOOKPROBE_MAX_TURNS:-} HOOKPROBE_MEMORY_AUTO_APPLY: ${HOOKPROBE_MEMORY_AUTO_APPLY:-} - HOOKPROBE_POSTURE_CHECK: ${HOOKPROBE_POSTURE_CHECK:-} + # `enforce` is the code's default; spelled out here so the deploy states + # the intent the containment page describes. `off` is the recorded opt-out. + HOOKPROBE_POSTURE_CHECK: ${HOOKPROBE_POSTURE_CHECK:-enforce} HOOKPROBE_REMEDIATION_COOLDOWN_SECONDS: ${HOOKPROBE_REMEDIATION_COOLDOWN_SECONDS:-} # How long an executed procedure waits for the condition to answer before # the board calls it held; exit 0 alone verifies nothing. 0 turns it off. diff --git a/hookprobe/requirements.lock b/hookprobe/requirements.lock index 91c84482..a9525a5f 100644 --- a/hookprobe/requirements.lock +++ b/hookprobe/requirements.lock @@ -11,6 +11,7 @@ cffi==2.1.1 charset-normalizer==3.5.1 claude-agent-sdk==0.2.140 click==8.4.2 +coverage==7.16.2 cryptography==50.0.0 cyclonedx-python-lib==11.12.0 defusedxml==0.7.1 @@ -52,6 +53,7 @@ Pygments==2.21.0 PyJWT==2.15.1 pyparsing==3.3.2 pytest==9.1.1 +pytest-cov==7.1.0 python-dotenv==1.2.3 python-multipart==0.0.32 PyYAML==6.0.3 diff --git a/hookprobe/requirements.txt b/hookprobe/requirements.txt index b91340ff..71d0e76b 100644 --- a/hookprobe/requirements.txt +++ b/hookprobe/requirements.txt @@ -8,6 +8,7 @@ claude-agent-sdk>=0.1.0 # Test + lint (tests never import the SDK; they inject fake engines) pytest>=9.1.1 httpx>=0.28.1 +pytest-cov>=7.0 ruff==0.16.10 bandit>=1.9.4 pip-audit>=2.10.1 diff --git a/hookprobe/scripts/gate.sh b/hookprobe/scripts/gate.sh index 8e4d0f8b..fde392ef 100755 --- a/hookprobe/scripts/gate.sh +++ b/hookprobe/scripts/gate.sh @@ -53,8 +53,12 @@ step "shell scripts parse" find examples scripts -name '*.sh' -print0 | xargs -0 -r -n1 bash -n echo "shell: OK" -step "pytest" -$PY -m pytest -q +step "pytest (with a coverage floor)" +# The floor is a REGRESSION catch, not a target: set just below what the suite +# covers today, so a change that deletes tests or leaves a branch unreached fails +# the gate instead of quietly lowering the number. hookprobe measured 88% when +# this step was added; `--cov-fail-under` is the part that must not move. +$PY -m pytest -q --cov=hookprobe --cov-fail-under=85 step "pip-audit (known-vulnerable dependencies)" $PY -m pip_audit --progress-spinner off diff --git a/hookprobe/tests/test_ops.py b/hookprobe/tests/test_ops.py index 8729fc0b..20cbb6de 100644 --- a/hookprobe/tests/test_ops.py +++ b/hookprobe/tests/test_ops.py @@ -485,7 +485,7 @@ def runs(text: str, command: str) -> bool: "ruff format --check hookprobe tests", "mypy hookprobe", "bandit -q -r hookprobe", - "pytest -q", + "pytest -q --cov=hookprobe --cov-fail-under=85", "pip_audit --progress-spinner off", ): assert runs(gate, command), f"gate.sh does not run {command!r}" diff --git a/hookrelay/README.md b/hookrelay/README.md index 3686177b..a8f7e9f3 100644 --- a/hookrelay/README.md +++ b/hookrelay/README.md @@ -8,12 +8,12 @@ workflow are its own. Receive webhooks. Decide. Fan out to channels. Nothing else. -A pluggable router (under 6,300 source lines, five dependencies) that takes JSON +A pluggable router (under 6,400 source lines, five dependencies) that takes JSON webhooks in at one door, walks each event through three named gates, and delivers to a chat bridge / generic HTTP (DingTalk and WeCom as shipped plugins) — with retries, per-channel rate limits, and a dead-letter queue you can see. -Both numbers are **budgets, not descriptions**. 6,300 source lines is the +Both numbers are **budgets, not descriptions**. 6,400 source lines is the ceiling and five dependencies is the count; `scripts/assert_weight.py` enforces the first alongside the other stack checks, and crossing it is meant to cost a conversation rather than a commit. Tests are counted and printed but never capped diff --git a/hookrelay/deploy/docker-compose.yml b/hookrelay/deploy/docker-compose.yml index bd67da11..4f1dd700 100644 --- a/hookrelay/deploy/docker-compose.yml +++ b/hookrelay/deploy/docker-compose.yml @@ -30,6 +30,7 @@ services: HOOKRELAY_DB: /data/hookrelay.db HOOKRELAY_CONFIG: /etc/hookrelay/config.yaml HOOKRELAY_WORKER_INTERVAL: "0.5" + HOOKRELAY_SIGNATURE_REPLAY_SECONDS: ${HOOKRELAY_SIGNATURE_REPLAY_SECONDS:-300} # closes the captured-request replay window (0 = bounded only) HOOKRELAY_PLUGINS: /data/plugins volumes: - ${HOOKRELAY_CONFIG_FILE:?path to your config.yaml — sources, routes, channels}:/etc/hookrelay/config.yaml:ro diff --git a/hookrelay/docs/reference.md b/hookrelay/docs/reference.md index 6de237dc..48ed64b0 100644 --- a/hookrelay/docs/reference.md +++ b/hookrelay/docs/reference.md @@ -7,7 +7,7 @@ routes and their first sentence from the handlers. Correct it by editing the comment beside the field or the handler's docstring — an edit here is lost on the next run, and `scripts/gen_reference.py --check` will say so. -## Environment (20) +## Environment (21) | variable | default | what it is | | --- | --- | --- | @@ -17,6 +17,7 @@ the next run, and `scripts/gen_reference.py --check` will say so. | `HOOKRELAY_ADMIN_TOKEN` | *(empty)* | Required for silence management; empty disables those endpoints entirely (403), so an unconfigured instance cannot be muted by a stranger | | `HOOKRELAY_READ_TOKEN` | *(empty)* | Empty leaves /status open — single-operator dev mode | | `HOOKRELAY_MAX_BODY_BYTES` | `256 * 1024` | Inbound body cap | +| `HOOKRELAY_SIGNATURE_REPLAY_SECONDS` | `300` | How long an accepted inbound signature is remembered, so the same signed bytes replayed inside the timestamp's freshness window are answered as a duplicate instead of walking the p… | | `HOOKRELAY_MAX_ATTEMPTS` | `8` | Attempts per delivery before the row is dead-lettered | | `HOOKRELAY_RETENTION_DAYS` | `14` | How long delivered rows are kept; 0 disables purging entirely | | `HOOKRELAY_ALARM_URL` | *(empty)* | Self-alarm for dead letters (who watches the watchman) | diff --git a/hookrelay/hookrelay/app.py b/hookrelay/hookrelay/app.py index 2db4030c..4f044881 100644 --- a/hookrelay/hookrelay/app.py +++ b/hookrelay/hookrelay/app.py @@ -36,7 +36,7 @@ from hookrelay.fuse import StormFuse from hookrelay.live import Live from hookrelay.pipeline import handle_hook, record_storm_suppressed, settle_folds -from hookrelay.security import token_ok, verify_signature +from hookrelay.security import replay_token, token_ok, verify_signature from hookrelay.settings import Settings from hookrelay.store import Store, button_labels, now_ts from hookrelay.timeline import render as render_timeline @@ -422,6 +422,19 @@ async def hook(source_name: str, request: Request) -> JSONResponse: headers = {key.lower(): value for key, value in request.headers.items()} if not adapter.verify(source, body, headers): raise HTTPException(status_code=401, detail="bad signature") + # A captured request replayed inside the skew is the same signed BYTES, + # and the content fingerprint cannot see that: it folds a restatement of + # the same alert however long ago the content window closed. Claiming the + # signature here answers the second copy as a skip — BEFORE the fuse and + # the pipeline, so it cannot spend a second verdict. Off when the window + # is 0, and a scheme this cannot name (a plugin adapter's own header) + # yields no token, which is a refusal to guess rather than a miss. + if app_settings.signature_replay_seconds > 0: + token = replay_token(source.name, headers.get("x-hook-signature")) + if token is not None and not await app.state.store.claim_signature( + token, app_settings.signature_replay_seconds, now_ts() + ): + return JSONResponse({"outcome": "skipped", "skip_code": "replayed_signature"}) try: payload = json.loads(body or b"{}") except ValueError: diff --git a/hookrelay/hookrelay/security.py b/hookrelay/hookrelay/security.py index 805b4009..ab829dbf 100644 --- a/hookrelay/hookrelay/security.py +++ b/hookrelay/hookrelay/security.py @@ -3,8 +3,9 @@ Inbound: X-Hook-Signature = hex HMAC-SHA256 with the source's secret ("sha256=" prefix tolerated). With X-Hook-Timestamp the signature covers "{timestamp}.{body}" and must be FRESH, which BOUNDS how long a captured -request stays useful rather than preventing replay outright (see -verify_signature); without it the legacy body-only form is +request stays useful rather than preventing replay outright; the door pairs it +with `replay_token` + `Store.claim_signature` to close that window (see +verify_signature). Without a timestamp the legacy body-only form is accepted unless the door sets require_timestamp. A source configured without a secret is accepted unsigned — a deliberate, documented decision for trusted private networks, not a fallback. @@ -56,16 +57,26 @@ def verify_signature( WHAT THE TIMESTAMP ACTUALLY BUYS, stated exactly, because this used to claim it made a captured request "un-replayable" and that is more than it does: it BOUNDS the replay window to max_skew seconds. Inside that window the same - signature is accepted again — there is no nonce cache, so a capture replayed - within five minutes is indistinguishable from the original. - - That is a deliberate position, not an oversight. Closing it needs a store of - spent signatures, and the window it would close is already narrower than the - dedup window a door carries (120s by default), so a replay landing inside it - mostly arrives as a duplicate and is skipped as one. The card-action tokens - in hookrelay/actions.py DO get single use, because those spend money and - restart services; an inbound alert delivered twice costs a duplicate row. - Revisit this if a door ever grows an action whose second execution matters. + signature is accepted again — a capture replayed within five minutes is + indistinguishable from the original. + + CLOSED BY THE DOOR, not here (2026-10-12). A pure function has no store to + write to, and that is deliberate: `verify_signature` is the one thing both + the pipe's doors and any second implementation must agree on, so it stays + pure. The door pairs it with `replay_token` and `Store.claim_signature`, + which together answer the second copy as a duplicate before the pipeline and + keep it from spending a second verdict. The old reasoning — "the window is + narrower than the dedup window, so a replay lands as a duplicate anyway" — + was true only up to the dedup window: an alert replayed at t=200s is inside + the 300s skew, outside the 120s content window, and was a fresh judge call. + + The cache tracks the DEFAULT scheme's `X-Hook-Signature` and only that; a + plugin adapter speaking `X-Hub-Signature-256` or Stripe's scheme is not + tracked, which `replay_token` says out loud rather than guessing at a header + it cannot own. The bridge's inbound door keeps its own skew check and no + store, so its window stays bounded and `docs/containment.md` still says so. + The card-action tokens in hookrelay/actions.py still get single use for their + own reason: those spend money and restart services. """ if not secret: return True @@ -91,6 +102,24 @@ def verify_signature( return constant_time_eq(expected, provided) +def replay_token(source_name: str, signature: str | None) -> str | None: + """One value identifying one signed request, or None if it cannot be one. + + Source-scoped so two sources that happen to share a secret cannot spend each + other's rows, and normalised exactly the way `verify_signature` normalises + (lowercased, `sha256=` prefix stripped) so the two agree about what "the same + signature" means. Returns None when there is nothing worth tracking — no + source name, or no signature header (a source with no secret, or a plugin + scheme this does not know) — and a None means "let it through un-cached", not + a cache miss. A guard that guessed at a plugin's header would either miss or + refuse a legitimate retry, and neither is visible from here. + """ + token = (signature or "").strip().lower().removeprefix("sha256=") + if not source_name or not token: + return None + return f"{source_name}:{token}" + + def token_ok(configured: str, presented: str | None) -> bool: """Empty configured token = the check is disabled (dev mode for read, endpoint disabled for admin — the CALLER decides which semantic applies).""" diff --git a/hookrelay/hookrelay/settings.py b/hookrelay/hookrelay/settings.py index a39edc66..fe61f39d 100644 --- a/hookrelay/hookrelay/settings.py +++ b/hookrelay/hookrelay/settings.py @@ -50,6 +50,12 @@ class Settings: breaker_cooldown_seconds: int # Delivery-loop tick, in seconds. worker_interval_seconds: float + # How long an accepted inbound signature is remembered, so the same signed + # bytes replayed inside the timestamp's freshness window are answered as a + # duplicate instead of walking the pipeline again. 0 turns the cache off and + # leaves the window BOUNDED rather than closed, which is what the pipe did + # before 2026-10-12 — see security.verify_signature for what that costs. + signature_replay_seconds: int = 300 # Signs the buttons on a notification card and verifies them on the way # back. Empty = no card carries an action and /card-action refuses # everything, which is the right default: an unsigned button is a URL @@ -83,6 +89,7 @@ def load(cls) -> Settings: admin_token=os.environ.get("HOOKRELAY_ADMIN_TOKEN", ""), read_token=os.environ.get("HOOKRELAY_READ_TOKEN", ""), max_body_bytes=_int("HOOKRELAY_MAX_BODY_BYTES", 256 * 1024), + signature_replay_seconds=_int("HOOKRELAY_SIGNATURE_REPLAY_SECONDS", 300), max_attempts=_int("HOOKRELAY_MAX_ATTEMPTS", 8), retention_days=_int("HOOKRELAY_RETENTION_DAYS", 14), alarm_url=os.environ.get("HOOKRELAY_ALARM_URL", ""), diff --git a/hookrelay/hookrelay/store.py b/hookrelay/hookrelay/store.py index 59e93950..17841c77 100644 --- a/hookrelay/hookrelay/store.py +++ b/hookrelay/hookrelay/store.py @@ -124,6 +124,27 @@ ); CREATE INDEX IF NOT EXISTS ix_card_actions_event ON card_actions (event_id); CREATE INDEX IF NOT EXISTS ix_card_actions_correlation ON card_actions (correlation_id); + +-- One row per inbound signature this pipe has already accepted, so a capture +-- replayed inside the skew window is answered like a duplicate instead of being +-- walked through the pipeline a second time. The window is the signature's own +-- (`max_skew_seconds`, the same 300s `verify_signature` bounds a captured +-- request to), and the token carrying the source name keeps two sources that +-- happen to share a secret from spending each other's rows. Rows are pruned to +-- the window on every claim, so the table is bounded by the window rather than +-- by uptime. +-- +-- Deliberately a SECOND identity beside the content fingerprint +-- (`recent_duplicate`): that one folds a restatement of the same alert however +-- it was signed, this one folds the same signed BYTES however long ago the +-- content window closed. An alert replayed at t=200s is inside the skew but +-- outside the 120s content window, and without this it would buy a fresh judge +-- call — which is the whole reason a replay cache exists. +CREATE TABLE IF NOT EXISTS spent_signatures ( + token TEXT NOT NULL PRIMARY KEY, + seen_at REAL NOT NULL +); +CREATE INDEX IF NOT EXISTS ix_spent_signatures_seen ON spent_signatures (seen_at); """ @@ -893,6 +914,25 @@ async def record_action_outcome(self, jti: str, outcome: str) -> None: async with self._write(): await self.db.execute("UPDATE card_actions SET outcome = ? WHERE jti = ?", (outcome[:200], jti)) + # ── inbound signatures ──────────────────────────────────────────────── + + async def claim_signature(self, token: str, window_seconds: int, now: float) -> bool: + """True the first time this signature is seen inside the window, else False. + + Same shape as spend_action: claim BEFORE the pipeline, let the PRIMARY + KEY be the decision rather than a read-then-write with a gap. The prune + runs inside the same write lock, so the table is the size of the window + and not of uptime. `window_seconds <= 0` is the caller's business, not + this method's — a zero window is a refusal to cache, not a claim with + nothing to compare against. + """ + async with self._write(): + await self.db.execute("DELETE FROM spent_signatures WHERE seen_at < ?", (now - window_seconds,)) + cursor = await self.db.execute( + "INSERT OR IGNORE INTO spent_signatures (token, seen_at) VALUES (?, ?)", (token, now) + ) + return cursor.rowcount == 1 + async def recent_actions(self, limit: int = 50) -> list[dict[str, Any]]: cursor = await self.read.execute( "SELECT kind, event_id, correlation_id, actor, outcome, pressed_at " diff --git a/hookrelay/requirements.lock b/hookrelay/requirements.lock index 11825abb..38161790 100644 --- a/hookrelay/requirements.lock +++ b/hookrelay/requirements.lock @@ -9,6 +9,7 @@ CacheControl==0.14.4 certifi==2026.7.22 charset-normalizer==3.5.1 click==8.4.2 +coverage==7.16.2 cyclonedx-python-lib==11.12.0 defusedxml==0.7.1 fastapi==0.142.2 @@ -41,6 +42,7 @@ pydantic_core==2.46.4 Pygments==2.21.0 pyparsing==3.3.2 pytest==9.1.1 +pytest-cov==7.1.0 pytest-asyncio==1.4.0 python-dotenv==1.2.3 PyYAML==6.0.3 diff --git a/hookrelay/requirements.txt b/hookrelay/requirements.txt index 9b411800..9b1ce2a5 100644 --- a/hookrelay/requirements.txt +++ b/hookrelay/requirements.txt @@ -8,6 +8,7 @@ pyyaml>=6.0.3 # Test + lint pytest>=9.1.1 pytest-asyncio>=1.4.0 +pytest-cov>=7.0 ruff==0.16.10 bandit>=1.9.4 pip-audit>=2.10.1 diff --git a/hookrelay/scripts/gate.sh b/hookrelay/scripts/gate.sh index 6d8c85cb..0009a849 100755 --- a/hookrelay/scripts/gate.sh +++ b/hookrelay/scripts/gate.sh @@ -78,8 +78,12 @@ cfg = Config.from_file("config.example.yaml") print(f"example config: {len(cfg.sources)} sources, {len(cfg.channels)} channels, {len(cfg.routes)} routes") PYEOF -step "pytest" -$PY -m pytest -q +step "pytest (with a coverage floor)" +# The floor is a REGRESSION catch, not a target: set just below what the suite +# covers today, so a change that deletes tests or leaves a branch unreached fails +# the gate instead of quietly lowering the number. hookrelay measured 93% when +# this step was added; `--cov-fail-under` is the part that must not move. +$PY -m pytest -q --cov=hookrelay --cov-fail-under=90 step "pip-audit (known-vulnerable dependencies)" $PY -m pip_audit --progress-spinner off diff --git a/hookrelay/tests/test_extensibility.py b/hookrelay/tests/test_extensibility.py index d1cb8924..7364df2b 100644 --- a/hookrelay/tests/test_extensibility.py +++ b/hookrelay/tests/test_extensibility.py @@ -460,7 +460,7 @@ def runs(text: str, command: str) -> bool: "ruff format --check hookrelay tests", "mypy hookrelay", "bandit -q -r hookrelay", - "pytest -q", + "pytest -q --cov=hookrelay --cov-fail-under=90", "pip_audit --progress-spinner off", ): assert runs(gate, command), f"gate.sh does not run {command!r}" @@ -500,6 +500,8 @@ def test_the_stack_gate_matches_its_workflow() -> None: "scripts/assert_copies.py", "scripts/assert_ordering.py", "scripts/assert_docs.py", + # Every headline number: reproducible here or named as measured elsewhere. + "scripts/reproduce_claims.py", # With --check, deliberately. Without it the generator WRITES, so a # runner that dropped the flag would rewrite the files it was meant to be # verifying and pass every time. diff --git a/hookrelay/tests/test_signature_replay.py b/hookrelay/tests/test_signature_replay.py new file mode 100644 index 00000000..bdafb3d4 --- /dev/null +++ b/hookrelay/tests/test_signature_replay.py @@ -0,0 +1,130 @@ +"""A replayed signed request is folded, not walked through the pipeline again. + +The timestamp on a signed door bounds how long a capture stays useful; on its +own it does not close the window, because the same signature inside it is +accepted again. `replay_token` + `Store.claim_signature` close it at the door, +BEFORE the fuse and the pipeline, so the second copy cannot spend a second +verdict. These tests pin both halves: the door's observable answer, and the +`None` cases where there is nothing a guard can track without guessing. +""" + +from __future__ import annotations + +import hashlib +import hmac +import json + +from hookrelay.security import replay_token + +PAYLOAD = {"title": "db down", "message": "x", "state": "alerting"} + + +def _sig(secret: str, body: bytes) -> str: + return hmac.new(secret.encode(), body, hashlib.sha256).hexdigest() + + +def _ts_sig(secret: str, body: bytes, ts: int) -> str: + return hmac.new(secret.encode(), str(ts).encode() + b"." + body, hashlib.sha256).hexdigest() + + +# ── the door ────────────────────────────────────────────────────────────── + + +async def test_same_signature_twice_is_folded_the_second_time(client): + body = json.dumps(PAYLOAD).encode() + headers = {"X-Hook-Signature": _sig("s3cret", body), "content-type": "application/json"} + first = await client.post("/hook/grafana", content=body, headers=headers) + assert first.status_code == 200 + assert first.json()["outcome"] == "routed" + + second = await client.post("/hook/grafana", content=body, headers=headers) + assert second.status_code == 200 + assert second.json() == {"outcome": "skipped", "skip_code": "replayed_signature"} + + +async def test_a_replay_is_folded_even_outside_the_content_window(client): + """The point of a SECOND identity: the content fingerprint only folds inside + its own window, and the skew outlives it. Changing the source's content + fingerprint cannot make a replayed capture look new here.""" + body = json.dumps(PAYLOAD).encode() + headers = {"X-Hook-Signature": _sig("s3cret", body)} + assert (await client.post("/hook/grafana", content=body, headers=headers)).json()["outcome"] == "routed" + replay = await client.post("/hook/grafana", content=body, headers=headers) + assert replay.json()["skip_code"] == "replayed_signature" + + +async def test_a_timestamped_capture_is_folded_too(client): + body = json.dumps(PAYLOAD).encode() + ts = int(__import__("time").time()) + headers = {"X-Hook-Signature": _ts_sig("s3cret", body, ts), "X-Hook-Timestamp": str(ts)} + assert (await client.post("/hook/grafana", content=body, headers=headers)).json()["outcome"] == "routed" + replay = await client.post("/hook/grafana", content=body, headers=headers) + assert replay.json()["skip_code"] == "replayed_signature" + + +async def test_a_different_signature_is_not_folded(client): + """A second alert with its own signature must reach the pipeline as before: + this stage folds signed BYTES, not the content dedup's business.""" + body = json.dumps(PAYLOAD).encode() + other = json.dumps({**PAYLOAD, "title": "cache miss", "message": "y"}).encode() + first = await client.post("/hook/grafana", content=body, headers={"X-Hook-Signature": _sig("s3cret", body)}) + second = await client.post("/hook/grafana", content=other, headers={"X-Hook-Signature": _sig("s3cret", other)}) + assert first.json()["outcome"] == "routed" + assert second.json()["outcome"] == "routed" + + +async def test_the_cache_is_off_when_the_window_is_zero(settings, cfg): + """0 is a refusal to cache: the door keeps its bounded-window behaviour and + neither copy is folded by THIS stage. The knob has to actually do something.""" + import dataclasses + + import httpx + + from hookrelay.app import create_app + + body = json.dumps(PAYLOAD).encode() + headers = {"X-Hook-Signature": _sig("s3cret", body)} + off = dataclasses.replace(settings, signature_replay_seconds=0) + app = create_app(settings=off, cfg=cfg) + async with ( + httpx.ASGITransport(app=app) as transport, + app.router.lifespan_context(app), + httpx.AsyncClient(transport=transport, base_url="http://t") as c, + ): + first = await c.post("/hook/grafana", content=body, headers=headers) + second = await c.post("/hook/grafana", content=body, headers=headers) + assert first.json()["outcome"] == "routed" + # Unfolded by the replay cache, so the second copy is the content dedup's. + assert second.json()["skip_code"] != "replayed_signature" + + +# ── the token ───────────────────────────────────────────────────────────── + + +def test_replay_token_is_normalised_like_verify_signature(): + assert replay_token("grafana", "SHA256=ABCdef") == "grafana:abcdef" + assert replay_token("grafana", "abc") == replay_token("grafana", "sha256=abc") + + +def test_replay_token_is_source_scoped(): + """Two sources sharing a secret must not spend each other's rows.""" + assert replay_token("a", "sig") != replay_token("b", "sig") + + +def test_replay_token_is_none_when_there_is_nothing_to_track(): + assert replay_token("grafana", None) is None + assert replay_token("grafana", "") is None + assert replay_token("", "sig") is None + + +async def test_claim_signature_is_true_once_then_false(store): + now = 1_000.0 + assert await store.claim_signature("grafana:abc", 300, now) is True + assert await store.claim_signature("grafana:abc", 300, now + 1) is False + + +async def test_claim_signature_forgets_outside_the_window(store): + """The prune is what keeps the table the size of the window, not of uptime — + and what lets a genuinely new request with recycled bytes through later.""" + assert await store.claim_signature("grafana:abc", 300, 1_000.0) is True + assert await store.claim_signature("grafana:abc", 300, 1_301.0) is True diff --git a/scripts/assert_weight.py b/scripts/assert_weight.py index 697c0aed..6b9dac9b 100755 --- a/scripts/assert_weight.py +++ b/scripts/assert_weight.py @@ -295,7 +295,18 @@ # base window later, through the stages after the fold, the walk an # arriving event takes. Replayed on the retired deployment's judge ledger it # folds 55% of the wake=yes cards where the rule as shipped folded 24%. - "hookrelay": (6300, Path("hookrelay/README.md")), + # 6300 -> 6400 on 2026-10-12, for the inbound replay cache: a signature the + # pipe has already accepted is claimed once (`Store.claim_signature`) and the + # second copy is answered as a duplicate before the pipeline, so a capture + # replayed inside the timestamp's own freshness window cannot buy a second + # verdict. Split, measured: +~55 source, +~30 code, so the doctrine question + # applies and has the easy answer — this is the DOOR's business and not a + # judgment about an alert: it reads the pipe's own `X-Hook-Signature` header, + # keys on the pipe's own source name, and never looks at a payload. Same class + # as the dedup fingerprint beside it. The rest of the raise is the docstring + # that says so; `docs/containment.md` used to read "there is no nonce cache", + # and every line that changed that sentence lives next to the code that did. + "hookrelay": (6400, Path("hookrelay/README.md")), # 2900 -> 3000 on 2026-08-21, for the judge's second axis (`wake_someone`). # Raised rather than trimmed because the thing that pushed it over is the one # measurement that says whether this service earns its model calls at all: @@ -420,9 +431,19 @@ def main() -> int: ceiling, readme = CEILINGS[service] headroom = ceiling - source + # The heaviest modules ride the PASSING line too, not only the failure + # message. A headroom of +4 was invisible until the next feature went red; + # showing where the weight already sits is what turns "raise it again" from + # a formality into a look at the distribution. It is printed, never capped + # — a per-module ceiling would price the pipe's own decomposition, which + # .agents/notes/implemented/2026-09-10-the-ceiling-stays-per-service.md + # already declined. + heaviest_modules = ", ".join( + f"{name} {count}" for name, count in sorted(per_module.items(), key=lambda item: -item[1])[:3] + ) lines.append( f" {service:10} {source:>5} source ({source_code} code) · {tests:>5} tests · " - f"ceiling {ceiling} ({headroom:+d})" + f"ceiling {ceiling} ({headroom:+d}) · heaviest {heaviest_modules}" ) if source > ceiling: diff --git a/scripts/gate.sh b/scripts/gate.sh index 185415d1..0bbe8323 100755 --- a/scripts/gate.sh +++ b/scripts/gate.sh @@ -92,6 +92,11 @@ python3 scripts/assert_python_pins.py # hold its own number, but not the fact that all three are measured the same way # and that only two of them are capped. python3 scripts/assert_weight.py +# Every headline number, in one of two classes: reproducible here (run NOW, and +# fail if it has stopped holding) or measured on the retired deployments and +# named as such. The venvs the component gates just built are what make the two +# pytest-backed claims a real run rather than a skip. +python3 scripts/reproduce_claims.py # And whether the settings each service reads can be SET on a deployment. There # is no env_file: the compose `environment:` block is the whole of what reaches a # container, so a knob no compose names is a default with a docstring — which is diff --git a/scripts/gen_front_pages.py b/scripts/gen_front_pages.py index d8548793..a4667e67 100755 --- a/scripts/gen_front_pages.py +++ b/scripts/gen_front_pages.py @@ -69,6 +69,13 @@ ("README.zh-CN.md", "docs/zh/index.md", "../", "[English](../) · **中文**", "## 延伸阅读", ZH_DESCRIPTION, ZH_TAIL), ) _SERVICE_LINK = re.compile(r"\[(`hook(?:relay|judge|probe)`)\]\(hook(?:relay|judge|probe)\)") +# A link to a repository-root file the site does NOT serve (OVERVIEW.md, +# STACK.md, CONTRIBUTING.md…). `docs/` pages keep their handling below; anything +# else ending in `.md` becomes an absolute repository URL, which is exactly the +# answer the tail already gives. Without this a body link to STACK.md was a dead +# relative link on the site page — found by check-docs.py the first time the +# README linked one above the tail. +_ROOT_MD = re.compile(r"\]\(((?!docs/|https?://|mailto:|/|#)[^)\s#]+\.md)(#[^)]*)?\)") _OWNER = re.compile(r"https://github\.com/([^/\s]+)/hookstack/") # The README's own language switch, in either language's form. _SWITCH = re.compile(r"^(\*\*English\*\*|\[English\]).*中文") @@ -92,8 +99,9 @@ def render(readme: str, prefix: str, language_line: str, tail_heading: str, desc if not (line.startswith("# ") or line.startswith("[![") or _SWITCH.match(line)) ] text = "\n".join(lines).strip("\n") + "\n" - text = text.replace("](docs/", f"]({prefix}") text = _SERVICE_LINK.sub(r"\1", text) + text = _ROOT_MD.sub(lambda m: f"]({repo}/blob/main/{m.group(1)}{m.group(2) or ''})", text) + text = text.replace("](docs/", f"]({prefix}") front = f"---\ntitle: hookstack\ndescription: {description}\n---\n\n{language_line}\n\n" return front + text + "\n" + tail.format(repo=repo, owner=who) diff --git a/scripts/reproduce_claims.py b/scripts/reproduce_claims.py new file mode 100755 index 00000000..ca655273 --- /dev/null +++ b/scripts/reproduce_claims.py @@ -0,0 +1,152 @@ +#!/usr/bin/env python3 +"""Every headline number: reproduce it here, or say out loud that you cannot. + +`docs/evidence.md` is the prose table; this is the half of it a machine can +check. The repository's most persuasive numbers come from two deployments on +somebody else's infrastructure, whose ledgers are runtime data and therefore not +in the tree — so the useful thing is not a green tick over all of them, it is a +line that says which class each number is in. + +Two classes, and the failure modes are different: + + reproducible a command in this repository produces the number. THIS script + runs it and fails if it has stopped agreeing. + reported measured on the retired deployments. Nothing here can run it, so + nothing here claims to. The row names the artefact that would + back it, because "trust me" and "ask me for this file" are + different sentences. + + python3 scripts/reproduce_claims.py # check the reproducible half + python3 scripts/reproduce_claims.py --list # the table, changed nothing + +WHAT THIS DOES NOT DO. It does not audit the reported numbers — it cannot, and a +script that pretended to would be the worst outcome this file could produce: a +green light over a figure nobody checked. It also does not run the two +pytest-backed claims unless the owning service has a venv; without one it says +so and points at the command, because a check that silently skips is one that +passes by finding nothing. +""" + +from __future__ import annotations + +import subprocess # nosec B404 — runs this repository's own checks, fixed argv +import sys +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent + + +def _run(args: list[str]) -> tuple[bool, str]: + """(passed, first line of output). Never raises on a failing check.""" + try: + result = subprocess.run( # nosec B603 B607 — fixed argv, this repo's own scripts + args, cwd=ROOT, capture_output=True, text=True, timeout=180, check=False + ) + except (OSError, subprocess.SubprocessError) as exc: + return False, str(exc) + first = (result.stdout.strip() or result.stderr.strip()).splitlines() + return result.returncode == 0, (first[-1] if first else "") + + +def _pytest_claim(test_path: str, venv_service: str) -> tuple[bool | None, str]: + """Run one test file in a service's venv, or say it could not run. + + `venv_service` is the venv to borrow, not necessarily the test's own service — + the lark bridge has no venv and is tested from hookrelay's, which is what the + stack gate does too. + + A missing venv is `None` (UNRUN), never `True`: a reader running this on a + fresh clone gets the honest answer and the command to fix it, and the whole + repository gate — where the venvs always exist, because the component gates + built them a moment earlier — gets a real run of both files. + """ + python = ROOT / venv_service / ".venv" / "bin" / "python" + if not python.is_file(): + return None, f"no {venv_service}/.venv — run `cd {venv_service} && bash scripts/gate.sh` once, then re-run" + passed, detail = _run([str(python), "-m", "pytest", "-q", test_path]) + return passed, detail + + +# (claim, how to check it, artefact for the reader) +REPRODUCIBLE: tuple[tuple[str, str, object], ...] = ( + ( + "hookrelay stays at or under 6,400 source lines", + "hookrelay/README.md", + lambda: _run([sys.executable, "scripts/assert_weight.py"]), + ), + ( + "hookjudge stays at or under 3,400 source lines", + "hookjudge/README.md", + lambda: _run([sys.executable, "scripts/assert_weight.py"]), + ), + ( + "every route, knob and containment row is written up or named as not", + "docs/containment.md", + lambda: _run([sys.executable, "scripts/assert_docs.py"]), + ), + ( + "the three boards share one design language", + "hookrelay · hookjudge · hookprobe pages", + lambda: _run([sys.executable, "scripts/assert_design.py"]), + ), + ( + "a replayed signed request cannot buy a second verdict", + "docs/containment.md", + lambda: _pytest_claim("hookrelay/tests/test_signature_replay.py", "hookrelay"), + ), + ( + "an empty BRIDGE_INBOUND_SECRET refuses to start", + "docs/containment.md", + lambda: _pytest_claim("deploy/lark-bridge/tests/test_fail_closed.py", "hookrelay"), + ), +) + +REPORTED: tuple[tuple[str, str], ...] = ( + ("795 production alerts; 28 of 29 rules answered identically", "hookjudge/eval/dataset.jsonl (gitignored)"), + ("the fold sends 55% fewer cards than the rule as shipped", "the retired judge ledger"), + ("the first real write happened 2026-09-30", "the work deployment's run records and audit log"), + ("kubectl was absent from the work image for ten days", "the release job history and posture records"), + ("one HTTP POST from inside a probe reached the chat MCP port", "the 2026-09-30 measurement"), +) + + +def main() -> int: + if "--list" in sys.argv: + for claim, artefact in REPORTED: + print(f" reported {claim} · {artefact}") + return 0 + + failed: list[str] = [] + unrun = 0 + print("reproducible claims (run here):") + for claim, artefact, check in REPRODUCIBLE: + passed, detail = check() + if passed is None: + mark, unrun = "UNRUN", unrun + 1 + else: + mark = "OK " if passed else "FAIL" + print(f" {mark} {claim}") + print(f" backed by {artefact}") + if passed is False: + failed.append(f"{claim} — {detail}") + print(f" {detail}") + elif passed is None: + print(f" {detail}") + + print("\nreported claims (measured on the retired deployments; not in this repository):") + for claim, artefact in REPORTED: + print(f" -- {claim}") + print(f" would be backed by {artefact}") + + if failed: + print(f"\n{len(failed)} reproducible claim(s) no longer hold:") + for line in failed: + print(f" {line}") + return 1 + tail = f"; {unrun} could not run here (no venv)" if unrun else "" + print(f"\nevidence: every claim this repository can check still holds{tail}; the rest say so out loud.") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) From 44d191ff9836e5a92f3e24090998fb34219f7be4 Mon Sep 17 00:00:00 2001 From: itswl Date: Mon, 12 Oct 2026 23:36:28 +0800 Subject: [PATCH 2/5] hookrelay: claim an inbound signature only after the pipeline answers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The replay cache wrote its row immediately after `adapter.verify`, before JSON parsing, the storm fuse and `handle_hook`. A retry of identical bytes after a 5xx, a client timeout, or the fuse's 429 was then answered as a replay, and the event was lost while both sides believed it had been handled — the bounded window this replaced had tolerated exactly those retries, and the fuse's reject stores nothing at all. The door now PEEKS (`Store.signature_seen`) before the pipeline and CLAIMS (`Store.claim_signature`) only once the pipeline has returned, so a failed attempt leaves nothing behind for the retry to trip over. The cost is the race between two concurrent identical requests, which degrades to the content dedup — the behaviour before this cache existed, and acceptable. Tests: a retry after a 5xx and a retry after a 429 are both processed, the peek has no side effect of the claim, and the success path still folds. The docstring names the legacy body-only case: an identical body carries an identical token, so a legitimate identical re-send between the 120s content window and the 300s replay window now folds, where a timestamped signature would not. --- .../proving-the-boundaries-run.md | 0 hookrelay/hookrelay/app.py | 28 +++-- hookrelay/hookrelay/security.py | 8 ++ hookrelay/hookrelay/store.py | 31 ++++- hookrelay/tests/test_signature_replay.py | 108 +++++++++++++++--- 5 files changed, 144 insertions(+), 31 deletions(-) rename {docs/blog => blog}/proving-the-boundaries-run.md (100%) diff --git a/docs/blog/proving-the-boundaries-run.md b/blog/proving-the-boundaries-run.md similarity index 100% rename from docs/blog/proving-the-boundaries-run.md rename to blog/proving-the-boundaries-run.md diff --git a/hookrelay/hookrelay/app.py b/hookrelay/hookrelay/app.py index 4f044881..99288946 100644 --- a/hookrelay/hookrelay/app.py +++ b/hookrelay/hookrelay/app.py @@ -424,15 +424,19 @@ async def hook(source_name: str, request: Request) -> JSONResponse: raise HTTPException(status_code=401, detail="bad signature") # A captured request replayed inside the skew is the same signed BYTES, # and the content fingerprint cannot see that: it folds a restatement of - # the same alert however long ago the content window closed. Claiming the - # signature here answers the second copy as a skip — BEFORE the fuse and - # the pipeline, so it cannot spend a second verdict. Off when the window - # is 0, and a scheme this cannot name (a plugin adapter's own header) - # yields no token, which is a refusal to guess rather than a miss. + # the same alert however long ago the content window closed. So ask here + # and answer the second copy as a skip — but PEEK, do not claim: the row + # is written only after the pipeline has answered (below), because a + # claim made before it would turn a sender's identical retry after a 5xx, + # a timeout or the fuse's 429 into a replay, losing the event while both + # sides believed it was handled. Off when the window is 0, and a scheme + # this cannot name (a plugin adapter's own header) yields no token, which + # is a refusal to guess rather than a miss. + replay: str | None = None if app_settings.signature_replay_seconds > 0: - token = replay_token(source.name, headers.get("x-hook-signature")) - if token is not None and not await app.state.store.claim_signature( - token, app_settings.signature_replay_seconds, now_ts() + replay = replay_token(source.name, headers.get("x-hook-signature")) + if replay is not None and await app.state.store.signature_seen( + replay, app_settings.signature_replay_seconds, now_ts() ): return JSONResponse({"outcome": "skipped", "skip_code": "replayed_signature"}) try: @@ -463,6 +467,8 @@ async def hook(source_name: str, request: Request) -> JSONResponse: source.storm_threshold, extracted=extracted, ) + if replay is not None: + await app.state.store.claim_signature(replay, app_settings.signature_replay_seconds, now_ts()) return JSONResponse({"event_id": event_id, "outcome": "skipped", "skip_code": "storm_suppressed"}) result = await handle_hook( @@ -475,6 +481,12 @@ async def hook(source_name: str, request: Request) -> JSONResponse: client=app.state.http_client, extracted=extracted, ) + # The pipeline answered, so the event was handled once: remember the + # signature now. Anything that raised above — the 400 for a body that is + # not JSON, the 429 from the hard fuse, a fault inside handle_hook — + # reaches here having claimed nothing, so the sender's retry is processed. + if replay is not None: + await app.state.store.claim_signature(replay, app_settings.signature_replay_seconds, now_ts()) return JSONResponse(result) @app.post("/explain/{source_name}") diff --git a/hookrelay/hookrelay/security.py b/hookrelay/hookrelay/security.py index ab829dbf..ba7cb2f6 100644 --- a/hookrelay/hookrelay/security.py +++ b/hookrelay/hookrelay/security.py @@ -113,6 +113,14 @@ def replay_token(source_name: str, signature: str | None) -> str | None: scheme this does not know) — and a None means "let it through un-cached", not a cache miss. A guard that guessed at a plugin's header would either miss or refuse a legitimate retry, and neither is visible from here. + + One consequence worth stating, because it is the case a sender hits: a + LEGACY body-only signature is a function of the body alone, so two identical + bodies with no timestamp carry the SAME token. A legitimate identical re-send + that lands between the content window (120s) and this cache's window (300s) is + therefore folded where it used to be processed again. A TIMESTAMPED signature + covers the timestamp, so a re-send there is a different token and is not + folded — which is one more reason the timestamped form is the preferred one. """ token = (signature or "").strip().lower().removeprefix("sha256=") if not source_name or not token: diff --git a/hookrelay/hookrelay/store.py b/hookrelay/hookrelay/store.py index 17841c77..2c271695 100644 --- a/hookrelay/hookrelay/store.py +++ b/hookrelay/hookrelay/store.py @@ -916,15 +916,34 @@ async def record_action_outcome(self, jti: str, outcome: str) -> None: # ── inbound signatures ──────────────────────────────────────────────── + async def signature_seen(self, token: str, window_seconds: int, now: float) -> bool: + """Whether this signature was CLAIMED inside the window. A read, no claim. + + Paired with `claim_signature` so the door can ask before the pipeline and + record only once the pipeline has answered. Claiming up front looked + tidier and lost events: a row written before a 5xx, a client timeout or + the fuse's 429 makes the sender's identical RETRY answer as a replay, and + both sides then believe the event was handled. The cost of the split is + the race between two concurrent identical requests, which degrades to the + content dedup — the behaviour before this cache existed, and acceptable. + + The prune lives in `claim_signature`, which runs on every HANDLED + request, so this stays a read and the table stays bounded by the window. + """ + cursor = await self.read.execute( + "SELECT 1 FROM spent_signatures WHERE token = ? AND seen_at >= ? LIMIT 1", + (token, now - window_seconds), + ) + return await cursor.fetchone() is not None + async def claim_signature(self, token: str, window_seconds: int, now: float) -> bool: """True the first time this signature is seen inside the window, else False. - Same shape as spend_action: claim BEFORE the pipeline, let the PRIMARY - KEY be the decision rather than a read-then-write with a gap. The prune - runs inside the same write lock, so the table is the size of the window - and not of uptime. `window_seconds <= 0` is the caller's business, not - this method's — a zero window is a refusal to cache, not a claim with - nothing to compare against. + Same shape as spend_action: the PRIMARY KEY is the decision rather than a + read-then-write with a gap. The prune runs inside the same write lock, so + the table is the size of the window and not of uptime. + `window_seconds <= 0` is the caller's business, not this method's — a zero + window is a refusal to cache, not a claim with nothing to compare against. """ async with self._write(): await self.db.execute("DELETE FROM spent_signatures WHERE seen_at < ?", (now - window_seconds,)) diff --git a/hookrelay/tests/test_signature_replay.py b/hookrelay/tests/test_signature_replay.py index bdafb3d4..913674ba 100644 --- a/hookrelay/tests/test_signature_replay.py +++ b/hookrelay/tests/test_signature_replay.py @@ -2,18 +2,26 @@ The timestamp on a signed door bounds how long a capture stays useful; on its own it does not close the window, because the same signature inside it is -accepted again. `replay_token` + `Store.claim_signature` close it at the door, -BEFORE the fuse and the pipeline, so the second copy cannot spend a second -verdict. These tests pin both halves: the door's observable answer, and the -`None` cases where there is nothing a guard can track without guessing. +accepted again. `replay_token` + the store's `signature_seen`/`claim_signature` +pair close it at the door. These tests pin three things: the door's observable +answer, the `None` cases where there is nothing a guard can track without +guessing, and — the one a first cut got wrong — that the signature is claimed +only once the pipeline has ANSWERED, so a retry after a 5xx or a 429 is +processed rather than folded as a replay. """ from __future__ import annotations +import contextlib +import dataclasses import hashlib import hmac import json +import httpx + +import hookrelay.app as app_module +from hookrelay.app import create_app from hookrelay.security import replay_token PAYLOAD = {"title": "db down", "message": "x", "state": "alerting"} @@ -27,6 +35,22 @@ def _ts_sig(secret: str, body: bytes, ts: int) -> str: return hmac.new(secret.encode(), str(ts).encode() + b"." + body, hashlib.sha256).hexdigest() +@contextlib.asynccontextmanager +async def _serve(app, *, raise_app_exceptions: bool = True): + """An app served in process, so a test can reach `app.state` and monkeypatch. + + `raise_app_exceptions=False` is what turns a fault in the pipeline into a 500 + RESPONSE, which is the case a sender retries against. + """ + transport = httpx.ASGITransport(app=app, raise_app_exceptions=raise_app_exceptions) + async with ( + transport, + app.router.lifespan_context(app), + httpx.AsyncClient(transport=transport, base_url="http://t") as client, + ): + yield client + + # ── the door ────────────────────────────────────────────────────────────── @@ -76,21 +100,10 @@ async def test_a_different_signature_is_not_folded(client): async def test_the_cache_is_off_when_the_window_is_zero(settings, cfg): """0 is a refusal to cache: the door keeps its bounded-window behaviour and neither copy is folded by THIS stage. The knob has to actually do something.""" - import dataclasses - - import httpx - - from hookrelay.app import create_app - body = json.dumps(PAYLOAD).encode() headers = {"X-Hook-Signature": _sig("s3cret", body)} - off = dataclasses.replace(settings, signature_replay_seconds=0) - app = create_app(settings=off, cfg=cfg) - async with ( - httpx.ASGITransport(app=app) as transport, - app.router.lifespan_context(app), - httpx.AsyncClient(transport=transport, base_url="http://t") as c, - ): + app = create_app(settings=dataclasses.replace(settings, signature_replay_seconds=0), cfg=cfg) + async with _serve(app) as c: first = await c.post("/hook/grafana", content=body, headers=headers) second = await c.post("/hook/grafana", content=body, headers=headers) assert first.json()["outcome"] == "routed" @@ -98,6 +111,58 @@ async def test_the_cache_is_off_when_the_window_is_zero(settings, cfg): assert second.json()["skip_code"] != "replayed_signature" +async def test_a_retry_after_a_5xx_is_processed(settings, cfg, monkeypatch): + """The claim is written only once the pipeline has ANSWERED. Claimed up front, + a fault here would make the sender's identical retry answer as a replay and + the event would be lost while both sides believed it had been handled.""" + real = app_module.handle_hook + calls = {"n": 0} + + async def flaky(*args, **kwargs): + calls["n"] += 1 + if calls["n"] == 1: + raise RuntimeError("the pipeline fell over") + return await real(*args, **kwargs) + + monkeypatch.setattr(app_module, "handle_hook", flaky) + body = json.dumps(PAYLOAD).encode() + headers = {"X-Hook-Signature": _sig("s3cret", body)} + app = create_app(settings=settings, cfg=cfg) + async with _serve(app, raise_app_exceptions=False) as c: + first = await c.post("/hook/grafana", content=body, headers=headers) + assert first.status_code == 500 + # The same bytes again: the retry must be processed, not folded. + second = await c.post("/hook/grafana", content=body, headers=headers) + assert second.status_code == 200 + assert second.json()["outcome"] == "routed" + + +async def test_a_retry_after_the_fuse_429_is_processed(settings, cfg): + """The hard fuse stores nothing, so nothing about it may burn the signature: + once the storm passes, the retry is processed rather than folded.""" + body = json.dumps(PAYLOAD).encode() + headers = {"X-Hook-Signature": _sig("s3cret", body)} + app = create_app(settings=settings, cfg=cfg) + verdicts = iter(["reject", "pass"]) + app.state.fuse.check = lambda *args, **kwargs: next(verdicts) + async with _serve(app) as c: + first = await c.post("/hook/grafana", content=body, headers=headers) + assert first.status_code == 429 + second = await c.post("/hook/grafana", content=body, headers=headers) + assert second.status_code == 200 + assert second.json()["outcome"] == "routed" + + +async def test_a_second_request_after_success_is_still_folded(client): + """And the claim DOES happen on the success path — the fold must not have been + traded away for the retry behaviour above.""" + body = json.dumps(PAYLOAD).encode() + headers = {"X-Hook-Signature": _sig("s3cret", body)} + assert (await client.post("/hook/grafana", content=body, headers=headers)).json()["outcome"] == "routed" + again = await client.post("/hook/grafana", content=body, headers=headers) + assert again.json() == {"outcome": "skipped", "skip_code": "replayed_signature"} + + # ── the token ───────────────────────────────────────────────────────────── @@ -123,6 +188,15 @@ async def test_claim_signature_is_true_once_then_false(store): assert await store.claim_signature("grafana:abc", 300, now + 1) is False +async def test_signature_seen_asks_without_claiming(store): + """The peek must not have the side effect of the claim, or the door's + peek-then-claim split would be a claim wearing two names.""" + assert await store.signature_seen("grafana:abc", 300, 1_000.0) is False + assert await store.signature_seen("grafana:abc", 300, 1_000.0) is False + assert await store.claim_signature("grafana:abc", 300, 1_000.0) is True + assert await store.signature_seen("grafana:abc", 300, 1_000.0) is True + + async def test_claim_signature_forgets_outside_the_window(store): """The prune is what keeps the table the size of the window, not of uptime — and what lets a genuinely new request with recycled bytes through later.""" From 71112457f66ca061e5e4f95db32c92f4c2d288bf Mon Sep 17 00:00:00 2001 From: itswl Date: Mon, 12 Oct 2026 23:36:44 +0800 Subject: [PATCH 3/5] docs: keep the runnable demo on the front pages, and the draft out of Pages MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three adjustments from review: * the three demo commands come back INLINE under "10 minutes" / "10 分钟" on both front pages. Short front pages are right; taking the only runnable commands off both of them is not, and the pages they pointed at are English. * the blog outline was a self-described DRAFT under `docs/`, which is the Pages root, so merging would publish it. It moves to `blog/`, content unchanged, until it is a post. * docs/releasing.md loses the step that asked for a GitHub Release object. This repository releases with an annotated tag whose message is the note and has no Release objects (0.5.0 and 0.6.0 shipped that way); the page now says so. --- README.md | 13 ++++++++++++- README.zh-CN.md | 11 ++++++++++- docs/index.md | 13 ++++++++++++- docs/releasing.md | 5 ++--- docs/zh/index.md | 11 ++++++++++- 5 files changed, 46 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 25ce93c4..27b8f7d8 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,18 @@ The investigator is the product. The pipe and the judge are the smallest alertin ## Start here - **30 seconds:** [docs/quickstart.md](docs/quickstart.md) opens with the shape of it — three services, one diagram. -- **10 minutes, no keys, no bill:** the same page's second section runs the whole loop on a rehearsal: an alert judged, an investigation replayed through the real read-only gate, a card, an approve press, a resolved alert. +- **10 minutes, no keys, no bill:** pipe + judge + a stub model + the investigator on a rehearsal + a readable sink. Docker is the only prerequisite. + + ```bash + curl -fsSLO https://raw.githubusercontent.com/itswl/hookstack/main/docker-compose.quickstart.yml + docker compose -f docker-compose.quickstart.yml up -d + bash <(curl -fsSL https://raw.githubusercontent.com/itswl/hookstack/main/scripts/demo.sh) + ``` + + The demo runs the whole loop: an alert judged, a recorded investigation replayed + through the real read-only gate, the report as a card, an approve press running + two allowlisted commands, the alert resolved. [docs/quickstart.md](docs/quickstart.md) + has the rest of it — a real model, the boards, the half-day path. - **Half a day, for real:** [STACK.md](STACK.md) runs all three from source; [docs/containment.md](docs/containment.md) is the boundary map you read *before* pointing it at a cluster. - **What it does, in full:** [docs/what-you-get.md](docs/what-you-get.md). diff --git a/README.zh-CN.md b/README.zh-CN.md index 12e85bbf..911ddaa2 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -17,7 +17,16 @@ ## 从这里开始 - **30 秒：**[docs/quickstart.md](docs/quickstart.md) 开头就讲清了它的形状 —— 三个服务，一张图。 -- **10 分钟，不要 key，不花钱：** 同一页的第二段会在排练上跑完整条闭环：告警被判定，调查经过真实的只读门回放，卡片，按下批准，告警恢复。 +- **10 分钟，不要 key，不花钱：** 管道 + 判官 + 桩模型 + 跑排练的调查员 + 可读的接收端，只要有 Docker。 + + ```bash + curl -fsSLO https://raw.githubusercontent.com/itswl/hookstack/main/docker-compose.quickstart.yml + docker compose -f docker-compose.quickstart.yml up -d + bash <(curl -fsSL https://raw.githubusercontent.com/itswl/hookstack/main/scripts/demo.sh) + ``` + + 演示会跑完整条闭环：告警被判定，一次录好的调查经过真实的只读门回放，报告变成卡片，按下批准后跑两条允许清单里的命令，告警恢复。[docs/quickstart.md](docs/quickstart.md) + 里有其余内容 —— 换成真实模型、三块看板、半天的路径。 - **半天，真上：**[STACK.md](STACK.md) 从源码一步步跑起三个服务；[docs/containment.md](docs/containment.md) 是指向真实集群**之前**要读的边界地图。 - **完整功能清单：**[docs/what-you-get.md](docs/what-you-get.md)（英文）。 diff --git a/docs/index.md b/docs/index.md index 02488fc9..d4e28150 100644 --- a/docs/index.md +++ b/docs/index.md @@ -20,7 +20,18 @@ The investigator is the product. The pipe and the judge are the smallest alertin ## Start here - **30 seconds:** [docs/quickstart.md](quickstart.md) opens with the shape of it — three services, one diagram. -- **10 minutes, no keys, no bill:** the same page's second section runs the whole loop on a rehearsal: an alert judged, an investigation replayed through the real read-only gate, a card, an approve press, a resolved alert. +- **10 minutes, no keys, no bill:** pipe + judge + a stub model + the investigator on a rehearsal + a readable sink. Docker is the only prerequisite. + + ```bash + curl -fsSLO https://raw.githubusercontent.com/itswl/hookstack/main/docker-compose.quickstart.yml + docker compose -f docker-compose.quickstart.yml up -d + bash <(curl -fsSL https://raw.githubusercontent.com/itswl/hookstack/main/scripts/demo.sh) + ``` + + The demo runs the whole loop: an alert judged, a recorded investigation replayed + through the real read-only gate, the report as a card, an approve press running + two allowlisted commands, the alert resolved. [docs/quickstart.md](quickstart.md) + has the rest of it — a real model, the boards, the half-day path. - **Half a day, for real:** [STACK.md](https://github.com/itswl/hookstack/blob/main/STACK.md) runs all three from source; [docs/containment.md](containment.md) is the boundary map you read *before* pointing it at a cluster. - **What it does, in full:** [docs/what-you-get.md](what-you-get.md). diff --git a/docs/releasing.md b/docs/releasing.md index 16e596ac..cd5ab6ed 100644 --- a/docs/releasing.md +++ b/docs/releasing.md @@ -14,7 +14,8 @@ registry but the container one. The release notes are not a separate document. They are the implemented decision records that landed since the last tag — that is what the bucket is for, and a second place to write them is the duplicate-source mistake this repository keeps -finding its way back to. +finding its way back to. There are **no GitHub Release objects**: the annotated +tag's own message is the note, which is how 0.5.0 and 0.6.0 shipped. ## The steps @@ -40,8 +41,6 @@ finding its way back to. 4. **Re-run the quickstart the way a stranger does** — `curl` the published compose, `docker compose up`, `bash scripts/demo.sh`. The file's own comment records three releases that were wrong until somebody actually ran it. -5. **A GitHub Release** whose body is the notes since the last tag, linking the - tag. The images already exist; this is the human-facing index of them. ## A note on tags vs. digests diff --git a/docs/zh/index.md b/docs/zh/index.md index da76c5f7..e61f7a65 100644 --- a/docs/zh/index.md +++ b/docs/zh/index.md @@ -20,7 +20,16 @@ description: 让 agent 值班，但不把钥匙交出去 —— 管道给每条 ## 从这里开始 - **30 秒：**[docs/quickstart.md](../quickstart.md) 开头就讲清了它的形状 —— 三个服务，一张图。 -- **10 分钟，不要 key，不花钱：** 同一页的第二段会在排练上跑完整条闭环：告警被判定，调查经过真实的只读门回放，卡片，按下批准，告警恢复。 +- **10 分钟，不要 key，不花钱：** 管道 + 判官 + 桩模型 + 跑排练的调查员 + 可读的接收端，只要有 Docker。 + + ```bash + curl -fsSLO https://raw.githubusercontent.com/itswl/hookstack/main/docker-compose.quickstart.yml + docker compose -f docker-compose.quickstart.yml up -d + bash <(curl -fsSL https://raw.githubusercontent.com/itswl/hookstack/main/scripts/demo.sh) + ``` + + 演示会跑完整条闭环：告警被判定，一次录好的调查经过真实的只读门回放，报告变成卡片，按下批准后跑两条允许清单里的命令，告警恢复。[docs/quickstart.md](../quickstart.md) + 里有其余内容 —— 换成真实模型、三块看板、半天的路径。 - **半天，真上：**[STACK.md](https://github.com/itswl/hookstack/blob/main/STACK.md) 从源码一步步跑起三个服务；[docs/containment.md](../containment.md) 是指向真实集群**之前**要读的边界地图。 - **完整功能清单：**[docs/what-you-get.md](../what-you-get.md)（英文）。 From d6dbff9449fd0b58079df88b90677097f584c913 Mon Sep 17 00:00:00 2001 From: itswl Date: Mon, 12 Oct 2026 23:36:44 +0800 Subject: [PATCH 4/5] locks: put the two new pins where relock.sh would MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `pytest-cov` sat before `pytest-asyncio` in two locks, and the next `scripts/relock.sh` would reorder them. Position only — no version moves. The investigator's lock has no `pytest-asyncio`, so its order was already right. --- hookjudge/requirements.lock | 2 +- hookrelay/requirements.lock | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/hookjudge/requirements.lock b/hookjudge/requirements.lock index db89f966..860b4593 100644 --- a/hookjudge/requirements.lock +++ b/hookjudge/requirements.lock @@ -42,8 +42,8 @@ pydantic_core==2.46.4 Pygments==2.21.0 pyparsing==3.3.2 pytest==9.1.1 -pytest-cov==7.1.0 pytest-asyncio==1.4.0 +pytest-cov==7.1.0 python-dotenv==1.2.3 PyYAML==6.0.3 requests==2.34.2 diff --git a/hookrelay/requirements.lock b/hookrelay/requirements.lock index 38161790..68ce6cc4 100644 --- a/hookrelay/requirements.lock +++ b/hookrelay/requirements.lock @@ -42,8 +42,8 @@ pydantic_core==2.46.4 Pygments==2.21.0 pyparsing==3.3.2 pytest==9.1.1 -pytest-cov==7.1.0 pytest-asyncio==1.4.0 +pytest-cov==7.1.0 python-dotenv==1.2.3 PyYAML==6.0.3 requests==2.34.2 From 1dfd449efd9a238d9bc0261b7ff0338c553112e5 Mon Sep 17 00:00:00 2001 From: itswl Date: Mon, 12 Oct 2026 23:39:14 +0800 Subject: [PATCH 5/5] ware: the pipe's ceiling moves 6,400 -> 6,450 for the peek-then-claim fix MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Splitting the replay cache into a read before the pipeline and a claim after it answers is thirty-odd source lines the ceiling set earlier in this branch does not have. Raising is the sanctioned path; trimming the docstring that explains WHY a peek and a claim are two calls is not, so this adds a second entry for the same day with its split (+~30 source, +~8 code) and the same doctrine answer as the entry above — the door's business, no payload read, no judgment about an alert. hookrelay/README.md and both front pages state the new number. --- README.md | 2 +- README.zh-CN.md | 2 +- hookrelay/README.md | 4 ++-- scripts/assert_weight.py | 12 +++++++++++- 4 files changed, 15 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 27b8f7d8..18f99e81 100644 --- a/README.md +++ b/README.md @@ -64,6 +64,6 @@ The same pipe also carries an operator's own work signals to a planner and, on a ## Developing -`bash scripts/gate.sh` runs every service's gate plus the stack checks. Read its verdict before you commit. The pipe caps itself at 6,400 source lines and the judge at 3,400; the investigator is uncapped. +`bash scripts/gate.sh` runs every service's gate plus the stack checks. Read its verdict before you commit. The pipe caps itself at 6,450 source lines and the judge at 3,400; the investigator is uncapped. MIT licensed. diff --git a/README.zh-CN.md b/README.zh-CN.md index 911ddaa2..4c06ecef 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -60,6 +60,6 @@ ## 开发 -`bash scripts/gate.sh` 会跑每个服务的 gate 和整个栈的检查。提交前读它的结论。管道的源码上限是 6,400 行，判官 3,400 行，调查员不设上限。 +`bash scripts/gate.sh` 会跑每个服务的 gate 和整个栈的检查。提交前读它的结论。管道的源码上限是 6,450 行，判官 3,400 行，调查员不设上限。 MIT 协议。 diff --git a/hookrelay/README.md b/hookrelay/README.md index a8f7e9f3..a8297a22 100644 --- a/hookrelay/README.md +++ b/hookrelay/README.md @@ -8,12 +8,12 @@ workflow are its own. Receive webhooks. Decide. Fan out to channels. Nothing else. -A pluggable router (under 6,400 source lines, five dependencies) that takes JSON +A pluggable router (under 6,450 source lines, five dependencies) that takes JSON webhooks in at one door, walks each event through three named gates, and delivers to a chat bridge / generic HTTP (DingTalk and WeCom as shipped plugins) — with retries, per-channel rate limits, and a dead-letter queue you can see. -Both numbers are **budgets, not descriptions**. 6,400 source lines is the +Both numbers are **budgets, not descriptions**. 6,450 source lines is the ceiling and five dependencies is the count; `scripts/assert_weight.py` enforces the first alongside the other stack checks, and crossing it is meant to cost a conversation rather than a commit. Tests are counted and printed but never capped diff --git a/scripts/assert_weight.py b/scripts/assert_weight.py index 6b9dac9b..bed40c8d 100755 --- a/scripts/assert_weight.py +++ b/scripts/assert_weight.py @@ -306,7 +306,17 @@ # as the dedup fingerprint beside it. The rest of the raise is the docstring # that says so; `docs/containment.md` used to read "there is no nonce cache", # and every line that changed that sentence lives next to the code that did. - "hookrelay": (6400, Path("hookrelay/README.md")), + # 6400 -> 6450 on 2026-10-12, in review, for the peek-then-claim split the + # first version got wrong: the row is now READ before the pipeline + # (`signature_seen`) and CLAIMED only after it answers (`claim_signature`), so + # a retry following a 5xx or the fuse's 429 is processed instead of folded — + # the up-front claim lost the event while both sides believed it handled. + # Split: +~30 source, +~8 code, the rest the docstring saying why a peek and a + # claim are two calls. Still the door's business and still no payload read, so + # the doctrine answer is the entry above with no new argument; two raises in + # one day is the ceiling working, not a formality — each needed a reason + # written down, and neither added a judgment about an alert's worth. + "hookrelay": (6450, Path("hookrelay/README.md")), # 2900 -> 3000 on 2026-08-21, for the judge's second axis (`wake_someone`). # Raised rather than trimmed because the thing that pushed it over is the one # measurement that says whether this service earns its model calls at all: