docs: reconcile 43 markdown files with the current repository state, and gate diagram/link rot

.github/workflows/quality.yml

@@ -1589,6 +1589,32 @@ jobs:
             home: true
             run: python scripts/check-docs-reachable.py
 
+          # #3395: #2458 scans only README.md and docs/**, so the component
+          # trees under arcane/**, analysis/**, sandbox/** and branding/** were
+          # never link-checked -- the agent-intrusion-corpus README's dead
+          # `../../docs/...` hop (two levels short of a five-deep path) lived
+          # there unnoticed. Whole-tree relative-link resolution. Fenced blocks
+          # are skipped: their paths belong to the reader's project, not ours.
+          - name: Every tracked doc's relative links resolve (#3395)
+            home: true
+            run: python scripts/check-doc-links.py
+
+          # #3395: two of the forty mermaid diagrams in the tree did not parse
+          # at all -- AI_TRIAGE.md named a node `call` (a reserved mermaid
+          # token) and ghidra/README.md left a colon unquoted inside an edge
+          # label. Both rendered as an error box on GitHub and nothing noticed,
+          # so mermaid parsing is now a gate rather than a review habit. Renders
+          # every block in one headless browser; mermaid is resolved from the
+          # npx cache, so the step is self-bootstrapping (no pinned version to
+          # drift out of date with the docs). The warm-up below is what makes
+          # that true: a fresh runner's npx cache is empty, so the lookup inside
+          # check-mermaid.mjs finds nothing and the gate cannot run at all.
+          - name: Mermaid diagrams parse (#3395)
+            home: true
+            run: |
+              npx -y @mermaid-js/mermaid-cli --version
+              node scripts/check-mermaid.mjs
+
           # #2576: the CAPE implementation-plan doc claimed the detail page
           # (/cape/{sha256}) was admin-gated. Nothing in backend-service
           # enforces that -- require_service_token is the actual gate on

README.md

@@ -25,9 +25,12 @@ flowchart LR
   wg --> home["home APIARY stacks<br/>@ 10.8.0.2"]
 ```
 
-**All core sensors run without compose profiles.** The only profile is the
-optional on-demand `geoip-update` maintenance job. 38 deployment pieces —
-32 independent Arcane-managed stacks under `arcane/home/` plus 6 more at
+**All core sensors run without compose profiles.** The only profiles are the
+optional on-demand maintenance jobs `geoip-update` and `threat-intel` (both in
+`honeypot-init`); the four `["legacy"]` worker stacks are defined for rollback
+but do not run, and `sandbox/ghosts`'s `["test"]` client is not a sensor.
+39 deployment pieces —
+33 independent Arcane-managed stacks under `arcane/home/` plus 6 more at
 their own repository-root paths, all at home, plus the VPS (see
 [docs/ARCANE-GIT-SYNC.md](docs/ARCANE-GIT-SYNC.md) for how a repo commit
 reaches the live host, and [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for
@@ -38,16 +41,17 @@ why the home side split into this many Compose stacks):
 | `honeypot-keycloak` ([arcane/home/honeypot-keycloak/compose.yml](arcane/home/honeypot-keycloak/compose.yml)) | **home** | Arcane-managed Keycloak/PostgreSQL identity stack; only Keycloak is reachable from VPS Traefik over WireGuard |
 | `honeypot-init` ([arcane/home/honeypot-init/compose.yml](arcane/home/honeypot-init/compose.yml)) | **home** | one-shot bootstrap jobs: log paths, Elasticsearch templates, Arkime schema, persona validation |
 | `honeypot-cowrie`, `honeypot-dionaea`, `honeypot-conpot`, `honeypot-dnp3`, `honeypot-http`, `honeypot-multipot` (`arcane/home/honeypot-<name>/compose.yml`, one directory each) | **home** | the sensors: Cowrie, Dionaea (+ TFTP relay), Conpot personas, DNP3, HTTP/API honeypots, multipot |
-| `honeypot-dicompot`, `honeypot-dns-honeypot`, `honeypot-citrix`, `honeypot-cisco-asa`, `honeypot-rdp`, `honeypot-endlessh`, `honeypot-beelzebub`, `honeypot-hellpot`, `honeypot-elasticpot`, `honeypot-galah`, `honeypot-sentrypeer`, `honeypot-mailoney` (`arcane/home/honeypot-<name>/compose.yml`, one directory each) | **home** | more sensors: DICOM medical-imaging decoy, DNS UDP reflection bait (response-capped, never a real amplification vector), Citrix ADC/NetScaler Gateway decoy (CVE-2019-19781), Cisco ASA WebVPN+IKE decoy (CVE-2018-0101), RDP decoy, SSH pre-auth tarpit, vendored multi-protocol deception runtime (SSH/LDAP/MCP/HTTP, #1418), vendored HTTP bot tarpit (#1419), vendored Elasticsearch decoy distinct from multipot's own (#1423), vendored LLM-powered HTTP honeypot behind its own broker-guarded bridge onto the shared Ollama instance (#1420), vendored SIP/VoIP fraud-detection honeypot (#1424), vendored SMTP honeypot taking over port 25 from multipot's own retired handler (#1422) — the row's `honeypot-wordpot` / WordPress/CMS decoy slot was removed when wordpot retired (#2381) |
+| `honeypot-dicompot`, `honeypot-dns-honeypot`, `honeypot-citrix-honeypot`, `honeypot-cisco-asa-honeypot`, `honeypot-sonicwall-sma`, `honeypot-rdp-honeypot`, `honeypot-endlessh`, `honeypot-beelzebub`, `honeypot-hellpot`, `honeypot-elasticpot`, `honeypot-galah`, `honeypot-sentrypeer`, `honeypot-mailoney` (`arcane/home/honeypot-<name>/compose.yml`, one directory each) | **home** | more sensors: DICOM medical-imaging decoy, DNS UDP reflection bait (response-capped, never a real amplification vector), Citrix ADC/NetScaler Gateway decoy (CVE-2019-19781), Cisco ASA WebVPN+IKE decoy (CVE-2018-0101), SonicWall SMA1000 Work Place/AMC decoy (CVE-2026-83548 Work Place SSRF), RDP decoy, SSH pre-auth tarpit, vendored multi-protocol deception runtime (SSH/LDAP/MCP/HTTP, #1418), vendored HTTP bot tarpit (#1419), vendored Elasticsearch decoy distinct from multipot's own (#1423), vendored LLM-powered HTTP honeypot behind its own broker-guarded bridge onto the shared Ollama instance (#1420), vendored SIP/VoIP fraud-detection honeypot (#1424), vendored SMTP honeypot taking over port 25 from multipot's own retired handler (#1422) — the row's `honeypot-wordpot` / WordPress/CMS decoy slot was removed when wordpot retired (#2381) |
 | `honeypot-canarytokens` ([arcane/home/honeypot-canarytokens/compose.yml](arcane/home/honeypot-canarytokens/compose.yml)) | **home** | self-hosted honeytoken platform (#1426) -- planted-artifact deception, not a listening protocol decoy; `canarytokens-adapter` translates its webhook alerts into this repo's shared JSON event shape. The dashboard's Settings > Canarytokens pane (#1487) creates PDF/Word/Excel/custom-image/Windows-Folder/QR tokens on demand for use *outside* this honeypot (#1662 dropped the stale design doc that described the pre-cutover plan; the shipped pane is authoritative) |
 | `honeypot-tanner` ([arcane/home/honeypot-tanner/compose.yml](arcane/home/honeypot-tanner/compose.yml)) | **home** | SNARE + TANNER application-emulation boundary |
 | `honeypot-elk` ([arcane/home/honeypot-elk/compose.yml](arcane/home/honeypot-elk/compose.yml)) | **home** | Filebeat, Elasticsearch, Kibana, EveBox, Arkime |
-| `honeypot-agent-intrusion-worker` ([arcane/home/honeypot-agent-intrusion-worker/compose.yml](arcane/home/honeypot-agent-intrusion-worker/compose.yml)) | **home** | correlates sensor/Suricata events into campaigns, scores them against deterministic criticality rules, writes the `agent-intrusion-campaigns` index the dashboard's `/agent-campaigns` route reads |
-| `honeypot-attacker-identity-worker`, `honeypot-correlator-worker`, `honeypot-payload-inventory-worker` (`arcane/home/honeypot-<name>/compose.yml`, one directory each) | **home** | three more workers that had their own top-level compose file but had drifted out of the deploy/installer inventory before #1502's audit caught it (same class of gap #560 and #891 each fixed once before) -- attacker-identity correlation, cross-sensor campaign correlation, and payload inventory tracking |
+| `honeypot-agent-intrusion-worker` ([arcane/home/honeypot-agent-intrusion-worker/compose.yml](arcane/home/honeypot-agent-intrusion-worker/compose.yml)) | **home** | the labelled corpus plus the Tier 1 contract benchmark. The worker itself was ported to Rust in #1610 and now runs as `WORKER_LOOPS=agent-intrusion` inside `honeypot-dashboard`'s `backend-worker`; the Python stack is retained under the `legacy` profile for rollback only, and the live `agent-intrusion-campaigns` index is written by the Rust loop |
+| `honeypot-attacker-identity-worker`, `honeypot-correlator-worker`, `honeypot-payload-inventory-worker` (`arcane/home/honeypot-<name>/compose.yml`, one directory each) | **home** | three more workers that had their own top-level compose file but had drifted out of the deploy/installer inventory before #1502's audit caught it (same class of gap #560 and #891 each fixed once before) -- attacker-identity correlation, cross-sensor campaign correlation, and payload inventory tracking. All three were retired by the same #1649 pass as the agent-intrusion worker: #1610 ported them to Rust, where they now run as `WORKER_LOOPS=attacker-identity` and `WORKER_LOOPS=correlator` on `honeypot-dashboard`'s `backend-worker` and `WORKER_LOOPS=payload-inventory` on `backend-worker-payload-inventory`. Like row above, the Python stacks are kept under the `legacy` profile for rollback only and are not the live writers |
 | `honeypot-dashboard` ([arcane/home/honeypot-dashboard/compose.yml](arcane/home/honeypot-dashboard/compose.yml)) | **home** | the live investigation dashboard: TanStack Start frontend (`dashboard-next`) in front of the Rust axum `backend-service` API tier, plus its worker containers (importer, networkless enrichment, the `WORKER_LOOPS` aggregation loops) and the services-adapter Docker control surface. Live since #1628's cutover completed 2026-08-22 -- the Go dashboard is deleted; see [docs/DASHBOARD-CUTOVER.md](docs/DASHBOARD-CUTOVER.md) and [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) |
-| `honeypot-dashboard-backend` ([arcane/home/honeypot-dashboard-backend/compose.yml](arcane/home/honeypot-dashboard-backend/compose.yml)) | **home** | the write-capable, host-mounted `backend-service` instance (:8082), split out from `honeypot-dashboard` by #1622 -- same route table plus the analysis request-spool mounts; only this instance can dispatch `analysis/ghidra`/sandbox jobs |
+| `honeypot-dashboard-backend` ([arcane/home/honeypot-dashboard-backend/compose.yml](arcane/home/honeypot-dashboard-backend/compose.yml)) | **home** | the unprivileged read-only `backend-service` API tier (:8081), split out from `honeypot-dashboard` by #1622 so Arcane can redeploy the API tier without touching `dashboard-next`; the write-capable, host-spool-mounted instance is `backend-service-mounted` (:8082), which stayed in `honeypot-dashboard` |
 | `honeypot-payload-analysis` ([arcane/home/honeypot-payload-analysis/compose.yml](arcane/home/honeypot-payload-analysis/compose.yml)) | **home** | payload dedup + YARA scanning |
 | `honeypot-utilities` ([arcane/home/honeypot-utilities/compose.yml](arcane/home/honeypot-utilities/compose.yml)) | **home** | autoheal, log rotation, disk-space monitoring, reporting |
+| `unsloth` ([arcane/home/unsloth/compose.yml](arcane/home/unsloth/compose.yml)) | **home** | Unsloth Studio, the interactive leg of the round-7 training work area (#3080); operator starts and stops it in Arcane so it can release VRAM between cold-benchmark legs |
 | [`vps/`](vps/) | **VPS** | Traefik, portbridge raw tunnels, Suricata, WireGuard HTTP bridges, and isolated Keycloak OIDC gateways |
 
 Every stack above is a directory-aware Arcane Git sync driven by
@@ -90,7 +94,7 @@ for where those fit.
 | [docs/BACKUP-ESSENTIALS.md](docs/BACKUP-ESSENTIALS.md) | What is backed up so the stack can be rebuilt, where the three copies go, and the full restore procedure |
 | [scripts/install.sh](scripts/install.sh) | Single entry point for host provisioning — `sudo ./scripts/install.sh --profile home\|vps`, which dispatches to the installer below (or [scripts/install-vps.sh](scripts/install-vps.sh)) with that profile's answers file. Both share their retry/step/logging framework via [scripts/lib/install-common.sh](scripts/lib/install-common.sh) ([#1609](https://github.com/Xore/APIARY/issues/1609)) |
 | [scripts/install-homeserver.sh](scripts/install-homeserver.sh) | Unattended provisioning script (Docker, GPU/NVIDIA, WireGuard, Arcane, the stacks themselves) for a manually-installed base Ubuntu system — fill in [scripts/install-homeserver.conf.example](scripts/install-homeserver.conf.example) first, same idea as a Windows `autounattend.xml` answer file. First cut, see [#518](https://github.com/Xore/APIARY/issues/518) |
-| [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | System architecture and data flow — trust boundaries, container map, event ingestion, correlation/enrichment (p0f, HASSH/JA3/JA4, GeoIP), payload lifecycle, sandbox detonation, evidence types (6 diagrams) |
+| [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | System architecture and data flow — trust boundaries, container map, event ingestion, correlation/enrichment (p0f, HASSH/JA3/JA4, GeoIP), payload lifecycle, sandbox detonation, evidence types (4 diagrams) |
 | [docs/SENSORS.md](docs/SENSORS.md) | The sensor table, resource budgets, investigation UIs, SNARE+TANNER, Suricata, Arkime, and how real attacker IPs survive the tunnel |
 | [docs/OPERATIONS.md](docs/OPERATIONS.md) | Persona inventory, the seeded cowrie filesystem, GeoIP, and how to actually read the data (dashboard, Kibana, Arkime, backups) |
 | [docs/ip-reporting-plan.md](docs/ip-reporting-plan.md) | Defensive IP-blocklist reporting (AbuseIPDB/Blocklist.de), dry-run by default |
@@ -103,7 +107,7 @@ for where those fit.
 | [docs/CONTAINER-UPDATES.md](docs/CONTAINER-UPDATES.md) | How to check pinned images for updates, assess compatibility, verify empirically, and pin by digest |
 | [docs/TESTING.md](docs/TESTING.md) | The three testing tiers -- CI, live feature smoke tests, and the full clean-reinstall release gate -- and how to repeat each one |
 | [docs/STACK-REBUILD.md](docs/STACK-REBUILD.md) | Runbook for a full deliberate reset — stop order, what's preserved vs wiped, and the ordering/permission pitfalls to avoid |
-| [deploy-profiles/](deploy-profiles/) | Named deployment shapes (full / ICS-only / web-only) — which of the 20 split home stacks run for a given deployment, plus a validator catching cross-stack drift before deploy |
+| [deploy-profiles/](deploy-profiles/) | Named deployment shapes (full / ICS-only / web-only) — which of the 26 split home stacks run for a given deployment, plus a validator catching cross-stack drift before deploy |
 | [docs/RECOVERY.md](docs/RECOVERY.md) | `factory-reset.sh` — one entry point for "back up, optionally wipe/reset, restart" on the same host |
 | [docs/ROADMAP.md](docs/ROADMAP.md) / [docs/WORK-LEDGER.md](docs/WORK-LEDGER.md) | What order work happens in, and how issues are claimed/reviewed |
 | [docs/ml-worker-plan.md](docs/ml-worker-plan.md), [docs/gpu-llm-analysis-worker.md](docs/gpu-llm-analysis-worker.md), [docs/gpu-ml-worker-acceleration.md](docs/gpu-ml-worker-acceleration.md) | The homeserver's NVIDIA GPU running local LLM log/payload analysis and CUDA-accelerated anomaly detection — no data leaves the machine |

SECURITY.md

@@ -13,3 +13,12 @@ live malware, private keys, production `.env` files, packet captures containing
 private traffic, or unredacted logs to a public issue.
 
 Supported security fixes target the current `main` branch.
+
+`scripts/check-public-leaks.py` enforces the "leak a real secret" half of this
+policy on every change, and fails CI on private keys, GitHub/AWS/Slack tokens,
+literal credential assignments, credentials embedded in URLs, deployment
+`.env` files, private-key and packet-capture binaries, and the
+deployment-specific addresses and default password this repository must never
+name. Exactly one tracked `.env` is exempt, and it is the decoy honeyfs file
+under `arcane/home/honeypot-cowrie/` that exists for attackers to find — not a
+credential, and not something to "fix" by deleting.

arcane/home/honeypot-agent-intrusion-worker/analysis/agent-intrusion-corpus/README.md

@@ -15,7 +15,7 @@ original consumer and is retired) that renders its output — all proven
 against that one corpus rather than hand-built fixtures alone. The
 prerequisite research (mapping the campaign to APIARY's actual trust
 boundaries) is
-[`docs/agent-intrusion-threat-model.md`](../../docs/agent-intrusion-threat-model.md);
+[`docs/agent-intrusion-threat-model.md`](../../../../../docs/agent-intrusion-threat-model.md);
 phase 4's preventive-control audits (Dockerfile digest pinning, an
 assessed ARKIME secret-delivery finding) are documented there, not here,
 since they touch the wider repo rather than this directory. Phases 1-5
@@ -259,7 +259,8 @@ change reviewed content. To extend the corpus, add events directly to
 
 `corpus.jsonl` is not itself schema-versioned (no top-level `version`
 field) — the file's own git history is the version record, matching how
-this repo treats `analysis/yara/` and other reviewed-fixture directories.
+this repo treats `arcane/home/honeypot-payload-analysis/analysis/yara/` and
+other reviewed-fixture directories.
 A future breaking change to `schema.json` (a required field added/removed,
 an enum value changed) should bump `schema.json`'s own `$id` and add a
 note here, not silently reinterpret old corpus rows under a new meaning.