Repository navigation
A reproducible Linux end-to-end lab, and the three defects building it found - #6
Merged
Merged
Conversation
Awaiting approval; no implementation yet. Four defects shipped in the DNS rescue channel and the 3257-test suite found none of them. Two were platform defects that cannot show on macOS, and the VM that did show one lost its network mid-run. This describes a lab where the Linux end-to-end run is reproducible and cheap enough for CI. Severance is by topology: the beacon sits on a network the team server is not on, so HTTPS, SMB and QUIC fail because they do not route. Every constraint in the document was measured on this machine rather than assumed, including that container-to-container UDP works while published UDP on macOS does not -- which is why the beacon is a container. A builder check found three things the first draft got wrong, all now fixed in the document: a statically severed beacon can never be attributed (the implant assigns its agent id only in the HTTPS registration path), 'nothing published' would have left the driver unable to read the record it asserts on, and the smb-up scenario as drafted could not pass. It also surfaced a product gap the lab does not fix: unattributed_rescues is never served over HTTP, so a beacon that never registered is invisible to every operator.
Measured before writing the workflow, by running the suite in a Linux container on python 3.9. 3438 tests ran there against 3258 on macOS -- 180 that had never executed anywhere. Without git, ffmpeg and procps in the image: 19 errors, 6 failures, none of which say anything about the code. With them, three remain, and all three are real defects on the platform the implant targets. The Unix one-liner cannot be executed on Linux at all: the stage is 135,492 bytes and the kernel refuses any argv entry over 131,072 (measured exactly at the boundary). No quoting avoids it, and the POSIX stager's eval path hits the same wall. docs/TUNNEL.md calls that path the common case on Linux, which is false. And stop() on the NTLM relay closes a socket another thread is blocked in accept() on, which is undefined in POSIX: macOS wakes it, Linux leaves the port bound and accepting. The operator believes the relay is down while the target still has an open port. In scope by the operator's decision, sequenced first so they can be split out if the two worktrees working in that code conflict.
The operator brought the unattributed_rescues gap into scope. Four surfaces, because each reaches a different reader: a log event on arrival so it is never silent, an authenticated operator route, a separate key on the agent tool (separate because there is no beacon id to task and a model must not mistake one for something actionable), and a bell notification, because visible in a drawer nobody opens is not addressed. The notification is the only part that edits _OPERATOR_HTML, so its acceptance is a browser rather than a regex. A source-level assertion that a string is present is exactly what let escHtml(info.stuck.summary) ship and destroy the whole agent panel.
Nine tasks. The two platform fixes come first, in their own commits, so they can be split out if the two worktrees working in tunnel and implant code conflict. Then the four surfaces that make an unattributed rescue visible, then the lab's containers, its driver and the three scenarios, then CI. Each scenario must fail when its subject is reverted, and the plan names which commit to revert for two of them; the third is exempt because it guards a property of the lab rather than a product change, which is stated so its absence is not read as an oversight. The notification task's acceptance is a browser, not a regex. A source-level assertion that a string is present is what let escHtml(info.stuck.summary) ship and destroy the whole agent panel.
Task 2's test monkeypatched relay._server.shutdown. Measured: socket objects use __slots__ and reject attribute assignment, so the test could not have been written. Replaced with a recording proxy on the relay's attribute, which also pins the call order -- shutdown after close would fix nothing, and the monkeypatch left that to chance. Task 6 said to parse the compose file with yaml 'if available, otherwise a small reader'. PyYAML 6.0.3 is installed and software/requirements.txt already requires pyyaml>=6.0, so the reader is now forbidden. And a variable in Task 1 named 'stage' while holding the one-liner, which matters because the new form legitimately contains a quote and a pipe.
…cribed python3 -c
Both were wrong in my document, not in the code, and an implementer caught them by reading the design against the source. The image table said the beacon container installs requests. It installs nothing: beacon.py is standard-library only -- its HTTP session is _HttpSession over urllib and says so -- and an unused dependency would have contradicted the stdlib-only constraint this same document states. And the smb-up scenario spoke of giving the SMB listener a valid token. Only the DNS listener has that concept: listeners/dns.py:79 is the one constructor taking listener_token, and the shipped compose gives the variable to the DNS service alone.
The lab runs. On arm64 driving linux/aarch64 containers, with the compose form detected rather than assumed: the beacon registers for real while attached to c2-net, docker network disconnect severs it, and the driver confirms from INSIDE the beacon container that the team server's beacon socket, the QUIC address and the operator socket are all closed. So the legs fail because they do not route, not because a stub returned None. Evidence comes from the components. dnsmasq's own query log shows four tasking queries and exactly one rescue across four stuck cycles, 130 characters with a longest label of 48 -- both inside the DNS limits, checked against what actually crossed the resolver. The operator record carries the diagnosis. And it fails when it should: with the trigger reverted to `tasking is None`, zero rescues cross and the operator sees nothing. That is the defect a hand-kept VM found last week before losing its network; it is now one command. Four defects of my own, each found only by running it: _compose passed no env, so the generated key never reached compose and the server exited with its own correct complaint; the resolver's queries are in a file, as resolver.conf's comment says, not on stdout; the fleet key is agent_id and not id, which is the same distinction the rescue's attribution turns on; and python 3.9's argparse rejects the empty list nargs="*" produces when choices is set.
stale-token gives the DNS listener a token the server no longer accepts. Tasking over DNS needs it; the rescue does not, because it goes through the beacon handler. So the ladder runs all the way down, SMB and QUIC are really attempted, and the operator record reads SMB: unreachable; QUIC: unreachable instead of not tried. That is the fix from 3dad39e proved end to end on Linux, in the scenario a hand-kept VM lost its network before reaching. smb-up keeps that stale token on purpose. Disabling DNS would make 'no rescue was sent' vacuous: _maybe_rescue returns immediately when DNS is off, so the latch would never be armed and the absence would prove nothing. Here the resolver answers, the latch is armed and could fire, and it does not -- because an upstream leg delivered. Each scenario now declares what it claims, and a test refuses one that claims nothing. The expectations differ, which matters: the Task 7 check would have reported smb-up's correct 'no diagnosis' as a failure. One defect of my own, worth naming. The first smb-up check compared last_channel to 'smb'. A listener relays to the beacon handler over HTTP, so an SMB delivery lands as last_channel: http and that field cannot distinguish SMB from HTTPS at all -- the check printed 'expected smb', saw 'http', and passed anyway. A central claim proved by nothing, which is the same defect class I flagged three times in other people's work this week. It now rests on an invariant of the implant instead: a leg reaches _leg_status only when it FAILS, and the ladder is ordered. If SMB delivered, smb is absent from every cycle and quic was never reached; if SMB failed, both are there. Both directions are demonstrated -- removing the listener makes three independent assertions fire.
This repository had no CI at all. The reason it has some now is a
measurement: four defects shipped in the DNS rescue channel and a
3528-test suite found none of them, and two of the four could not surface
on macOS -- a dead QUIC endpoint raises on Darwin and merely times out on
Linux, and Linux refuses an argv entry over 131072 bytes where Darwin has
no such limit. The suite had never run on Linux.
Two jobs, because they fail for different reasons and a reader looking at
a red run should be able to tell which broke.
Each apt package carries its measured consequence: without git, ffmpeg
and procps the suite reports 19 errors and 6 failures that say nothing
about the code. Three packages with no reason beside them are three
packages the next person deletes.
node is installed deliberately, not incidentally. Several tests are
skipUnless(which('node')), including the operator console's notification
dedupe and batching, which run the page's real JavaScript -- without node
they skip SILENTLY and the job reports green over logic it never ran,
which is the exact failure mode this workflow exists to prevent. A
separate step proves the tools are present so a missing one is a loud
setup failure rather than a quiet gap. util-linux and bash-completion are
there for the same reason: on Linux the suite runs MORE than on a
developer's Mac, and one of the local skips says so itself.
Two defects of my own. CiTests.setUpClass raised SkipTest when the
workflow was absent, so deleting CI would have removed its own guard --
the same silent green it checks for. And the secret test forbade the
string 'secrets.', which the comment explaining why there are none then
failed; it now forbids the reference syntax, verified by adding a real
one and watching it fire.
Found by the final review, and the fifth defect of this class on this branch. The tests for node, the apt packages and the python version asserted against the workflow file's raw TEXT -- which contains the comments explaining why each matters. The words node, ffmpeg and 3.9 are all in that prose, so the tests stayed green when the step they describe was deleted. Measured: removing actions/setup-node left all ten CI tests passing. They now assert against the parsed structure: the setup-node step, the actual apt-get command line, the real python-version value. Two counter-checks confirm the guard fires -- removing the step reports 'no setup-node step', and removing ffmpeg from the apt line while keeping its comment reports 'named in prose but never installed'. That distinction is what this branch failed on four times before. And the workflow's central claim is now measured rather than argued. With node hidden from PATH and ffmpeg, git and ps left in place, three modules run 294 tests and report OK either way: 0 skips with node, 17 without, seven of them the operator console's notification tests. Green both times, which is exactly the silent-green this workflow exists to prevent. The numbers are in the file.
Every decision I took on the operator's behalf, in order, with what each costs if it is wrong. Committed rather than left in the git-ignored ledger, and rather than pasted into a chat window that cannot hold them: a ruling that reaches nobody was a decision made in secret. Eleven of them record a defect of my own. Five record the same class -- a test that asserts a claim I invented rather than a property of the code -- which is the class I flagged four times in other people's work this week. Each was caught by mutating the subject and watching the test, not by reading it.
This was referenced Nov 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four defects shipped in the DNS rescue channel (#4, #5) and a 3538-test suite
found none of them. Two were platform defects that cannot surface on
macOS. The VM that did show one lost its network mid-run.
This makes the Linux end-to-end run reproducible, cheap enough for CI, and
extensible to the transports that come later.
Design:
docs/superpowers/specs/2026-11-15-e2e-docker-lab-design.mdPlan:
docs/superpowers/plans/2026-11-16-e2e-docker-lab.mdThe 56 rulings this was built on:
docs/superpowers/plans/2026-11-16-e2e-docker-lab-rulings.mdThe lab
Severance is by topology: the beacon sits on a network the team server is
not on, so HTTPS, SMB and QUIC fail because they do not route.
dnsmasqis areal recursive resolver in front of the DNS listener, and
_DNS_PORTisuntouched — so the hop the rescue channel rests on is exercised, not bypassed.
The beacon registers for real first, because the implant assigns its agent
id only in the HTTPS registration path — a beacon that never registered
fingerprints its implant id and matches no fleet key. So the life-cycle is
work, then be cut off, which is also the only one that matters operationally.
Evidence comes from the components: dnsmasq's own query log, the operator API,
the beacon container's output. Never the driver's bookkeeping.
Three scenarios, each falsifiable.
rescue;stale-token, which the VMdied before reaching and which is the only one that proves the leg-recording
fix on Linux (
SMB: unreachable; QUIC: unreachable, not "not tried"); andsmb-up, which keeps the stale token on purpose, because disabling DNS wouldmake "no rescue was sent" vacuous.
Mutation-proved, not asserted: revert the rescue trigger to
tasking is None→ 0 rescues, exit 1. Remove the SMB listener → three independentassertions fire. Remove
actions/setup-node→ the CI guard fires.The three Linux defects building it found
MAX_ARG_STRLENof 131,072, measured at the boundary. No quoting avoids it, and the POSIX stager'sevalhits the same wall. The payload now travels on stdin through a shell builtin.stop()left the relay's port accepting on Linuxaccept()on is undefined in POSIX. macOS wakes it; Linux does not.shutdown()beforeclose()— and the twotryblocks are load-bearing, because macOS raisesENOTCONNon a listening socket.unattributed_rescueswas appended to, trimmed, and never served. Now: a log event, an authenticated route, a separate key on the agent tool, and one batched notification.CI, where this repository had none
Two jobs on
ubuntu-latest. Each apt package carries its measuredconsequence: without
git,ffmpegandprocpsthe suite reports 19 errorsand 6 failures that say nothing about the code.
node is not optional, and the claim is measured. With node hidden from
PATH and the rest left in place, three modules run 294 tests and report
OKeither way — 0 skips with node, 17 without, seven of them the operator
console's notification tests. Green both times. That is the silent-green this
workflow exists to prevent.
Notes for review
refused by the provider, so Tasks 6–9 were implemented and reviewed by the
same context, declared as self-reviews in the rulings document. A fresh
reader found this feature's two worst defects earlier; self-review did not.
test asserting an invented claim rather than a property of the code. The
worst: three CI guards asserted against the workflow's prose, so deleting
setup-nodeleft all ten green. Each was caught by mutation, not by reading.stale-tokenreportsunreachable, nottimed_out.After severance the name is unresolvable, so the leg raises — the silent
Linux timeout path is not exercised. Closing it needs a blackhole
address on
beacon-net.software/c2/docker-compose.ymlis untouched. The listener container is theshipped artefact, built from the shipped Dockerfile, and a test compares its
flags against the shipped compose so the lab cannot drift into proving a
shape nobody deploys.
3538 tests green; all three scenarios exit 0.