Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
1b88f61
docs: design for a reproducible end-to-end lab in Docker
BenjiTrapp Nov 15, 2026
a12577f
docs: the three Linux defects the lab's CI job uncovered
BenjiTrapp Nov 16, 2026
50b8ef0
docs: close the gap too -- a rescue nobody can see
BenjiTrapp Nov 16, 2026
7075b29
docs: plan for the Linux end-to-end lab and the three defects
BenjiTrapp Nov 16, 2026
b687ec0
docs: two plan corrections the pre-flight scan found
BenjiTrapp Nov 16, 2026
1165329
fix(stager): the Unix one-liner could not run on Linux at all
BenjiTrapp Nov 16, 2026
1876a2b
docs: correct the one-liner's shape and size where comments still des…
BenjiTrapp Nov 16, 2026
050a0ef
test: make the argv test cover the stager and say what the printf tes…
BenjiTrapp Nov 16, 2026
9e22826
fix(relay): stop() left the port accepting on Linux
BenjiTrapp Nov 16, 2026
7ec5f01
feat(rescue): a rescue nobody can claim is no longer silent
BenjiTrapp Nov 16, 2026
9971bee
feat(rescue): the agent sees the rescues it cannot act on
BenjiTrapp Nov 16, 2026
63a4ecf
fix(rescue): an unaskable server is not 'no rescues'; show the finger…
BenjiTrapp Nov 16, 2026
7e5a3e3
test(rescue): pin the premise that an unrouted GET answers 405
BenjiTrapp Nov 16, 2026
d83cc1e
feat(rescue): the operator is told when something unknown is stuck
BenjiTrapp Nov 16, 2026
69a9ca5
fix(rescue): several unseen rescues in one poll are one notification
BenjiTrapp Nov 16, 2026
30bcef0
test(lab): the containers and networks for a Linux end-to-end run
BenjiTrapp Nov 16, 2026
4ff2111
docs: correct two stale claims in the lab design
BenjiTrapp Nov 16, 2026
7469e77
test(lab): a driver, and the rescue scenario through a real resolver
BenjiTrapp Nov 16, 2026
b6f610d
test(lab): the scenario the VM died before reaching
BenjiTrapp Nov 16, 2026
8db943e
ci: run the suite and the lab on Linux
BenjiTrapp Nov 16, 2026
42777b5
test(ci): three guards asserted against prose, not against the workflow
BenjiTrapp Nov 16, 2026
b20244c
docs: the 56 rulings this branch was built on
BenjiTrapp Nov 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
159 changes: 159 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
# This repository had no CI at all until this file, and the reason it has
# one now is a measurement: four defects shipped in the DNS rescue channel
# and a 3528-test suite found none of them. Two of the four could not
# surface on macOS at all -- a dead QUIC endpoint raises on Darwin and
# merely times out on Linux, and Linux refuses an argv entry over 131072
# bytes where Darwin has no such limit. The suite had never run on Linux.
#
# Two jobs, not one. They fail for different reasons, and a reader looking
# at a red run should be able to tell which broke without opening it: the
# suite is the primary guard, and the lab covers what unit tests
# structurally cannot.
#
# Neither job takes a secret. The lab generates its AES key per run with
# os.urandom, and a workflow that referenced `secrets.` would be a standing
# invitation to add one.
name: CI

on:
push:
branches: ["**"]
pull_request:
workflow_dispatch:

permissions:
contents: read

jobs:
# ── The suite ──
#
# Python 3.9 because software/tests/requirements.txt says the suite runs
# in a 3.9 venv and has only ever run against it. Testing a version
# nobody develops against would report failures nobody asked for.
tests:
name: suite (python 3.9, linux)
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- uses: actions/checkout@v4
with:
# The requirements-floor and leak-walk tests shell out to git and
# read what is TRACKED, so a shallow clone with no history is
# fine but a detached blobless one is not.
fetch-depth: 1

- uses: actions/setup-python@v5
with:
python-version: "3.9"

# ── Three packages, each with a measured consequence ──
#
# Without them the suite reports 19 errors and 6 failures that say
# nothing whatever about the code. Measured in a python:3.9-slim
# container before this file existed:
#
# git the requirements-floor and leak-walk tests shell out to
# it; 4 errors without it
# ffmpeg the figure and GIF tools encode with it; 9 errors and
# 2 failures
# procps `ps`, which the supervisor and tunnel-spawn tests read a
# child's command line with; 5 errors and 1 failure
#
# Delete one and the run goes red for a reason that looks like a
# product fault. That is why each is named here.
- name: System packages the suite shells out to
run: |
sudo apt-get update -qq
sudo apt-get install -y -qq --no-install-recommends \
git ffmpeg procps bash-completion util-linux

# ── node is not optional ──
#
# Several tests are @skipUnless(shutil.which("node")), including the
# operator console's notification dedupe, batching and escaping,
# which run the page's real JavaScript. Without node they SKIP
# SILENTLY and the job still reports green over logic it never
# executed -- which is the exact failure mode this whole workflow
# exists to prevent.
#
# Measured rather than asserted, with node hidden from PATH and
# ffmpeg, git and ps left in place: test_c2_hardening,
# test_dns_rescue and test_brand_assets run 294 tests and report
# `OK` either way -- 0 skips with node, **17 skips without**, seven
# of them the notification tests above. Green both times.
#
# So node is installed deliberately, and the step below proves it is
# on PATH, which turns a failed setup into a loud error instead of a
# quiet gap.
- uses: actions/setup-node@v4
with:
node-version: "20"

- name: Prove the environment provides what the suite needs
run: |
set -eu
for tool in git ffmpeg ps node setsid; do
command -v "$tool" >/dev/null || { echo "missing: $tool" >&2; exit 1; }
done
python3 -c "import sys; assert sys.version_info[:2] == (3, 9), sys.version"
echo "arch: $(uname -m), $(uname -s)"

- name: Install the suite's dependencies
run: python3 -m pip install -r software/tests/requirements.txt

# The documented invocation, unchanged: pytest is not installed and
# the suite is not written for it.
- name: The suite
run: python3 -m unittest discover -s software/tests -t .

# On Linux the suite runs MORE than it does on a developer's Mac: the
# shell-session tests that skip on Darwin for want of setsid(1) take
# the real path here, and their own skip message says the fallback
# "really has no ^C" on the deployment target. A count that silently
# dropped would hide a module that stopped importing, which has
# happened here before -- 26 reported errors once hid 968 tests that
# never ran.
- name: Report what actually ran
if: always()
run: |
python3 -m unittest discover -s software/tests -t . 2>&1 \
| tail -3 || true

# ── The lab ──
#
# The integration proof: a beacon whose transports fail because they do
# not route, through a real recursive resolver, on Linux. Separate from
# the suite because it needs Docker and because a red lab means something
# different from a red suite.
lab:
name: end-to-end lab (docker, linux)
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4

- uses: actions/setup-python@v5
with:
python-version: "3.9"

# The driver detects which compose spelling exists rather than
# hardcoding one: `docker compose` is the plugin form these runners
# ship, while the project's development machine has only the
# standalone `docker-compose`. Both were measured.
- name: What docker and compose are here
run: |
docker --version
docker compose version || docker-compose version
echo "arch: $(uname -m)"

- name: Build the lab's images
run: |
docker compose -p raccoon-lab \
-f software/tests/lab/docker-compose.yml --profile smb build

# Exit 2 means "I could not run" and is distinct from 1, "it failed".
# If Docker were missing the driver would say so and exit 2 rather
# than failing in a way that looks like a product fault -- but on
# this runner it is present, so a 2 here is itself a finding.
- name: Every scenario
run: python3 tools/e2e_lab.py
42 changes: 28 additions & 14 deletions docs/TUNNEL.md
Original file line number Diff line number Diff line change
Expand Up @@ -484,9 +484,9 @@ check — and not about the package being present.

## Staging onto a host with no Python

The ordinary artifact is `python3 -c "…"`, and a one-liner cannot install
Python. The second artifact shape from the same generator is a **stager**
— a POSIX shell script and a PowerShell script — which:
The ordinary artifact is `printf %s "…" | python3 -`, and a one-liner
cannot install Python. The second artifact shape from the same generator
is a **stager** — a POSIX shell script and a PowerShell script — which:

1. **probes** for a usable interpreter and, finding one, runs the
ordinary one-liner and stops there;
Expand Down Expand Up @@ -514,16 +514,28 @@ Then, as an operator step rather than something the stager does:
only when it finds nothing.

**On Windows the stager is the route, and the inline one-liner is not.**
At roughly 115 KB the generated `python -c "…"` exceeds both the 8191
character `cmd.exe` command line and the 32767 character `CreateProcess`
limit, so it cannot be delivered on that platform at any quoting. It is
still emitted, because it is the text the Windows stager runs through
`Invoke-Expression` when its probe finds a Python — where there is no
length limit — and because an operator may paste it into something that
does not go through `cmd.exe`. Both one-liners are therefore
double-quoted with single quotes inside, and the stage carries no `$`,
backtick, backslash or `"`, so one text survives `sh -c`, `cmd.exe` and
`Invoke-Expression` alike.
At roughly 135 KB (135,453 characters with one layer, about 138 KB at
the default of three, up to 143 KB at six) the generated `python -c "…"`
exceeds both the 8191 character `cmd.exe` command line and the 32767
character `CreateProcess` limit, so it cannot be delivered on that
platform at any quoting. It is still emitted, because it is the text the
Windows stager runs through `Invoke-Expression` when its probe finds a
Python — where there is no length limit — and because an operator may
paste it into something that does not go through `cmd.exe`. Both
one-liners are therefore double-quoted with single quotes inside, and the
stage carries no `$`, backtick, backslash or `"`, so the same stage
survives `sh`, `cmd.exe` and `Invoke-Expression` alike.

**On Linux the payload cannot be an argument at all.** The kernel refuses
any single `argv` entry over 131072 bytes (`MAX_ARG_STRLEN`), and the
generated stage is larger, so `python3 -c "…"` fails to exec with
`Argument list too long` however it is quoted. macOS has no such limit,
which is why this went unnoticed. The Unix one-liner is therefore
`printf %s "…" | python3 -`: the payload travels on python's stdin, and
`printf` is a builtin of dash, bash and busybox ash, so nothing execs it
and the artifact is still one line. `%s` keeps the stage out of the format
string, so a `%` inside it is data. The POSIX stager carries the same
text through `eval '…'`, which is why it needed no change.

### The dead-drop list, and the hash that has no override

Expand Down Expand Up @@ -584,7 +596,9 @@ source. Point it at a drop and it will fetch, verify and unpack whatever
is there; nothing in this repository produces that file for a POSIX host.
Its value today is its **probe-and-delegate** path: on a host that has a
Python it runs the ordinary one-liner, which is the common case on Linux
and macOS.
and macOS. That one-liner is the `printf … | python3 -` form because a
single `argv` entry over 131072 bytes cannot be exec'd on Linux (see
above); a `python3 -c "…"` form would make this path fail there.

**The stager installs no persistence**, although an earlier design step
said it would. The per-build autorun name, the gate and the payload name
Expand Down
Loading
Loading