Skip to content

development.md

Latest commit

 

History

History
1088 lines (942 loc) · 58.8 KB

development.md

File metadata and controls

1088 lines (942 loc) · 58.8 KB
Raw

Development

This page tells how to build podssh, test it, measure it in a box like the target sandbox, and release it.

Requirements

  • Rust 1.99 or later, edition 2024: each crate declares the newest stable Rust of the day, as the crates of PodUtils/podcrates do (the operator, 2026-11-05; T-685). The gate runs the build image's stable Rust.
  • A C compiler for the binary. The SSH client uses aws-lc (aws-lc-sys, through russh). On Windows x86_64, also NASM: without it, aws-lc's build fails unless AWS_LC_SYS_PREBUILT_NASM=1 asks for its prebuilt objects (aws-lc-sys's README).
  • cmake and perl for the ts feature only.
  • Network to github.com for the first build: some crates come from PodUtils/podcrates by a tag of that repository (T-684), the one source of that kind that deny.toml allows.
  • Python 3 for the repository checks.
  • For the container gate on Windows: Git Bash, PowerShell, and the operator's wsl-toolkit. On Linux: Docker or Podman, which sh scripts/cloud-setup.sh installs or starts (section "A Linux host, or a cloud session").
  • For the box like the target sandbox: Podman. On Windows, the Podman machine.

Build and test natively

CAUTION: Set the job limit first. See "Memory".

export CARGO_BUILD_JOBS=4
cargo build                        # target/debug/podssh
cargo test --no-fail-fast          # the default members
cargo test -p podssh-core          # one crate
cargo build -p podssh-cli --features ts                          # with Tailscale
cargo test -p podssh-ts -p podssh-cli --features podssh-cli/ts   # its tests
cargo build -p podssh-cli --features iroh                        # with the iroh road
cargo test -p podssh-iroh -p podssh-cli --features podssh-cli/iroh-test   # its tests
cargo test -p podssh-ws --features plain-ws --test plain_loopback   # plain ws:// to loopback, for tests
cargo test -p podssh-ws --features plain-ws --test cloudflare_access # the road of cloudflared (T-264)
cargo build -p podssh-cli --features warp                        # with WARP's road (T-352)
cargo test -p podssh-warp -p podssh-cli --features podssh-cli/warp --lib   # its tests
cargo test -p podssh-warp --test small_frames                    # its tunnel under volume (T-398)
cargo test -p podssh-warp --features stand-in --test road        # the probe of an open tunnel (T-560)
cargo test -p podssh-relay --features warp-stand-in --test quick_edge_warp # the edge inside it (T-374)
  • A cargo build or cargo test at the root uses default-members. These do not include crates/podssh-ts or crates/podssh-iroh. Use --workspace to include them.
  • Use --no-fail-fast. Without it, cargo stops at the first test target that fails, and the remaining targets do not run.
  • The workspace builds and tests natively on Windows (MSVC) and Linux. The static Linux binary needs the musl target. The container gate builds it. CI's job windows runs the default tests on windows-2025 at each push.
  • The Tailscale fork in vendor/tailscale-rs is a workspace of its own: podssh's builds cap its lints, and cargo does not test a crate of it that has dev-dependencies. Run its clippy and its tests with its manifest, in a target directory of their own: cargo clippy --manifest-path vendor/tailscale-rs/Cargo.toml --target-dir .work/fork-target -p ts_derp -p ts_runtime -p ts_http_util -p ts_tls_util -p ts_control -p tailscale --all-targets -- -D warnings, and cargo test the same way. scripts/ts-derp-prove.sh runs both in the build image (T-276).

Memory

Cargo runs one job for each CPU. On a machine with 20 threads, that is 20 compilers and linkers at the same time.

CAUTION: On 2026-10-07, three uncapped builds at the same time almost used all the memory of a 64 GB machine.

  1. For native builds, set CARGO_BUILD_JOBS=4. You can also put jobs = 4 under [build] in ~/.cargo/config.toml.
  2. For container builds, scripts/dev.sh gives PODSSH_JOBS (default 4). scripts/gate.sh limits itself to one job for each 3 GiB of free memory.
  3. Do not run two container builds at the same time. scripts/dev.sh holds a lock in .work/dev.lock and refuses a second run.

NOTE: A memory limit on a container does not protect the machine here. The wsl-toolkit base gives no cgroup delegation to its Podman account, so it accepts a memory or CPU limit and does not enforce it. The job limit and the lock are the protection.

NOTE: To limit WSL itself, the operator can set memory=24GB and swap=8GB under [wsl2] in %USERPROFILE%\.wslconfig, and restart WSL. No script changes this file.

Disk

What grows on the developer machine, and what removes it:

  1. A run of scripts/dev.sh is a job of wsl-toolkit with --container-lifecycle ephemeral: its container and its copy of the tree (up to 10 GiB with a build) go when it ends. A run that was cut can leave both; wsl-toolkit gc --older-than 24h lists them, and removes them with --apply, for each job of the tool, not only podssh's. On 2026-10-10, 90 jobs of runs from before 2026-10-08 held 203 GiB.
  2. The base's disk image, ext4.vhdx in the tool's state directory, does not shrink when the guest frees space. To give the space back, the operator stops the distribution (wsl --terminate wsl-toolkit) and, in an elevated shell, runs diskpart with select vdisk file=PATH, attach vdisk readonly, compact vdisk and detach vdisk. A session does not run wsl.exe (AGENTS.md, section 4).
  3. A run of the tests leaves nothing in the temporary directory, and the gate fails when it does (T-272). target/ is Cargo's; cargo clean empties it.

Checks

python scripts/check-repo.py        # 500-line rule, doc links, credentials, LF, pinned images, no listener, rule 6 in the code, STATUS's release marks
python scripts/check-scripts.py     # shell scripts parse under dash
python scripts/check-relay-spec.py  # the live relay still matches what podssh uses: 0 holds, 1 a fact changed, 2 not read
python scripts/check-relay-spec.py --plant-moved-line     # each fact one line down holds, with a note (T-338)
python scripts/relay-spec-agree.py target/debug/podssh   # podssh relay spec and the script agree
python scripts/check-domain.py      # the relay's domain expires 90 days or more ahead (relay-watch.yml runs both each day)
python scripts/relay_watch.py --self-test   # the issues that the watch opens, on made-up days
cargo todo check                    # the work record in TODO/ agrees: counts, statuses, cited paths and lines, no citation left unmoved, no line number in plain text after a citation
cargo todo remap FILE...            # after an edit of FILE: move its citations by a diff against HEAD
cargo todo check --base HEAD~1      # the same check against the last commit's parent, as CI runs it; remap takes --base too
cargo deny --locked check advisories licenses bans sources   # the dependencies (deny.toml)
cargo fmt -p podssh-cli -p podssh-ssh -p podssh-relay -p podssh-ws -p podssh-core -p podssh-probe -p podssh-ts -p podssh-todo -- --check
cargo clippy --locked --all-targets -p podssh-cli -p podssh-ssh -p podssh-relay -p podssh-ws -p podssh-core -p podssh-probe -p podssh-ts -p podssh-todo -- -D warnings -A clippy::collapsible_if
cargo deny --locked --all-features check licenses sources    # the same, with the Tailscale fork

The code has the format of rustfmt.toml, and clippy gives no warning; the gate checks both first (scripts/gate.sh). Name the packages: cargo fmt --all also formats the fork in vendor/, which keeps its own format and its patches. A table keeps one row to a line (#[rustfmt::skip], with a comment that says so). A lint is repaired, or allowed at its item with a comment that says why; never for a whole crate.

The clippy that counts is the gate's, in the build image. Its Rust can be newer than the host's, with new lints (clippy 1.99 found a lint that 1.98 passes), and clippy on Windows does not compile the code for Unix only. Before a push that changes Rust code, run the gate (sh scripts/dev.sh check), or at least its step lint:

sh scripts/dev.sh run -- 'sh /work/scripts/gate.sh lint'

check-repo.py also reads each Rust file under crates/ for a listener: TcpListener, UnixListener, UdpSocket, bind(, listen( and socket2, outside comment lines. A crate's tests/ may hold one, for a server on the loopback; elsewhere only the bind probes of podssh doctor may. A listener that the user asks for and a probe allows joins the table in the commit that adds it. The scan fails when it reads fewer than 100 files, and when a listener that it plants in a temporary tree is not found.

It reads each code file too (each tracked file but the documents and vendor/) for rule 6 of AGENTS.md: a stop-sign marker, a line number of a document (NAME.md and a line), or a path of the documents that moved (under docs/spec/ or docs/TODO/). A comment names a document's section, or states the fact. The tests of the record's checker, which hold Markdown locations as data, and one test of podssh-probe are allowed the last two; no file is allowed a marker. Three planted lines must be found at each run.

And it reads Cargo.toml and fuzz/Cargo.toml with their lock files: each package of [patch.crates-io], a copy of vendor/, must come from its path in the lock, with no [[patch.unused]] (T-460). An update of russh or of iroh that did not move the copy builds the release in its place, with none of podssh's patch, and cargo only warns. Two planted locks must be found.

And it reads the rows of docs/STATUS.md, the page of main (T-594): a row whose first cell has a date of 2026-11-02 or later, after v1.0.0, names a release that has its notes in docs/releases/, or says unreleased; a row that says unreleased with a date before the newest release that the head names fails, as the release step left it. It reads no git, so a checkout of depth 1 checks the same. Three planted rows must be found.

And it reads each tracked file for the repository's old name (T-683): the repository is PodUtils/podssh since 2026-11-05, and only the dated records (docs/releases/, docs/STATUS.md, docs/decisions.md, docs/research/, the entries of TODO/, vendor/) and two tests of podssh update's refusal may name the old one (scripts/repo_names.py). A planted line of Cargo.toml must be found.

deny.toml allows each license that a crate of the graph needs, by name, and crates.io as the only source; an advisory is ignored only with its reason and the date it was read. CI runs both commands on each push, each pull request and each day (.github/workflows/deny.yml). Each release publishes THIRD-PARTY-LICENSES.md, the licenses of the crates in its binaries with their texts (about.toml, about.hbs, made by cargo-about).

Read each exit code directly. cmd | tail gives the exit code of tail.

Each check of check-repo.py counts the files that it read, and fails below a floor (100 Rust files, 10 Markdown files, 150 tracked files, 5 shell scripts, one image, 100 files for the listeners, 100 code files for rule 6, 100 rows of docs/STATUS.md), so a scan that read nothing never passes. python scripts/check-repo.py --plant-empty runs each check on an empty repository and must exit 1; CI runs both.

CI also scans each commit of the history for credentials, on each push, each pull request and each week (.github/workflows/secrets.yml): TruffleHog, pinned in images/trufflehog/Dockerfile, in its git mode only, as its filesystem mode on a developer's machine would read .env/. A key made for the run and committed in a repository of its own must be found first. scripts/secrets-report.py prints each finding with no secret in it, as the logs of a public repository are public; a verified finding, or one that could not be verified, fails the run, and an unverified one is a note. The test values that look like credentials (a proxy URL with a made-up user and password) are listed in .github/secrets-allow.txt, each by its detector, file and line, and the commit that added it, with the reason; the same value in another commit is a new finding.

On a finding: revoke the credential first, because the history is public and copies of it may exist; then remove it from the tree. Never rewrite the published history (AGENTS.md, section 6).

The container gate

sh scripts/dev.sh check    # the host checks, then scripts/gate.sh in the build image
sh scripts/dev.sh gate     # scripts/gate.sh alone, in the build image
sh scripts/dev.sh plant    # shows that the check of a static binary refuses a dynamic one
sh scripts/dev.sh images   # the build image runs: its uname, rustc and cargo
sh scripts/dev.sh test -p podssh-core
sh scripts/dev.sh run -- 'uname -a'
sh scripts/dev.sh help

scripts/dev.sh reaches the container through a transport that it sources: on Windows scripts/dev-wsl.sh (wsl-toolkit, called through PowerShell from Git Bash), on Linux scripts/dev-native.sh (the host's own Docker or Podman). Each gives a container of the pinned image a copy of the tree, and returns the run's own exit code. PODSSH_TRANSPORT=wsl or native chooses; by default a host with wsl-toolkit, or a Windows shell, uses wsl.

Each image is named in one place, pinned to the digest of its multi-platform index: images/build/Dockerfile (rust:1-alpine, for the gate, CI and the release), images/box/Dockerfile (alpine:3.24, the box and its SSH server) and images/box-proxy/Dockerfile (python:3.15-alpine, the box's proxy). The scripts and the workflows read their FROM line, so two runs of one commit use one compiler, and python scripts/check-repo.py fails on a second name. CI pulls each through scripts/pull-image.sh: when Docker Hub refuses a pull with no account, past its limit for the runner's address, the same digest comes from a mirror (mirror.gcr.io, then public.ecr.aws/docker/library or ghcr.io), and a pull by digest gives the same bytes or none. secrets asks a registry that does not exist for the scanner, and a mirror must serve it. To move to a newer image, read the digest of its index (docker buildx imagetools inspect REF; the index must list linux/amd64 and linux/arm64, which the release builds), write it into the Dockerfile, and run the gate.

scripts/gate.sh is the gate. CI runs the same file in the same image. Its steps, in order: lint, libs, record, ssh, ts, iroh, tor, m6 and release; sh scripts/gate.sh --list prints them. With no argument, the gate runs each step; with names, those steps, in the order given. CI runs each step in a job of its own, at the same time: its job plan reads the list, so CI cannot miss a step that the gate has. python scripts/check-scripts.py fails when a function step_NAME of the gate is not in its list, or a name has no function. One step, in the container:

sh scripts/dev.sh run -- 'sh /work/scripts/gate.sh ssh'

lint checks the format and clippy's lints (see "Checks"). tor runs the tests of podssh-tor and of the CLI with the feature tor. The gate has no step at a minimum Rust and no build with no C compiler since 2026-11-05 (T-685): each crate may use C, zig and the newest stable Rust, and the one rule of the build, a statically compiled binary, is checked in the step release (scripts/check-static.sh).

The Wisp client's tests (T-438) run against a stand-in written from the protocol's document in each run, and against wisp-server-python, Mercury Workshop's own server, where a Python has it; each of those says that it did not run where none has it, and fails instead with PODSSH_TEST_WISP_REQUIRED set, as in .github/workflows/forwarder-live.yml, which installs it in a venv on the runner. By hand:

python3 -m venv .work/wisp-venv
.work/wisp-venv/bin/pip install wisp-python==0.9.0 websockets==15.0.1
PODSSH_TEST_WISP_PYTHON="$PWD/.work/wisp-venv/bin/python" cargo test -p podssh-ws --test wisp

The Tor road's tests against a Tor network of C tor (T-353) run in CI's job tor-net, on the runner: scripts/tor-net.sh starts chutney's network basic there, chutney at a pinned commit, with no daemon of Tor's own network (Q60). By hand, with tor and tor-gencert installed:

toml=$(sh scripts/tor-net.sh start .work/tor-net "$(hostname -I | awk '{print $1}')")
PODSSH_TEST_TOR_NETWORK=$toml cargo test -p podssh-tor --test private_network
PODSSH_TEST_TOR_NETWORK=$toml cargo test -p podssh-cli --features tor --test tor_private
sh scripts/tor-net.sh stop .work/tor-net

The relays listen on the address given, as podssh sends no loopback address through a proxy; on the loopback, the CLI's test checks that no first hop went through the proxy instead.

The road through a bridge (T-385) runs in the same job, after that network stopped: scripts/tor-net.sh start-bridges builds Tor's own meek-server and WebTunnel server with Go (from Go's module proxy), starts a network of podssh's own whose bridge runs both, and prints the path of the bridge's lines, one for each transport. By hand, with go installed too:

bridges=$(sh scripts/tor-net.sh start-bridges .work/tor-net-bridges "$(hostname -I | awk '{print $1}')")
PODSSH_TEST_TOR_BRIDGES=$bridges cargo test -p podssh-cli --features tor --test tor_bridges
sh scripts/tor-net.sh stop .work/tor-net-bridges

cargo test -p podssh-tor --test transports needs no network: Arti's own client of a transport reaches stand-ins of both servers through the sentinels. The other steps make sure that:

  1. (libs) The library crates build and pass their tests. The test of the feature plain-ws of podssh-ws runs too, and scripts/no-plain-ws.sh shows that the binary does not enable it (T-068). tmux, installed here and named by PODSSH_TEST_TMUX, shows each sequence of the terminal oracle's corpus as its model does (T-200, docs/terminal.md, "Tests").
  2. (record) The work record agrees with itself (crates/podssh-todo, a wrapper of pod-todo). Its checker passes its tests, where each planted disagreement must be found, and then checks TODO/: the counts against the rows, the status, title and milestone of each row against its entry, each id that a document names, each cited path and line, the work order, and docs/ROADMAP.md.
  3. (ssh) The SSH client and the command line pass their tests.
  4. (ts, iroh) The tests of the ts feature pass, and those of the iroh feature: two endpoints with no UDP, through a stand-in CONNECT proxy and a relay on the loopback that only the proxy can name; and, with iroh-test (the feature iroh and a relay server for the tests, never in a release), podssh node --iroh and podssh ssh iroh:TICKET from end to end, through iroh's relay server on the loopback.
  5. (m6) The exit of M6 (T-156): a session survives a stopped relay host, a new address of its client and a stall of 3 minutes, with 20 MiB up and back and the fault in their middle, on the resumable layer over the reverse road and on the iroh road; the same faults end a session of the forward road with 255. The stand-in relay serves the reverse road (scripts/fake_reverse.py, on scripts/fake_ws.py) as the live relay showed it, and iroh's relay server runs under a name that only the stand-in proxies resolve, with no UDP (PODSSH_IROH_UDP=off). The proxies make the faults: the client's makes each, the far end's only a stopped relay host, which ends each connection to it. About 15 minutes; natively, cargo test -p podssh-cli --features iroh-test --test m6_exit -- --ignored --test-threads 1.
  6. (release, with 7 to 11) The static musl binary has no dynamic dependencies and no program interpreter.
  7. The binary works against real servers. scripts/interop.sh installs OpenSSH and Dropbear in the container, starts them on 127.0.0.1, and runs podssh ssh --direct against them: exit statuses and signals, streams and digests, each authentication method, host keys, -W, -J, -s, ptys through pipes, and a real pty (scripts/interop-pty.py: resize, Ctrl-C, vi, less, top, ~.). A session whose output nobody reads for 30 s, while the server writes 50 MB, must peak under half of that (scripts/interop-limits.sh, T-201). Each exit code of the matrix and of the refusals is checked twice (T-225): ssh.ID against its literal, and ref.ID against OpenSSH's own client, run first with the same arguments (scripts/interop-reference.sh). A difference on purpose is a row of that script's table, with its reason; a difference that no row names fails, and so does a row that the run does not meet.
  8. OpenSSH accepts the keys of podssh keygen (scripts/interop-keygen.sh): its ssh-keygen reads them and its sshd accepts them for login.
  9. podssh handles a relay that fails (scripts/interop-faults.sh). A stand-in relay (scripts/fake-relay.py, with TLS from a CA made for the run) and a stand-in proxy (scripts/fake-proxy.py) fail in one way each: a host that is down, a 503, a host that does not answer after TLS, a host that does not start TLS, a proxy 502, a Close during a transfer (1011) and at the byte limit (1009), a stall, and a host that stops during a session. Since T-203, the stand-in relay also shapes the traffic: 2 s each way (with a planted control: a ConnectTimeout of 5 s must fail), a jitter of 0 to 1.5 s (5,000,000 bytes up and back, unchanged), 64 KiB/s (2,000,000 bytes near the time computed), and a cut with no Close. The stand-in proxy's --move, --pause and --stop-after make the faults of the step m6. The same checks run natively in an ignored test, cargo test -p podssh-cli --test faults -- --ignored --test-threads 1.
  10. The man page renders (scripts/interop-man.sh): groff and mandoc show each flag that --help shows, groff gives no warning, and mandoc -Tlint gives no error. A planted page, with one flag's term removed, must fail.
  11. Each check of 7 to 10 prints one line, VERDICT ID TEXT (scripts/interop-lib.sh), with an id that does not change from run to run, and the step scores the logs against scripts/interop-baseline.txt (T-199, cargo todo interop --logs FILE...). An id that must pass and gave FAIL, skip or env (a missing precondition of the host), an id with no line or with two, an id that the baseline does not name, and a harness under its floor of passes each fail the step; it prints a row score PREFIX pass fail env skip missing floor for each harness. With no log, cargo todo interop checks that the scripts print exactly the ids of the baseline, and the tests of podssh-todo run that check on each system. The job windows of CI scores the console checks the same way. Change the baseline in the commit that adds, removes or renames a check.

NOTE: The stand-ins are part of this repository. They test how podssh handles each fault. The live tests show that podssh works with the real relay.

The containers are temporary. They get the tree as CI does, without these patterns of scripts/dev-wsl.sh (EXCLUDES): target/** and vendor/tailscale-rs/target/** (build output, which would pass the copy's limit of 1 GiB), .git/**, .work/**, .tmp/**, .env/** (live credentials) and .codegraph/** (a local index). A pattern matches a file of that name at any depth.

A run of GitHub Actions asked of the owner's agent

Where GitHub Actions start no workflow for the account of a session (a cloud session of 2026-11-06), the owner's local agent runs them, and it is a smaller model (the owner's rule of 2026-11-06, relayed by the podcrates session). So each run is asked like this:

  • a NEW issue of PodUtils/podssh, titled exactly [gh action] requested, for each run;
  • one line of why, then ONE sh block: set -e first, each gh command in full (-R OWNER/REPO and every flag), nothing that needs judgement, and a last line that echoes the run ids;
  • the body ends with: "Run the block as it is, once. Then write here: the output of the last line, or the line where it stopped and its error.";
  • the block of an open request is never amended, and no second block goes in a comment: close it and open a new issue;
  • while a block checks the head of main, nothing is pushed to main until it ran.

Each such issue is mapped in TODO/issues.md and TODO/github.txt as any issue.

A Linux host, or a cloud session

sh scripts/cloud-setup.sh (also sh scripts/dev.sh setup) makes a Linux host with root ready for this work: a Claude Code cloud container, a new VM. Each step probes first and changes nothing that is ready, so a second run changes nothing, and a session can run it at each start. sh scripts/cloud-setup.sh --check reports and changes nothing.

  1. The packages (apt): Podman and its helpers, OpenSSH (client, server, sftp-server), Dropbear, groff, mandoc, tmux, the musl tools, and the tools of the scripts.
  2. A Docker daemon that is installed and not running (a container has no init system to start it), with the session's proxy for its pulls.
  3. Podman's settings for a host whose PID 1 is not systemd: cgroupfs, and files for events and logs; runc where the cgroups are hybrid, as crun refuses each container there.
  4. The pinned images: the build image into Docker (scripts/pull-image.sh), the box's two images into Podman.
  5. The gate's Rust as the host's toolchain, with clippy, rustfmt and the musl target: the clippy that counts is the gate's.
  6. The repository's git identity, when it has none of its own: that of its newest commit by a person. A cloud session's global identity is its own, and each commit is the operator's (docs/decisions.md).

The native transport runs each container with --rm, --init and a copy of the tree in .work/native/, removed after the run (PODSSH_KEEP=1 keeps it, with its target/). It gives the container the host's network when the host's proxy listens on the host's loopback, the proxy variables, and the host's CA (SSL_CERT_FILE) in the image's store, for apk, rustup, cargo and git; podssh's tests in the container get no CA variable, as in CI. The crates that cargo downloads stay in a volume between runs (PODSSH_NO_CACHE=1 starts empty). sh scripts/dev.sh clean removes podssh's own containers and copies that a cut run left.

In a Claude Code cloud session:

  • The committed .claude/settings.json allows each tool with no prompt (a cloud session cannot bypass permissions), sets CARGO_BUILD_JOBS=4, and adds no line of attribution to a commit.
  • The environment's setup script (the cloud environment's settings) can run the setup at the start of each session: cd podssh && sh scripts/cloud-setup.sh. That script starts one directory above the repository.
  • The way out is the session's agent proxy (HTTPS_PROXY on the loopback), which re-terminates TLS for some names with a CA of its own, and passes the relay's own chain; direct TLS on port 443 meets an egress that intercepts it (T-311). podssh's tests ignore the machine's proxy where they use the stand-in relay (T-313), and the live handshakes go through the proxy (T-315).
  • The session's disk is an allowance of about 40 GB, and df reads it wrong: on 2026-10-30, clippy and the tests of the workspace with full debug information filled it, target/debug at 15 GB. Build with less: CARGO_INCREMENTAL=0 CARGO_PROFILE_DEV_DEBUG=line-tables-only CARGO_PROFILE_TEST_DEBUG=line-tables-only; cargo clean frees it.

A box like the target sandbox

scripts/test_in_box.sh uses Podman to build a box with the properties of the target sandbox, and measures podssh in the box.

  1. Get a static Linux binary of podssh. Use the release workflow's artifact or the gate's build.

  2. On Windows, start the Podman machine: podman machine start.

  3. Run the script:

    sh scripts/test_in_box.sh path/to/podssh-x86_64-unknown-linux-musl

The box has these properties:

  • No route out. The only way out is a CONNECT proxy (scripts/box/proxy.py). It allows ports 443, 80 and 8443 to public hosts, and refuses other ports and private addresses with the texts of the sandbox's proxy.
  • A resolver that does not answer, no capabilities, no_new_privs, and a seccomp filter that refuses bind and UDP (scripts/box/seccomp.json).
  • No /dev/ptmx, and uid 0 with no name.
  • A /dev/tty that opens and never answers, and no controlling terminal, as in the target sandbox. scripts/box/deadtty.py holds a pty with nobody at its master on the Podman host (the Podman machine on Windows), and the box has its slave at /dev/tty. It needs Python 3 and setsid on that host.

BOX_RUN=tt sh scripts/test_in_box.sh BINARY runs an interactive session over -tt instead of sandbox-check.sh (T-004): scripts/box/tt-session.sh writes keys for vi, less, top and Ctrl-C through a pipe to OpenSSH in a container next to the box, which the box's proxy lets through (proxy.py --allow), and checks five markers.

scripts/box/probe.sh compares the box with the operator's sandprobe report of the target sandbox. If a required property is different, the script stops and does not run podssh. Then scripts/sandbox-check.sh runs doctor, proxy, keygen, ssh, a prompt with nobody to answer it, and OpenSSH with podssh as its ProxyCommand.

NOTE: The box does not give EACCES for connect() to loopback and to some ports, as the sandbox does.

NOTE: The sandprobe report is .work/sandprobe-run1.txt. It is not in the repository, because it describes the operator's own sandbox.

To measure a real sandbox, run sh scripts/sandbox-check.sh in it. With no argument, the script builds podssh first, and finds the binary where cargo put it: CARGO_TARGET_DIR, the target_directory of cargo metadata, or target/. It runs the binary once before the steps. Each step prints ok or FAIL against its expected result, or skip with the reason when it cannot run on that host (no ssh, or an OpenSSH that needs a user database entry). The script exits 1 when a step failed.

In a sandbox where /tmp or $HOME is mounted noexec, CARGO_HOME cannot be written, or /tmp is small, give cargo a directory where programs can run. podssh doctor names such directories in its exec lines; with no binary there, copy /bin/true into a directory and run it from there.

export CARGO_HOME=/work/cargo CARGO_TARGET_DIR=/work/target TMPDIR=/work/tmp
mkdir -p "$TMPDIR"
sh scripts/sandbox-check.sh

Rules for tests

These rules are necessary because tests here passed while the code was wrong: the tests built their input with the same assumptions as the code.

  1. Test protocol code against software that podssh did not write: an OpenSSH or Dropbear server, a real IRC server, the live relay, or bytes captured from one.
  2. Trust a new check only after it fails on a planted defect and passes on correct input. A plant that must run again (before a release, T-251) is a file of scripts/plants/: the defect as a patch, or a switch of its harness, and the line that the failed check prints. sh scripts/plant-each.sh --here ID runs one here; the workflow plants.yml runs each group in the build image.
  3. Run the real binary with stdout and stderr on different pipes. A stray println! passes every test that runs in memory.
  4. In a live test that expects a failure, send nothing after the trigger. Extra bytes can arrive in place of the close code.
  5. Do not use the network in a test. The tests that run the binary set PODSSH_OFFLINE=1, which makes each connection attempt fail at once with a message. The tests that run in the process use a destination that the relay cannot take, so podssh refuses it before it connects.
  6. Remove what a test makes. A scratch directory under the temporary directory goes to cleanup::at_test_end (tests/cleanup/mod.rs of its crate), which removes it when the test ends, also when it fails; the gate fails when a run leaves an entry podssh-* there (T-272). A test that starts podssh sets PODSSH_SSH_CONFIG=none, so that the ssh_config of the machine cannot change it (T-043).

Property tests

tests/properties.rs of podssh-ws, podssh-relay, podssh-ssh and podssh-core (and of pod-term in PodUtils/podcrates) state what must hold for each input of a state machine, and proptest checks it on inputs that it makes (T-202): frames and fragments of WebSocket, the sanitizer, relay lists and paths, known_hosts patterns, the escape filter, the IRC line framing and the line discipline. Each runs in the gate, on stable Rust.

cargo test -p podssh-ws --test properties
PROPTEST_CASES=4096 cargo test -p podssh-ssh --test properties   # more inputs, by hand
  1. Each property takes 256 inputs, unless PROPTEST_CASES says otherwise. The property of a message past 16 MiB takes 4, as each input moves more than 16 MiB.
  2. A property that fails prints the smallest input that fails, and writes it to tests/properties.proptest-regressions beside its test. Commit that file with the repair: proptest tries its inputs first at each run.
  3. A property names what holds for each input. A case that a person chose stays a test of its own.

Fuzzing

fuzz/ holds a fuzz target for each parser of the bytes of a peer or of a file (T-198): cargo-fuzz and libFuzzer, on a pinned nightly Rust. It is a workspace of its own, which the root workspace excludes, so the gate builds none of it. Each target checks more than the absence of a panic; its first lines say what. cargo +nightly fuzz list names the targets.

export PODSSH_BUILD_IMAGE=docker.io/library/rust:1-trixie   # glibc, for this machine's runs
sh scripts/dev.sh run -- 'sh /work/scripts/fuzz.sh 120'      # each target for 120 s
sh scripts/dev.sh run -- 'sh /work/scripts/fuzz.sh 60 ws_frame close_reason'
gh workflow run fuzz.yml -f seconds=300                       # in CI
  1. Linux with glibc only, and always with the address sanitizer. On Windows, libFuzzer's coverage needs the runtime of MSVC's address sanitizer, which these build tools lack: the link fails with __start___sancov_pcs unresolved.
  2. On musl, the build image's C library, each target died of a signal within five inputs (2026-10-11), so scripts/fuzz.sh refuses musl with 69. PODSSH_BUILD_IMAGE gives scripts/dev.sh a glibc image instead.
  3. The workflow fuzz runs each target for 120 s each Monday, and by hand. A crash fails it, and the crash's input is its artifact.
  4. A crash becomes a unit test first, then a repair. The test names the target that found it.
  5. scripts/fuzz-seeds.py seeds the targets with bytes that podssh did not make: the frames of RFC 6455, the IRC grammar fixture, the hosts of the relay's document; the messages of Cap'n Proto that the reference encoder wrote for the quick tunnel, and connect-ip-go's capsules (T-403).
  6. A private parser is reached through a function under cfg(fuzzing), which cargo-fuzz sets: podssh_ws::fuzz_exchange and podssh_ssh::known_hosts::fuzz_parse_line. Their crates declare the cfg ([lints.rust]), so that clippy takes it.
  7. The readers of the quick tunnel's Cap'n Proto and of WARP's capsules (capnp_message, quick_connect_request, quick_control and warp_capsule, T-403) run under -malloc_limit_mb=64: one allocation of 64 MiB from an input of some KiB is a defect, as a list of 2^23 structs of no size was. fuzz/Cargo.toml names podssh-warp, which the default build of podssh-cli leaves out.

Live tests

These tests use the network. Run them only on request.

cargo test -p podssh-cli --test proxy_live -- --ignored   # the proxy path, with a token
cargo test -p podssh-cli --test doctor -- --ignored       # doctor, end to end
cargo test -p podssh-relay --test live -- --ignored       # the relay answers pings
cargo test -p podssh-relay --features pair --test pair_live -- --ignored     # a pair, made and stopped
cargo test -p podssh-relay --features pair --test reverse_live -- --ignored  # a node and its operators
cargo test -p podssh-relay --features blocking --test blocking_live -- --ignored  # the same, and a forward session, through the facade
cargo test -p podssh-relay --features piping --test piping_live -- --ignored     # a session through ppng.io (Q65)

The Piping Server road's tests on the loopback (T-437) start nwtgck's two servers by the commands of PODSSH_TEST_PIPING_JS (nwtgck/piping-server, from npm install -g piping-server@1.12.9) and PODSSH_TEST_PIPING_RUST (piping-server-rust's binary); each test whose server is not named says that it did not run. The workflow piping.yml installs both, runs them, and then the live test, directly and through the box's proxy of names:

PODSSH_TEST_PIPING_JS=piping-server PODSSH_TEST_PIPING_RUST=.work/piping-server \
    cargo test -p podssh-relay --features piping --test piping
gh workflow run piping.yml --ref REF

For interactive use on Windows, scripts/interop-conpty.py runs podssh ssh -t in a real pseudo console against an SSH server that you give:

python scripts/interop-conpty.py target/debug/podssh.exe root@HOST --direct

CI's job windows runs it at each push against the OpenSSH server of MSYS2 on 127.0.0.1, which the runner image has at C:\msys64 (scripts/interop-conpty-msys2.py). The user key and the host key are made for the run and deleted after it. A second run builds a podssh that leaves the console raw; it must fail the three restore checks and no other. With MSYS2 on a developer's machine, the same:

python scripts/interop-conpty-msys2.py target/debug/podssh.exe   # 14 of 14, or exit 1
python scripts/interop-conpty-msys2.py --plant                   # builds the plant, then podssh again

--install first adds openssh, vim, less and procps-ng with pacman; MSYS2_ROOT names a directory other than C:\msys64. Exit 2 means that the checks did not run (no MSYS2, or the server did not start).

A console check that failed at random is repaired, then passes 20 runs in a row (Q36). The workflow conpty-repeat runs the checks on CI's Windows runner, run after run, and the first that fails ends it (T-283):

gh workflow run conpty-repeat.yml -f runs=20

scripts/relay-measure.py measures the relay hosts and the roads through podssh's own binary (T-605), adapted from the operator's archive of relay research (docs/research/relay-measurement.md): each host's leg by ping, the relay's own dial by relay trace, GitHub's host key through each host by doctor --quick, a road's checks by doctor with its flags, and a login and the throughput to a destination by ping DEST. It runs one podssh at a time, in rounds in a random order from a seed that the result keeps, each run within a limit and the whole within --budget, with a cache directory of its own. The result is one JSON file with each run, each host's summary and a rank; it keeps the proxy's kind alone, as podssh's own lines name the proxy that they went through. Its self-test runs offline against a stand-in podssh, and fails on two runs at once (RELAY_MEASURE_PLANT=parallel):

python3 -I scripts/relay-measure.py --self-test
python3 -I scripts/relay-measure.py --podssh target/debug/podssh \
    --stages pool,trace,forward --hosts 4 --rounds 2 --out .work/relay-measure.json
python3 -I scripts/relay-measure.py --podssh target/debug/podssh --stages dest \
    --destination USER@HOST --identity KEY --ping-road '--direct'
gh workflow run relay-measure.yml --ref main -f hosts=all -f rounds=2

The workflow, by hand, uploads the JSON as an artifact and writes the rank in the step summary. Its stage dest reaches the operator's server when the repository's secrets PODSSH_THROUGHPUT_SSH and PODSSH_THROUGHPUT_KEY are both set, and is left out with a line otherwise; its input roads takes the flags of each road of the stage roads, separated by ;, for the forwarders of Q56, short.

scripts/iroh-ab.py measures the arms of PODSSH_IROH_QUIC, QUIC's settings on the iroh road, one against another (T-665), by T-157's method: a new connection for each run, down and up in separate sessions, the arms in turn in each round, and the minimum, p50, p95 and maximum by nearest rank. With OpenSSH, it copies a static podssh to the far server into a private directory, runs a podssh node --iroh of each arm there in front of the server's own sshd, logs in to each with podssh ssh -v, which says the path and the arm of each run, and removes the nodes and the directory at the end. The result keeps figures, paths, arms and each end's rmem_max and wmem_max; no user, host, address, key or ticket. The workflow, by hand, builds the static binary in the gate's image and needs both secrets of the stage dest above:

python3 -I scripts/iroh-ab.py --self-test
python3 -I scripts/iroh-ab.py --podssh target/x86_64-unknown-linux-musl/release/podssh \
    --destination USER@HOST --identity KEY --arms 'iroh;tuned' --runs 5 --mib 4 --out .work/iroh-ab.json
gh workflow run iroh-ab.yml --ref main -f arms='iroh;tuned' -f runs=5 -f mib=4

scripts/capture-close.py captures the payload of a Close that the live relay sends, with a client of the Python standard library: bytes that podssh did not make, for a fixture of the Close parser (crates/podssh-ws/tests/control_frames.rs). It mints a token that it keeps in memory and never prints:

python scripts/capture-close.py              # github.com:22: 1000 target closed
python scripts/capture-close.py --client-close   # what the relay does after the client's Close
python scripts/capture-reverse.py            # one session of the reverse road, frame by frame

scripts/capture-reverse.py makes a pair, opens the node's and an operator's socket, and records each frame of one session (hello, open, ready, data both ways, close) and whether the relay answers a Ping on the node's socket and on the operator's; it stops the pair at the end and prints no token.

The examples open a forward session as podssh ssh does (podssh_relay::open): a token from the environment, the cache or a mint, never printed.

cargo run -p podssh-relay --example live_forward          # github.com:22: its SSH banner
cargo run -p podssh-cli --example live_irc                 # an IRC registration through the relay

The stand-in tailnet (T-334)

scripts/ts-headscale.sh runs the live parts of podssh ts against headscale and two nodes of tailscaled in Podman, else Docker, in place of the operator's tailnet (docs/tailscale.md gives its parts and its runs). Run it on a Linux host with an engine, not in the gate's container, one build at a time:

sh scripts/ts-headscale.sh                           # builds podssh --features ts first
sh scripts/ts-headscale.sh target/debug/podssh       # a build with the feature ts
TS_HS_PLANT=unknown-key sh scripts/ts-headscale.sh target/debug/podssh   # must exit 1

It needs podman or docker, openssl, curl, python3, OpenSSH's ssh, timeout and sha256sum, and the ports 45380 to 45385 of the loopback (PODSSH_HS_PORT moves them); PODSSH_ENGINE names the engine. It pulls images/headscale/Dockerfile and images/tailscale/Dockerfile through scripts/pull-image.sh (PULL_ENGINE=podman for Podman). Each line is a result of scripts/interop-lib.sh; the exit is 0 when each run passed, 1 when one failed, and 3 when none failed but this host has no route to the live relay, which three runs reach (env; GitHub Actions has one). The CA, the pre-auth keys and the containers live for the run alone, and no key is printed.

Line endings

  • .gitattributes stores the sources with LF.
  • Shell scripts must use LF. dash refuses CRLF.
  • The IRC wire fixtures in crates/podssh-core/tests/fixtures/ are stored byte for byte (-text), because CRLF ends each IRC line.
  • Do not change the line endings of a file that a change does not otherwise touch.

Updates

Dependabot (.github/dependabot.yml) looks each week at the crates of the workspace's Cargo.lock, the actions of the workflows, and the pinned images of images/ that it lists, the stand-in tailnet's two among them (T-334). The images are not under .github/: there, Dependabot's service refused each pull request of the docker ecosystem with a 400, "The request contains invalid or unauthorized changes" (T-308). Minor and patch updates of the crates come in one pull request, and each major update alone; the actions come in one. The fork in vendor/tailscale-rs has no entry: it follows its own patches. CI checks the configuration against Dependabot's published schema, with a planted copy that must fail.

Each pull request of Dependabot runs the whole CI: the gate, whose step release checks that the binary stays static; the plant; and the live check of the relay's document. An update whose CI is green is applied as the operator's own commit, then its pull request is closed:

git fetch origin pull/NUMBER/head
git cherry-pick --no-commit FETCH_HEAD   # the change, not the bot's commit
git commit                               # the operator's commit; then the checks, and push

A merge in GitHub's page, or a plain git cherry-pick, would keep the bot as the author, and each commit of this repository is the operator's (docs/decisions.md). Each crate declares the newest stable Rust of the day (T-685), and an update that needs a newer one raises it.

Dependabot moves no path package, so the crates of vendor/ have two watches of their own (T-341). .github/workflows/upstream.yml reads each week crates.io's newest release of russh and of iroh-relay, and the latest release of tailscale-rs on GitHub, against each copy and the line "Upstream read" of its section of vendor/README.md (scripts/check-vendored.py). And cargo-deny passes over a path package (an advisory planted for russh passed it while vendor/russh held 0.64.1), so .github/workflows/deny.yml reads its copy of advisory-db for each package of the lock files with no source, but podssh's own (scripts/check-vendored-advisories.py), on each push and each day. When the first is red, read the release that it names and judge it: move the copy, or write the release and the day in the line "Upstream read", with the reason. When the second is red, move the copy to a patched release, or judge the advisory and ignore it in deny.toml with the reason, as for any crate. Each check runs its own plant beside it.

Release builds

The released Linux binary is podssh-cli for x86_64-unknown-linux-musl, built with RUSTFLAGS=-Ctarget-feature=+crt-static and the release profile (optimized for size, fat LTO). scripts/gate.sh builds it, and CI uploads it.

.github/workflows/publish.yml (T-686) publishes the crates to crates.io, apart from the binaries: on a tag v*, or dispatched by release.yml's job tag on the tag it pushed (a tag pushed with GITHUB_TOKEN starts no workflow). Its job dry-run packs and builds each crate that the workspace lets out (cargo publish --workspace --dry-run; none while each crate says publish = false); its job publish runs in the environment crates-io, which crates.io trusts, with a token of rust-lang/crates-io-auth-action for 30 minutes, and only when [workspace.metadata.podssh] crates-io in Cargo.toml is true at the tag and build.yml is green at its commit. The switch stays false until the owner rules that podssh publishes (T-687); a new crate name is reserved by the owner first.

.github/workflows/release.yml makes the release binaries, each with the features of its variable RELEASE_FEATURES, the same on each target: the iroh road (T-348) and WARP's road (T-352, Q59). scripts/release-short.sh checks that each binary has both roads, and the gate's step release builds the same features:

  • static musl binaries for x86_64 and aarch64, each built in rust:1-alpine on a native runner, with a check for dynamic dependencies;
  • a static musl binary for armv7 (a Raspberry Pi with 32 bits), built with Alpine's own rust in a container of linux/arm/v7 on an arm64 runner (natively where the runner runs 32-bit ARM, else under QEMU), with the same check (T-218);
  • a static musl binary for riscv64, cross-compiled with cargo-zigbuild (zig 0.16.0, checked by its sum) in rust:1-alpine on a native runner, with the same check; it runs on the runner under QEMU (T-218). Zig comes from a community mirror of scripts/zig-mirrors.txt in a random order, else from ziglang.org, which asks CI not to depend on it; a mirror that serves other bytes is named and passed over (scripts/fetch-zig.sh, T-343). The armv7 job first checks Alpine's rustc against the highest rust-version of the workspace's crates in the binary (scripts/check-rustc.sh), and stops with both named; the weekly upstream.yml compares the copy of the mirrors, and the pin of check-jsonschema, with their sources;
  • Mac binaries for aarch64 and x86_64, built on the macOS runners, with a check (otool -L) that they link the system's libraries only;
  • Windows binaries for x86_64 and ARM64 with a static C runtime, with a check (dumpbin /headers /dependents) of the machine, and that each needs no C runtime DLL;
  • each binary names the commit of its build in --version (the workflow gives PODSSH_BUILD_COMMIT to the compiler; T-210);
  • each binary then runs on its own platform the short form of the release check, scripts/release-short.sh BINARY --net: --version against Cargo.toml and the commit, the whole manual, keygen and keygen -y, and GitHub's SSH banner through the live relay. No target ships that no run executed.

The actions of each job, each named by its commit, with its tag in a comment: a tag can move to another commit after its review, and a commit cannot (T-337; scripts/check-repo.py fails a uses: that names no commit, and Dependabot moves the commit and its comment together):

  • each build job: actions/checkout and actions/upload-artifact; the Windows x86-64 build also ilammy/setup-nasm, for aws-lc. When NASM cannot be set up, aws-lc takes its prebuilt objects (AWS_LC_SYS_PREBUILT_NASM=1), with a warning; the input nasm=prebuilt of a run by hand asks for them, to prove that fallback;
  • the riscv64 job, and the armv7 job where the runner does not run 32-bit ARM, register QEMU's handler with scripts/binfmt.sh, from the image of images/binfmt pinned to its digest and pulled through the mirrors of scripts/pull-image.sh, in place of a third-party action;
  • the job notes, which may only read the repository: actions/checkout, taiki-e/install-action for cargo-about and git-cliff (release binaries, checked by their sums), and actions/upload-artifact. It writes THIRD-PARTY-LICENSES.md and the list of the commits;
  • the job publish, which holds the rights to write the release and to sign as this workflow: actions/checkout, actions/download-artifact, actions/attest, Sigstore's cosign-installer and actions/upload-artifact, and no other tool. Any step of a job with id-token: write can ask for the job's token, so a tool of a third party there could sign other files as this workflow; the tools that the notes need run in notes instead.

To build and check the binaries without a release, run the workflow by hand:

gh workflow run release.yml --ref main

To publish a release:

  1. The version of the tag in Cargo.toml (version), then cargo update -w for Cargo.lock and fuzz/Cargo.lock: each build runs with --locked, and a tag's binary must name the tag's version.

  2. Write the notes in docs/releases/<tag>.md. The workflow fails without this file.

  3. Push the tag vX.Y.Z or vX.Y.Z-pre. A tag with a suffix (-beta.1) becomes a prerelease. Where git may not push a tag (a Claude Code cloud session answers HTTP 403), run the workflow by hand with the input publish set to the tag:

    gh workflow run release.yml --ref main -f publish=vX.Y.Z

    That run builds nothing: its job tag makes the tag at the commit that the run was started on, once (it fails when the tag exists), and starts the workflow by hand on the tag, as a tag that the job's token pushes starts no workflow. The run on the tag builds, attests, signs and publishes. When main already holds newer commits than the one to tag, push a branch at that commit and give it to --ref (v1.0.0-beta.2 went so, from beta/v1.0.0-beta.2).

  4. On the tag, the workflow adds SHA256SUMS, attests the provenance of each binary (actions/attest, one attestation of SLSA for all of them, T-210), signs SHA256SUMS with Sigstore's keyless signing (cosign sign-blob, T-211), checks the signature with the identity of the workflow at the tag, and checks that a changed copy and a branch's identity fail. It publishes the binaries, SHA256SUMS, SHA256SUMS.sigstore.json and provenance.sigstore.json (the bundle of the attestation). The body is the notes file, then the list of the commits since the previous tag: each commit, grouped by its subject (cliff.toml, read by a pinned git-cliff). No CHANGELOG.md is kept in the tree: git holds the history. A run by hand on a branch makes no proof: its identity would name the branch. The attestation and the signature use Sigstore's public instance: when it is down, the step fails before anything is published, and the failed jobs of the tag's run are run again (gh run rerun RUN --failed); the tag stays where it is (T-343).

  5. In the commit after the tag, docs/STATUS.md names the tag as the newest release in its head, with its date and commit, and each row that says unreleased and whose commit the tag holds names the tag in its place (git merge-base --is-ancestor COMMIT TAG, or git tag --contains COMMIT). python scripts/check-repo.py fails a row that still says unreleased with a date before the tag's (T-594).

The list can be read at any time, as an artifact named changes; the run also checks that no commit is dropped:

gh workflow run changelog.yml --ref main

The plan of releases (the operator, 2026-10-30): betas v1.0.0-beta.N, each a prerelease, then v1.0.0, the final release (T-309, T-250). A beta is tagged on a commit where the gate passed, after the release workflow and the changelog workflow ran by hand on it, and after the checks of its binary (--version, static, man, doctor, proxy and ssh through the relay). v1.0.0 comes when each other entry is done, except the relay's: the check from end to end with no human (T-251) runs before the tag, on the gate's artifact of the last push and a local build; then v1.0.0 is tagged once, and the check runs again on the published files. A tag never moves: a defect found after a tag goes into the next beta, or into v1.0.1. From T-210 and T-211 on (v1.0.0-beta.2), each release also has provenance and Sigstore signatures; the README tells how to check them.

Once a tag's release is published, its proofs are checked as a user checks them, on a fresh runner with the job's token and cosign (this host may have neither):

gh workflow run release-proofs.yml --ref main -f tag=vX.Y.Z

The run downloads the published files, checks SHA256SUMS against each binary, the provenance of each binary (gh attestation verify, through the API and with provenance.sigstore.json: the release workflow at the tag, and the tag's commit) and the signature of SHA256SUMS (cosign 3.1.4 and 2.6.5: the release workflow at the tag), requires that a changed copy fails each check, and runs scripts/release-short.sh with --net on the published x86_64 binary, whose --version must name the tag's commit. Then podssh update (T-375): the published binary replaces itself with its release's, and the binary of the release before updates to it, each then naming the tag's commit; a binary from before T-375 is passed over, as it has no podssh update.

The check of a release from end to end, with no human (T-251), is scripts/release-check.sh. Before the tag it takes the gate's static binary of the last push (the artifact of build.yml, which gh fetches with its login, gh run download) and a build of this host; after the tag it downloads the published files and checks SHA256SUMS, its signature and the provenance of each binary as the run above does, before any binary runs (a missing cosign or gh fails). Then each check that this host can run, each with a time limit, and a row for each run of GitHub Actions that the others need, whose conclusion it reads through the public API when the session names the run. docs/STATUS.md ("The release check") tells how a session runs it and records its table:

sh scripts/release-check.sh --list --local       # each row: its entry, where it runs, its limit
gh run download RUN -R PodUtils/podssh -n podssh-x86_64-unknown-linux-musl -D .work/artifact
sh scripts/release-check.sh --local --binary .work/artifact/podssh --actions-run build=RUN
sh scripts/release-check.sh vX.Y.Z --actions-run release=RUN    # from a checkout of the tag

Its own plant, the one of T-251's Prove: a byte of a binary changed in a download of the release (--dir DIR), or after --sums before the tag, fails the row sums.check, and the run stops with "no check ran" and exit 1.

podssh update from a build of a commit, on Linux, macOS and Windows, to a published release (the newest by default):

gh workflow run update-live.yml --ref main -f tag=vX.Y.Z

What a user feels (T-660)

scripts/feel-matrix.py measures what a user feels on each road, through each of the four links of osewa's lab (docs/research/osewa-paper.md, section 3.1): a key's echo idle and beside a download on the same connection, cold connects with true, and the download's rate while one types. Each cell's raw file goes under --out, with summary.tsv. It shapes the links in one of two ways:

  • --links proxy, the default: the stand-in proxy of scripts/fake-proxy.py, with its --shape, in front of each client's first hop, with the links of scripts/fake_shape.py. No root; each road of podssh runs, and OpenSSH's client as the reference.
  • --links netem: tc netem on the loopback (scripts/feel_netem.py), the same links read from scripts/fake_shape.py, each way on a netem of its own for the far ends' ports alone, the loopback's MTU at 1500 and its offloads off; all put back at the end. It needs root, or sudo -n. Only the roads of one host run: OpenSSH's client, --direct, and osewa's release binary over QUIC and over its WebSocket (scripts/feel_osewa.py, a reference and never a road), whose server and files live in the run's scratch directory and never update themselves. It is the reference for the links in user space.
python3 scripts/feel-matrix.py target/x86_64-unknown-linux-musl/release/podssh --runs 3
python3 -I scripts/feel-matrix.py PODSSH --links netem --osewa OSEWA --runs 3   # root, or sudo -n
gh workflow run feel-netem.yml --ref main -f tag=v1.1.0-beta.3
gh workflow run feel-netem.yml --ref main -f run=RUN   # a run's artifact podssh-x86_64-unknown-linux-musl

The workflow feel-netem.yml, by hand, runs the matrix with netem on a runner of GitHub Actions, which has root: podssh from a release, checked against its SHA256SUMS, or from a run's artifact; osewa's release, pinned by its version and the SHA-256 of its file. It uploads the raw files as an artifact, writes the summary in the step summary, and puts the loopback back after the run, also when the run ended early.