From c48b6417f1d19b27815958a7ca80645de33219a6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 15 Jun 2026 12:44:34 +0200 Subject: [PATCH 001/138] docs(plans): add NetBird dynamic WireGuard implementation plan Co-authored-by: Cursor --- .../2026-06-15-netbird-dynamic-wireguard.md | 913 ++++++++++++++++++ 1 file changed, 913 insertions(+) create mode 100644 docs/superpowers/plans/2026-06-15-netbird-dynamic-wireguard.md diff --git a/docs/superpowers/plans/2026-06-15-netbird-dynamic-wireguard.md b/docs/superpowers/plans/2026-06-15-netbird-dynamic-wireguard.md new file mode 100644 index 00000000..c2ac7ac1 --- /dev/null +++ b/docs/superpowers/plans/2026-06-15-netbird-dynamic-wireguard.md @@ -0,0 +1,913 @@ +# NetBird Dynamic WireGuard Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a NetBird client container to sandcat's proxy stack so that the WireGuard network layer becomes dynamically controllable at runtime — enabling routes and peer access to be added or removed without restarting any container. + +**Architecture:** A new `netbird` service runs as a companion to `wg-client` in `compose-proxy.yml`. It connects to a NetBird management server (self-hosted or NetBird Cloud) using an enrollment key from user settings. `wg-client-init.sh` continues to manage the mitmproxy WireGuard tunnel unchanged; NetBird owns a separate `wg` interface (`wgnetbird`) that provides the dynamic overlay network. The `sandcat netbird` CLI subcommand surfaces `up`, `down`, `route add`, `route remove`, and `status` for runtime control via the NetBird management API (no container restarts required). This is pure infrastructure — no research instrumentation in this plan. + +**Tech Stack:** Bash (`bats`, `bats-mock-ext`, `yq`), Docker Compose, NetBird client Docker image (`netbirdio/netbird`), NetBird management REST API (v1). + +--- + +## File Structure and Responsibilities + +- Create: `cli/templates/devcontainer/sandcat/compose-netbird.yml` + - Defines the `netbird` service: image, volumes, caps, env, healthcheck. +- Modify: `cli/templates/devcontainer/compose-all.yml` + - Add `include` for `compose-netbird.yml` (opt-in; only included when NetBird is enabled). +- Modify: `cli/lib/composefile.bash` + - Add `enable_netbird` function that patches `compose-all.yml` to include `compose-netbird.yml`. +- Modify: `cli/libexec/init/devcontainer` + - Accept `--netbird` flag; call `enable_netbird` when present. +- Modify: `cli/libexec/init/init` + - Add `--netbird` flag; seed `netbird_enrollment_key` in user settings when flag is set; pass flag to `devcontainer`. +- Create: `cli/libexec/netbird/netbird` + - Subcommand dispatcher: `up`, `down`, `route`, `status` — all implemented as calls to the NetBird management REST API via `curl` + the enrollment key from settings. +- Create: `cli/lib/netbird.bash` + - Pure helper functions: `netbird_api`, `netbird_up`, `netbird_down`, `netbird_route_add`, `netbird_route_remove`, `netbird_status`. All testable without Docker. +- Modify: `cli/libexec/init/settings` + - Add `netbird_enrollment_key` field handling. +- Create: `cli/test/composefile/netbird.bats` + - Tests for `enable_netbird` function. +- Create: `cli/test/netbird/test_helper.bash` + - Standard test setup for netbird tests. +- Create: `cli/test/netbird/netbird_api.bats` + - Unit tests for `netbird.bash` helper functions. +- Create: `cli/test/netbird/netbird.bats` + - Integration tests for the `netbird` subcommand dispatcher. +- Modify: `cli/test/init/init.bats` + - Add tests for `--netbird` flag plumbing. +- Modify: `cli/README.md` + - Document `--netbird` init flag, settings key, and `sandcat netbird` subcommand. + +--- + +### Task 1: Add `netbird` service Compose definition + +**Files:** +- Create: `cli/templates/devcontainer/sandcat/compose-netbird.yml` + +- [ ] **Step 1: Write the compose service definition** + +```yaml +# cli/templates/devcontainer/sandcat/compose-netbird.yml +services: + netbird: + image: netbirdio/netbird:latest + cap_add: + - NET_ADMIN + - SYS_ADMIN + - SYS_RESOURCE + environment: + - NB_SETUP_KEY + volumes: + - netbird-config:/etc/netbird + restart: unless-stopped + healthcheck: + test: ["CMD", "netbird", "status"] + interval: 5s + timeout: 5s + retries: 12 + +volumes: + netbird-config: +``` + +- [ ] **Step 2: Validate the file parses correctly** + +Run: `yq '.' cli/templates/devcontainer/sandcat/compose-netbird.yml` +Expected: YAML printed without errors. + +- [ ] **Step 3: Commit** + +```bash +git add cli/templates/devcontainer/sandcat/compose-netbird.yml +git commit -m "feat(netbird): add netbird service compose definition" +``` + +--- + +### Task 2: Add `enable_netbird` compose function and tests + +**Files:** +- Modify: `cli/lib/composefile.bash` +- Create: `cli/test/composefile/netbird.bats` + +- [ ] **Step 1: Write the failing BATS test** + +```bash +# cli/test/composefile/netbird.bats +#!/usr/bin/env bats + +setup() { + load test_helper + source "$SCT_LIBDIR/composefile.bash" + + COMPOSE_FILE="$BATS_TEST_TMPDIR/compose-all.yml" + cat >"$COMPOSE_FILE" <<'YAML' +include: + - path: sandcat/compose-proxy.yml +services: + agent: + image: placeholder +YAML +} + +teardown() { + unstub_all +} + +@test "enable_netbird adds netbird include to compose-all.yml" { + enable_netbird "$COMPOSE_FILE" + + yq -e '.include[] | select(.path == "sandcat/compose-netbird.yml")' "$COMPOSE_FILE" +} + +@test "enable_netbird is idempotent" { + enable_netbird "$COMPOSE_FILE" + enable_netbird "$COMPOSE_FILE" + + run yq '[.include[] | select(.path == "sandcat/compose-netbird.yml")] | length' "$COMPOSE_FILE" + assert_output "1" +} +``` + +- [ ] **Step 2: Run the tests to confirm they fail** + +Run: `cd cli && bats test/composefile/netbird.bats` +Expected: FAIL — `enable_netbird` is not defined. + +- [ ] **Step 3: Implement `enable_netbird` in `cli/lib/composefile.bash`** + +Add at the end of the file: + +```bash +# Adds the NetBird compose include to compose-all.yml if not already present. +# Args: +# $1 - Path to compose-all.yml +enable_netbird() { + require yq + local compose_file=$1 + + local already_included + already_included=$(yq '[.include[] | select(.path == "sandcat/compose-netbird.yml")] | length' "$compose_file") + + if [[ "$already_included" -eq 0 ]]; then + yq -i '.include += [{"path": "sandcat/compose-netbird.yml"}]' "$compose_file" + fi +} +``` + +- [ ] **Step 4: Run the tests to confirm they pass** + +Run: `cd cli && bats test/composefile/netbird.bats` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add cli/lib/composefile.bash cli/test/composefile/netbird.bats +git commit -m "feat(netbird): add enable_netbird compose helper with idempotency" +``` + +--- + +### Task 3: Add NetBird pure API helpers and unit tests + +**Files:** +- Create: `cli/lib/netbird.bash` +- Create: `cli/test/netbird/test_helper.bash` +- Create: `cli/test/netbird/netbird_api.bats` + +- [ ] **Step 1: Create the test helper** + +```bash +# cli/test/netbird/test_helper.bash +#!/bin/bash +bats_require_minimum_version 1.5.0 +if shopt -s compat32 2>/dev/null; then + export BASH_COMPAT=3.2 +fi +set -uo pipefail +export SHELLOPTS + +SCT_ROOT="$BATS_TEST_DIRNAME/../.." +BATS_LIB_PATH="$SCT_ROOT/support":${BATS_LIB_PATH-} + +bats_load_library bats-ext +bats_load_library bats-support +bats_load_library bats-assert +bats_load_library bats-mock-ext + +export SCT_ROOT SCT_LIBDIR="$SCT_ROOT/lib" +``` + +- [ ] **Step 2: Write the failing unit tests** + +```bash +# cli/test/netbird/netbird_api.bats +#!/usr/bin/env bats + +setup() { + load test_helper + source "$SCT_LIBDIR/netbird.bash" + + export NB_MANAGEMENT_URL="https://api.netbird.io" + export NB_API_TOKEN="test-token" +} + +teardown() { + unstub_all +} + +@test "netbird_api calls curl with auth header and endpoint" { + stub curl \ + "-sf -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '{\"peers\":[]}\n'" + run netbird_api "GET" "/api/peers" + assert_success +} + +@test "netbird_api fails when NB_API_TOKEN is unset" { + unset NB_API_TOKEN + run netbird_api "GET" "/api/peers" + assert_failure + assert_output --partial "NB_API_TOKEN" +} + +@test "netbird_status returns structured peer list" { + stub curl \ + "-sf -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[{\"id\":\"peer1\",\"name\":\"test\",\"connected\":true}]'" + run netbird_status + assert_success + assert_output --partial "peer1" +} + +@test "netbird_route_add calls routes API with network and peer" { + stub curl \ + "-sf -X POST -H 'Authorization: Token test-token' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"peer1\",\"enabled\":true}' https://api.netbird.io/api/routes : echo '{\"id\":\"route1\"}'" + run netbird_route_add "10.8.0.0/24" "peer1" + assert_success +} + +@test "netbird_route_remove calls DELETE on routes API" { + stub curl \ + "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/routes/route1 : :" + run netbird_route_remove "route1" + assert_success +} + +@test "netbird_up calls peers API to enable peer" { + stub curl \ + "-sf -X PUT -H 'Authorization: Token test-token' -H 'Content-Type: application/json' -d '{\"login_expiration_enabled\":false}' https://api.netbird.io/api/peers/peer1 : echo '{\"id\":\"peer1\"}'" + run netbird_up "peer1" + assert_success +} + +@test "netbird_down calls peers API to disable peer" { + stub curl \ + "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers/peer1 : :" + run netbird_down "peer1" + assert_success +} +``` + +- [ ] **Step 3: Run the tests to confirm they fail** + +Run: `cd cli && bats test/netbird/netbird_api.bats` +Expected: FAIL — `cli/lib/netbird.bash` does not exist. + +- [ ] **Step 4: Implement `cli/lib/netbird.bash`** + +```bash +# cli/lib/netbird.bash +#!/usr/bin/env bash + +# Calls the NetBird management REST API. +# Requires NB_MANAGEMENT_URL and NB_API_TOKEN to be set in the environment. +# Args: +# $1 - HTTP method (GET, POST, PUT, DELETE) +# $2 - API path (e.g. /api/peers) +# $3 - Optional JSON body string +netbird_api() { + local method=$1 + local path=$2 + local body=${3:-} + + if [[ -z "${NB_API_TOKEN:-}" ]]; then + echo "NB_API_TOKEN is not set" >&2 + return 1 + fi + + local url="${NB_MANAGEMENT_URL:-https://api.netbird.io}${path}" + local args=(-sf -X "$method" + -H "Authorization: Token $NB_API_TOKEN" + -H "Content-Type: application/json") + + if [[ -n "$body" ]]; then + args+=(-d "$body") + fi + + curl "${args[@]}" "$url" +} + +# Returns the current list of peers from the NetBird management server. +netbird_status() { + netbird_api "GET" "/api/peers" +} + +# Enables a peer by ID on the NetBird management server. +# Args: +# $1 - Peer ID +netbird_up() { + local peer_id=$1 + netbird_api "PUT" "/api/peers/$peer_id" '{"login_expiration_enabled":false}' +} + +# Removes a peer by ID from the NetBird management server. +# Args: +# $1 - Peer ID +netbird_down() { + local peer_id=$1 + netbird_api "DELETE" "/api/peers/$peer_id" +} + +# Adds a network route to the NetBird management server. +# Args: +# $1 - Network CIDR (e.g. 10.8.0.0/24) +# $2 - Peer ID that serves the route +netbird_route_add() { + local network=$1 + local peer_id=$2 + netbird_api "POST" "/api/routes" "{\"network\":\"$network\",\"peer\":\"$peer_id\",\"enabled\":true}" +} + +# Removes a network route by ID from the NetBird management server. +# Args: +# $1 - Route ID +netbird_route_remove() { + local route_id=$1 + netbird_api "DELETE" "/api/routes/$route_id" +} +``` + +- [ ] **Step 5: Run the tests to confirm they pass** + +Run: `cd cli && bats test/netbird/netbird_api.bats` +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add cli/lib/netbird.bash cli/test/netbird/test_helper.bash cli/test/netbird/netbird_api.bats +git commit -m "feat(netbird): add pure netbird API helpers with unit tests" +``` + +--- + +### Task 4: Add `sandcat netbird` subcommand dispatcher and integration tests + +**Files:** +- Create: `cli/libexec/netbird/netbird` +- Create: `cli/test/netbird/netbird.bats` + +- [ ] **Step 1: Write the failing integration tests** + +```bash +# cli/test/netbird/netbird.bats +#!/usr/bin/env bats + +setup() { + load test_helper + + NETBIRD_CMD="$SCT_LIBEXECDIR/netbird/netbird" + export NB_MANAGEMENT_URL="https://api.netbird.io" + export NB_API_TOKEN="test-token" +} + +teardown() { + unstub_all +} + +@test "netbird status calls netbird_status" { + stub curl \ + "-sf -X GET -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[]'" + run bash "$NETBIRD_CMD" status + assert_success +} + +@test "netbird up requires peer-id argument" { + run bash "$NETBIRD_CMD" up + assert_failure + assert_output --partial "peer-id" +} + +@test "netbird up calls netbird_up with peer-id" { + stub curl \ + "-sf -X PUT -H 'Authorization: Token test-token' -H 'Content-Type: application/json' -d '{\"login_expiration_enabled\":false}' https://api.netbird.io/api/peers/abc123 : echo '{\"id\":\"abc123\"}'" + run bash "$NETBIRD_CMD" up --peer-id abc123 + assert_success +} + +@test "netbird down requires peer-id argument" { + run bash "$NETBIRD_CMD" down + assert_failure + assert_output --partial "peer-id" +} + +@test "netbird down calls netbird_down with peer-id" { + stub curl \ + "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers/abc123 : :" + run bash "$NETBIRD_CMD" down --peer-id abc123 + assert_success +} + +@test "netbird route add requires --network and --peer-id" { + run bash "$NETBIRD_CMD" route add --network 10.8.0.0/24 + assert_failure + assert_output --partial "peer-id" +} + +@test "netbird route add calls netbird_route_add" { + stub curl \ + "-sf -X POST -H 'Authorization: Token test-token' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"abc123\",\"enabled\":true}' https://api.netbird.io/api/routes : echo '{\"id\":\"route1\"}'" + run bash "$NETBIRD_CMD" route add --network 10.8.0.0/24 --peer-id abc123 + assert_success +} + +@test "netbird route remove requires --route-id" { + run bash "$NETBIRD_CMD" route remove + assert_failure + assert_output --partial "route-id" +} + +@test "netbird route remove calls netbird_route_remove" { + stub curl \ + "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/routes/route1 : :" + run bash "$NETBIRD_CMD" route remove --route-id route1 + assert_success +} + +@test "netbird with unknown subcommand prints usage and fails" { + run bash "$NETBIRD_CMD" bogus + assert_failure + assert_output --partial "Usage" +} +``` + +- [ ] **Step 2: Run the tests to confirm they fail** + +Run: `cd cli && bats test/netbird/netbird.bats` +Expected: FAIL — `cli/libexec/netbird/netbird` does not exist. + +- [ ] **Step 3: Create the dispatcher** + +```bash +#!/usr/bin/env bash +# cli/libexec/netbird/netbird +set -euo pipefail + +# shellcheck source=../../lib/logging.bash +source "$SCT_LIBDIR/logging.bash" +# shellcheck source=../../lib/netbird.bash +source "$SCT_LIBDIR/netbird.bash" + +usage() { + cat <<'EOF' +Usage: sandcat netbird [options] + +Subcommands: + status List connected NetBird peers + up --peer-id Enable a peer on the NetBird management server + down --peer-id Remove a peer from the NetBird management server + route add --network --peer-id Add a route + route remove --route-id Remove a route by ID +EOF +} + +cmd_status() { + netbird_status +} + +cmd_up() { + local peer_id="" + while [[ $# -gt 0 ]]; do + case $1 in + --peer-id) peer_id="$2"; shift 2 ;; + *) echo "Unknown option: $1" | error; return 1 ;; + esac + done + if [[ -z "$peer_id" ]]; then + echo "Missing required option: --peer-id" | error + return 1 + fi + netbird_up "$peer_id" +} + +cmd_down() { + local peer_id="" + while [[ $# -gt 0 ]]; do + case $1 in + --peer-id) peer_id="$2"; shift 2 ;; + *) echo "Unknown option: $1" | error; return 1 ;; + esac + done + if [[ -z "$peer_id" ]]; then + echo "Missing required option: --peer-id" | error + return 1 + fi + netbird_down "$peer_id" +} + +cmd_route() { + local subcmd="${1:-}" + shift || true + case "$subcmd" in + add) + local network="" peer_id="" + while [[ $# -gt 0 ]]; do + case $1 in + --network) network="$2"; shift 2 ;; + --peer-id) peer_id="$2"; shift 2 ;; + *) echo "Unknown option: $1" | error; return 1 ;; + esac + done + if [[ -z "$network" ]]; then + echo "Missing required option: --network" | error; return 1 + fi + if [[ -z "$peer_id" ]]; then + echo "Missing required option: --peer-id" | error; return 1 + fi + netbird_route_add "$network" "$peer_id" + ;; + remove) + local route_id="" + while [[ $# -gt 0 ]]; do + case $1 in + --route-id) route_id="$2"; shift 2 ;; + *) echo "Unknown option: $1" | error; return 1 ;; + esac + done + if [[ -z "$route_id" ]]; then + echo "Missing required option: --route-id" | error; return 1 + fi + netbird_route_remove "$route_id" + ;; + *) + echo "Unknown route subcommand: $subcmd" | error + usage; return 1 + ;; + esac +} + +main() { + local subcmd="${1:-}" + shift || true + case "$subcmd" in + status) cmd_status "$@" ;; + up) cmd_up "$@" ;; + down) cmd_down "$@" ;; + route) cmd_route "$@" ;; + *) + usage + return 1 + ;; + esac +} + +if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + main "$@" +fi +``` + +- [ ] **Step 4: Make it executable** + +Run: `chmod +x cli/libexec/netbird/netbird` + +- [ ] **Step 5: Run the integration tests to confirm they pass** + +Run: `cd cli && bats test/netbird/netbird.bats` +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add cli/libexec/netbird/netbird cli/test/netbird/netbird.bats +git commit -m "feat(netbird): add sandcat netbird subcommand dispatcher" +``` + +--- + +### Task 5: Wire `--netbird` flag into `init devcontainer` and `init` + +**Files:** +- Modify: `cli/libexec/init/devcontainer` +- Modify: `cli/libexec/init/init` +- Modify: `cli/test/init/init.bats` + +- [ ] **Step 1: Write the failing BATS tests for the new flags** + +Add to `cli/test/init/init.bats`: + +```bash +@test "init --netbird passes netbird flag to devcontainer" { + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird : :" + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --netbird + assert_success +} + +@test "init --netbird seeds netbird_enrollment_key in user settings" { + stub devcontainer "*: :" + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --netbird + run yq '.netbird_enrollment_key' "$SCT_HOME_DIR/settings.json" + assert_output '""' +} +``` + +- [ ] **Step 2: Run the targeted tests to confirm they fail** + +Run: `cd cli && bats test/init/init.bats --filter "netbird"` +Expected: FAIL — `--netbird` is not recognized by `init`. + +- [ ] **Step 3: Add `--netbird` to `cli/libexec/init/devcontainer`** + +In the `while [[ $# -gt 0 ]]` argument-parsing loop, add: + +```bash +--netbird) + netbird="true" + shift 1 + ;; +``` + +Add `local netbird="false"` at the top of the function alongside the other locals. After the `apply_secret_provider` call, add: + +```bash +if [[ "$netbird" == "true" ]]; then + enable_netbird "$compose_file" +fi +``` + +Also add `--netbird` to the case branch that checks for options requiring values: + +```bash +--settings-file|--project-path|--agent|--ide|--name|--stacks|--proxy|--secret-provider) +``` + +becomes: + +```bash +--settings-file|--project-path|--agent|--ide|--name|--stacks|--proxy|--secret-provider) +``` + +`--netbird` takes no value so it stays outside that branch. + +- [ ] **Step 4: Add `--netbird` to `cli/libexec/init/init`** + +Add `local netbird="false"` near the other locals. In the argument-parsing loop add: + +```bash +--netbird) + netbird="true" + shift 1 + ;; +``` + +In the `add_secret_provider_tokens_to_user_settings` call site, add alongside it: + +```bash +if [[ "$netbird" == "true" ]]; then + yq -i -o json '.netbird_enrollment_key = (.netbird_enrollment_key // "")' "$user_settings_file" +fi +``` + +In the `devcontainer` invocation, append `${netbird:+--netbird}`: + +```bash +devcontainer \ + --settings-file "$rel_settings_file" \ + ... \ + --secret-provider "$secret_provider" \ + ${netbird:+--netbird} +``` + +- [ ] **Step 5: Run all init tests** + +Run: `cd cli && bats test/init/init.bats` +Expected: PASS for all existing and new tests. + +- [ ] **Step 6: Commit** + +```bash +git add cli/libexec/init/devcontainer cli/libexec/init/init cli/test/init/init.bats +git commit -m "feat(init): add --netbird flag and enrollment key seeding" +``` + +--- + +### Task 6: Add compose contract tests for the netbird include + +**Files:** +- Create: `cli/test/composefile/netbird_contract.bats` + +- [ ] **Step 1: Write the contract tests** + +```bash +# cli/test/composefile/netbird_contract.bats +#!/usr/bin/env bats +# +# Verifies that compose-netbird.yml and compose-all.yml meet structural contracts +# that other components rely on. +# + +setup() { + load test_helper + COMPOSE_NETBIRD="$SCT_TEMPLATEDIR/devcontainer/sandcat/compose-netbird.yml" + COMPOSE_ALL="$SCT_TEMPLATEDIR/devcontainer/compose-all.yml" +} + +@test "compose-netbird.yml defines netbird service" { + yq -e '.services.netbird' "$COMPOSE_NETBIRD" +} + +@test "compose-netbird.yml netbird service has NET_ADMIN capability" { + yq -e '.services.netbird.cap_add[] | select(. == "NET_ADMIN")' "$COMPOSE_NETBIRD" +} + +@test "compose-netbird.yml netbird service has NB_SETUP_KEY environment entry" { + yq -e '.services.netbird.environment[] | select(. == "NB_SETUP_KEY")' "$COMPOSE_NETBIRD" +} + +@test "compose-netbird.yml netbird service has healthcheck" { + yq -e '.services.netbird.healthcheck' "$COMPOSE_NETBIRD" +} + +@test "compose-netbird.yml netbird service uses netbird-config named volume" { + yq -e '.volumes.netbird-config' "$COMPOSE_NETBIRD" +} + +@test "compose-all.yml does not include compose-netbird.yml by default" { + run yq '.include[] | select(.path == "sandcat/compose-netbird.yml")' "$COMPOSE_ALL" + assert_output "" +} +``` + +- [ ] **Step 2: Run the contract tests** + +Run: `cd cli && bats test/composefile/netbird_contract.bats` +Expected: PASS. (The last test verifies the template ships without NetBird by default.) + +- [ ] **Step 3: Commit** + +```bash +git add cli/test/composefile/netbird_contract.bats +git commit -m "test(netbird): add compose contract tests for netbird service" +``` + +--- + +### Task 7: Update CLI docs + +**Files:** +- Modify: `cli/README.md` + +- [ ] **Step 1: Add NetBird section to README** + +Locate the `--secret-provider` option description in `cli/README.md` and add immediately after it: + +```markdown +- `--netbird` - Enable dynamic WireGuard control via NetBird. Adds a companion + `netbird` container to the proxy stack and seeds `netbird_enrollment_key` in + your user settings (`~/.config/sandcat/settings.json`). Fill in the key before + starting the devcontainer. +``` + +Find the init usage examples and add: + +```bash +# With NetBird dynamic WireGuard +sandcat init --agent claude --ide vscode --netbird --name myproject +``` + +Add a new section `## Dynamic networking (NetBird)`: + +```markdown +## Dynamic networking (NetBird) + +When initialized with `--netbird`, sandcat adds a companion [NetBird](https://netbird.io) +container that connects to a NetBird management server. This makes the WireGuard +network layer controllable at runtime — routes and peer access can be added or +removed without restarting any container. + +### Setup + +1. Create a NetBird account at or self-host the management server. +2. Generate a setup key in the NetBird dashboard under **Setup Keys**. +3. Add the key to your sandcat user settings: + +```json +{ + "netbird_enrollment_key": "your-setup-key-here" +} +``` + +4. Set the NetBird API token in your shell before using `sandcat netbird` commands: + +```bash +export NB_API_TOKEN="your-management-api-token" +export NB_MANAGEMENT_URL="https://api.netbird.io" # or your self-hosted URL +``` + +### Runtime control + +```bash +# List connected peers +sandcat netbird status + +# Enable a peer +sandcat netbird up --peer-id + +# Remove a peer +sandcat netbird down --peer-id + +# Add a network route served by a peer +sandcat netbird route add --network 10.8.0.0/24 --peer-id + +# Remove a route by ID (ID returned by route add) +sandcat netbird route remove --route-id +``` +``` + +- [ ] **Step 2: Verify no old NetBird references remain undefined** + +Run: `rg --line-number "netbird" cli/README.md` +Expected: Only the newly added lines appear. + +- [ ] **Step 3: Commit** + +```bash +git add cli/README.md +git commit -m "docs(cli): document --netbird flag and sandcat netbird subcommand" +``` + +--- + +### Task 8: Final verification + +**Files:** None modified (verification only). + +- [ ] **Step 1: Run full composefile test suite** + +Run: `cd cli && bats test/composefile/` +Expected: PASS (all composefile tests including new netbird tests). + +- [ ] **Step 2: Run full netbird test suite** + +Run: `cd cli && bats test/netbird/` +Expected: PASS. + +- [ ] **Step 3: Run full init test suite** + +Run: `cd cli && bats test/init/` +Expected: PASS. + +- [ ] **Step 4: Confirm git status is clean** + +Run: `git status --short` +Expected: No untracked or modified files. + +- [ ] **Step 5: Confirm all new files are committed** + +Run: `git log --oneline -8` +Expected: All tasks committed individually with descriptive messages. + +--- + +## Plan Self-Review + +### 1. Spec coverage check + +| Requirement | Covered by | +|---|---| +| NetBird companion container definition | Task 1 | +| `enable_netbird` compose helper | Task 2 | +| Pure API helpers (status, up, down, route add/remove) | Task 3 | +| `sandcat netbird` subcommand dispatcher | Task 4 | +| `--netbird` flag in `init` and `init devcontainer` | Task 5 | +| `netbird_enrollment_key` seeded in user settings | Task 5 | +| Compose contract tests (no default include) | Task 6 | +| README docs | Task 7 | + +No spec gaps. + +### 2. Placeholder scan + +No TBD/TODO in any step. Every code step contains concrete implementations. Every run step contains expected output. + +### 3. Type/name consistency + +- `NB_API_TOKEN` and `NB_MANAGEMENT_URL` used consistently across `netbird.bash`, the dispatcher, and README. +- `NB_SETUP_KEY` (container enrollment env var) is distinct from `NB_API_TOKEN` (management API token) — both are explicitly named throughout. +- `enable_netbird` called consistently from `composefile.bash` and `init devcontainer`. +- `netbird_enrollment_key` (settings JSON key) used consistently in `init` and README. From 1c3a385a447f2128688bf710882b704d5131f81b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 15 Jun 2026 13:39:10 +0200 Subject: [PATCH 002/138] =?UTF-8?q?docs(plans):=20revise=20netbird=20plan?= =?UTF-8?q?=20=E2=80=94=20manager/consumer=20split,=20no=20caps=20on=20net?= =?UTF-8?q?bird=20container?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Cursor --- .../2026-06-15-netbird-dynamic-wireguard.md | 1117 ++++++++++------- 1 file changed, 691 insertions(+), 426 deletions(-) diff --git a/docs/superpowers/plans/2026-06-15-netbird-dynamic-wireguard.md b/docs/superpowers/plans/2026-06-15-netbird-dynamic-wireguard.md index c2ac7ac1..e4759336 100644 --- a/docs/superpowers/plans/2026-06-15-netbird-dynamic-wireguard.md +++ b/docs/superpowers/plans/2026-06-15-netbird-dynamic-wireguard.md @@ -2,93 +2,395 @@ > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. -**Goal:** Add a NetBird client container to sandcat's proxy stack so that the WireGuard network layer becomes dynamically controllable at runtime — enabling routes and peer access to be added or removed without restarting any container. +**Goal:** Add a NetBird sidecar container to sandcat's proxy stack so that the WireGuard peer configuration on `wg0` becomes dynamically controllable at runtime — enabling routes and peer access to be added or removed without restarting any container. -**Architecture:** A new `netbird` service runs as a companion to `wg-client` in `compose-proxy.yml`. It connects to a NetBird management server (self-hosted or NetBird Cloud) using an enrollment key from user settings. `wg-client-init.sh` continues to manage the mitmproxy WireGuard tunnel unchanged; NetBird owns a separate `wg` interface (`wgnetbird`) that provides the dynamic overlay network. The `sandcat netbird` CLI subcommand surfaces `up`, `down`, `route add`, `route remove`, and `status` for runtime control via the NetBird management API (no container restarts required). This is pure infrastructure — no research instrumentation in this plan. +**Architecture:** The system follows a strict manager/consumer split that mirrors the existing `mitmproxy → wg-client` pattern. A new `netbird` container is a low-privilege management API client: it polls the NetBird management server and writes a WireGuard `syncconf`-compatible peer config file (`peers.conf`) to a shared volume. It owns no network interfaces and requires no kernel capabilities. `wg-client` remains the sole owner of `NET_ADMIN`, `wg0`, and iptables; it gains a `supervise_netbird_config` watcher loop (alongside the existing `supervise_dnsmasq` loop) that detects changes to `peers.conf` and applies them via `wg syncconf wg0`. Physical capability revocation therefore flows: NetBird management server removes a peer → `netbird` container writes updated `peers.conf` → `wg-client` calls `wg syncconf` → agent loses routing to that endpoint. -**Tech Stack:** Bash (`bats`, `bats-mock-ext`, `yq`), Docker Compose, NetBird client Docker image (`netbirdio/netbird`), NetBird management REST API (v1). +**Tech Stack:** Bash (`bats`, `bats-mock-ext`, `yq`, `inotifywait` from `inotify-tools`), Docker Compose, Debian slim + curl + jq (custom `netbird` image), NetBird management REST API (v1). --- ## File Structure and Responsibilities +- Create: `cli/templates/devcontainer/sandcat/Dockerfile.netbird` + - Minimal Debian image with `curl`, `jq`. No caps. Custom entrypoint only. +- Create: `cli/templates/devcontainer/sandcat/scripts/netbird-sync.sh` + - Polls NetBird management API; writes WireGuard-format `peers.conf` to shared volume; loops. - Create: `cli/templates/devcontainer/sandcat/compose-netbird.yml` - - Defines the `netbird` service: image, volumes, caps, env, healthcheck. + - Defines the `netbird` service: no `cap_add`, mounts `netbird-config` volume writable, mounts `mitmproxy-config` read-only (for CA cert). +- Modify: `cli/templates/devcontainer/sandcat/compose-proxy.yml` + - Extend `wg-client` service to mount `netbird-config` volume read-only. +- Modify: `cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh` + - Add `supervise_netbird_config` function: watches `/run/netbird/peers.conf` and calls `wg syncconf wg0` on change. +- Modify: `cli/templates/devcontainer/sandcat/Dockerfile.wg-client` + - Install `inotify-tools` so `inotifywait` is available for the watcher. - Modify: `cli/templates/devcontainer/compose-all.yml` - - Add `include` for `compose-netbird.yml` (opt-in; only included when NetBird is enabled). + - Add `include` for `compose-netbird.yml` (opt-in; included only when NetBird is enabled). - Modify: `cli/lib/composefile.bash` - - Add `enable_netbird` function that patches `compose-all.yml` to include `compose-netbird.yml`. + - Add `enable_netbird` function. - Modify: `cli/libexec/init/devcontainer` - Accept `--netbird` flag; call `enable_netbird` when present. - Modify: `cli/libexec/init/init` - - Add `--netbird` flag; seed `netbird_enrollment_key` in user settings when flag is set; pass flag to `devcontainer`. -- Create: `cli/libexec/netbird/netbird` - - Subcommand dispatcher: `up`, `down`, `route`, `status` — all implemented as calls to the NetBird management REST API via `curl` + the enrollment key from settings. + - Add `--netbird` flag; seed `netbird_enrollment_key` in user settings; pass flag to `devcontainer`. - Create: `cli/lib/netbird.bash` - - Pure helper functions: `netbird_api`, `netbird_up`, `netbird_down`, `netbird_route_add`, `netbird_route_remove`, `netbird_status`. All testable without Docker. -- Modify: `cli/libexec/init/settings` - - Add `netbird_enrollment_key` field handling. + - Pure helper functions for the NetBird management REST API. Testable without Docker. +- Create: `cli/libexec/netbird/netbird` + - Subcommand dispatcher: `up`, `down`, `route`, `status`. - Create: `cli/test/composefile/netbird.bats` - Tests for `enable_netbird` function. +- Create: `cli/test/composefile/netbird_contract.bats` + - Structural contracts: no caps on `netbird` service; `wg-client` mounts `netbird-config` read-only; default template excludes NetBird. - Create: `cli/test/netbird/test_helper.bash` - - Standard test setup for netbird tests. + - Standard test setup. - Create: `cli/test/netbird/netbird_api.bats` - - Unit tests for `netbird.bash` helper functions. + - Unit tests for `netbird.bash` helpers. - Create: `cli/test/netbird/netbird.bats` - - Integration tests for the `netbird` subcommand dispatcher. + - Integration tests for the `netbird` subcommand. +- Create: `cli/test/wg-client/netbird_config.bats` + - Unit tests for `supervise_netbird_config` and `apply_netbird_peers`. - Modify: `cli/test/init/init.bats` - - Add tests for `--netbird` flag plumbing. + - Tests for `--netbird` flag plumbing. - Modify: `cli/README.md` - Document `--netbird` init flag, settings key, and `sandcat netbird` subcommand. --- -### Task 1: Add `netbird` service Compose definition +### Task 1: Add the `netbird` sync container (Dockerfile + script + Compose) **Files:** +- Create: `cli/templates/devcontainer/sandcat/Dockerfile.netbird` +- Create: `cli/templates/devcontainer/sandcat/scripts/netbird-sync.sh` - Create: `cli/templates/devcontainer/sandcat/compose-netbird.yml` -- [ ] **Step 1: Write the compose service definition** +- [ ] **Step 1: Write `Dockerfile.netbird`** + +```dockerfile +# cli/templates/devcontainer/sandcat/Dockerfile.netbird +FROM debian:trixie-slim + +# curl - NetBird management REST API calls +# jq - parse API responses +RUN apt-get update \ + && apt-get install -y --no-install-recommends curl jq \ + && rm -rf /var/lib/apt/lists/* + +COPY scripts/netbird-sync.sh /usr/local/bin/netbird-sync.sh +RUN chmod +x /usr/local/bin/netbird-sync.sh + +ENTRYPOINT ["/usr/local/bin/netbird-sync.sh"] +``` + +- [ ] **Step 2: Write `netbird-sync.sh`** + +```bash +#!/bin/bash +# cli/templates/devcontainer/sandcat/scripts/netbird-sync.sh +# +# Polls the NetBird management API for the current peer list and writes +# a WireGuard syncconf-compatible peer config to /run/netbird/peers.conf. +# wg-client reads this file and applies it to wg0 via `wg syncconf`. +# +# Environment: +# NB_MANAGEMENT_URL - NetBird management server base URL (default: https://api.netbird.io) +# NB_API_TOKEN - NetBird management API token (required) +# NB_POLL_INTERVAL - Seconds between polls (default: 5) +# +set -euo pipefail + +PEERS_CONF="/run/netbird/peers.conf" +PEERS_CONF_TMP="${PEERS_CONF}.tmp" +NB_URL="${NB_MANAGEMENT_URL:-https://api.netbird.io}" +POLL_INTERVAL="${NB_POLL_INTERVAL:-5}" + +if [[ -z "${NB_API_TOKEN:-}" ]]; then + echo "NB_API_TOKEN is required" >&2 + exit 1 +fi + +mkdir -p "$(dirname "$PEERS_CONF")" + +# Fetch the current peer list from NetBird management and write WireGuard +# peer stanzas to $1. Each peer entry from the API provides: +# - public_key : WireGuard public key +# - ip : allowed IP in the overlay (e.g. 100.64.0.1) +# - routes : additional CIDRs this peer serves (may be empty) +write_peers_conf() { + local out="$1" + local peers_json + peers_json=$(curl -sf \ + -H "Authorization: Token $NB_API_TOKEN" \ + -H "Content-Type: application/json" \ + "$NB_URL/api/peers") || { + echo "[netbird-sync] Failed to fetch peers from $NB_URL" >&2 + return 1 + } + + { + echo "# Auto-generated by netbird-sync.sh — do not edit." + echo "$peers_json" | jq -r ' + .[] | select(.connected == true) | + "[Peer]\n" + + "PublicKey = " + .public_key + "\n" + + "AllowedIPs = " + (.ip + "/32," + ( + if .routes then (.routes | join(",")) else "" end + ) | gsub(",+$"; "")) + "\n" + ' + } > "$out" +} + +main() { + echo "[netbird-sync] Starting. Polling $NB_URL every ${POLL_INTERVAL}s." + while true; do + if write_peers_conf "$PEERS_CONF_TMP"; then + if ! diff -q "$PEERS_CONF_TMP" "$PEERS_CONF" >/dev/null 2>&1; then + mv "$PEERS_CONF_TMP" "$PEERS_CONF" + echo "[netbird-sync] peers.conf updated." + fi + fi + sleep "$POLL_INTERVAL" + done +} + +if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + main "$@" +fi +``` + +- [ ] **Step 3: Write `compose-netbird.yml`** ```yaml # cli/templates/devcontainer/sandcat/compose-netbird.yml services: netbird: - image: netbirdio/netbird:latest - cap_add: - - NET_ADMIN - - SYS_ADMIN - - SYS_RESOURCE + build: + context: . + dockerfile: Dockerfile.netbird + # No cap_add — this container only calls the NetBird management REST API + # and writes a config file. It never touches network interfaces or iptables. environment: - - NB_SETUP_KEY + - NB_API_TOKEN + - NB_MANAGEMENT_URL=https://api.netbird.io + - NB_POLL_INTERVAL=5 volumes: - - netbird-config:/etc/netbird + # Write-only: netbird-sync.sh writes peers.conf here. + # wg-client mounts this volume read-only (see compose-proxy.yml). + - netbird-config:/run/netbird + # Read-only: CA cert for TLS verification when calling the management API. + - mitmproxy-config:/mitmproxy-config:ro restart: unless-stopped healthcheck: - test: ["CMD", "netbird", "status"] + test: ["CMD-SHELL", "test -f /run/netbird/peers.conf"] interval: 5s - timeout: 5s + timeout: 3s retries: 12 volumes: netbird-config: ``` -- [ ] **Step 2: Validate the file parses correctly** +- [ ] **Step 4: Validate both YAML files parse** Run: `yq '.' cli/templates/devcontainer/sandcat/compose-netbird.yml` Expected: YAML printed without errors. -- [ ] **Step 3: Commit** +- [ ] **Step 5: Commit** + +```bash +git add \ + cli/templates/devcontainer/sandcat/Dockerfile.netbird \ + cli/templates/devcontainer/sandcat/scripts/netbird-sync.sh \ + cli/templates/devcontainer/sandcat/compose-netbird.yml +git commit -m "feat(netbird): add netbird sync container (Dockerfile, sync script, compose)" +``` + +--- + +### Task 2: Mount `netbird-config` in `wg-client` and add `inotify-tools` to its image + +**Files:** +- Modify: `cli/templates/devcontainer/sandcat/compose-proxy.yml` +- Modify: `cli/templates/devcontainer/sandcat/Dockerfile.wg-client` + +- [ ] **Step 1: Add `netbird-config` read-only volume mount to `wg-client` in `compose-proxy.yml`** + +In the `wg-client` service `volumes` list, add: + +```yaml + # Read-only: peers.conf written by the netbird container. + # wg-client watches this file and calls wg syncconf wg0 on change. + - netbird-config:/run/netbird:ro +``` + +Also declare the external volume at the bottom of the file: + +```yaml +volumes: + mitmproxy-config: + wg-runtime: + netbird-config: + external: true +``` + +(`external: true` because `netbird-config` is defined and owned by `compose-netbird.yml`.) + +- [ ] **Step 2: Add `inotify-tools` to `Dockerfile.wg-client`** + +Change the `apt-get install` line in `Dockerfile.wg-client` from: + +```dockerfile + wireguard-tools iproute2 iptables jq openresolv dnsmasq \ +``` + +to: + +```dockerfile + wireguard-tools iproute2 iptables jq openresolv dnsmasq inotify-tools \ +``` + +- [ ] **Step 3: Verify both files are valid** + +Run: `yq '.' cli/templates/devcontainer/sandcat/compose-proxy.yml` +Expected: YAML printed without errors. + +Run: `grep inotify cli/templates/devcontainer/sandcat/Dockerfile.wg-client` +Expected: Line with `inotify-tools` present. + +- [ ] **Step 4: Commit** + +```bash +git add \ + cli/templates/devcontainer/sandcat/compose-proxy.yml \ + cli/templates/devcontainer/sandcat/Dockerfile.wg-client +git commit -m "feat(netbird): mount netbird-config in wg-client and add inotify-tools" +``` + +--- + +### Task 3: Add `supervise_netbird_config` watcher to `wg-client-init.sh` with tests + +**Files:** +- Modify: `cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh` +- Create: `cli/test/wg-client/netbird_config.bats` + +- [ ] **Step 1: Write the failing BATS tests** + +```bash +# cli/test/wg-client/netbird_config.bats +#!/usr/bin/env bats + +setup() { + load test_helper + PEERS_CONF="$BATS_TEST_TMPDIR/peers.conf" + WG_IFACE="wg0" +} + +teardown() { + unstub_all +} + +@test "apply_netbird_peers calls wg syncconf with peers.conf" { + touch "$PEERS_CONF" + stub wg \ + "syncconf wg0 $PEERS_CONF : :" + apply_netbird_peers "$WG_IFACE" "$PEERS_CONF" +} + +@test "apply_netbird_peers does nothing when peers.conf is absent" { + run apply_netbird_peers "$WG_IFACE" "$BATS_TEST_TMPDIR/missing.conf" + assert_success +} + +@test "supervise_netbird_config returns immediately when netbird is not enabled" { + # No peers.conf present and no inotifywait stub — function must return 0 + # without blocking. + run supervise_netbird_config "$WG_IFACE" "$BATS_TEST_TMPDIR/missing.conf" + assert_success +} +``` + +- [ ] **Step 2: Run the tests to confirm they fail** + +Run: `cd cli && bats test/wg-client/netbird_config.bats` +Expected: FAIL — `apply_netbird_peers` and `supervise_netbird_config` are not defined. + +- [ ] **Step 3: Add the functions to `wg-client-init.sh`** + +Add before the `main()` function: + +```bash +# Apply the current NetBird peer config to the WireGuard interface. +# Uses `wg syncconf` so only the peer list changes; keys and listen port +# on wg0 (managed by main()) are untouched. +# Args: +# $1 - WireGuard interface name (e.g. wg0) +# $2 - Path to the peers.conf file written by the netbird container +apply_netbird_peers() { + local iface=$1 + local peers_conf=$2 + [[ -f "$peers_conf" ]] || return 0 + wg syncconf "$iface" "$peers_conf" +} + +# Watch peers.conf for changes and call apply_netbird_peers on each change. +# Returns immediately (no-op) if peers.conf does not exist, so this function +# is safe to call unconditionally regardless of whether NetBird is enabled. +# Args: +# $1 - WireGuard interface name (e.g. wg0) +# $2 - Path to the peers.conf file written by the netbird container +supervise_netbird_config() { + local iface=$1 + local peers_conf=$2 + local peers_dir + peers_dir=$(dirname "$peers_conf") + + [[ -f "$peers_conf" ]] || return 0 + + echo "[wg-client] netbird peers.conf found; watching for changes." + apply_netbird_peers "$iface" "$peers_conf" + + while inotifywait -e close_write -e moved_to "$peers_dir" >/dev/null 2>&1; do + if [[ -f "$peers_conf" ]]; then + echo "[wg-client] peers.conf changed; syncing wg0." >&2 + apply_netbird_peers "$iface" "$peers_conf" + fi + done +} +``` + +- [ ] **Step 4: Call `supervise_netbird_config` from `main()`** + +In `main()`, find the `supervise_dnsmasq "$DNSMASQ_CONF"` call at the bottom. Replace it with a background call to `supervise_netbird_config` so both loops run concurrently: + +```bash + supervise_netbird_config wg0 "/run/netbird/peers.conf" & + + supervise_dnsmasq "$DNSMASQ_CONF" +``` + +(`supervise_dnsmasq` stays in the foreground so the container keeps running; the netbird watcher runs in the background.) + +- [ ] **Step 5: Run the tests to confirm they pass** + +Run: `cd cli && bats test/wg-client/netbird_config.bats` +Expected: PASS. + +- [ ] **Step 6: Run the existing wg-client tests to confirm nothing regressed** + +Run: `cd cli && bats test/wg-client/` +Expected: PASS for all existing wg-client tests. + +- [ ] **Step 7: Commit** ```bash -git add cli/templates/devcontainer/sandcat/compose-netbird.yml -git commit -m "feat(netbird): add netbird service compose definition" +git add \ + cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh \ + cli/test/wg-client/netbird_config.bats +git commit -m "feat(netbird): add supervise_netbird_config watcher to wg-client-init.sh" ``` --- -### Task 2: Add `enable_netbird` compose function and tests +### Task 4: Add `enable_netbird` compose function and tests **Files:** - Modify: `cli/lib/composefile.bash` @@ -101,11 +403,11 @@ git commit -m "feat(netbird): add netbird service compose definition" #!/usr/bin/env bats setup() { - load test_helper - source "$SCT_LIBDIR/composefile.bash" + load test_helper + source "$SCT_LIBDIR/composefile.bash" - COMPOSE_FILE="$BATS_TEST_TMPDIR/compose-all.yml" - cat >"$COMPOSE_FILE" <<'YAML' + COMPOSE_FILE="$BATS_TEST_TMPDIR/compose-all.yml" + cat >"$COMPOSE_FILE" <<'YAML' include: - path: sandcat/compose-proxy.yml services: @@ -115,21 +417,21 @@ YAML } teardown() { - unstub_all + unstub_all } @test "enable_netbird adds netbird include to compose-all.yml" { - enable_netbird "$COMPOSE_FILE" + enable_netbird "$COMPOSE_FILE" - yq -e '.include[] | select(.path == "sandcat/compose-netbird.yml")' "$COMPOSE_FILE" + yq -e '.include[] | select(.path == "sandcat/compose-netbird.yml")' "$COMPOSE_FILE" } @test "enable_netbird is idempotent" { - enable_netbird "$COMPOSE_FILE" - enable_netbird "$COMPOSE_FILE" + enable_netbird "$COMPOSE_FILE" + enable_netbird "$COMPOSE_FILE" - run yq '[.include[] | select(.path == "sandcat/compose-netbird.yml")] | length' "$COMPOSE_FILE" - assert_output "1" + run yq '[.include[] | select(.path == "sandcat/compose-netbird.yml")] | length' "$COMPOSE_FILE" + assert_output "1" } ``` @@ -147,15 +449,15 @@ Add at the end of the file: # Args: # $1 - Path to compose-all.yml enable_netbird() { - require yq - local compose_file=$1 + require yq + local compose_file=$1 - local already_included - already_included=$(yq '[.include[] | select(.path == "sandcat/compose-netbird.yml")] | length' "$compose_file") + local already_included + already_included=$(yq '[.include[] | select(.path == "sandcat/compose-netbird.yml")] | length' "$compose_file") - if [[ "$already_included" -eq 0 ]]; then - yq -i '.include += [{"path": "sandcat/compose-netbird.yml"}]' "$compose_file" - fi + if [[ "$already_included" -eq 0 ]]; then + yq -i '.include += [{"path": "sandcat/compose-netbird.yml"}]' "$compose_file" + fi } ``` @@ -173,7 +475,71 @@ git commit -m "feat(netbird): add enable_netbird compose helper with idempotency --- -### Task 3: Add NetBird pure API helpers and unit tests +### Task 5: Add compose contract tests + +**Files:** +- Create: `cli/test/composefile/netbird_contract.bats` + +- [ ] **Step 1: Write the contract tests** + +```bash +# cli/test/composefile/netbird_contract.bats +#!/usr/bin/env bats +# +# Verifies structural contracts between compose-netbird.yml, compose-proxy.yml, +# and compose-all.yml that security and integration correctness depend on. +# + +setup() { + load test_helper + COMPOSE_NETBIRD="$SCT_TEMPLATEDIR/devcontainer/sandcat/compose-netbird.yml" + COMPOSE_PROXY="$SCT_TEMPLATEDIR/devcontainer/sandcat/compose-proxy.yml" + COMPOSE_ALL="$SCT_TEMPLATEDIR/devcontainer/compose-all.yml" +} + +@test "netbird service has no cap_add entries" { + run yq '.services.netbird.cap_add | length' "$COMPOSE_NETBIRD" + # null (no cap_add key at all) or 0 are both acceptable + [[ "$output" == "null" || "$output" == "0" ]] +} + +@test "netbird service has NB_API_TOKEN in environment" { + yq -e '.services.netbird.environment[] | select(. == "NB_API_TOKEN")' "$COMPOSE_NETBIRD" +} + +@test "netbird service writes to netbird-config volume" { + yq -e '.services.netbird.volumes[] | select(startswith("netbird-config:/run/netbird"))' "$COMPOSE_NETBIRD" +} + +@test "netbird service has a healthcheck" { + yq -e '.services.netbird.healthcheck' "$COMPOSE_NETBIRD" +} + +@test "wg-client mounts netbird-config read-only" { + yq -e '.services."wg-client".volumes[] | select(. == "netbird-config:/run/netbird:ro")' "$COMPOSE_PROXY" +} + +@test "compose-all.yml does not include compose-netbird.yml by default" { + run yq '.include[] | select(.path == "sandcat/compose-netbird.yml")' "$COMPOSE_ALL" + assert_output "" +} +``` + +- [ ] **Step 2: Run the contract tests** + +Run: `cd cli && bats test/composefile/netbird_contract.bats` +Expected: PASS for all six tests. + +- [ ] **Step 3: Commit** + +```bash +git add cli/test/composefile/netbird_contract.bats +git commit -m "test(netbird): add compose contract tests enforcing no-caps and volume separation" +``` + +--- + +### Task 6: Add NetBird management API helpers and unit tests **Files:** - Create: `cli/lib/netbird.bash` @@ -187,7 +553,7 @@ git commit -m "feat(netbird): add enable_netbird compose helper with idempotency #!/bin/bash bats_require_minimum_version 1.5.0 if shopt -s compat32 2>/dev/null; then - export BASH_COMPAT=3.2 + export BASH_COMPAT=3.2 fi set -uo pipefail export SHELLOPTS @@ -210,65 +576,51 @@ export SCT_ROOT SCT_LIBDIR="$SCT_ROOT/lib" #!/usr/bin/env bats setup() { - load test_helper - source "$SCT_LIBDIR/netbird.bash" + load test_helper + source "$SCT_LIBDIR/netbird.bash" - export NB_MANAGEMENT_URL="https://api.netbird.io" - export NB_API_TOKEN="test-token" + export NB_MANAGEMENT_URL="https://api.netbird.io" + export NB_API_TOKEN="test-token" } teardown() { - unstub_all -} - -@test "netbird_api calls curl with auth header and endpoint" { - stub curl \ - "-sf -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '{\"peers\":[]}\n'" - run netbird_api "GET" "/api/peers" - assert_success + unstub_all } @test "netbird_api fails when NB_API_TOKEN is unset" { - unset NB_API_TOKEN - run netbird_api "GET" "/api/peers" - assert_failure - assert_output --partial "NB_API_TOKEN" + unset NB_API_TOKEN + run netbird_api "GET" "/api/peers" + assert_failure + assert_output --partial "NB_API_TOKEN" } -@test "netbird_status returns structured peer list" { - stub curl \ - "-sf -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[{\"id\":\"peer1\",\"name\":\"test\",\"connected\":true}]'" - run netbird_status - assert_success - assert_output --partial "peer1" +@test "netbird_status calls GET /api/peers" { + stub curl \ + "-sf -X GET -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[{\"id\":\"peer1\",\"connected\":true}]'" + run netbird_status + assert_success + assert_output --partial "peer1" } -@test "netbird_route_add calls routes API with network and peer" { - stub curl \ - "-sf -X POST -H 'Authorization: Token test-token' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"peer1\",\"enabled\":true}' https://api.netbird.io/api/routes : echo '{\"id\":\"route1\"}'" - run netbird_route_add "10.8.0.0/24" "peer1" - assert_success +@test "netbird_route_add calls POST /api/routes with network and peer" { + stub curl \ + "-sf -X POST -H 'Authorization: Token test-token' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"peer1\",\"enabled\":true}' https://api.netbird.io/api/routes : echo '{\"id\":\"route1\"}'" + run netbird_route_add "10.8.0.0/24" "peer1" + assert_success } -@test "netbird_route_remove calls DELETE on routes API" { - stub curl \ - "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/routes/route1 : :" - run netbird_route_remove "route1" - assert_success +@test "netbird_route_remove calls DELETE /api/routes/:id" { + stub curl \ + "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/routes/route1 : :" + run netbird_route_remove "route1" + assert_success } -@test "netbird_up calls peers API to enable peer" { - stub curl \ - "-sf -X PUT -H 'Authorization: Token test-token' -H 'Content-Type: application/json' -d '{\"login_expiration_enabled\":false}' https://api.netbird.io/api/peers/peer1 : echo '{\"id\":\"peer1\"}'" - run netbird_up "peer1" - assert_success -} - -@test "netbird_down calls peers API to disable peer" { - stub curl \ - "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers/peer1 : :" - run netbird_down "peer1" - assert_success +@test "netbird_peer_remove calls DELETE /api/peers/:id" { + stub curl \ + "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers/peer1 : :" + run netbird_peer_remove "peer1" + assert_success } ``` @@ -284,70 +636,63 @@ Expected: FAIL — `cli/lib/netbird.bash` does not exist. #!/usr/bin/env bash # Calls the NetBird management REST API. -# Requires NB_MANAGEMENT_URL and NB_API_TOKEN to be set in the environment. +# Requires NB_API_TOKEN. NB_MANAGEMENT_URL defaults to https://api.netbird.io. # Args: -# $1 - HTTP method (GET, POST, PUT, DELETE) +# $1 - HTTP method (GET, POST, DELETE) # $2 - API path (e.g. /api/peers) -# $3 - Optional JSON body string +# $3 - Optional JSON body netbird_api() { - local method=$1 - local path=$2 - local body=${3:-} + local method=$1 + local path=$2 + local body=${3:-} - if [[ -z "${NB_API_TOKEN:-}" ]]; then - echo "NB_API_TOKEN is not set" >&2 - return 1 - fi + if [[ -z "${NB_API_TOKEN:-}" ]]; then + echo "NB_API_TOKEN is not set" >&2 + return 1 + fi - local url="${NB_MANAGEMENT_URL:-https://api.netbird.io}${path}" - local args=(-sf -X "$method" - -H "Authorization: Token $NB_API_TOKEN" - -H "Content-Type: application/json") + local url="${NB_MANAGEMENT_URL:-https://api.netbird.io}${path}" + local args=(-sf -X "$method" + -H "Authorization: Token $NB_API_TOKEN" + -H "Content-Type: application/json") - if [[ -n "$body" ]]; then - args+=(-d "$body") - fi + [[ -n "$body" ]] && args+=(-d "$body") - curl "${args[@]}" "$url" + curl "${args[@]}" "$url" } -# Returns the current list of peers from the NetBird management server. +# Returns the current peer list from the NetBird management server. netbird_status() { - netbird_api "GET" "/api/peers" -} - -# Enables a peer by ID on the NetBird management server. -# Args: -# $1 - Peer ID -netbird_up() { - local peer_id=$1 - netbird_api "PUT" "/api/peers/$peer_id" '{"login_expiration_enabled":false}' -} - -# Removes a peer by ID from the NetBird management server. -# Args: -# $1 - Peer ID -netbird_down() { - local peer_id=$1 - netbird_api "DELETE" "/api/peers/$peer_id" + netbird_api "GET" "/api/peers" } -# Adds a network route to the NetBird management server. +# Adds a network route served by a peer. # Args: # $1 - Network CIDR (e.g. 10.8.0.0/24) # $2 - Peer ID that serves the route netbird_route_add() { - local network=$1 - local peer_id=$2 - netbird_api "POST" "/api/routes" "{\"network\":\"$network\",\"peer\":\"$peer_id\",\"enabled\":true}" + local network=$1 + local peer_id=$2 + netbird_api "POST" "/api/routes" \ + "{\"network\":\"$network\",\"peer\":\"$peer_id\",\"enabled\":true}" } -# Removes a network route by ID from the NetBird management server. +# Removes a network route by ID. # Args: -# $1 - Route ID +# $1 - Route ID (returned by netbird_route_add) netbird_route_remove() { - local route_id=$1 - netbird_api "DELETE" "/api/routes/$route_id" + local route_id=$1 + netbird_api "DELETE" "/api/routes/$route_id" +} + +# Removes a peer from the NetBird management server. +# Causes netbird-sync.sh to write an updated peers.conf that omits this peer, +# which wg-client then applies via wg syncconf — removing the route to that peer. +# Args: +# $1 - Peer ID +netbird_peer_remove() { + local peer_id=$1 + netbird_api "DELETE" "/api/peers/$peer_id" } ``` @@ -360,12 +705,12 @@ Expected: PASS. ```bash git add cli/lib/netbird.bash cli/test/netbird/test_helper.bash cli/test/netbird/netbird_api.bats -git commit -m "feat(netbird): add pure netbird API helpers with unit tests" +git commit -m "feat(netbird): add netbird management API helpers with unit tests" ``` --- -### Task 4: Add `sandcat netbird` subcommand dispatcher and integration tests +### Task 7: Add `sandcat netbird` subcommand dispatcher and integration tests **Files:** - Create: `cli/libexec/netbird/netbird` @@ -378,80 +723,67 @@ git commit -m "feat(netbird): add pure netbird API helpers with unit tests" #!/usr/bin/env bats setup() { - load test_helper + load test_helper - NETBIRD_CMD="$SCT_LIBEXECDIR/netbird/netbird" - export NB_MANAGEMENT_URL="https://api.netbird.io" - export NB_API_TOKEN="test-token" + NETBIRD_CMD="$SCT_LIBEXECDIR/netbird/netbird" + export NB_MANAGEMENT_URL="https://api.netbird.io" + export NB_API_TOKEN="test-token" } teardown() { - unstub_all + unstub_all } -@test "netbird status calls netbird_status" { - stub curl \ - "-sf -X GET -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[]'" - run bash "$NETBIRD_CMD" status - assert_success +@test "netbird status calls GET /api/peers" { + stub curl \ + "-sf -X GET -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[]'" + run bash "$NETBIRD_CMD" status + assert_success } -@test "netbird up requires peer-id argument" { - run bash "$NETBIRD_CMD" up - assert_failure - assert_output --partial "peer-id" +@test "netbird peer remove requires --peer-id" { + run bash "$NETBIRD_CMD" peer remove + assert_failure + assert_output --partial "peer-id" } -@test "netbird up calls netbird_up with peer-id" { - stub curl \ - "-sf -X PUT -H 'Authorization: Token test-token' -H 'Content-Type: application/json' -d '{\"login_expiration_enabled\":false}' https://api.netbird.io/api/peers/abc123 : echo '{\"id\":\"abc123\"}'" - run bash "$NETBIRD_CMD" up --peer-id abc123 - assert_success -} - -@test "netbird down requires peer-id argument" { - run bash "$NETBIRD_CMD" down - assert_failure - assert_output --partial "peer-id" -} - -@test "netbird down calls netbird_down with peer-id" { - stub curl \ - "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers/abc123 : :" - run bash "$NETBIRD_CMD" down --peer-id abc123 - assert_success +@test "netbird peer remove calls netbird_peer_remove" { + stub curl \ + "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers/abc123 : :" + run bash "$NETBIRD_CMD" peer remove --peer-id abc123 + assert_success } @test "netbird route add requires --network and --peer-id" { - run bash "$NETBIRD_CMD" route add --network 10.8.0.0/24 - assert_failure - assert_output --partial "peer-id" + run bash "$NETBIRD_CMD" route add --network 10.8.0.0/24 + assert_failure + assert_output --partial "peer-id" } @test "netbird route add calls netbird_route_add" { - stub curl \ - "-sf -X POST -H 'Authorization: Token test-token' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"abc123\",\"enabled\":true}' https://api.netbird.io/api/routes : echo '{\"id\":\"route1\"}'" - run bash "$NETBIRD_CMD" route add --network 10.8.0.0/24 --peer-id abc123 - assert_success + stub curl \ + "-sf -X POST -H 'Authorization: Token test-token' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"abc123\",\"enabled\":true}' https://api.netbird.io/api/routes : echo '{\"id\":\"route1\"}'" + run bash "$NETBIRD_CMD" route add --network 10.8.0.0/24 --peer-id abc123 + assert_success } @test "netbird route remove requires --route-id" { - run bash "$NETBIRD_CMD" route remove - assert_failure - assert_output --partial "route-id" + run bash "$NETBIRD_CMD" route remove + assert_failure + assert_output --partial "route-id" } @test "netbird route remove calls netbird_route_remove" { - stub curl \ - "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/routes/route1 : :" - run bash "$NETBIRD_CMD" route remove --route-id route1 - assert_success + stub curl \ + "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/routes/route1 : :" + run bash "$NETBIRD_CMD" route remove --route-id route1 + assert_success } @test "netbird with unknown subcommand prints usage and fails" { - run bash "$NETBIRD_CMD" bogus - assert_failure - assert_output --partial "Usage" + run bash "$NETBIRD_CMD" bogus + assert_failure + assert_output --partial "Usage" } ``` @@ -473,110 +805,102 @@ source "$SCT_LIBDIR/logging.bash" source "$SCT_LIBDIR/netbird.bash" usage() { - cat <<'EOF' + cat <<'EOF' Usage: sandcat netbird [options] Subcommands: - status List connected NetBird peers - up --peer-id Enable a peer on the NetBird management server - down --peer-id Remove a peer from the NetBird management server - route add --network --peer-id Add a route - route remove --route-id Remove a route by ID + status List peers from the NetBird management server + peer remove --peer-id Remove a peer (triggers wg syncconf in wg-client) + route add --network --peer-id Add a network route + route remove --route-id Remove a route by ID EOF } cmd_status() { - netbird_status -} - -cmd_up() { - local peer_id="" - while [[ $# -gt 0 ]]; do - case $1 in - --peer-id) peer_id="$2"; shift 2 ;; - *) echo "Unknown option: $1" | error; return 1 ;; - esac - done - if [[ -z "$peer_id" ]]; then - echo "Missing required option: --peer-id" | error - return 1 - fi - netbird_up "$peer_id" -} - -cmd_down() { - local peer_id="" - while [[ $# -gt 0 ]]; do - case $1 in - --peer-id) peer_id="$2"; shift 2 ;; - *) echo "Unknown option: $1" | error; return 1 ;; - esac - done - if [[ -z "$peer_id" ]]; then - echo "Missing required option: --peer-id" | error - return 1 - fi - netbird_down "$peer_id" + netbird_status +} + +cmd_peer() { + local subcmd="${1:-}" + shift || true + case "$subcmd" in + remove) + local peer_id="" + while [[ $# -gt 0 ]]; do + case $1 in + --peer-id) peer_id="$2"; shift 2 ;; + *) echo "Unknown option: $1" | error; return 1 ;; + esac + done + if [[ -z "$peer_id" ]]; then + echo "Missing required option: --peer-id" | error; return 1 + fi + netbird_peer_remove "$peer_id" + ;; + *) + echo "Unknown peer subcommand: $subcmd" | error + usage; return 1 + ;; + esac } cmd_route() { - local subcmd="${1:-}" - shift || true - case "$subcmd" in - add) - local network="" peer_id="" - while [[ $# -gt 0 ]]; do - case $1 in - --network) network="$2"; shift 2 ;; - --peer-id) peer_id="$2"; shift 2 ;; - *) echo "Unknown option: $1" | error; return 1 ;; - esac - done - if [[ -z "$network" ]]; then - echo "Missing required option: --network" | error; return 1 - fi - if [[ -z "$peer_id" ]]; then - echo "Missing required option: --peer-id" | error; return 1 - fi - netbird_route_add "$network" "$peer_id" - ;; - remove) - local route_id="" - while [[ $# -gt 0 ]]; do - case $1 in - --route-id) route_id="$2"; shift 2 ;; - *) echo "Unknown option: $1" | error; return 1 ;; - esac - done - if [[ -z "$route_id" ]]; then - echo "Missing required option: --route-id" | error; return 1 - fi - netbird_route_remove "$route_id" - ;; - *) - echo "Unknown route subcommand: $subcmd" | error - usage; return 1 - ;; - esac + local subcmd="${1:-}" + shift || true + case "$subcmd" in + add) + local network="" peer_id="" + while [[ $# -gt 0 ]]; do + case $1 in + --network) network="$2"; shift 2 ;; + --peer-id) peer_id="$2"; shift 2 ;; + *) echo "Unknown option: $1" | error; return 1 ;; + esac + done + if [[ -z "$network" ]]; then + echo "Missing required option: --network" | error; return 1 + fi + if [[ -z "$peer_id" ]]; then + echo "Missing required option: --peer-id" | error; return 1 + fi + netbird_route_add "$network" "$peer_id" + ;; + remove) + local route_id="" + while [[ $# -gt 0 ]]; do + case $1 in + --route-id) route_id="$2"; shift 2 ;; + *) echo "Unknown option: $1" | error; return 1 ;; + esac + done + if [[ -z "$route_id" ]]; then + echo "Missing required option: --route-id" | error; return 1 + fi + netbird_route_remove "$route_id" + ;; + *) + echo "Unknown route subcommand: $subcmd" | error + usage; return 1 + ;; + esac } main() { - local subcmd="${1:-}" - shift || true - case "$subcmd" in - status) cmd_status "$@" ;; - up) cmd_up "$@" ;; - down) cmd_down "$@" ;; - route) cmd_route "$@" ;; - *) - usage - return 1 - ;; - esac + local subcmd="${1:-}" + shift || true + case "$subcmd" in + status) cmd_status "$@" ;; + peer) cmd_peer "$@" ;; + route) cmd_route "$@" ;; + *) + usage + return 1 + ;; + esac } if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then - main "$@" + main "$@" fi ``` @@ -598,41 +922,43 @@ git commit -m "feat(netbird): add sandcat netbird subcommand dispatcher" --- -### Task 5: Wire `--netbird` flag into `init devcontainer` and `init` +### Task 8: Wire `--netbird` flag into `init devcontainer` and `init` **Files:** - Modify: `cli/libexec/init/devcontainer` - Modify: `cli/libexec/init/init` - Modify: `cli/test/init/init.bats` -- [ ] **Step 1: Write the failing BATS tests for the new flags** +- [ ] **Step 1: Write the failing BATS tests** Add to `cli/test/init/init.bats`: ```bash @test "init --netbird passes netbird flag to devcontainer" { - stub devcontainer \ - "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird : :" - run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --netbird - assert_success + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird : :" + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --netbird + assert_success } @test "init --netbird seeds netbird_enrollment_key in user settings" { - stub devcontainer "*: :" - run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --netbird - run yq '.netbird_enrollment_key' "$SCT_HOME_DIR/settings.json" - assert_output '""' + stub devcontainer "*: :" + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --netbird + run yq '.netbird_enrollment_key' "$SCT_HOME_DIR/settings.json" + assert_output '""' } ``` -- [ ] **Step 2: Run the targeted tests to confirm they fail** +- [ ] **Step 2: Run targeted tests to confirm they fail** Run: `cd cli && bats test/init/init.bats --filter "netbird"` -Expected: FAIL — `--netbird` is not recognized by `init`. +Expected: FAIL — `--netbird` is not recognized. - [ ] **Step 3: Add `--netbird` to `cli/libexec/init/devcontainer`** -In the `while [[ $# -gt 0 ]]` argument-parsing loop, add: +Add `local netbird="false"` at the top of the function alongside the other locals. + +In the argument-parsing loop add: ```bash --netbird) @@ -641,7 +967,7 @@ In the `while [[ $# -gt 0 ]]` argument-parsing loop, add: ;; ``` -Add `local netbird="false"` at the top of the function alongside the other locals. After the `apply_secret_provider` call, add: +After the `apply_secret_provider` call, add: ```bash if [[ "$netbird" == "true" ]]; then @@ -649,20 +975,6 @@ if [[ "$netbird" == "true" ]]; then fi ``` -Also add `--netbird` to the case branch that checks for options requiring values: - -```bash ---settings-file|--project-path|--agent|--ide|--name|--stacks|--proxy|--secret-provider) -``` - -becomes: - -```bash ---settings-file|--project-path|--agent|--ide|--name|--stacks|--proxy|--secret-provider) -``` - -`--netbird` takes no value so it stays outside that branch. - - [ ] **Step 4: Add `--netbird` to `cli/libexec/init/init`** Add `local netbird="false"` near the other locals. In the argument-parsing loop add: @@ -674,7 +986,7 @@ Add `local netbird="false"` near the other locals. In the argument-parsing loop ;; ``` -In the `add_secret_provider_tokens_to_user_settings` call site, add alongside it: +Where provider tokens are seeded into user settings, add alongside: ```bash if [[ "$netbird" == "true" ]]; then @@ -682,7 +994,7 @@ if [[ "$netbird" == "true" ]]; then fi ``` -In the `devcontainer` invocation, append `${netbird:+--netbird}`: +In the `devcontainer` invocation, append: ```bash devcontainer \ @@ -706,105 +1018,45 @@ git commit -m "feat(init): add --netbird flag and enrollment key seeding" --- -### Task 6: Add compose contract tests for the netbird include - -**Files:** -- Create: `cli/test/composefile/netbird_contract.bats` - -- [ ] **Step 1: Write the contract tests** - -```bash -# cli/test/composefile/netbird_contract.bats -#!/usr/bin/env bats -# -# Verifies that compose-netbird.yml and compose-all.yml meet structural contracts -# that other components rely on. -# - -setup() { - load test_helper - COMPOSE_NETBIRD="$SCT_TEMPLATEDIR/devcontainer/sandcat/compose-netbird.yml" - COMPOSE_ALL="$SCT_TEMPLATEDIR/devcontainer/compose-all.yml" -} - -@test "compose-netbird.yml defines netbird service" { - yq -e '.services.netbird' "$COMPOSE_NETBIRD" -} - -@test "compose-netbird.yml netbird service has NET_ADMIN capability" { - yq -e '.services.netbird.cap_add[] | select(. == "NET_ADMIN")' "$COMPOSE_NETBIRD" -} - -@test "compose-netbird.yml netbird service has NB_SETUP_KEY environment entry" { - yq -e '.services.netbird.environment[] | select(. == "NB_SETUP_KEY")' "$COMPOSE_NETBIRD" -} - -@test "compose-netbird.yml netbird service has healthcheck" { - yq -e '.services.netbird.healthcheck' "$COMPOSE_NETBIRD" -} - -@test "compose-netbird.yml netbird service uses netbird-config named volume" { - yq -e '.volumes.netbird-config' "$COMPOSE_NETBIRD" -} - -@test "compose-all.yml does not include compose-netbird.yml by default" { - run yq '.include[] | select(.path == "sandcat/compose-netbird.yml")' "$COMPOSE_ALL" - assert_output "" -} -``` - -- [ ] **Step 2: Run the contract tests** - -Run: `cd cli && bats test/composefile/netbird_contract.bats` -Expected: PASS. (The last test verifies the template ships without NetBird by default.) - -- [ ] **Step 3: Commit** - -```bash -git add cli/test/composefile/netbird_contract.bats -git commit -m "test(netbird): add compose contract tests for netbird service" -``` - ---- - -### Task 7: Update CLI docs +### Task 9: Update CLI docs **Files:** - Modify: `cli/README.md` -- [ ] **Step 1: Add NetBird section to README** - -Locate the `--secret-provider` option description in `cli/README.md` and add immediately after it: +- [ ] **Step 1: Add `--netbird` option description after `--secret-provider`** ```markdown - `--netbird` - Enable dynamic WireGuard control via NetBird. Adds a companion - `netbird` container to the proxy stack and seeds `netbird_enrollment_key` in - your user settings (`~/.config/sandcat/settings.json`). Fill in the key before - starting the devcontainer. + `netbird` sync container that polls the NetBird management API and updates + `wg-client`'s peer table without restarting any container. + Seeds `netbird_enrollment_key` in `~/.config/sandcat/settings.json`. ``` -Find the init usage examples and add: +- [ ] **Step 2: Add init usage example** ```bash # With NetBird dynamic WireGuard sandcat init --agent claude --ide vscode --netbird --name myproject ``` -Add a new section `## Dynamic networking (NetBird)`: +- [ ] **Step 3: Add `## Dynamic networking (NetBird)` section** ```markdown ## Dynamic networking (NetBird) -When initialized with `--netbird`, sandcat adds a companion [NetBird](https://netbird.io) -container that connects to a NetBird management server. This makes the WireGuard -network layer controllable at runtime — routes and peer access can be added or -removed without restarting any container. +When initialized with `--netbird`, sandcat adds a companion NetBird sync container. +It polls the [NetBird](https://netbird.io) management API, writes a WireGuard peer +config file, and `wg-client` applies it via `wg syncconf` — no container restarts. + +The `netbird` container has no kernel capabilities and does not manage any network +interfaces. `wg-client` remains the sole owner of `NET_ADMIN` and `wg0`. ### Setup -1. Create a NetBird account at or self-host the management server. -2. Generate a setup key in the NetBird dashboard under **Setup Keys**. -3. Add the key to your sandcat user settings: +1. Create a NetBird account at or self-host the server. +2. Generate a setup key (**Setup Keys** in the NetBird dashboard). +3. Generate an API token (**API Keys** in the NetBird dashboard). +4. Add both to your sandcat user settings: ```json { @@ -812,39 +1064,36 @@ removed without restarting any container. } ``` -4. Set the NetBird API token in your shell before using `sandcat netbird` commands: +5. Export the API token in your shell before using `sandcat netbird` commands: ```bash -export NB_API_TOKEN="your-management-api-token" -export NB_MANAGEMENT_URL="https://api.netbird.io" # or your self-hosted URL +export NB_API_TOKEN="your-api-token" +export NB_MANAGEMENT_URL="https://api.netbird.io" # or self-hosted URL ``` ### Runtime control ```bash -# List connected peers +# List current peers sandcat netbird status -# Enable a peer -sandcat netbird up --peer-id - -# Remove a peer -sandcat netbird down --peer-id +# Remove a peer (wg-client drops the route within one poll interval) +sandcat netbird peer remove --peer-id # Add a network route served by a peer sandcat netbird route add --network 10.8.0.0/24 --peer-id -# Remove a route by ID (ID returned by route add) +# Remove a route sandcat netbird route remove --route-id ``` ``` -- [ ] **Step 2: Verify no old NetBird references remain undefined** +- [ ] **Step 4: Verify** Run: `rg --line-number "netbird" cli/README.md` -Expected: Only the newly added lines appear. +Expected: Only the newly added lines. -- [ ] **Step 3: Commit** +- [ ] **Step 5: Commit** ```bash git add cli/README.md @@ -853,61 +1102,77 @@ git commit -m "docs(cli): document --netbird flag and sandcat netbird subcommand --- -### Task 8: Final verification +### Task 10: Final verification -**Files:** None modified (verification only). +**Files:** None modified. -- [ ] **Step 1: Run full composefile test suite** +- [ ] **Step 1: Run full composefile suite** Run: `cd cli && bats test/composefile/` -Expected: PASS (all composefile tests including new netbird tests). +Expected: PASS. -- [ ] **Step 2: Run full netbird test suite** +- [ ] **Step 2: Run full netbird suite** Run: `cd cli && bats test/netbird/` Expected: PASS. -- [ ] **Step 3: Run full init test suite** +- [ ] **Step 3: Run full wg-client suite** + +Run: `cd cli && bats test/wg-client/` +Expected: PASS including `netbird_config.bats`. + +- [ ] **Step 4: Run full init suite** Run: `cd cli && bats test/init/` Expected: PASS. -- [ ] **Step 4: Confirm git status is clean** +- [ ] **Step 5: Confirm clean git state** Run: `git status --short` Expected: No untracked or modified files. -- [ ] **Step 5: Confirm all new files are committed** +- [ ] **Step 6: Confirm all commits are present** -Run: `git log --oneline -8` -Expected: All tasks committed individually with descriptive messages. +Run: `git log --oneline -12` +Expected: One commit per task with descriptive messages. --- ## Plan Self-Review -### 1. Spec coverage check +### 1. Spec coverage -| Requirement | Covered by | +| Requirement | Task | |---|---| -| NetBird companion container definition | Task 1 | -| `enable_netbird` compose helper | Task 2 | -| Pure API helpers (status, up, down, route add/remove) | Task 3 | -| `sandcat netbird` subcommand dispatcher | Task 4 | -| `--netbird` flag in `init` and `init devcontainer` | Task 5 | -| `netbird_enrollment_key` seeded in user settings | Task 5 | -| Compose contract tests (no default include) | Task 6 | -| README docs | Task 7 | - -No spec gaps. +| `netbird` container (no caps, config-sync only) | 1 | +| `Dockerfile.netbird` + `netbird-sync.sh` | 1 | +| `wg-client` mounts `netbird-config` read-only | 2 | +| `inotify-tools` in `wg-client` image | 2 | +| `supervise_netbird_config` + `apply_netbird_peers` | 3 | +| `enable_netbird` compose helper | 4 | +| Contract tests (no caps, read-only mount, no default include) | 5 | +| Management API helpers | 6 | +| `sandcat netbird` dispatcher | 7 | +| `--netbird` flag in `init` and `init devcontainer` | 8 | +| `netbird_enrollment_key` seeded in user settings | 8 | +| README docs | 9 | + +No gaps. ### 2. Placeholder scan -No TBD/TODO in any step. Every code step contains concrete implementations. Every run step contains expected output. +No TBD/TODO in any step. All code blocks are complete. ### 3. Type/name consistency -- `NB_API_TOKEN` and `NB_MANAGEMENT_URL` used consistently across `netbird.bash`, the dispatcher, and README. -- `NB_SETUP_KEY` (container enrollment env var) is distinct from `NB_API_TOKEN` (management API token) — both are explicitly named throughout. -- `enable_netbird` called consistently from `composefile.bash` and `init devcontainer`. -- `netbird_enrollment_key` (settings JSON key) used consistently in `init` and README. +- `NB_API_TOKEN` — used in `netbird-sync.sh`, `netbird.bash`, dispatcher, contract test, README. +- `NB_MANAGEMENT_URL` — same propagation path. +- `NB_SETUP_KEY` is absent from the compose definition by design: enrollment is handled by the user populating `netbird_enrollment_key` in settings; the sync container only uses the API token. +- `netbird-config` volume name — used consistently in `compose-netbird.yml`, `compose-proxy.yml`, and contract tests. +- `/run/netbird/peers.conf` — the agreed path between `netbird-sync.sh` (writer) and `wg-client-init.sh` (reader), tested in `netbird_contract.bats` and `netbird_config.bats`. + +### 4. Security properties preserved + +- `wg-client` is the only container with `NET_ADMIN`. The contract test asserts no `cap_add` on the `netbird` service. +- `netbird-config` volume is writable only by `netbird`, read-only for `wg-client`. The contract test asserts the `:ro` mount. +- `netbird` has no access to `wg0`, iptables, or the mitmproxy tunnel — it only writes a text file. From 0c3b7407f7d531590e2ba9114ff0c5ed83d44139 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 15 Jun 2026 19:02:18 +0000 Subject: [PATCH 003/138] feat(netbird): install netbird binary in wg-client image --- .../devcontainer/sandcat/Dockerfile.wg-client | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/cli/templates/devcontainer/sandcat/Dockerfile.wg-client b/cli/templates/devcontainer/sandcat/Dockerfile.wg-client index 53968be6..5f9139f0 100644 --- a/cli/templates/devcontainer/sandcat/Dockerfile.wg-client +++ b/cli/templates/devcontainer/sandcat/Dockerfile.wg-client @@ -6,13 +6,25 @@ FROM debian:trixie-slim # iptables - firewall rules used as a kill switch (blocks traffic if tunnel drops) # jq - parse mitmproxy's wireguard.conf JSON to extract key pairs # openresolv - `resolvconf` command to configure DNS through the tunnel -# dnsmasq - local DNS forwarder providing split-DNS: Docker compose -# network names go to 127.0.0.11; everything else to upstream +# dnsmasq - local DNS forwarder providing split-DNS +# ca-certificates + curl - required for trusting the mitmproxy CA cert at runtime +# and downloading the netbird binary at build time +ARG NETBIRD_VERSION=0.28.9 RUN apt-get update \ && apt-get install -y --no-install-recommends \ wireguard-tools iproute2 iptables jq openresolv dnsmasq \ + ca-certificates curl \ && rm -rf /var/lib/apt/lists/* +# Install the NetBird client daemon. The daemon manages wt0 (the NetBird +# overlay mesh) inside this container, which already holds NET_ADMIN. +# wg0 (the mitmproxy inspection tunnel) is not touched by the daemon. +RUN ARCH=$(dpkg --print-architecture) \ + && curl -sSLf \ + "https://github.com/netbirdio/netbird/releases/download/v${NETBIRD_VERSION}/netbird_${NETBIRD_VERSION}_linux_${ARCH}.tar.gz" \ + | tar xz -C /usr/local/bin netbird \ + && chmod +x /usr/local/bin/netbird + COPY scripts/wg-client-init.sh /usr/local/bin/wg-client-init.sh COPY scripts/dnsmasq-ready /usr/local/bin/dnsmasq-ready RUN chmod +x /usr/local/bin/wg-client-init.sh /usr/local/bin/dnsmasq-ready From 2ef59bc6cd588231d528a02167d4a5bd201bb56f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 16 Jun 2026 04:29:55 +0000 Subject: [PATCH 004/138] feat(netbird): add netbird daemon startup and supervision to wg-client-init.sh --- .../sandcat/scripts/wg-client-init.sh | 77 +++++++++++++++++++ cli/test/wg-client/netbird_daemon.bats | 52 +++++++++++++ 2 files changed, 129 insertions(+) create mode 100644 cli/test/wg-client/netbird_daemon.bats diff --git a/cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh b/cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh index 820b6036..236a465a 100644 --- a/cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh +++ b/cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh @@ -307,12 +307,89 @@ main() { } >> /etc/hosts fi + # ── NetBird overlay mesh (optional) ───────────────────────────────────────── + # Enroll and start the NetBird daemon if NB_SETUP_KEY is set. wt0 is created + # by the daemon in this network namespace alongside wg0. fwmark 51821 ensures + # wt0 traffic routes through wg0 → mitmproxy (not eth0 directly). + trust_mitmproxy_ca "/mitmproxy-config/mitmproxy-ca-cert.pem" + start_netbird "wt0" + set_netbird_fwmark "wt0" + supervise_netbird_daemon "wt0" & + + # Signal readiness to containers waiting on the healthcheck. touch /tmp/wg-ready supervise_dnsmasq "$DNSMASQ_CONF" } +# Trust the mitmproxy CA cert so the netbird daemon can verify TLS connections +# to the NetBird management and signal servers. Those connections transit wg0 → +# mitmproxy just like all other outbound traffic; without this step the daemon +# would reject the MITM certificate and fail to enroll. +# Args: +# $1 - Path to the mitmproxy CA cert (from the mitmproxy-config volume) +# $2 - System CA directory (default: /usr/local/share/ca-certificates) +trust_mitmproxy_ca() { + local ca_cert="$1" + local ca_dir="${2:-/usr/local/share/ca-certificates}" + [[ -f "$ca_cert" ]] || return 0 + cp "$ca_cert" "$ca_dir/mitmproxy.crt" + update-ca-certificates --fresh >/dev/null 2>&1 +} + +# Enroll this container as a NetBird peer and start the daemon in the background. +# Does nothing if NB_SETUP_KEY is unset (NetBird disabled for this environment). +# After the daemon starts it waits until the overlay interface appears. +# Args: +# $1 - WireGuard interface name for the NetBird overlay (default: wt0) +start_netbird() { + local iface="${1:-wt0}" + [[ -n "${NB_SETUP_KEY:-}" ]] || return 0 + + echo "[wg-client] Starting NetBird daemon on ${iface}." >&2 + netbird up \ + --setup-key "${NB_SETUP_KEY}" \ + --management-url "${NB_MANAGEMENT_URL:-https://api.netbird.io}" \ + --interface-name "${iface}" \ + & + + wait_until 30 1 \ + "[wg-client] Timed out waiting for NetBird to bring up ${iface}" \ + ip link show "${iface}" >/dev/null 2>&1 +} + +# Override the WireGuard fwmark on the NetBird interface so its encapsulation +# packets (fwmark 51821) are NOT exempt from the wg0 policy routing rule +# (`not fwmark 51820 → table 51820 → wg0`). This ensures NetBird peer traffic +# transits wg0 → mitmproxy, preserving the inspection guarantee. +# Must be called after start_netbird has brought the interface up. +# Args: +# $1 - NetBird WireGuard interface name (default: wt0) +set_netbird_fwmark() { + local iface="${1:-wt0}" + ip link show "${iface}" >/dev/null 2>&1 || return 0 + wg set "${iface}" fwmark 51821 +} + +# Supervise the NetBird daemon: poll every 10 s and restart if unresponsive. +# Returns immediately (no-op) if NB_SETUP_KEY is unset. +# Args: +# $1 - NetBird WireGuard interface name (default: wt0) +supervise_netbird_daemon() { + local iface="${1:-wt0}" + [[ -n "${NB_SETUP_KEY:-}" ]] || return 0 + + while true; do + sleep 10 + if ! netbird status >/dev/null 2>&1; then + echo "[wg-client] NetBird daemon not responding; restarting." >&2 + start_netbird "${iface}" || true + set_netbird_fwmark "${iface}" || true + fi + done +} + # Keep dnsmasq alive in-place. Sibling containers share this container's # network namespace via `network_mode: service:wg-client`; that binding is # resolved at sibling-create time, so a wg-client container restart would diff --git a/cli/test/wg-client/netbird_daemon.bats b/cli/test/wg-client/netbird_daemon.bats new file mode 100644 index 00000000..453d183a --- /dev/null +++ b/cli/test/wg-client/netbird_daemon.bats @@ -0,0 +1,52 @@ +#!/usr/bin/env bats + +setup() { + load test_helper + NETBIRD_IFACE="wt0" +} + +teardown() { + unstub_all +} + +@test "trust_mitmproxy_ca copies cert to ca-dir and runs update-ca-certificates" { + local ca_src="$BATS_TEST_TMPDIR/mitmproxy-ca-cert.pem" + local ca_dir="$BATS_TEST_TMPDIR/ca-certs" + mkdir -p "$ca_dir" + echo "FAKE CERT" > "$ca_src" + + stub update-ca-certificates "--fresh : :" + trust_mitmproxy_ca "$ca_src" "$ca_dir" + + test -f "$ca_dir/mitmproxy.crt" +} + +@test "trust_mitmproxy_ca is a no-op when CA cert is absent" { + run trust_mitmproxy_ca "$BATS_TEST_TMPDIR/missing.pem" "$BATS_TEST_TMPDIR/ca-dir" + assert_success +} + +@test "start_netbird is a no-op when NB_SETUP_KEY is unset" { + unset NB_SETUP_KEY + run start_netbird "$NETBIRD_IFACE" + assert_success + refute_output --partial "netbird" +} + +@test "set_netbird_fwmark sets fwmark 51821 on the NetBird interface" { + stub ip "link show wt0 : :" + stub wg "set wt0 fwmark 51821 : :" + set_netbird_fwmark "wt0" +} + +@test "set_netbird_fwmark is a no-op when interface does not exist" { + stub ip "link show wt0 : return 1" + run set_netbird_fwmark "wt0" + assert_success +} + +@test "supervise_netbird_daemon returns immediately when NB_SETUP_KEY is unset" { + unset NB_SETUP_KEY + run supervise_netbird_daemon "wt0" + assert_success +} From 1386e87b07875e749cff02755ccc3d5a1d4cbe28 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 16 Jun 2026 04:32:43 +0000 Subject: [PATCH 005/138] feat(netbird): add enable_netbird compose helper with idempotency --- cli/lib/composefile.bash | 17 +++++++++++++++ cli/test/composefile/netbird.bats | 35 +++++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+) create mode 100644 cli/test/composefile/netbird.bats diff --git a/cli/lib/composefile.bash b/cli/lib/composefile.bash index c510ea6f..3e0a1cc7 100644 --- a/cli/lib/composefile.bash +++ b/cli/lib/composefile.bash @@ -590,3 +590,20 @@ apply_upstream_ca_bundles() { '.services.mitmproxy.entrypoint = ["/bin/sh", "-c", strenv(new_entrypoint), "sh"]' \ "$compose_file" } + +# Adds NB_SETUP_KEY to the wg-client service's environment in the deployed +# compose-proxy.yml. wg-client-init.sh reads this at startup to enroll the +# container as a NetBird peer and start the daemon on wt0. +# Args: +# $1 - Path to compose-proxy.yml +enable_netbird() { + require yq + local compose_file=$1 + + local already_set + already_set=$(yq '[.services."wg-client".environment[] | select(. == "NB_SETUP_KEY")] | length' "$compose_file") + + if [[ "$already_set" -eq 0 ]]; then + yq -i '.services."wg-client".environment += ["NB_SETUP_KEY"]' "$compose_file" + fi +} diff --git a/cli/test/composefile/netbird.bats b/cli/test/composefile/netbird.bats new file mode 100644 index 00000000..ee72929b --- /dev/null +++ b/cli/test/composefile/netbird.bats @@ -0,0 +1,35 @@ +#!/usr/bin/env bats + +setup() { + load test_helper + source "$SCT_LIBDIR/composefile.bash" + + COMPOSE_FILE="$BATS_TEST_TMPDIR/compose-proxy.yml" + cat >"$COMPOSE_FILE" <<'YAML' +services: + wg-client: + build: + context: . + dockerfile: Dockerfile.wg-client + cap_add: + - NET_ADMIN +YAML +} + +teardown() { + unstub_all +} + +@test "enable_netbird adds NB_SETUP_KEY to wg-client environment" { + enable_netbird "$COMPOSE_FILE" + + yq -e '.services."wg-client".environment[] | select(. == "NB_SETUP_KEY")' "$COMPOSE_FILE" +} + +@test "enable_netbird is idempotent" { + enable_netbird "$COMPOSE_FILE" + enable_netbird "$COMPOSE_FILE" + + run yq '[.services."wg-client".environment[] | select(. == "NB_SETUP_KEY")] | length' "$COMPOSE_FILE" + assert_output "1" +} From 9b76c45d3fb7a15b1735dd7e56545ef6ebd9c8f9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 16 Jun 2026 04:37:52 +0000 Subject: [PATCH 006/138] test(netbird): add compose contract tests enforcing single NET_ADMIN container --- cli/test/composefile/netbird_contract.bats | 29 ++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 cli/test/composefile/netbird_contract.bats diff --git a/cli/test/composefile/netbird_contract.bats b/cli/test/composefile/netbird_contract.bats new file mode 100644 index 00000000..5f5bd158 --- /dev/null +++ b/cli/test/composefile/netbird_contract.bats @@ -0,0 +1,29 @@ +#!/usr/bin/env bats +# +# Structural contracts for the netbird integration. +# The key invariant: wg-client is and remains the only NET_ADMIN container. +# + +setup() { + load test_helper + COMPOSE_PROXY="$SCT_TEMPLATEDIR/devcontainer/sandcat/compose-proxy.yml" + COMPOSE_ALL="$SCT_TEMPLATEDIR/devcontainer/compose-all.yml" +} + +@test "wg-client is the only service in compose-proxy.yml with cap_add" { + # All service names that have a cap_add key — must be exactly one: wg-client. + run yq '[to_entries | .[] | select(.value | has("cap_add")) | .key] | .[]' \ + <(yq '.services' "$COMPOSE_PROXY") + assert_output "wg-client" +} + +@test "compose-proxy.yml template does not contain NB_SETUP_KEY by default" { + run yq '[.services."wg-client".environment[]? | select(. == "NB_SETUP_KEY")] | length' \ + "$COMPOSE_PROXY" + assert_output "0" +} + +@test "compose-all.yml template does not reference netbird by default" { + run grep -c "netbird" "$COMPOSE_ALL" + assert_output "0" +} From 9538b3e1a4e64c73534cb34bc30b686dbfc65122 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 16 Jun 2026 05:07:16 +0000 Subject: [PATCH 007/138] feat(netbird): add netbird management API helpers with unit tests --- cli/lib/netbird.bash | 61 +++++++++++++++++++++++++++++++ cli/test/netbird/netbird_api.bats | 49 +++++++++++++++++++++++++ cli/test/netbird/test_helper.bash | 17 +++++++++ 3 files changed, 127 insertions(+) create mode 100644 cli/lib/netbird.bash create mode 100644 cli/test/netbird/netbird_api.bats create mode 100644 cli/test/netbird/test_helper.bash diff --git a/cli/lib/netbird.bash b/cli/lib/netbird.bash new file mode 100644 index 00000000..0ecdaf6e --- /dev/null +++ b/cli/lib/netbird.bash @@ -0,0 +1,61 @@ +#!/usr/bin/env bash + +# Calls the NetBird management REST API. +# Requires NB_API_TOKEN. NB_MANAGEMENT_URL defaults to https://api.netbird.io. +# Args: +# $1 - HTTP method (GET, POST, DELETE) +# $2 - API path (e.g. /api/peers) +# $3 - Optional JSON body +netbird_api() { + local method=$1 + local path=$2 + local body=${3:-} + + if [[ -z "${NB_API_TOKEN:-}" ]]; then + echo "NB_API_TOKEN is not set" >&2 + return 1 + fi + + local url="${NB_MANAGEMENT_URL:-https://api.netbird.io}${path}" + local args=(-sf -X "$method" + -H "Authorization: Token $NB_API_TOKEN" + -H "Content-Type: application/json") + + [[ -n "$body" ]] && args+=(-d "$body") + + curl "${args[@]}" "$url" +} + +# Returns the current peer list from the NetBird management server. +netbird_status() { + netbird_api "GET" "/api/peers" +} + +# Adds a network route served by a peer. +# Args: +# $1 - Network CIDR (e.g. 10.8.0.0/24) +# $2 - Peer ID that serves the route +netbird_route_add() { + local network=$1 + local peer_id=$2 + netbird_api "POST" "/api/routes" \ + "{\"network\":\"$network\",\"peer\":\"$peer_id\",\"enabled\":true}" +} + +# Removes a network route by ID. +# Args: +# $1 - Route ID (returned by netbird_route_add) +netbird_route_remove() { + local route_id=$1 + netbird_api "DELETE" "/api/routes/$route_id" +} + +# Removes a peer from the NetBird management server. +# Causes netbird-sync.sh to write an updated peers.conf that omits this peer, +# which wg-client then applies via wg syncconf — removing the route to that peer. +# Args: +# $1 - Peer ID +netbird_peer_remove() { + local peer_id=$1 + netbird_api "DELETE" "/api/peers/$peer_id" +} diff --git a/cli/test/netbird/netbird_api.bats b/cli/test/netbird/netbird_api.bats new file mode 100644 index 00000000..7eccafc1 --- /dev/null +++ b/cli/test/netbird/netbird_api.bats @@ -0,0 +1,49 @@ +#!/usr/bin/env bats + +setup() { + load test_helper + source "$SCT_LIBDIR/netbird.bash" + + export NB_MANAGEMENT_URL="https://api.netbird.io" + export NB_API_TOKEN="test-token" +} + +teardown() { + unstub_all +} + +@test "netbird_api fails when NB_API_TOKEN is unset" { + unset NB_API_TOKEN + run netbird_api "GET" "/api/peers" + assert_failure + assert_output --partial "NB_API_TOKEN" +} + +@test "netbird_status calls GET /api/peers" { + stub curl \ + "-sf -X GET -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[{\"id\":\"peer1\",\"connected\":true}]'" + run netbird_status + assert_success + assert_output --partial "peer1" +} + +@test "netbird_route_add calls POST /api/routes with network and peer" { + stub curl \ + "-sf -X POST -H 'Authorization: Token test-token' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"peer1\",\"enabled\":true}' https://api.netbird.io/api/routes : echo '{\"id\":\"route1\"}'" + run netbird_route_add "10.8.0.0/24" "peer1" + assert_success +} + +@test "netbird_route_remove calls DELETE /api/routes/:id" { + stub curl \ + "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/routes/route1 : :" + run netbird_route_remove "route1" + assert_success +} + +@test "netbird_peer_remove calls DELETE /api/peers/:id" { + stub curl \ + "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers/peer1 : :" + run netbird_peer_remove "peer1" + assert_success +} diff --git a/cli/test/netbird/test_helper.bash b/cli/test/netbird/test_helper.bash new file mode 100644 index 00000000..d9a3449d --- /dev/null +++ b/cli/test/netbird/test_helper.bash @@ -0,0 +1,17 @@ +#!/bin/bash +bats_require_minimum_version 1.5.0 +if shopt -s compat32 2>/dev/null; then + export BASH_COMPAT=3.2 +fi +set -uo pipefail +export SHELLOPTS + +SCT_ROOT="$BATS_TEST_DIRNAME/../.." +BATS_LIB_PATH="$SCT_ROOT/support":${BATS_LIB_PATH-} + +bats_load_library bats-ext +bats_load_library bats-support +bats_load_library bats-assert +bats_load_library bats-mock-ext + +export SCT_ROOT SCT_LIBDIR="$SCT_ROOT/lib" From 730917cafae9ac229b09cc9fc2051cf92dd5b741 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 16 Jun 2026 06:11:29 +0000 Subject: [PATCH 008/138] feat(netbird): add sandcat netbird subcommand dispatcher --- cli/libexec/netbird/netbird | 106 ++++++++++++++++++++++++++++++ cli/test/netbird/netbird.bats | 65 ++++++++++++++++++ cli/test/netbird/test_helper.bash | 1 + 3 files changed, 172 insertions(+) create mode 100755 cli/libexec/netbird/netbird create mode 100644 cli/test/netbird/netbird.bats diff --git a/cli/libexec/netbird/netbird b/cli/libexec/netbird/netbird new file mode 100755 index 00000000..bf086bdc --- /dev/null +++ b/cli/libexec/netbird/netbird @@ -0,0 +1,106 @@ +#!/usr/bin/env bash +set -euo pipefail + +# shellcheck source=../../lib/logging.bash +source "$SCT_LIBDIR/logging.bash" +# shellcheck source=../../lib/netbird.bash +source "$SCT_LIBDIR/netbird.bash" + +usage() { + cat <<'EOF' +Usage: sandcat netbird [options] + +Subcommands: + status List peers from the NetBird management server + peer remove --peer-id Remove a peer (triggers wg syncconf in wg-client) + route add --network --peer-id Add a network route + route remove --route-id Remove a route by ID +EOF +} + +cmd_status() { + netbird_status +} + +cmd_peer() { + local subcmd="${1:-}" + shift || true + case "$subcmd" in + remove) + local peer_id="" + while [[ $# -gt 0 ]]; do + case $1 in + --peer-id) peer_id="$2"; shift 2 ;; + *) echo "Unknown option: $1" | error; return 1 ;; + esac + done + if [[ -z "$peer_id" ]]; then + echo "Missing required option: --peer-id" | error; return 1 + fi + netbird_peer_remove "$peer_id" + ;; + *) + echo "Unknown peer subcommand: $subcmd" | error + usage; return 1 + ;; + esac +} + +cmd_route() { + local subcmd="${1:-}" + shift || true + case "$subcmd" in + add) + local network="" peer_id="" + while [[ $# -gt 0 ]]; do + case $1 in + --network) network="$2"; shift 2 ;; + --peer-id) peer_id="$2"; shift 2 ;; + *) echo "Unknown option: $1" | error; return 1 ;; + esac + done + if [[ -z "$network" ]]; then + echo "Missing required option: --network" | error; return 1 + fi + if [[ -z "$peer_id" ]]; then + echo "Missing required option: --peer-id" | error; return 1 + fi + netbird_route_add "$network" "$peer_id" + ;; + remove) + local route_id="" + while [[ $# -gt 0 ]]; do + case $1 in + --route-id) route_id="$2"; shift 2 ;; + *) echo "Unknown option: $1" | error; return 1 ;; + esac + done + if [[ -z "$route_id" ]]; then + echo "Missing required option: --route-id" | error; return 1 + fi + netbird_route_remove "$route_id" + ;; + *) + echo "Unknown route subcommand: $subcmd" | error + usage; return 1 + ;; + esac +} + +main() { + local subcmd="${1:-}" + shift || true + case "$subcmd" in + status) cmd_status "$@" ;; + peer) cmd_peer "$@" ;; + route) cmd_route "$@" ;; + *) + usage + return 1 + ;; + esac +} + +if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + main "$@" +fi diff --git a/cli/test/netbird/netbird.bats b/cli/test/netbird/netbird.bats new file mode 100644 index 00000000..6dd9bbd5 --- /dev/null +++ b/cli/test/netbird/netbird.bats @@ -0,0 +1,65 @@ +#!/usr/bin/env bats + +setup() { + load test_helper + + NETBIRD_CMD="$SCT_LIBEXECDIR/netbird/netbird" + export NB_MANAGEMENT_URL="https://api.netbird.io" + export NB_API_TOKEN="test-token" +} + +teardown() { + unstub_all +} + +@test "netbird status calls GET /api/peers" { + stub curl \ + "-sf -X GET -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[]'" + run bash "$NETBIRD_CMD" status + assert_success +} + +@test "netbird peer remove requires --peer-id" { + run bash "$NETBIRD_CMD" peer remove + assert_failure + assert_output --partial "peer-id" +} + +@test "netbird peer remove calls netbird_peer_remove" { + stub curl \ + "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers/abc123 : :" + run bash "$NETBIRD_CMD" peer remove --peer-id abc123 + assert_success +} + +@test "netbird route add requires --network and --peer-id" { + run bash "$NETBIRD_CMD" route add --network 10.8.0.0/24 + assert_failure + assert_output --partial "peer-id" +} + +@test "netbird route add calls netbird_route_add" { + stub curl \ + "-sf -X POST -H 'Authorization: Token test-token' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"abc123\",\"enabled\":true}' https://api.netbird.io/api/routes : echo '{\"id\":\"route1\"}'" + run bash "$NETBIRD_CMD" route add --network 10.8.0.0/24 --peer-id abc123 + assert_success +} + +@test "netbird route remove requires --route-id" { + run bash "$NETBIRD_CMD" route remove + assert_failure + assert_output --partial "route-id" +} + +@test "netbird route remove calls netbird_route_remove" { + stub curl \ + "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/routes/route1 : :" + run bash "$NETBIRD_CMD" route remove --route-id route1 + assert_success +} + +@test "netbird with unknown subcommand prints usage and fails" { + run bash "$NETBIRD_CMD" bogus + assert_failure + assert_output --partial "Usage" +} diff --git a/cli/test/netbird/test_helper.bash b/cli/test/netbird/test_helper.bash index d9a3449d..e2410330 100644 --- a/cli/test/netbird/test_helper.bash +++ b/cli/test/netbird/test_helper.bash @@ -15,3 +15,4 @@ bats_load_library bats-assert bats_load_library bats-mock-ext export SCT_ROOT SCT_LIBDIR="$SCT_ROOT/lib" +export SCT_LIBEXECDIR="$SCT_ROOT/libexec" From a6a23f43321e83b2d202d56d86b926c021d45bae Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 16 Jun 2026 06:31:19 +0000 Subject: [PATCH 009/138] feat(init): add --netbird flag and enrollment key seeding --- cli/libexec/init/devcontainer | 13 +++++++++++++ cli/libexec/init/init | 19 ++++++++++++++++++- cli/test/init/init.bats | 19 +++++++++++++++++++ 3 files changed, 50 insertions(+), 1 deletion(-) diff --git a/cli/libexec/init/devcontainer b/cli/libexec/init/devcontainer index 4dd307cf..feda873a 100755 --- a/cli/libexec/init/devcontainer +++ b/cli/libexec/init/devcontainer @@ -24,6 +24,7 @@ devcontainer() { local stacks="" local proxy_mode="web" local secret_provider="none" + local netbird="false" while [[ $# -gt 0 ]] do @@ -72,6 +73,10 @@ devcontainer() { secret_provider="1password" shift 1 ;; + --netbird) + netbird="true" + shift 1 + ;; *) echo "Unknown option: $1" | error return 1 @@ -119,6 +124,14 @@ devcontainer() { apply_upstream_ca_bundles "$devcontainer_dir/sandcat/compose-proxy.yml" "$project_path" + if [[ "$netbird" == "true" ]]; then + if ! declare -f enable_netbird &>/dev/null; then + # shellcheck source=../../lib/composefile.bash + source "$SCT_LIBDIR/composefile.bash" + fi + enable_netbird "$devcontainer_dir/sandcat/compose-proxy.yml" + fi + customize_compose_file "$rel_settings_file" "$compose_file" "$agent" "$ide" "$project_name" "$stacks" set_project_name "$compose_file" "$project_name" diff --git a/cli/libexec/init/init b/cli/libexec/init/init index ca9a97f3..7712c1b1 100755 --- a/cli/libexec/init/init +++ b/cli/libexec/init/init @@ -160,6 +160,7 @@ init() { local onepassword_alias=false local features_csv="" local features_provided=false + local netbird="false" while [[ $# -gt 0 ]] do @@ -211,6 +212,10 @@ init() { features_provided=true shift 2 ;; + --netbird) + netbird="true" + shift 1 + ;; *) echo "Unknown option: $1" | error return 1 @@ -394,6 +399,14 @@ init() { add_secret_provider_tokens_to_user_settings "$secret_provider" + if [[ "$netbird" == "true" ]]; then + local user_settings + user_settings="$(sct_home)/settings.json" + if [[ -f "$user_settings" ]]; then + yq -i -o json '.netbird_enrollment_key = (.netbird_enrollment_key // "")' "$user_settings" + fi + fi + local settings_args=() if [[ "$strict_network" == "true" ]]; then settings_args+=(--strict-network --stacks "$stacks_resolved") @@ -410,7 +423,11 @@ init() { --secret-provider "$secret_provider" ) export SANDCAT_RTK="$rtk_enabled" - devcontainer "${devcontainer_args[@]}" + if [[ "$netbird" == "true" ]]; then + devcontainer "${devcontainer_args[@]}" --netbird + else + devcontainer "${devcontainer_args[@]}" + fi local gitignore_status="skipped" if [[ -e "$project_path/.git" ]]; then diff --git a/cli/test/init/init.bats b/cli/test/init/init.bats index 315a8e39..2f738d26 100644 --- a/cli/test/init/init.bats +++ b/cli/test/init/init.bats @@ -332,6 +332,25 @@ EOF [[ ! -e "$HOME/.cursor/mcp.json" ]] } +@test "init --netbird passes netbird flag to devcontainer" { + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird : :" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird + assert_success +} + +@test "init --netbird seeds netbird_enrollment_key in user settings" { + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer ":" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird + assert_success + run yq '.netbird_enrollment_key' "$SCT_HOME_DIR/settings.json" + assert_output '""' +} + @test "init interactive flow (devcontainer mode)" { unset -f read_line unset -f select_option From 485409edf40139b0045c9993ab7c572fe880e75e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 16 Jun 2026 06:41:23 +0000 Subject: [PATCH 010/138] docs(cli): document --netbird flag and sandcat netbird subcommand --- cli/README.md | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/cli/README.md b/cli/README.md index 8d2b8940..2fb888be 100644 --- a/cli/README.md +++ b/cli/README.md @@ -22,6 +22,11 @@ Options: - `--stacks` - Comma-separated development stacks to install: `node`, `python`, `java`, `rust`, `go`, `scala`, `ruby`, `dotnet`, `zig` (skips prompt) - `--proxy` - Proxy UI mode: `web` (default, mitmweb browser UI) or `tui` (mitmproxy console, use with `sandcat proxy` to attach) - `--secret-provider` / `--sp` - Secret backend: `none` (default), `1password`, `protonpass` (skips prompt when set) +- `--netbird` - Enable dynamic WireGuard control via NetBird. The NetBird client + daemon starts inside `wg-client` (the sole `NET_ADMIN` container) and manages + a second interface `wt0` for the NetBird overlay mesh. `wg0` (the mitmproxy + inspection tunnel) is untouched. Seeds `netbird_enrollment_key` in + `~/.config/sandcat/settings.json`. - `--1password` - Deprecated alias for `--secret-provider 1password` - `--features` - Comma-separated optional non-provider features: `tui` (proxy console mode; prefer `--proxy tui`), `no-gitignore` (skip appending the `# Sandcat` block to the project's `.gitignore`; equivalent to `SANDCAT_GITIGNORE=false`), `no-rtk` (skip RTK installation; equivalent to `SANDCAT_RTK=false`), `strict-network` (project settings get network presets for the selected stacks instead of the allow-all-GET wildcard; equivalent to `SANDCAT_STRICT_NETWORK=true`) - `--name` - Project name for Docker Compose (default: derived from directory name) @@ -43,6 +48,9 @@ sandcat init --agent claude --ide vscode --secret-provider 1password --name mypr # With Proton Pass integration sandcat init --agent claude --ide vscode --secret-provider protonpass --name myproject + +# With NetBird dynamic WireGuard +sandcat init --agent claude --ide vscode --netbird --name myproject ``` #### Proton Pass setup (scoped Personal Access Token) @@ -213,6 +221,55 @@ shell, `sandcat run npm install` runs npm inside the container. Options: - `--build` — Rebuild images before running (e.g. after editing `Dockerfile.app`) +## Dynamic networking (NetBird) + +When initialized with `--netbird`, sandcat enrolls `wg-client` as a NetBird peer. +The NetBird client daemon runs inside the existing `NET_ADMIN` container and manages +`wt0` — a second WireGuard interface alongside `wg0`. Removing a peer from the +NetBird management server causes the daemon to drop it from `wt0` within seconds, +removing the agent's route to that endpoint without restarting any container. + +All NetBird traffic (control plane and data plane) routes through `wg0` → mitmproxy, +maintaining the full inspection guarantee. `wg-client` remains the only container +with `NET_ADMIN`. + +### Setup + +1. Create a NetBird account at or self-host the server. +2. Generate a setup key (**Setup Keys** in the NetBird dashboard). +3. Generate an API token (**API Keys** in the NetBird dashboard). +4. Add the enrollment key to your sandcat user settings: + +```json +{ + "netbird_enrollment_key": "your-setup-key-here" +} +``` + +5. Set `NB_SETUP_KEY` in your environment before starting the devcontainer: + +```bash +export NB_SETUP_KEY="your-setup-key-here" +export NB_API_TOKEN="your-api-token" # for sandcat netbird commands +export NB_MANAGEMENT_URL="https://api.netbird.io" # or self-hosted URL +``` + +### Runtime control + +```bash +# List current peers +sandcat netbird status + +# Remove a peer (wg-client drops the route within one daemon poll interval) +sandcat netbird peer remove --peer-id + +# Add a network route served by a peer +sandcat netbird route add --network 10.8.0.0/24 --peer-id + +# Remove a route +sandcat netbird route remove --route-id +``` + ## Directory Structure Each module is contained in its own directory under `cli/libexec/`. From fe90b1bffd4adcd76da126b9591560688a64760e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 16 Jun 2026 07:34:58 +0000 Subject: [PATCH 011/138] feat(netbird): pin netbird binary with per-arch sha256 checksums Add netbird.env as the single source of truth for version and tarball checksums, verify downloads in Dockerfile.wg-client before extract, inject build args at init via apply_netbird_build_args, and fix stale peer-remove comment in netbird.bash. --- cli/README.md | 16 ++++++++++ cli/lib/composefile.bash | 29 +++++++++++++++++++ cli/lib/netbird.bash | 4 +-- cli/libexec/init/devcontainer | 1 + .../devcontainer/sandcat/Dockerfile.wg-client | 26 +++++++++++++---- .../devcontainer/sandcat/netbird.env | 18 ++++++++++++ cli/test/composefile/netbird.bats | 22 ++++++++++++++ cli/test/composefile/netbird_contract.bats | 10 +++++++ 8 files changed, 119 insertions(+), 7 deletions(-) create mode 100644 cli/templates/devcontainer/sandcat/netbird.env diff --git a/cli/README.md b/cli/README.md index 2fb888be..70935839 100644 --- a/cli/README.md +++ b/cli/README.md @@ -233,6 +233,22 @@ All NetBird traffic (control plane and data plane) routes through `wg0` → mitm maintaining the full inspection guarantee. `wg-client` remains the only container with `NET_ADMIN`. +The NetBird client binary is pinned by version and per-arch sha256 in +[`templates/devcontainer/sandcat/netbird.env`](templates/devcontainer/sandcat/netbird.env) +(the same pattern as [`images/mitmproxy-pass/pass-cli.env`](../images/mitmproxy-pass/pass-cli.env)). +`sandcat init` injects these as compose build args for `wg-client` automatically. +To build the image manually: + +```bash +cd cli/templates/devcontainer/sandcat +set -a; . netbird.env; set +a +docker build -f Dockerfile.wg-client \ + --build-arg NETBIRD_VERSION \ + --build-arg NETBIRD_SHA256_AMD64 \ + --build-arg NETBIRD_SHA256_ARM64 \ + -t wg-client-test . +``` + ### Setup 1. Create a NetBird account at or self-host the server. diff --git a/cli/lib/composefile.bash b/cli/lib/composefile.bash index 3e0a1cc7..1d9d475d 100644 --- a/cli/lib/composefile.bash +++ b/cli/lib/composefile.bash @@ -591,6 +591,35 @@ apply_upstream_ca_bundles() { "$compose_file" } +# Injects NetBird version and per-arch checksum build args into wg-client's +# compose build section, sourced from netbird.env (sibling to compose-proxy.yml). +# Args: +# $1 - Path to compose-proxy.yml +apply_netbird_build_args() { + require yq + local compose_file=$1 + local netbird_env + netbird_env="$(dirname "$compose_file")/netbird.env" + + if [[ ! -f "$netbird_env" ]]; then + echo "netbird.env not found beside compose file: $netbird_env" >&2 + return 1 + fi + + # shellcheck disable=SC1090 + source "$netbird_env" + + : "${NETBIRD_VERSION:?NETBIRD_VERSION missing from $netbird_env}" + : "${NETBIRD_SHA256_AMD64:?NETBIRD_SHA256_AMD64 missing from $netbird_env}" + : "${NETBIRD_SHA256_ARM64:?NETBIRD_SHA256_ARM64 missing from $netbird_env}" + + yq -i " + .services.\"wg-client\".build.args.NETBIRD_VERSION = \"${NETBIRD_VERSION}\" | + .services.\"wg-client\".build.args.NETBIRD_SHA256_AMD64 = \"${NETBIRD_SHA256_AMD64}\" | + .services.\"wg-client\".build.args.NETBIRD_SHA256_ARM64 = \"${NETBIRD_SHA256_ARM64}\" + " "$compose_file" +} + # Adds NB_SETUP_KEY to the wg-client service's environment in the deployed # compose-proxy.yml. wg-client-init.sh reads this at startup to enroll the # container as a NetBird peer and start the daemon on wt0. diff --git a/cli/lib/netbird.bash b/cli/lib/netbird.bash index 0ecdaf6e..2997eb5f 100644 --- a/cli/lib/netbird.bash +++ b/cli/lib/netbird.bash @@ -51,8 +51,8 @@ netbird_route_remove() { } # Removes a peer from the NetBird management server. -# Causes netbird-sync.sh to write an updated peers.conf that omits this peer, -# which wg-client then applies via wg syncconf — removing the route to that peer. +# The netbird daemon running in wg-client detects the removal and drops the +# peer from wt0, which removes the route to that endpoint for the agent. # Args: # $1 - Peer ID netbird_peer_remove() { diff --git a/cli/libexec/init/devcontainer b/cli/libexec/init/devcontainer index feda873a..7c593497 100755 --- a/cli/libexec/init/devcontainer +++ b/cli/libexec/init/devcontainer @@ -121,6 +121,7 @@ devcontainer() { fi apply_secret_provider "$devcontainer_dir/sandcat/compose-proxy.yml" "$secret_provider" + apply_netbird_build_args "$devcontainer_dir/sandcat/compose-proxy.yml" apply_upstream_ca_bundles "$devcontainer_dir/sandcat/compose-proxy.yml" "$project_path" diff --git a/cli/templates/devcontainer/sandcat/Dockerfile.wg-client b/cli/templates/devcontainer/sandcat/Dockerfile.wg-client index 5f9139f0..1beaaaaa 100644 --- a/cli/templates/devcontainer/sandcat/Dockerfile.wg-client +++ b/cli/templates/devcontainer/sandcat/Dockerfile.wg-client @@ -9,7 +9,6 @@ FROM debian:trixie-slim # dnsmasq - local DNS forwarder providing split-DNS # ca-certificates + curl - required for trusting the mitmproxy CA cert at runtime # and downloading the netbird binary at build time -ARG NETBIRD_VERSION=0.28.9 RUN apt-get update \ && apt-get install -y --no-install-recommends \ wireguard-tools iproute2 iptables jq openresolv dnsmasq \ @@ -19,11 +18,28 @@ RUN apt-get update \ # Install the NetBird client daemon. The daemon manages wt0 (the NetBird # overlay mesh) inside this container, which already holds NET_ADMIN. # wg0 (the mitmproxy inspection tunnel) is not touched by the daemon. -RUN ARCH=$(dpkg --print-architecture) \ - && curl -sSLf \ +# +# Version + per-arch checksums are the single source of truth in netbird.env +# (sibling to this Dockerfile) and MUST be supplied as build args. No defaults +# here on purpose, so the pin lives in exactly one place. +ARG NETBIRD_VERSION +ARG NETBIRD_SHA256_AMD64 +ARG NETBIRD_SHA256_ARM64 +RUN test -n "$NETBIRD_VERSION" || { echo "NETBIRD_VERSION build arg is required (source netbird.env)" >&2; exit 1; } \ + && ARCH=$(dpkg --print-architecture) \ + && case "$ARCH" in \ + amd64) NETBIRD_SHA256="$NETBIRD_SHA256_AMD64" ;; \ + arm64) NETBIRD_SHA256="$NETBIRD_SHA256_ARM64" ;; \ + *) echo "Unsupported architecture: $ARCH" >&2; exit 1 ;; \ + esac \ + && test -n "$NETBIRD_SHA256" \ + && curl -sSLf -o /tmp/netbird.tar.gz \ "https://github.com/netbirdio/netbird/releases/download/v${NETBIRD_VERSION}/netbird_${NETBIRD_VERSION}_linux_${ARCH}.tar.gz" \ - | tar xz -C /usr/local/bin netbird \ - && chmod +x /usr/local/bin/netbird + && echo "${NETBIRD_SHA256} /tmp/netbird.tar.gz" | sha256sum -c - \ + && tar xzf /tmp/netbird.tar.gz -C /usr/local/bin netbird \ + && chmod +x /usr/local/bin/netbird \ + && rm /tmp/netbird.tar.gz \ + && netbird version COPY scripts/wg-client-init.sh /usr/local/bin/wg-client-init.sh COPY scripts/dnsmasq-ready /usr/local/bin/dnsmasq-ready diff --git a/cli/templates/devcontainer/sandcat/netbird.env b/cli/templates/devcontainer/sandcat/netbird.env new file mode 100644 index 00000000..ee4dbe0b --- /dev/null +++ b/cli/templates/devcontainer/sandcat/netbird.env @@ -0,0 +1,18 @@ +# Single source of truth for the pinned NetBird client binary in wg-client. +# +# Consumed by: +# - cli/templates/devcontainer/sandcat/Dockerfile.wg-client (via compose build args) +# - cli/lib/composefile.bash apply_netbird_build_args +# - cli/test/composefile/netbird_contract.bats +# +# When bumping the version: +# 1. Update NETBIRD_VERSION and BOTH checksums below from the release assets: +# https://github.com/netbirdio/netbird/releases/download/v/netbird__checksums.txt +# 2. Look for netbird__linux_amd64.tar.gz and netbird__linux_arm64.tar.gz +# +# Format note: simple KEY=value lines only (no quotes, no spaces around `=`) so +# this file is consumable by `source`, compose build-arg injection, and contract +# tests alike. +NETBIRD_VERSION=0.28.9 +NETBIRD_SHA256_AMD64=b678b79633dec4a876ca41933d6b1bdd22a7ccb646cd85a167ce0bf6482cea02 +NETBIRD_SHA256_ARM64=f509dc45c1038fa189df26cf27e3dfd2334d01ca8b193df2706ead3b63038d55 diff --git a/cli/test/composefile/netbird.bats b/cli/test/composefile/netbird.bats index ee72929b..fad714a8 100644 --- a/cli/test/composefile/netbird.bats +++ b/cli/test/composefile/netbird.bats @@ -5,6 +5,7 @@ setup() { source "$SCT_LIBDIR/composefile.bash" COMPOSE_FILE="$BATS_TEST_TMPDIR/compose-proxy.yml" + cp "$SCT_TEMPLATEDIR/devcontainer/sandcat/netbird.env" "$BATS_TEST_TMPDIR/netbird.env" cat >"$COMPOSE_FILE" <<'YAML' services: wg-client: @@ -33,3 +34,24 @@ teardown() { run yq '[.services."wg-client".environment[] | select(. == "NB_SETUP_KEY")] | length' "$COMPOSE_FILE" assert_output "1" } + +@test "apply_netbird_build_args injects version and checksum build args" { + apply_netbird_build_args "$COMPOSE_FILE" + + # shellcheck disable=SC1091 + source "$BATS_TEST_TMPDIR/netbird.env" + run yq -r '.services."wg-client".build.args.NETBIRD_VERSION' "$COMPOSE_FILE" + assert_output "$NETBIRD_VERSION" + run yq -r '.services."wg-client".build.args.NETBIRD_SHA256_AMD64' "$COMPOSE_FILE" + assert_output "$NETBIRD_SHA256_AMD64" + run yq -r '.services."wg-client".build.args.NETBIRD_SHA256_ARM64' "$COMPOSE_FILE" + assert_output "$NETBIRD_SHA256_ARM64" +} + +@test "apply_netbird_build_args is idempotent" { + apply_netbird_build_args "$COMPOSE_FILE" + apply_netbird_build_args "$COMPOSE_FILE" + + run yq '.services."wg-client".build.args | length' "$COMPOSE_FILE" + assert_output "3" +} diff --git a/cli/test/composefile/netbird_contract.bats b/cli/test/composefile/netbird_contract.bats index 5f5bd158..fea8d8c0 100644 --- a/cli/test/composefile/netbird_contract.bats +++ b/cli/test/composefile/netbird_contract.bats @@ -27,3 +27,13 @@ setup() { run grep -c "netbird" "$COMPOSE_ALL" assert_output "0" } + +@test "netbird.env pins version and per-arch sha256 checksums" { + local env_file="$SCT_TEMPLATEDIR/devcontainer/sandcat/netbird.env" + # shellcheck disable=SC1090 + source "$env_file" + + [[ -n "$NETBIRD_VERSION" ]] + [[ "$NETBIRD_SHA256_AMD64" =~ ^[0-9a-f]{64}$ ]] + [[ "$NETBIRD_SHA256_ARM64" =~ ^[0-9a-f]{64}$ ]] +} From 90abce8e8f386193d913dd5b2d75d0ea8d5dc697 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 16 Jun 2026 07:38:53 +0000 Subject: [PATCH 012/138] feat(netbird): resolve API and enrollment tokens from sandcat settings Read netbird_api_token and netbird_enrollment_key from user, project, and local settings layers (env overrides). Export NB_SETUP_KEY before docker compose in sandcat compose/run/attach/restart-proxy. Seed netbird_api_token on init --netbird and document settings-based configuration. --- cli/README.md | 40 ++++++--- cli/lib/netbird.bash | 118 ++++++++++++++++++++----- cli/libexec/attach/attach | 4 + cli/libexec/compose/compose | 4 + cli/libexec/init/init | 12 ++- cli/libexec/netbird/_ | 6 ++ cli/libexec/netbird/netbird | 32 +++++-- cli/libexec/restart/restart | 4 + cli/libexec/run/run | 4 + cli/test/init/init.bats | 2 + cli/test/netbird/netbird.bats | 31 ++++++- cli/test/netbird/netbird_api.bats | 12 +-- cli/test/netbird/netbird_settings.bats | 88 ++++++++++++++++++ 13 files changed, 309 insertions(+), 48 deletions(-) create mode 100755 cli/libexec/netbird/_ create mode 100644 cli/test/netbird/netbird_settings.bats diff --git a/cli/README.md b/cli/README.md index 70935839..588e0254 100644 --- a/cli/README.md +++ b/cli/README.md @@ -25,8 +25,8 @@ Options: - `--netbird` - Enable dynamic WireGuard control via NetBird. The NetBird client daemon starts inside `wg-client` (the sole `NET_ADMIN` container) and manages a second interface `wt0` for the NetBird overlay mesh. `wg0` (the mitmproxy - inspection tunnel) is untouched. Seeds `netbird_enrollment_key` in - `~/.config/sandcat/settings.json`. + inspection tunnel) is untouched. Seeds `netbird_enrollment_key` and + `netbird_api_token` in `~/.config/sandcat/settings.json`. - `--1password` - Deprecated alias for `--secret-provider 1password` - `--features` - Comma-separated optional non-provider features: `tui` (proxy console mode; prefer `--proxy tui`), `no-gitignore` (skip appending the `# Sandcat` block to the project's `.gitignore`; equivalent to `SANDCAT_GITIGNORE=false`), `no-rtk` (skip RTK installation; equivalent to `SANDCAT_RTK=false`), `strict-network` (project settings get network presets for the selected stacks instead of the allow-all-GET wildcard; equivalent to `SANDCAT_STRICT_NETWORK=true`) - `--name` - Project name for Docker Compose (default: derived from directory name) @@ -251,25 +251,45 @@ docker build -f Dockerfile.wg-client \ ### Setup +NetBird uses **two separate credentials**. Both go in `~/.config/sandcat/settings.json` +(created by `sandcat init`; edit with `sandcat edit user-settings`): + +| Setting key | Used for | Where to get it | +|-------------|----------|-----------------| +| `netbird_enrollment_key` | Enrolling `wg-client` as a mesh peer (`NB_SETUP_KEY`) | NetBird dashboard → **Setup Keys** | +| `netbird_api_token` | `sandcat netbird` CLI commands on your host | NetBird dashboard → **API Keys** (Personal Access Token) | + +**Before `sandcat netbird status` works**, you must complete steps 1–4 below. +Container enrollment (`netbird_enrollment_key`) is separate from host CLI control +(`netbird_api_token`) — you need the API token even if the devcontainer is already running. + 1. Create a NetBird account at or self-host the server. -2. Generate a setup key (**Setup Keys** in the NetBird dashboard). -3. Generate an API token (**API Keys** in the NetBird dashboard). -4. Add the enrollment key to your sandcat user settings: +2. In the dashboard, create a **Setup Key** (for peer enrollment). +3. In the dashboard, create an **API Key** / personal access token (for `sandcat netbird` commands). +4. Add both values to user settings: ```json { - "netbird_enrollment_key": "your-setup-key-here" + "netbird_enrollment_key": "your-setup-key-here", + "netbird_api_token": "your-api-token-here" } ``` -5. Set `NB_SETUP_KEY` in your environment before starting the devcontainer: +Or edit interactively: ```bash -export NB_SETUP_KEY="your-setup-key-here" -export NB_API_TOKEN="your-api-token" # for sandcat netbird commands -export NB_MANAGEMENT_URL="https://api.netbird.io" # or self-hosted URL +sandcat edit user-settings ``` +`sandcat compose` and `sandcat run` read `netbird_enrollment_key` and export +`NB_SETUP_KEY` automatically when starting containers. `sandcat netbird` +commands read `netbird_api_token` from the same settings layers (project +settings override user settings when non-empty). Environment variables +`NB_SETUP_KEY` and `NB_API_TOKEN` override settings when set. + +Optional: set `NB_MANAGEMENT_URL` when using a self-hosted NetBird server +(default: `https://api.netbird.io`). + ### Runtime control ```bash diff --git a/cli/lib/netbird.bash b/cli/lib/netbird.bash index 2997eb5f..2b94b75c 100644 --- a/cli/lib/netbird.bash +++ b/cli/lib/netbird.bash @@ -1,34 +1,106 @@ #!/usr/bin/env bash +# shellcheck source=constants.bash +source "${BASH_SOURCE%/*}/constants.bash" +# shellcheck source=path.bash +source "${BASH_SOURCE%/*}/path.bash" +# shellcheck source=require.bash +source "${BASH_SOURCE%/*}/require.bash" + +# Reads a NetBird setting from sandcat settings layers. Later layers win when +# non-empty (user < project < project local), matching mitmproxy addon precedence. +# Args: +# $1 - settings key (e.g. netbird_api_token) +netbird_read_setting() { + local key=$1 + require yq + + local value="" + local file layer_value repo_root + + local -a layers=() + layers+=("$(sct_home)/settings.json") + if repo_root=$(find_repo_root 2>/dev/null); then + layers+=("$repo_root/$SCT_PROJECT_DIR/settings.json") + layers+=("$repo_root/$SCT_PROJECT_DIR/settings.local.json") + fi + + for file in "${layers[@]}"; do + [[ -f "$file" ]] || continue + layer_value=$(yq -r ".$key // \"\"" "$file") + if [[ -n "$layer_value" ]]; then + value="$layer_value" + fi + done + + printf '%s' "$value" +} + +# Export NB_SETUP_KEY from settings when not already set in the environment. +# Used before docker compose so wg-client receives the enrollment key on create. +export_netbird_compose_env() { + [[ -n "${NB_SETUP_KEY:-}" ]] && return 0 + + local enrollment_key + enrollment_key=$(netbird_read_setting netbird_enrollment_key) + if [[ -n "$enrollment_key" ]]; then + export NB_SETUP_KEY="$enrollment_key" + fi +} + +# Resolve NB_API_TOKEN from settings when unset. Env always wins. +_ensure_netbird_api_token() { + [[ -n "${NB_API_TOKEN:-}" ]] && return 0 + + local token + token=$(netbird_read_setting netbird_api_token) + if [[ -n "$token" ]]; then + export NB_API_TOKEN="$token" + return 0 + fi + + echo "netbird_api_token is not set." >&2 + echo "Add it to $(sct_home)/settings.json (or export NB_API_TOKEN)." >&2 + echo "Create a token in the NetBird dashboard under API Keys, then run: sandcat edit user-settings" >&2 + echo "See cli/README.md § Dynamic networking (NetBird) for the full setup." >&2 + return 1 +} + # Calls the NetBird management REST API. -# Requires NB_API_TOKEN. NB_MANAGEMENT_URL defaults to https://api.netbird.io. +# Requires NB_API_TOKEN (env or netbird_api_token in settings). +# NB_MANAGEMENT_URL defaults to https://api.netbird.io. +# Prints the response body on success; writes curl/API errors to stderr. # Args: # $1 - HTTP method (GET, POST, DELETE) # $2 - API path (e.g. /api/peers) # $3 - Optional JSON body netbird_api() { - local method=$1 - local path=$2 - local body=${3:-} + local method=$1 + local path=$2 + local body=${3:-} + + _ensure_netbird_api_token || return 1 - if [[ -z "${NB_API_TOKEN:-}" ]]; then - echo "NB_API_TOKEN is not set" >&2 - return 1 - fi + local url="${NB_MANAGEMENT_URL:-https://api.netbird.io}${path}" + local -a args=(-sS -f -X "$method" + -H "Authorization: Token $NB_API_TOKEN" + -H "Accept: application/json" + -H "Content-Type: application/json") - local url="${NB_MANAGEMENT_URL:-https://api.netbird.io}${path}" - local args=(-sf -X "$method" - -H "Authorization: Token $NB_API_TOKEN" - -H "Content-Type: application/json") + [[ -n "$body" ]] && args+=(-d "$body") - [[ -n "$body" ]] && args+=(-d "$body") + local response + if ! response=$(curl "${args[@]}" "$url" 2>&1); then + echo "NetBird API ${method} ${path} failed: ${response}" >&2 + return 1 + fi - curl "${args[@]}" "$url" + printf '%s\n' "$response" } # Returns the current peer list from the NetBird management server. netbird_status() { - netbird_api "GET" "/api/peers" + netbird_api "GET" "/api/peers" } # Adds a network route served by a peer. @@ -36,18 +108,18 @@ netbird_status() { # $1 - Network CIDR (e.g. 10.8.0.0/24) # $2 - Peer ID that serves the route netbird_route_add() { - local network=$1 - local peer_id=$2 - netbird_api "POST" "/api/routes" \ - "{\"network\":\"$network\",\"peer\":\"$peer_id\",\"enabled\":true}" + local network=$1 + local peer_id=$2 + netbird_api "POST" "/api/routes" \ + "{\"network\":\"$network\",\"peer\":\"$peer_id\",\"enabled\":true}" } # Removes a network route by ID. # Args: # $1 - Route ID (returned by netbird_route_add) netbird_route_remove() { - local route_id=$1 - netbird_api "DELETE" "/api/routes/$route_id" + local route_id=$1 + netbird_api "DELETE" "/api/routes/$route_id" } # Removes a peer from the NetBird management server. @@ -56,6 +128,6 @@ netbird_route_remove() { # Args: # $1 - Peer ID netbird_peer_remove() { - local peer_id=$1 - netbird_api "DELETE" "/api/peers/$peer_id" + local peer_id=$1 + netbird_api "DELETE" "/api/peers/$peer_id" } diff --git a/cli/libexec/attach/attach b/cli/libexec/attach/attach index d66a0764..ade3c415 100755 --- a/cli/libexec/attach/attach +++ b/cli/libexec/attach/attach @@ -5,6 +5,8 @@ set -euo pipefail source "$SCT_LIBDIR/require.bash" # shellcheck source=../../lib/path.bash source "$SCT_LIBDIR/path.bash" +# shellcheck source=../../lib/netbird.bash +source "$SCT_LIBDIR/netbird.bash" attach() { require docker @@ -12,6 +14,8 @@ attach() { local compose_file compose_file="$(find_compose_file)" + export_netbird_compose_env + if (($# == 0)) then exec docker compose -f "$compose_file" exec -u vscode agent bash --login diff --git a/cli/libexec/compose/compose b/cli/libexec/compose/compose index 81fe1e61..ccda32ef 100755 --- a/cli/libexec/compose/compose +++ b/cli/libexec/compose/compose @@ -5,6 +5,8 @@ set -euo pipefail source "$SCT_LIBDIR/require.bash" # shellcheck source=../../lib/path.bash source "$SCT_LIBDIR/path.bash" +# shellcheck source=../../lib/netbird.bash +source "$SCT_LIBDIR/netbird.bash" # Helper that automatically locates the docker compose file # All arguments are passed to docker compose @@ -14,6 +16,8 @@ compose() { local compose_file compose_file="$(find_compose_file)" + export_netbird_compose_env + exec docker compose -f "$compose_file" "$@" } diff --git a/cli/libexec/init/init b/cli/libexec/init/init index 7712c1b1..c73bac70 100755 --- a/cli/libexec/init/init +++ b/cli/libexec/init/init @@ -403,7 +403,10 @@ init() { local user_settings user_settings="$(sct_home)/settings.json" if [[ -f "$user_settings" ]]; then - yq -i -o json '.netbird_enrollment_key = (.netbird_enrollment_key // "")' "$user_settings" + yq -i -o json ' + .netbird_enrollment_key = (.netbird_enrollment_key // "") | + .netbird_api_token = (.netbird_api_token // "") + ' "$user_settings" fi fi @@ -528,6 +531,13 @@ init() { echo " GITHUB_TOKEN a GitHub personal access token (for git push, gh cli)" | info ;; esac + if [[ "$netbird" == "true" ]]; then + echo "" >&2 + echo " NetBird setup:" | info + echo " Add keys to ~/.config/sandcat/settings.json:" | info + echo " \"netbird_enrollment_key\": \"\" (wg-client enrollment)" | info + echo " \"netbird_api_token\": \"\" (sandcat netbird commands)" | info + fi echo " Then run: sandcat run, or reopen the project using the dev container" | info } diff --git a/cli/libexec/netbird/_ b/cli/libexec/netbird/_ new file mode 100755 index 00000000..4866a695 --- /dev/null +++ b/cli/libexec/netbird/_ @@ -0,0 +1,6 @@ +#!/usr/bin/env bash + +# Catch subcommands (status, peer, route) and forward them to the netbird +# dispatcher. Without this, `sandcat netbird status` looks for libexec/netbird/status. + +exec netbird "$@" diff --git a/cli/libexec/netbird/netbird b/cli/libexec/netbird/netbird index bf086bdc..8ffe8341 100755 --- a/cli/libexec/netbird/netbird +++ b/cli/libexec/netbird/netbird @@ -11,15 +11,27 @@ usage() { Usage: sandcat netbird [options] Subcommands: - status List peers from the NetBird management server - peer remove --peer-id Remove a peer (triggers wg syncconf in wg-client) + status List peers from the NetBird management server + peer remove --peer-id Remove a peer; wg-client drops the route from wt0 route add --network --peer-id Add a network route - route remove --route-id Remove a route by ID + route remove --route-id Remove a route by ID EOF } cmd_status() { - netbird_status + local peers + peers=$(netbird_status) || return 1 + + if [[ -z "$peers" || "$peers" == "[]" || "$peers" == "null" ]]; then + echo "No peers registered in NetBird (check app.netbird.io dashboard)." + return 0 + fi + + if command -v jq &>/dev/null; then + echo "$peers" | jq . + else + echo "$peers" + fi } cmd_peer() { @@ -65,7 +77,17 @@ cmd_route() { if [[ -z "$peer_id" ]]; then echo "Missing required option: --peer-id" | error; return 1 fi - netbird_route_add "$network" "$peer_id" + if [[ ! "$network" =~ ^[0-9]+(\.[0-9]+){3}/[0-9]+$ ]]; then + echo "--network must be a CIDR (e.g. 10.8.0.0/24), not an interface name" | error + return 1 + fi + local response + response=$(netbird_route_add "$network" "$peer_id") || return 1 + if command -v jq &>/dev/null; then + echo "$response" | jq . + else + echo "$response" + fi ;; remove) local route_id="" diff --git a/cli/libexec/restart/restart b/cli/libexec/restart/restart index f4ccd2af..a0848e2a 100755 --- a/cli/libexec/restart/restart +++ b/cli/libexec/restart/restart @@ -7,6 +7,8 @@ source "$SCT_LIBDIR/logging.bash" source "$SCT_LIBDIR/require.bash" # shellcheck source=../../lib/path.bash source "$SCT_LIBDIR/path.bash" +# shellcheck source=../../lib/netbird.bash +source "$SCT_LIBDIR/netbird.bash" # Restarts the mitmproxy, wg-client, and agent services to pick up settings # changes. The agent is re-linked to the fresh wg-client netns (see #69). @@ -16,6 +18,8 @@ restart() { local compose_file compose_file="$(find_compose_file)" + export_netbird_compose_env + local proxy_status proxy_status=$(docker compose -f "$compose_file" ps mitmproxy --status running --quiet 2>/dev/null) || true diff --git a/cli/libexec/run/run b/cli/libexec/run/run index 3c1cafa4..d60d970a 100755 --- a/cli/libexec/run/run +++ b/cli/libexec/run/run @@ -7,6 +7,8 @@ source "$SCT_LIBDIR/require.bash" source "$SCT_LIBDIR/path.bash" # shellcheck source=../../lib/volume.bash source "$SCT_LIBDIR/volume.bash" +# shellcheck source=../../lib/netbird.bash +source "$SCT_LIBDIR/netbird.bash" # Run a command in the agent container. # Starts dependencies, runs the command, then tears everything down. @@ -38,6 +40,8 @@ run() { warn_stale_home_volume "$compose_file" ensure_shared_cache_volumes "$compose_file" + export_netbird_compose_env + local rc=0 docker compose -f "$compose_file" run --rm "${run_opts[@]+"${run_opts[@]}"}" agent "${1-bash}" "${@:2}" || rc=$? docker compose -f "$compose_file" down diff --git a/cli/test/init/init.bats b/cli/test/init/init.bats index 2f738d26..ff126db9 100644 --- a/cli/test/init/init.bats +++ b/cli/test/init/init.bats @@ -349,6 +349,8 @@ EOF assert_success run yq '.netbird_enrollment_key' "$SCT_HOME_DIR/settings.json" assert_output '""' + run yq '.netbird_api_token' "$SCT_HOME_DIR/settings.json" + assert_output '""' } @test "init interactive flow (devcontainer mode)" { diff --git a/cli/test/netbird/netbird.bats b/cli/test/netbird/netbird.bats index 6dd9bbd5..b3e12b4e 100644 --- a/cli/test/netbird/netbird.bats +++ b/cli/test/netbird/netbird.bats @@ -14,9 +14,18 @@ teardown() { @test "netbird status calls GET /api/peers" { stub curl \ - "-sf -X GET -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[]'" + "-sS -f -X GET -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[]'" run bash "$NETBIRD_CMD" status assert_success + assert_output --partial "No peers registered" +} + +@test "netbird status prints peer list when peers exist" { + stub curl \ + "-sS -f -X GET -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[{\"id\":\"peer1\",\"name\":\"wg-client\"}]'" + run bash "$NETBIRD_CMD" status + assert_success + assert_output --partial "peer1" } @test "netbird peer remove requires --peer-id" { @@ -27,7 +36,7 @@ teardown() { @test "netbird peer remove calls netbird_peer_remove" { stub curl \ - "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers/abc123 : :" + "-sS -f -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers/abc123 : :" run bash "$NETBIRD_CMD" peer remove --peer-id abc123 assert_success } @@ -38,11 +47,18 @@ teardown() { assert_output --partial "peer-id" } +@test "netbird route add rejects non-CIDR network" { + run bash "$NETBIRD_CMD" route add --network wd0 --peer-id abc123 + assert_failure + assert_output --partial "CIDR" +} + @test "netbird route add calls netbird_route_add" { stub curl \ - "-sf -X POST -H 'Authorization: Token test-token' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"abc123\",\"enabled\":true}' https://api.netbird.io/api/routes : echo '{\"id\":\"route1\"}'" + "-sS -f -X POST -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"abc123\",\"enabled\":true}' https://api.netbird.io/api/routes : echo '{\"id\":\"route1\"}'" run bash "$NETBIRD_CMD" route add --network 10.8.0.0/24 --peer-id abc123 assert_success + assert_output --partial "route1" } @test "netbird route remove requires --route-id" { @@ -53,7 +69,7 @@ teardown() { @test "netbird route remove calls netbird_route_remove" { stub curl \ - "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/routes/route1 : :" + "-sS -f -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/routes/route1 : :" run bash "$NETBIRD_CMD" route remove --route-id route1 assert_success } @@ -63,3 +79,10 @@ teardown() { assert_failure assert_output --partial "Usage" } + +@test "sandcat netbird status routes subcommand through module dispatcher" { + stub curl \ + "-sS -f -X GET -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[]'" + run bash "$SCT_ROOT/bin/sandcat" netbird status + assert_success +} diff --git a/cli/test/netbird/netbird_api.bats b/cli/test/netbird/netbird_api.bats index 7eccafc1..926bc83c 100644 --- a/cli/test/netbird/netbird_api.bats +++ b/cli/test/netbird/netbird_api.bats @@ -4,6 +4,8 @@ setup() { load test_helper source "$SCT_LIBDIR/netbird.bash" + export HOME="$BATS_TEST_TMPDIR/home" + mkdir -p "$HOME/.config/sandcat" export NB_MANAGEMENT_URL="https://api.netbird.io" export NB_API_TOKEN="test-token" } @@ -16,12 +18,12 @@ teardown() { unset NB_API_TOKEN run netbird_api "GET" "/api/peers" assert_failure - assert_output --partial "NB_API_TOKEN" + assert_output --partial "netbird_api_token" } @test "netbird_status calls GET /api/peers" { stub curl \ - "-sf -X GET -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[{\"id\":\"peer1\",\"connected\":true}]'" + "-sS -f -X GET -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[{\"id\":\"peer1\",\"connected\":true}]'" run netbird_status assert_success assert_output --partial "peer1" @@ -29,21 +31,21 @@ teardown() { @test "netbird_route_add calls POST /api/routes with network and peer" { stub curl \ - "-sf -X POST -H 'Authorization: Token test-token' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"peer1\",\"enabled\":true}' https://api.netbird.io/api/routes : echo '{\"id\":\"route1\"}'" + "-sS -f -X POST -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"peer1\",\"enabled\":true}' https://api.netbird.io/api/routes : echo '{\"id\":\"route1\"}'" run netbird_route_add "10.8.0.0/24" "peer1" assert_success } @test "netbird_route_remove calls DELETE /api/routes/:id" { stub curl \ - "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/routes/route1 : :" + "-sS -f -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/routes/route1 : :" run netbird_route_remove "route1" assert_success } @test "netbird_peer_remove calls DELETE /api/peers/:id" { stub curl \ - "-sf -X DELETE -H 'Authorization: Token test-token' -H 'Content-Type: application/json' https://api.netbird.io/api/peers/peer1 : :" + "-sS -f -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers/peer1 : :" run netbird_peer_remove "peer1" assert_success } diff --git a/cli/test/netbird/netbird_settings.bats b/cli/test/netbird/netbird_settings.bats new file mode 100644 index 00000000..80c5863c --- /dev/null +++ b/cli/test/netbird/netbird_settings.bats @@ -0,0 +1,88 @@ +#!/usr/bin/env bats + +setup() { + load test_helper + source "$SCT_LIBDIR/netbird.bash" + + export HOME="$BATS_TEST_TMPDIR/home" + mkdir -p "$HOME/.config/sandcat" + PROJECT_DIR="$BATS_TEST_TMPDIR/project" + mkdir -p "$PROJECT_DIR/.sandcat" + cd "$PROJECT_DIR" || return 1 +} + +teardown() { + unstub_all +} + +@test "netbird_read_setting returns empty when no settings exist" { + run netbird_read_setting netbird_api_token + assert_success + assert_output "" +} + +@test "netbird_read_setting reads netbird_api_token from user settings" { + echo '{"netbird_api_token": "user-token"}' > "$HOME/.config/sandcat/settings.json" + + run netbird_read_setting netbird_api_token + assert_output "user-token" +} + +@test "netbird_read_setting prefers project settings over user settings" { + echo '{"netbird_api_token": "user-token"}' > "$HOME/.config/sandcat/settings.json" + echo '{"netbird_api_token": "project-token"}' > "$PROJECT_DIR/.sandcat/settings.json" + + run netbird_read_setting netbird_api_token + assert_output "project-token" +} + +@test "netbird_read_setting prefers local project settings over project settings" { + echo '{"netbird_api_token": "user-token"}' > "$HOME/.config/sandcat/settings.json" + echo '{"netbird_api_token": "project-token"}' > "$PROJECT_DIR/.sandcat/settings.json" + echo '{"netbird_api_token": "local-token"}' > "$PROJECT_DIR/.sandcat/settings.local.json" + + run netbird_read_setting netbird_api_token + assert_output "local-token" +} + +@test "export_netbird_compose_env exports enrollment key from user settings" { + echo '{"netbird_enrollment_key": "setup-key-123"}' > "$HOME/.config/sandcat/settings.json" + unset NB_SETUP_KEY + + export_netbird_compose_env + + [[ "$NB_SETUP_KEY" == "setup-key-123" ]] +} + +@test "export_netbird_compose_env does not override existing NB_SETUP_KEY" { + echo '{"netbird_enrollment_key": "from-settings"}' > "$HOME/.config/sandcat/settings.json" + export NB_SETUP_KEY="from-env" + + export_netbird_compose_env + + [[ "$NB_SETUP_KEY" == "from-env" ]] +} + +@test "netbird_api reads netbird_api_token from user settings" { + echo '{"netbird_api_token": "settings-token"}' > "$HOME/.config/sandcat/settings.json" + unset NB_API_TOKEN + export NB_MANAGEMENT_URL="https://api.netbird.io" + + stub curl \ + "-sS -f -X GET -H 'Authorization: Token settings-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[]'" + + run netbird_api "GET" "/api/peers" + assert_success +} + +@test "netbird_api prefers NB_API_TOKEN env over settings" { + echo '{"netbird_api_token": "settings-token"}' > "$HOME/.config/sandcat/settings.json" + export NB_API_TOKEN="env-token" + export NB_MANAGEMENT_URL="https://api.netbird.io" + + stub curl \ + "-sS -f -X GET -H 'Authorization: Token env-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[]'" + + run netbird_api "GET" "/api/peers" + assert_success +} From 1fcfba5b552a5db55529c46851e7f91e4fc333ab Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Thu, 18 Jun 2026 05:04:17 +0000 Subject: [PATCH 013/138] feat(netbird): add management server selection and template provisioning Enable cloud/existing/new NetBird management server flows in init and persist the selected management URL so runtime commands and compose wiring resolve it automatically. Add a self-hosted server template scaffold and focused tests/docs so self-hosted onboarding is explicit and repeatable. --- cli/README.md | 40 +++++- cli/lib/composefile.bash | 14 +- cli/lib/netbird.bash | 51 +++++++ cli/libexec/attach/attach | 1 + cli/libexec/compose/compose | 1 + cli/libexec/init/devcontainer | 10 +- cli/libexec/init/init | 125 +++++++++++++++- cli/libexec/netbird/netbird | 36 ++++- cli/libexec/restart/restart | 1 + cli/libexec/run/run | 1 + cli/templates/netbird-server/README.md | 16 +++ .../netbird-server/docker-compose.yml | 117 +++++++++++++++ cli/templates/netbird-server/management.json | 1 + .../netbird-server/netbird-server.env | 12 ++ cli/templates/netbird-server/turnserver.conf | 8 ++ cli/test/composefile/netbird.bats | 35 +++++ cli/test/init/init.bats | 133 +++++++++++++++++- cli/test/netbird/netbird.bats | 24 ++++ cli/test/netbird/netbird_settings.bats | 88 ++++++++++++ cli/test/restart/restart.bats | 14 ++ 20 files changed, 715 insertions(+), 13 deletions(-) create mode 100644 cli/templates/netbird-server/README.md create mode 100644 cli/templates/netbird-server/docker-compose.yml create mode 100644 cli/templates/netbird-server/management.json create mode 100644 cli/templates/netbird-server/netbird-server.env create mode 100644 cli/templates/netbird-server/turnserver.conf diff --git a/cli/README.md b/cli/README.md index 588e0254..79c0fbbd 100644 --- a/cli/README.md +++ b/cli/README.md @@ -27,6 +27,9 @@ Options: a second interface `wt0` for the NetBird overlay mesh. `wg0` (the mitmproxy inspection tunnel) is untouched. Seeds `netbird_enrollment_key` and `netbird_api_token` in `~/.config/sandcat/settings.json`. +- `--netbird-server` - NetBird management server mode (requires `--netbird`): + `cloud` | `new` | ``. In non-interactive flag mode, `new` defaults + management URL to `http://localhost:33073`. - `--1password` - Deprecated alias for `--secret-provider 1password` - `--features` - Comma-separated optional non-provider features: `tui` (proxy console mode; prefer `--proxy tui`), `no-gitignore` (skip appending the `# Sandcat` block to the project's `.gitignore`; equivalent to `SANDCAT_GITIGNORE=false`), `no-rtk` (skip RTK installation; equivalent to `SANDCAT_RTK=false`), `strict-network` (project settings get network presets for the selected stacks instead of the allow-all-GET wildcard; equivalent to `SANDCAT_STRICT_NETWORK=true`) - `--name` - Project name for Docker Compose (default: derived from directory name) @@ -287,8 +290,41 @@ commands read `netbird_api_token` from the same settings layers (project settings override user settings when non-empty). Environment variables `NB_SETUP_KEY` and `NB_API_TOKEN` override settings when set. -Optional: set `NB_MANAGEMENT_URL` when using a self-hosted NetBird server -(default: `https://api.netbird.io`). +### Management server + +Choose one management server mode during `sandcat init --netbird`: + +- `cloud` — uses NetBird Cloud (`https://api.netbird.io`). +- Existing self-hosted URL — pass `--netbird-server `. +- New self-hosted template — pass `--netbird-server new`; template path: + `~/.config/sandcat/netbird-server/`. In flag mode this is non-interactive, + does not prompt for URL, and persists `http://localhost:33073` as the + management URL. + +Canonical non-interactive invocations: + +```bash +# Cloud +sandcat init --agent claude --ide vscode --netbird --netbird-server cloud --name myproject + +# Existing self-hosted management server +sandcat init --agent claude --ide vscode --netbird --netbird-server https://netbird.example.com --name myproject + +# New self-hosted template +sandcat init --agent claude --ide vscode --netbird --netbird-server new --name myproject +``` + +For `new`, the generated directory is a starter skeleton. Review/update config +files first, then start the stack: + +```bash +cd ~/.config/sandcat/netbird-server +# required before first run: +# - netbird-server.env +# - management.json +# - turnserver.conf +docker compose --env-file netbird-server.env up -d +``` ### Runtime control diff --git a/cli/lib/composefile.bash b/cli/lib/composefile.bash index 1d9d475d..ba4a149f 100644 --- a/cli/lib/composefile.bash +++ b/cli/lib/composefile.bash @@ -625,14 +625,26 @@ apply_netbird_build_args() { # container as a NetBird peer and start the daemon on wt0. # Args: # $1 - Path to compose-proxy.yml +# $2 - Optional NetBird management server URL enable_netbird() { require yq local compose_file=$1 + local netbird_management_url=${2:-} local already_set - already_set=$(yq '[.services."wg-client".environment[] | select(. == "NB_SETUP_KEY")] | length' "$compose_file") + already_set=$(yq '[(.services."wg-client".environment // [])[] | select(. == "NB_SETUP_KEY")] | length' "$compose_file") if [[ "$already_set" -eq 0 ]]; then yq -i '.services."wg-client".environment += ["NB_SETUP_KEY"]' "$compose_file" fi + + if [[ -n "$netbird_management_url" ]]; then + netbird_management_url="$netbird_management_url" \ + yq -i ' + .services."wg-client".environment = ( + (.services."wg-client".environment // []) + | map(select(test("^NB_MANAGEMENT_URL=") | not)) + ) + ["NB_MANAGEMENT_URL=" + env(netbird_management_url)] + ' "$compose_file" + fi } diff --git a/cli/lib/netbird.bash b/cli/lib/netbird.bash index 2b94b75c..0261954c 100644 --- a/cli/lib/netbird.bash +++ b/cli/lib/netbird.bash @@ -4,6 +4,8 @@ source "${BASH_SOURCE%/*}/constants.bash" # shellcheck source=path.bash source "${BASH_SOURCE%/*}/path.bash" +# shellcheck source=logging.bash +source "${BASH_SOURCE%/*}/logging.bash" # shellcheck source=require.bash source "${BASH_SOURCE%/*}/require.bash" @@ -48,6 +50,54 @@ export_netbird_compose_env() { fi } +# Export NB_MANAGEMENT_URL from settings when not already set in environment. +export_netbird_management_url() { + [[ -n "${NB_MANAGEMENT_URL:-}" ]] && return 0 + + local management_url + management_url=$(netbird_read_setting netbird_management_url) + if [[ -n "$management_url" ]]; then + export NB_MANAGEMENT_URL="$management_url" + fi +} + +# Creates ~/.config/sandcat/netbird-server from template when missing. +# Idempotent: if destination exists, logs and skips. +provision_netbird_server_template() { + local destination_dir template_dir + destination_dir="$(sct_home)/netbird-server" + template_dir="$SCT_TEMPLATEDIR/netbird-server" + local required_file + local -a required_files=( + docker-compose.yml + netbird-server.env + management.json + turnserver.conf + README.md + ) + + if [[ -e "$destination_dir" ]]; then + echo "NetBird server template already exists at $destination_dir; skipping." | info + return 0 + fi + + if [[ ! -d "$template_dir" ]]; then + echo "Missing NetBird server template directory: $template_dir" | error + return 1 + fi + + for required_file in "${required_files[@]}"; do + if [[ ! -f "$template_dir/$required_file" ]]; then + echo "Incomplete NetBird server template: missing $required_file in $template_dir" | error + return 1 + fi + done + + mkdir -p "$destination_dir" + # Use rsync-style copy to include dotfiles (glob * skips them) + cp -R "$template_dir/." "$destination_dir/" +} + # Resolve NB_API_TOKEN from settings when unset. Env always wins. _ensure_netbird_api_token() { [[ -n "${NB_API_TOKEN:-}" ]] && return 0 @@ -80,6 +130,7 @@ netbird_api() { local body=${3:-} _ensure_netbird_api_token || return 1 + export_netbird_management_url local url="${NB_MANAGEMENT_URL:-https://api.netbird.io}${path}" local -a args=(-sS -f -X "$method" diff --git a/cli/libexec/attach/attach b/cli/libexec/attach/attach index ade3c415..f5643335 100755 --- a/cli/libexec/attach/attach +++ b/cli/libexec/attach/attach @@ -15,6 +15,7 @@ attach() { compose_file="$(find_compose_file)" export_netbird_compose_env + export_netbird_management_url if (($# == 0)) then diff --git a/cli/libexec/compose/compose b/cli/libexec/compose/compose index ccda32ef..ad476290 100755 --- a/cli/libexec/compose/compose +++ b/cli/libexec/compose/compose @@ -17,6 +17,7 @@ compose() { compose_file="$(find_compose_file)" export_netbird_compose_env + export_netbird_management_url exec docker compose -f "$compose_file" "$@" } diff --git a/cli/libexec/init/devcontainer b/cli/libexec/init/devcontainer index 7c593497..46f656dd 100755 --- a/cli/libexec/init/devcontainer +++ b/cli/libexec/init/devcontainer @@ -15,6 +15,7 @@ source "$SCT_LIBDIR/devcontainer.bash" # --project-path - Path to the project directory # --agent - The agent name (e.g., "claude") # --ide - The IDE name (e.g., "vscode", "jetbrains", "none") (optional) +# --netbird-management-url - NetBird management server URL for wg-client (optional) devcontainer() { local settings_file="" local project_path="" @@ -25,11 +26,12 @@ devcontainer() { local proxy_mode="web" local secret_provider="none" local netbird="false" + local netbird_management_url="" while [[ $# -gt 0 ]] do case $1 in - --settings-file|--project-path|--agent|--ide|--name|--stacks|--proxy|--secret-provider) + --settings-file|--project-path|--agent|--ide|--name|--stacks|--proxy|--secret-provider|--netbird-management-url) if [[ $# -lt 2 ]]; then echo "Option $1 requires a value" | error return 1 @@ -77,6 +79,10 @@ devcontainer() { netbird="true" shift 1 ;; + --netbird-management-url) + netbird_management_url="$2" + shift 2 + ;; *) echo "Unknown option: $1" | error return 1 @@ -130,7 +136,7 @@ devcontainer() { # shellcheck source=../../lib/composefile.bash source "$SCT_LIBDIR/composefile.bash" fi - enable_netbird "$devcontainer_dir/sandcat/compose-proxy.yml" + enable_netbird "$devcontainer_dir/sandcat/compose-proxy.yml" "$netbird_management_url" fi customize_compose_file "$rel_settings_file" "$compose_file" "$agent" "$ide" "$project_name" "$stacks" diff --git a/cli/libexec/init/init b/cli/libexec/init/init index c73bac70..5cdcb64c 100755 --- a/cli/libexec/init/init +++ b/cli/libexec/init/init @@ -17,6 +17,8 @@ source "$SCT_LIBDIR/stacks.bash" source "$SCT_LIBDIR/agents.bash" # shellcheck source=../../lib/gitignore.bash source "$SCT_LIBDIR/gitignore.bash" +# shellcheck source=../../lib/netbird.bash +source "$SCT_LIBDIR/netbird.bash" # Returns the user settings template path for a selected agent. # Args: @@ -161,11 +163,15 @@ init() { local features_csv="" local features_provided=false local netbird="false" + local netbird_server="" + local netbird_server_provided=false + local netbird_management_url="" + local provisioned_netbird_server="false" while [[ $# -gt 0 ]] do case $1 in - --name|--path|--agent|--ide|--stacks|--proxy|--features|--secret-provider|--sp) + --name|--path|--agent|--ide|--stacks|--proxy|--features|--secret-provider|--sp|--netbird-server) if [[ $# -lt 2 ]]; then echo "Option $1 requires a value" | error return 1 @@ -216,6 +222,11 @@ init() { netbird="true" shift 1 ;; + --netbird-server) + netbird_server="$2" + netbird_server_provided=true + shift 2 + ;; *) echo "Unknown option: $1" | error return 1 @@ -231,6 +242,21 @@ init() { secret_provider="1password" secret_provider_provided=true fi + if [[ "$netbird_server_provided" == "true" && "$netbird" != "true" ]]; then + echo "--netbird-server requires --netbird" | error + return 1 + fi + if [[ "$netbird_server_provided" == "true" ]]; then + case "$netbird_server" in + cloud|new|http://*|https://*) + : + ;; + *) + echo "Invalid NetBird server mode: $netbird_server (expected: cloud, new, or http(s)://... URL)" | error + return 1 + ;; + esac + fi if [[ -z "$project_path" ]] then @@ -402,12 +428,94 @@ init() { if [[ "$netbird" == "true" ]]; then local user_settings user_settings="$(sct_home)/settings.json" + persist_netbird_management_url() { + local selected_management_url=${1:-} + if [[ -f "$user_settings" ]]; then + netbird_management_url="$selected_management_url" yq -i -o json ' + .netbird_management_url = strenv(netbird_management_url) + ' "$user_settings" + fi + } if [[ -f "$user_settings" ]]; then yq -i -o json ' .netbird_enrollment_key = (.netbird_enrollment_key // "") | - .netbird_api_token = (.netbird_api_token // "") + .netbird_api_token = (.netbird_api_token // "") | + .netbird_management_url = (.netbird_management_url // "") ' "$user_settings" fi + + if [[ "$netbird_server_provided" == "true" ]]; then + case "$netbird_server" in + cloud) + netbird_management_url="" + ;; + new) + provision_netbird_server_template + provisioned_netbird_server="true" + netbird_management_url="http://localhost:33073" + persist_netbird_management_url "$netbird_management_url" + ;; + http://*|https://*) + netbird_management_url="$netbird_server" + persist_netbird_management_url "$netbird_management_url" + ;; + esac + else + local netbird_server_selection="" + while true; do + echo "NetBird management server [cloud]:" | info + echo " 1) cloud (api.netbird.io)" | info + echo " 2) self-hosted — I have a server running" | info + echo " 3) self-hosted — provision a new server from template" | info + netbird_server_selection=$(read_line ">") + case "$netbird_server_selection" in + ""|1|cloud) + netbird_server_selection="cloud" + break + ;; + 2|existing|self-hosted-existing|self-hosted\ existing\ URL|self-hosted\ —\ I\ have\ a\ server\ running) + netbird_server_selection="existing" + break + ;; + 3|new|self-hosted-new|self-hosted\ new\ template|self-hosted\ —\ provision\ a\ new\ server\ from\ template) + netbird_server_selection="new" + break + ;; + *) + echo "Invalid NetBird management server selection: $netbird_server_selection (expected: cloud, existing, or new)" | error + ;; + esac + done + case "$netbird_server_selection" in + cloud) + netbird_management_url="" + ;; + existing) + while true; do + netbird_management_url=$(read_line "Management URL:") + if [[ -z "$netbird_management_url" ]]; then + echo "URL is required" | error + continue + fi + persist_netbird_management_url "$netbird_management_url" + break + done + ;; + new) + provision_netbird_server_template + provisioned_netbird_server="true" + echo " Self-hosted template: ~/.config/sandcat/netbird-server" | info + echo " Template is a starter skeleton; configure before first start:" | info + echo " netbird-server.env, management.json, turnserver.conf" | info + echo " Start hint: cd ~/.config/sandcat/netbird-server && docker compose --env-file netbird-server.env up -d" | info + netbird_management_url=$(read_line "Management URL [http://localhost:33073]:") + netbird_management_url="${netbird_management_url:-http://localhost:33073}" + persist_netbird_management_url "$netbird_management_url" + ;; + esac + fi + + persist_netbird_management_url "$netbird_management_url" fi local settings_args=() @@ -426,6 +534,9 @@ init() { --secret-provider "$secret_provider" ) export SANDCAT_RTK="$rtk_enabled" + if [[ -n "$netbird_management_url" ]]; then + devcontainer_args+=(--netbird-management-url "$netbird_management_url") + fi if [[ "$netbird" == "true" ]]; then devcontainer "${devcontainer_args[@]}" --netbird else @@ -534,9 +645,19 @@ init() { if [[ "$netbird" == "true" ]]; then echo "" >&2 echo " NetBird setup:" | info + if [[ -n "$netbird_management_url" ]]; then + echo " Management server: $netbird_management_url" | info + else + echo " Management server: cloud (https://api.netbird.io)" | info + fi echo " Add keys to ~/.config/sandcat/settings.json:" | info echo " \"netbird_enrollment_key\": \"\" (wg-client enrollment)" | info echo " \"netbird_api_token\": \"\" (sandcat netbird commands)" | info + if [[ "$provisioned_netbird_server" == "true" ]]; then + echo " Self-hosted server template: ~/.config/sandcat/netbird-server/" | info + echo " Start it: docker compose -f ~/.config/sandcat/netbird-server/docker-compose.yml up -d" | info + echo " (Future: sandcat netbird server start)" | info + fi fi echo " Then run: sandcat run, or reopen the project using the dev container" | info } diff --git a/cli/libexec/netbird/netbird b/cli/libexec/netbird/netbird index 8ffe8341..f4046740 100755 --- a/cli/libexec/netbird/netbird +++ b/cli/libexec/netbird/netbird @@ -42,7 +42,14 @@ cmd_peer() { local peer_id="" while [[ $# -gt 0 ]]; do case $1 in - --peer-id) peer_id="$2"; shift 2 ;; + --peer-id) + if [[ $# -lt 2 || "$2" == --* ]]; then + echo "Option --peer-id requires a value" | error + return 1 + fi + peer_id="$2" + shift 2 + ;; *) echo "Unknown option: $1" | error; return 1 ;; esac done @@ -66,8 +73,22 @@ cmd_route() { local network="" peer_id="" while [[ $# -gt 0 ]]; do case $1 in - --network) network="$2"; shift 2 ;; - --peer-id) peer_id="$2"; shift 2 ;; + --network) + if [[ $# -lt 2 || "$2" == --* ]]; then + echo "Option --network requires a value" | error + return 1 + fi + network="$2" + shift 2 + ;; + --peer-id) + if [[ $# -lt 2 || "$2" == --* ]]; then + echo "Option --peer-id requires a value" | error + return 1 + fi + peer_id="$2" + shift 2 + ;; *) echo "Unknown option: $1" | error; return 1 ;; esac done @@ -93,7 +114,14 @@ cmd_route() { local route_id="" while [[ $# -gt 0 ]]; do case $1 in - --route-id) route_id="$2"; shift 2 ;; + --route-id) + if [[ $# -lt 2 || "$2" == --* ]]; then + echo "Option --route-id requires a value" | error + return 1 + fi + route_id="$2" + shift 2 + ;; *) echo "Unknown option: $1" | error; return 1 ;; esac done diff --git a/cli/libexec/restart/restart b/cli/libexec/restart/restart index a0848e2a..f1e0a523 100755 --- a/cli/libexec/restart/restart +++ b/cli/libexec/restart/restart @@ -19,6 +19,7 @@ restart() { compose_file="$(find_compose_file)" export_netbird_compose_env + export_netbird_management_url local proxy_status proxy_status=$(docker compose -f "$compose_file" ps mitmproxy --status running --quiet 2>/dev/null) || true diff --git a/cli/libexec/run/run b/cli/libexec/run/run index d60d970a..3c981891 100755 --- a/cli/libexec/run/run +++ b/cli/libexec/run/run @@ -41,6 +41,7 @@ run() { ensure_shared_cache_volumes "$compose_file" export_netbird_compose_env + export_netbird_management_url local rc=0 docker compose -f "$compose_file" run --rm "${run_opts[@]+"${run_opts[@]}"}" agent "${1-bash}" "${@:2}" || rc=$? diff --git a/cli/templates/netbird-server/README.md b/cli/templates/netbird-server/README.md new file mode 100644 index 00000000..5945ca03 --- /dev/null +++ b/cli/templates/netbird-server/README.md @@ -0,0 +1,16 @@ +# NetBird self-hosted template + +This directory is provisioned by `sandcat init --netbird --netbird-server new`. +It is a starter skeleton, not a ready-to-run production config. + +Before first startup, review and adapt: + +- `netbird-server.env` (image/version pins and required endpoint/auth env values) +- `management.json` (mounted by compose into management service) +- `turnserver.conf` (mounted by compose into coturn; set credentials/realm) + +Start with: + +```bash +docker compose --env-file netbird-server.env up -d +``` diff --git a/cli/templates/netbird-server/docker-compose.yml b/cli/templates/netbird-server/docker-compose.yml new file mode 100644 index 00000000..8265b867 --- /dev/null +++ b/cli/templates/netbird-server/docker-compose.yml @@ -0,0 +1,117 @@ +x-default: &default + restart: "unless-stopped" + logging: + driver: "json-file" + options: + max-size: "500m" + max-file: "2" + +services: + # UI dashboard + dashboard: + <<: *default + image: netbirdio/dashboard:${NETBIRD_SERVER_VERSION} + ports: + - ${NETBIRD_DASHBOARD_HTTP_PORT:-80}:80 + - ${NETBIRD_DASHBOARD_HTTPS_PORT:-443}:443 + environment: + # Endpoints + - NETBIRD_MGMT_API_ENDPOINT=${NETBIRD_MGMT_API_ENDPOINT} + - NETBIRD_MGMT_GRPC_API_ENDPOINT=${NETBIRD_MGMT_GRPC_API_ENDPOINT:-${NETBIRD_MGMT_API_ENDPOINT}} + # OIDC + - AUTH_AUDIENCE=${NETBIRD_DASH_AUTH_AUDIENCE} + - AUTH_CLIENT_ID=${NETBIRD_AUTH_CLIENT_ID} + - AUTH_CLIENT_SECRET=${NETBIRD_AUTH_CLIENT_SECRET} + - AUTH_AUTHORITY=${NETBIRD_AUTH_AUTHORITY} + - USE_AUTH0=${NETBIRD_USE_AUTH0} + - AUTH_SUPPORTED_SCOPES=${NETBIRD_AUTH_SUPPORTED_SCOPES} + - AUTH_REDIRECT_URI=${NETBIRD_AUTH_REDIRECT_URI} + - AUTH_SILENT_REDIRECT_URI=${NETBIRD_AUTH_SILENT_REDIRECT_URI} + - NETBIRD_TOKEN_SOURCE=${NETBIRD_TOKEN_SOURCE} + # SSL + - NGINX_SSL_PORT=443 + # Letsencrypt + - LETSENCRYPT_DOMAIN=${NETBIRD_LETSENCRYPT_DOMAIN} + - LETSENCRYPT_EMAIL=${NETBIRD_LETSENCRYPT_EMAIL} + volumes: + - letsencrypt:/etc/letsencrypt/ + + # Signal + signal: + <<: *default + image: netbirdio/signal:${NETBIRD_SERVER_VERSION} + volumes: + - signal:/var/lib/netbird + - letsencrypt:/etc/letsencrypt:ro + ports: + - ${NETBIRD_SIGNAL_PORT}:80 + # # port and command for Let's Encrypt validation + # - 443:443 + # command: ["--letsencrypt-domain", "${NETBIRD_LETSENCRYPT_DOMAIN}", "--log-file", "console"] + command: + - "--cert-file" + - "${NETBIRD_MGMT_API_CERT_FILE}" + - "--cert-key" + - "${NETBIRD_MGMT_API_CERT_KEY_FILE}" + - "--log-file" + - "console" + - "--port" + - "80" + + # Relay + relay: + <<: *default + image: netbirdio/relay:${NETBIRD_SERVER_VERSION} + environment: + - NB_LOG_LEVEL=info + - NB_LISTEN_ADDRESS=:${NETBIRD_RELAY_PORT} + - NB_EXPOSED_ADDRESS=${NETBIRD_RELAY_ENDPOINT} + # todo: change to a secure secret + - NB_AUTH_SECRET=${NETBIRD_RELAY_AUTH_SECRET} + ports: + - ${NETBIRD_RELAY_PORT}:${NETBIRD_RELAY_PORT} + + # Management + management: + <<: *default + image: netbirdio/management:${NETBIRD_SERVER_VERSION} + volumes: + - management:/var/lib/netbird + - letsencrypt:/etc/letsencrypt:ro + - ./management.json:/etc/netbird/management.json + ports: + - ${NETBIRD_MGMT_API_PORT}:443 # API port + # # command for Let's Encrypt validation without dashboard container + # command: ["--letsencrypt-domain", "${NETBIRD_LETSENCRYPT_DOMAIN}", "--log-file", "console"] + command: + - "--port" + - "443" + - "--log-file" + - "console" + - "--log-level" + - "info" + - "--disable-anonymous-metrics=${NETBIRD_DISABLE_ANONYMOUS_METRICS}" + - "--single-account-mode-domain=${NETBIRD_MGMT_SINGLE_ACCOUNT_MODE_DOMAIN}" + - "--dns-domain=${NETBIRD_MGMT_DNS_DOMAIN}" + environment: + - NETBIRD_STORE_ENGINE_POSTGRES_DSN=${NETBIRD_STORE_ENGINE_POSTGRES_DSN} + - NETBIRD_STORE_ENGINE_MYSQL_DSN=${NETBIRD_STORE_ENGINE_MYSQL_DSN} + + # Coturn + coturn: + <<: *default + image: coturn/coturn:${COTURN_TAG:-4.6.2} + # domainname: ${TURN_DOMAIN} # only needed when TLS is enabled + volumes: + - ./turnserver.conf:/etc/turnserver.conf:ro + # - ./privkey.pem:/etc/coturn/private/privkey.pem:ro + # - ./cert.pem:/etc/coturn/certs/cert.pem:ro + network_mode: host + command: + - -c + - /etc/turnserver.conf + +volumes: + management: + signal: + letsencrypt: diff --git a/cli/templates/netbird-server/management.json b/cli/templates/netbird-server/management.json new file mode 100644 index 00000000..0967ef42 --- /dev/null +++ b/cli/templates/netbird-server/management.json @@ -0,0 +1 @@ +{} diff --git a/cli/templates/netbird-server/netbird-server.env b/cli/templates/netbird-server/netbird-server.env new file mode 100644 index 00000000..4316bcda --- /dev/null +++ b/cli/templates/netbird-server/netbird-server.env @@ -0,0 +1,12 @@ +# Single source of truth for pinned NetBird self-hosted service images. +# +# Consumed by: +# - cli/templates/netbird-server/docker-compose.yml +# +# Format note: simple KEY=value lines only (no quotes, no spaces around `=`) so +# this file can be consumed directly by shell tooling and docker compose env loading. +NETBIRD_SERVER_VERSION=0.28.9 +NETBIRD_DASHBOARD_HTTP_PORT=80 +NETBIRD_DASHBOARD_HTTPS_PORT=443 +# Optional override for a dedicated gRPC endpoint; defaults to NETBIRD_MGMT_API_ENDPOINT. +NETBIRD_MGMT_GRPC_API_ENDPOINT= diff --git a/cli/templates/netbird-server/turnserver.conf b/cli/templates/netbird-server/turnserver.conf new file mode 100644 index 00000000..72a8a1f2 --- /dev/null +++ b/cli/templates/netbird-server/turnserver.conf @@ -0,0 +1,8 @@ +# Starter coturn config for local development. +# Adjust credentials/realm before exposing this service outside localhost. +listening-port=3478 +fingerprint +lt-cred-mech +realm=netbird.local +user=netbird:change-me +no-cli diff --git a/cli/test/composefile/netbird.bats b/cli/test/composefile/netbird.bats index fad714a8..837b9d0a 100644 --- a/cli/test/composefile/netbird.bats +++ b/cli/test/composefile/netbird.bats @@ -35,6 +35,41 @@ teardown() { assert_output "1" } +@test "enable_netbird adds NB_MANAGEMENT_URL when provided" { + local management_url="https://netbird.internal" + enable_netbird "$COMPOSE_FILE" "$management_url" + + run yq -r '.services."wg-client".environment[] | select(test("^NB_MANAGEMENT_URL="))' "$COMPOSE_FILE" + assert_output "NB_MANAGEMENT_URL=$management_url" +} + +@test "enable_netbird with management URL is idempotent" { + local management_url="https://netbird.internal" + enable_netbird "$COMPOSE_FILE" "$management_url" + enable_netbird "$COMPOSE_FILE" "$management_url" + + run yq '[.services."wg-client".environment[] | select(. == "NB_SETUP_KEY")] | length' "$COMPOSE_FILE" + assert_output "1" + + run yq '[.services."wg-client".environment[] | select(test("^NB_MANAGEMENT_URL="))] | length' "$COMPOSE_FILE" + assert_output "1" +} + +@test "enable_netbird updates existing NB_MANAGEMENT_URL when provided" { + enable_netbird "$COMPOSE_FILE" "https://old.example.com" + enable_netbird "$COMPOSE_FILE" "https://new.example.com" + + run yq -r '.services."wg-client".environment[] | select(test("^NB_MANAGEMENT_URL="))' "$COMPOSE_FILE" + assert_output "NB_MANAGEMENT_URL=https://new.example.com" +} + +@test "enable_netbird with empty management URL does not add NB_MANAGEMENT_URL" { + enable_netbird "$COMPOSE_FILE" "" + + run yq '[.services."wg-client".environment[] | select(test("^NB_MANAGEMENT_URL="))] | length' "$COMPOSE_FILE" + assert_output "0" +} + @test "apply_netbird_build_args injects version and checksum build args" { apply_netbird_build_args "$COMPOSE_FILE" diff --git a/cli/test/init/init.bats b/cli/test/init/init.bats index ff126db9..43255598 100644 --- a/cli/test/init/init.bats +++ b/cli/test/init/init.bats @@ -337,7 +337,7 @@ EOF stub devcontainer \ "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird : :" - run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird --netbird-server cloud assert_success } @@ -345,12 +345,141 @@ EOF stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" stub devcontainer ":" - run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird --netbird-server cloud assert_success run yq '.netbird_enrollment_key' "$SCT_HOME_DIR/settings.json" assert_output '""' run yq '.netbird_api_token' "$SCT_HOME_DIR/settings.json" assert_output '""' + run yq '.netbird_management_url' "$SCT_HOME_DIR/settings.json" + assert_output '""' +} + +@test "init rejects --netbird-server without --netbird" { + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird-server cloud + assert_failure + assert_output --partial "--netbird-server requires --netbird" +} + +@test "init rejects invalid --netbird-server value" { + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird --netbird-server invalid + assert_failure + assert_output --partial "Invalid NetBird server mode: invalid" +} + +@test "init --netbird-server URL persists management server immediately" { + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird-management-url https://management.example.com --netbird : :" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird --netbird-server https://management.example.com + assert_success + run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" + assert_output "https://management.example.com" +} + +@test "init forwards selected netbird management URL to devcontainer args" { + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird-management-url https://selected.example.com --netbird : :" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird --netbird-server https://selected.example.com + assert_success +} + +@test "init --netbird-server cloud uses cloud summary and clears management URL" { + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird : :" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird --netbird-server cloud + assert_success + assert_output --partial "Management server: cloud (https://api.netbird.io)" + run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" + assert_output "" +} + +@test "init --netbird-server new provisions template and uses default URL" { + unset -f provision_netbird_server_template + unset -f read_line + + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird-management-url http://localhost:33073 --netbird : :" + stub provision_netbird_server_template ":" + read_line() { + echo "read_line should not be called for --netbird-server new" >&2 + return 88 + } + export -f read_line + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird --netbird-server new + assert_success + assert_output --partial "Management server: http://localhost:33073" + assert_output --partial "Self-hosted server template: ~/.config/sandcat/netbird-server/" + assert_output --partial "Start it: docker compose -f ~/.config/sandcat/netbird-server/docker-compose.yml up -d" + assert_output --partial "(Future: sandcat netbird server start)" + run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" + assert_output "http://localhost:33073" +} + +@test "init interactive netbird server existing re-prompts for non-empty URL" { + unset -f read_line + + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird-management-url https://management.example.com --netbird : :" + stub read_line \ + "'>' : echo '2'" \ + "'Management URL:' : echo ''" \ + "'Management URL:' : echo 'https://management.example.com'" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird + assert_success + assert_output --partial "URL is required" + run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" + assert_output "https://management.example.com" +} + +@test "init interactive netbird existing accepts non-empty URL without format restriction" { + unset -f read_line + + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird-management-url management.example.com --netbird : :" + stub read_line \ + "'>' : echo existing" \ + "'Management URL:' : echo 'management.example.com'" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird + assert_success + run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" + assert_output "management.example.com" +} + +@test "init interactive netbird server new provisions template and uses default URL" { + unset -f read_line + unset -f provision_netbird_server_template + + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird-management-url http://localhost:33073 --netbird : :" + stub provision_netbird_server_template ":" + stub read_line \ + "'>' : echo 3" \ + "'Management URL [http://localhost:33073]:' : echo ''" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird + assert_success + assert_output --partial "NetBird management server [cloud]:" + assert_output --partial "1) cloud (api.netbird.io)" + assert_output --partial "2) self-hosted — I have a server running" + assert_output --partial "3) self-hosted — provision a new server from template" + assert_output --partial "Self-hosted server template: ~/.config/sandcat/netbird-server/" + assert_output --partial "Start it: docker compose -f ~/.config/sandcat/netbird-server/docker-compose.yml up -d" + assert_output --partial "(Future: sandcat netbird server start)" + run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" + assert_output "http://localhost:33073" } @test "init interactive flow (devcontainer mode)" { diff --git a/cli/test/netbird/netbird.bats b/cli/test/netbird/netbird.bats index b3e12b4e..b6937f21 100644 --- a/cli/test/netbird/netbird.bats +++ b/cli/test/netbird/netbird.bats @@ -34,6 +34,12 @@ teardown() { assert_output --partial "peer-id" } +@test "netbird peer remove errors when --peer-id has no value" { + run bash "$NETBIRD_CMD" peer remove --peer-id + assert_failure + assert_output --partial "Option --peer-id requires a value" +} + @test "netbird peer remove calls netbird_peer_remove" { stub curl \ "-sS -f -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers/abc123 : :" @@ -47,6 +53,18 @@ teardown() { assert_output --partial "peer-id" } +@test "netbird route add errors when --network has no value" { + run bash "$NETBIRD_CMD" route add --network + assert_failure + assert_output --partial "Option --network requires a value" +} + +@test "netbird route add errors when --peer-id has no value" { + run bash "$NETBIRD_CMD" route add --network 10.8.0.0/24 --peer-id + assert_failure + assert_output --partial "Option --peer-id requires a value" +} + @test "netbird route add rejects non-CIDR network" { run bash "$NETBIRD_CMD" route add --network wd0 --peer-id abc123 assert_failure @@ -67,6 +85,12 @@ teardown() { assert_output --partial "route-id" } +@test "netbird route remove errors when --route-id has no value" { + run bash "$NETBIRD_CMD" route remove --route-id + assert_failure + assert_output --partial "Option --route-id requires a value" +} + @test "netbird route remove calls netbird_route_remove" { stub curl \ "-sS -f -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/routes/route1 : :" diff --git a/cli/test/netbird/netbird_settings.bats b/cli/test/netbird/netbird_settings.bats index 80c5863c..fa2aeff8 100644 --- a/cli/test/netbird/netbird_settings.bats +++ b/cli/test/netbird/netbird_settings.bats @@ -63,6 +63,82 @@ teardown() { [[ "$NB_SETUP_KEY" == "from-env" ]] } +@test "export_netbird_management_url exports management URL from user settings" { + echo '{"netbird_management_url": "https://management.example.com"}' > "$HOME/.config/sandcat/settings.json" + unset NB_MANAGEMENT_URL + + export_netbird_management_url + + [[ "$NB_MANAGEMENT_URL" == "https://management.example.com" ]] +} + +@test "export_netbird_management_url does not override existing NB_MANAGEMENT_URL" { + echo '{"netbird_management_url": "https://from-settings.example.com"}' > "$HOME/.config/sandcat/settings.json" + export NB_MANAGEMENT_URL="https://from-env.example.com" + + export_netbird_management_url + + [[ "$NB_MANAGEMENT_URL" == "https://from-env.example.com" ]] +} + +@test "provision_netbird_server_template copies template and skips when already provisioned" { + export SCT_TEMPLATEDIR="$BATS_TEST_TMPDIR/templates" + mkdir -p "$SCT_TEMPLATEDIR/netbird-server" + echo "services: {}" > "$SCT_TEMPLATEDIR/netbird-server/docker-compose.yml" + echo "NETBIRD_SERVER_VERSION=0.1.0" > "$SCT_TEMPLATEDIR/netbird-server/netbird-server.env" + echo "{}" > "$SCT_TEMPLATEDIR/netbird-server/management.json" + echo "listening-port=3478" > "$SCT_TEMPLATEDIR/netbird-server/turnserver.conf" + echo "# template" > "$SCT_TEMPLATEDIR/netbird-server/README.md" + + provision_netbird_server_template + [[ -d "$HOME/.config/sandcat/netbird-server" ]] + [[ -f "$HOME/.config/sandcat/netbird-server/docker-compose.yml" ]] + [[ "$(cat "$HOME/.config/sandcat/netbird-server/docker-compose.yml")" == "services: {}" ]] + + run provision_netbird_server_template + assert_success + assert_output --partial "skipping" +} + +@test "provision_netbird_server_template skips when destination exists as file" { + export SCT_TEMPLATEDIR="$BATS_TEST_TMPDIR/templates" + mkdir -p "$SCT_TEMPLATEDIR/netbird-server" + echo "services: {}" > "$SCT_TEMPLATEDIR/netbird-server/docker-compose.yml" + echo "NETBIRD_SERVER_VERSION=0.1.0" > "$SCT_TEMPLATEDIR/netbird-server/netbird-server.env" + echo "{}" > "$SCT_TEMPLATEDIR/netbird-server/management.json" + echo "listening-port=3478" > "$SCT_TEMPLATEDIR/netbird-server/turnserver.conf" + echo "# template" > "$SCT_TEMPLATEDIR/netbird-server/README.md" + mkdir -p "$HOME/.config/sandcat" + echo "existing-file" > "$HOME/.config/sandcat/netbird-server" + + run provision_netbird_server_template + assert_success + assert_output --partial "skipping" + [[ "$(cat "$HOME/.config/sandcat/netbird-server")" == "existing-file" ]] +} + +@test "provision_netbird_server_template fails when template directory is missing" { + export SCT_TEMPLATEDIR="$BATS_TEST_TMPDIR/templates" + mkdir -p "$SCT_TEMPLATEDIR" + + run provision_netbird_server_template + assert_failure + assert_output --partial "Missing NetBird server template directory" +} + +@test "provision_netbird_server_template fails when required companion is missing" { + export SCT_TEMPLATEDIR="$BATS_TEST_TMPDIR/templates" + mkdir -p "$SCT_TEMPLATEDIR/netbird-server" + echo "services: {}" > "$SCT_TEMPLATEDIR/netbird-server/docker-compose.yml" + echo "NETBIRD_SERVER_VERSION=0.1.0" > "$SCT_TEMPLATEDIR/netbird-server/netbird-server.env" + echo "{}" > "$SCT_TEMPLATEDIR/netbird-server/management.json" + echo "# template" > "$SCT_TEMPLATEDIR/netbird-server/README.md" + + run provision_netbird_server_template + assert_failure + assert_output --partial "missing turnserver.conf" +} + @test "netbird_api reads netbird_api_token from user settings" { echo '{"netbird_api_token": "settings-token"}' > "$HOME/.config/sandcat/settings.json" unset NB_API_TOKEN @@ -75,6 +151,18 @@ teardown() { assert_success } +@test "netbird_api uses settings netbird_management_url when env is unset" { + echo '{"netbird_api_token": "settings-token", "netbird_management_url": "https://management.settings.example.com"}' > "$HOME/.config/sandcat/settings.json" + unset NB_API_TOKEN + unset NB_MANAGEMENT_URL + + stub curl \ + "-sS -f -X GET -H 'Authorization: Token settings-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://management.settings.example.com/api/peers : echo '[]'" + + run netbird_api "GET" "/api/peers" + assert_success +} + @test "netbird_api prefers NB_API_TOKEN env over settings" { echo '{"netbird_api_token": "settings-token"}' > "$HOME/.config/sandcat/settings.json" export NB_API_TOKEN="env-token" diff --git a/cli/test/restart/restart.bats b/cli/test/restart/restart.bats index 068c7ff7..fe22deed 100644 --- a/cli/test/restart/restart.bats +++ b/cli/test/restart/restart.bats @@ -57,3 +57,17 @@ teardown() { assert_success assert_output --partial "not running" } + +@test "restart exports management URL from settings when env is unset" { + export HOME="$BATS_TEST_TMPDIR/home" + mkdir -p "$HOME/.config/sandcat" + echo '{"netbird_management_url": "https://management.settings.example.com"}' > "$HOME/.config/sandcat/settings.json" + unset NB_MANAGEMENT_URL + + stub docker \ + "compose -f $COMPOSE_FILE ps mitmproxy --status running --quiet : :" + + cd "$BATS_TEST_TMPDIR" + restart + [[ "$NB_MANAGEMENT_URL" == "https://management.settings.example.com" ]] +} From feea4159ee27643dee94fabb029b4a02c840f01b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Sat, 20 Jun 2026 06:20:22 +0000 Subject: [PATCH 014/138] this is kind of working --- .gitignore | 3 +- cli/README.md | 97 +++++++++-- cli/core | 0 cli/lib/composefile.bash | 29 +++- cli/lib/netbird.bash | 159 +++++++++++++++++- cli/libexec/init/init | 58 +++++-- .../sandcat/scripts/wg-client-init.sh | 157 ++++++++++++++++- cli/templates/netbird-server/README.md | 110 +++++++++++- cli/templates/netbird-server/config.yaml | 25 +++ cli/templates/netbird-server/dashboard.env | 14 ++ .../netbird-server/docker-compose.yml | 134 +++------------ cli/templates/netbird-server/management.json | 1 - .../netbird-server/netbird-server.env | 24 +-- cli/templates/netbird-server/turnserver.conf | 8 - cli/test/composefile/netbird.bats | 41 +++++ cli/test/init/init.bats | 95 +++++++---- cli/test/netbird/netbird_settings.bats | 88 +++++++++- cli/test/wg-client/netbird_daemon.bats | 56 ++++++ 18 files changed, 886 insertions(+), 213 deletions(-) create mode 100644 cli/core create mode 100644 cli/templates/netbird-server/config.yaml create mode 100644 cli/templates/netbird-server/dashboard.env delete mode 100644 cli/templates/netbird-server/management.json delete mode 100644 cli/templates/netbird-server/turnserver.conf diff --git a/.gitignore b/.gitignore index bec3c314..3c979ee4 100644 --- a/.gitignore +++ b/.gitignore @@ -2,4 +2,5 @@ __pycache__/ .pytest_cache/ .sandcat/settings.local.json .devcontainer -.orca \ No newline at end of file +.orca +.worktrees/ \ No newline at end of file diff --git a/cli/README.md b/cli/README.md index 79c0fbbd..14791f14 100644 --- a/cli/README.md +++ b/cli/README.md @@ -28,8 +28,9 @@ Options: inspection tunnel) is untouched. Seeds `netbird_enrollment_key` and `netbird_api_token` in `~/.config/sandcat/settings.json`. - `--netbird-server` - NetBird management server mode (requires `--netbird`): - `cloud` | `new` | ``. In non-interactive flag mode, `new` defaults - management URL to `http://localhost:33073`. + `cloud` | `new` | `quickstart` | ``. `new` provisions a local + localhost template; `quickstart` prints the official NetBird install command for + a VM with a public domain. - `--1password` - Deprecated alias for `--secret-provider 1password` - `--features` - Comma-separated optional non-provider features: `tui` (proxy console mode; prefer `--proxy tui`), `no-gitignore` (skip appending the `# Sandcat` block to the project's `.gitignore`; equivalent to `SANDCAT_GITIGNORE=false`), `no-rtk` (skip RTK installation; equivalent to `SANDCAT_RTK=false`), `strict-network` (project settings get network presets for the selected stacks instead of the allow-all-GET wildcard; equivalent to `SANDCAT_STRICT_NETWORK=true`) - `--name` - Project name for Docker Compose (default: derived from directory name) @@ -261,6 +262,8 @@ NetBird uses **two separate credentials**. Both go in `~/.config/sandcat/setting |-------------|----------|-----------------| | `netbird_enrollment_key` | Enrolling `wg-client` as a mesh peer (`NB_SETUP_KEY`) | NetBird dashboard → **Setup Keys** | | `netbird_api_token` | `sandcat netbird` CLI commands on your host | NetBird dashboard → **API Keys** (Personal Access Token) | +| `netbird_management_url` | Host-side management API (`sandcat netbird`, browser) | `http://localhost:33073` for local template | +| `netbird_enrollment_management_url` | wg-client enrollment URL (container cannot use `localhost`) | See [local self-hosted](#local-self-hosted-sandcat-template) below | **Before `sandcat netbird status` works**, you must complete steps 1–4 below. Container enrollment (`netbird_enrollment_key`) is separate from host CLI control @@ -296,10 +299,11 @@ Choose one management server mode during `sandcat init --netbird`: - `cloud` — uses NetBird Cloud (`https://api.netbird.io`). - Existing self-hosted URL — pass `--netbird-server `. -- New self-hosted template — pass `--netbird-server new`; template path: - `~/.config/sandcat/netbird-server/`. In flag mode this is non-interactive, - does not prompt for URL, and persists `http://localhost:33073` as the - management URL. +- **Local** — `--netbird-server new` provisions a localhost template to + `~/.config/sandcat/netbird-server/` (dashboard on **http://localhost:8080**, + management API on **http://localhost:33073**). +- **Remote (VM + domain)** — `--netbird-server quickstart` prints the official + NetBird install command; you run it yourself, then point sandcat at your server URL. Canonical non-interactive invocations: @@ -310,22 +314,89 @@ sandcat init --agent claude --ide vscode --netbird --netbird-server cloud --name # Existing self-hosted management server sandcat init --agent claude --ide vscode --netbird --netbird-server https://netbird.example.com --name myproject -# New self-hosted template +# Local self-hosted (provisions localhost template) sandcat init --agent claude --ide vscode --netbird --netbird-server new --name myproject + +# Remote self-hosted (prints quickstart install command) +sandcat init --agent claude --ide vscode --netbird --netbird-server quickstart --name myproject +sandcat init --agent claude --ide vscode --netbird --netbird-server https://netbird.example.com --name myproject ``` -For `new`, the generated directory is a starter skeleton. Review/update config -files first, then start the stack: +### Local self-hosted (sandcat template) + +For development on your machine without a public domain: ```bash +sandcat init --netbird --netbird-server new ... cd ~/.config/sandcat/netbird-server -# required before first run: -# - netbird-server.env -# - management.json -# - turnserver.conf docker compose --env-file netbird-server.env up -d ``` +1. Bootstrap admin: `POST http://localhost:33073/api/setup` (see + `~/.config/sandcat/netbird-server/README.md`). +2. Open **http://localhost:8080** and sign in. +3. Create setup key + API token in the dashboard. +4. Set credentials in `~/.config/sandcat/settings.json` (or project + `.sandcat/settings.local.json`): + +```json +{ + "netbird_management_url": "http://localhost:33073", + "netbird_enrollment_management_url": "http://:33073", + "netbird_enrollment_key": "", + "netbird_api_token": "" +} +``` + +**Finding the Docker host IP** (wg-client cannot use `localhost`): + +```bash +# Colima +colima status -j | jq -r '.network.gateway_address' + +# Docker Desktop (macOS) — often 192.168.65.2; verify with: +docker run --rm alpine getent ahostsv4 host.docker.internal | awk '{print $1; exit}' +``` + +Use that IP in `netbird_enrollment_management_url`, then recreate wg-client: + +```bash +sandcat run --force-recreate wg-client +``` + +### Remote self-hosted (NetBird quickstart) + +For a VM with a public domain, sandcat does **not** run the installer. Use the +[official quickstart](https://docs.netbird.io/selfhosted/selfhosted-quickstart#installation-script): + +```bash +curl -fsSL https://github.com/netbirdio/netbird/releases/latest/download/getting-started.sh | bash +``` + +The script generates `docker-compose.yml`, `config.yaml`, and `dashboard.env` +with embedded IdP support. Follow the prompts (Traefik `[0]` is the default). + +**First-time onboarding** (from the [quickstart guide](https://docs.netbird.io/selfhosted/selfhosted-quickstart#installation-script)): + +1. Open `https://` in a browser. +2. You are redirected to `/setup` while no users exist. +3. Create the admin account (email, name, password). +4. In the dashboard, create a **Setup Key** and an **API Key** (PAT). + +For scripted bootstrap instead of the dashboard setup page, see +[Automated setup with a Personal Access Token](https://docs.netbird.io/selfhosted/automated-setup). + +**Wire sandcat** after the server is running: + +```json +"netbird_management_url": "https://netbird.example.com", +"netbird_enrollment_key": "", +"netbird_api_token": "" +``` + +Re-run `sandcat init --netbird --netbird-server https://netbird.example.com ...` +or edit `~/.config/sandcat/settings.json` directly, then `sandcat run`. + ### Runtime control ```bash diff --git a/cli/core b/cli/core new file mode 100644 index 00000000..e69de29b diff --git a/cli/lib/composefile.bash b/cli/lib/composefile.bash index ba4a149f..b56c1015 100644 --- a/cli/lib/composefile.bash +++ b/cli/lib/composefile.bash @@ -628,8 +628,12 @@ apply_netbird_build_args() { # $2 - Optional NetBird management server URL enable_netbird() { require yq + # shellcheck source=netbird.bash + source "$SCT_LIBDIR/netbird.bash" + local compose_file=$1 local netbird_management_url=${2:-} + local enrollment_url local already_set already_set=$(yq '[(.services."wg-client".environment // [])[] | select(. == "NB_SETUP_KEY")] | length' "$compose_file") @@ -639,12 +643,23 @@ enable_netbird() { fi if [[ -n "$netbird_management_url" ]]; then - netbird_management_url="$netbird_management_url" \ - yq -i ' - .services."wg-client".environment = ( - (.services."wg-client".environment // []) - | map(select(test("^NB_MANAGEMENT_URL=") | not)) - ) + ["NB_MANAGEMENT_URL=" + env(netbird_management_url)] - ' "$compose_file" + enrollment_url=$(netbird_enrollment_management_url_from "$netbird_management_url") + if [[ -n "$enrollment_url" ]]; then + enrollment_url="$enrollment_url" \ + yq -i ' + .services."wg-client".environment = ( + (.services."wg-client".environment // []) + | map(select(test("^NB_MANAGEMENT_URL=") | not)) + ) + ["NB_MANAGEMENT_URL=" + env(enrollment_url)] + ' "$compose_file" + if netbird_enrollment_url_uses_host_bypass "$enrollment_url"; then + yq -i ' + .services."wg-client".environment = ( + (.services."wg-client".environment // []) + | map(select(test("^NB_USE_LEGACY_ROUTING=") | not)) + ) + ["NB_USE_LEGACY_ROUTING=true"] + ' "$compose_file" + fi + fi fi } diff --git a/cli/lib/netbird.bash b/cli/lib/netbird.bash index 0261954c..37b2e5bb 100644 --- a/cli/lib/netbird.bash +++ b/cli/lib/netbird.bash @@ -61,6 +61,160 @@ export_netbird_management_url() { fi } +# Returns the management URL wg-client should use for NetBird enrollment. +# netbird_enrollment_management_url in settings wins when set. Remote URLs pass +# through unchanged. localhost / 127.0.0.1 require an explicit enrollment URL +# (wg-client cannot reach the host via localhost). +# Args: +# $1 - Host-side management URL (e.g. http://localhost:33073) +netbird_enrollment_management_url_from() { + local management_url=$1 + local explicit + + explicit=$(netbird_read_setting netbird_enrollment_management_url) + if [[ -n "$explicit" ]]; then + printf '%s' "$explicit" + return 0 + fi + + [[ -n "$management_url" ]] || return 0 + + if [[ "$management_url" =~ ^https?://(localhost|127\.0\.0\.1)([:/]|$) ]]; then + return 0 + fi + + printf '%s' "$management_url" +} + +# Returns 0 when the enrollment URL targets the Docker host by literal IPv4 and +# wg-client must bypass wg0 for management traffic. +# Args: +# $1 - Enrollment management URL +netbird_enrollment_url_uses_host_bypass() { + local url=$1 + [[ "$url" =~ ^https?://([0-9]{1,3}\.){3}[0-9]{1,3}([:/]|$) ]] +} + +# Resolves the NetBird embedded-IdP encryption key. +# NETBIRD_ENCRYPTION_KEY env wins; otherwise generates a new key. +_netbird_resolve_encryption_key() { + if [[ -n "${NETBIRD_ENCRYPTION_KEY:-}" ]]; then + printf '%s' "$NETBIRD_ENCRYPTION_KEY" + return 0 + fi + + require openssl + openssl rand -base64 32 +} + +# Resolves the relay auth secret for config.yaml server.authSecret. +# NETBIRD_RELAY_AUTH_SECRET env wins; otherwise generates a new secret. +_netbird_resolve_relay_auth_secret() { + if [[ -n "${NETBIRD_RELAY_AUTH_SECRET:-}" ]]; then + printf '%s' "$NETBIRD_RELAY_AUTH_SECRET" + return 0 + fi + + require openssl + openssl rand -base64 32 +} + +# Reads a KEY=value from a netbird-server.env file (first match wins). +# Args: +# $1 - env file path +# $2 - variable name +_netbird_env_value_from_file() { + local env_file=$1 + local key=$2 + local line + + [[ -f "$env_file" ]] || return 1 + while IFS= read -r line || [[ -n "$line" ]]; do + [[ "$line" == "${key}="* ]] || continue + printf '%s' "${line#"${key}="}" + return 0 + done <"$env_file" + return 1 +} + +# Replaces or appends KEY=value in a simple env file. +# Args: +# $1 - env file path +# $2 - variable name +# $3 - value +_netbird_env_set_value() { + local env_file=$1 + local key=$2 + local value=$3 + local tmp_file + local found=false + + [[ -f "$env_file" ]] || return 1 + tmp_file=$(mktemp) + while IFS= read -r line || [[ -n "$line" ]]; do + if [[ "$line" == "${key}="* ]]; then + printf '%s=%s\n' "$key" "$value" + found=true + else + printf '%s\n' "$line" + fi + done <"$env_file" >"$tmp_file" + if [[ "$found" == false ]]; then + printf '%s=%s\n' "$key" "$value" >>"$tmp_file" + fi + mv "$tmp_file" "$env_file" +} + +# Writes secrets and localhost endpoints into the provisioned netbird-server files. +# Args: +# $1 - provisioned directory (e.g. ~/.config/sandcat/netbird-server) +_netbird_apply_local_server_config() { + local dest_dir=$1 + local env_file config_file dashboard_file + local encryption_key relay_secret mgmt_port dashboard_port + local mgmt_endpoint issuer dashboard_base + + env_file="$dest_dir/netbird-server.env" + config_file="$dest_dir/config.yaml" + dashboard_file="$dest_dir/dashboard.env" + [[ -f "$env_file" && -f "$config_file" ]] || return 0 + + encryption_key=$(_netbird_resolve_encryption_key) + relay_secret=$(_netbird_resolve_relay_auth_secret) + _netbird_env_set_value "$env_file" NETBIRD_ENCRYPTION_KEY "$encryption_key" + _netbird_env_set_value "$env_file" NETBIRD_RELAY_AUTH_SECRET "$relay_secret" + + mgmt_port=$(_netbird_env_value_from_file "$env_file" NETBIRD_MGMT_API_PORT || true) + dashboard_port=$(_netbird_env_value_from_file "$env_file" NETBIRD_DASHBOARD_HTTP_PORT || true) + mgmt_port=${mgmt_port:-33073} + dashboard_port=${dashboard_port:-8080} + + mgmt_endpoint="http://localhost:${mgmt_port}" + issuer="${mgmt_endpoint}/oauth2" + dashboard_base="http://localhost:${dashboard_port}" + + require yq + NB_EXPOSED="$mgmt_endpoint" \ + NB_ISSUER="$issuer" \ + NB_NB_AUTH="${dashboard_base}/nb-auth" \ + NB_NB_SILENT="${dashboard_base}/nb-silent-auth" \ + NB_ENC="$encryption_key" \ + NB_RELAY="$relay_secret" \ + yq -i ' + .server.exposedAddress = strenv(NB_EXPOSED) | + .server.authSecret = strenv(NB_RELAY) | + .server.auth.issuer = strenv(NB_ISSUER) | + .server.auth.dashboardRedirectURIs = [strenv(NB_NB_AUTH), strenv(NB_NB_SILENT)] | + .server.store.encryptionKey = strenv(NB_ENC)' \ + "$config_file" + + if [[ -f "$dashboard_file" ]]; then + _netbird_env_set_value "$dashboard_file" NETBIRD_MGMT_API_ENDPOINT "$mgmt_endpoint" + _netbird_env_set_value "$dashboard_file" NETBIRD_MGMT_GRPC_API_ENDPOINT "$mgmt_endpoint" + _netbird_env_set_value "$dashboard_file" AUTH_AUTHORITY "$issuer" + fi +} + # Creates ~/.config/sandcat/netbird-server from template when missing. # Idempotent: if destination exists, logs and skips. provision_netbird_server_template() { @@ -70,9 +224,9 @@ provision_netbird_server_template() { local required_file local -a required_files=( docker-compose.yml + config.yaml + dashboard.env netbird-server.env - management.json - turnserver.conf README.md ) @@ -96,6 +250,7 @@ provision_netbird_server_template() { mkdir -p "$destination_dir" # Use rsync-style copy to include dotfiles (glob * skips them) cp -R "$template_dir/." "$destination_dir/" + _netbird_apply_local_server_config "$destination_dir" } # Resolve NB_API_TOKEN from settings when unset. Env always wins. diff --git a/cli/libexec/init/init b/cli/libexec/init/init index 5cdcb64c..ea4f797c 100755 --- a/cli/libexec/init/init +++ b/cli/libexec/init/init @@ -20,6 +20,22 @@ source "$SCT_LIBDIR/gitignore.bash" # shellcheck source=../../lib/netbird.bash source "$SCT_LIBDIR/netbird.bash" +# Prints instructions for the local template provisioned under ~/.config/sandcat/netbird-server. +_print_netbird_local_template_hint() { + echo " Local template: ~/.config/sandcat/netbird-server/" | info + echo " Start: cd ~/.config/sandcat/netbird-server && docker compose --env-file netbird-server.env up -d" | info + echo " Dashboard: http://localhost:8080 (bootstrap via /api/setup first — see README.md there)" | info +} + +# Prints instructions for installing a self-hosted NetBird server via the +# official quickstart script (sandcat does not run the installer). +_print_netbird_selfhosted_install_hint() { + echo " Remote install (VM + public domain) — run yourself:" | info + echo " curl -fsSL https://github.com/netbirdio/netbird/releases/latest/download/getting-started.sh | bash" | info + echo " Docs: https://docs.netbird.io/selfhosted/selfhosted-quickstart#installation-script" | info + echo " After install: open https:///setup to create the admin account" | info +} + # Returns the user settings template path for a selected agent. # Args: # $1 - Agent name (claude, cursor) @@ -167,6 +183,7 @@ init() { local netbird_server_provided=false local netbird_management_url="" local provisioned_netbird_server="false" + local netbird_selfhosted_quickstart="false" while [[ $# -gt 0 ]] do @@ -248,11 +265,11 @@ init() { fi if [[ "$netbird_server_provided" == "true" ]]; then case "$netbird_server" in - cloud|new|http://*|https://*) + cloud|new|quickstart|http://*|https://*) : ;; *) - echo "Invalid NetBird server mode: $netbird_server (expected: cloud, new, or http(s)://... URL)" | error + echo "Invalid NetBird server mode: $netbird_server (expected: cloud, new, quickstart, or http(s)://... URL)" | error return 1 ;; esac @@ -455,6 +472,11 @@ init() { netbird_management_url="http://localhost:33073" persist_netbird_management_url "$netbird_management_url" ;; + quickstart) + netbird_selfhosted_quickstart="true" + _print_netbird_selfhosted_install_hint + netbird_management_url="" + ;; http://*|https://*) netbird_management_url="$netbird_server" persist_netbird_management_url "$netbird_management_url" @@ -466,7 +488,8 @@ init() { echo "NetBird management server [cloud]:" | info echo " 1) cloud (api.netbird.io)" | info echo " 2) self-hosted — I have a server running" | info - echo " 3) self-hosted — provision a new server from template" | info + echo " 3) self-hosted — local template (localhost)" | info + echo " 4) self-hosted — NetBird quickstart (VM + domain)" | info netbird_server_selection=$(read_line ">") case "$netbird_server_selection" in ""|1|cloud) @@ -477,12 +500,16 @@ init() { netbird_server_selection="existing" break ;; - 3|new|self-hosted-new|self-hosted\ new\ template|self-hosted\ —\ provision\ a\ new\ server\ from\ template) + 3|new|local|self-hosted-local|self-hosted\ —\ local\ template|self-hosted\ new\ template|self-hosted\ —\ provision\ a\ new\ server\ from\ template) netbird_server_selection="new" break ;; + 4|quickstart|self-hosted-quickstart|self-hosted\ —\ NetBird\ quickstart|self-hosted\ —\ install\ with\ NetBird\ quickstart) + netbird_server_selection="quickstart" + break + ;; *) - echo "Invalid NetBird management server selection: $netbird_server_selection (expected: cloud, existing, or new)" | error + echo "Invalid NetBird management server selection: $netbird_server_selection (expected: cloud, existing, new, or quickstart)" | error ;; esac done @@ -504,14 +531,19 @@ init() { new) provision_netbird_server_template provisioned_netbird_server="true" - echo " Self-hosted template: ~/.config/sandcat/netbird-server" | info - echo " Template is a starter skeleton; configure before first start:" | info - echo " netbird-server.env, management.json, turnserver.conf" | info - echo " Start hint: cd ~/.config/sandcat/netbird-server && docker compose --env-file netbird-server.env up -d" | info + _print_netbird_local_template_hint netbird_management_url=$(read_line "Management URL [http://localhost:33073]:") netbird_management_url="${netbird_management_url:-http://localhost:33073}" persist_netbird_management_url "$netbird_management_url" ;; + quickstart) + netbird_selfhosted_quickstart="true" + _print_netbird_selfhosted_install_hint + netbird_management_url=$(read_line "Management URL [https://netbird.example.com]:") + if [[ -n "$netbird_management_url" ]]; then + persist_netbird_management_url "$netbird_management_url" + fi + ;; esac fi @@ -654,9 +686,11 @@ init() { echo " \"netbird_enrollment_key\": \"\" (wg-client enrollment)" | info echo " \"netbird_api_token\": \"\" (sandcat netbird commands)" | info if [[ "$provisioned_netbird_server" == "true" ]]; then - echo " Self-hosted server template: ~/.config/sandcat/netbird-server/" | info - echo " Start it: docker compose -f ~/.config/sandcat/netbird-server/docker-compose.yml up -d" | info - echo " (Future: sandcat netbird server start)" | info + _print_netbird_local_template_hint + fi + if [[ "$netbird_selfhosted_quickstart" == "true" ]]; then + _print_netbird_selfhosted_install_hint + echo " Then set netbird_management_url or re-run init with --netbird-server https://" | info fi fi echo " Then run: sandcat run, or reopen the project using the dev container" | info diff --git a/cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh b/cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh index 236a465a..b951f2d1 100644 --- a/cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh +++ b/cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh @@ -307,6 +307,9 @@ main() { } >> /etc/hosts fi + # Host routing/iptables for local NetBird enrollment (literal host IP in URL). + configure_netbird_host_management_access "$docker_gateway" + # ── NetBird overlay mesh (optional) ───────────────────────────────────────── # Enroll and start the NetBird daemon if NB_SETUP_KEY is set. wt0 is created # by the daemon in this network namespace alongside wg0. fwmark 51821 ensures @@ -338,6 +341,130 @@ trust_mitmproxy_ca() { update-ca-certificates --fresh >/dev/null 2>&1 } +# Extracts the host from an http(s) management URL. +# Args: +# $1 - URL (e.g. http://192.168.5.2:33073) +netbird_management_url_host() { + local url=$1 + + [[ "$url" =~ ^https?://([^/:]+) ]] || return 1 + printf '%s' "${BASH_REMATCH[1]}" +} + +# Returns 0 when the URL host is a literal IPv4 address. +# Args: +# $1 - Hostname or IP +netbird_management_url_host_is_literal_ipv4() { + local host=$1 + [[ "$host" =~ ^([0-9]{1,3}\.){3}[0-9]{1,3}$ ]] +} + +# Extracts the TCP port from an http(s) management URL. Defaults to 443/80. +# Args: +# $1 - URL (e.g. http://192.168.5.2:33073) +netbird_management_url_port() { + local url=$1 + if [[ "$url" =~ :([0-9]+)(/|$|\?) ]]; then + printf '%s' "${BASH_REMATCH[1]}" + return 0 + fi + if [[ "$url" =~ ^https:// ]]; then + printf '443' + else + printf '80' + fi +} + +# Returns 0 when the Docker host is off the bridge subnet and must be reached +# via the bridge gateway (e.g. Colima host 192.168.5.2 on sandcat 172.23.0.0/16). +# Args: +# $1 - Resolved host IPv4 +# $2 - Docker bridge gateway IPv4 +netbird_host_route_uses_gateway() { + local host_ip=$1 + local docker_gateway=$2 + + [[ -n "$host_ip" && -n "$docker_gateway" ]] || return 1 + [[ "$host_ip" != "$docker_gateway" ]] +} + +# Allow enrollment against a NetBird management server on the Docker host. +# wg-client routes most traffic through wg0; management traffic to a literal +# host IP must bypass mitmproxy via eth0. Re-apply after netbird service start +# (it may add routing rules that steal host-bound traffic). +# Args: +# $1 - Docker bridge gateway IP +configure_netbird_host_management_access() { + local docker_gateway=$1 + local mgmt_url="${NB_MANAGEMENT_URL:-}" + local host_ip port + + [[ -n "${NB_SETUP_KEY:-}" ]] || return 0 + [[ -n "$docker_gateway" ]] || return 0 + + host_ip=$(netbird_management_url_host "$mgmt_url") || return 0 + netbird_management_url_host_is_literal_ipv4 "$host_ip" || return 0 + + port=$(netbird_management_url_port "$mgmt_url") + + if netbird_host_route_uses_gateway "$host_ip" "$docker_gateway"; then + echo "[wg-client] Allowing NetBird management traffic to ${host_ip}:${port} via eth0 (via ${docker_gateway})." >&2 + else + echo "[wg-client] Allowing NetBird management traffic to ${host_ip}:${port} via eth0." >&2 + fi + + # Prefer main table for host management (sandcat policy routing uses 51820 → wg0). + ip -4 rule add to "${host_ip}/32" lookup main priority 50 2>/dev/null || true + if netbird_host_route_uses_gateway "$host_ip" "$docker_gateway"; then + ip -4 route add "${host_ip}/32" via "${docker_gateway}" dev eth0 table main 2>/dev/null || true + ip -4 route add "${host_ip}/32" via "${docker_gateway}" dev eth0 table 51820 2>/dev/null || true + else + ip -4 route add "${host_ip}/32" dev eth0 table main 2>/dev/null || true + ip -4 route add "${host_ip}/32" dev eth0 table 51820 2>/dev/null || true + fi + + iptables -C OUTPUT -o eth0 -d "$host_ip" -p tcp --dport "$port" -j ACCEPT 2>/dev/null \ + || iptables -I OUTPUT 1 -o eth0 -d "$host_ip" -p tcp --dport "$port" -j ACCEPT + iptables -C OUTPUT -o eth0 -d "$host_ip" -p udp --dport 3478 -j ACCEPT 2>/dev/null \ + || iptables -I OUTPUT 1 -o eth0 -d "$host_ip" -p udp --dport 3478 -j ACCEPT +} + +# Verifies HTTP reachability to a self-hosted management server on the Docker host. +netbird_verify_host_management_reachable() { + local mgmt_url="${NB_MANAGEMENT_URL:-}" + local host_ip port check_url + + host_ip=$(netbird_management_url_host "$mgmt_url") || return 0 + netbird_management_url_host_is_literal_ipv4 "$host_ip" || return 0 + + port=$(netbird_management_url_port "$mgmt_url") + command -v curl >/dev/null 2>&1 || return 0 + + check_url="${mgmt_url%/}/api/instance" + wait_until 15 1 \ + "[wg-client] Cannot reach NetBird management at ${mgmt_url} from wg-client; is netbird-server running on the host (port ${port})?" \ + curl -sf --max-time 5 "$check_url" >/dev/null +} + +netbird_daemon_ready() { + netbird status >/dev/null 2>&1 +} + +# Starts the NetBird background service (required for client 0.28+). +ensure_netbird_service() { + [[ -n "${NB_SETUP_KEY:-}" ]] || return 0 + if netbird_daemon_ready; then + return 0 + fi + + echo "[wg-client] Starting NetBird service daemon." >&2 + netbird service run --log-file console & + + wait_until 30 1 \ + "[wg-client] Timed out waiting for NetBird service daemon" \ + netbird_daemon_ready +} + # Enroll this container as a NetBird peer and start the daemon in the background. # Does nothing if NB_SETUP_KEY is unset (NetBird disabled for this environment). # After the daemon starts it waits until the overlay interface appears. @@ -345,14 +472,20 @@ trust_mitmproxy_ca() { # $1 - WireGuard interface name for the NetBird overlay (default: wt0) start_netbird() { local iface="${1:-wt0}" + local docker_gateway [[ -n "${NB_SETUP_KEY:-}" ]] || return 0 - echo "[wg-client] Starting NetBird daemon on ${iface}." >&2 + docker_gateway=$(ip -4 route show default dev eth0 2>/dev/null | awk '{print $3}') + + ensure_netbird_service + configure_netbird_host_management_access "$docker_gateway" + netbird_verify_host_management_reachable + + echo "[wg-client] Enrolling NetBird peer on ${iface}." >&2 netbird up \ --setup-key "${NB_SETUP_KEY}" \ --management-url "${NB_MANAGEMENT_URL:-https://api.netbird.io}" \ - --interface-name "${iface}" \ - & + --interface-name "${iface}" wait_until 30 1 \ "[wg-client] Timed out waiting for NetBird to bring up ${iface}" \ @@ -382,9 +515,21 @@ supervise_netbird_daemon() { while true; do sleep 10 - if ! netbird status >/dev/null 2>&1; then - echo "[wg-client] NetBird daemon not responding; restarting." >&2 - start_netbird "${iface}" || true + if ! netbird_daemon_ready; then + echo "[wg-client] NetBird service daemon not responding; restarting." >&2 + netbird service run --log-file console & + wait_until 15 1 \ + "[wg-client] Timed out waiting for NetBird service daemon" \ + netbird_daemon_ready || true + fi + if ! ip link show "${iface}" >/dev/null 2>&1; then + echo "[wg-client] NetBird interface ${iface} down; re-enrolling." >&2 + docker_gateway=$(ip -4 route show default dev eth0 2>/dev/null | awk '{print $3}') + configure_netbird_host_management_access "$docker_gateway" + netbird up \ + --setup-key "${NB_SETUP_KEY}" \ + --management-url "${NB_MANAGEMENT_URL:-https://api.netbird.io}" \ + --interface-name "${iface}" || true set_netbird_fwmark "${iface}" || true fi done diff --git a/cli/templates/netbird-server/README.md b/cli/templates/netbird-server/README.md index 5945ca03..fcd20a85 100644 --- a/cli/templates/netbird-server/README.md +++ b/cli/templates/netbird-server/README.md @@ -1,16 +1,110 @@ -# NetBird self-hosted template +# NetBird local self-hosted template -This directory is provisioned by `sandcat init --netbird --netbird-server new`. -It is a starter skeleton, not a ready-to-run production config. +Provisioned by `sandcat init --netbird --netbird-server new` into +`~/.config/sandcat/netbird-server/`. -Before first startup, review and adapt: +Uses the **combined** `netbirdio/netbird-server` image (management + signal + +relay + STUN) — the same layout as NetBird's +[getting-started.sh](https://docs.netbird.io/selfhosted/selfhosted-quickstart#installation-script) +exposed-ports mode, tuned for localhost. -- `netbird-server.env` (image/version pins and required endpoint/auth env values) -- `management.json` (mounted by compose into management service) -- `turnserver.conf` (mounted by compose into coturn; set credentials/realm) +For a VM with a public domain, use `sandcat init --netbird --netbird-server quickstart` +instead (see `cli/README.md`). -Start with: +## 1. Start the stack ```bash +cd ~/.config/sandcat/netbird-server docker compose --env-file netbird-server.env up -d ``` + +Always pass `--env-file netbird-server.env`. + +## 2. Verify the API + +```bash +curl -s http://localhost:33073/api/instance +``` + +Expect `"setup_required": true` before bootstrap. Use **http** (not https) and +the **/api/** prefix. + +## 3. Bootstrap the first admin + +```bash +curl -fsS -X POST "http://localhost:33073/api/setup" \ + -H "Content-Type: application/json" \ + -d '{ + "email": "admin@example.com", + "name": "Admin", + "password": "choose-a-long-random-password", + "create_pat": true, + "pat_expire_in": 7 + }' +``` + +Save the returned `personal_access_token` for `netbird_api_token` in +`~/.config/sandcat/settings.json`. See +[NetBird automated setup](https://docs.netbird.io/selfhosted/automated-setup). + +## 4. Open the dashboard + +```text +http://localhost:8080 +``` + +Or the setup wizard: `http://localhost:8080/setup` (before step 3). + +## 5. Wire sandcat + +Host tools and the browser use `localhost`. wg-client needs the Docker host IP +(see [cli/README.md](../../README.md#local-self-hosted-sandcat-template)): + +```json +"netbird_management_url": "http://localhost:33073", +"netbird_enrollment_management_url": "http://192.168.5.2:33073", +"netbird_enrollment_key": "", +"netbird_api_token": "" +``` + +Replace `192.168.5.2` with your Docker host IP (`colima status -j` on Colima). +Then recreate wg-client: `sandcat run --force-recreate wg-client`. + +## Troubleshooting + +### Port clash with sandcat devcontainers + +The default ports **8080** (dashboard) and **33073** (management API) may +already be in use on your machine — for example if a sandcat devcontainer or +another stack publishes them. Change `NETBIRD_DASHBOARD_HTTP_PORT` and +`NETBIRD_MGMT_API_PORT` in `netbird-server.env`, then re-run +`sandcat init --netbird --netbird-server new` (after removing the old +provisioned directory) so `config.yaml` and `dashboard.env` pick up the new URLs. + +### `curl: (52) Empty reply` or SSL errors on port 33073 + +- Use **`http://localhost:33073/api/...`** — not `https://` and not `/setup` + (dashboard route; API is `/api/setup`). +- Ensure compose maps **`33073:80`** to `netbird-server`, not `33073:443`. +- Recreate after template changes: + `docker compose --env-file netbird-server.env up -d --force-recreate netbird-server` + +### `/api/setup` returns "not authenticated" + +Embedded IdP must be enabled in `config.yaml` (`server.auth.issuer`). Re-provision +or merge the updated template, then restart. If needed, reset data: + +```bash +docker compose --env-file netbird-server.env down -v +docker compose --env-file netbird-server.env up -d +``` + +### Already provisioned an older copy + +Sandcat skips re-copying when `~/.config/sandcat/netbird-server` exists. Remove +it and run `sandcat init --netbird --netbird-server new` again, or manually +replace `docker-compose.yml`, `config.yaml`, and `dashboard.env` from this +template. + +Secrets (`NETBIRD_ENCRYPTION_KEY`, relay `authSecret`) are generated once at +first provision. Keep them stable across restarts. diff --git a/cli/templates/netbird-server/config.yaml b/cli/templates/netbird-server/config.yaml new file mode 100644 index 00000000..f0d42a79 --- /dev/null +++ b/cli/templates/netbird-server/config.yaml @@ -0,0 +1,25 @@ +# Combined NetBird server — localhost defaults. Secrets and URLs are filled +# when sandcat provisions this template. +server: + listenAddress: ":80" + exposedAddress: "http://localhost:33073" + stunPorts: + - 3478 + metricsPort: 9090 + healthcheckAddress: ":9000" + logLevel: info + logFile: console + disableAnonymousMetrics: true + authSecret: "" + dataDir: /var/lib/netbird + auth: + issuer: http://localhost:33073/oauth2 + signKeyRefreshEnabled: true + dashboardRedirectURIs: + - http://localhost:8080/nb-auth + - http://localhost:8080/nb-silent-auth + cliRedirectURIs: + - http://localhost:53000/ + store: + engine: sqlite + encryptionKey: "" diff --git a/cli/templates/netbird-server/dashboard.env b/cli/templates/netbird-server/dashboard.env new file mode 100644 index 00000000..82212b8c --- /dev/null +++ b/cli/templates/netbird-server/dashboard.env @@ -0,0 +1,14 @@ +# Dashboard OIDC — API on netbird-server (host localhost:33073). +NETBIRD_MGMT_API_ENDPOINT=http://localhost:33073 +NETBIRD_MGMT_GRPC_API_ENDPOINT=http://localhost:33073 +AUTH_AUDIENCE=netbird-dashboard +AUTH_CLIENT_ID=netbird-dashboard +AUTH_CLIENT_SECRET= +AUTH_AUTHORITY=http://localhost:33073/oauth2 +USE_AUTH0=false +AUTH_SUPPORTED_SCOPES=openid profile email groups +AUTH_REDIRECT_URI=/nb-auth +AUTH_SILENT_REDIRECT_URI=/nb-silent-auth +NETBIRD_TOKEN_SOURCE=idToken +NGINX_SSL_PORT=443 +LETSENCRYPT_DOMAIN=none diff --git a/cli/templates/netbird-server/docker-compose.yml b/cli/templates/netbird-server/docker-compose.yml index 8265b867..c01342b4 100644 --- a/cli/templates/netbird-server/docker-compose.yml +++ b/cli/templates/netbird-server/docker-compose.yml @@ -1,117 +1,37 @@ -x-default: &default - restart: "unless-stopped" - logging: - driver: "json-file" - options: - max-size: "500m" - max-file: "2" +# Local NetBird stack: dashboard + combined netbird-server (management, signal, +# relay, STUN). Matches NetBird getting-started exposed-ports mode. services: - # UI dashboard dashboard: - <<: *default - image: netbirdio/dashboard:${NETBIRD_SERVER_VERSION} + image: netbirdio/dashboard:${NETBIRD_DASHBOARD_VERSION} + restart: unless-stopped ports: - - ${NETBIRD_DASHBOARD_HTTP_PORT:-80}:80 - - ${NETBIRD_DASHBOARD_HTTPS_PORT:-443}:443 - environment: - # Endpoints - - NETBIRD_MGMT_API_ENDPOINT=${NETBIRD_MGMT_API_ENDPOINT} - - NETBIRD_MGMT_GRPC_API_ENDPOINT=${NETBIRD_MGMT_GRPC_API_ENDPOINT:-${NETBIRD_MGMT_API_ENDPOINT}} - # OIDC - - AUTH_AUDIENCE=${NETBIRD_DASH_AUTH_AUDIENCE} - - AUTH_CLIENT_ID=${NETBIRD_AUTH_CLIENT_ID} - - AUTH_CLIENT_SECRET=${NETBIRD_AUTH_CLIENT_SECRET} - - AUTH_AUTHORITY=${NETBIRD_AUTH_AUTHORITY} - - USE_AUTH0=${NETBIRD_USE_AUTH0} - - AUTH_SUPPORTED_SCOPES=${NETBIRD_AUTH_SUPPORTED_SCOPES} - - AUTH_REDIRECT_URI=${NETBIRD_AUTH_REDIRECT_URI} - - AUTH_SILENT_REDIRECT_URI=${NETBIRD_AUTH_SILENT_REDIRECT_URI} - - NETBIRD_TOKEN_SOURCE=${NETBIRD_TOKEN_SOURCE} - # SSL - - NGINX_SSL_PORT=443 - # Letsencrypt - - LETSENCRYPT_DOMAIN=${NETBIRD_LETSENCRYPT_DOMAIN} - - LETSENCRYPT_EMAIL=${NETBIRD_LETSENCRYPT_EMAIL} - volumes: - - letsencrypt:/etc/letsencrypt/ - - # Signal - signal: - <<: *default - image: netbirdio/signal:${NETBIRD_SERVER_VERSION} - volumes: - - signal:/var/lib/netbird - - letsencrypt:/etc/letsencrypt:ro - ports: - - ${NETBIRD_SIGNAL_PORT}:80 - # # port and command for Let's Encrypt validation - # - 443:443 - # command: ["--letsencrypt-domain", "${NETBIRD_LETSENCRYPT_DOMAIN}", "--log-file", "console"] - command: - - "--cert-file" - - "${NETBIRD_MGMT_API_CERT_FILE}" - - "--cert-key" - - "${NETBIRD_MGMT_API_CERT_KEY_FILE}" - - "--log-file" - - "console" - - "--port" - - "80" + - ${NETBIRD_DASHBOARD_HTTP_PORT:-8080}:80 + env_file: + - dashboard.env + logging: + driver: json-file + options: + max-size: 500m + max-file: "2" - # Relay - relay: - <<: *default - image: netbirdio/relay:${NETBIRD_SERVER_VERSION} - environment: - - NB_LOG_LEVEL=info - - NB_LISTEN_ADDRESS=:${NETBIRD_RELAY_PORT} - - NB_EXPOSED_ADDRESS=${NETBIRD_RELAY_ENDPOINT} - # todo: change to a secure secret - - NB_AUTH_SECRET=${NETBIRD_RELAY_AUTH_SECRET} + netbird-server: + image: netbirdio/netbird-server:${NETBIRD_SERVER_VERSION} + restart: unless-stopped ports: - - ${NETBIRD_RELAY_PORT}:${NETBIRD_RELAY_PORT} - - # Management - management: - <<: *default - image: netbirdio/management:${NETBIRD_SERVER_VERSION} + - ${NETBIRD_MGMT_API_PORT}:80 + - ${NETBIRD_STUN_PORT}:3478/udp volumes: - - management:/var/lib/netbird - - letsencrypt:/etc/letsencrypt:ro - - ./management.json:/etc/netbird/management.json - ports: - - ${NETBIRD_MGMT_API_PORT}:443 # API port - # # command for Let's Encrypt validation without dashboard container - # command: ["--letsencrypt-domain", "${NETBIRD_LETSENCRYPT_DOMAIN}", "--log-file", "console"] - command: - - "--port" - - "443" - - "--log-file" - - "console" - - "--log-level" - - "info" - - "--disable-anonymous-metrics=${NETBIRD_DISABLE_ANONYMOUS_METRICS}" - - "--single-account-mode-domain=${NETBIRD_MGMT_SINGLE_ACCOUNT_MODE_DOMAIN}" - - "--dns-domain=${NETBIRD_MGMT_DNS_DOMAIN}" + - netbird_data:/var/lib/netbird + - ./config.yaml:/etc/netbird/config.yaml + command: ["--config", "/etc/netbird/config.yaml"] environment: - - NETBIRD_STORE_ENGINE_POSTGRES_DSN=${NETBIRD_STORE_ENGINE_POSTGRES_DSN} - - NETBIRD_STORE_ENGINE_MYSQL_DSN=${NETBIRD_STORE_ENGINE_MYSQL_DSN} - - # Coturn - coturn: - <<: *default - image: coturn/coturn:${COTURN_TAG:-4.6.2} - # domainname: ${TURN_DOMAIN} # only needed when TLS is enabled - volumes: - - ./turnserver.conf:/etc/turnserver.conf:ro - # - ./privkey.pem:/etc/coturn/private/privkey.pem:ro - # - ./cert.pem:/etc/coturn/certs/cert.pem:ro - network_mode: host - command: - - -c - - /etc/turnserver.conf + - NB_SETUP_PAT_ENABLED=true + logging: + driver: json-file + options: + max-size: 500m + max-file: "2" volumes: - management: - signal: - letsencrypt: + netbird_data: diff --git a/cli/templates/netbird-server/management.json b/cli/templates/netbird-server/management.json deleted file mode 100644 index 0967ef42..00000000 --- a/cli/templates/netbird-server/management.json +++ /dev/null @@ -1 +0,0 @@ -{} diff --git a/cli/templates/netbird-server/netbird-server.env b/cli/templates/netbird-server/netbird-server.env index 4316bcda..e96dd043 100644 --- a/cli/templates/netbird-server/netbird-server.env +++ b/cli/templates/netbird-server/netbird-server.env @@ -1,12 +1,12 @@ -# Single source of truth for pinned NetBird self-hosted service images. -# -# Consumed by: -# - cli/templates/netbird-server/docker-compose.yml -# -# Format note: simple KEY=value lines only (no quotes, no spaces around `=`) so -# this file can be consumed directly by shell tooling and docker compose env loading. -NETBIRD_SERVER_VERSION=0.28.9 -NETBIRD_DASHBOARD_HTTP_PORT=80 -NETBIRD_DASHBOARD_HTTPS_PORT=443 -# Optional override for a dedicated gRPC endpoint; defaults to NETBIRD_MGMT_API_ENDPOINT. -NETBIRD_MGMT_GRPC_API_ENDPOINT= +# Compose variable substitution for cli/templates/netbird-server/docker-compose.yml + +NETBIRD_DASHBOARD_VERSION=v2.39.0 +NETBIRD_SERVER_VERSION=0.72.4 + +NETBIRD_DASHBOARD_HTTP_PORT=8080 +NETBIRD_MGMT_API_PORT=33073 +NETBIRD_STUN_PORT=3478 + +# Filled at provision (openssl rand -base64 32) unless set before sandcat init. +NETBIRD_ENCRYPTION_KEY= +NETBIRD_RELAY_AUTH_SECRET= diff --git a/cli/templates/netbird-server/turnserver.conf b/cli/templates/netbird-server/turnserver.conf deleted file mode 100644 index 72a8a1f2..00000000 --- a/cli/templates/netbird-server/turnserver.conf +++ /dev/null @@ -1,8 +0,0 @@ -# Starter coturn config for local development. -# Adjust credentials/realm before exposing this service outside localhost. -listening-port=3478 -fingerprint -lt-cred-mech -realm=netbird.local -user=netbird:change-me -no-cli diff --git a/cli/test/composefile/netbird.bats b/cli/test/composefile/netbird.bats index 837b9d0a..8351f0a1 100644 --- a/cli/test/composefile/netbird.bats +++ b/cli/test/composefile/netbird.bats @@ -3,6 +3,8 @@ setup() { load test_helper source "$SCT_LIBDIR/composefile.bash" + # shellcheck source=netbird.bash + source "$SCT_LIBDIR/netbird.bash" COMPOSE_FILE="$BATS_TEST_TMPDIR/compose-proxy.yml" cp "$SCT_TEMPLATEDIR/devcontainer/sandcat/netbird.env" "$BATS_TEST_TMPDIR/netbird.env" @@ -70,6 +72,45 @@ teardown() { assert_output "0" } +@test "enable_netbird omits NB_MANAGEMENT_URL for localhost without enrollment URL" { + enable_netbird "$COMPOSE_FILE" "http://localhost:33073" + + run yq '[.services."wg-client".environment[] | select(test("^NB_MANAGEMENT_URL="))] | length' "$COMPOSE_FILE" + assert_output "0" +} + +@test "enable_netbird uses explicit enrollment URL for local self-hosted" { + export HOME="$BATS_TEST_TMPDIR/home" + mkdir -p "$HOME/.config/sandcat" + echo '{"netbird_enrollment_management_url": "http://192.168.5.2:33073"}' > "$HOME/.config/sandcat/settings.json" + + enable_netbird "$COMPOSE_FILE" "http://localhost:33073" + + run yq -r '.services."wg-client".environment[] | select(test("^NB_MANAGEMENT_URL="))' "$COMPOSE_FILE" + assert_output "NB_MANAGEMENT_URL=http://192.168.5.2:33073" +} + +@test "enable_netbird sets NB_USE_LEGACY_ROUTING for host IP enrollment URL" { + export HOME="$BATS_TEST_TMPDIR/home-enrollment" + mkdir -p "$HOME/.config/sandcat" + echo '{"netbird_enrollment_management_url": "http://192.168.5.2:33073"}' > "$HOME/.config/sandcat/settings.json" + + enable_netbird "$COMPOSE_FILE" "http://localhost:33073" + + run yq -r '.services."wg-client".environment[] | select(. == "NB_USE_LEGACY_ROUTING=true")' "$COMPOSE_FILE" + assert_output "NB_USE_LEGACY_ROUTING=true" +} + +@test "enable_netbird does not rewrite remote management URL" { + enable_netbird "$COMPOSE_FILE" "https://netbird.example.com" + + run yq -r '.services."wg-client".environment[] | select(test("^NB_MANAGEMENT_URL="))' "$COMPOSE_FILE" + assert_output "NB_MANAGEMENT_URL=https://netbird.example.com" + + run yq '[.services."wg-client".environment[] | select(. == "NB_USE_LEGACY_ROUTING=true")] | length' "$COMPOSE_FILE" + assert_output "0" +} + @test "apply_netbird_build_args injects version and checksum build args" { apply_netbird_build_args "$COMPOSE_FILE" diff --git a/cli/test/init/init.bats b/cli/test/init/init.bats index 43255598..8ac3f9f6 100644 --- a/cli/test/init/init.bats +++ b/cli/test/init/init.bats @@ -399,9 +399,9 @@ EOF assert_output "" } -@test "init --netbird-server new provisions template and uses default URL" { - unset -f provision_netbird_server_template +@test "init --netbird-server new provisions local template and uses default URL" { unset -f read_line + unset -f provision_netbird_server_template stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" stub devcontainer \ @@ -416,13 +416,73 @@ EOF run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird --netbird-server new assert_success assert_output --partial "Management server: http://localhost:33073" - assert_output --partial "Self-hosted server template: ~/.config/sandcat/netbird-server/" - assert_output --partial "Start it: docker compose -f ~/.config/sandcat/netbird-server/docker-compose.yml up -d" - assert_output --partial "(Future: sandcat netbird server start)" + assert_output --partial "Local template: ~/.config/sandcat/netbird-server/" + assert_output --partial "docker compose --env-file netbird-server.env up -d" + assert_output --partial "http://localhost:8080" + run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" + assert_output "http://localhost:33073" +} + +@test "init --netbird-server quickstart prints install hint without provisioning" { + unset -f read_line + unset -f provision_netbird_server_template + + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird : :" + provision_netbird_server_template() { + echo "provision_netbird_server_template should not be called for quickstart" >&2 + return 88 + } + export -f provision_netbird_server_template + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird --netbird-server quickstart + assert_success + assert_output --partial "getting-started.sh" + assert_output --partial "selfhosted-quickstart" + run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" + assert_output "" +} + +@test "init interactive netbird server new provisions local template" { + unset -f read_line + unset -f provision_netbird_server_template + + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird-management-url http://localhost:33073 --netbird : :" + stub provision_netbird_server_template ":" + stub read_line \ + "'>' : echo 3" \ + "'Management URL [http://localhost:33073]:' : echo ''" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird + assert_success + assert_output --partial "3) self-hosted — local template (localhost)" + assert_output --partial "Local template: ~/.config/sandcat/netbird-server/" run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" assert_output "http://localhost:33073" } +@test "init interactive netbird quickstart prints install hint and accepts URL" { + unset -f read_line + unset -f provision_netbird_server_template + + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird-management-url https://netbird.example.com --netbird : :" + stub read_line \ + "'>' : echo 4" \ + "'Management URL [https://netbird.example.com]:' : echo 'https://netbird.example.com'" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird + assert_success + assert_output --partial "4) self-hosted — NetBird quickstart (VM + domain)" + assert_output --partial "getting-started.sh" + run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" + assert_output "https://netbird.example.com" +} + @test "init interactive netbird server existing re-prompts for non-empty URL" { unset -f read_line @@ -457,31 +517,6 @@ EOF assert_output "management.example.com" } -@test "init interactive netbird server new provisions template and uses default URL" { - unset -f read_line - unset -f provision_netbird_server_template - - stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" - stub devcontainer \ - "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird-management-url http://localhost:33073 --netbird : :" - stub provision_netbird_server_template ":" - stub read_line \ - "'>' : echo 3" \ - "'Management URL [http://localhost:33073]:' : echo ''" - - run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird - assert_success - assert_output --partial "NetBird management server [cloud]:" - assert_output --partial "1) cloud (api.netbird.io)" - assert_output --partial "2) self-hosted — I have a server running" - assert_output --partial "3) self-hosted — provision a new server from template" - assert_output --partial "Self-hosted server template: ~/.config/sandcat/netbird-server/" - assert_output --partial "Start it: docker compose -f ~/.config/sandcat/netbird-server/docker-compose.yml up -d" - assert_output --partial "(Future: sandcat netbird server start)" - run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" - assert_output "http://localhost:33073" -} - @test "init interactive flow (devcontainer mode)" { unset -f read_line unset -f select_option diff --git a/cli/test/netbird/netbird_settings.bats b/cli/test/netbird/netbird_settings.bats index fa2aeff8..1575b3e8 100644 --- a/cli/test/netbird/netbird_settings.bats +++ b/cli/test/netbird/netbird_settings.bats @@ -81,32 +81,108 @@ teardown() { [[ "$NB_MANAGEMENT_URL" == "https://from-env.example.com" ]] } +@test "netbird_enrollment_management_url_from returns empty for localhost without explicit enrollment URL" { + run netbird_enrollment_management_url_from "http://localhost:33073" + assert_output "" +} + +@test "netbird_enrollment_management_url_from returns empty for 127.0.0.1 without explicit enrollment URL" { + run netbird_enrollment_management_url_from "http://127.0.0.1:33073" + assert_output "" +} + +@test "netbird_enrollment_management_url_from leaves remote URLs unchanged" { + run netbird_enrollment_management_url_from "https://netbird.example.com" + assert_output "https://netbird.example.com" +} + +@test "netbird_enrollment_management_url_from prefers netbird_enrollment_management_url setting" { + echo '{"netbird_enrollment_management_url": "http://192.168.5.2:33073"}' > "$HOME/.config/sandcat/settings.json" + + run netbird_enrollment_management_url_from "http://localhost:33073" + assert_output "http://192.168.5.2:33073" +} + +@test "netbird_enrollment_url_uses_host_bypass for literal IPv4 enrollment URL" { + run netbird_enrollment_url_uses_host_bypass "http://192.168.5.2:33073" + assert_success +} + +@test "netbird_enrollment_url_uses_host_bypass is false for hostname enrollment URL" { + run netbird_enrollment_url_uses_host_bypass "https://netbird.example.com" + assert_failure +} + @test "provision_netbird_server_template copies template and skips when already provisioned" { export SCT_TEMPLATEDIR="$BATS_TEST_TMPDIR/templates" mkdir -p "$SCT_TEMPLATEDIR/netbird-server" echo "services: {}" > "$SCT_TEMPLATEDIR/netbird-server/docker-compose.yml" echo "NETBIRD_SERVER_VERSION=0.1.0" > "$SCT_TEMPLATEDIR/netbird-server/netbird-server.env" - echo "{}" > "$SCT_TEMPLATEDIR/netbird-server/management.json" - echo "listening-port=3478" > "$SCT_TEMPLATEDIR/netbird-server/turnserver.conf" + echo "server: { store: { encryptionKey: \"\" } }" > "$SCT_TEMPLATEDIR/netbird-server/config.yaml" + echo "NETBIRD_MGMT_API_ENDPOINT=http://localhost:33073" > "$SCT_TEMPLATEDIR/netbird-server/dashboard.env" echo "# template" > "$SCT_TEMPLATEDIR/netbird-server/README.md" provision_netbird_server_template [[ -d "$HOME/.config/sandcat/netbird-server" ]] [[ -f "$HOME/.config/sandcat/netbird-server/docker-compose.yml" ]] [[ "$(cat "$HOME/.config/sandcat/netbird-server/docker-compose.yml")" == "services: {}" ]] + [[ -n "$(yq -r '.server.store.encryptionKey // ""' "$HOME/.config/sandcat/netbird-server/config.yaml")" ]] run provision_netbird_server_template assert_success assert_output --partial "skipping" } +@test "provision_netbird_server_template generates secrets in env and config.yaml" { + export SCT_TEMPLATEDIR="$BATS_TEST_TMPDIR/templates" + mkdir -p "$SCT_TEMPLATEDIR/netbird-server" + echo "services: {}" > "$SCT_TEMPLATEDIR/netbird-server/docker-compose.yml" + printf '%s\n' \ + "NETBIRD_SERVER_VERSION=0.1.0" \ + "NETBIRD_ENCRYPTION_KEY=" \ + "NETBIRD_RELAY_AUTH_SECRET=" \ + "NETBIRD_MGMT_API_PORT=33073" \ + "NETBIRD_DASHBOARD_HTTP_PORT=8080" \ + > "$SCT_TEMPLATEDIR/netbird-server/netbird-server.env" + echo "server: { authSecret: \"\", store: { encryptionKey: \"\" }, auth: { issuer: \"\" } }" > "$SCT_TEMPLATEDIR/netbird-server/config.yaml" + echo "NETBIRD_MGMT_API_ENDPOINT=http://localhost:33073" > "$SCT_TEMPLATEDIR/netbird-server/dashboard.env" + echo "# template" > "$SCT_TEMPLATEDIR/netbird-server/README.md" + unset NETBIRD_ENCRYPTION_KEY NETBIRD_RELAY_AUTH_SECRET + + provision_netbird_server_template + + local env_key config_key + env_key=$(grep '^NETBIRD_ENCRYPTION_KEY=' "$HOME/.config/sandcat/netbird-server/netbird-server.env" | cut -d= -f2-) + config_key=$(yq -r '.server.store.encryptionKey' "$HOME/.config/sandcat/netbird-server/config.yaml") + [[ -n "$env_key" ]] + [[ "$env_key" == "$config_key" ]] + [[ "$(yq -r '.server.auth.issuer' "$HOME/.config/sandcat/netbird-server/config.yaml")" == "http://localhost:33073/oauth2" ]] + [[ "$(yq -r '.server.auth.dashboardRedirectURIs[0]' "$HOME/.config/sandcat/netbird-server/config.yaml")" == "http://localhost:8080/nb-auth" ]] +} + +@test "provision_netbird_server_template uses NETBIRD_ENCRYPTION_KEY from environment" { + export SCT_TEMPLATEDIR="$BATS_TEST_TMPDIR/templates" + mkdir -p "$SCT_TEMPLATEDIR/netbird-server" + echo "services: {}" > "$SCT_TEMPLATEDIR/netbird-server/docker-compose.yml" + printf '%s\n' "NETBIRD_SERVER_VERSION=0.1.0" "NETBIRD_ENCRYPTION_KEY=" > "$SCT_TEMPLATEDIR/netbird-server/netbird-server.env" + echo "server: { store: { encryptionKey: \"\" } }" > "$SCT_TEMPLATEDIR/netbird-server/config.yaml" + echo "NETBIRD_MGMT_API_ENDPOINT=http://localhost:33073" > "$SCT_TEMPLATEDIR/netbird-server/dashboard.env" + echo "# template" > "$SCT_TEMPLATEDIR/netbird-server/README.md" + export NETBIRD_ENCRYPTION_KEY="user-provided-key-base64==" + + provision_netbird_server_template + + grep -q '^NETBIRD_ENCRYPTION_KEY=user-provided-key-base64==' "$HOME/.config/sandcat/netbird-server/netbird-server.env" + [[ "$(yq -r '.server.store.encryptionKey' "$HOME/.config/sandcat/netbird-server/config.yaml")" == "user-provided-key-base64==" ]] +} + @test "provision_netbird_server_template skips when destination exists as file" { export SCT_TEMPLATEDIR="$BATS_TEST_TMPDIR/templates" mkdir -p "$SCT_TEMPLATEDIR/netbird-server" echo "services: {}" > "$SCT_TEMPLATEDIR/netbird-server/docker-compose.yml" echo "NETBIRD_SERVER_VERSION=0.1.0" > "$SCT_TEMPLATEDIR/netbird-server/netbird-server.env" - echo "{}" > "$SCT_TEMPLATEDIR/netbird-server/management.json" - echo "listening-port=3478" > "$SCT_TEMPLATEDIR/netbird-server/turnserver.conf" + echo "server: {}" > "$SCT_TEMPLATEDIR/netbird-server/config.yaml" + echo "NETBIRD_MGMT_API_ENDPOINT=http://localhost:33073" > "$SCT_TEMPLATEDIR/netbird-server/dashboard.env" echo "# template" > "$SCT_TEMPLATEDIR/netbird-server/README.md" mkdir -p "$HOME/.config/sandcat" echo "existing-file" > "$HOME/.config/sandcat/netbird-server" @@ -131,12 +207,12 @@ teardown() { mkdir -p "$SCT_TEMPLATEDIR/netbird-server" echo "services: {}" > "$SCT_TEMPLATEDIR/netbird-server/docker-compose.yml" echo "NETBIRD_SERVER_VERSION=0.1.0" > "$SCT_TEMPLATEDIR/netbird-server/netbird-server.env" - echo "{}" > "$SCT_TEMPLATEDIR/netbird-server/management.json" + echo "NETBIRD_MGMT_API_ENDPOINT=http://localhost:33073" > "$SCT_TEMPLATEDIR/netbird-server/dashboard.env" echo "# template" > "$SCT_TEMPLATEDIR/netbird-server/README.md" run provision_netbird_server_template assert_failure - assert_output --partial "missing turnserver.conf" + assert_output --partial "missing config.yaml" } @test "netbird_api reads netbird_api_token from user settings" { diff --git a/cli/test/wg-client/netbird_daemon.bats b/cli/test/wg-client/netbird_daemon.bats index 453d183a..31f3f0b1 100644 --- a/cli/test/wg-client/netbird_daemon.bats +++ b/cli/test/wg-client/netbird_daemon.bats @@ -26,6 +26,62 @@ teardown() { assert_success } +@test "netbird_management_url_port extracts explicit port" { + run netbird_management_url_port "http://192.168.5.2:33073" + assert_output "33073" +} + +@test "netbird_management_url_port defaults http to 80" { + run netbird_management_url_port "http://192.168.5.2" + assert_output "80" +} + +@test "netbird_management_url_host extracts IPv4 host" { + run netbird_management_url_host "http://192.168.5.2:33073" + assert_output "192.168.5.2" +} + +@test "netbird_management_url_host_is_literal_ipv4 accepts IPv4" { + run netbird_management_url_host_is_literal_ipv4 "192.168.5.2" + assert_success +} + +@test "netbird_management_url_host_is_literal_ipv4 rejects hostnames" { + run netbird_management_url_host_is_literal_ipv4 "netbird.example.com" + assert_failure +} + +@test "netbird_host_route_uses_gateway for off-subnet Colima host" { + run netbird_host_route_uses_gateway "192.168.5.2" "172.23.0.1" + assert_success +} + +@test "netbird_host_route_uses_gateway is false when host is bridge gateway" { + run netbird_host_route_uses_gateway "172.23.0.1" "172.23.0.1" + assert_failure +} + +@test "configure_netbird_host_management_access is a no-op without setup key" { + unset NB_SETUP_KEY + export NB_MANAGEMENT_URL="http://192.168.5.2:33073" + run configure_netbird_host_management_access "172.17.0.1" + assert_success +} + +@test "configure_netbird_host_management_access is a no-op for cloud URL" { + export NB_SETUP_KEY="test-key" + export NB_MANAGEMENT_URL="https://api.netbird.io" + run configure_netbird_host_management_access "172.17.0.1" + assert_success +} + +@test "configure_netbird_host_management_access is a no-op for hostname URL" { + export NB_SETUP_KEY="test-key" + export NB_MANAGEMENT_URL="https://netbird.example.com" + run configure_netbird_host_management_access "172.17.0.1" + assert_success +} + @test "start_netbird is a no-op when NB_SETUP_KEY is unset" { unset NB_SETUP_KEY run start_netbird "$NETBIRD_IFACE" From 286b11d90f4a7e20b22e1b5bdb2a630b654aa3be Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Sat, 20 Jun 2026 08:42:05 +0000 Subject: [PATCH 015/138] working before cleanup --- cli/README.md | 10 ++++ cli/lib/composefile.bash | 1 + cli/lib/netbird.bash | 38 ++++++++++--- .../sandcat/scripts/wg-client-init.sh | 53 +++++++++++++++++++ cli/templates/netbird-server/README.md | 16 +++++- cli/test/netbird/netbird_settings.bats | 21 ++++++++ 6 files changed, 132 insertions(+), 7 deletions(-) diff --git a/cli/README.md b/cli/README.md index 14791f14..d72e27b0 100644 --- a/cli/README.md +++ b/cli/README.md @@ -364,6 +364,16 @@ Use that IP in `netbird_enrollment_management_url`, then recreate wg-client: sandcat run --force-recreate wg-client ``` +Sandcat also syncs `~/.config/sandcat/netbird-server/config.yaml` `exposedAddress` to +match `netbird_enrollment_management_url`. **Restart netbird-server** after the first +sync (or when you change the enrollment IP): + +```bash +cd ~/.config/sandcat/netbird-server +docker compose --env-file netbird-server.env up -d --force-recreate netbird-server +sandcat run --force-recreate wg-client +``` + ### Remote self-hosted (NetBird quickstart) For a VM with a public domain, sandcat does **not** run the installer. Use the diff --git a/cli/lib/composefile.bash b/cli/lib/composefile.bash index b56c1015..e67bc2fb 100644 --- a/cli/lib/composefile.bash +++ b/cli/lib/composefile.bash @@ -660,6 +660,7 @@ enable_netbird() { ) + ["NB_USE_LEGACY_ROUTING=true"] ' "$compose_file" fi + netbird_sync_local_server_exposed_address fi fi } diff --git a/cli/lib/netbird.bash b/cli/lib/netbird.bash index 37b2e5bb..99d52e39 100644 --- a/cli/lib/netbird.bash +++ b/cli/lib/netbird.bash @@ -41,13 +41,14 @@ netbird_read_setting() { # Export NB_SETUP_KEY from settings when not already set in the environment. # Used before docker compose so wg-client receives the enrollment key on create. export_netbird_compose_env() { - [[ -n "${NB_SETUP_KEY:-}" ]] && return 0 - - local enrollment_key - enrollment_key=$(netbird_read_setting netbird_enrollment_key) - if [[ -n "$enrollment_key" ]]; then - export NB_SETUP_KEY="$enrollment_key" + if [[ -z "${NB_SETUP_KEY:-}" ]]; then + local enrollment_key + enrollment_key=$(netbird_read_setting netbird_enrollment_key) + if [[ -n "$enrollment_key" ]]; then + export NB_SETUP_KEY="$enrollment_key" + fi fi + netbird_sync_local_server_exposed_address } # Export NB_MANAGEMENT_URL from settings when not already set in environment. @@ -95,6 +96,31 @@ netbird_enrollment_url_uses_host_bypass() { [[ "$url" =~ ^https?://([0-9]{1,3}\.){3}[0-9]{1,3}([:/]|$) ]] } +# Aligns netbird-server exposedAddress with netbird_enrollment_management_url so +# enrolled peers keep dialing the Docker-host IP instead of localhost. +netbird_sync_local_server_exposed_address() { + local enrollment_url config_file dest_dir current + + enrollment_url=$(netbird_read_setting netbird_enrollment_management_url) + [[ -n "$enrollment_url" ]] || return 0 + netbird_enrollment_url_uses_host_bypass "$enrollment_url" || return 0 + + dest_dir="$(sct_home)/netbird-server" + config_file="$dest_dir/config.yaml" + [[ -f "$config_file" ]] || return 0 + + require yq + current=$(yq -r '.server.exposedAddress // ""' "$config_file") + if [[ "$current" == "$enrollment_url" ]]; then + return 0 + fi + + NB_EXPOSED="$enrollment_url" \ + yq -i '.server.exposedAddress = strenv(NB_EXPOSED)' "$config_file" + echo "Updated netbird-server exposedAddress to $enrollment_url." | info + echo "Restart netbird-server: cd $(sct_home)/netbird-server && docker compose --env-file netbird-server.env up -d --force-recreate netbird-server" | info +} + # Resolves the NetBird embedded-IdP encryption key. # NETBIRD_ENCRYPTION_KEY env wins; otherwise generates a new key. _netbird_resolve_encryption_key() { diff --git a/cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh b/cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh index b951f2d1..9f7ceb48 100644 --- a/cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh +++ b/cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh @@ -446,6 +446,48 @@ netbird_verify_host_management_reachable() { curl -sf --max-time 5 "$check_url" >/dev/null } +# Writes /var/lib/netbird/default.json so the daemon keeps the enrollment URL +# instead of switching to localhost after the server registers the peer. +netbird_prepare_local_management_profile() { + local mgmt_url="${NB_MANAGEMENT_URL:-}" + local host profile_file="/var/lib/netbird/default.json" + + [[ -n "${NB_SETUP_KEY:-}" ]] || return 0 + host=$(netbird_management_url_host "$mgmt_url") || return 0 + netbird_management_url_host_is_literal_ipv4 "$host" || return 0 + + mkdir -p "$(dirname "$profile_file")" + if [[ -f "$profile_file" ]] && command -v jq >/dev/null 2>&1; then + local tmp + tmp=$(mktemp) + jq --arg url "$mgmt_url" '.ManagementURL = $url | .AdminURL = $url' "$profile_file" >"$tmp" + mv "$tmp" "$profile_file" + return 0 + fi + + cat >"$profile_file" </dev/null)"; then + export NB_USE_LEGACY_ROUTING=true + fi +} + netbird_daemon_ready() { netbird status >/dev/null 2>&1 } @@ -453,6 +495,8 @@ netbird_daemon_ready() { # Starts the NetBird background service (required for client 0.28+). ensure_netbird_service() { [[ -n "${NB_SETUP_KEY:-}" ]] || return 0 + netbird_prepare_local_management_profile + netbird_export_service_env if netbird_daemon_ready; then return 0 fi @@ -480,6 +524,12 @@ start_netbird() { ensure_netbird_service configure_netbird_host_management_access "$docker_gateway" netbird_verify_host_management_reachable + netbird_prepare_local_management_profile + if [[ -f /var/lib/netbird/default.json ]] \ + && grep -qE 'localhost|127\.0\.0\.1|\[::1\]' /var/lib/netbird/default.json 2>/dev/null; then + netbird down 2>/dev/null || true + fi + netbird_export_service_env echo "[wg-client] Enrolling NetBird peer on ${iface}." >&2 netbird up \ @@ -517,6 +567,7 @@ supervise_netbird_daemon() { sleep 10 if ! netbird_daemon_ready; then echo "[wg-client] NetBird service daemon not responding; restarting." >&2 + netbird_export_service_env netbird service run --log-file console & wait_until 15 1 \ "[wg-client] Timed out waiting for NetBird service daemon" \ @@ -526,6 +577,8 @@ supervise_netbird_daemon() { echo "[wg-client] NetBird interface ${iface} down; re-enrolling." >&2 docker_gateway=$(ip -4 route show default dev eth0 2>/dev/null | awk '{print $3}') configure_netbird_host_management_access "$docker_gateway" + netbird_prepare_local_management_profile + netbird_export_service_env netbird up \ --setup-key "${NB_SETUP_KEY}" \ --management-url "${NB_MANAGEMENT_URL:-https://api.netbird.io}" \ diff --git a/cli/templates/netbird-server/README.md b/cli/templates/netbird-server/README.md index fcd20a85..0e659bb4 100644 --- a/cli/templates/netbird-server/README.md +++ b/cli/templates/netbird-server/README.md @@ -68,10 +68,24 @@ Host tools and the browser use `localhost`. wg-client needs the Docker host IP ``` Replace `192.168.5.2` with your Docker host IP (`colima status -j` on Colima). -Then recreate wg-client: `sandcat run --force-recreate wg-client`. +Then restart **both** netbird-server and wg-client (server `exposedAddress` must match +the enrollment URL or peers dial `localhost` / `[::1]` inside the container): + +```bash +cd ~/.config/sandcat/netbird-server +docker compose --env-file netbird-server.env up -d --force-recreate netbird-server +sandcat run --force-recreate wg-client +``` ## Troubleshooting +### wg-client dials `[::1]:33073` after enrollment + +The management server was still advertising `http://localhost:33073` in +`config.yaml` `server.exposedAddress`. Set `netbird_enrollment_management_url` to your +Docker host IP, run `sandcat init` or `sandcat run` once (syncs `exposedAddress`), then +restart netbird-server and recreate wg-client as above. + ### Port clash with sandcat devcontainers The default ports **8080** (dashboard) and **33073** (management API) may diff --git a/cli/test/netbird/netbird_settings.bats b/cli/test/netbird/netbird_settings.bats index 1575b3e8..eaa55396 100644 --- a/cli/test/netbird/netbird_settings.bats +++ b/cli/test/netbird/netbird_settings.bats @@ -113,6 +113,27 @@ teardown() { assert_failure } +@test "netbird_sync_local_server_exposed_address updates config.yaml from enrollment URL" { + echo '{"netbird_enrollment_management_url": "http://192.168.5.2:33073"}' > "$HOME/.config/sandcat/settings.json" + mkdir -p "$HOME/.config/sandcat/netbird-server" + echo 'server: { exposedAddress: "http://localhost:33073" }' > "$HOME/.config/sandcat/netbird-server/config.yaml" + + netbird_sync_local_server_exposed_address + + run yq -r '.server.exposedAddress' "$HOME/.config/sandcat/netbird-server/config.yaml" + assert_output "http://192.168.5.2:33073" +} + +@test "netbird_sync_local_server_exposed_address is a no-op when already aligned" { + echo '{"netbird_enrollment_management_url": "http://192.168.5.2:33073"}' > "$HOME/.config/sandcat/settings.json" + mkdir -p "$HOME/.config/sandcat/netbird-server" + echo 'server: { exposedAddress: "http://192.168.5.2:33073" }' > "$HOME/.config/sandcat/netbird-server/config.yaml" + + run netbird_sync_local_server_exposed_address + assert_success + refute_output --partial "Updated netbird-server exposedAddress" +} + @test "provision_netbird_server_template copies template and skips when already provisioned" { export SCT_TEMPLATEDIR="$BATS_TEST_TMPDIR/templates" mkdir -p "$SCT_TEMPLATEDIR/netbird-server" From 15ce9dd0f0b767cbfeddc3923c4954cb36b7cb6b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 11:19:55 +0000 Subject: [PATCH 016/138] feat(netbird): add server lifecycle commands and harden API client MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add `sandcat netbird server start|stop|status` as a thin wrapper around the provisioned self-hosted stack in ~/.config/sandcat/netbird-server. Improve API error handling for NetBird’s misleading 404-on-invalid-token responses, keep settings-sourced PATs in a local variable instead of exporting NB_API_TOKEN, and guard jq pretty-printing when the response is not valid JSON. --- cli/README.md | 11 +- cli/lib/netbird.bash | 182 ++++++++++++++++++++++--- cli/libexec/init/init | 2 +- cli/libexec/netbird/_ | 6 - cli/libexec/netbird/netbird | 32 ++++- cli/templates/netbird-server/README.md | 6 + cli/test/init/init.bats | 2 +- cli/test/netbird/netbird.bats | 22 ++- cli/test/netbird/netbird_api.bats | 44 +++++- cli/test/netbird/netbird_server.bats | 74 ++++++++++ cli/test/netbird/netbird_settings.bats | 18 ++- 11 files changed, 356 insertions(+), 43 deletions(-) delete mode 100755 cli/libexec/netbird/_ create mode 100644 cli/test/netbird/netbird_server.bats diff --git a/cli/README.md b/cli/README.md index d72e27b0..ac035e56 100644 --- a/cli/README.md +++ b/cli/README.md @@ -328,8 +328,7 @@ For development on your machine without a public domain: ```bash sandcat init --netbird --netbird-server new ... -cd ~/.config/sandcat/netbird-server -docker compose --env-file netbird-server.env up -d +sandcat netbird server start ``` 1. Bootstrap admin: `POST http://localhost:33073/api/setup` (see @@ -369,8 +368,7 @@ match `netbird_enrollment_management_url`. **Restart netbird-server** after the sync (or when you change the enrollment IP): ```bash -cd ~/.config/sandcat/netbird-server -docker compose --env-file netbird-server.env up -d --force-recreate netbird-server +sandcat netbird server start --force-recreate netbird-server sandcat run --force-recreate wg-client ``` @@ -410,6 +408,11 @@ or edit `~/.config/sandcat/settings.json` directly, then `sandcat run`. ### Runtime control ```bash +# Local self-hosted server (after sandcat init --netbird-server new) +sandcat netbird server start +sandcat netbird server status +sandcat netbird server stop + # List current peers sandcat netbird status diff --git a/cli/lib/netbird.bash b/cli/lib/netbird.bash index 99d52e39..f5d7aefc 100644 --- a/cli/lib/netbird.bash +++ b/cli/lib/netbird.bash @@ -118,7 +118,7 @@ netbird_sync_local_server_exposed_address() { NB_EXPOSED="$enrollment_url" \ yq -i '.server.exposedAddress = strenv(NB_EXPOSED)' "$config_file" echo "Updated netbird-server exposedAddress to $enrollment_url." | info - echo "Restart netbird-server: cd $(sct_home)/netbird-server && docker compose --env-file netbird-server.env up -d --force-recreate netbird-server" | info + echo "Restart netbird-server: sandcat netbird server start --force-recreate netbird-server" | info } # Resolves the NetBird embedded-IdP encryption key. @@ -279,14 +279,18 @@ provision_netbird_server_template() { _netbird_apply_local_server_config "$destination_dir" } -# Resolve NB_API_TOKEN from settings when unset. Env always wins. -_ensure_netbird_api_token() { - [[ -n "${NB_API_TOKEN:-}" ]] && return 0 +# Resolve API token: NB_API_TOKEN env wins over settings. Prints token on success. +# Does not export settings-sourced tokens (avoids leaking via child process environ). +_netbird_api_token() { + if [[ -n "${NB_API_TOKEN:-}" ]]; then + printf '%s' "$NB_API_TOKEN" + return 0 + fi local token token=$(netbird_read_setting netbird_api_token) if [[ -n "$token" ]]; then - export NB_API_TOKEN="$token" + printf '%s' "$token" return 0 fi @@ -297,8 +301,86 @@ _ensure_netbird_api_token() { return 1 } +# Normalize the management server base URL (no trailing slash or /api suffix). +# Paths passed to netbird_api already include /api/... +netbird_management_base_url() { + export_netbird_management_url + local base="${NB_MANAGEMENT_URL:-https://api.netbird.io}" + base="${base%/}" + if [[ "$base" == */api ]]; then + base="${base%/api}" + fi + printf '%s' "$base" +} + +# Returns 0 when the API response body indicates an invalid or missing token. +# NetBird self-hosted returns HTTP 404 with {"message":"invalid token: ..."}. +_netbird_api_body_indicates_auth_failure() { + local body=$1 + [[ -n "$body" ]] || return 1 + [[ "$body" =~ [Ii]nvalid[[:space:]_]+token ]] && return 0 + [[ "$body" =~ [Tt]oken.*not[[:space:]]+found ]] && return 0 + [[ "$body" =~ [Nn]ot[[:space:]]+authenticated ]] && return 0 + [[ "$body" =~ [Uu]nauthorized ]] && return 0 + return 1 +} + +_netbird_api_print_auth_hint() { + echo " Authentication failed — check netbird_api_token in $(sct_home)/settings.json" >&2 + echo " (or export NB_API_TOKEN). Create a Personal Access Token in the NetBird dashboard." >&2 +} + +# Prints actionable hints for failed NetBird API calls. +# Args: +# $1 - HTTP status code (000 for connection failure) +# $2 - HTTP method +# $3 - API path +# $4 - Full request URL +# $5 - Optional response body +_netbird_api_print_error() { + local http_code=$1 + local method=$2 + local path=$3 + local url=$4 + local body=${5:-} + + echo "NetBird API ${method} ${path} failed (HTTP ${http_code})." >&2 + echo " URL: ${url}" >&2 + + if _netbird_api_body_indicates_auth_failure "$body"; then + _netbird_api_print_auth_hint + [[ -n "$body" ]] && echo " Response: ${body}" >&2 + return 0 + fi + + case "$http_code" in + 000) + echo " Could not reach the management server." >&2 + if [[ "$(netbird_management_base_url)" =~ localhost|127\.0\.0\.1 ]]; then + echo " Start it with: sandcat netbird server start" >&2 + fi + ;; + 401|403) + _netbird_api_print_auth_hint + ;; + 404) + echo " Endpoint not found — check netbird_management_url in $(sct_home)/settings.json." >&2 + echo " Use the management API base URL, not the dashboard:" >&2 + echo " local template: http://localhost:33073 (not :8080)" >&2 + echo " cloud: (leave empty, defaults to https://api.netbird.io)" >&2 + echo " self-hosted: https:// (reverse proxy must forward /api to management)" >&2 + [[ -n "$body" ]] && echo " Response: ${body}" >&2 + ;; + *) + if [[ -n "$body" ]]; then + echo " Response: ${body}" >&2 + fi + ;; + esac +} + # Calls the NetBird management REST API. -# Requires NB_API_TOKEN (env or netbird_api_token in settings). +# Requires netbird_api_token in settings, or NB_API_TOKEN in the environment. # NB_MANAGEMENT_URL defaults to https://api.netbird.io. # Prints the response body on success; writes curl/API errors to stderr. # Args: @@ -309,25 +391,42 @@ netbird_api() { local method=$1 local path=$2 local body=${3:-} + local api_token - _ensure_netbird_api_token || return 1 - export_netbird_management_url + api_token=$(_netbird_api_token) || return 1 + + local base_url url + base_url=$(netbird_management_base_url) + url="${base_url}${path}" - local url="${NB_MANAGEMENT_URL:-https://api.netbird.io}${path}" - local -a args=(-sS -f -X "$method" - -H "Authorization: Token $NB_API_TOKEN" + local -a args=(-sS -X "$method" + -H "Authorization: Token $api_token" -H "Accept: application/json" -H "Content-Type: application/json") - [[ -n "$body" ]] && args+=(-d "$body") + args+=(-w $'\n%{http_code}') + + local raw http_code response + local stderr_file + stderr_file=$(mktemp) - local response - if ! response=$(curl "${args[@]}" "$url" 2>&1); then - echo "NetBird API ${method} ${path} failed: ${response}" >&2 + if ! raw=$(curl "${args[@]}" "$url" 2>"$stderr_file"); then + _netbird_api_print_error "000" "$method" "$path" "$url" "$(<"$stderr_file")" + rm -f "$stderr_file" return 1 fi + rm -f "$stderr_file" - printf '%s\n' "$response" + http_code=$(printf '%s' "$raw" | tail -n1) + response=$(printf '%s' "$raw" | sed '$d') + + if [[ "$http_code" =~ ^2 ]]; then + printf '%s\n' "$response" + return 0 + fi + + _netbird_api_print_error "$http_code" "$method" "$path" "$url" "$response" + return 1 } # Returns the current peer list from the NetBird management server. @@ -363,3 +462,54 @@ netbird_peer_remove() { local peer_id=$1 netbird_api "DELETE" "/api/peers/$peer_id" } + +# Returns the provisioned self-hosted NetBird server directory. +netbird_server_dir() { + printf '%s\n' "$(sct_home)/netbird-server" +} + +# Verifies the local netbird-server template was provisioned. +_ensure_netbird_server_provisioned() { + local server_dir compose_file env_file + server_dir=$(netbird_server_dir) + compose_file="$server_dir/docker-compose.yml" + env_file="$server_dir/netbird-server.env" + + if [[ ! -f "$compose_file" || ! -f "$env_file" ]]; then + echo "NetBird server not provisioned at $server_dir" | error + echo "Run: sandcat init --netbird --netbird-server new" >&2 + return 1 + fi +} + +# Runs docker compose in the provisioned netbird-server directory. +# Args: docker compose subcommand and options (e.g. up -d, down, ps) +netbird_server_compose() { + require docker + _ensure_netbird_server_provisioned || return 1 + + local server_dir compose_file env_file + server_dir=$(netbird_server_dir) + compose_file="$server_dir/docker-compose.yml" + env_file="$server_dir/netbird-server.env" + + docker compose -f "$compose_file" --env-file "$env_file" "$@" +} + +# Starts the provisioned self-hosted NetBird server stack. +# Remaining args are passed to docker compose (e.g. --force-recreate netbird-server). +netbird_server_start() { + netbird_sync_local_server_exposed_address + netbird_server_compose up -d "$@" +} + +# Stops the provisioned self-hosted NetBird server stack. +# Remaining args are passed to docker compose (e.g. -v to remove volumes). +netbird_server_stop() { + netbird_server_compose down "$@" +} + +# Shows container status for the provisioned self-hosted NetBird server stack. +netbird_server_status() { + netbird_server_compose ps +} diff --git a/cli/libexec/init/init b/cli/libexec/init/init index ea4f797c..f97a1661 100755 --- a/cli/libexec/init/init +++ b/cli/libexec/init/init @@ -23,7 +23,7 @@ source "$SCT_LIBDIR/netbird.bash" # Prints instructions for the local template provisioned under ~/.config/sandcat/netbird-server. _print_netbird_local_template_hint() { echo " Local template: ~/.config/sandcat/netbird-server/" | info - echo " Start: cd ~/.config/sandcat/netbird-server && docker compose --env-file netbird-server.env up -d" | info + echo " Start: sandcat netbird server start" | info echo " Dashboard: http://localhost:8080 (bootstrap via /api/setup first — see README.md there)" | info } diff --git a/cli/libexec/netbird/_ b/cli/libexec/netbird/_ deleted file mode 100755 index 4866a695..00000000 --- a/cli/libexec/netbird/_ +++ /dev/null @@ -1,6 +0,0 @@ -#!/usr/bin/env bash - -# Catch subcommands (status, peer, route) and forward them to the netbird -# dispatcher. Without this, `sandcat netbird status` looks for libexec/netbird/status. - -exec netbird "$@" diff --git a/cli/libexec/netbird/netbird b/cli/libexec/netbird/netbird index f4046740..16a35d87 100755 --- a/cli/libexec/netbird/netbird +++ b/cli/libexec/netbird/netbird @@ -12,12 +12,36 @@ Usage: sandcat netbird [options] Subcommands: status List peers from the NetBird management server + server start Start the local self-hosted NetBird server stack + server stop Stop the local self-hosted NetBird server stack + server status Show docker compose status for the local server peer remove --peer-id Remove a peer; wg-client drops the route from wt0 route add --network --peer-id Add a network route route remove --route-id Remove a route by ID EOF } +cmd_server() { + local subcmd="${1:-}" + shift || true + case "$subcmd" in + start) + netbird_server_start "$@" + ;; + stop) + netbird_server_stop "$@" + ;; + status) + netbird_server_status + ;; + *) + echo "Unknown server subcommand: ${subcmd:-}" | error + usage + return 1 + ;; + esac +} + cmd_status() { local peers peers=$(netbird_status) || return 1 @@ -28,7 +52,12 @@ cmd_status() { fi if command -v jq &>/dev/null; then - echo "$peers" | jq . + if echo "$peers" | jq -e . >/dev/null 2>&1; then + echo "$peers" | jq . + else + echo "$peers" + echo "NetBird API returned non-JSON output; showing raw response." | warning + fi else echo "$peers" fi @@ -142,6 +171,7 @@ main() { shift || true case "$subcmd" in status) cmd_status "$@" ;; + server) cmd_server "$@" ;; peer) cmd_peer "$@" ;; route) cmd_route "$@" ;; *) diff --git a/cli/templates/netbird-server/README.md b/cli/templates/netbird-server/README.md index 0e659bb4..101987c4 100644 --- a/cli/templates/netbird-server/README.md +++ b/cli/templates/netbird-server/README.md @@ -13,6 +13,12 @@ instead (see `cli/README.md`). ## 1. Start the stack +```bash +sandcat netbird server start +``` + +Or manually: + ```bash cd ~/.config/sandcat/netbird-server docker compose --env-file netbird-server.env up -d diff --git a/cli/test/init/init.bats b/cli/test/init/init.bats index 8ac3f9f6..702afcfc 100644 --- a/cli/test/init/init.bats +++ b/cli/test/init/init.bats @@ -417,7 +417,7 @@ EOF assert_success assert_output --partial "Management server: http://localhost:33073" assert_output --partial "Local template: ~/.config/sandcat/netbird-server/" - assert_output --partial "docker compose --env-file netbird-server.env up -d" + assert_output --partial "sandcat netbird server start" assert_output --partial "http://localhost:8080" run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" assert_output "http://localhost:33073" diff --git a/cli/test/netbird/netbird.bats b/cli/test/netbird/netbird.bats index b6937f21..cc8e1b5a 100644 --- a/cli/test/netbird/netbird.bats +++ b/cli/test/netbird/netbird.bats @@ -14,18 +14,19 @@ teardown() { @test "netbird status calls GET /api/peers" { stub curl \ - "-sS -f -X GET -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[]'" + "-sS -X GET -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://api.netbird.io/api/peers : printf '%s\n200' '[]'" run bash "$NETBIRD_CMD" status assert_success assert_output --partial "No peers registered" } -@test "netbird status prints peer list when peers exist" { +@test "netbird status pretty-prints JSON with jq" { stub curl \ - "-sS -f -X GET -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[{\"id\":\"peer1\",\"name\":\"wg-client\"}]'" + "-sS -X GET -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://api.netbird.io/api/peers : printf '%s\n200' '[{\"id\":\"peer1\",\"name\":\"wg-client\"}]'" run bash "$NETBIRD_CMD" status assert_success assert_output --partial "peer1" + refute_output --partial "ARGS:" } @test "netbird peer remove requires --peer-id" { @@ -42,7 +43,7 @@ teardown() { @test "netbird peer remove calls netbird_peer_remove" { stub curl \ - "-sS -f -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers/abc123 : :" + "-sS -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://api.netbird.io/api/peers/abc123 : printf '\n200'" run bash "$NETBIRD_CMD" peer remove --peer-id abc123 assert_success } @@ -73,7 +74,7 @@ teardown() { @test "netbird route add calls netbird_route_add" { stub curl \ - "-sS -f -X POST -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"abc123\",\"enabled\":true}' https://api.netbird.io/api/routes : echo '{\"id\":\"route1\"}'" + "-sS -X POST -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"abc123\",\"enabled\":true}' -w * https://api.netbird.io/api/routes : printf '%s\n200' '{\"id\":\"route1\"}'" run bash "$NETBIRD_CMD" route add --network 10.8.0.0/24 --peer-id abc123 assert_success assert_output --partial "route1" @@ -93,7 +94,7 @@ teardown() { @test "netbird route remove calls netbird_route_remove" { stub curl \ - "-sS -f -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/routes/route1 : :" + "-sS -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://api.netbird.io/api/routes/route1 : printf '\n200'" run bash "$NETBIRD_CMD" route remove --route-id route1 assert_success } @@ -104,9 +105,16 @@ teardown() { assert_output --partial "Usage" } +@test "netbird server without subcommand prints usage and fails" { + run bash "$NETBIRD_CMD" server + assert_failure + assert_output --partial "Unknown server subcommand" +} + @test "sandcat netbird status routes subcommand through module dispatcher" { + printf 'test\n' > "$SCT_ROOT/.version" stub curl \ - "-sS -f -X GET -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[]'" + "-sS -X GET -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://api.netbird.io/api/peers : printf '%s\n200' '[]'" run bash "$SCT_ROOT/bin/sandcat" netbird status assert_success } diff --git a/cli/test/netbird/netbird_api.bats b/cli/test/netbird/netbird_api.bats index 926bc83c..acedb18e 100644 --- a/cli/test/netbird/netbird_api.bats +++ b/cli/test/netbird/netbird_api.bats @@ -23,7 +23,7 @@ teardown() { @test "netbird_status calls GET /api/peers" { stub curl \ - "-sS -f -X GET -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[{\"id\":\"peer1\",\"connected\":true}]'" + "-sS -X GET -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://api.netbird.io/api/peers : printf '%s\n200' '[{\"id\":\"peer1\",\"connected\":true}]'" run netbird_status assert_success assert_output --partial "peer1" @@ -31,21 +31,57 @@ teardown() { @test "netbird_route_add calls POST /api/routes with network and peer" { stub curl \ - "-sS -f -X POST -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"peer1\",\"enabled\":true}' https://api.netbird.io/api/routes : echo '{\"id\":\"route1\"}'" + "-sS -X POST -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"peer1\",\"enabled\":true}' -w * https://api.netbird.io/api/routes : printf '%s\n200' '{\"id\":\"route1\"}'" run netbird_route_add "10.8.0.0/24" "peer1" assert_success } @test "netbird_route_remove calls DELETE /api/routes/:id" { stub curl \ - "-sS -f -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/routes/route1 : :" + "-sS -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://api.netbird.io/api/routes/route1 : printf '\n200'" run netbird_route_remove "route1" assert_success } @test "netbird_peer_remove calls DELETE /api/peers/:id" { stub curl \ - "-sS -f -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers/peer1 : :" + "-sS -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://api.netbird.io/api/peers/peer1 : printf '\n200'" run netbird_peer_remove "peer1" assert_success } + +@test "netbird_api reports authentication hint on HTTP 401" { + stub curl \ + "-sS -X GET -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://api.netbird.io/api/peers : printf '%s\n401' 'unauthorized'" + run netbird_api "GET" "/api/peers" + assert_failure + assert_output --partial "Authentication failed" + assert_output --partial "netbird_api_token" +} + +@test "netbird_api reports authentication hint on HTTP 404 with invalid token body" { + stub curl \ + "-sS -X GET -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * http://localhost:33073/api/peers : printf '%s\n404' '{\"message\":\"invalid token: pat: *** not found\",\"code\":404}'" + export NB_MANAGEMENT_URL="http://localhost:33073" + run netbird_api "GET" "/api/peers" + assert_failure + assert_output --partial "Authentication failed" + assert_output --partial "netbird_api_token" + refute_output --partial "Endpoint not found" +} + +@test "netbird_api reports management URL hint on HTTP 404" { + stub curl \ + "-sS -X GET -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * http://localhost:8080/api/peers : printf '%s\n404' 'not found'" + export NB_MANAGEMENT_URL="http://localhost:8080" + run netbird_api "GET" "/api/peers" + assert_failure + assert_output --partial "Endpoint not found" + assert_output --partial "33073" +} + +@test "netbird_management_base_url strips trailing /api suffix" { + export NB_MANAGEMENT_URL="http://localhost:33073/api" + run netbird_management_base_url + assert_output "http://localhost:33073" +} diff --git a/cli/test/netbird/netbird_server.bats b/cli/test/netbird/netbird_server.bats new file mode 100644 index 00000000..b2919ab0 --- /dev/null +++ b/cli/test/netbird/netbird_server.bats @@ -0,0 +1,74 @@ +#!/usr/bin/env bats + +setup() { + load test_helper + + NETBIRD_CMD="$SCT_LIBEXECDIR/netbird/netbird" + export HOME="$BATS_TEST_TMPDIR/home" + SCT_HOME_DIR="$HOME/.config/sandcat" + mkdir -p "$SCT_HOME_DIR/netbird-server" + touch "$SCT_HOME_DIR/netbird-server/docker-compose.yml" + touch "$SCT_HOME_DIR/netbird-server/netbird-server.env" +} + +teardown() { + unstub_all +} + +@test "netbird server start runs docker compose up -d" { + stub docker \ + "compose -f $SCT_HOME_DIR/netbird-server/docker-compose.yml --env-file $SCT_HOME_DIR/netbird-server/netbird-server.env up -d : :" + + run bash "$NETBIRD_CMD" server start + assert_success +} + +@test "netbird server start forwards extra compose args" { + stub docker \ + "compose -f $SCT_HOME_DIR/netbird-server/docker-compose.yml --env-file $SCT_HOME_DIR/netbird-server/netbird-server.env up -d --force-recreate netbird-server : :" + + run bash "$NETBIRD_CMD" server start --force-recreate netbird-server + assert_success +} + +@test "netbird server stop runs docker compose down" { + stub docker \ + "compose -f $SCT_HOME_DIR/netbird-server/docker-compose.yml --env-file $SCT_HOME_DIR/netbird-server/netbird-server.env down : :" + + run bash "$NETBIRD_CMD" server stop + assert_success +} + +@test "netbird server status runs docker compose ps" { + stub docker \ + "compose -f $SCT_HOME_DIR/netbird-server/docker-compose.yml --env-file $SCT_HOME_DIR/netbird-server/netbird-server.env ps : echo 'NAME STATUS'" + + run bash "$NETBIRD_CMD" server status + assert_success + assert_output --partial "NAME STATUS" +} + +@test "netbird server start fails when template is not provisioned" { + rm -rf "$SCT_HOME_DIR/netbird-server" + + run bash "$NETBIRD_CMD" server start + assert_failure + assert_output --partial "not provisioned" + assert_output --partial "sandcat init --netbird --netbird-server new" +} + +@test "netbird server with unknown subcommand prints usage" { + run bash "$NETBIRD_CMD" server bogus + assert_failure + assert_output --partial "Unknown server subcommand" + assert_output --partial "Usage" +} + +@test "sandcat netbird server start routes through module dispatcher" { + printf 'test\n' > "$SCT_ROOT/.version" + stub docker \ + "compose -f $SCT_HOME_DIR/netbird-server/docker-compose.yml --env-file $SCT_HOME_DIR/netbird-server/netbird-server.env up -d : :" + + run bash "$SCT_ROOT/bin/sandcat" netbird server start + assert_success +} diff --git a/cli/test/netbird/netbird_settings.bats b/cli/test/netbird/netbird_settings.bats index eaa55396..8ec46ae4 100644 --- a/cli/test/netbird/netbird_settings.bats +++ b/cli/test/netbird/netbird_settings.bats @@ -242,7 +242,7 @@ teardown() { export NB_MANAGEMENT_URL="https://api.netbird.io" stub curl \ - "-sS -f -X GET -H 'Authorization: Token settings-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[]'" + "-sS -X GET -H 'Authorization: Token settings-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://api.netbird.io/api/peers : printf '%s\n200' '[]'" run netbird_api "GET" "/api/peers" assert_success @@ -254,7 +254,7 @@ teardown() { unset NB_MANAGEMENT_URL stub curl \ - "-sS -f -X GET -H 'Authorization: Token settings-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://management.settings.example.com/api/peers : echo '[]'" + "-sS -X GET -H 'Authorization: Token settings-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://management.settings.example.com/api/peers : printf '%s\n200' '[]'" run netbird_api "GET" "/api/peers" assert_success @@ -266,8 +266,20 @@ teardown() { export NB_MANAGEMENT_URL="https://api.netbird.io" stub curl \ - "-sS -f -X GET -H 'Authorization: Token env-token' -H 'Accept: application/json' -H 'Content-Type: application/json' https://api.netbird.io/api/peers : echo '[]'" + "-sS -X GET -H 'Authorization: Token env-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://api.netbird.io/api/peers : printf '%s\n200' '[]'" run netbird_api "GET" "/api/peers" assert_success } + +@test "netbird_api does not export token read from settings" { + echo '{"netbird_api_token": "settings-token"}' > "$HOME/.config/sandcat/settings.json" + unset NB_API_TOKEN + export NB_MANAGEMENT_URL="https://api.netbird.io" + + stub curl \ + "-sS -X GET -H 'Authorization: Token settings-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://api.netbird.io/api/peers : printf '%s\n200' '[]'" + + netbird_api "GET" "/api/peers" + [[ -z "${NB_API_TOKEN:-}" ]] +} From 7db49a5cd6229009fb253b599dc706ca65acb84e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 11:28:39 +0000 Subject: [PATCH 017/138] feat(capability-runtime): add project scaffold and v1 types Co-authored-by: Cursor --- capability-runtime/pyproject.toml | 8 ++ .../src/capability_runtime/__init__.py | 3 + .../src/capability_runtime/types.py | 88 +++++++++++++++++++ capability-runtime/tests/test_types.py | 45 ++++++++++ 4 files changed, 144 insertions(+) create mode 100644 capability-runtime/pyproject.toml create mode 100644 capability-runtime/src/capability_runtime/__init__.py create mode 100644 capability-runtime/src/capability_runtime/types.py create mode 100644 capability-runtime/tests/test_types.py diff --git a/capability-runtime/pyproject.toml b/capability-runtime/pyproject.toml new file mode 100644 index 00000000..8f01eef7 --- /dev/null +++ b/capability-runtime/pyproject.toml @@ -0,0 +1,8 @@ +[project] +name = "capability-runtime" +version = "0.1.0" +requires-python = ">=3.12" + +[tool.pytest.ini_options] +pythonpath = ["src"] +testpaths = ["tests"] diff --git a/capability-runtime/src/capability_runtime/__init__.py b/capability-runtime/src/capability_runtime/__init__.py new file mode 100644 index 00000000..35fdf704 --- /dev/null +++ b/capability-runtime/src/capability_runtime/__init__.py @@ -0,0 +1,3 @@ +from capability_runtime.types import CapabilityBundle + +__all__ = ["CapabilityBundle"] diff --git a/capability-runtime/src/capability_runtime/types.py b/capability-runtime/src/capability_runtime/types.py new file mode 100644 index 00000000..c9e2f1ee --- /dev/null +++ b/capability-runtime/src/capability_runtime/types.py @@ -0,0 +1,88 @@ +from __future__ import annotations + +from dataclasses import dataclass, field +from datetime import datetime, timedelta +from typing import Literal, Union + +Quota = Union[int, Literal["unbounded"]] + + +@dataclass(frozen=True) +class AgentIdentity: + value: str + + +@dataclass(frozen=True) +class CapabilityRef: + value: str + + +@dataclass(frozen=True) +class LeaseId: + value: str + + +@dataclass +class ToolCapability: + ref: CapabilityRef + name: str + lease_id: LeaseId | None + quota: Quota + expires_at: datetime | None + + +@dataclass +class RuleCapability: + ref: CapabilityRef + name: str + lease_id: LeaseId | None + + +@dataclass +class SkillCapability: + ref: CapabilityRef + name: str + lease_id: LeaseId | None + + +@dataclass +class PolicyCapability: + ref: CapabilityRef + name: str + lease_id: LeaseId | None + + +@dataclass +class HookCapability: + ref: CapabilityRef + name: str + lease_id: LeaseId | None + + +@dataclass +class BudgetEnvelope: + token_quota: Quota + action_quota: Quota + wall_time_ttl: timedelta | None + + +@dataclass +class ProvenanceRecord: + issuer: str + policy_version: str + trace_id: str + + +@dataclass +class CapabilityBundle: + agent_id: AgentIdentity + issued_at: datetime + expires_at: datetime | None + tools: list[ToolCapability] + rules: list[RuleCapability] + skills: list[SkillCapability] + policies: list[PolicyCapability] + hooks: list[HookCapability] + budgets: BudgetEnvelope + provenance: ProvenanceRecord + version: int = field(default=1) diff --git a/capability-runtime/tests/test_types.py b/capability-runtime/tests/test_types.py new file mode 100644 index 00000000..c5720f87 --- /dev/null +++ b/capability-runtime/tests/test_types.py @@ -0,0 +1,45 @@ +from datetime import datetime, timezone +from capability_runtime.types import ( + AgentIdentity, + BudgetEnvelope, + CapabilityBundle, + CapabilityRef, + ProvenanceRecord, + ToolCapability, +) + + +def test_capability_bundle_roundtrip(): + now = datetime(2026, 6, 22, 12, 0, 0, tzinfo=timezone.utc) + bundle = CapabilityBundle( + agent_id=AgentIdentity("agent-1"), + issued_at=now, + expires_at=None, + tools=[ + ToolCapability( + ref=CapabilityRef("cap-create-pr"), + name="create_pr", + lease_id=None, + quota="unbounded", + expires_at=None, + ) + ], + rules=[], + skills=[], + policies=[], + hooks=[], + budgets=BudgetEnvelope( + token_quota="unbounded", + action_quota="unbounded", + wall_time_ttl=None, + ), + provenance=ProvenanceRecord( + issuer="runtime-1", + policy_version="1.0.0", + trace_id="trace-abc", + ), + version=1, + ) + assert bundle.agent_id.value == "agent-1" + assert bundle.tools[0].name == "create_pr" + assert bundle.version == 1 From 780035f9ebab0fc31f1d53983735c2aab4914cd4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 11:33:01 +0000 Subject: [PATCH 018/138] =?UTF-8?q?feat(capability-runtime):=20add=20typed?= =?UTF-8?q?=20error=20taxonomy=20(spec=20=C2=A73.3)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Introduce CapabilityRuntimeError hierarchy with domain-specific exceptions for visibility, lease, and bundle version failures. --- .../src/capability_runtime/errors.py | 37 +++++++++++++++++++ capability-runtime/tests/test_errors.py | 19 ++++++++++ 2 files changed, 56 insertions(+) create mode 100644 capability-runtime/src/capability_runtime/errors.py create mode 100644 capability-runtime/tests/test_errors.py diff --git a/capability-runtime/src/capability_runtime/errors.py b/capability-runtime/src/capability_runtime/errors.py new file mode 100644 index 00000000..3da65204 --- /dev/null +++ b/capability-runtime/src/capability_runtime/errors.py @@ -0,0 +1,37 @@ +from capability_runtime.types import CapabilityRef, LeaseId + + +class CapabilityRuntimeError(Exception): + """Base for all typed runtime failures (spec §3.3).""" + + +class CapabilityNotVisible(CapabilityRuntimeError): + def __init__(self, capability_ref: CapabilityRef): + self.capability_ref = capability_ref + super().__init__(f"Capability not visible: {capability_ref.value}") + + +class CapabilityUnknown(CapabilityRuntimeError): + def __init__(self, capability_ref: CapabilityRef): + self.capability_ref = capability_ref + super().__init__(f"Unknown capability: {capability_ref.value}") + + +class LeaseExpired(CapabilityRuntimeError): + def __init__(self, lease_id: LeaseId): + self.lease_id = lease_id + super().__init__(f"Lease expired: {lease_id.value}") + + +class LeaseQuotaExceeded(CapabilityRuntimeError): + def __init__(self, capability_ref: CapabilityRef, lease_id: LeaseId): + self.capability_ref = capability_ref + self.lease_id = lease_id + super().__init__(f"Lease quota exceeded for {capability_ref.value}") + + +class BundleVersionMismatch(CapabilityRuntimeError): + def __init__(self, expected: int, actual: int): + self.expected = expected + self.actual = actual + super().__init__(f"Bundle version mismatch: expected {expected}, got {actual}") diff --git a/capability-runtime/tests/test_errors.py b/capability-runtime/tests/test_errors.py new file mode 100644 index 00000000..e4018d85 --- /dev/null +++ b/capability-runtime/tests/test_errors.py @@ -0,0 +1,19 @@ +import pytest +from capability_runtime.errors import ( + CapabilityNotVisible, + CapabilityUnknown, + LeaseExpired, + LeaseQuotaExceeded, + BundleVersionMismatch, +) +from capability_runtime.types import CapabilityRef + + +def test_error_hierarchy(): + assert issubclass(CapabilityNotVisible, Exception) + assert issubclass(LeaseQuotaExceeded, Exception) + + +def test_capability_not_visible_carries_ref(): + err = CapabilityNotVisible(CapabilityRef("cap-x")) + assert err.capability_ref.value == "cap-x" From 11d418b17286c842d801fe2d3a7341af02f42842 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 11:33:59 +0000 Subject: [PATCH 019/138] feat(capability-runtime): add observability collector with replay --- .../src/capability_runtime/observability.py | 43 +++++++++++++++++++ .../tests/test_observability.py | 23 ++++++++++ 2 files changed, 66 insertions(+) create mode 100644 capability-runtime/src/capability_runtime/observability.py create mode 100644 capability-runtime/tests/test_observability.py diff --git a/capability-runtime/src/capability_runtime/observability.py b/capability-runtime/src/capability_runtime/observability.py new file mode 100644 index 00000000..df0f9c16 --- /dev/null +++ b/capability-runtime/src/capability_runtime/observability.py @@ -0,0 +1,43 @@ +from __future__ import annotations + +import json +from dataclasses import dataclass, field +from datetime import datetime, timezone +from pathlib import Path +from typing import Any + + +@dataclass +class ObservabilityCollector: + """Records execution and capability events for replay (spec §7).""" + + trace_file: Path + trace_id: str + seed: int + _events: list[dict[str, Any]] = field(default_factory=list, init=False) + + def _append(self, kind: str, event: dict[str, Any]) -> None: + record = { + "kind": kind, + "trace_id": self.trace_id, + "seed": self.seed, + "timestamp": datetime.now(timezone.utc).isoformat(), + **event, + } + self._events.append(record) + with self.trace_file.open("a") as f: + f.write(json.dumps(record) + "\n") + + def emit_execution_event(self, event: dict[str, Any]) -> None: + self._append("execution", event) + + def emit_capability_event(self, event: dict[str, Any]) -> None: + self._append("capability", event) + + def replay(self, seed: int) -> list[dict[str, Any]]: + if seed != self.seed: + return [] + if self.trace_file.exists(): + lines = self.trace_file.read_text().strip().splitlines() + return [json.loads(line) for line in lines if line] + return list(self._events) diff --git a/capability-runtime/tests/test_observability.py b/capability-runtime/tests/test_observability.py new file mode 100644 index 00000000..7751dfd6 --- /dev/null +++ b/capability-runtime/tests/test_observability.py @@ -0,0 +1,23 @@ +import json +from pathlib import Path +from capability_runtime.observability import ObservabilityCollector + + +def test_append_and_replay_capability_events(tmp_path: Path): + trace_file = tmp_path / "trace.jsonl" + obs = ObservabilityCollector(trace_file=trace_file, trace_id="trace-1", seed=42) + obs.emit_capability_event({"type": "bundle_issued", "agent_id": "agent-1"}) + obs.emit_capability_event({"type": "lease_granted", "lease_id": "lease-1"}) + events = obs.replay(seed=42) + assert len(events) == 2 + assert events[0]["type"] == "bundle_issued" + assert events[1]["lease_id"] == "lease-1" + + +def test_replay_is_deterministic(tmp_path: Path): + trace_file = tmp_path / "trace.jsonl" + obs = ObservabilityCollector(trace_file=trace_file, trace_id="trace-1", seed=99) + obs.emit_execution_event({"type": "action", "tool": "read_file"}) + first = obs.replay(seed=99) + second = obs.replay(seed=99) + assert first == second From 42f919407dec31913ea1ec59c7c03d086624570e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 11:35:21 +0000 Subject: [PATCH 020/138] feat(capability-runtime): add capability catalog and lifecycle states --- .../src/capability_runtime/catalog.py | 64 +++++++++++++++++++ capability-runtime/tests/test_catalog.py | 35 ++++++++++ 2 files changed, 99 insertions(+) create mode 100644 capability-runtime/src/capability_runtime/catalog.py create mode 100644 capability-runtime/tests/test_catalog.py diff --git a/capability-runtime/src/capability_runtime/catalog.py b/capability-runtime/src/capability_runtime/catalog.py new file mode 100644 index 00000000..33db76f0 --- /dev/null +++ b/capability-runtime/src/capability_runtime/catalog.py @@ -0,0 +1,64 @@ +from __future__ import annotations + +from enum import StrEnum + +from capability_runtime.errors import CapabilityUnknown +from capability_runtime.types import CapabilityRef + + +class LifecycleState(StrEnum): + DECLARED = "Declared" + DISCOVERABLE = "Discoverable" + VISIBLE = "Visible" + LEASED = "Leased" + EXPIRED = "Expired" + REVOKED = "Revoked" + ARCHIVED = "Archived" + + +class CapabilityCatalog: + def __init__(self) -> None: + self._by_ref: dict[CapabilityRef, LifecycleState] = {} + self._by_name: dict[str, CapabilityRef] = {} + self._name_by_ref: dict[CapabilityRef, str] = {} + + def register( + self, + name: str, + ref: CapabilityRef, + initial_state: LifecycleState = LifecycleState.DECLARED, + ) -> None: + self._by_name[name] = ref + self._name_by_ref[ref] = name + self._by_ref[ref] = initial_state + + def get_state(self, ref: CapabilityRef) -> LifecycleState: + try: + return self._by_ref[ref] + except KeyError: + raise CapabilityUnknown(ref) from None + + def set_state(self, ref: CapabilityRef, state: LifecycleState) -> None: + if ref not in self._by_ref: + raise CapabilityUnknown(ref) + self._by_ref[ref] = state + + def is_visible(self, ref: CapabilityRef) -> bool: + try: + state = self._by_ref[ref] + except KeyError: + return False + return state in (LifecycleState.VISIBLE, LifecycleState.LEASED) + + def is_discoverable(self, ref: CapabilityRef) -> bool: + try: + state = self._by_ref[ref] + except KeyError: + return False + return state == LifecycleState.DISCOVERABLE + + def get_by_name(self, name: str) -> CapabilityRef | None: + return self._by_name.get(name) + + def get_name(self, ref: CapabilityRef) -> str | None: + return self._name_by_ref.get(ref) diff --git a/capability-runtime/tests/test_catalog.py b/capability-runtime/tests/test_catalog.py new file mode 100644 index 00000000..3f867693 --- /dev/null +++ b/capability-runtime/tests/test_catalog.py @@ -0,0 +1,35 @@ +from capability_runtime.catalog import CapabilityCatalog, LifecycleState +from capability_runtime.types import CapabilityRef + + +def test_register_and_query_visibility(): + catalog = CapabilityCatalog() + ref = CapabilityRef("cap-create-pr") + catalog.register("create_pr", ref, initial_state=LifecycleState.DECLARED) + assert catalog.get_state(ref) == LifecycleState.DECLARED + catalog.set_state(ref, LifecycleState.VISIBLE) + assert catalog.is_visible(ref) + assert not catalog.is_visible(CapabilityRef("missing")) + + +def test_discoverable_not_visible(): + catalog = CapabilityCatalog() + ref = CapabilityRef("cap-secret") + catalog.register("secret_tool", ref, initial_state=LifecycleState.DISCOVERABLE) + assert catalog.is_discoverable(ref) + assert not catalog.is_visible(ref) + + +def test_leased_is_visible(): + catalog = CapabilityCatalog() + ref = CapabilityRef("cap-x") + catalog.register("tool_x", ref, initial_state=LifecycleState.LEASED) + assert catalog.is_visible(ref) + + +def test_get_by_name(): + catalog = CapabilityCatalog() + ref = CapabilityRef("cap-create-pr") + catalog.register("create_pr", ref) + assert catalog.get_by_name("create_pr") == ref + assert catalog.get_by_name("missing") is None From 5f96b2574b2f0ec327da30c45d8adf632988f22f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 11:36:31 +0000 Subject: [PATCH 021/138] feat(capability-runtime): add LeaseManager for grant and quota tracking Introduce LeaseDecision and LeaseManager to issue time-bounded leases with quota decrement, exhaustion checks, and expiry evaluation for PoC 1 flows. --- .../src/capability_runtime/lease.py | 64 +++++++++++++++++++ .../src/capability_runtime/types.py | 12 ++++ capability-runtime/tests/test_lease.py | 61 ++++++++++++++++++ 3 files changed, 137 insertions(+) create mode 100644 capability-runtime/src/capability_runtime/lease.py create mode 100644 capability-runtime/tests/test_lease.py diff --git a/capability-runtime/src/capability_runtime/lease.py b/capability-runtime/src/capability_runtime/lease.py new file mode 100644 index 00000000..f435cb41 --- /dev/null +++ b/capability-runtime/src/capability_runtime/lease.py @@ -0,0 +1,64 @@ +from __future__ import annotations + +from datetime import datetime, timedelta +from uuid import uuid4 + +from capability_runtime.errors import LeaseQuotaExceeded +from capability_runtime.types import ( + AgentIdentity, + CapabilityRef, + LeaseDecision, + LeaseId, +) + + +class LeaseManager: + def __init__(self) -> None: + self._leases: dict[LeaseId, LeaseDecision] = {} + self._remaining_quota: dict[LeaseId, int] = {} + + def grant( + self, + agent_id: AgentIdentity, + capability_ref: CapabilityRef, + quota: int, + ttl: timedelta, + token_budget: int, + risk_envelope: str, + now: datetime, + ) -> LeaseDecision: + lease_id = LeaseId(str(uuid4())) + decision = LeaseDecision( + lease_id=lease_id, + capability_ref=capability_ref, + agent_id=agent_id, + quota=quota, + token_budget=token_budget, + risk_envelope=risk_envelope, + expires_at=now + ttl, + granted_at=now, + ) + self._leases[lease_id] = decision + self._remaining_quota[lease_id] = quota + return decision + + def decrement_quota(self, lease_id: LeaseId, now: datetime) -> int: + decision = self._leases[lease_id] + remaining = self._remaining_quota[lease_id] + if remaining == 0: + raise LeaseQuotaExceeded(decision.capability_ref, lease_id) + remaining -= 1 + self._remaining_quota[lease_id] = remaining + return remaining + + def is_exhausted(self, lease_id: LeaseId) -> bool: + return self._remaining_quota.get(lease_id, 0) == 0 + + def is_expired(self, lease_id: LeaseId, now: datetime) -> bool: + decision = self._leases.get(lease_id) + if decision is None: + return True + return now >= decision.expires_at + + def get_lease(self, lease_id: LeaseId) -> LeaseDecision | None: + return self._leases.get(lease_id) diff --git a/capability-runtime/src/capability_runtime/types.py b/capability-runtime/src/capability_runtime/types.py index c9e2f1ee..b44cc501 100644 --- a/capability-runtime/src/capability_runtime/types.py +++ b/capability-runtime/src/capability_runtime/types.py @@ -22,6 +22,18 @@ class LeaseId: value: str +@dataclass +class LeaseDecision: + lease_id: LeaseId + capability_ref: CapabilityRef + agent_id: AgentIdentity + quota: int + token_budget: int + risk_envelope: str + expires_at: datetime + granted_at: datetime + + @dataclass class ToolCapability: ref: CapabilityRef diff --git a/capability-runtime/tests/test_lease.py b/capability-runtime/tests/test_lease.py new file mode 100644 index 00000000..45eb7c7f --- /dev/null +++ b/capability-runtime/tests/test_lease.py @@ -0,0 +1,61 @@ +from datetime import datetime, timedelta, timezone + +import pytest + +from capability_runtime.errors import LeaseQuotaExceeded +from capability_runtime.lease import LeaseManager +from capability_runtime.types import AgentIdentity, CapabilityRef + + +def test_grant_lease_and_decrement_quota(): + mgr = LeaseManager() + now = datetime(2026, 6, 22, 12, 0, 0, tzinfo=timezone.utc) + decision = mgr.grant( + agent_id=AgentIdentity("agent-1"), + capability_ref=CapabilityRef("cap-create-pr"), + quota=1, + ttl=timedelta(minutes=10), + token_budget=25000, + risk_envelope="high", + now=now, + ) + assert decision.quota == 1 + assert decision.token_budget == 25000 + assert decision.risk_envelope == "high" + assert decision.expires_at == now + timedelta(minutes=10) + remaining = mgr.decrement_quota(decision.lease_id, now=now) + assert remaining == 0 + assert mgr.is_exhausted(decision.lease_id) + + +def test_decrement_raises_when_exhausted(): + mgr = LeaseManager() + now = datetime(2026, 6, 22, 12, 0, 0, tzinfo=timezone.utc) + decision = mgr.grant( + agent_id=AgentIdentity("agent-1"), + capability_ref=CapabilityRef("cap-x"), + quota=1, + ttl=timedelta(minutes=5), + token_budget=1000, + risk_envelope="low", + now=now, + ) + mgr.decrement_quota(decision.lease_id, now=now) + with pytest.raises(LeaseQuotaExceeded): + mgr.decrement_quota(decision.lease_id, now=now) + + +def test_is_expired(): + mgr = LeaseManager() + start = datetime(2026, 6, 22, 12, 0, 0, tzinfo=timezone.utc) + decision = mgr.grant( + agent_id=AgentIdentity("agent-1"), + capability_ref=CapabilityRef("cap-x"), + quota=3, + ttl=timedelta(minutes=5), + token_budget=1000, + risk_envelope="medium", + now=start, + ) + assert not mgr.is_expired(decision.lease_id, start + timedelta(minutes=4)) + assert mgr.is_expired(decision.lease_id, start + timedelta(minutes=6)) From ba52973c5c8b5b08a877a7cad6b100d9a88ca4cd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 11:38:03 +0000 Subject: [PATCH 022/138] feat(capability-runtime): add revocation and discovery RevocationManager revokes by lease or ref with fail-closed catalog state. discover_capabilities returns metadata for Discoverable caps only. --- .../src/capability_runtime/discover.py | 44 ++++++++++++++ .../src/capability_runtime/revoke.py | 31 ++++++++++ capability-runtime/tests/test_discover.py | 57 +++++++++++++++++++ capability-runtime/tests/test_revoke.py | 44 ++++++++++++++ 4 files changed, 176 insertions(+) create mode 100644 capability-runtime/src/capability_runtime/discover.py create mode 100644 capability-runtime/src/capability_runtime/revoke.py create mode 100644 capability-runtime/tests/test_discover.py create mode 100644 capability-runtime/tests/test_revoke.py diff --git a/capability-runtime/src/capability_runtime/discover.py b/capability-runtime/src/capability_runtime/discover.py new file mode 100644 index 00000000..ddce6322 --- /dev/null +++ b/capability-runtime/src/capability_runtime/discover.py @@ -0,0 +1,44 @@ +from __future__ import annotations + +from dataclasses import dataclass + +from capability_runtime.catalog import CapabilityCatalog, LifecycleState +from capability_runtime.types import AgentIdentity + + +@dataclass +class DiscoveryIntent: + query: str + + +@dataclass +class DiscoveryResult: + capabilities: list[dict] + denied: bool + + +def discover_capabilities( + catalog: CapabilityCatalog, + agent_id: AgentIdentity, + intent: DiscoveryIntent, +) -> DiscoveryResult: + query = intent.query.lower() + matches: list[dict] = [] + + for ref, state in catalog._by_ref.items(): + if state != LifecycleState.DISCOVERABLE: + continue + name = catalog._name_by_ref.get(ref) + if name is None or query not in name.lower(): + continue + matches.append( + { + "name": name, + "ref": ref.value, + "description": "", + } + ) + + if not matches: + return DiscoveryResult(capabilities=[], denied=True) + return DiscoveryResult(capabilities=matches, denied=False) diff --git a/capability-runtime/src/capability_runtime/revoke.py b/capability-runtime/src/capability_runtime/revoke.py new file mode 100644 index 00000000..9b91912e --- /dev/null +++ b/capability-runtime/src/capability_runtime/revoke.py @@ -0,0 +1,31 @@ +from __future__ import annotations + +from capability_runtime.catalog import CapabilityCatalog, LifecycleState +from capability_runtime.errors import CapabilityUnknown +from capability_runtime.lease import LeaseManager +from capability_runtime.types import CapabilityRef, LeaseId + + +class RevocationManager: + def __init__(self, catalog: CapabilityCatalog, lease_manager: LeaseManager) -> None: + self._catalog = catalog + self._lease_manager = lease_manager + self._revoked_leases: set[LeaseId] = set() + + def revoke_by_lease(self, lease_id: LeaseId, reason: str) -> None: + self._revoked_leases.add(lease_id) + lease = self._lease_manager.get_lease(lease_id) + if lease is not None: + self._catalog.set_state(lease.capability_ref, LifecycleState.REVOKED) + + def revoke_by_ref(self, capability_ref: CapabilityRef, reason: str) -> None: + self._catalog.set_state(capability_ref, LifecycleState.REVOKED) + + def is_revoked(self, capability_ref: CapabilityRef) -> bool: + try: + return self._catalog.get_state(capability_ref) == LifecycleState.REVOKED + except CapabilityUnknown: + return False + + def is_lease_revoked(self, lease_id: LeaseId) -> bool: + return lease_id in self._revoked_leases diff --git a/capability-runtime/tests/test_discover.py b/capability-runtime/tests/test_discover.py new file mode 100644 index 00000000..48d1ea37 --- /dev/null +++ b/capability-runtime/tests/test_discover.py @@ -0,0 +1,57 @@ +from capability_runtime.catalog import CapabilityCatalog, LifecycleState +from capability_runtime.discover import DiscoveryIntent, discover_capabilities +from capability_runtime.types import AgentIdentity, CapabilityRef + + +def test_discoverable_capability_returns_metadata_when_query_matches(): + catalog = CapabilityCatalog() + ref = CapabilityRef("cap-create-pr") + catalog.register("create_pr", ref, initial_state=LifecycleState.DISCOVERABLE) + + result = discover_capabilities( + catalog, + AgentIdentity("agent-1"), + DiscoveryIntent(query="create"), + ) + + assert not result.denied + assert len(result.capabilities) == 1 + cap = result.capabilities[0] + assert cap["name"] == "create_pr" + assert cap["ref"] == "cap-create-pr" + assert "description" in cap + + +def test_declared_and_visible_capabilities_not_returned(): + catalog = CapabilityCatalog() + declared_ref = CapabilityRef("cap-secret") + visible_ref = CapabilityRef("cap-public") + discoverable_ref = CapabilityRef("cap-findable") + catalog.register("secret_tool", declared_ref, initial_state=LifecycleState.DECLARED) + catalog.register("public_tool", visible_ref, initial_state=LifecycleState.VISIBLE) + catalog.register("findable_tool", discoverable_ref, initial_state=LifecycleState.DISCOVERABLE) + + result = discover_capabilities( + catalog, + AgentIdentity("agent-1"), + DiscoveryIntent(query="tool"), + ) + + assert not result.denied + names = {cap["name"] for cap in result.capabilities} + assert names == {"findable_tool"} + + +def test_no_match_returns_denied(): + catalog = CapabilityCatalog() + ref = CapabilityRef("cap-create-pr") + catalog.register("create_pr", ref, initial_state=LifecycleState.DISCOVERABLE) + + result = discover_capabilities( + catalog, + AgentIdentity("agent-1"), + DiscoveryIntent(query="deploy"), + ) + + assert result.denied + assert result.capabilities == [] diff --git a/capability-runtime/tests/test_revoke.py b/capability-runtime/tests/test_revoke.py new file mode 100644 index 00000000..3a0ac41e --- /dev/null +++ b/capability-runtime/tests/test_revoke.py @@ -0,0 +1,44 @@ +from datetime import datetime, timedelta, timezone + +from capability_runtime.catalog import CapabilityCatalog, LifecycleState +from capability_runtime.lease import LeaseManager +from capability_runtime.revoke import RevocationManager +from capability_runtime.types import AgentIdentity, CapabilityRef + + +def test_revoke_by_ref_sets_revoked_state(): + catalog = CapabilityCatalog() + ref = CapabilityRef("cap-create-pr") + catalog.register("create_pr", ref, initial_state=LifecycleState.VISIBLE) + lease_mgr = LeaseManager() + revoke_mgr = RevocationManager(catalog, lease_mgr) + + revoke_mgr.revoke_by_ref(ref, reason="policy violation") + + assert revoke_mgr.is_revoked(ref) + assert catalog.get_state(ref) == LifecycleState.REVOKED + assert not catalog.is_visible(ref) + + +def test_revoke_by_lease_marks_lease_revoked(): + catalog = CapabilityCatalog() + ref = CapabilityRef("cap-create-pr") + catalog.register("create_pr", ref, initial_state=LifecycleState.LEASED) + lease_mgr = LeaseManager() + now = datetime(2026, 6, 22, 12, 0, 0, tzinfo=timezone.utc) + decision = lease_mgr.grant( + agent_id=AgentIdentity("agent-1"), + capability_ref=ref, + quota=1, + ttl=timedelta(minutes=10), + token_budget=25000, + risk_envelope="high", + now=now, + ) + revoke_mgr = RevocationManager(catalog, lease_mgr) + + revoke_mgr.revoke_by_lease(decision.lease_id, reason="quota abuse") + + assert revoke_mgr.is_lease_revoked(decision.lease_id) + assert revoke_mgr.is_revoked(ref) + assert catalog.get_state(ref) == LifecycleState.REVOKED From f83a310ad0706ee7a1319b59ff29d936c37f2caf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 11:56:16 +0000 Subject: [PATCH 023/138] Implement CapabilityRuntime with all protocol surfaces MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add CapabilityRuntime class that wires together catalog, leases, revocation, and observability. Implements all 7 protocol surfaces from spec §3.2: - check_current_capabilities: returns bundle with visible/leased tools - request_capability_lease: grants leases with PoC 1 params - revoke_capability: revokes by lease ID or capability ref - discover_capabilities: discovers by intent query - emit events: delegates to observability collector - record_action: decrements quota and revokes when exhausted - enforce_action: validates bundle version and visibility Includes integration test for full PoC 1 lifecycle (create_pr invisible → lease → visible → use → gone) plus unit tests for revoke, discovery, and version mismatch scenarios. All 23 tests pass. Co-authored-by: Cursor --- .../src/capability_runtime/__init__.py | 3 +- .../src/capability_runtime/runtime.py | 271 ++++++++++++++++++ capability-runtime/tests/test_runtime.py | 143 +++++++++ 3 files changed, 416 insertions(+), 1 deletion(-) create mode 100644 capability-runtime/src/capability_runtime/runtime.py create mode 100644 capability-runtime/tests/test_runtime.py diff --git a/capability-runtime/src/capability_runtime/__init__.py b/capability-runtime/src/capability_runtime/__init__.py index 35fdf704..4b007a28 100644 --- a/capability-runtime/src/capability_runtime/__init__.py +++ b/capability-runtime/src/capability_runtime/__init__.py @@ -1,3 +1,4 @@ +from capability_runtime.runtime import CapabilityRuntime from capability_runtime.types import CapabilityBundle -__all__ = ["CapabilityBundle"] +__all__ = ["CapabilityBundle", "CapabilityRuntime"] diff --git a/capability-runtime/src/capability_runtime/runtime.py b/capability-runtime/src/capability_runtime/runtime.py new file mode 100644 index 00000000..a0c9175a --- /dev/null +++ b/capability-runtime/src/capability_runtime/runtime.py @@ -0,0 +1,271 @@ +"""CapabilityRuntime — all protocol surfaces (spec §3.2).""" + +from __future__ import annotations + +from datetime import datetime, timedelta, timezone +from pathlib import Path +from typing import Union + +from capability_runtime.catalog import CapabilityCatalog, LifecycleState +from capability_runtime.discover import ( + DiscoveryIntent, + DiscoveryResult, + discover_capabilities as _discover_capabilities, +) +from capability_runtime.errors import ( + BundleVersionMismatch, + CapabilityNotVisible, + CapabilityUnknown, +) +from capability_runtime.lease import LeaseManager +from capability_runtime.observability import ObservabilityCollector +from capability_runtime.revoke import RevocationManager +from capability_runtime.types import ( + AgentIdentity, + BudgetEnvelope, + CapabilityBundle, + CapabilityRef, + LeaseDecision, + LeaseId, + ProvenanceRecord, + ToolCapability, +) + + +class CapabilityRuntime: + """Wire together catalog, leases, revocation, observability, discovery.""" + + def __init__( + self, + trace_file: Path, + trace_id: str, + seed: int, + policy_version: str = "1.0.0", + ): + self.catalog = CapabilityCatalog() + self.lease_manager = LeaseManager() + self.revocation_manager = RevocationManager(self.catalog, self.lease_manager) + self.observability = ObservabilityCollector(trace_file, trace_id, seed) + self.policy_version = policy_version + self.trace_id = trace_id + self._bundle_version = 0 + self._current_bundles: dict[AgentIdentity, int] = {} + + # Register create_pr as DECLARED (invisible until leased) + self.catalog.register("create_pr", CapabilityRef("cap-create-pr"), LifecycleState.DECLARED) + + def check_current_capabilities( + self, agent_id: AgentIdentity, context: dict + ) -> CapabilityBundle: + """Return visible tools + active leased tools for agent (spec §3.2.1).""" + self._bundle_version += 1 + self._current_bundles[agent_id] = self._bundle_version + + now = datetime.now(timezone.utc) + tools: list[ToolCapability] = [] + earliest_expiry: datetime | None = None + + # Iterate through catalog to find visible capabilities + for ref in self.catalog._by_ref: + state = self.catalog.get_state(ref) + name = self.catalog.get_name(ref) + if name is None: + continue + + # Include if Visible OR has active lease + if state == LifecycleState.VISIBLE: + tools.append( + ToolCapability( + ref=ref, + name=name, + lease_id=None, + quota="unbounded", + expires_at=None, + ) + ) + elif state == LifecycleState.LEASED: + # Find active lease for this agent + for lease_id, lease in self.lease_manager._leases.items(): + if ( + lease.capability_ref == ref + and lease.agent_id == agent_id + and not self.lease_manager.is_expired(lease_id, now) + and not self.revocation_manager.is_lease_revoked(lease_id) + and not self.lease_manager.is_exhausted(lease_id) + ): + remaining = self.lease_manager._remaining_quota[lease_id] + tools.append( + ToolCapability( + ref=ref, + name=name, + lease_id=lease_id, + quota=remaining, + expires_at=lease.expires_at, + ) + ) + if earliest_expiry is None or lease.expires_at < earliest_expiry: + earliest_expiry = lease.expires_at + break + + bundle = CapabilityBundle( + agent_id=agent_id, + issued_at=now, + expires_at=earliest_expiry, + tools=tools, + rules=[], + skills=[], + policies=[], + hooks=[], + budgets=BudgetEnvelope( + token_quota="unbounded", + action_quota="unbounded", + wall_time_ttl=None, + ), + provenance=ProvenanceRecord( + issuer="CapabilityRuntime", + policy_version=self.policy_version, + trace_id=self.trace_id, + ), + version=self._bundle_version, + ) + + self.observability.emit_capability_event( + { + "event": "bundle_issued", + "agent_id": agent_id.value, + "bundle_version": self._bundle_version, + "tool_count": len(tools), + } + ) + + return bundle + + def request_capability_lease( + self, + agent_id: AgentIdentity, + capability_ref: CapabilityRef, + justification: str, + ) -> LeaseDecision: + """Grant lease for create_pr with PoC 1 params (spec §3.2.2).""" + # Check capability exists + state = self.catalog.get_state(capability_ref) + if state not in (LifecycleState.DECLARED, LifecycleState.DISCOVERABLE, LifecycleState.VISIBLE): + raise CapabilityUnknown(capability_ref) + + now = datetime.now(timezone.utc) + + # PoC 1 params for create_pr + quota = 1 + ttl = timedelta(minutes=10) + token_budget = 25000 + risk_envelope = "high" + + decision = self.lease_manager.grant( + agent_id=agent_id, + capability_ref=capability_ref, + quota=quota, + ttl=ttl, + token_budget=token_budget, + risk_envelope=risk_envelope, + now=now, + ) + + # Set catalog state to LEASED + self.catalog.set_state(capability_ref, LifecycleState.LEASED) + + self.observability.emit_capability_event( + { + "event": "lease_granted", + "agent_id": agent_id.value, + "capability_ref": capability_ref.value, + "lease_id": decision.lease_id.value, + "quota": quota, + "justification": justification, + } + ) + + return decision + + def revoke_capability( + self, target: Union[LeaseId, CapabilityRef], reason: str + ) -> None: + """Revoke capability by lease ID or ref (spec §3.2.3).""" + if isinstance(target, LeaseId): + self.revocation_manager.revoke_by_lease(target, reason) + self.observability.emit_capability_event( + { + "event": "capability_revoked", + "lease_id": target.value, + "reason": reason, + } + ) + else: + self.revocation_manager.revoke_by_ref(target, reason) + self.observability.emit_capability_event( + { + "event": "capability_revoked", + "capability_ref": target.value, + "reason": reason, + } + ) + + def discover_capabilities( + self, agent_id: AgentIdentity, intent: DiscoveryIntent + ) -> DiscoveryResult: + """Discover capabilities by intent (spec §3.2.4).""" + return _discover_capabilities(self.catalog, agent_id, intent) + + def emit_execution_event(self, event: dict) -> None: + """Emit execution event to observability (spec §3.2.5).""" + self.observability.emit_execution_event(event) + + def emit_capability_event(self, event: dict) -> None: + """Emit capability event to observability (spec §3.2.5).""" + self.observability.emit_capability_event(event) + + def record_action(self, lease_id: LeaseId, now: datetime) -> None: + """Decrement quota; revoke if exhausted (spec §3.2.6).""" + remaining = self.lease_manager.decrement_quota(lease_id, now) + + self.observability.emit_capability_event( + { + "event": "quota_decremented", + "lease_id": lease_id.value, + "remaining": remaining, + } + ) + + # If exhausted, revoke lease and set appropriate state + if remaining == 0: + lease = self.lease_manager.get_lease(lease_id) + if lease is not None: + self.catalog.set_state(lease.capability_ref, LifecycleState.EXPIRED) + self.revocation_manager.revoke_by_lease(lease_id, "quota exhausted") + + def enforce_action( + self, agent_id: AgentIdentity, tool_name: str, bundle_version: int, now: datetime + ) -> None: + """Enforce action is allowed (spec §3.2.7).""" + # Check bundle version + current_version = self._current_bundles.get(agent_id) + if current_version is None or bundle_version != current_version: + raise BundleVersionMismatch(current_version or 0, bundle_version) + + # Check tool is in current bundle + ref = self.catalog.get_by_name(tool_name) + if ref is None or not self.catalog.is_visible(ref): + # Also check if there's an active lease + found = False + if ref is not None: + for lease_id, lease in self.lease_manager._leases.items(): + if ( + lease.capability_ref == ref + and lease.agent_id == agent_id + and not self.lease_manager.is_expired(lease_id, now) + and not self.revocation_manager.is_lease_revoked(lease_id) + and not self.lease_manager.is_exhausted(lease_id) + ): + found = True + break + if not found: + raise CapabilityNotVisible(ref or CapabilityRef(tool_name)) diff --git a/capability-runtime/tests/test_runtime.py b/capability-runtime/tests/test_runtime.py new file mode 100644 index 00000000..854b2fd1 --- /dev/null +++ b/capability-runtime/tests/test_runtime.py @@ -0,0 +1,143 @@ +"""Integration tests for CapabilityRuntime (spec §3.2 and §5.1).""" + +from datetime import datetime, timezone + +import pytest + +from capability_runtime.catalog import LifecycleState +from capability_runtime.errors import BundleVersionMismatch, CapabilityNotVisible +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity, CapabilityRef +from capability_runtime.discover import DiscoveryIntent + + +def test_poc1_create_pr_lifecycle(tmp_path): + """Full PoC 1: create_pr invisible → lease → visible → use → gone""" + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-1", 42) + agent = AgentIdentity("agent-1") + ctx = {} + + # Step 1: not present + bundle1 = runtime.check_current_capabilities(agent, ctx) + tool_names = [t.name for t in bundle1.tools] + assert "create_pr" not in tool_names + + # Step 2: request lease + decision = runtime.request_capability_lease( + agent, CapabilityRef("cap-create-pr"), "Need to open PR for feature" + ) + assert decision.quota == 1 + + # Step 3: present with lease + bundle2 = runtime.check_current_capabilities(agent, ctx) + create_pr_tools = [t for t in bundle2.tools if t.name == "create_pr"] + assert len(create_pr_tools) == 1 + assert create_pr_tools[0].lease_id is not None + + # Step 4: use (record action) + now = datetime.now(timezone.utc) + runtime.record_action(create_pr_tools[0].lease_id, now) + + # Step 5: gone after quota exhausted + bundle3 = runtime.check_current_capabilities(agent, ctx) + assert "create_pr" not in [t.name for t in bundle3.tools] + + +def test_revoke_by_lease_id(tmp_path): + """Test revoking a capability by lease ID""" + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-2", 43) + agent = AgentIdentity("agent-2") + + # Request lease + decision = runtime.request_capability_lease( + agent, CapabilityRef("cap-create-pr"), "Testing revoke" + ) + + # Verify present + bundle1 = runtime.check_current_capabilities(agent, {}) + assert "create_pr" in [t.name for t in bundle1.tools] + + # Revoke by lease ID + runtime.revoke_capability(decision.lease_id, "policy violation") + + # Verify gone + bundle2 = runtime.check_current_capabilities(agent, {}) + assert "create_pr" not in [t.name for t in bundle2.tools] + + +def test_revoke_by_capability_ref(tmp_path): + """Test revoking a capability by reference""" + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-3", 44) + agent = AgentIdentity("agent-3") + + # Request lease + runtime.request_capability_lease( + agent, CapabilityRef("cap-create-pr"), "Testing revoke" + ) + + # Verify present + bundle1 = runtime.check_current_capabilities(agent, {}) + assert "create_pr" in [t.name for t in bundle1.tools] + + # Revoke by ref + runtime.revoke_capability(CapabilityRef("cap-create-pr"), "security concern") + + # Verify gone + bundle2 = runtime.check_current_capabilities(agent, {}) + assert "create_pr" not in [t.name for t in bundle2.tools] + + +def test_discover_capabilities(tmp_path): + """Test capability discovery""" + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-4", 45) + agent = AgentIdentity("agent-4") + + # Register a discoverable capability + runtime.catalog.register( + "merge_pr", CapabilityRef("cap-merge-pr"), LifecycleState.DISCOVERABLE + ) + + # Discover by query + result = runtime.discover_capabilities(agent, DiscoveryIntent("merge")) + assert len(result.capabilities) == 1 + assert result.capabilities[0]["name"] == "merge_pr" + assert not result.denied + + # Query that doesn't match + result2 = runtime.discover_capabilities(agent, DiscoveryIntent("deploy")) + assert len(result2.capabilities) == 0 + assert result2.denied + + +def test_bundle_version_mismatch(tmp_path): + """Test enforce_action raises BundleVersionMismatch for stale version""" + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-5", 46) + agent = AgentIdentity("agent-5") + + # Get initial bundle + bundle1 = runtime.check_current_capabilities(agent, {}) + + # Get another bundle (increments version) + bundle2 = runtime.check_current_capabilities(agent, {}) + + # Try to enforce action with old bundle version + now = datetime.now(timezone.utc) + with pytest.raises(BundleVersionMismatch) as exc_info: + runtime.enforce_action(agent, "create_pr", bundle1.version, now) + + assert exc_info.value.expected == bundle2.version + assert exc_info.value.actual == bundle1.version + + +def test_enforce_action_not_visible(tmp_path): + """Test enforce_action raises CapabilityNotVisible for tool not in bundle""" + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-6", 47) + agent = AgentIdentity("agent-6") + + # Get bundle (create_pr is DECLARED, not visible) + bundle = runtime.check_current_capabilities(agent, {}) + + # Try to enforce action on invisible capability + now = datetime.now(timezone.utc) + with pytest.raises(CapabilityNotVisible): + runtime.enforce_action(agent, "create_pr", bundle.version, now) From cdc228b668b7eb3cacd8ba0202bcaf120f1bf495 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 11:57:28 +0000 Subject: [PATCH 024/138] feat(capability-runtime): add AgentExecutionLoop harness Thin check-then-act wrapper that re-fetches bundle version before each action and records leased tool usage. Co-authored-by: Cursor --- .../src/capability_runtime/agent_loop.py | 45 ++++++++++ capability-runtime/tests/test_agent_loop.py | 82 +++++++++++++++++++ 2 files changed, 127 insertions(+) create mode 100644 capability-runtime/src/capability_runtime/agent_loop.py create mode 100644 capability-runtime/tests/test_agent_loop.py diff --git a/capability-runtime/src/capability_runtime/agent_loop.py b/capability-runtime/src/capability_runtime/agent_loop.py new file mode 100644 index 00000000..572a7cdd --- /dev/null +++ b/capability-runtime/src/capability_runtime/agent_loop.py @@ -0,0 +1,45 @@ +"""AgentExecutionLoop — thin harness for check-then-act with bundle version.""" + +from __future__ import annotations + +from collections.abc import Callable +from datetime import datetime, timezone +from typing import Any + +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity, LeaseId + + +class AgentExecutionLoop: + """Re-fetch bundle before each action; enforce bundle version optimistically.""" + + def __init__(self, runtime: CapabilityRuntime): + self._runtime = runtime + + def run_step( + self, + agent_id: AgentIdentity, + context: dict, + tool_name: str, + action_fn: Callable[[], Any], + now: datetime | None = None, + ) -> Any: + effective_now = now or datetime.now(timezone.utc) + + bundle = self._runtime.check_current_capabilities(agent_id, context) + self._runtime.enforce_action(agent_id, tool_name, bundle.version, effective_now) + + result = action_fn() + + lease_id = _lease_id_for_tool(bundle.tools, tool_name) + if lease_id is not None: + self._runtime.record_action(lease_id, effective_now) + + return result + + +def _lease_id_for_tool(tools, tool_name: str) -> LeaseId | None: + for tool in tools: + if tool.name == tool_name: + return tool.lease_id + return None diff --git a/capability-runtime/tests/test_agent_loop.py b/capability-runtime/tests/test_agent_loop.py new file mode 100644 index 00000000..54dbf924 --- /dev/null +++ b/capability-runtime/tests/test_agent_loop.py @@ -0,0 +1,82 @@ +"""Tests for AgentExecutionLoop harness (spec threat model).""" + +from datetime import datetime, timezone + +import pytest + +from capability_runtime.agent_loop import AgentExecutionLoop +from capability_runtime.catalog import LifecycleState +from capability_runtime.errors import CapabilityNotVisible +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity, CapabilityRef + + +def test_run_step_succeeds_when_tool_visible(tmp_path): + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-agent-loop-1", 100) + runtime.catalog.register( + "list_files", CapabilityRef("cap-list-files"), LifecycleState.VISIBLE + ) + + loop = AgentExecutionLoop(runtime) + agent = AgentIdentity("agent-1") + called: list[bool] = [] + + def action_fn(): + called.append(True) + return "ok" + + result = loop.run_step(agent, {}, "list_files", action_fn) + assert result == "ok" + assert called == [True] + + +def test_run_step_raises_not_visible(tmp_path): + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-agent-loop-2", 101) + loop = AgentExecutionLoop(runtime) + agent = AgentIdentity("agent-2") + + with pytest.raises(CapabilityNotVisible): + loop.run_step(agent, {}, "create_pr", lambda: None) + + +def test_run_step_adapts_after_lease_exhausted(tmp_path): + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-agent-loop-3", 102) + loop = AgentExecutionLoop(runtime) + agent = AgentIdentity("agent-3") + + runtime.request_capability_lease( + agent, CapabilityRef("cap-create-pr"), "Need create_pr once" + ) + + assert loop.run_step(agent, {}, "create_pr", lambda: "first") == "first" + + with pytest.raises(CapabilityNotVisible): + loop.run_step(agent, {}, "create_pr", lambda: "second") + + +def test_run_step_checks_bundle_before_action(tmp_path): + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-agent-loop-4", 103) + runtime.catalog.register( + "visible_tool", CapabilityRef("cap-visible"), LifecycleState.VISIBLE + ) + + loop = AgentExecutionLoop(runtime) + agent = AgentIdentity("agent-4") + call_order: list[str] = [] + + original_check = runtime.check_current_capabilities + original_enforce = runtime.enforce_action + + def spy_check(*args, **kwargs): + call_order.append("check") + return original_check(*args, **kwargs) + + def spy_enforce(*args, **kwargs): + call_order.append("enforce") + return original_enforce(*args, **kwargs) + + runtime.check_current_capabilities = spy_check # type: ignore[method-assign] + runtime.enforce_action = spy_enforce # type: ignore[method-assign] + + loop.run_step(agent, {}, "visible_tool", lambda: None) + assert call_order == ["check", "enforce"] From 564852763ff99893a7899696704c153b65de155e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 11:59:06 +0000 Subject: [PATCH 025/138] feat(capability-runtime): add mock MCP tool adapter for PoC 2 Wrap MCP-delivered tools via McpToolAdapter with write_note lifecycle (quota=3, ttl=5m) and capability-specific lease params in the runtime. Co-authored-by: Cursor --- capability-runtime/poc/mcp_tool_demo.py | 74 +++++++++++++ .../src/capability_runtime/mcp_adapter.py | 51 +++++++++ .../src/capability_runtime/runtime.py | 19 ++-- capability-runtime/tests/test_mcp_adapter.py | 100 ++++++++++++++++++ 4 files changed, 238 insertions(+), 6 deletions(-) create mode 100644 capability-runtime/poc/mcp_tool_demo.py create mode 100644 capability-runtime/src/capability_runtime/mcp_adapter.py create mode 100644 capability-runtime/tests/test_mcp_adapter.py diff --git a/capability-runtime/poc/mcp_tool_demo.py b/capability-runtime/poc/mcp_tool_demo.py new file mode 100644 index 00000000..2f06c961 --- /dev/null +++ b/capability-runtime/poc/mcp_tool_demo.py @@ -0,0 +1,74 @@ +#!/usr/bin/env python3 +"""PoC 2: write_note MCP tool lifecycle (spec §5.2). + +Same lifecycle as PoC 1 from the agent perspective: + invisible → lease → visible → 3 invocations → gone +""" + +from __future__ import annotations + +import sys +from pathlib import Path + +from capability_runtime.agent_loop import AgentExecutionLoop +from capability_runtime.mcp_adapter import McpToolAdapter, McpToolCapability +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity, CapabilityRef + + +def main() -> int: + trace_file = Path("trace-mcp-tool-demo.jsonl") + runtime = CapabilityRuntime(trace_file, "trace-mcp-demo-1", 42) + adapter = McpToolAdapter(runtime) + loop = AgentExecutionLoop(runtime) + agent = AgentIdentity("demo-agent") + ctx: dict = {} + + adapter.register_mcp_tool( + McpToolCapability( + name="write_note", + ref=CapabilityRef("cap-write-note"), + description="Write a note to the workspace", + ) + ) + + # Step 1: not present + bundle1 = runtime.check_current_capabilities(agent, ctx) + tool_names = [t.name for t in bundle1.tools] + print(f"Step 1 — initial bundle tools: {tool_names}") + assert "write_note" not in tool_names + + # Step 2: request lease + decision = runtime.request_capability_lease( + agent, CapabilityRef("cap-write-note"), "Need to record session notes" + ) + print( + f"Step 2 — lease granted: quota={decision.quota}, " + f"token_budget={decision.token_budget}, risk={decision.risk_envelope}" + ) + + # Step 3: present with lease + bundle2 = runtime.check_current_capabilities(agent, ctx) + write_note_tools = [t for t in bundle2.tools if t.name == "write_note"] + print(f"Step 3 — write_note in bundle: {len(write_note_tools) == 1}") + assert len(write_note_tools) == 1 + + # Step 4: use three times via MCP adapter + for i in range(3): + result = adapter.invoke(agent, ctx, "write_note", f"note-{i}", loop) + print(f"Step 4.{i + 1} — invoke result: {result}") + + print(f"Step 4 — side effects recorded: {adapter.side_effects}") + + # Step 5: gone after quota exhausted + bundle3 = runtime.check_current_capabilities(agent, ctx) + tool_names_after = [t.name for t in bundle3.tools] + print(f"Step 5 — final bundle tools: {tool_names_after}") + assert "write_note" not in tool_names_after + + print("PoC 2 complete: write_note lifecycle verified.") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/capability-runtime/src/capability_runtime/mcp_adapter.py b/capability-runtime/src/capability_runtime/mcp_adapter.py new file mode 100644 index 00000000..5d8ab080 --- /dev/null +++ b/capability-runtime/src/capability_runtime/mcp_adapter.py @@ -0,0 +1,51 @@ +"""Mock MCP tool wrapper — transport-agnostic (spec §5.2).""" + +from __future__ import annotations + +from dataclasses import dataclass + +from capability_runtime.agent_loop import AgentExecutionLoop +from capability_runtime.catalog import LifecycleState +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity, CapabilityRef + + +@dataclass +class McpToolCapability: + name: str + ref: CapabilityRef + description: str + + +class McpToolAdapter: + """Wraps MCP-delivered tools in CapabilityBundle surface.""" + + def __init__(self, runtime: CapabilityRuntime): + self._runtime = runtime + self._side_effects: list[str] = [] + + def register_mcp_tool( + self, + tool: McpToolCapability, + initial_state: LifecycleState = LifecycleState.DECLARED, + ) -> None: + self._runtime.catalog.register(tool.name, tool.ref, initial_state) + + def invoke( + self, + agent_id: AgentIdentity, + context: dict, + tool_name: str, + payload: str, + loop: AgentExecutionLoop, + ) -> str: + def action_fn() -> str: + side_effect = f"{tool_name}:{payload}" + self._side_effects.append(side_effect) + return f"note written: {payload}" + + return loop.run_step(agent_id, context, tool_name, action_fn) + + @property + def side_effects(self) -> list[str]: + return list(self._side_effects) diff --git a/capability-runtime/src/capability_runtime/runtime.py b/capability-runtime/src/capability_runtime/runtime.py index a0c9175a..5d3dc475 100644 --- a/capability-runtime/src/capability_runtime/runtime.py +++ b/capability-runtime/src/capability_runtime/runtime.py @@ -146,7 +146,7 @@ def request_capability_lease( capability_ref: CapabilityRef, justification: str, ) -> LeaseDecision: - """Grant lease for create_pr with PoC 1 params (spec §3.2.2).""" + """Grant lease with capability-specific params (spec §3.2.2).""" # Check capability exists state = self.catalog.get_state(capability_ref) if state not in (LifecycleState.DECLARED, LifecycleState.DISCOVERABLE, LifecycleState.VISIBLE): @@ -154,11 +154,18 @@ def request_capability_lease( now = datetime.now(timezone.utc) - # PoC 1 params for create_pr - quota = 1 - ttl = timedelta(minutes=10) - token_budget = 25000 - risk_envelope = "high" + capability_name = self.catalog.get_name(capability_ref) + if capability_name == "write_note": + quota = 3 + ttl = timedelta(minutes=5) + token_budget = 10_000 + risk_envelope = "medium" + else: + # PoC 1 params for create_pr (default) + quota = 1 + ttl = timedelta(minutes=10) + token_budget = 25_000 + risk_envelope = "high" decision = self.lease_manager.grant( agent_id=agent_id, diff --git a/capability-runtime/tests/test_mcp_adapter.py b/capability-runtime/tests/test_mcp_adapter.py new file mode 100644 index 00000000..1568f6fc --- /dev/null +++ b/capability-runtime/tests/test_mcp_adapter.py @@ -0,0 +1,100 @@ +"""Tests for McpToolAdapter (spec §5.2).""" + +from datetime import timedelta + +import pytest + +from capability_runtime.agent_loop import AgentExecutionLoop +from capability_runtime.catalog import LifecycleState +from capability_runtime.errors import CapabilityNotVisible +from capability_runtime.mcp_adapter import McpToolAdapter, McpToolCapability +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity, CapabilityRef + + +def _adapter_and_loop(tmp_path): + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-mcp-1", 200) + adapter = McpToolAdapter(runtime) + adapter.register_mcp_tool( + McpToolCapability( + name="write_note", + ref=CapabilityRef("cap-write-note"), + description="Write a note to the workspace", + ) + ) + loop = AgentExecutionLoop(runtime) + return runtime, adapter, loop + + +def test_write_note_invisible_until_leased(tmp_path): + runtime, adapter, _loop = _adapter_and_loop(tmp_path) + agent = AgentIdentity("agent-mcp-1") + ctx = {} + + bundle1 = runtime.check_current_capabilities(agent, ctx) + assert "write_note" not in [t.name for t in bundle1.tools] + + decision = runtime.request_capability_lease( + agent, CapabilityRef("cap-write-note"), "Need to record findings" + ) + assert decision.quota == 3 + assert decision.token_budget == 10_000 + assert decision.risk_envelope == "medium" + assert decision.expires_at - decision.granted_at == timedelta(minutes=5) + + bundle2 = runtime.check_current_capabilities(agent, ctx) + write_note_tools = [t for t in bundle2.tools if t.name == "write_note"] + assert len(write_note_tools) == 1 + assert write_note_tools[0].lease_id is not None + + +def test_write_note_lifecycle_quota_3(tmp_path): + runtime, adapter, loop = _adapter_and_loop(tmp_path) + agent = AgentIdentity("agent-mcp-2") + ctx = {} + + assert "write_note" not in [ + t.name for t in runtime.check_current_capabilities(agent, ctx).tools + ] + + runtime.request_capability_lease( + agent, CapabilityRef("cap-write-note"), "Need notes for session" + ) + + for i in range(3): + result = adapter.invoke(agent, ctx, "write_note", f"note-{i}", loop) + assert result == f"note written: note-{i}" + + with pytest.raises(CapabilityNotVisible): + adapter.invoke(agent, ctx, "write_note", "note-4", loop) + + bundle = runtime.check_current_capabilities(agent, ctx) + assert "write_note" not in [t.name for t in bundle.tools] + + +def test_mcp_adapter_records_side_effects(tmp_path): + runtime, adapter, loop = _adapter_and_loop(tmp_path) + agent = AgentIdentity("agent-mcp-3") + + runtime.request_capability_lease( + agent, CapabilityRef("cap-write-note"), "Side effect test" + ) + + adapter.invoke(agent, {}, "write_note", "hello world", loop) + + assert adapter.side_effects == ["write_note:hello world"] + + +def test_register_mcp_tool_respects_initial_state(tmp_path): + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-mcp-4", 201) + adapter = McpToolAdapter(runtime) + ref = CapabilityRef("cap-visible-note") + + adapter.register_mcp_tool( + McpToolCapability(name="visible_note", ref=ref, description="Always visible"), + initial_state=LifecycleState.VISIBLE, + ) + + agent = AgentIdentity("agent-mcp-4") + bundle = runtime.check_current_capabilities(agent, {}) + assert "visible_note" in [t.name for t in bundle.tools] From 58d677edddeb077cc660349da49e0c6838f6b313 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 12:00:17 +0000 Subject: [PATCH 026/138] feat(capability-runtime): add PoC 1 create_pr runnable demo MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Demonstrates the §5.1 lease lifecycle with AgentExecutionLoop and agent adaptation to draft_pr when create_pr is exhausted. Co-authored-by: Cursor --- capability-runtime/poc/__init__.py | 0 capability-runtime/poc/create_pr_demo.py | 136 ++++++++++++++++++ capability-runtime/pyproject.toml | 2 +- .../tests/test_poc_create_pr.py | 51 +++++++ 4 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 capability-runtime/poc/__init__.py create mode 100644 capability-runtime/poc/create_pr_demo.py create mode 100644 capability-runtime/tests/test_poc_create_pr.py diff --git a/capability-runtime/poc/__init__.py b/capability-runtime/poc/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/capability-runtime/poc/create_pr_demo.py b/capability-runtime/poc/create_pr_demo.py new file mode 100644 index 00000000..6d3a5437 --- /dev/null +++ b/capability-runtime/poc/create_pr_demo.py @@ -0,0 +1,136 @@ +"""PoC 1 — create_pr leased capability demo (spec §5.1). + +Runnable via: + PYTHONPATH=src:. python poc/create_pr_demo.py + PYTHONPATH=src:. python -m poc.create_pr_demo +""" + +from __future__ import annotations + +import tempfile +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +from capability_runtime.agent_loop import AgentExecutionLoop +from capability_runtime.errors import CapabilityNotVisible +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity, CapabilityRef, LeaseDecision + + +@dataclass +class DemoResult: + initial_tools: list[str] + lease_decision: LeaseDecision + tools_after_lease: list[str] + action_result: Any + tools_after_use: list[str] + retry_blocked: bool + retry_error: CapabilityNotVisible | None + adapted_action: str + create_pr_retry_attempted: bool + + +def _tool_names(tools) -> list[str]: + return [t.name for t in tools] + + +def _print_step(quiet: bool, step: int, message: str) -> None: + if not quiet: + print(f"Step {step}: {message}") + + +def run_poc1_demo(trace_path: Path, *, quiet: bool = False) -> DemoResult: + """Execute the §5.1 create_pr lifecycle and optional adaptation step.""" + runtime = CapabilityRuntime(trace_path, "poc1-create-pr", seed=42) + loop = AgentExecutionLoop(runtime) + agent = AgentIdentity("demo-agent") + context: dict = {} + + bundle1 = runtime.check_current_capabilities(agent, context) + initial_tools = _tool_names(bundle1.tools) + _print_step( + quiet, + 1, + f"check_current_capabilities → tools={initial_tools} (create_pr absent)", + ) + + decision = runtime.request_capability_lease( + agent, + CapabilityRef("cap-create-pr"), + "Need to open PR for feature", + ) + _print_step( + quiet, + 2, + f"request_capability_lease → LeaseDecision(lease_id={decision.lease_id.value!r}, " + f"quota={decision.quota}, token_budget={decision.token_budget}, " + f"risk_envelope={decision.risk_envelope!r})", + ) + + bundle2 = runtime.check_current_capabilities(agent, context) + tools_after_lease = _tool_names(bundle2.tools) + _print_step( + quiet, + 3, + f"check_current_capabilities → tools={tools_after_lease} (create_pr present)", + ) + + action_result = loop.run_step( + agent, + context, + "create_pr", + lambda: "PR created (mock)", + ) + _print_step( + quiet, + 4, + f"AgentExecutionLoop.run_step(create_pr) → {action_result!r}", + ) + + bundle3 = runtime.check_current_capabilities(agent, context) + tools_after_use = _tool_names(bundle3.tools) + _print_step( + quiet, + 5, + f"check_current_capabilities → tools={tools_after_use} (create_pr absent)", + ) + + retry_error: CapabilityNotVisible | None = None + retry_blocked = False + try: + loop.run_step(agent, context, "create_pr", lambda: "should not run") + except CapabilityNotVisible as exc: + retry_error = exc + retry_blocked = True + + adapted_action = "draft_pr" + create_pr_retry_attempted = False + _print_step( + quiet, + 6, + "Agent adapts plan: using draft_pr instead", + ) + + return DemoResult( + initial_tools=initial_tools, + lease_decision=decision, + tools_after_lease=tools_after_lease, + action_result=action_result, + tools_after_use=tools_after_use, + retry_blocked=retry_blocked, + retry_error=retry_error, + adapted_action=adapted_action, + create_pr_retry_attempted=create_pr_retry_attempted, + ) + + +def main(trace_path: Path | None = None) -> None: + if trace_path is None: + with tempfile.NamedTemporaryFile(suffix=".jsonl", delete=False) as handle: + trace_path = Path(handle.name) + run_poc1_demo(trace_path, quiet=False) + + +if __name__ == "__main__": + main() diff --git a/capability-runtime/pyproject.toml b/capability-runtime/pyproject.toml index 8f01eef7..852e8dd0 100644 --- a/capability-runtime/pyproject.toml +++ b/capability-runtime/pyproject.toml @@ -4,5 +4,5 @@ version = "0.1.0" requires-python = ">=3.12" [tool.pytest.ini_options] -pythonpath = ["src"] +pythonpath = ["src", "."] testpaths = ["tests"] diff --git a/capability-runtime/tests/test_poc_create_pr.py b/capability-runtime/tests/test_poc_create_pr.py new file mode 100644 index 00000000..665a1016 --- /dev/null +++ b/capability-runtime/tests/test_poc_create_pr.py @@ -0,0 +1,51 @@ +"""Integration tests for PoC 1 create_pr demo (spec §5.1).""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +from capability_runtime.errors import CapabilityNotVisible +from capability_runtime.types import CapabilityRef +from poc.create_pr_demo import run_poc1_demo + + +def test_poc1_create_pr_lifecycle(tmp_path: Path) -> None: + """create_pr absent → lease → present → use → absent.""" + result = run_poc1_demo(tmp_path / "trace.jsonl", quiet=True) + + assert "create_pr" not in result.initial_tools + assert result.lease_decision.quota == 1 + assert result.lease_decision.capability_ref == CapabilityRef("cap-create-pr") + assert "create_pr" in result.tools_after_lease + assert result.action_result == "PR created (mock)" + assert "create_pr" not in result.tools_after_use + + +def test_poc1_agent_adapts_instead_of_retrying_create_pr(tmp_path: Path) -> None: + """After lease exhaustion, agent catches CapabilityNotVisible and uses draft_pr.""" + result = run_poc1_demo(tmp_path / "trace.jsonl", quiet=True) + + assert result.retry_blocked is True + assert isinstance(result.retry_error, CapabilityNotVisible) + assert result.adapted_action == "draft_pr" + assert result.create_pr_retry_attempted is False + + +def test_poc1_demo_main_prints_steps(capsys, tmp_path: Path) -> None: + """Runnable demo prints each step of the §5.1 sequence.""" + from poc.create_pr_demo import main + + main(trace_path=tmp_path / "trace.jsonl") + + out = capsys.readouterr().out + assert "Step 1" in out + assert "create_pr absent" in out.lower() or "create_pr not" in out.lower() + assert "Step 2" in out + assert "LeaseDecision" in out or "lease" in out.lower() + assert "Step 3" in out + assert "create_pr present" in out.lower() or "create_pr" in out + assert "Step 4" in out + assert "Step 5" in out + assert "Agent adapts plan: using draft_pr instead" in out From f629cea1ba4887a23361d91e7ca06497f96a3fb6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 19:30:57 +0000 Subject: [PATCH 027/138] feat(capability-runtime): add network capability types and physical backend protocol Co-authored-by: Cursor --- .../src/capability_runtime/network.py | 20 +++++++++++++++ .../src/capability_runtime/types.py | 12 +++++++++ capability-runtime/tests/test_network.py | 25 +++++++++++++++++++ 3 files changed, 57 insertions(+) create mode 100644 capability-runtime/src/capability_runtime/network.py create mode 100644 capability-runtime/tests/test_network.py diff --git a/capability-runtime/src/capability_runtime/network.py b/capability-runtime/src/capability_runtime/network.py new file mode 100644 index 00000000..461ed6fe --- /dev/null +++ b/capability-runtime/src/capability_runtime/network.py @@ -0,0 +1,20 @@ +from __future__ import annotations + +from dataclasses import dataclass +from typing import Protocol + +from capability_runtime.types import CapabilityRef + + +@dataclass +class NetworkBinding: + capability_ref: CapabilityRef + peer_id: str + network: str + route_id: str | None + + +class PhysicalRevocationBackend(Protocol): + def revoke_peer(self, peer_id: str, reason: str) -> None: ... + + def revoke_route(self, route_id: str, reason: str) -> None: ... diff --git a/capability-runtime/src/capability_runtime/types.py b/capability-runtime/src/capability_runtime/types.py index b44cc501..087bb05a 100644 --- a/capability-runtime/src/capability_runtime/types.py +++ b/capability-runtime/src/capability_runtime/types.py @@ -43,6 +43,18 @@ class ToolCapability: expires_at: datetime | None +@dataclass +class NetworkCapability: + ref: CapabilityRef + name: str + peer_id: str + network: str + route_id: str | None + lease_id: LeaseId | None + quota: Quota = "unbounded" + expires_at: datetime | None = None + + @dataclass class RuleCapability: ref: CapabilityRef diff --git a/capability-runtime/tests/test_network.py b/capability-runtime/tests/test_network.py new file mode 100644 index 00000000..5d70af95 --- /dev/null +++ b/capability-runtime/tests/test_network.py @@ -0,0 +1,25 @@ +from capability_runtime.network import NetworkBinding, PhysicalRevocationBackend +from capability_runtime.types import CapabilityRef, NetworkCapability + + +def test_network_capability_in_bundle(): + cap = NetworkCapability( + ref=CapabilityRef("cap-reach-api"), + name="reach_api", + peer_id="peer-abc", + network="10.8.0.0/24", + route_id="route-1", + lease_id=None, + ) + assert cap.peer_id == "peer-abc" + assert cap.network == "10.8.0.0/24" + + +def test_network_binding_dataclass(): + binding = NetworkBinding( + capability_ref=CapabilityRef("cap-reach-api"), + peer_id="peer-abc", + network="10.8.0.0/24", + route_id="route-1", + ) + assert binding.route_id == "route-1" From 9ee62740563a04df9ab65ae62553f42ab06b1753 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 19:34:46 +0000 Subject: [PATCH 028/138] feat(capability-runtime): add NetBird client with mock and REST implementations Co-authored-by: Cursor --- .../src/capability_runtime/netbird_client.py | 117 ++++++++++++++ .../tests/test_netbird_client.py | 146 ++++++++++++++++++ 2 files changed, 263 insertions(+) create mode 100644 capability-runtime/src/capability_runtime/netbird_client.py create mode 100644 capability-runtime/tests/test_netbird_client.py diff --git a/capability-runtime/src/capability_runtime/netbird_client.py b/capability-runtime/src/capability_runtime/netbird_client.py new file mode 100644 index 00000000..76e88ac5 --- /dev/null +++ b/capability-runtime/src/capability_runtime/netbird_client.py @@ -0,0 +1,117 @@ +from __future__ import annotations + +import json +import os +from typing import Protocol +from urllib.error import HTTPError +from urllib.request import Request, urlopen + + +class NetBirdClient(Protocol): + def list_peers(self) -> list[dict]: ... + + def list_routes(self) -> list[dict]: ... + + def remove_peer(self, peer_id: str) -> None: ... + + def remove_route(self, route_id: str) -> None: ... + + def peer_exists(self, peer_id: str) -> bool: ... + + def route_exists(self, route_id: str) -> bool: ... + + +class MockNetBirdClient: + def __init__( + self, + peers: list[dict] | None = None, + routes: list[dict] | None = None, + ) -> None: + self._peers = list(peers or []) + self._routes = list(routes or []) + + def list_peers(self) -> list[dict]: + return list(self._peers) + + def list_routes(self) -> list[dict]: + return list(self._routes) + + def remove_peer(self, peer_id: str) -> None: + self._peers = [peer for peer in self._peers if peer.get("id") != peer_id] + + def remove_route(self, route_id: str) -> None: + self._routes = [route for route in self._routes if route.get("id") != route_id] + + def peer_exists(self, peer_id: str) -> bool: + return any(peer.get("id") == peer_id for peer in self._peers) + + def route_exists(self, route_id: str) -> bool: + return any(route.get("id") == route_id for route in self._routes) + + +class RestNetBirdClient: + def __init__( + self, + *, + api_token: str | None = None, + management_url: str | None = None, + ) -> None: + self._token = api_token if api_token is not None else os.environ.get("NB_API_TOKEN") + self._management_url = ( + management_url + if management_url is not None + else os.environ.get("NB_MANAGEMENT_URL", "https://api.netbird.io") + ) + + def list_peers(self) -> list[dict]: + return self._request("GET", "/api/peers") + + def list_routes(self) -> list[dict]: + return self._request("GET", "/api/routes") + + def remove_peer(self, peer_id: str) -> None: + self._request("DELETE", f"/api/peers/{peer_id}") + + def remove_route(self, route_id: str) -> None: + self._request("DELETE", f"/api/routes/{route_id}") + + def peer_exists(self, peer_id: str) -> bool: + return any(peer.get("id") == peer_id for peer in self.list_peers()) + + def route_exists(self, route_id: str) -> bool: + return any(route.get("id") == route_id for route in self.list_routes()) + + def _management_base_url(self) -> str: + base = self._management_url.rstrip("/") + if base.endswith("/api"): + base = base[: -len("/api")] + return base + + def _require_token(self) -> str: + if not self._token: + raise RuntimeError("NB_API_TOKEN is required for NetBird REST API calls") + return self._token + + def _request(self, method: str, path: str) -> list[dict]: + url = f"{self._management_base_url()}{path}" + request = Request( + url, + method=method, + headers={ + "Authorization": f"Token {self._require_token()}", + "Accept": "application/json", + "Content-Type": "application/json", + }, + ) + try: + with urlopen(request) as response: + body = response.read().decode() + except HTTPError: + raise + + if not body: + return [] + data = json.loads(body) + if isinstance(data, list): + return data + return [data] diff --git a/capability-runtime/tests/test_netbird_client.py b/capability-runtime/tests/test_netbird_client.py new file mode 100644 index 00000000..27c0c841 --- /dev/null +++ b/capability-runtime/tests/test_netbird_client.py @@ -0,0 +1,146 @@ +"""Tests for NetBird client (mock + REST).""" + +from __future__ import annotations + +import json +from io import BytesIO +from urllib.error import HTTPError + +import pytest + +from capability_runtime.netbird_client import MockNetBirdClient, RestNetBirdClient + + +def test_mock_client_tracks_peers_and_routes(): + client = MockNetBirdClient( + peers=[{"id": "peer-abc", "connected": True}], + routes=[{"id": "route-1", "network": "10.8.0.0/24"}], + ) + assert client.peer_exists("peer-abc") + assert client.list_peers() == [{"id": "peer-abc", "connected": True}] + assert client.list_routes() == [{"id": "route-1", "network": "10.8.0.0/24"}] + client.remove_peer("peer-abc") + assert not client.peer_exists("peer-abc") + assert client.list_peers() == [] + + +def test_mock_remove_route(): + client = MockNetBirdClient(routes=[{"id": "route-1", "network": "10.8.0.0/24"}]) + assert client.route_exists("route-1") + client.remove_route("route-1") + assert not client.route_exists("route-1") + assert client.list_routes() == [] + + +def test_mock_defaults_to_empty_state(): + client = MockNetBirdClient() + assert client.list_peers() == [] + assert client.list_routes() == [] + assert not client.peer_exists("missing") + assert not client.route_exists("missing") + + +def test_rest_client_remove_peer(monkeypatch): + captured: dict = {} + + def fake_urlopen(request): + captured["url"] = request.full_url + captured["method"] = request.get_method() + captured["headers"] = dict(request.header_items()) + return BytesIO(b"") + + monkeypatch.setenv("NB_API_TOKEN", "test-token") + monkeypatch.setenv("NB_MANAGEMENT_URL", "https://api.netbird.io") + monkeypatch.setattr("capability_runtime.netbird_client.urlopen", fake_urlopen) + + client = RestNetBirdClient() + client.remove_peer("peer-abc") + + assert captured["method"] == "DELETE" + assert captured["url"] == "https://api.netbird.io/api/peers/peer-abc" + assert captured["headers"]["Authorization"] == "Token test-token" + + +def test_rest_client_remove_route(monkeypatch): + captured: dict = {} + + def fake_urlopen(request): + captured["url"] = request.full_url + captured["method"] = request.get_method() + return BytesIO(b"") + + monkeypatch.setenv("NB_API_TOKEN", "test-token") + monkeypatch.setattr("capability_runtime.netbird_client.urlopen", fake_urlopen) + + client = RestNetBirdClient() + client.remove_route("route-1") + + assert captured["method"] == "DELETE" + assert captured["url"] == "https://api.netbird.io/api/routes/route-1" + + +def test_rest_client_list_peers(monkeypatch): + peers = [{"id": "peer-abc", "connected": True}] + + def fake_urlopen(request): + assert request.get_method() == "GET" + assert request.full_url == "https://api.netbird.io/api/peers" + return BytesIO(json.dumps(peers).encode()) + + monkeypatch.setenv("NB_API_TOKEN", "test-token") + monkeypatch.setattr("capability_runtime.netbird_client.urlopen", fake_urlopen) + + client = RestNetBirdClient() + assert client.list_peers() == peers + + +def test_rest_client_peer_exists(monkeypatch): + peers = [{"id": "peer-abc", "connected": True}] + + def fake_urlopen(request): + return BytesIO(json.dumps(peers).encode()) + + monkeypatch.setenv("NB_API_TOKEN", "test-token") + monkeypatch.setattr("capability_runtime.netbird_client.urlopen", fake_urlopen) + + client = RestNetBirdClient() + assert client.peer_exists("peer-abc") + assert not client.peer_exists("peer-missing") + + +def test_rest_client_strips_trailing_slash_and_api_suffix(monkeypatch): + captured: dict = {} + + def fake_urlopen(request): + captured["url"] = request.full_url + return BytesIO(b"[]") + + monkeypatch.setenv("NB_API_TOKEN", "test-token") + monkeypatch.setenv("NB_MANAGEMENT_URL", "https://example.com/api/") + monkeypatch.setattr("capability_runtime.netbird_client.urlopen", fake_urlopen) + + RestNetBirdClient().list_peers() + assert captured["url"] == "https://example.com/api/peers" + + +def test_rest_client_requires_api_token(monkeypatch): + monkeypatch.delenv("NB_API_TOKEN", raising=False) + with pytest.raises(RuntimeError, match="NB_API_TOKEN"): + RestNetBirdClient().list_peers() + + +def test_rest_client_raises_on_http_error(monkeypatch): + def fake_urlopen(request): + raise HTTPError( + request.full_url, + 404, + "Not Found", + hdrs=None, + fp=BytesIO(b'{"message":"not found"}'), + ) + + monkeypatch.setenv("NB_API_TOKEN", "test-token") + monkeypatch.setattr("capability_runtime.netbird_client.urlopen", fake_urlopen) + + with pytest.raises(HTTPError): + RestNetBirdClient().remove_peer("peer-missing") From a440e11af3ad7d18fb8f0a9fba8abf2565704010 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 19:35:38 +0000 Subject: [PATCH 029/138] feat(capability-runtime): add NetBirdRevocationBackend for physical network revoke Implements PhysicalRevocationBackend by delegating route and peer removal to an injectable NetBirdClient. Co-authored-by: Cursor --- .../src/capability_runtime/netbird_backend.py | 21 +++++++++++++++++ .../tests/test_netbird_backend.py | 23 +++++++++++++++++++ 2 files changed, 44 insertions(+) create mode 100644 capability-runtime/src/capability_runtime/netbird_backend.py create mode 100644 capability-runtime/tests/test_netbird_backend.py diff --git a/capability-runtime/src/capability_runtime/netbird_backend.py b/capability-runtime/src/capability_runtime/netbird_backend.py new file mode 100644 index 00000000..a831ff84 --- /dev/null +++ b/capability-runtime/src/capability_runtime/netbird_backend.py @@ -0,0 +1,21 @@ +from __future__ import annotations + +from capability_runtime.netbird_client import NetBirdClient +from capability_runtime.network import NetworkBinding + + +class NetBirdRevocationBackend: + def __init__(self, client: NetBirdClient) -> None: + self._client = client + + def revoke_binding(self, binding: NetworkBinding, reason: str) -> None: + if binding.route_id: + self.revoke_route(binding.route_id, reason) + if binding.peer_id: + self.revoke_peer(binding.peer_id, reason) + + def revoke_peer(self, peer_id: str, reason: str) -> None: + self._client.remove_peer(peer_id) + + def revoke_route(self, route_id: str, reason: str) -> None: + self._client.remove_route(route_id) diff --git a/capability-runtime/tests/test_netbird_backend.py b/capability-runtime/tests/test_netbird_backend.py new file mode 100644 index 00000000..0fbd1140 --- /dev/null +++ b/capability-runtime/tests/test_netbird_backend.py @@ -0,0 +1,23 @@ +"""Tests for NetBirdRevocationBackend.""" + +from capability_runtime.netbird_backend import NetBirdRevocationBackend +from capability_runtime.netbird_client import MockNetBirdClient +from capability_runtime.network import NetworkBinding +from capability_runtime.types import CapabilityRef + + +def test_backend_revokes_route_then_peer(): + client = MockNetBirdClient( + peers=[{"id": "peer-abc", "connected": True}], + routes=[{"id": "route-1", "network": "10.8.0.0/24", "peer": "peer-abc"}], + ) + backend = NetBirdRevocationBackend(client) + binding = NetworkBinding( + capability_ref=CapabilityRef("cap-reach-api"), + peer_id="peer-abc", + network="10.8.0.0/24", + route_id="route-1", + ) + backend.revoke_binding(binding, reason="policy") + assert not client.route_exists("route-1") + assert not client.peer_exists("peer-abc") From 3b306125989d442aac49c1f46de36e557529aad8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 19:38:52 +0000 Subject: [PATCH 030/138] Add network capability catalog and bundle assembly Implements Task 4: Catalog network bindings + bundle assembly - Add network binding storage to catalog (set/get/iter methods) - Add networks field to CapabilityBundle with default empty list - Add register_network_capability() to runtime - Include network capabilities in check_current_capabilities bundle - Add CallerIdentityMismatch error for test compatibility - Network capabilities with bindings appear in bundle.networks - Test verifies visible network caps are included in bundle TDD approach: wrote test first, watched it fail, implemented minimal code. All 51 tests pass (excluding pre-existing test_security.py failures). Co-authored-by: Cursor --- .../src/capability_runtime/catalog.py | 14 ++++ .../src/capability_runtime/errors.py | 5 ++ .../src/capability_runtime/runtime.py | 82 +++++++++++++++---- .../src/capability_runtime/types.py | 1 + .../tests/test_runtime_network.py | 17 ++++ 5 files changed, 103 insertions(+), 16 deletions(-) create mode 100644 capability-runtime/tests/test_runtime_network.py diff --git a/capability-runtime/src/capability_runtime/catalog.py b/capability-runtime/src/capability_runtime/catalog.py index 33db76f0..db6c7134 100644 --- a/capability-runtime/src/capability_runtime/catalog.py +++ b/capability-runtime/src/capability_runtime/catalog.py @@ -1,10 +1,14 @@ from __future__ import annotations from enum import StrEnum +from typing import TYPE_CHECKING from capability_runtime.errors import CapabilityUnknown from capability_runtime.types import CapabilityRef +if TYPE_CHECKING: + from capability_runtime.network import NetworkBinding + class LifecycleState(StrEnum): DECLARED = "Declared" @@ -21,6 +25,7 @@ def __init__(self) -> None: self._by_ref: dict[CapabilityRef, LifecycleState] = {} self._by_name: dict[str, CapabilityRef] = {} self._name_by_ref: dict[CapabilityRef, str] = {} + self._network_bindings: dict[CapabilityRef, NetworkBinding] = {} def register( self, @@ -62,3 +67,12 @@ def get_by_name(self, name: str) -> CapabilityRef | None: def get_name(self, ref: CapabilityRef) -> str | None: return self._name_by_ref.get(ref) + + def set_network_binding(self, ref: CapabilityRef, binding: NetworkBinding) -> None: + self._network_bindings[ref] = binding + + def get_network_binding(self, ref: CapabilityRef) -> NetworkBinding | None: + return self._network_bindings.get(ref) + + def iter_network_bindings(self): + return iter(self._network_bindings.items()) diff --git a/capability-runtime/src/capability_runtime/errors.py b/capability-runtime/src/capability_runtime/errors.py index 3da65204..75eedce1 100644 --- a/capability-runtime/src/capability_runtime/errors.py +++ b/capability-runtime/src/capability_runtime/errors.py @@ -35,3 +35,8 @@ def __init__(self, expected: int, actual: int): self.expected = expected self.actual = actual super().__init__(f"Bundle version mismatch: expected {expected}, got {actual}") + + +class CallerIdentityMismatch(CapabilityRuntimeError): + def __init__(self, message: str): + super().__init__(message) diff --git a/capability-runtime/src/capability_runtime/runtime.py b/capability-runtime/src/capability_runtime/runtime.py index 5d3dc475..94bde42a 100644 --- a/capability-runtime/src/capability_runtime/runtime.py +++ b/capability-runtime/src/capability_runtime/runtime.py @@ -18,6 +18,8 @@ CapabilityUnknown, ) from capability_runtime.lease import LeaseManager +from capability_runtime.network import NetworkBinding +from capability_runtime.netbird_client import NetBirdClient from capability_runtime.observability import ObservabilityCollector from capability_runtime.revoke import RevocationManager from capability_runtime.types import ( @@ -27,6 +29,7 @@ CapabilityRef, LeaseDecision, LeaseId, + NetworkCapability, ProvenanceRecord, ToolCapability, ) @@ -41,6 +44,7 @@ def __init__( trace_id: str, seed: int, policy_version: str = "1.0.0", + netbird_client: NetBirdClient | None = None, ): self.catalog = CapabilityCatalog() self.lease_manager = LeaseManager() @@ -50,10 +54,22 @@ def __init__( self.trace_id = trace_id self._bundle_version = 0 self._current_bundles: dict[AgentIdentity, int] = {} + self._netbird_client = netbird_client # Register create_pr as DECLARED (invisible until leased) self.catalog.register("create_pr", CapabilityRef("cap-create-pr"), LifecycleState.DECLARED) + def register_network_capability( + self, + name: str, + ref: CapabilityRef, + binding: NetworkBinding, + initial_state: LifecycleState = LifecycleState.DECLARED, + ) -> None: + """Register a network capability with its binding.""" + self.catalog.register(name, ref, initial_state) + self.catalog.set_network_binding(ref, binding) + def check_current_capabilities( self, agent_id: AgentIdentity, context: dict ) -> CapabilityBundle: @@ -63,6 +79,7 @@ def check_current_capabilities( now = datetime.now(timezone.utc) tools: list[ToolCapability] = [] + networks: list[NetworkCapability] = [] earliest_expiry: datetime | None = None # Iterate through catalog to find visible capabilities @@ -74,15 +91,31 @@ def check_current_capabilities( # Include if Visible OR has active lease if state == LifecycleState.VISIBLE: - tools.append( - ToolCapability( - ref=ref, - name=name, - lease_id=None, - quota="unbounded", - expires_at=None, + # Check if this is a network capability + binding = self.catalog.get_network_binding(ref) + if binding is not None: + networks.append( + NetworkCapability( + ref=ref, + name=name, + peer_id=binding.peer_id, + network=binding.network, + route_id=binding.route_id, + lease_id=None, + quota="unbounded", + expires_at=None, + ) + ) + else: + tools.append( + ToolCapability( + ref=ref, + name=name, + lease_id=None, + quota="unbounded", + expires_at=None, + ) ) - ) elif state == LifecycleState.LEASED: # Find active lease for this agent for lease_id, lease in self.lease_manager._leases.items(): @@ -94,15 +127,31 @@ def check_current_capabilities( and not self.lease_manager.is_exhausted(lease_id) ): remaining = self.lease_manager._remaining_quota[lease_id] - tools.append( - ToolCapability( - ref=ref, - name=name, - lease_id=lease_id, - quota=remaining, - expires_at=lease.expires_at, + # Check if this is a network capability + binding = self.catalog.get_network_binding(ref) + if binding is not None: + networks.append( + NetworkCapability( + ref=ref, + name=name, + peer_id=binding.peer_id, + network=binding.network, + route_id=binding.route_id, + lease_id=lease_id, + quota=remaining, + expires_at=lease.expires_at, + ) + ) + else: + tools.append( + ToolCapability( + ref=ref, + name=name, + lease_id=lease_id, + quota=remaining, + expires_at=lease.expires_at, + ) ) - ) if earliest_expiry is None or lease.expires_at < earliest_expiry: earliest_expiry = lease.expires_at break @@ -126,6 +175,7 @@ def check_current_capabilities( policy_version=self.policy_version, trace_id=self.trace_id, ), + networks=networks, version=self._bundle_version, ) diff --git a/capability-runtime/src/capability_runtime/types.py b/capability-runtime/src/capability_runtime/types.py index 087bb05a..12765cc5 100644 --- a/capability-runtime/src/capability_runtime/types.py +++ b/capability-runtime/src/capability_runtime/types.py @@ -109,4 +109,5 @@ class CapabilityBundle: hooks: list[HookCapability] budgets: BudgetEnvelope provenance: ProvenanceRecord + networks: list[NetworkCapability] = field(default_factory=list) version: int = field(default=1) diff --git a/capability-runtime/tests/test_runtime_network.py b/capability-runtime/tests/test_runtime_network.py new file mode 100644 index 00000000..f7a0054b --- /dev/null +++ b/capability-runtime/tests/test_runtime_network.py @@ -0,0 +1,17 @@ +def test_register_network_capability_appears_in_bundle_when_visible(tmp_path): + from capability_runtime.catalog import LifecycleState + from capability_runtime.netbird_client import MockNetBirdClient + from capability_runtime.network import NetworkBinding + from capability_runtime.runtime import CapabilityRuntime + from capability_runtime.types import AgentIdentity, CapabilityRef + + client = MockNetBirdClient() + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-n1", 1, netbird_client=client) + agent = AgentIdentity("agent-1") + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", "route-1") + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + bundle = runtime.check_current_capabilities(agent, {}) + assert any(n.name == "reach_api" for n in bundle.networks) + cap = next(n for n in bundle.networks if n.name == "reach_api") + assert cap.peer_id == "peer-abc" From 5d14c154b16658eebb512b0079ddde94375f8eac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 19:41:48 +0000 Subject: [PATCH 031/138] Wire logical revoke to physical NetBird backend Implements Task 5: - Create NetBirdRevocationBackend in CapabilityRuntime.__init__ when netbird_client is provided - When revoke_capability is called with a CapabilityRef that has a network binding: - Call NetBirdRevocationBackend.revoke_binding(binding, reason) using self._netbird_client - Perform logical revoke via revocation_manager - Invalidate agent bundle cache by incrementing _bundle_version - Emit capability event with "physical_revocation": true - Add tests for network capability revocation calling NetBird backend - Add test verifying non-network tool revoke does NOT call NetBird client All 53 tests pass. Co-authored-by: Cursor --- .../src/capability_runtime/runtime.py | 19 +++++++++ .../tests/test_runtime_network.py | 42 +++++++++++++++++++ 2 files changed, 61 insertions(+) diff --git a/capability-runtime/src/capability_runtime/runtime.py b/capability-runtime/src/capability_runtime/runtime.py index 94bde42a..c5d2d855 100644 --- a/capability-runtime/src/capability_runtime/runtime.py +++ b/capability-runtime/src/capability_runtime/runtime.py @@ -19,6 +19,7 @@ ) from capability_runtime.lease import LeaseManager from capability_runtime.network import NetworkBinding +from capability_runtime.netbird_backend import NetBirdRevocationBackend from capability_runtime.netbird_client import NetBirdClient from capability_runtime.observability import ObservabilityCollector from capability_runtime.revoke import RevocationManager @@ -55,6 +56,9 @@ def __init__( self._bundle_version = 0 self._current_bundles: dict[AgentIdentity, int] = {} self._netbird_client = netbird_client + self._netbird_backend = ( + NetBirdRevocationBackend(netbird_client) if netbird_client is not None else None + ) # Register create_pr as DECLARED (invisible until leased) self.catalog.register("create_pr", CapabilityRef("cap-create-pr"), LifecycleState.DECLARED) @@ -247,6 +251,8 @@ def revoke_capability( self, target: Union[LeaseId, CapabilityRef], reason: str ) -> None: """Revoke capability by lease ID or ref (spec §3.2.3).""" + physical_revocation = False + if isinstance(target, LeaseId): self.revocation_manager.revoke_by_lease(target, reason) self.observability.emit_capability_event( @@ -257,12 +263,25 @@ def revoke_capability( } ) else: + # Check if this is a network capability + binding = self.catalog.get_network_binding(target) + if binding is not None and self._netbird_backend is not None: + # Perform physical revocation via NetBird backend + self._netbird_backend.revoke_binding(binding, reason) + physical_revocation = True + + # Perform logical revocation self.revocation_manager.revoke_by_ref(target, reason) + + # Invalidate agent bundle cache + self._bundle_version += 1 + self.observability.emit_capability_event( { "event": "capability_revoked", "capability_ref": target.value, "reason": reason, + "physical_revocation": physical_revocation, } ) diff --git a/capability-runtime/tests/test_runtime_network.py b/capability-runtime/tests/test_runtime_network.py index f7a0054b..8abff527 100644 --- a/capability-runtime/tests/test_runtime_network.py +++ b/capability-runtime/tests/test_runtime_network.py @@ -15,3 +15,45 @@ def test_register_network_capability_appears_in_bundle_when_visible(tmp_path): assert any(n.name == "reach_api" for n in bundle.networks) cap = next(n for n in bundle.networks if n.name == "reach_api") assert cap.peer_id == "peer-abc" + + +def test_revoke_network_capability_calls_netbird_backend(tmp_path): + from capability_runtime.catalog import LifecycleState + from capability_runtime.netbird_client import MockNetBirdClient + from capability_runtime.network import NetworkBinding + from capability_runtime.runtime import CapabilityRuntime + from capability_runtime.types import AgentIdentity, CapabilityRef + + client = MockNetBirdClient( + peers=[{"id": "peer-abc", "connected": True}], + routes=[{"id": "route-1", "network": "10.8.0.0/24"}], + ) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-n2", 2, netbird_client=client) + agent = AgentIdentity("agent-1") + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", "route-1") + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + runtime.revoke_capability(ref, "security") + assert not client.peer_exists("peer-abc") + bundle = runtime.check_current_capabilities(agent, {}) + assert "reach_api" not in [n.name for n in bundle.networks] + + +def test_revoke_non_network_capability_does_not_call_netbird(tmp_path): + from capability_runtime.catalog import LifecycleState + from capability_runtime.netbird_client import MockNetBirdClient + from capability_runtime.runtime import CapabilityRuntime + from capability_runtime.types import AgentIdentity, CapabilityRef + + client = MockNetBirdClient( + peers=[{"id": "peer-xyz", "connected": True}], + ) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-n3", 3, netbird_client=client) + agent = AgentIdentity("agent-1") + ref = CapabilityRef("cap-create-pr") + runtime.catalog.register("create_pr_tool", ref, LifecycleState.VISIBLE) + runtime.revoke_capability(ref, "security") + # Peer should still exist because we didn't revoke a network capability + assert client.peer_exists("peer-xyz") + bundle = runtime.check_current_capabilities(agent, {}) + assert "create_pr_tool" not in [t.name for t in bundle.tools] From 60db8c71073c43a85e3247c2e855a218d7ecc317 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 19:45:31 +0000 Subject: [PATCH 032/138] feat: implement RouteDisappearanceWatcher for physical-to-logical revocation Add RouteDisappearanceWatcher to detect when NetBird peers disappear externally (e.g., via wg syncconf) and perform logical-only revocation. Changes: - Add route_watcher.py with RouteDisappearanceWatcher class - Add revoke_from_physical() method to CapabilityRuntime - Add 5 comprehensive tests in test_route_watcher.py - Emit capability events with physical_trigger flag The watcher polls NetBird client and compares against catalog bindings, revoking capabilities whose peers no longer exist without calling the NetBird API (since physical resource already gone). Tests: 58 passing (5 new route watcher tests) Co-authored-by: Cursor --- .../src/capability_runtime/route_watcher.py | 51 ++++++ .../src/capability_runtime/runtime.py | 28 +++ .../tests/test_route_watcher.py | 166 ++++++++++++++++++ 3 files changed, 245 insertions(+) create mode 100644 capability-runtime/src/capability_runtime/route_watcher.py create mode 100644 capability-runtime/tests/test_route_watcher.py diff --git a/capability-runtime/src/capability_runtime/route_watcher.py b/capability-runtime/src/capability_runtime/route_watcher.py new file mode 100644 index 00000000..0b8ff53e --- /dev/null +++ b/capability-runtime/src/capability_runtime/route_watcher.py @@ -0,0 +1,51 @@ +"""RouteDisappearanceWatcher — detect physical route removal and trigger logical revoke.""" + +from __future__ import annotations + +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from capability_runtime.netbird_client import NetBirdClient + from capability_runtime.runtime import CapabilityRuntime + + +class RouteDisappearanceWatcher: + """Monitor NetBird peers and trigger logical revoke when they disappear. + + This watcher detects when a peer (and its associated routes) no longer exist + in NetBird, typically because they were removed externally via `wg syncconf` + or NetBird API. When detected, it performs a logical-only revocation since + the physical resource is already gone. + """ + + def __init__(self, runtime: CapabilityRuntime, client: NetBirdClient): + """Initialize watcher with runtime and NetBird client. + + Args: + runtime: The capability runtime to revoke from + client: NetBird client to query current peer state + """ + self._runtime = runtime + self._client = client + + def poll_once(self) -> None: + """Poll once for disappeared peers and revoke their capabilities. + + For each network binding registered in the catalog, checks if the + associated peer_id still exists in NetBird. If not, performs a + logical-only revocation via runtime.revoke_from_physical(). + """ + # Collect all network bindings from catalog + bindings_to_revoke = [] + + for ref in self._runtime.catalog._by_ref: + binding = self._runtime.catalog.get_network_binding(ref) + if binding is not None: + # Check if peer still exists + if not self._client.peer_exists(binding.peer_id): + bindings_to_revoke.append(binding) + + # Revoke all disappeared bindings + for binding in bindings_to_revoke: + reason = f"peer {binding.peer_id} disappeared from NetBird" + self._runtime.revoke_from_physical(binding, reason) diff --git a/capability-runtime/src/capability_runtime/runtime.py b/capability-runtime/src/capability_runtime/runtime.py index c5d2d855..3b64e528 100644 --- a/capability-runtime/src/capability_runtime/runtime.py +++ b/capability-runtime/src/capability_runtime/runtime.py @@ -285,6 +285,34 @@ def revoke_capability( } ) + def revoke_from_physical(self, binding: NetworkBinding, reason: str) -> None: + """Perform logical revoke when physical route already disappeared. + + This is called by RouteDisappearanceWatcher when it detects that a peer + or route no longer exists in NetBird. Since the physical resource is + already gone, we only perform logical revocation without calling the + NetBird backend. + + Args: + binding: The network binding that disappeared + reason: Why the revocation occurred + """ + # Perform logical revocation only (no NetBird API call) + self.revocation_manager.revoke_by_ref(binding.capability_ref, reason) + + # Invalidate agent bundle cache + self._bundle_version += 1 + + # Emit event with physical_trigger flag + self.observability.emit_capability_event( + { + "event": "capability_revoked", + "capability_ref": binding.capability_ref.value, + "reason": reason, + "physical_trigger": True, + } + ) + def discover_capabilities( self, agent_id: AgentIdentity, intent: DiscoveryIntent ) -> DiscoveryResult: diff --git a/capability-runtime/tests/test_route_watcher.py b/capability-runtime/tests/test_route_watcher.py new file mode 100644 index 00000000..f76b0b5b --- /dev/null +++ b/capability-runtime/tests/test_route_watcher.py @@ -0,0 +1,166 @@ +"""Tests for RouteDisappearanceWatcher (physical to logical revoke).""" + +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from capability_runtime.catalog import LifecycleState +from capability_runtime.netbird_client import MockNetBirdClient +from capability_runtime.network import NetworkBinding +from capability_runtime.route_watcher import RouteDisappearanceWatcher +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity, CapabilityRef + + +def test_watcher_detects_removed_peer_and_revokes_runtime(tmp_path: Path): + """Watcher detects peer removal and performs logical revoke.""" + client = MockNetBirdClient( + peers=[{"id": "peer-abc", "connected": True}], + routes=[], + ) + runtime = CapabilityRuntime( + tmp_path / "t.jsonl", + "trace-w1", + 3, + netbird_client=client, + ) + + # Register a network capability + agent = AgentIdentity("agent-1") + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", None) + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + + # Verify capability is visible + bundle_before = runtime.check_current_capabilities(agent, {}) + assert "reach_api" in [n.name for n in bundle_before.networks] + + # Simulate external wg syncconf removing peer + client.remove_peer("peer-abc") + + # Run watcher + watcher = RouteDisappearanceWatcher(runtime, client) + watcher.poll_once() + + # Verify capability is revoked + bundle_after = runtime.check_current_capabilities(agent, {}) + assert "reach_api" not in [n.name for n in bundle_after.networks] + + +def test_watcher_handles_multiple_peers_selectively(tmp_path: Path): + """Watcher only revokes capabilities for removed peers.""" + client = MockNetBirdClient( + peers=[ + {"id": "peer-abc", "connected": True}, + {"id": "peer-def", "connected": True}, + ], + routes=[], + ) + runtime = CapabilityRuntime( + tmp_path / "t.jsonl", + "trace-w2", + 4, + netbird_client=client, + ) + + agent = AgentIdentity("agent-1") + ref1 = CapabilityRef("cap-reach-api") + ref2 = CapabilityRef("cap-reach-db") + binding1 = NetworkBinding(ref1, "peer-abc", "10.8.0.0/24", None) + binding2 = NetworkBinding(ref2, "peer-def", "10.8.1.0/24", None) + + runtime.register_network_capability("reach_api", ref1, binding1, LifecycleState.VISIBLE) + runtime.register_network_capability("reach_db", ref2, binding2, LifecycleState.VISIBLE) + + # Remove only one peer + client.remove_peer("peer-abc") + + watcher = RouteDisappearanceWatcher(runtime, client) + watcher.poll_once() + + bundle = runtime.check_current_capabilities(agent, {}) + names = [n.name for n in bundle.networks] + assert "reach_api" not in names + assert "reach_db" in names + + +def test_watcher_emits_capability_event_with_physical_trigger(tmp_path: Path): + """Watcher emits event with physical_trigger flag.""" + client = MockNetBirdClient( + peers=[{"id": "peer-abc", "connected": True}], + routes=[], + ) + runtime = CapabilityRuntime( + tmp_path / "t.jsonl", + "trace-w3", + 5, + netbird_client=client, + ) + + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", None) + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + + client.remove_peer("peer-abc") + + watcher = RouteDisappearanceWatcher(runtime, client) + watcher.poll_once() + + # Read trace file to verify event + trace_lines = (tmp_path / "t.jsonl").read_text().strip().split("\n") + events = [json.loads(line) for line in trace_lines if line] + + revoke_events = [e for e in events if e.get("event") == "capability_revoked"] + assert len(revoke_events) > 0 + assert any(e.get("physical_trigger") is True for e in revoke_events) + + +def test_watcher_does_nothing_when_all_peers_exist(tmp_path: Path): + """Watcher is idempotent when no peers removed.""" + client = MockNetBirdClient( + peers=[{"id": "peer-abc", "connected": True}], + routes=[], + ) + runtime = CapabilityRuntime( + tmp_path / "t.jsonl", + "trace-w4", + 6, + netbird_client=client, + ) + + agent = AgentIdentity("agent-1") + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", None) + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + + bundle_before = runtime.check_current_capabilities(agent, {}) + version_before = bundle_before.version + + watcher = RouteDisappearanceWatcher(runtime, client) + watcher.poll_once() + + bundle_after = runtime.check_current_capabilities(agent, {}) + assert "reach_api" in [n.name for n in bundle_after.networks] + # Bundle version should have incremented due to check_current_capabilities calls + assert bundle_after.version > version_before + + +def test_watcher_handles_empty_catalog(tmp_path: Path): + """Watcher handles runtime with no network capabilities.""" + client = MockNetBirdClient( + peers=[{"id": "peer-abc", "connected": True}], + routes=[], + ) + runtime = CapabilityRuntime( + tmp_path / "t.jsonl", + "trace-w5", + 7, + netbird_client=client, + ) + + watcher = RouteDisappearanceWatcher(runtime, client) + # Should not raise + watcher.poll_once() From 11e334337207931d45dccd6c4e2c73bd01cb7d01 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 19:47:07 +0000 Subject: [PATCH 033/138] feat(capability-runtime): add PoC 3 network route demo Runnable vertical slice demonstrating reach_api lease, logical revoke with NetBird peer removal, and external disappearance via watcher. Co-authored-by: Cursor --- capability-runtime/poc/network_route_demo.py | 144 ++++++++++++++++++ .../tests/test_poc_network_route.py | 42 +++++ 2 files changed, 186 insertions(+) create mode 100644 capability-runtime/poc/network_route_demo.py create mode 100644 capability-runtime/tests/test_poc_network_route.py diff --git a/capability-runtime/poc/network_route_demo.py b/capability-runtime/poc/network_route_demo.py new file mode 100644 index 00000000..161a4d65 --- /dev/null +++ b/capability-runtime/poc/network_route_demo.py @@ -0,0 +1,144 @@ +"""PoC 3 — network route reachability demo (spec Phase 3). + +Runnable via: + PYTHONPATH=src:. python poc/network_route_demo.py + PYTHONPATH=src:. python -m poc.network_route_demo +""" + +from __future__ import annotations + +import tempfile +from dataclasses import dataclass +from pathlib import Path +from capability_runtime.catalog import LifecycleState +from capability_runtime.netbird_client import MockNetBirdClient +from capability_runtime.network import NetworkBinding +from capability_runtime.route_watcher import RouteDisappearanceWatcher +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity, CapabilityRef, LeaseDecision + + +@dataclass +class DemoResult: + initial_networks: list[str] + lease_decision: LeaseDecision + networks_after_lease: list[str] + peer_id_after_lease: str | None + peer_removed_by_revoke: bool + networks_after_revoke: list[str] + networks_after_external_removal: list[str] + external_removal_peer_gone: bool + + +def _network_names(bundle) -> list[str]: + return [n.name for n in bundle.networks] + + +def _peer_id_for(bundle, name: str) -> str | None: + for cap in bundle.networks: + if cap.name == name: + return cap.peer_id + return None + + +def _print_step(quiet: bool, step: int, message: str) -> None: + if not quiet: + print(f"Step {step}: {message}") + + +def run_poc3_demo(trace_path: Path, *, quiet: bool = False) -> DemoResult: + """Execute the Phase 3 network route lifecycle and external disappearance.""" + client = MockNetBirdClient( + peers=[{"id": "peer-abc", "connected": True}], + routes=[{"id": "route-1", "network": "10.8.0.0/24"}], + ) + runtime = CapabilityRuntime(trace_path, "poc3-network-route", seed=42, netbird_client=client) + watcher = RouteDisappearanceWatcher(runtime, client) + agent = AgentIdentity("demo-agent") + context: dict = {} + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", "route-1") + + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.DECLARED) + + bundle1 = runtime.check_current_capabilities(agent, context) + initial_networks = _network_names(bundle1) + _print_step( + quiet, + 1, + f"check_current_capabilities → networks={initial_networks} (reach_api absent)", + ) + + decision = runtime.request_capability_lease( + agent, + ref, + "Need API reachability for integration test", + ) + _print_step( + quiet, + 2, + f"register + request_capability_lease → LeaseDecision(lease_id={decision.lease_id.value!r}, " + f"quota={decision.quota}, token_budget={decision.token_budget}, " + f"risk_envelope={decision.risk_envelope!r})", + ) + + bundle2 = runtime.check_current_capabilities(agent, context) + networks_after_lease = _network_names(bundle2) + peer_id = _peer_id_for(bundle2, "reach_api") + _print_step( + quiet, + 3, + f"check_current_capabilities → networks={networks_after_lease} " + f"(reach_api present, peer_id={peer_id!r})", + ) + + runtime.revoke_capability(ref, "security policy") + peer_removed = not client.peer_exists("peer-abc") + _print_step( + quiet, + 4, + f"revoke_capability → mock client peer removed (peer_exists={client.peer_exists('peer-abc')})", + ) + + bundle3 = runtime.check_current_capabilities(agent, context) + networks_after_revoke = _network_names(bundle3) + _print_step( + quiet, + 5, + f"check_current_capabilities → networks={networks_after_revoke} (reach_api absent)", + ) + + client._peers.append({"id": "peer-abc", "connected": True}) + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + client.remove_peer("peer-abc") + watcher.poll_once() + bundle4 = runtime.check_current_capabilities(agent, context) + networks_after_external = _network_names(bundle4) + _print_step( + quiet, + 6, + f"re-register visible cap, external remove_peer + watcher.poll_once → " + f"networks={networks_after_external} (reach_api absent)", + ) + + return DemoResult( + initial_networks=initial_networks, + lease_decision=decision, + networks_after_lease=networks_after_lease, + peer_id_after_lease=peer_id, + peer_removed_by_revoke=peer_removed, + networks_after_revoke=networks_after_revoke, + networks_after_external_removal=networks_after_external, + external_removal_peer_gone=not client.peer_exists("peer-abc"), + ) + + +def main(trace_path: Path | None = None) -> None: + if trace_path is None: + with tempfile.NamedTemporaryFile(suffix=".jsonl", delete=False) as handle: + trace_path = Path(handle.name) + run_poc3_demo(trace_path, quiet=False) + + +if __name__ == "__main__": + main() diff --git a/capability-runtime/tests/test_poc_network_route.py b/capability-runtime/tests/test_poc_network_route.py new file mode 100644 index 00000000..eea63916 --- /dev/null +++ b/capability-runtime/tests/test_poc_network_route.py @@ -0,0 +1,42 @@ +"""Integration tests for PoC 3 network route demo (spec Phase 3).""" + +from __future__ import annotations + +from pathlib import Path + +from capability_runtime.types import CapabilityRef +from poc.network_route_demo import run_poc3_demo + + +def test_poc3_network_route_lifecycle(tmp_path: Path) -> None: + """reach_api absent → lease → present → revoke → absent → external removal stays absent.""" + result = run_poc3_demo(tmp_path / "trace.jsonl", quiet=True) + + assert "reach_api" not in result.initial_networks + assert result.lease_decision.capability_ref == CapabilityRef("cap-reach-api") + assert "reach_api" in result.networks_after_lease + assert result.peer_id_after_lease == "peer-abc" + assert result.peer_removed_by_revoke is True + assert "reach_api" not in result.networks_after_revoke + assert "reach_api" not in result.networks_after_external_removal + assert result.external_removal_peer_gone is True + + +def test_poc3_demo_main_prints_steps(capsys, tmp_path: Path) -> None: + """Runnable demo prints each step of the Phase 3 sequence.""" + from poc.network_route_demo import main + + main(trace_path=tmp_path / "trace.jsonl") + + out = capsys.readouterr().out + assert "Step 1" in out + assert "reach_api absent" in out.lower() + assert "Step 2" in out + assert "lease" in out.lower() + assert "Step 3" in out + assert "peer_id=" in out + assert "Step 4" in out + assert "revoke_capability" in out.lower() + assert "Step 5" in out + assert "Step 6" in out + assert "watcher.poll_once" in out.lower() or "watcher" in out.lower() From e1f259eb0df34afff85178cf04d05a7322bef37b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 22 Jun 2026 19:47:51 +0000 Subject: [PATCH 034/138] docs(capability-runtime): document Phase 3 NetBird networking scope Explain logical/physical revocation bridge, RouteDisappearanceWatcher, and PoC 3 quick start aligned with the Reachability == Capability thesis. Co-authored-by: Cursor --- capability-runtime/README.md | 106 +++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100644 capability-runtime/README.md diff --git a/capability-runtime/README.md b/capability-runtime/README.md new file mode 100644 index 00000000..97acd230 --- /dev/null +++ b/capability-runtime/README.md @@ -0,0 +1,106 @@ +# capability-runtime + +Phase 0+1+3 proof-of-concept for [capability-oriented networking](../../docs/superpowers/specs/2026-06-15-capability-oriented-networking-design.md). + +Implements the v1 `CapabilityRuntime` protocol: dynamic capability bundles, leased tools, revocation, observability with replay, an `AgentExecutionLoop` harness with optimistic check-then-act, and Phase 3 network reachability as a first-class capability. + +## Spec thesis: Reachability == Capability + +The design spec states that **reachability is capability**: an agent only has a network route when the runtime grants it in the current `CapabilityBundle`. Network endpoints are not ambient permissions — they appear after lease (or visibility grant) and disappear on revoke or physical route loss, with the same lifecycle semantics as leased tools. + +Phase 3 realizes this thesis by binding each network capability to a NetBird peer/route (`NetworkBinding`) and keeping logical runtime state aligned with physical WireGuard reachability. + +## Scope + +**In scope (Phase 0+1):** + +- `check_current_capabilities`, `request_capability_lease`, `revoke_capability`, `discover_capabilities` +- JSONL execution/capability event tracing with seed-gated replay +- PoC 1: `create_pr` (quota=1, ttl=10m) +- PoC 2: mock MCP `write_note` (quota=3, ttl=5m) + +**In scope (Phase 3 — networking realization):** + +- Network capabilities in `CapabilityBundle.networks` with `NetworkBinding` (peer, route, CIDR) +- **Logical revoke → physical:** `revoke_capability(ref, reason)` calls `NetBirdRevocationBackend` to remove peer/route via injectable `NetBirdClient`, then performs logical catalog revocation +- **Physical disappearance → logical revoke:** `RouteDisappearanceWatcher` polls NetBird peer state; when a bound peer vanishes (e.g. after external `wg syncconf`), calls `runtime.revoke_from_physical()` — logical-only, no duplicate NetBird API call +- PoC 3: `reach_api` network route lifecycle demo +- Observability events with `physical_revocation` and `physical_trigger` flags + +**Out of scope / known limitations:** + +- Token budget enforcement (`token_budget` is stored but not decremented) +- In-flight revocation policy (`allow_finish` / `interrupt`) — pre-action stale bundles fail closed; mid-action policy not implemented +- `TaskContext`-driven visibility rules +- Non-tool capability kinds (`rules`, `skills`, etc.) in bundles +- Multi-agent leasing on the same capability (global `LEASED` catalog state) +- Production sandcat wiring (`RestNetBirdClient` tokens, `sandcat capability` subcommand) — injectable client only in PoC + +## NetBird bridge (logical ↔ physical) + +Phase 3 connects the capability runtime to the sandcat NetBird deployment model described in [NetBird dynamic WireGuard plan](../../docs/superpowers/plans/2026-06-15-netbird-dynamic-wireguard.md): + +``` +Logical revoke (runtime) Physical path (sandcat) +───────────────────────── ─────────────────────── +revoke_capability(ref, reason) + → NetBirdRevocationBackend + → remove_peer / remove_route → NetBird management API + → catalog REVOKED → netbird container writes peers.conf + → emit physical_revocation → wg-client: wg syncconf wg0 + → agent loses route to endpoint +``` + +When revocation originates from the runtime, `NetBirdRevocationBackend.revoke_binding()` removes the bound peer and/or route through the injected `NetBirdClient`. In production this maps to NetBird API calls that eventually rewrite `peers.conf` and trigger `wg syncconf` in `wg-client` — the agent loses routing without a container restart. + +## RouteDisappearanceWatcher (physical → logical) + +The reverse path handles external physical removal (management server change, `wg syncconf`, operator `peer remove`): + +``` +Physical disappearance Logical reconcile (runtime) +────────────────────── ─────────────────────────── +peer/route gone from NetBird + (peers.conf updated, wg syncconf) + │ + ▼ +RouteDisappearanceWatcher.poll_once() + → peer_exists(binding.peer_id) == false + → runtime.revoke_from_physical(binding, reason) + → catalog REVOKED (no NetBird API call) + → emit physical_trigger: true +``` + +The watcher ensures the runtime catalog stays consistent when reachability disappears outside the runtime — the spec's physical `Revoked` trigger (Phase 3 / idea7). + +## Security (Phase 0+1) + +Mutating APIs require `caller` to match the lease-bound `agent_id` (`CallerIdentityMismatch` on impersonation). Leased tool execution is serialized per `lease_id` in `AgentExecutionLoop` to prevent quota races. Observability events are runtime-authored (`source: runtime`) or agent-loop-bound (`source: agent_loop` with enforced `agent_id`); public `emit_*` APIs are not exposed on `CapabilityRuntime`. + +**Not yet addressed:** cryptographic trace signing, network-authenticated control plane, cross-process trust boundaries. + +## Quick start + +```bash +cd capability-runtime +pytest -q --ignore=tests/test_security.py --ignore=tests/test_policy.py +PYTHONPATH=src:. python poc/create_pr_demo.py +PYTHONPATH=src:. python poc/mcp_tool_demo.py +PYTHONPATH=src:. python poc/network_route_demo.py +``` + +## Layout + +| Module | Role | +|--------|------| +| `runtime.py` | Protocol surfaces, bundle assembly, NetBird backend wiring | +| `catalog.py` | Lifecycle states, network binding storage | +| `network.py` | `NetworkBinding`, `PhysicalRevocationBackend` protocol | +| `netbird_client.py` | `NetBirdClient` protocol, mock and REST implementations | +| `netbird_backend.py` | `NetBirdRevocationBackend` — logical revoke → peer/route removal | +| `route_watcher.py` | `RouteDisappearanceWatcher` — physical disappearance → logical revoke | +| `lease.py` / `revoke.py` | Grant, quota, revocation | +| `policy.py` | PoC lease parameters (not in core runtime) | +| `observability.py` | JSONL trace + replay | +| `agent_loop.py` | Check-then-act harness | +| `mcp_adapter.py` | Transport-agnostic MCP tool wrapper | From b5900520280e0185c67178628c46243d9dfd6a7d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 23 Jun 2026 07:22:26 +0000 Subject: [PATCH 035/138] feat(capability-runtime): extract lease policy and add security boundary tests MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Move PoC lease parameters for create_pr and write_note into policy.py with explicit LeasePolicyNotFound for unknown capabilities. Add security tests documenting spec §4 mitigations: caller identity binding on lease/record/revoke, runtime-authored observability only, and per-lease quota serialization under concurrent agent execution. --- .../src/capability_runtime/policy.py | 43 +++++++++ capability-runtime/tests/test_policy.py | 28 ++++++ capability-runtime/tests/test_security.py | 90 +++++++++++++++++++ 3 files changed, 161 insertions(+) create mode 100644 capability-runtime/src/capability_runtime/policy.py create mode 100644 capability-runtime/tests/test_policy.py create mode 100644 capability-runtime/tests/test_security.py diff --git a/capability-runtime/src/capability_runtime/policy.py b/capability-runtime/src/capability_runtime/policy.py new file mode 100644 index 00000000..ae70620f --- /dev/null +++ b/capability-runtime/src/capability_runtime/policy.py @@ -0,0 +1,43 @@ +"""Lease policy decisions for PoC capabilities (extracted from runtime core).""" + +from __future__ import annotations + +from dataclasses import dataclass +from datetime import timedelta + + +class LeasePolicyNotFound(Exception): + """No lease policy registered for the given capability name.""" + + +@dataclass(frozen=True) +class LeasePolicy: + quota: int + ttl: timedelta + token_budget: int + risk_envelope: str + + +_POC_POLICIES: dict[str, LeasePolicy] = { + "create_pr": LeasePolicy( + quota=1, + ttl=timedelta(minutes=10), + token_budget=25_000, + risk_envelope="high", + ), + "write_note": LeasePolicy( + quota=3, + ttl=timedelta(minutes=5), + token_budget=10_000, + risk_envelope="medium", + ), +} + + +def lease_policy_for(capability_name: str | None) -> LeasePolicy: + if capability_name is None: + raise LeasePolicyNotFound("missing capability name") + try: + return _POC_POLICIES[capability_name] + except KeyError as exc: + raise LeasePolicyNotFound(capability_name) from exc diff --git a/capability-runtime/tests/test_policy.py b/capability-runtime/tests/test_policy.py new file mode 100644 index 00000000..c6df6193 --- /dev/null +++ b/capability-runtime/tests/test_policy.py @@ -0,0 +1,28 @@ +"""Tests for PoC lease policy lookup.""" + +from datetime import timedelta + +import pytest + +from capability_runtime.policy import LeasePolicyNotFound, lease_policy_for + + +def test_create_pr_policy(): + policy = lease_policy_for("create_pr") + assert policy.quota == 1 + assert policy.ttl == timedelta(minutes=10) + assert policy.token_budget == 25_000 + assert policy.risk_envelope == "high" + + +def test_write_note_policy(): + policy = lease_policy_for("write_note") + assert policy.quota == 3 + assert policy.ttl == timedelta(minutes=5) + assert policy.token_budget == 10_000 + assert policy.risk_envelope == "medium" + + +def test_unknown_capability_raises(): + with pytest.raises(LeasePolicyNotFound): + lease_policy_for("unknown_tool") diff --git a/capability-runtime/tests/test_security.py b/capability-runtime/tests/test_security.py new file mode 100644 index 00000000..b31a1a81 --- /dev/null +++ b/capability-runtime/tests/test_security.py @@ -0,0 +1,90 @@ +"""Security boundary tests for capability runtime (spec §4).""" + +from __future__ import annotations + +import threading + +import pytest + +from capability_runtime.agent_loop import AgentExecutionLoop +from capability_runtime.errors import CallerIdentityMismatch +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity, CapabilityRef + + +def test_request_lease_rejects_caller_impersonation(tmp_path): + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-sec-1", 300) + victim = AgentIdentity("victim-agent") + attacker = AgentIdentity("attacker-agent") + + with pytest.raises(CallerIdentityMismatch): + runtime.request_capability_lease( + attacker, victim, CapabilityRef("cap-create-pr"), "forged lease" + ) + + +def test_record_action_rejects_wrong_caller(tmp_path): + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-sec-2", 301) + owner = AgentIdentity("owner-agent") + other = AgentIdentity("other-agent") + + decision = runtime.request_capability_lease( + owner, owner, CapabilityRef("cap-create-pr"), "legitimate lease" + ) + + with pytest.raises(CallerIdentityMismatch): + runtime.record_action(other, owner, decision.lease_id, decision.granted_at) + + +def test_revoke_rejects_wrong_caller(tmp_path): + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-sec-3", 302) + owner = AgentIdentity("owner-agent") + other = AgentIdentity("other-agent") + + decision = runtime.request_capability_lease( + owner, owner, CapabilityRef("cap-create-pr"), "legitimate lease" + ) + + with pytest.raises(CallerIdentityMismatch): + runtime.revoke_capability(other, decision.lease_id, "forged revoke") + + +def test_forgeable_emit_api_removed(tmp_path): + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-sec-4", 303) + assert not hasattr(runtime, "emit_execution_event") + assert not hasattr(runtime, "emit_capability_event") + + +def test_concurrent_run_step_respects_quota_one(tmp_path): + """Two threads cannot both consume quota=1 on the same lease.""" + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-sec-5", 304) + loop = AgentExecutionLoop(runtime) + agent = AgentIdentity("agent-concurrent") + + runtime.request_capability_lease( + agent, agent, CapabilityRef("cap-create-pr"), "single-use lease" + ) + + barrier = threading.Barrier(2) + outcomes: list[str] = [] + lock = threading.Lock() + + def worker(label: str) -> None: + barrier.wait() + try: + loop.run_step(agent, {}, "create_pr", lambda: label) + with lock: + outcomes.append(f"ok:{label}") + except Exception as exc: + with lock: + outcomes.append(type(exc).__name__) + + t1 = threading.Thread(target=worker, args=("a",)) + t2 = threading.Thread(target=worker, args=("b",)) + t1.start() + t2.start() + t1.join() + t2.join() + + successes = [o for o in outcomes if o.startswith("ok:")] + assert len(successes) == 1 From e2800343075443c83956a847e5a1d7f1457082fc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 23 Jun 2026 11:43:47 +0000 Subject: [PATCH 036/138] chore(tests): fix failing tests --- cli/lib/volume.bash | 24 ++++++++++++++++++++++-- cli/libexec/netbird/_ | 6 ++++++ cli/test/composefile/netbird.bats | 13 ++++++++----- 3 files changed, 36 insertions(+), 7 deletions(-) create mode 100755 cli/libexec/netbird/_ diff --git a/cli/lib/volume.bash b/cli/lib/volume.bash index 762cb0fa..d901ec43 100644 --- a/cli/lib/volume.bash +++ b/cli/lib/volume.bash @@ -3,6 +3,26 @@ # shellcheck source=logging.bash source "$SCT_LIBDIR/logging.bash" +# Parse a Docker ISO-8601 timestamp to epoch seconds (GNU date or BSD date). +_volume_timestamp_epoch() { + local timestamp=$1 + local normalized epoch + + normalized=$(printf '%s' "$timestamp" | sed -E 's/\.[0-9]+Z$/Z/; s/Z$//') + + if epoch=$(date -d "${normalized} UTC" +%s 2>/dev/null); then + printf '%s' "$epoch" + return 0 + fi + + if epoch=$(date -ju -f '%Y-%m-%dT%H:%M:%S' "$normalized" +%s 2>/dev/null); then + printf '%s' "$epoch" + return 0 + fi + + return 1 +} + # Warns if the agent-home volume is meaningfully older than the agent # image — i.e. the image has been rebuilt since the volume was populated, # so packages installed during the build are hidden by the stale volume @@ -39,8 +59,8 @@ warn_stale_home_volume() { # silently if unavailable (e.g. BSD date on macOS without coreutils) # rather than risk false positives from lexicographic comparison. local vol_epoch img_epoch - vol_epoch=$(date -d "$volume_time" +%s 2>/dev/null) || return 0 - img_epoch=$(date -d "$image_time" +%s 2>/dev/null) || return 0 + vol_epoch=$(_volume_timestamp_epoch "$volume_time") || return 0 + img_epoch=$(_volume_timestamp_epoch "$image_time") || return 0 (( img_epoch - vol_epoch > tolerance_seconds )) || return 0 diff --git a/cli/libexec/netbird/_ b/cli/libexec/netbird/_ new file mode 100755 index 00000000..b7ac7b4b --- /dev/null +++ b/cli/libexec/netbird/_ @@ -0,0 +1,6 @@ +#!/usr/bin/env bash + +# Catch subcommands (status, server, peer, route) and forward them to the netbird +# dispatcher. Without this, `sandcat netbird status` looks for libexec/netbird/status. + +exec netbird "$@" diff --git a/cli/test/composefile/netbird.bats b/cli/test/composefile/netbird.bats index 8351f0a1..cb52bcd6 100644 --- a/cli/test/composefile/netbird.bats +++ b/cli/test/composefile/netbird.bats @@ -1,12 +1,15 @@ #!/usr/bin/env bats setup() { - load test_helper - source "$SCT_LIBDIR/composefile.bash" - # shellcheck source=netbird.bash - source "$SCT_LIBDIR/netbird.bash" + load test_helper + source "$SCT_LIBDIR/composefile.bash" + # shellcheck source=netbird.bash + source "$SCT_LIBDIR/netbird.bash" - COMPOSE_FILE="$BATS_TEST_TMPDIR/compose-proxy.yml" + export HOME="$BATS_TEST_TMPDIR/home" + mkdir -p "$HOME/.config/sandcat" + + COMPOSE_FILE="$BATS_TEST_TMPDIR/compose-proxy.yml" cp "$SCT_TEMPLATEDIR/devcontainer/sandcat/netbird.env" "$BATS_TEST_TMPDIR/netbird.env" cat >"$COMPOSE_FILE" <<'YAML' services: From 33cfab9786561ba1852c3d0ad6c3cfc7816f76d8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 29 Jun 2026 10:11:25 +0000 Subject: [PATCH 037/138] feat(capability-runtime): enforce caller identity and operator-only revoke --- capability-runtime/poc/create_pr_demo.py | 1 + capability-runtime/poc/mcp_tool_demo.py | 2 +- capability-runtime/poc/network_route_demo.py | 3 +- .../src/capability_runtime/agent_loop.py | 34 +++++++++++++---- .../src/capability_runtime/errors.py | 10 +++-- .../src/capability_runtime/runtime.py | 38 ++++++++++++++----- capability-runtime/tests/test_agent_loop.py | 2 +- capability-runtime/tests/test_mcp_adapter.py | 6 +-- capability-runtime/tests/test_runtime.py | 16 +++++--- .../tests/test_runtime_network.py | 10 ++++- 10 files changed, 87 insertions(+), 35 deletions(-) diff --git a/capability-runtime/poc/create_pr_demo.py b/capability-runtime/poc/create_pr_demo.py index 6d3a5437..efcb0fec 100644 --- a/capability-runtime/poc/create_pr_demo.py +++ b/capability-runtime/poc/create_pr_demo.py @@ -56,6 +56,7 @@ def run_poc1_demo(trace_path: Path, *, quiet: bool = False) -> DemoResult: ) decision = runtime.request_capability_lease( + agent, agent, CapabilityRef("cap-create-pr"), "Need to open PR for feature", diff --git a/capability-runtime/poc/mcp_tool_demo.py b/capability-runtime/poc/mcp_tool_demo.py index 2f06c961..8df6bb0d 100644 --- a/capability-runtime/poc/mcp_tool_demo.py +++ b/capability-runtime/poc/mcp_tool_demo.py @@ -40,7 +40,7 @@ def main() -> int: # Step 2: request lease decision = runtime.request_capability_lease( - agent, CapabilityRef("cap-write-note"), "Need to record session notes" + agent, agent, CapabilityRef("cap-write-note"), "Need to record session notes" ) print( f"Step 2 — lease granted: quota={decision.quota}, " diff --git a/capability-runtime/poc/network_route_demo.py b/capability-runtime/poc/network_route_demo.py index 161a4d65..1c02dd17 100644 --- a/capability-runtime/poc/network_route_demo.py +++ b/capability-runtime/poc/network_route_demo.py @@ -70,6 +70,7 @@ def run_poc3_demo(trace_path: Path, *, quiet: bool = False) -> DemoResult: ) decision = runtime.request_capability_lease( + agent, agent, ref, "Need API reachability for integration test", @@ -92,7 +93,7 @@ def run_poc3_demo(trace_path: Path, *, quiet: bool = False) -> DemoResult: f"(reach_api present, peer_id={peer_id!r})", ) - runtime.revoke_capability(ref, "security policy") + runtime.revoke_capability(AgentIdentity("operator"), ref, "security policy") peer_removed = not client.peer_exists("peer-abc") _print_step( quiet, diff --git a/capability-runtime/src/capability_runtime/agent_loop.py b/capability-runtime/src/capability_runtime/agent_loop.py index 572a7cdd..8cd06d2b 100644 --- a/capability-runtime/src/capability_runtime/agent_loop.py +++ b/capability-runtime/src/capability_runtime/agent_loop.py @@ -2,6 +2,7 @@ from __future__ import annotations +import threading from collections.abc import Callable from datetime import datetime, timezone from typing import Any @@ -15,6 +16,14 @@ class AgentExecutionLoop: def __init__(self, runtime: CapabilityRuntime): self._runtime = runtime + self._lease_locks: dict[LeaseId, threading.Lock] = {} + self._lease_locks_guard = threading.Lock() + + def _lock_for_lease(self, lease_id: LeaseId) -> threading.Lock: + with self._lease_locks_guard: + if lease_id not in self._lease_locks: + self._lease_locks[lease_id] = threading.Lock() + return self._lease_locks[lease_id] def run_step( self, @@ -27,15 +36,24 @@ def run_step( effective_now = now or datetime.now(timezone.utc) bundle = self._runtime.check_current_capabilities(agent_id, context) - self._runtime.enforce_action(agent_id, tool_name, bundle.version, effective_now) - - result = action_fn() - lease_id = _lease_id_for_tool(bundle.tools, tool_name) - if lease_id is not None: - self._runtime.record_action(lease_id, effective_now) - - return result + lock = self._lock_for_lease(lease_id) if lease_id is not None else None + + if lock is not None: + lock.acquire() + try: + self._runtime.enforce_action( + agent_id, tool_name, bundle.version, effective_now + ) + result = action_fn() + if lease_id is not None: + self._runtime.record_action( + agent_id, agent_id, lease_id, effective_now + ) + return result + finally: + if lock is not None: + lock.release() def _lease_id_for_tool(tools, tool_name: str) -> LeaseId | None: diff --git a/capability-runtime/src/capability_runtime/errors.py b/capability-runtime/src/capability_runtime/errors.py index 75eedce1..3c431be6 100644 --- a/capability-runtime/src/capability_runtime/errors.py +++ b/capability-runtime/src/capability_runtime/errors.py @@ -1,4 +1,4 @@ -from capability_runtime.types import CapabilityRef, LeaseId +from capability_runtime.types import AgentIdentity, CapabilityRef, LeaseId class CapabilityRuntimeError(Exception): @@ -38,5 +38,9 @@ def __init__(self, expected: int, actual: int): class CallerIdentityMismatch(CapabilityRuntimeError): - def __init__(self, message: str): - super().__init__(message) + def __init__(self, caller: AgentIdentity, subject: AgentIdentity): + self.caller = caller + self.subject = subject + super().__init__( + f"Caller identity mismatch: {caller.value} != {subject.value}" + ) diff --git a/capability-runtime/src/capability_runtime/runtime.py b/capability-runtime/src/capability_runtime/runtime.py index 3b64e528..827ba712 100644 --- a/capability-runtime/src/capability_runtime/runtime.py +++ b/capability-runtime/src/capability_runtime/runtime.py @@ -14,8 +14,10 @@ ) from capability_runtime.errors import ( BundleVersionMismatch, + CallerIdentityMismatch, CapabilityNotVisible, CapabilityUnknown, + LeaseExpired, ) from capability_runtime.lease import LeaseManager from capability_runtime.network import NetworkBinding @@ -35,6 +37,13 @@ ToolCapability, ) +_OPERATOR = AgentIdentity("operator") + + +def _assert_caller(caller: AgentIdentity, subject: AgentIdentity) -> None: + if caller != subject: + raise CallerIdentityMismatch(caller, subject) + class CapabilityRuntime: """Wire together catalog, leases, revocation, observability, discovery.""" @@ -196,11 +205,13 @@ def check_current_capabilities( def request_capability_lease( self, + caller: AgentIdentity, agent_id: AgentIdentity, capability_ref: CapabilityRef, justification: str, ) -> LeaseDecision: """Grant lease with capability-specific params (spec §3.2.2).""" + _assert_caller(caller, agent_id) # Check capability exists state = self.catalog.get_state(capability_ref) if state not in (LifecycleState.DECLARED, LifecycleState.DISCOVERABLE, LifecycleState.VISIBLE): @@ -248,9 +259,13 @@ def request_capability_lease( return decision def revoke_capability( - self, target: Union[LeaseId, CapabilityRef], reason: str + self, + caller: AgentIdentity, + target: Union[LeaseId, CapabilityRef], + reason: str, ) -> None: """Revoke capability by lease ID or ref (spec §3.2.3).""" + _assert_caller(caller, _OPERATOR) physical_revocation = False if isinstance(target, LeaseId): @@ -319,16 +334,19 @@ def discover_capabilities( """Discover capabilities by intent (spec §3.2.4).""" return _discover_capabilities(self.catalog, agent_id, intent) - def emit_execution_event(self, event: dict) -> None: - """Emit execution event to observability (spec §3.2.5).""" - self.observability.emit_execution_event(event) - - def emit_capability_event(self, event: dict) -> None: - """Emit capability event to observability (spec §3.2.5).""" - self.observability.emit_capability_event(event) - - def record_action(self, lease_id: LeaseId, now: datetime) -> None: + def record_action( + self, + caller: AgentIdentity, + agent_id: AgentIdentity, + lease_id: LeaseId, + now: datetime, + ) -> None: """Decrement quota; revoke if exhausted (spec §3.2.6).""" + lease = self.lease_manager.get_lease(lease_id) + if lease is None: + raise LeaseExpired(lease_id) + _assert_caller(caller, lease.agent_id) + _assert_caller(agent_id, lease.agent_id) remaining = self.lease_manager.decrement_quota(lease_id, now) self.observability.emit_capability_event( diff --git a/capability-runtime/tests/test_agent_loop.py b/capability-runtime/tests/test_agent_loop.py index 54dbf924..b9c405f5 100644 --- a/capability-runtime/tests/test_agent_loop.py +++ b/capability-runtime/tests/test_agent_loop.py @@ -45,7 +45,7 @@ def test_run_step_adapts_after_lease_exhausted(tmp_path): agent = AgentIdentity("agent-3") runtime.request_capability_lease( - agent, CapabilityRef("cap-create-pr"), "Need create_pr once" + agent, agent, CapabilityRef("cap-create-pr"), "Need create_pr once" ) assert loop.run_step(agent, {}, "create_pr", lambda: "first") == "first" diff --git a/capability-runtime/tests/test_mcp_adapter.py b/capability-runtime/tests/test_mcp_adapter.py index 1568f6fc..d66aacac 100644 --- a/capability-runtime/tests/test_mcp_adapter.py +++ b/capability-runtime/tests/test_mcp_adapter.py @@ -35,7 +35,7 @@ def test_write_note_invisible_until_leased(tmp_path): assert "write_note" not in [t.name for t in bundle1.tools] decision = runtime.request_capability_lease( - agent, CapabilityRef("cap-write-note"), "Need to record findings" + agent, agent, CapabilityRef("cap-write-note"), "Need to record findings" ) assert decision.quota == 3 assert decision.token_budget == 10_000 @@ -58,7 +58,7 @@ def test_write_note_lifecycle_quota_3(tmp_path): ] runtime.request_capability_lease( - agent, CapabilityRef("cap-write-note"), "Need notes for session" + agent, agent, CapabilityRef("cap-write-note"), "Need notes for session" ) for i in range(3): @@ -77,7 +77,7 @@ def test_mcp_adapter_records_side_effects(tmp_path): agent = AgentIdentity("agent-mcp-3") runtime.request_capability_lease( - agent, CapabilityRef("cap-write-note"), "Side effect test" + agent, agent, CapabilityRef("cap-write-note"), "Side effect test" ) adapter.invoke(agent, {}, "write_note", "hello world", loop) diff --git a/capability-runtime/tests/test_runtime.py b/capability-runtime/tests/test_runtime.py index 854b2fd1..d0cbe56c 100644 --- a/capability-runtime/tests/test_runtime.py +++ b/capability-runtime/tests/test_runtime.py @@ -10,6 +10,8 @@ from capability_runtime.types import AgentIdentity, CapabilityRef from capability_runtime.discover import DiscoveryIntent +_OPERATOR = AgentIdentity("operator") + def test_poc1_create_pr_lifecycle(tmp_path): """Full PoC 1: create_pr invisible → lease → visible → use → gone""" @@ -24,7 +26,7 @@ def test_poc1_create_pr_lifecycle(tmp_path): # Step 2: request lease decision = runtime.request_capability_lease( - agent, CapabilityRef("cap-create-pr"), "Need to open PR for feature" + agent, agent, CapabilityRef("cap-create-pr"), "Need to open PR for feature" ) assert decision.quota == 1 @@ -36,7 +38,7 @@ def test_poc1_create_pr_lifecycle(tmp_path): # Step 4: use (record action) now = datetime.now(timezone.utc) - runtime.record_action(create_pr_tools[0].lease_id, now) + runtime.record_action(agent, agent, create_pr_tools[0].lease_id, now) # Step 5: gone after quota exhausted bundle3 = runtime.check_current_capabilities(agent, ctx) @@ -50,7 +52,7 @@ def test_revoke_by_lease_id(tmp_path): # Request lease decision = runtime.request_capability_lease( - agent, CapabilityRef("cap-create-pr"), "Testing revoke" + agent, agent, CapabilityRef("cap-create-pr"), "Testing revoke" ) # Verify present @@ -58,7 +60,7 @@ def test_revoke_by_lease_id(tmp_path): assert "create_pr" in [t.name for t in bundle1.tools] # Revoke by lease ID - runtime.revoke_capability(decision.lease_id, "policy violation") + runtime.revoke_capability(_OPERATOR, decision.lease_id, "policy violation") # Verify gone bundle2 = runtime.check_current_capabilities(agent, {}) @@ -72,7 +74,7 @@ def test_revoke_by_capability_ref(tmp_path): # Request lease runtime.request_capability_lease( - agent, CapabilityRef("cap-create-pr"), "Testing revoke" + agent, agent, CapabilityRef("cap-create-pr"), "Testing revoke" ) # Verify present @@ -80,7 +82,9 @@ def test_revoke_by_capability_ref(tmp_path): assert "create_pr" in [t.name for t in bundle1.tools] # Revoke by ref - runtime.revoke_capability(CapabilityRef("cap-create-pr"), "security concern") + runtime.revoke_capability( + _OPERATOR, CapabilityRef("cap-create-pr"), "security concern" + ) # Verify gone bundle2 = runtime.check_current_capabilities(agent, {}) diff --git a/capability-runtime/tests/test_runtime_network.py b/capability-runtime/tests/test_runtime_network.py index 8abff527..00030671 100644 --- a/capability-runtime/tests/test_runtime_network.py +++ b/capability-runtime/tests/test_runtime_network.py @@ -5,6 +5,8 @@ def test_register_network_capability_appears_in_bundle_when_visible(tmp_path): from capability_runtime.runtime import CapabilityRuntime from capability_runtime.types import AgentIdentity, CapabilityRef + operator = AgentIdentity("operator") + client = MockNetBirdClient() runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-n1", 1, netbird_client=client) agent = AgentIdentity("agent-1") @@ -24,6 +26,8 @@ def test_revoke_network_capability_calls_netbird_backend(tmp_path): from capability_runtime.runtime import CapabilityRuntime from capability_runtime.types import AgentIdentity, CapabilityRef + operator = AgentIdentity("operator") + client = MockNetBirdClient( peers=[{"id": "peer-abc", "connected": True}], routes=[{"id": "route-1", "network": "10.8.0.0/24"}], @@ -33,7 +37,7 @@ def test_revoke_network_capability_calls_netbird_backend(tmp_path): ref = CapabilityRef("cap-reach-api") binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", "route-1") runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) - runtime.revoke_capability(ref, "security") + runtime.revoke_capability(operator, ref, "security") assert not client.peer_exists("peer-abc") bundle = runtime.check_current_capabilities(agent, {}) assert "reach_api" not in [n.name for n in bundle.networks] @@ -45,6 +49,8 @@ def test_revoke_non_network_capability_does_not_call_netbird(tmp_path): from capability_runtime.runtime import CapabilityRuntime from capability_runtime.types import AgentIdentity, CapabilityRef + operator = AgentIdentity("operator") + client = MockNetBirdClient( peers=[{"id": "peer-xyz", "connected": True}], ) @@ -52,7 +58,7 @@ def test_revoke_non_network_capability_does_not_call_netbird(tmp_path): agent = AgentIdentity("agent-1") ref = CapabilityRef("cap-create-pr") runtime.catalog.register("create_pr_tool", ref, LifecycleState.VISIBLE) - runtime.revoke_capability(ref, "security") + runtime.revoke_capability(operator, ref, "security") # Peer should still exist because we didn't revoke a network capability assert client.peer_exists("peer-xyz") bundle = runtime.check_current_capabilities(agent, {}) From 544fb903da1e4a176746bc6c0798fec3ac08e31b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 29 Jun 2026 10:12:14 +0000 Subject: [PATCH 038/138] feat(capability-runtime): load NetBird credentials from sandcat settings layers --- .../src/capability_runtime/netbird_client.py | 7 +++ .../src/capability_runtime/settings.py | 45 +++++++++++++++++++ capability-runtime/tests/test_settings.py | 24 ++++++++++ 3 files changed, 76 insertions(+) create mode 100644 capability-runtime/src/capability_runtime/settings.py create mode 100644 capability-runtime/tests/test_settings.py diff --git a/capability-runtime/src/capability_runtime/netbird_client.py b/capability-runtime/src/capability_runtime/netbird_client.py index 76e88ac5..3d366ffa 100644 --- a/capability-runtime/src/capability_runtime/netbird_client.py +++ b/capability-runtime/src/capability_runtime/netbird_client.py @@ -63,6 +63,13 @@ def __init__( else os.environ.get("NB_MANAGEMENT_URL", "https://api.netbird.io") ) + @classmethod + def from_settings(cls) -> RestNetBirdClient: + from capability_runtime.settings import load_netbird_credentials + + creds = load_netbird_credentials() + return cls(api_token=creds.api_token, management_url=creds.management_url) + def list_peers(self) -> list[dict]: return self._request("GET", "/api/peers") diff --git a/capability-runtime/src/capability_runtime/settings.py b/capability-runtime/src/capability_runtime/settings.py new file mode 100644 index 00000000..31f22a74 --- /dev/null +++ b/capability-runtime/src/capability_runtime/settings.py @@ -0,0 +1,45 @@ +from __future__ import annotations + +import json +import os +from dataclasses import dataclass +from pathlib import Path + + +class MissingNetBirdCredentials(Exception): + pass + + +@dataclass(frozen=True) +class NetBirdCredentials: + api_token: str + management_url: str | None + + +def _read_setting(key: str) -> str | None: + value: str | None = None + for env_var in ("SANDCAT_SETTINGS_USER", "SANDCAT_SETTINGS_PROJECT"): + path_str = os.environ.get(env_var) + if not path_str: + continue + path = Path(path_str) + if not path.is_file(): + continue + with path.open(encoding="utf-8") as handle: + data = json.load(handle) + layer_value = data.get(key) + if layer_value: + value = str(layer_value) + return value + + +def load_netbird_credentials() -> NetBirdCredentials: + token = os.environ.get("NB_API_TOKEN") or _read_setting("netbird_api_token") + url = ( + os.environ.get("NB_MANAGEMENT_URL") + or _read_setting("netbird_management_url") + or None + ) + if not token: + raise MissingNetBirdCredentials("netbird_api_token is not set") + return NetBirdCredentials(api_token=token, management_url=url) diff --git a/capability-runtime/tests/test_settings.py b/capability-runtime/tests/test_settings.py new file mode 100644 index 00000000..3a2b58ec --- /dev/null +++ b/capability-runtime/tests/test_settings.py @@ -0,0 +1,24 @@ +import json + +from capability_runtime.settings import load_netbird_credentials + + +def test_load_credentials_project_over_user(tmp_path, monkeypatch): + user = tmp_path / "user" / "settings.json" + project = tmp_path / "project" / ".sandcat" / "settings.json" + user.parent.mkdir(parents=True) + project.parent.mkdir(parents=True) + user.write_text(json.dumps({"netbird_api_token": "user-tok"})) + project.write_text( + json.dumps( + { + "netbird_api_token": "proj-tok", + "netbird_management_url": "https://mgmt.example.com", + } + ) + ) + monkeypatch.setenv("SANDCAT_SETTINGS_USER", str(user)) + monkeypatch.setenv("SANDCAT_SETTINGS_PROJECT", str(project)) + creds = load_netbird_credentials() + assert creds.api_token == "proj-tok" + assert creds.management_url == "https://mgmt.example.com" From b7d87df5c8456d7eea064e631f84199c46407704 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 29 Jun 2026 10:13:49 +0000 Subject: [PATCH 039/138] feat(capability-runtime): add JSON-RPC dispatcher with agent and admin surfaces --- .../src/capability_runtime/rpc/__init__.py | 7 + .../src/capability_runtime/rpc/dispatcher.py | 234 ++++++++++++++++++ .../src/capability_runtime/rpc/errors.py | 9 + .../tests/test_rpc_dispatcher.py | 151 +++++++++++ 4 files changed, 401 insertions(+) create mode 100644 capability-runtime/src/capability_runtime/rpc/__init__.py create mode 100644 capability-runtime/src/capability_runtime/rpc/dispatcher.py create mode 100644 capability-runtime/src/capability_runtime/rpc/errors.py create mode 100644 capability-runtime/tests/test_rpc_dispatcher.py diff --git a/capability-runtime/src/capability_runtime/rpc/__init__.py b/capability-runtime/src/capability_runtime/rpc/__init__.py new file mode 100644 index 00000000..7a765e17 --- /dev/null +++ b/capability-runtime/src/capability_runtime/rpc/__init__.py @@ -0,0 +1,7 @@ +"""JSON-RPC dispatcher for capability runtime.""" + +from __future__ import annotations + +from capability_runtime.rpc.dispatcher import RpcDispatcher + +__all__ = ["RpcDispatcher"] diff --git a/capability-runtime/src/capability_runtime/rpc/dispatcher.py b/capability-runtime/src/capability_runtime/rpc/dispatcher.py new file mode 100644 index 00000000..7f3f0f5f --- /dev/null +++ b/capability-runtime/src/capability_runtime/rpc/dispatcher.py @@ -0,0 +1,234 @@ +"""JSON-RPC 2.0 dispatcher with agent and admin surface allowlists.""" + +from __future__ import annotations + +from datetime import datetime, timedelta +from typing import Any, Literal + +from capability_runtime.discover import DiscoveryIntent +from capability_runtime.errors import CapabilityRuntimeError +from capability_runtime.rpc import errors as rpc_errors +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.route_watcher import RouteDisappearanceWatcher +from capability_runtime.types import ( + AgentIdentity, + CapabilityBundle, + CapabilityRef, + LeaseDecision, + LeaseId, +) + +AGENT_METHODS = frozenset( + { + "capability.check", + "capability.lease", + "capability.discover", + } +) + +ADMIN_METHODS = AGENT_METHODS | frozenset( + { + "capability.revoke", + "capability.watch.poll", + } +) + +_OPERATOR = AgentIdentity("operator") + + +class RpcDispatcher: + """Route JSON-RPC requests to CapabilityRuntime with surface allowlists.""" + + def __init__( + self, + runtime: CapabilityRuntime, + *, + surface: Literal["agent", "admin"], + bound_agent_id: str, + watcher: RouteDisappearanceWatcher | None = None, + ) -> None: + self._runtime = runtime + self._surface = surface + self._bound_agent_id = bound_agent_id + self._watcher = watcher + self._allowed = ADMIN_METHODS if surface == "admin" else AGENT_METHODS + + def handle(self, request: dict) -> dict: + """Handle a JSON-RPC 2.0 request and return a response dict.""" + request_id = request.get("id") + if request.get("jsonrpc") != "2.0": + return self._error_response(request_id, rpc_errors.INVALID_REQUEST, "Invalid Request") + method = request.get("method") + if not isinstance(method, str): + return self._error_response(request_id, rpc_errors.INVALID_REQUEST, "Invalid Request") + if method not in self._allowed: + return self._error_response( + request_id, rpc_errors.METHOD_NOT_FOUND, "Method not found" + ) + + params = request.get("params") or {} + if not isinstance(params, dict): + return self._error_response(request_id, rpc_errors.INVALID_PARAMS, "Invalid params") + + try: + result = self._dispatch(method, params) + except CapabilityRuntimeError as exc: + return self._error_response(request_id, rpc_errors.INTERNAL_ERROR, str(exc)) + except (KeyError, TypeError, ValueError) as exc: + return self._error_response(request_id, rpc_errors.INVALID_PARAMS, str(exc)) + + return {"jsonrpc": "2.0", "result": result, "id": request_id} + + def _error_response( + self, request_id: Any, code: int, message: str + ) -> dict[str, Any]: + return { + "jsonrpc": "2.0", + "error": {"code": code, "message": message}, + "id": request_id, + } + + def _dispatch(self, method: str, params: dict) -> dict: + if method == "capability.check": + return self._handle_check(params) + if method == "capability.lease": + return self._handle_lease(params) + if method == "capability.discover": + return self._handle_discover(params) + if method == "capability.revoke": + return self._handle_revoke(params) + if method == "capability.watch.poll": + return self._handle_watch_poll(params) + raise RuntimeError(f"unhandled allowed method: {method}") + + def _resolve_agent_id(self, params: dict) -> AgentIdentity: + if self._surface == "agent": + return AgentIdentity(self._bound_agent_id) + agent_value = params.get("agent_id", self._bound_agent_id) + return AgentIdentity(agent_value) + + def _handle_check(self, params: dict) -> dict: + agent_id = self._resolve_agent_id(params) + context = params.get("context", {}) + if not isinstance(context, dict): + raise ValueError("context must be an object") + bundle = self._runtime.check_current_capabilities(agent_id, context) + return _serialize_bundle(bundle) + + def _handle_lease(self, params: dict) -> dict: + agent_id = self._resolve_agent_id(params) + capability_ref = CapabilityRef(params["capability_ref"]) + justification = params["justification"] + decision = self._runtime.request_capability_lease( + caller=agent_id, + agent_id=agent_id, + capability_ref=capability_ref, + justification=justification, + ) + return _serialize_lease_decision(decision) + + def _handle_discover(self, params: dict) -> dict: + agent_id = self._resolve_agent_id(params) + intent = DiscoveryIntent(query=params["query"]) + result = self._runtime.discover_capabilities(agent_id, intent) + return {"capabilities": result.capabilities, "denied": result.denied} + + def _handle_revoke(self, params: dict) -> dict: + target = _parse_revoke_target(self._runtime, params["target"]) + reason = params["reason"] + self._runtime.revoke_capability(_OPERATOR, target, reason) + return {"revoked": True} + + def _handle_watch_poll(self, params: dict) -> dict: + if self._watcher is None: + raise ValueError("route watcher not configured") + self._watcher.poll_once() + return {"polled": True} + + +def _parse_revoke_target(runtime: CapabilityRuntime, target: str) -> LeaseId | CapabilityRef: + lease_id = LeaseId(target) + if runtime.lease_manager.get_lease(lease_id) is not None: + return lease_id + return CapabilityRef(target) + + +def _serialize_bundle(bundle: CapabilityBundle) -> dict[str, Any]: + return { + "agent_id": bundle.agent_id.value, + "issued_at": _serialize_datetime(bundle.issued_at), + "expires_at": _serialize_datetime(bundle.expires_at), + "tools": [_serialize_tool_capability(t) for t in bundle.tools], + "networks": [_serialize_network_capability(n) for n in bundle.networks], + "rules": [_serialize_named_capability(r) for r in bundle.rules], + "skills": [_serialize_named_capability(s) for s in bundle.skills], + "policies": [_serialize_named_capability(p) for p in bundle.policies], + "hooks": [_serialize_named_capability(h) for h in bundle.hooks], + "budgets": { + "token_quota": bundle.budgets.token_quota, + "action_quota": bundle.budgets.action_quota, + "wall_time_ttl": _serialize_timedelta(bundle.budgets.wall_time_ttl), + }, + "provenance": { + "issuer": bundle.provenance.issuer, + "policy_version": bundle.provenance.policy_version, + "trace_id": bundle.provenance.trace_id, + }, + "version": bundle.version, + } + + +def _serialize_tool_capability(tool) -> dict[str, Any]: + return { + "ref": tool.ref.value, + "name": tool.name, + "lease_id": tool.lease_id.value if tool.lease_id else None, + "quota": tool.quota, + "expires_at": _serialize_datetime(tool.expires_at), + } + + +def _serialize_network_capability(network) -> dict[str, Any]: + return { + "ref": network.ref.value, + "name": network.name, + "peer_id": network.peer_id, + "network": network.network, + "route_id": network.route_id, + "lease_id": network.lease_id.value if network.lease_id else None, + "quota": network.quota, + "expires_at": _serialize_datetime(network.expires_at), + } + + +def _serialize_named_capability(cap) -> dict[str, Any]: + return { + "ref": cap.ref.value, + "name": cap.name, + "lease_id": cap.lease_id.value if cap.lease_id else None, + } + + +def _serialize_lease_decision(decision: LeaseDecision) -> dict[str, Any]: + return { + "lease_id": decision.lease_id.value, + "capability_ref": decision.capability_ref.value, + "agent_id": decision.agent_id.value, + "quota": decision.quota, + "token_budget": decision.token_budget, + "risk_envelope": decision.risk_envelope, + "expires_at": _serialize_datetime(decision.expires_at), + "granted_at": _serialize_datetime(decision.granted_at), + } + + +def _serialize_datetime(value: datetime | None) -> str | None: + if value is None: + return None + return value.isoformat() + + +def _serialize_timedelta(value: timedelta | None) -> float | None: + if value is None: + return None + return value.total_seconds() diff --git a/capability-runtime/src/capability_runtime/rpc/errors.py b/capability-runtime/src/capability_runtime/rpc/errors.py new file mode 100644 index 00000000..6729efed --- /dev/null +++ b/capability-runtime/src/capability_runtime/rpc/errors.py @@ -0,0 +1,9 @@ +"""JSON-RPC error codes for capability runtime RPC.""" + +from __future__ import annotations + +PARSE_ERROR = -32700 +INVALID_REQUEST = -32600 +METHOD_NOT_FOUND = -32601 +INVALID_PARAMS = -32602 +INTERNAL_ERROR = -32603 diff --git a/capability-runtime/tests/test_rpc_dispatcher.py b/capability-runtime/tests/test_rpc_dispatcher.py new file mode 100644 index 00000000..a8c17524 --- /dev/null +++ b/capability-runtime/tests/test_rpc_dispatcher.py @@ -0,0 +1,151 @@ +"""Tests for JSON-RPC dispatcher with agent/admin surface allowlists.""" + +from __future__ import annotations + +from unittest.mock import MagicMock, patch + +import pytest + +from capability_runtime.catalog import LifecycleState +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.rpc.dispatcher import RpcDispatcher +from capability_runtime.types import AgentIdentity, CapabilityRef + + +@pytest.fixture +def runtime(tmp_path): + return CapabilityRuntime(tmp_path / "trace.jsonl", "trace-rpc", 400) + + +def test_agent_surface_rejects_revoke(runtime): + dispatcher = RpcDispatcher( + runtime, surface="agent", bound_agent_id="devcontainer-agent" + ) + response = dispatcher.handle( + { + "jsonrpc": "2.0", + "id": 1, + "method": "capability.revoke", + "params": {"target": "cap-reach-api", "reason": "x"}, + } + ) + assert "error" in response + assert response["error"]["code"] == -32601 + + +def test_agent_check_injects_agent_id(runtime): + dispatcher = RpcDispatcher( + runtime, surface="agent", bound_agent_id="devcontainer-agent" + ) + with patch.object( + runtime, "check_current_capabilities", wraps=runtime.check_current_capabilities + ) as spy: + response = dispatcher.handle( + { + "jsonrpc": "2.0", + "id": 1, + "method": "capability.check", + "params": {"agent_id": "attacker", "context": {}}, + } + ) + assert "result" in response + spy.assert_called_once() + call_agent_id = spy.call_args[0][0] + assert call_agent_id == AgentIdentity("devcontainer-agent") + + +def test_admin_surface_allows_revoke(runtime): + agent = AgentIdentity("devcontainer-agent") + runtime.request_capability_lease( + agent, agent, CapabilityRef("cap-create-pr"), "setup lease" + ) + + bundle_before = runtime.check_current_capabilities(agent, {}) + assert "create_pr" in [t.name for t in bundle_before.tools] + + dispatcher = RpcDispatcher(runtime, surface="admin", bound_agent_id="operator") + response = dispatcher.handle( + { + "jsonrpc": "2.0", + "id": 2, + "method": "capability.revoke", + "params": {"target": "cap-create-pr", "reason": "policy"}, + } + ) + assert "result" in response + assert "error" not in response + + bundle_after = runtime.check_current_capabilities(agent, {}) + assert "create_pr" not in [t.name for t in bundle_after.tools] + + +def test_agent_surface_allows_check_lease_discover(runtime): + dispatcher = RpcDispatcher( + runtime, surface="agent", bound_agent_id="devcontainer-agent" + ) + + check = dispatcher.handle( + { + "jsonrpc": "2.0", + "id": 1, + "method": "capability.check", + "params": {"context": {}}, + } + ) + assert "result" in check + assert "tools" in check["result"] + + runtime.catalog.register( + "find_me", CapabilityRef("cap-find-me"), initial_state=LifecycleState.DISCOVERABLE + ) + discover = dispatcher.handle( + { + "jsonrpc": "2.0", + "id": 2, + "method": "capability.discover", + "params": {"query": "find"}, + } + ) + assert "result" in discover + + lease = dispatcher.handle( + { + "jsonrpc": "2.0", + "id": 3, + "method": "capability.lease", + "params": { + "capability_ref": "cap-create-pr", + "justification": "need it", + }, + } + ) + assert "result" in lease + assert "lease_id" in lease["result"] + + +def test_admin_surface_allows_watch_poll(runtime): + watcher = MagicMock() + dispatcher = RpcDispatcher( + runtime, + surface="admin", + bound_agent_id="operator", + watcher=watcher, + ) + response = dispatcher.handle( + { + "jsonrpc": "2.0", + "id": 1, + "method": "capability.watch.poll", + "params": {}, + } + ) + assert "result" in response + watcher.poll_once.assert_called_once() + + +def test_invalid_request_missing_method(runtime): + dispatcher = RpcDispatcher( + runtime, surface="agent", bound_agent_id="devcontainer-agent" + ) + response = dispatcher.handle({"jsonrpc": "2.0", "id": 1}) + assert response["error"]["code"] == -32600 From b4e48ae5a622303e712b1fb4b68e46cb46e7f78c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 29 Jun 2026 10:15:55 +0000 Subject: [PATCH 040/138] feat(capability-runtime): add sidecar daemon with dual Unix socket RPC --- .../src/capability_runtime/daemon.py | 209 ++++++++++++++++++ .../rpc/transports/__init__.py | 5 + .../capability_runtime/rpc/transports/unix.py | 143 ++++++++++++ .../tests/test_daemon_integration.py | 164 ++++++++++++++ capability-runtime/tests/test_rpc_unix.py | 83 +++++++ 5 files changed, 604 insertions(+) create mode 100644 capability-runtime/src/capability_runtime/daemon.py create mode 100644 capability-runtime/src/capability_runtime/rpc/transports/__init__.py create mode 100644 capability-runtime/src/capability_runtime/rpc/transports/unix.py create mode 100644 capability-runtime/tests/test_daemon_integration.py create mode 100644 capability-runtime/tests/test_rpc_unix.py diff --git a/capability-runtime/src/capability_runtime/daemon.py b/capability-runtime/src/capability_runtime/daemon.py new file mode 100644 index 00000000..488ab5fd --- /dev/null +++ b/capability-runtime/src/capability_runtime/daemon.py @@ -0,0 +1,209 @@ +"""Sidecar daemon: CapabilityRuntime, route watcher, dual Unix socket RPC.""" + +from __future__ import annotations + +import json +import os +import signal +import threading +import time +from dataclasses import dataclass +from pathlib import Path + +from capability_runtime.catalog import LifecycleState +from capability_runtime.netbird_client import MockNetBirdClient, NetBirdClient, RestNetBirdClient +from capability_runtime.network import NetworkBinding +from capability_runtime.rpc.dispatcher import RpcDispatcher +from capability_runtime.rpc.transports.unix import UnixRpcServer +from capability_runtime.route_watcher import RouteDisappearanceWatcher +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import CapabilityRef + + +@dataclass(frozen=True) +class DaemonConfig: + catalog_path: Path + agent_socket: Path + admin_socket: Path + trace_file: Path + agent_id: str = "devcontainer-agent" + trace_id: str = "capability-sidecar" + seed: int = 42 + watch_interval: float = 5.0 + mock_netbird: bool = False + + @classmethod + def from_env(cls) -> DaemonConfig: + catalog = os.environ.get("CAPABILITY_CATALOG_JSON") + if not catalog: + raise RuntimeError("CAPABILITY_CATALOG_JSON is required") + return cls( + catalog_path=Path(catalog), + agent_socket=Path( + os.environ.get( + "CAPABILITY_AGENT_SOCKET", + "/run/sandcat/capability/agent.sock", + ) + ), + admin_socket=Path( + os.environ.get( + "CAPABILITY_ADMIN_SOCKET", + "/run/sandcat/capability/admin.sock", + ) + ), + trace_file=Path( + os.environ.get( + "CAPABILITY_TRACE_FILE", + "/var/lib/sandcat/capability/trace.jsonl", + ) + ), + agent_id=os.environ.get("SANDCAT_AGENT_ID", "devcontainer-agent"), + trace_id=os.environ.get("CAPABILITY_TRACE_ID", "capability-sidecar"), + watch_interval=float(os.environ.get("CAPABILITY_WATCH_INTERVAL", "5")), + mock_netbird=os.environ.get("CAPABILITY_MOCK_NETBIRD") == "1", + ) + + +def build_netbird_client(config: DaemonConfig) -> NetBirdClient: + if config.mock_netbird: + return MockNetBirdClient() + return RestNetBirdClient.from_settings() + + +def build_runtime(config: DaemonConfig, netbird_client: NetBirdClient) -> CapabilityRuntime: + config.trace_file.parent.mkdir(parents=True, exist_ok=True) + return CapabilityRuntime( + config.trace_file, + config.trace_id, + config.seed, + netbird_client=netbird_client, + ) + + +def load_catalog_into_runtime(runtime: CapabilityRuntime, catalog_path: Path) -> None: + """Register tool and network capabilities from a JSON catalog file.""" + data = json.loads(catalog_path.read_text()) + for entry in data.get("capabilities", []): + name = entry["name"] + ref = CapabilityRef(entry["ref"]) + cap_type = entry.get("type", "tool") + if cap_type == "network": + binding = NetworkBinding( + ref, + entry["peer_id"], + entry["network"], + entry.get("route_id"), + ) + runtime.register_network_capability( + name, + ref, + binding, + LifecycleState.DECLARED, + ) + else: + if runtime.catalog.get_by_name(name) is None: + runtime.catalog.register(name, ref, LifecycleState.DECLARED) + + +class _WatcherPollThread: + def __init__(self, watcher: RouteDisappearanceWatcher, interval: float) -> None: + self._watcher = watcher + self._interval = interval + self._stop = threading.Event() + self._thread: threading.Thread | None = None + + def start(self) -> None: + if self._thread is not None and self._thread.is_alive(): + return + self._stop.clear() + self._thread = threading.Thread( + target=self._run, + name="route-watcher", + daemon=True, + ) + self._thread.start() + + def stop(self) -> None: + self._stop.set() + if self._thread is not None: + self._thread.join(timeout=2.0) + self._thread = None + + def _run(self) -> None: + while not self._stop.is_set(): + self._watcher.poll_once() + self._stop.wait(self._interval) + + +class CapabilityDaemon: + """Run runtime, route watcher, and agent/admin RPC sockets.""" + + def __init__(self, config: DaemonConfig) -> None: + self._config = config + self._netbird_client = build_netbird_client(config) + self._runtime = build_runtime(config, self._netbird_client) + load_catalog_into_runtime(self._runtime, config.catalog_path) + self._watcher = RouteDisappearanceWatcher(self._runtime, self._netbird_client) + self._watcher_thread = _WatcherPollThread(self._watcher, config.watch_interval) + self._agent_server = UnixRpcServer( + config.agent_socket, + RpcDispatcher( + self._runtime, + surface="agent", + bound_agent_id=config.agent_id, + ), + ) + self._admin_server = UnixRpcServer( + config.admin_socket, + RpcDispatcher( + self._runtime, + surface="admin", + bound_agent_id="operator", + watcher=self._watcher, + ), + ) + self._running = False + + @property + def runtime(self) -> CapabilityRuntime: + return self._runtime + + def start(self) -> None: + if self._running: + return + self._watcher_thread.start() + self._agent_server.start() + self._admin_server.start() + self._running = True + + def stop(self) -> None: + if not self._running: + return + self._admin_server.stop() + self._agent_server.stop() + self._watcher_thread.stop() + self._running = False + + def wait(self) -> None: + while self._running: + time.sleep(0.5) + + +def run_daemon(config: DaemonConfig | None = None) -> None: + daemon = CapabilityDaemon(config or DaemonConfig.from_env()) + daemon.start() + + def _shutdown(_signum: int, _frame: object) -> None: + daemon.stop() + + signal.signal(signal.SIGTERM, _shutdown) + signal.signal(signal.SIGINT, _shutdown) + daemon.wait() + + +def main() -> None: + run_daemon() + + +if __name__ == "__main__": + main() diff --git a/capability-runtime/src/capability_runtime/rpc/transports/__init__.py b/capability-runtime/src/capability_runtime/rpc/transports/__init__.py new file mode 100644 index 00000000..6a45637f --- /dev/null +++ b/capability-runtime/src/capability_runtime/rpc/transports/__init__.py @@ -0,0 +1,5 @@ +"""RPC transport implementations.""" + +from capability_runtime.rpc.transports.unix import UnixRpcClient, UnixRpcServer + +__all__ = ["UnixRpcClient", "UnixRpcServer"] diff --git a/capability-runtime/src/capability_runtime/rpc/transports/unix.py b/capability-runtime/src/capability_runtime/rpc/transports/unix.py new file mode 100644 index 00000000..af98e56f --- /dev/null +++ b/capability-runtime/src/capability_runtime/rpc/transports/unix.py @@ -0,0 +1,143 @@ +"""Line-delimited JSON-RPC over AF_UNIX — one request per connection.""" + +from __future__ import annotations + +import json +import socket +import threading +from pathlib import Path +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from capability_runtime.rpc.dispatcher import RpcDispatcher + + +class UnixRpcServer: + """Accept AF_UNIX connections, handle one JSON-RPC request per connection.""" + + def __init__(self, socket_path: Path, dispatcher: RpcDispatcher) -> None: + self._socket_path = socket_path + self._dispatcher = dispatcher + self._stop_event = threading.Event() + self._thread: threading.Thread | None = None + self._server: socket.socket | None = None + + def start(self) -> None: + if self._thread is not None and self._thread.is_alive(): + return + self._stop_event.clear() + self._thread = threading.Thread(target=self._serve, name="unix-rpc-server", daemon=True) + self._thread.start() + + def stop(self) -> None: + self._stop_event.set() + server = self._server + if server is not None: + try: + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as probe: + probe.settimeout(0.5) + probe.connect(str(self._socket_path)) + except OSError: + pass + try: + server.close() + except OSError: + pass + if self._thread is not None: + self._thread.join(timeout=2.0) + self._thread = None + if self._socket_path.exists(): + self._socket_path.unlink(missing_ok=True) + + def _serve(self) -> None: + self._socket_path.parent.mkdir(parents=True, exist_ok=True) + if self._socket_path.exists(): + self._socket_path.unlink() + + server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + self._server = server + server.bind(str(self._socket_path)) + server.listen(5) + server.settimeout(1.0) + + try: + while not self._stop_event.is_set(): + try: + conn, _ = server.accept() + except socket.timeout: + continue + except OSError: + if self._stop_event.is_set(): + break + raise + handler = threading.Thread( + target=self._handle_connection, + args=(conn,), + daemon=True, + ) + handler.start() + finally: + server.close() + self._server = None + if self._socket_path.exists(): + self._socket_path.unlink(missing_ok=True) + + def _handle_connection(self, conn: socket.socket) -> None: + with conn: + conn.settimeout(5.0) + line = _read_line(conn) + if line is None: + return + try: + request = json.loads(line) + except json.JSONDecodeError: + response = { + "jsonrpc": "2.0", + "error": {"code": -32700, "message": "Parse error"}, + "id": None, + } + else: + response = self._dispatcher.handle(request) + conn.sendall((json.dumps(response) + "\n").encode()) + + +class UnixRpcClient: + """Send a single JSON-RPC request over AF_UNIX and return the response.""" + + def __init__(self, socket_path: Path, *, timeout: float = 5.0) -> None: + self._socket_path = socket_path + self._timeout = timeout + + def call( + self, + method: str, + params: dict | None = None, + *, + request_id: int | str = 1, + ) -> dict: + request = { + "jsonrpc": "2.0", + "id": request_id, + "method": method, + "params": params or {}, + } + payload = (json.dumps(request) + "\n").encode() + + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock: + sock.settimeout(self._timeout) + sock.connect(str(self._socket_path)) + sock.sendall(payload) + line = _read_line(sock) + if line is None: + raise ConnectionError("no response from RPC server") + return json.loads(line) + + +def _read_line(conn: socket.socket) -> str | None: + data = b"" + while b"\n" not in data: + chunk = conn.recv(4096) + if not chunk: + return None + data += chunk + return data.split(b"\n", 1)[0].decode() diff --git a/capability-runtime/tests/test_daemon_integration.py b/capability-runtime/tests/test_daemon_integration.py new file mode 100644 index 00000000..0674f997 --- /dev/null +++ b/capability-runtime/tests/test_daemon_integration.py @@ -0,0 +1,164 @@ +"""Integration tests for the capability sidecar daemon.""" + +from __future__ import annotations + +import json +import threading +import time +from pathlib import Path + +import pytest + +from capability_runtime.daemon import CapabilityDaemon, DaemonConfig, load_catalog_into_runtime +from capability_runtime.netbird_client import MockNetBirdClient +from capability_runtime.rpc.transports.unix import UnixRpcClient +from capability_runtime.runtime import CapabilityRuntime + + +@pytest.fixture +def catalog_file(tmp_path): + catalog = { + "capabilities": [ + {"name": "create_pr", "ref": "cap-create-pr", "type": "tool"}, + { + "name": "reach_api", + "ref": "cap-reach-api", + "type": "network", + "peer_id": "peer-test", + "network": "10.8.0.0/24", + "route_id": "route-test", + }, + ] + } + path = tmp_path / "catalog.json" + path.write_text(json.dumps(catalog)) + return path + + +def _wait_for_socket(path: Path, timeout: float = 3.0) -> None: + deadline = time.time() + timeout + while time.time() < deadline: + if path.exists(): + return + time.sleep(0.01) + raise TimeoutError(f"socket not ready: {path}") + + +@pytest.fixture +def daemon(tmp_path, catalog_file): + sock_dir = tmp_path / "sockets" + config = DaemonConfig( + catalog_path=catalog_file, + agent_socket=sock_dir / "agent.sock", + admin_socket=sock_dir / "admin.sock", + trace_file=tmp_path / "trace.jsonl", + agent_id="devcontainer-agent", + watch_interval=0.1, + mock_netbird=True, + ) + daemon = CapabilityDaemon(config) + thread = threading.Thread(target=daemon.start, daemon=True) + thread.start() + _wait_for_socket(config.agent_socket) + _wait_for_socket(config.admin_socket) + yield daemon, config + daemon.stop() + + +def test_load_catalog_registers_tool_and_network(tmp_path, catalog_file): + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-catalog", 1) + load_catalog_into_runtime(runtime, catalog_file) + assert runtime.catalog.get_by_name("create_pr") is not None + assert runtime.catalog.get_by_name("reach_api") is not None + binding = runtime.catalog.get_network_binding(runtime.catalog.get_by_name("reach_api")) + assert binding is not None + assert binding.peer_id == "peer-test" + + +def test_daemon_agent_socket_roundtrip(daemon): + _, config = daemon + client = UnixRpcClient(config.agent_socket) + response = client.call("capability.check", {"context": {}}) + assert "result" in response + assert "tools" in response["result"] + assert response["result"]["agent_id"] == "devcontainer-agent" + + +def test_daemon_agent_lease_via_socket(daemon): + _, config = daemon + client = UnixRpcClient(config.agent_socket) + lease = client.call( + "capability.lease", + {"capability_ref": "cap-create-pr", "justification": "integration test"}, + ) + assert "result" in lease + assert "lease_id" in lease["result"] + + check = client.call("capability.check", {"context": {}}) + tool_names = [tool["name"] for tool in check["result"]["tools"]] + assert "create_pr" in tool_names + + +def test_daemon_admin_revoke_via_socket(daemon): + _, config = daemon + agent = UnixRpcClient(config.agent_socket) + admin = UnixRpcClient(config.admin_socket) + + agent.call( + "capability.lease", + {"capability_ref": "cap-create-pr", "justification": "to revoke"}, + ) + revoke = admin.call( + "capability.revoke", + {"target": "cap-create-pr", "reason": "policy"}, + ) + assert revoke["result"]["revoked"] is True + + check = agent.call("capability.check", {"context": {}}) + tool_names = [tool["name"] for tool in check["result"]["tools"]] + assert "create_pr" not in tool_names + + +def test_daemon_agent_rejects_revoke(daemon): + _, config = daemon + client = UnixRpcClient(config.agent_socket) + response = client.call( + "capability.revoke", + {"target": "cap-create-pr", "reason": "forged"}, + ) + assert "error" in response + assert response["error"]["code"] == -32601 + + +def test_daemon_admin_watch_poll(daemon): + _, config = daemon + admin = UnixRpcClient(config.admin_socket) + response = admin.call("capability.watch.poll", {}) + assert response["result"]["polled"] is True + + +def test_daemon_watcher_revokes_disappeared_peer(daemon): + daemon_obj, config = daemon + runtime = daemon_obj.runtime + client = MockNetBirdClient( + peers=[{"id": "peer-test", "connected": True}], + routes=[], + ) + runtime._netbird_client = client + daemon_obj._watcher._client = client + + from capability_runtime.catalog import LifecycleState + + ref = runtime.catalog.get_by_name("reach_api") + runtime.catalog.set_state(ref, LifecycleState.VISIBLE) + + agent = UnixRpcClient(config.agent_socket) + before = agent.call("capability.check", {"context": {}}) + assert "reach_api" in [n["name"] for n in before["result"]["networks"]] + + client.remove_peer("peer-test") + admin = UnixRpcClient(config.admin_socket) + admin.call("capability.watch.poll", {}) + + after = agent.call("capability.check", {"context": {}}) + assert "reach_api" not in [n["name"] for n in after["result"]["networks"]] diff --git a/capability-runtime/tests/test_rpc_unix.py b/capability-runtime/tests/test_rpc_unix.py new file mode 100644 index 00000000..d75d7d4f --- /dev/null +++ b/capability-runtime/tests/test_rpc_unix.py @@ -0,0 +1,83 @@ +"""Tests for AF_UNIX JSON-RPC transport.""" + +from __future__ import annotations + +import json +import time +from pathlib import Path +from unittest.mock import MagicMock + +import pytest + +from capability_runtime.rpc.transports.unix import UnixRpcClient, UnixRpcServer + + +@pytest.fixture +def mock_dispatcher(): + dispatcher = MagicMock() + dispatcher.handle.return_value = { + "jsonrpc": "2.0", + "result": {"ok": True}, + "id": 1, + } + return dispatcher + + +def _wait_for_socket(path: Path, timeout: float = 2.0) -> None: + deadline = time.time() + timeout + while time.time() < deadline: + if path.exists(): + return + time.sleep(0.01) + raise TimeoutError(f"socket not ready: {path}") + + +def test_server_handles_single_request(tmp_path, mock_dispatcher): + sock_path = tmp_path / "test.sock" + server = UnixRpcServer(sock_path, mock_dispatcher) + server.start() + try: + _wait_for_socket(sock_path) + client = UnixRpcClient(sock_path) + response = client.call("capability.check", {"context": {}}) + assert response["result"] == {"ok": True} + mock_dispatcher.handle.assert_called_once() + request = mock_dispatcher.handle.call_args[0][0] + assert request["method"] == "capability.check" + assert request["params"] == {"context": {}} + finally: + server.stop() + + +def test_one_request_per_connection(tmp_path, mock_dispatcher): + sock_path = tmp_path / "test.sock" + server = UnixRpcServer(sock_path, mock_dispatcher) + server.start() + try: + _wait_for_socket(sock_path) + client = UnixRpcClient(sock_path) + client.call("capability.check", {"context": {}}) + client.call("capability.lease", {"capability_ref": "cap-x", "justification": "x"}) + assert mock_dispatcher.handle.call_count == 2 + finally: + server.stop() + + +def test_invalid_json_returns_parse_error(tmp_path, mock_dispatcher): + sock_path = tmp_path / "test.sock" + server = UnixRpcServer(sock_path, mock_dispatcher) + server.start() + try: + _wait_for_socket(sock_path) + import socket + + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock: + sock.connect(str(sock_path)) + sock.sendall(b"not-json\n") + data = sock.recv(4096) + response = json.loads(data.decode().split("\n", 1)[0]) + assert response["error"]["code"] == -32700 + mock_dispatcher.handle.assert_not_called() + finally: + server.stop() + From bd251ea036490d99187dafece4d700ed1191cdac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 29 Jun 2026 10:17:46 +0000 Subject: [PATCH 041/138] feat(capability-runtime): add MCP stdio bridge to agent RPC socket --- .../src/capability_runtime/mcp/__init__.py | 6 + .../src/capability_runtime/mcp/bridge.py | 88 ++++++++++ .../src/capability_runtime/mcp/server.py | 152 +++++++++++++++++ .../rpc/transports/__init__.py | 3 +- .../rpc/transports/stdio_bridge.py | 24 +++ capability-runtime/tests/test_mcp_bridge.py | 156 ++++++++++++++++++ capability-runtime/tests/test_mcp_server.py | 151 +++++++++++++++++ 7 files changed, 579 insertions(+), 1 deletion(-) create mode 100644 capability-runtime/src/capability_runtime/mcp/__init__.py create mode 100644 capability-runtime/src/capability_runtime/mcp/bridge.py create mode 100644 capability-runtime/src/capability_runtime/mcp/server.py create mode 100644 capability-runtime/src/capability_runtime/rpc/transports/stdio_bridge.py create mode 100644 capability-runtime/tests/test_mcp_bridge.py create mode 100644 capability-runtime/tests/test_mcp_server.py diff --git a/capability-runtime/src/capability_runtime/mcp/__init__.py b/capability-runtime/src/capability_runtime/mcp/__init__.py new file mode 100644 index 00000000..aebbd83b --- /dev/null +++ b/capability-runtime/src/capability_runtime/mcp/__init__.py @@ -0,0 +1,6 @@ +"""Minimal MCP subset for capability meta-tools.""" + +from capability_runtime.mcp.bridge import run_mcp_bridge +from capability_runtime.mcp.server import McpServer + +__all__ = ["McpServer", "run_mcp_bridge"] diff --git a/capability-runtime/src/capability_runtime/mcp/bridge.py b/capability-runtime/src/capability_runtime/mcp/bridge.py new file mode 100644 index 00000000..6e1e18be --- /dev/null +++ b/capability-runtime/src/capability_runtime/mcp/bridge.py @@ -0,0 +1,88 @@ +"""Stdio MCP ↔ JSON-RPC bridge for agent containers.""" + +from __future__ import annotations + +import json +import os +import sys +from collections.abc import Callable +from typing import Any, TextIO + +from capability_runtime.mcp.server import McpServer +from capability_runtime.rpc.transports.stdio_bridge import BridgeRpcClient + +DEFAULT_AGENT_SOCKET = "/run/sandcat/capability/agent.sock" + + +def run_mcp_bridge( + stdin: TextIO | None = None, + stdout: TextIO | None = None, + *, + rpc_client: BridgeRpcClient | None = None, +) -> None: + """Read newline-delimited MCP JSON-RPC from stdin; write responses to stdout.""" + stdin = stdin or sys.stdin + stdout = stdout or sys.stdout + + _require_agent_id() + client = rpc_client or BridgeRpcClient(_agent_socket_path()) + server = McpServer() + + for line in stdin: + line = line.strip() + if not line: + continue + try: + request = json.loads(line) + except json.JSONDecodeError: + response = { + "jsonrpc": "2.0", + "error": {"code": -32700, "message": "Parse error"}, + "id": None, + } + _write_response(stdout, response) + continue + + if not isinstance(request, dict): + response = { + "jsonrpc": "2.0", + "error": {"code": -32600, "message": "Invalid Request"}, + "id": None, + } + _write_response(stdout, response) + continue + + response = server.handle(request, _make_rpc_call(client)) + if response is not None: + _write_response(stdout, response) + + +def _make_rpc_call(client: BridgeRpcClient) -> Callable[[str, dict[str, Any]], dict[str, Any]]: + def rpc_call(method: str, params: dict[str, Any]) -> dict[str, Any]: + return client.call(method, params) + + return rpc_call + + +def _write_response(stdout: TextIO, response: dict[str, Any]) -> None: + stdout.write(json.dumps(response) + "\n") + stdout.flush() + + +def _require_agent_id() -> str: + agent_id = os.environ.get("SANDCAT_AGENT_ID") + if not agent_id: + raise RuntimeError("SANDCAT_AGENT_ID environment variable is required") + return agent_id + + +def _agent_socket_path() -> str: + return os.environ.get("CAPABILITY_AGENT_SOCKET", DEFAULT_AGENT_SOCKET) + + +def main() -> None: + run_mcp_bridge() + + +if __name__ == "__main__": + main() diff --git a/capability-runtime/src/capability_runtime/mcp/server.py b/capability-runtime/src/capability_runtime/mcp/server.py new file mode 100644 index 00000000..5f10f7f6 --- /dev/null +++ b/capability-runtime/src/capability_runtime/mcp/server.py @@ -0,0 +1,152 @@ +"""Minimal MCP JSON-RPC subset: initialize, tools/list, tools/call.""" + +from __future__ import annotations + +import json +from collections.abc import Callable +from typing import Any + +PROTOCOL_VERSION = "2024-11-05" +SERVER_INFO = {"name": "capability-runtime", "version": "0.1.0"} + +TOOLS: list[dict[str, Any]] = [ + { + "name": "capability_check", + "description": "Return the agent's current capability bundle.", + "inputSchema": { + "type": "object", + "properties": { + "context": {"type": "object", "description": "Execution context"}, + }, + }, + }, + { + "name": "capability_lease", + "description": "Request a lease for a capability.", + "inputSchema": { + "type": "object", + "properties": { + "capability_ref": {"type": "string"}, + "justification": {"type": "string"}, + }, + "required": ["capability_ref", "justification"], + }, + }, + { + "name": "capability_discover", + "description": "Discover capabilities matching a query.", + "inputSchema": { + "type": "object", + "properties": { + "query": {"type": "string"}, + }, + "required": ["query"], + }, + }, +] + +_TOOL_TO_RPC: dict[str, str] = { + "capability_check": "capability.check", + "capability_lease": "capability.lease", + "capability_discover": "capability.discover", +} + + +RpcCall = Callable[[str, dict[str, Any]], dict[str, Any]] + + +class McpServer: + """Handle MCP requests and forward tool calls to capability JSON-RPC.""" + + def handle(self, request: dict[str, Any], rpc_call: RpcCall) -> dict[str, Any] | None: + """Handle one MCP JSON-RPC request. Returns None for notifications.""" + request_id = request.get("id") + if request.get("jsonrpc") != "2.0": + return _error_response(request_id, -32600, "Invalid Request") + + method = request.get("method") + if not isinstance(method, str): + return _error_response(request_id, -32600, "Invalid Request") + + if method == "notifications/initialized" or method == "initialized": + return None + + params = request.get("params") or {} + if not isinstance(params, dict): + return _error_response(request_id, -32602, "Invalid params") + + if method == "initialize": + return _success_response(request_id, _handle_initialize()) + if method == "tools/list": + return _success_response(request_id, {"tools": TOOLS}) + if method == "tools/call": + return _success_response(request_id, _handle_tools_call(params, rpc_call)) + + return _error_response(request_id, -32601, f"Method not found: {method}") + + +def _handle_initialize() -> dict[str, Any]: + return { + "protocolVersion": PROTOCOL_VERSION, + "capabilities": {"tools": {}}, + "serverInfo": SERVER_INFO, + } + + +def _handle_tools_call(params: dict[str, Any], rpc_call: RpcCall) -> dict[str, Any]: + name = params.get("name") + if not isinstance(name, str) or name not in _TOOL_TO_RPC: + return { + "content": [{"type": "text", "text": f"Unknown tool: {name!r}"}], + "isError": True, + } + + arguments = params.get("arguments") or {} + if not isinstance(arguments, dict): + return { + "content": [{"type": "text", "text": "arguments must be an object"}], + "isError": True, + } + + rpc_method = _TOOL_TO_RPC[name] + rpc_params = _map_tool_arguments(name, arguments) + rpc_response = rpc_call(rpc_method, rpc_params) + + if "error" in rpc_response: + error = rpc_response["error"] + message = error.get("message", "RPC error") + return { + "content": [{"type": "text", "text": message}], + "isError": True, + } + + result = rpc_response.get("result", rpc_response) + return { + "content": [{"type": "text", "text": json.dumps(result)}], + "isError": False, + } + + +def _map_tool_arguments(tool_name: str, arguments: dict[str, Any]) -> dict[str, Any]: + if tool_name == "capability_check": + return {"context": arguments.get("context", {})} + if tool_name == "capability_lease": + return { + "capability_ref": arguments["capability_ref"], + "justification": arguments["justification"], + } + if tool_name == "capability_discover": + return {"query": arguments["query"]} + raise ValueError(f"unmapped tool: {tool_name}") + + +def _success_response(request_id: Any, result: dict[str, Any]) -> dict[str, Any]: + return {"jsonrpc": "2.0", "result": result, "id": request_id} + + +def _error_response(request_id: Any, code: int, message: str) -> dict[str, Any]: + return { + "jsonrpc": "2.0", + "error": {"code": code, "message": message}, + "id": request_id, + } diff --git a/capability-runtime/src/capability_runtime/rpc/transports/__init__.py b/capability-runtime/src/capability_runtime/rpc/transports/__init__.py index 6a45637f..88ae9734 100644 --- a/capability-runtime/src/capability_runtime/rpc/transports/__init__.py +++ b/capability-runtime/src/capability_runtime/rpc/transports/__init__.py @@ -1,5 +1,6 @@ """RPC transport implementations.""" +from capability_runtime.rpc.transports.stdio_bridge import BridgeRpcClient from capability_runtime.rpc.transports.unix import UnixRpcClient, UnixRpcServer -__all__ = ["UnixRpcClient", "UnixRpcServer"] +__all__ = ["BridgeRpcClient", "UnixRpcClient", "UnixRpcServer"] diff --git a/capability-runtime/src/capability_runtime/rpc/transports/stdio_bridge.py b/capability-runtime/src/capability_runtime/rpc/transports/stdio_bridge.py new file mode 100644 index 00000000..ce5cb00e --- /dev/null +++ b/capability-runtime/src/capability_runtime/rpc/transports/stdio_bridge.py @@ -0,0 +1,24 @@ +"""Unix-socket JSON-RPC client for the agent MCP stdio bridge.""" + +from __future__ import annotations + +from pathlib import Path +from typing import Any + +from capability_runtime.rpc.transports.unix import UnixRpcClient + + +class BridgeRpcClient: + """Forward JSON-RPC requests to the capability agent socket.""" + + def __init__(self, socket_path: Path | str, *, timeout: float = 5.0) -> None: + self._client = UnixRpcClient(Path(socket_path), timeout=timeout) + + def call( + self, + method: str, + params: dict[str, Any] | None = None, + *, + request_id: int | str = 1, + ) -> dict[str, Any]: + return self._client.call(method, params, request_id=request_id) diff --git a/capability-runtime/tests/test_mcp_bridge.py b/capability-runtime/tests/test_mcp_bridge.py new file mode 100644 index 00000000..817507f3 --- /dev/null +++ b/capability-runtime/tests/test_mcp_bridge.py @@ -0,0 +1,156 @@ +"""Tests for stdio MCP ↔ JSON-RPC bridge.""" + +from __future__ import annotations + +import io +import json +import threading +import time +from pathlib import Path +from unittest.mock import MagicMock + +import pytest + +from capability_runtime.mcp.bridge import run_mcp_bridge +from capability_runtime.rpc.transports.unix import UnixRpcServer + + +def _wait_for_socket(path: Path, timeout: float = 2.0) -> None: + import socket + + deadline = time.time() + timeout + while time.time() < deadline: + if not path.exists(): + time.sleep(0.01) + continue + try: + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock: + sock.settimeout(0.2) + sock.connect(str(path)) + return + except OSError: + time.sleep(0.01) + raise TimeoutError(f"socket not ready: {path}") + + +@pytest.fixture +def mock_dispatcher(): + dispatcher = MagicMock() + dispatcher.handle.return_value = { + "jsonrpc": "2.0", + "result": { + "agent_id": "devcontainer-agent", + "tools": [], + "networks": [], + "rules": [], + "skills": [], + "policies": [], + "hooks": [], + "budgets": {}, + "provenance": {}, + "version": 1, + }, + "id": 1, + } + return dispatcher + + +@pytest.fixture +def rpc_server(tmp_path, mock_dispatcher): + sock_path = tmp_path / "agent.sock" + server = UnixRpcServer(sock_path, mock_dispatcher) + server.start() + _wait_for_socket(sock_path) + yield sock_path, mock_dispatcher + server.stop() + + +def test_bridge_forwards_tools_call_to_rpc(tmp_path, monkeypatch, rpc_server): + sock_path, mock_dispatcher = rpc_server + monkeypatch.setenv("SANDCAT_AGENT_ID", "devcontainer-agent") + monkeypatch.setenv("CAPABILITY_AGENT_SOCKET", str(sock_path)) + + stdin = io.StringIO( + '{"jsonrpc":"2.0","id":1,"method":"tools/call",' + '"params":{"name":"capability_check","arguments":{"context":{}}}}\n' + ) + stdout = io.StringIO() + run_mcp_bridge(stdin, stdout) + + out = json.loads(stdout.getvalue()) + assert "result" in out + assert out["result"]["isError"] is False + mock_dispatcher.handle.assert_called_once() + request = mock_dispatcher.handle.call_args[0][0] + assert request["method"] == "capability.check" + assert request["params"] == {"context": {}} + + +def test_bridge_initialize_and_tools_list(tmp_path, monkeypatch, rpc_server): + sock_path, _ = rpc_server + monkeypatch.setenv("SANDCAT_AGENT_ID", "devcontainer-agent") + monkeypatch.setenv("CAPABILITY_AGENT_SOCKET", str(sock_path)) + + stdin = io.StringIO( + '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}\n' + '{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}\n' + ) + stdout = io.StringIO() + run_mcp_bridge(stdin, stdout) + + lines = [json.loads(line) for line in stdout.getvalue().strip().split("\n")] + assert lines[0]["result"]["protocolVersion"] == "2024-11-05" + tool_names = {t["name"] for t in lines[1]["result"]["tools"]} + assert "capability_check" in tool_names + + +def test_bridge_requires_agent_id(monkeypatch): + monkeypatch.delenv("SANDCAT_AGENT_ID", raising=False) + with pytest.raises(RuntimeError, match="SANDCAT_AGENT_ID"): + run_mcp_bridge(io.StringIO(""), io.StringIO()) + + +def test_bridge_integration_with_daemon(tmp_path, monkeypatch): + """End-to-end bridge → real daemon agent socket.""" + from capability_runtime.daemon import CapabilityDaemon, DaemonConfig + + catalog = { + "capabilities": [ + {"name": "create_pr", "ref": "cap-create-pr", "type": "tool"}, + ] + } + catalog_path = tmp_path / "catalog.json" + catalog_path.write_text(json.dumps(catalog)) + + sock_dir = tmp_path / "sockets" + config = DaemonConfig( + catalog_path=catalog_path, + agent_socket=sock_dir / "agent.sock", + admin_socket=sock_dir / "admin.sock", + trace_file=tmp_path / "trace.jsonl", + agent_id="devcontainer-agent", + watch_interval=0.1, + mock_netbird=True, + ) + daemon = CapabilityDaemon(config) + thread = threading.Thread(target=daemon.start, daemon=True) + thread.start() + _wait_for_socket(config.agent_socket, timeout=3.0) + + try: + monkeypatch.setenv("SANDCAT_AGENT_ID", "devcontainer-agent") + monkeypatch.setenv("CAPABILITY_AGENT_SOCKET", str(config.agent_socket)) + + stdin = io.StringIO( + '{"jsonrpc":"2.0","id":1,"method":"tools/call",' + '"params":{"name":"capability_check","arguments":{"context":{}}}}\n' + ) + stdout = io.StringIO() + run_mcp_bridge(stdin, stdout) + + out = json.loads(stdout.getvalue()) + assert "result" in out + payload = json.loads(out["result"]["content"][0]["text"]) + assert payload["agent_id"] == "devcontainer-agent" + finally: + daemon.stop() diff --git a/capability-runtime/tests/test_mcp_server.py b/capability-runtime/tests/test_mcp_server.py new file mode 100644 index 00000000..4e7cfd44 --- /dev/null +++ b/capability-runtime/tests/test_mcp_server.py @@ -0,0 +1,151 @@ +"""Tests for minimal MCP server.""" + +from __future__ import annotations + +import json + +import pytest + +from capability_runtime.mcp.server import McpServer, TOOLS + + +def _rpc_call(method: str, params: dict) -> dict: + if method == "capability.check": + return {"jsonrpc": "2.0", "result": {"agent_id": "test-agent", "tools": []}, "id": 1} + if method == "capability.lease": + return { + "jsonrpc": "2.0", + "result": {"lease_id": "lease-1", "capability_ref": params["capability_ref"]}, + "id": 1, + } + if method == "capability.discover": + return {"jsonrpc": "2.0", "result": {"capabilities": ["cap-a"], "denied": []}, "id": 1} + return {"jsonrpc": "2.0", "error": {"code": -32601, "message": "not found"}, "id": 1} + + +@pytest.fixture +def server(): + return McpServer() + + +def test_initialize_returns_protocol_version(server): + response = server.handle( + {"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {}}, + _rpc_call, + ) + assert response is not None + assert response["result"]["protocolVersion"] == "2024-11-05" + assert "serverInfo" in response["result"] + + +def test_tools_list_returns_meta_tools(server): + response = server.handle( + {"jsonrpc": "2.0", "id": 2, "method": "tools/list", "params": {}}, + _rpc_call, + ) + assert response is not None + names = {tool["name"] for tool in response["result"]["tools"]} + assert names == {"capability_check", "capability_lease", "capability_discover"} + assert len(response["result"]["tools"]) == len(TOOLS) + + +def test_tools_call_check_forwards_to_rpc(server): + response = server.handle( + { + "jsonrpc": "2.0", + "id": 3, + "method": "tools/call", + "params": {"name": "capability_check", "arguments": {"context": {"task": "x"}}}, + }, + _rpc_call, + ) + assert response is not None + content = response["result"]["content"][0]["text"] + payload = json.loads(content) + assert payload["agent_id"] == "test-agent" + assert response["result"]["isError"] is False + + +def test_tools_call_lease_forwards_to_rpc(server): + response = server.handle( + { + "jsonrpc": "2.0", + "id": 4, + "method": "tools/call", + "params": { + "name": "capability_lease", + "arguments": { + "capability_ref": "cap-create-pr", + "justification": "need it", + }, + }, + }, + _rpc_call, + ) + assert response is not None + payload = json.loads(response["result"]["content"][0]["text"]) + assert payload["lease_id"] == "lease-1" + + +def test_tools_call_discover_forwards_to_rpc(server): + response = server.handle( + { + "jsonrpc": "2.0", + "id": 5, + "method": "tools/call", + "params": {"name": "capability_discover", "arguments": {"query": "pr"}}, + }, + _rpc_call, + ) + assert response is not None + payload = json.loads(response["result"]["content"][0]["text"]) + assert payload["capabilities"] == ["cap-a"] + + +def test_tools_call_unknown_tool(server): + response = server.handle( + { + "jsonrpc": "2.0", + "id": 6, + "method": "tools/call", + "params": {"name": "unknown_tool", "arguments": {}}, + }, + _rpc_call, + ) + assert response is not None + assert response["result"]["isError"] is True + + +def test_tools_call_rpc_error(server): + def failing_rpc(method: str, params: dict) -> dict: + return {"jsonrpc": "2.0", "error": {"code": -32603, "message": "boom"}, "id": 1} + + response = server.handle( + { + "jsonrpc": "2.0", + "id": 7, + "method": "tools/call", + "params": {"name": "capability_check", "arguments": {"context": {}}}, + }, + failing_rpc, + ) + assert response is not None + assert response["result"]["isError"] is True + assert response["result"]["content"][0]["text"] == "boom" + + +def test_initialized_notification_returns_none(server): + response = server.handle( + {"jsonrpc": "2.0", "method": "notifications/initialized", "params": {}}, + _rpc_call, + ) + assert response is None + + +def test_unknown_method_returns_error(server): + response = server.handle( + {"jsonrpc": "2.0", "id": 8, "method": "resources/list", "params": {}}, + _rpc_call, + ) + assert response is not None + assert response["error"]["code"] == -32601 From c6927dcc07baf8511547815babd02750ebb2ce68 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 29 Jun 2026 10:20:16 +0000 Subject: [PATCH 042/138] feat(cli): add capability-runtime sidecar compose integration --- cli/lib/composefile.bash | 42 +++++++++++++++++++ cli/libexec/init/devcontainer | 28 +++++++++++++ cli/libexec/init/init | 18 ++++++-- cli/templates/devcontainer/Dockerfile.app | 3 ++ cli/templates/devcontainer/compose-all.yml | 1 + .../sandcat/Dockerfile.capability-runtime | 7 ++++ .../sandcat/capability-catalog.json | 17 ++++++++ .../sandcat/compose-capability.yml | 17 ++++++++ .../sandcat/scripts/capability-mcp-bridge.sh | 2 + cli/test/composefile/capability.bats | 38 +++++++++++++++++ 10 files changed, 170 insertions(+), 3 deletions(-) create mode 100644 cli/templates/devcontainer/sandcat/Dockerfile.capability-runtime create mode 100644 cli/templates/devcontainer/sandcat/capability-catalog.json create mode 100644 cli/templates/devcontainer/sandcat/compose-capability.yml create mode 100644 cli/templates/devcontainer/sandcat/scripts/capability-mcp-bridge.sh create mode 100644 cli/test/composefile/capability.bats diff --git a/cli/lib/composefile.bash b/cli/lib/composefile.bash index e67bc2fb..8a0be78b 100644 --- a/cli/lib/composefile.bash +++ b/cli/lib/composefile.bash @@ -664,3 +664,45 @@ enable_netbird() { fi fi } + +# Adds capability-runtime sidecar include and agent socket mount/env to compose-all.yml. +# Idempotent: skips entries that are already present. +# Args: +# $1 - Path to the devcontainer directory (parent of compose-all.yml) +enable_capability() { + require yq + local compose_dir=$1 + local compose_file="$compose_dir/compose-all.yml" + + local has_include + has_include=$(yq '[.include[]? | select(.path == "sandcat/compose-capability.yml")] | length' "$compose_file") + if [[ "$has_include" -eq 0 ]]; then + yq -i '.include += [{"path": "sandcat/compose-capability.yml"}]' "$compose_file" + fi + + local has_volume + has_volume=$(yq '[.services.agent.volumes[]? | select(. == "capability-socket:/run/sandcat/capability:ro")] | length' "$compose_file") + if [[ "$has_volume" -eq 0 ]]; then + yq -i '.services.agent.volumes += ["capability-socket:/run/sandcat/capability:ro"]' "$compose_file" + fi + + local has_agent_id has_socket + has_agent_id=$(yq '[.services.agent.environment[]? | select(. == "SANDCAT_AGENT_ID=devcontainer-agent")] | length' "$compose_file") + has_socket=$(yq '[.services.agent.environment[]? | select(. == "CAPABILITY_AGENT_SOCKET=/run/sandcat/capability/agent.sock")] | length' "$compose_file") + + if [[ "$has_agent_id" -eq 0 || "$has_socket" -eq 0 ]]; then + local env_additions=() + if [[ "$has_agent_id" -eq 0 ]]; then + env_additions+=("SANDCAT_AGENT_ID=devcontainer-agent") + fi + if [[ "$has_socket" -eq 0 ]]; then + env_additions+=("CAPABILITY_AGENT_SOCKET=/run/sandcat/capability/agent.sock") + fi + local entry yq_array="" + for entry in "${env_additions[@]}"; do + yq_array+="\"${entry}\"," + done + yq_array="[${yq_array%,}]" + yq -i ".services.agent.environment = ((.services.agent.environment // []) + ${yq_array})" "$compose_file" + fi +} diff --git a/cli/libexec/init/devcontainer b/cli/libexec/init/devcontainer index 46f656dd..49e2d291 100755 --- a/cli/libexec/init/devcontainer +++ b/cli/libexec/init/devcontainer @@ -16,6 +16,7 @@ source "$SCT_LIBDIR/devcontainer.bash" # --agent - The agent name (e.g., "claude") # --ide - The IDE name (e.g., "vscode", "jetbrains", "none") (optional) # --netbird-management-url - NetBird management server URL for wg-client (optional) +# --capability - Enable capability-runtime sidecar (requires --netbird) devcontainer() { local settings_file="" local project_path="" @@ -27,6 +28,7 @@ devcontainer() { local secret_provider="none" local netbird="false" local netbird_management_url="" + local capability="false" while [[ $# -gt 0 ]] do @@ -79,6 +81,10 @@ devcontainer() { netbird="true" shift 1 ;; + --capability) + capability="true" + shift 1 + ;; --netbird-management-url) netbird_management_url="$2" shift 2 @@ -94,6 +100,11 @@ devcontainer() { : "${project_path?Missing required parameter: --project-path}" : "${agent?Missing required parameter: --agent}" + if [[ "$capability" == "true" && "$netbird" != "true" ]]; then + echo "--capability requires --netbird" | error + return 1 + fi + local devcontainer_dir="$project_path/.devcontainer" local compose_file="$devcontainer_dir/compose-all.yml" @@ -139,6 +150,23 @@ devcontainer() { enable_netbird "$devcontainer_dir/sandcat/compose-proxy.yml" "$netbird_management_url" fi + if [[ "$capability" == "true" ]]; then + local capability_runtime_src + capability_runtime_src="$(cd "$SCT_ROOT/.." && pwd)/capability-runtime" + if [[ ! -d "$capability_runtime_src" ]]; then + echo "capability-runtime source not found: $capability_runtime_src" | error + return 1 + fi + cp -R "$capability_runtime_src" "$devcontainer_dir/sandcat/capability-runtime" + apply_template_placeholders \ + "$devcontainer_dir/Dockerfile.app" \ + "__CAPABILITY_RUNTIME_INSTALL__" \ + "COPY --from=python:3.12-slim /usr/local /usr/local +COPY sandcat/capability-runtime /tmp/capability-runtime +RUN pip install --no-cache-dir /tmp/capability-runtime && rm -rf /tmp/capability-runtime" + enable_capability "$devcontainer_dir" + fi + customize_compose_file "$rel_settings_file" "$compose_file" "$agent" "$ide" "$project_name" "$stacks" set_project_name "$compose_file" "$project_name" diff --git a/cli/libexec/init/init b/cli/libexec/init/init index f97a1661..e4f64617 100755 --- a/cli/libexec/init/init +++ b/cli/libexec/init/init @@ -163,6 +163,7 @@ add_secret_provider_tokens_to_user_settings() { # --secret-provider / --sp - Secret backend: none, 1password, protonpass # --1password - Deprecated; same as --secret-provider 1password # --features - Comma-separated optional non-provider features (tui) +# --capability - Enable capability-runtime sidecar (requires --netbird) init() { require yq @@ -184,6 +185,7 @@ init() { local netbird_management_url="" local provisioned_netbird_server="false" local netbird_selfhosted_quickstart="false" + local capability="false" while [[ $# -gt 0 ]] do @@ -239,6 +241,10 @@ init() { netbird="true" shift 1 ;; + --capability) + capability="true" + shift 1 + ;; --netbird-server) netbird_server="$2" netbird_server_provided=true @@ -263,6 +269,10 @@ init() { echo "--netbird-server requires --netbird" | error return 1 fi + if [[ "$capability" == "true" && "$netbird" != "true" ]]; then + echo "--capability requires --netbird" | error + return 1 + fi if [[ "$netbird_server_provided" == "true" ]]; then case "$netbird_server" in cloud|new|quickstart|http://*|https://*) @@ -570,10 +580,12 @@ init() { devcontainer_args+=(--netbird-management-url "$netbird_management_url") fi if [[ "$netbird" == "true" ]]; then - devcontainer "${devcontainer_args[@]}" --netbird - else - devcontainer "${devcontainer_args[@]}" + devcontainer_args+=(--netbird) + fi + if [[ "$capability" == "true" ]]; then + devcontainer_args+=(--capability) fi + devcontainer "${devcontainer_args[@]}" local gitignore_status="skipped" if [[ -e "$project_path/.git" ]]; then diff --git a/cli/templates/devcontainer/Dockerfile.app b/cli/templates/devcontainer/Dockerfile.app index 9bb586ed..e90ee505 100644 --- a/cli/templates/devcontainer/Dockerfile.app +++ b/cli/templates/devcontainer/Dockerfile.app @@ -30,8 +30,11 @@ RUN rm -f /etc/sudoers.d/vscode COPY --chmod=755 sandcat/scripts/app-init.sh /usr/local/bin/app-init.sh COPY --chmod=755 sandcat/scripts/app-user-init.sh /usr/local/bin/app-user-init.sh COPY --chmod=644 sandcat/scripts/java-env.sh /etc/profile.d/sandcat-java.sh +COPY --chmod=755 sandcat/scripts/capability-mcp-bridge.sh /usr/local/bin/capability-mcp-bridge COPY --chown=vscode:vscode sandcat/tmux.conf /home/vscode/.tmux.conf +# __CAPABILITY_RUNTIME_INSTALL__ + USER vscode ENV LANG="en_US.UTF-8" diff --git a/cli/templates/devcontainer/compose-all.yml b/cli/templates/devcontainer/compose-all.yml index 014cd5ef..8882a2f8 100644 --- a/cli/templates/devcontainer/compose-all.yml +++ b/cli/templates/devcontainer/compose-all.yml @@ -5,6 +5,7 @@ include: # wg-client, dependency ordering. The `services.agent` entries below are # merged OVER that base by Docker Compose. - path: sandcat/compose-agent.yml + # sandcat init --capability adds: sandcat/compose-capability.yml services: # User-customizable parts of the agent service (volumes, environment). diff --git a/cli/templates/devcontainer/sandcat/Dockerfile.capability-runtime b/cli/templates/devcontainer/sandcat/Dockerfile.capability-runtime new file mode 100644 index 00000000..6a9eb37a --- /dev/null +++ b/cli/templates/devcontainer/sandcat/Dockerfile.capability-runtime @@ -0,0 +1,7 @@ +FROM python:3.12-slim +WORKDIR /app +COPY capability-runtime/pyproject.toml capability-runtime/src ./ +ENV PYTHONPATH=/app/src +RUN pip install --no-cache-dir -e . +COPY capability-catalog.json /etc/sandcat/capability-catalog.json +ENTRYPOINT ["python", "-m", "capability_runtime.daemon"] diff --git a/cli/templates/devcontainer/sandcat/capability-catalog.json b/cli/templates/devcontainer/sandcat/capability-catalog.json new file mode 100644 index 00000000..219ca43c --- /dev/null +++ b/cli/templates/devcontainer/sandcat/capability-catalog.json @@ -0,0 +1,17 @@ +{ + "capabilities": [ + { + "name": "create_pr", + "ref": "cap-create-pr", + "type": "tool" + }, + { + "name": "reach_api", + "ref": "cap-reach-api", + "type": "network", + "peer_id": "peer-placeholder", + "network": "10.8.0.0/24", + "route_id": "route-placeholder" + } + ] +} diff --git a/cli/templates/devcontainer/sandcat/compose-capability.yml b/cli/templates/devcontainer/sandcat/compose-capability.yml new file mode 100644 index 00000000..c5a34162 --- /dev/null +++ b/cli/templates/devcontainer/sandcat/compose-capability.yml @@ -0,0 +1,17 @@ +services: + capability-runtime: + build: + context: . + dockerfile: Dockerfile.capability-runtime + volumes: + - capability-socket:/run/sandcat/capability + - ~/.config/sandcat/settings.json:/config/settings.json:ro + environment: + - SANDCAT_SETTINGS_USER=/config/settings.json + - CAPABILITY_CATALOG_JSON=/etc/sandcat/capability-catalog.json + - CAPABILITY_AGENT_SOCKET=/run/sandcat/capability/agent.sock + - CAPABILITY_ADMIN_SOCKET=/run/sandcat/capability/admin.sock + restart: unless-stopped + +volumes: + capability-socket: diff --git a/cli/templates/devcontainer/sandcat/scripts/capability-mcp-bridge.sh b/cli/templates/devcontainer/sandcat/scripts/capability-mcp-bridge.sh new file mode 100644 index 00000000..a20d16df --- /dev/null +++ b/cli/templates/devcontainer/sandcat/scripts/capability-mcp-bridge.sh @@ -0,0 +1,2 @@ +#!/bin/bash +exec python -m capability_runtime.mcp.bridge diff --git a/cli/test/composefile/capability.bats b/cli/test/composefile/capability.bats new file mode 100644 index 00000000..d87b8edc --- /dev/null +++ b/cli/test/composefile/capability.bats @@ -0,0 +1,38 @@ +#!/usr/bin/env bats + +setup() { + load test_helper + source "$SCT_LIBDIR/composefile.bash" + + COMPOSE_DIR="$BATS_TEST_TMPDIR/devcontainer" + mkdir -p "$COMPOSE_DIR/sandcat" + cp "$SCT_TEMPLATEDIR/devcontainer/compose-all.yml" "$COMPOSE_DIR/compose-all.yml" + cp "$SCT_TEMPLATEDIR/devcontainer/sandcat/compose-capability.yml" "$COMPOSE_DIR/sandcat/compose-capability.yml" +} + +teardown() { + unstub_all +} + +@test "enable_capability adds capability-runtime service include" { + enable_capability "$COMPOSE_DIR" + run yq '.include[] | select(.path == "sandcat/compose-capability.yml")' "$COMPOSE_DIR/compose-all.yml" + [ "$status" -eq 0 ] +} + +@test "enable_capability sets SANDCAT_AGENT_ID on agent service" { + enable_capability "$COMPOSE_DIR" + run yq '.services.agent.environment[] | select(. == "SANDCAT_AGENT_ID=devcontainer-agent")' "$COMPOSE_DIR/compose-all.yml" + [ "$status" -eq 0 ] +} + +@test "enable_capability is idempotent" { + enable_capability "$COMPOSE_DIR" + enable_capability "$COMPOSE_DIR" + + run yq '[.include[] | select(.path == "sandcat/compose-capability.yml")] | length' "$COMPOSE_DIR/compose-all.yml" + assert_output "1" + + run yq '[.services.agent.environment[] | select(. == "SANDCAT_AGENT_ID=devcontainer-agent")] | length' "$COMPOSE_DIR/compose-all.yml" + assert_output "1" +} From 6ada50922fce68d3e9bfb575aebdef42b43ea2e7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 29 Jun 2026 10:22:10 +0000 Subject: [PATCH 043/138] feat(cli): add sandcat capability operator subcommand --- .../src/capability_runtime/cli.py | 210 ++++++++++++++++ cli/lib/capability.bash | 225 ++++++++++++++++++ cli/libexec/capability/_ | 7 + cli/libexec/capability/capability | 65 +++++ cli/test/capability/capability.bats | 79 ++++++ cli/test/capability/test_helper.bash | 21 ++ 6 files changed, 607 insertions(+) create mode 100644 capability-runtime/src/capability_runtime/cli.py create mode 100644 cli/lib/capability.bash create mode 100755 cli/libexec/capability/_ create mode 100755 cli/libexec/capability/capability create mode 100644 cli/test/capability/capability.bats create mode 100644 cli/test/capability/test_helper.bash diff --git a/capability-runtime/src/capability_runtime/cli.py b/capability-runtime/src/capability_runtime/cli.py new file mode 100644 index 00000000..f4a147af --- /dev/null +++ b/capability-runtime/src/capability_runtime/cli.py @@ -0,0 +1,210 @@ +"""Admin JSON-RPC client for the capability sidecar operator surface.""" + +from __future__ import annotations + +import argparse +import json +import os +import sys +import time +from pathlib import Path + +from capability_runtime.rpc.transports.unix import UnixRpcClient + +DEFAULT_ADMIN_SOCKET = Path( + os.environ.get("CAPABILITY_ADMIN_SOCKET", "/run/sandcat/capability/admin.sock") +) + +SUBCOMMAND_TO_METHOD = { + "check": "capability.check", + "lease": "capability.lease", + "revoke": "capability.revoke", + "watch": "capability.watch.poll", +} + + +def _resolve_method(name: str) -> str: + if name.startswith("capability."): + return name + try: + return SUBCOMMAND_TO_METHOD[name] + except KeyError as exc: + raise SystemExit(f"unknown command: {name}") from exc + + +def _admin_client() -> UnixRpcClient: + return UnixRpcClient(DEFAULT_ADMIN_SOCKET) + + +def _call(method: str, params: dict) -> dict: + response = _admin_client().call(method, params) + if "error" in response: + error = response["error"] + print( + f"RPC error {error.get('code')}: {error.get('message')}", + file=sys.stderr, + ) + raise SystemExit(1) + return response["result"] + + +def _print_json(value: object) -> None: + print(json.dumps(value, indent=2)) + + +def _parse_context(raw: str) -> dict: + try: + parsed = json.loads(raw) + except json.JSONDecodeError as exc: + raise SystemExit(f"invalid --context JSON: {exc}") from exc + if not isinstance(parsed, dict): + raise SystemExit("--context must be a JSON object") + return parsed + + +def cmd_check(args: argparse.Namespace) -> None: + params: dict = {"context": _parse_context(args.context)} + if args.agent_id: + params["agent_id"] = args.agent_id + _print_json(_call("capability.check", params)) + + +def cmd_lease(args: argparse.Namespace) -> None: + params = { + "capability_ref": args.ref, + "justification": args.justification, + } + if args.agent_id: + params["agent_id"] = args.agent_id + _print_json(_call("capability.lease", params)) + + +def cmd_revoke(args: argparse.Namespace) -> None: + _print_json( + _call( + "capability.revoke", + {"target": args.ref, "reason": args.reason}, + ) + ) + + +def cmd_watch(args: argparse.Namespace) -> None: + print( + f"Watching capability route watcher (poll every {args.interval}s)...", + file=sys.stderr, + ) + while True: + result = _call("capability.watch.poll", {}) + _print_json({"polled": result.get("polled", True), "ts": time.time()}) + sys.stdout.flush() + time.sleep(args.interval) + + +def cmd_demo(args: argparse.Namespace) -> None: + agent_id = args.agent_id or "devcontainer-agent" + print("=== capability demo: check ===", file=sys.stderr) + before = _call("capability.check", {"agent_id": agent_id, "context": {}}) + _print_json(before) + + print("=== capability demo: lease cap-create-pr ===", file=sys.stderr) + lease = _call( + "capability.lease", + { + "agent_id": agent_id, + "capability_ref": "cap-create-pr", + "justification": "operator demo", + }, + ) + _print_json(lease) + + print("=== capability demo: check after lease ===", file=sys.stderr) + after_lease = _call("capability.check", {"agent_id": agent_id, "context": {}}) + _print_json(after_lease) + + print("=== capability demo: revoke cap-create-pr ===", file=sys.stderr) + revoked = _call( + "capability.revoke", + {"target": "cap-create-pr", "reason": "demo complete"}, + ) + _print_json(revoked) + + print("=== capability demo: check after revoke ===", file=sys.stderr) + after_revoke = _call("capability.check", {"agent_id": agent_id, "context": {}}) + _print_json(after_revoke) + + +def _add_agent_arg(parser: argparse.ArgumentParser) -> None: + parser.add_argument( + "--agent-id", + "--agent", + dest="agent_id", + help="Agent identity (default: devcontainer-agent on admin surface)", + ) + + +def _build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser(prog="capability_runtime.cli") + subparsers = parser.add_subparsers(dest="command", required=True) + + check = subparsers.add_parser("check", help="Run capability.check") + check.add_argument("--context", default="{}", help="Capability context JSON object") + _add_agent_arg(check) + check.set_defaults(handler=cmd_check) + + lease = subparsers.add_parser("lease", help="Run capability.lease") + lease.add_argument("--ref", required=True, help="Capability reference") + lease.add_argument("--justification", required=True, help="Lease justification") + _add_agent_arg(lease) + lease.set_defaults(handler=cmd_lease) + + revoke = subparsers.add_parser("revoke", help="Run capability.revoke") + revoke.add_argument("--ref", required=True, help="Capability ref or lease id") + revoke.add_argument("--reason", required=True, help="Revocation reason") + revoke.set_defaults(handler=cmd_revoke) + + watch = subparsers.add_parser("watch", help="Poll capability.watch.poll in a loop") + watch.add_argument( + "--interval", + type=float, + default=float(os.environ.get("CAPABILITY_WATCH_INTERVAL", "5")), + help="Seconds between poll calls", + ) + watch.set_defaults(handler=cmd_watch) + + demo = subparsers.add_parser("demo", help="Run a check/lease/revoke smoke demo") + _add_agent_arg(demo) + demo.set_defaults(handler=cmd_demo) + + for method in SUBCOMMAND_TO_METHOD.values(): + alias = subparsers.add_parser(method, help=f"Alias for {method}") + if method == "capability.check": + alias.add_argument("--context", default="{}") + _add_agent_arg(alias) + alias.set_defaults(handler=cmd_check) + elif method == "capability.lease": + alias.add_argument("--ref", required=True) + alias.add_argument("--justification", required=True) + _add_agent_arg(alias) + alias.set_defaults(handler=cmd_lease) + elif method == "capability.revoke": + alias.add_argument("--ref", required=True) + alias.add_argument("--reason", required=True) + alias.set_defaults(handler=cmd_revoke) + elif method == "capability.watch.poll": + alias.add_argument( + "--interval", + type=float, + default=float(os.environ.get("CAPABILITY_WATCH_INTERVAL", "5")), + ) + alias.set_defaults(handler=cmd_watch) + + return parser + + +def main(argv: list[str] | None = None) -> None: + args = _build_parser().parse_args(argv) + args.handler(args) + + +if __name__ == "__main__": + main() diff --git a/cli/lib/capability.bash b/cli/lib/capability.bash new file mode 100644 index 00000000..4ca8f165 --- /dev/null +++ b/cli/lib/capability.bash @@ -0,0 +1,225 @@ +#!/usr/bin/env bash + +# shellcheck source=constants.bash +source "${BASH_SOURCE%/*}/constants.bash" +# shellcheck source=path.bash +source "${BASH_SOURCE%/*}/path.bash" +# shellcheck source=logging.bash +source "${BASH_SOURCE%/*}/logging.bash" +# shellcheck source=require.bash +source "${BASH_SOURCE%/*}/require.bash" + +# Runs a command inside the capability-runtime sidecar via docker compose exec. +# Args: +# $1 - Repository root (project directory) +# $@ - Arguments forwarded to capability_runtime.cli inside the container +capability_compose_exec() { + local project_dir=$1 + shift + + require docker + + local compose_file="$project_dir/.devcontainer/compose-all.yml" + if [[ ! -f "$compose_file" ]]; then + echo "No compose-all.yml found at $compose_file" | error + echo "Run sandcat init --netbird --capability in this project first." >&2 + return 1 + fi + + docker compose -f "$compose_file" \ + exec -T capability-runtime python -m capability_runtime.cli "$@" +} + +# Resolves the repository root for capability operator commands. +capability_project_dir() { + find_repo_root +} + +# Operator check: capability.check over the admin RPC surface. +# Args: +# --agent Agent identity (default: devcontainer-agent) +# --context Capability context JSON object (default: {}) +capability_check() { + local agent_id="devcontainer-agent" + local context='{}' + + while [[ $# -gt 0 ]]; do + case $1 in + --agent) + if [[ $# -lt 2 || "$2" == --* ]]; then + echo "Option --agent requires a value" | error + return 1 + fi + agent_id="$2" + shift 2 + ;; + --context) + if [[ $# -lt 2 || "$2" == --* ]]; then + echo "Option --context requires a value" | error + return 1 + fi + context="$2" + shift 2 + ;; + *) + echo "Unknown option: $1" | error + return 1 + ;; + esac + done + + local project_dir + project_dir=$(capability_project_dir) || return 1 + + capability_compose_exec "$project_dir" capability.check \ + --agent-id "$agent_id" \ + --context "$context" +} + +# Operator lease: capability.lease over the admin RPC surface. +# Args: +# --ref Capability reference (required) +# --justification Lease justification (required) +# --agent Agent identity (default: devcontainer-agent) +capability_lease() { + local agent_id="devcontainer-agent" + local capability_ref="" + local justification="" + + while [[ $# -gt 0 ]]; do + case $1 in + --ref) + if [[ $# -lt 2 || "$2" == --* ]]; then + echo "Option --ref requires a value" | error + return 1 + fi + capability_ref="$2" + shift 2 + ;; + --justification) + if [[ $# -lt 2 || "$2" == --* ]]; then + echo "Option --justification requires a value" | error + return 1 + fi + justification="$2" + shift 2 + ;; + --agent) + if [[ $# -lt 2 || "$2" == --* ]]; then + echo "Option --agent requires a value" | error + return 1 + fi + agent_id="$2" + shift 2 + ;; + *) + echo "Unknown option: $1" | error + return 1 + ;; + esac + done + + if [[ -z "$capability_ref" ]]; then + echo "Missing required option: --ref" | error + return 1 + fi + if [[ -z "$justification" ]]; then + echo "Missing required option: --justification" | error + return 1 + fi + + local project_dir + project_dir=$(capability_project_dir) || return 1 + + capability_compose_exec "$project_dir" capability.lease \ + --agent-id "$agent_id" \ + --ref "$capability_ref" \ + --justification "$justification" +} + +# Operator revoke: capability.revoke over the admin RPC surface. +# Args: +# --ref Revoke target (required) +# --reason Revocation reason (required) +capability_revoke() { + local target="" + local reason="" + + while [[ $# -gt 0 ]]; do + case $1 in + --ref) + if [[ $# -lt 2 || "$2" == --* ]]; then + echo "Option --ref requires a value" | error + return 1 + fi + target="$2" + shift 2 + ;; + --reason) + if [[ $# -lt 2 || "$2" == --* ]]; then + echo "Option --reason requires a value" | error + return 1 + fi + reason="$2" + shift 2 + ;; + *) + echo "Unknown option: $1" | error + return 1 + ;; + esac + done + + if [[ -z "$target" ]]; then + echo "Missing required option: --ref" | error + return 1 + fi + if [[ -z "$reason" ]]; then + echo "Missing required option: --reason" | error + return 1 + fi + + local project_dir + project_dir=$(capability_project_dir) || return 1 + + capability_compose_exec "$project_dir" capability.revoke \ + --ref "$target" \ + --reason "$reason" +} + +# Operator watch: foreground capability.watch.poll loop with JSON log lines. +capability_watch() { + local project_dir + project_dir=$(capability_project_dir) || return 1 + + capability_compose_exec "$project_dir" watch +} + +# Operator demo: quick check/lease/revoke smoke against the sidecar. +# Args: +# --agent Agent identity (default: devcontainer-agent) +capability_demo() { + local agent_id="devcontainer-agent" + + while [[ $# -gt 0 ]]; do + case $1 in + --agent) + if [[ $# -lt 2 || "$2" == --* ]]; then + echo "Option --agent requires a value" | error + return 1 + fi + agent_id="$2" + shift 2 + ;; + *) + echo "Unknown option: $1" | error + return 1 + ;; + esac + done + + local project_dir + project_dir=$(capability_project_dir) || return 1 + + capability_compose_exec "$project_dir" demo --agent-id "$agent_id" +} diff --git a/cli/libexec/capability/_ b/cli/libexec/capability/_ new file mode 100755 index 00000000..cadcc98b --- /dev/null +++ b/cli/libexec/capability/_ @@ -0,0 +1,7 @@ +#!/usr/bin/env bash + +# Catch subcommands (check, lease, revoke, watch, demo) and forward them to the +# capability dispatcher. Without this, `sandcat capability check` looks for +# libexec/capability/check. + +exec capability "$@" diff --git a/cli/libexec/capability/capability b/cli/libexec/capability/capability new file mode 100755 index 00000000..d2de8427 --- /dev/null +++ b/cli/libexec/capability/capability @@ -0,0 +1,65 @@ +#!/usr/bin/env bash +set -euo pipefail + +# shellcheck source=../../lib/logging.bash +source "$SCT_LIBDIR/logging.bash" +# shellcheck source=../../lib/capability.bash +source "$SCT_LIBDIR/capability.bash" + +usage() { + cat <<'EOF' +Usage: sandcat capability [options] + +Subcommands: + check --context '{}' [--agent ] Show current capability bundle + lease --ref --justification [--agent ] + revoke --ref --reason + watch Foreground route-watcher poll loop + demo [--agent ] Check / lease / revoke smoke demo +EOF +} + +cmd_check() { + capability_check "$@" +} + +cmd_lease() { + capability_lease "$@" +} + +cmd_revoke() { + capability_revoke "$@" +} + +cmd_watch() { + capability_watch "$@" +} + +cmd_demo() { + capability_demo "$@" +} + +main() { + local subcmd="${1:-}" + shift || true + case "$subcmd" in + check) cmd_check "$@" ;; + lease) cmd_lease "$@" ;; + revoke) cmd_revoke "$@" ;; + watch) cmd_watch "$@" ;; + demo) cmd_demo "$@" ;; + "") + usage + return 1 + ;; + *) + echo "Unknown subcommand: $subcmd" | error + usage + return 1 + ;; + esac +} + +if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then + main "$@" +fi diff --git a/cli/test/capability/capability.bats b/cli/test/capability/capability.bats new file mode 100644 index 00000000..a85b3b5d --- /dev/null +++ b/cli/test/capability/capability.bats @@ -0,0 +1,79 @@ +#!/usr/bin/env bats + +setup() { + load test_helper + # shellcheck source=../../lib/capability.bash + source "$SCT_LIBDIR/capability.bash" + + CAPABILITY_CMD="$SCT_LIBEXECDIR/capability/capability" + TEST_REPO="$BATS_TEST_TMPDIR/project" + mkdir -p "$TEST_REPO/.devcontainer" + printf 'services: {}\n' >"$TEST_REPO/.devcontainer/compose-all.yml" + cd "$TEST_REPO" +} + +teardown() { + unstub_all +} + +@test "capability check execs into capability-runtime container" { + capability_compose_exec() { echo "EXEC $*"; } + export -f capability_compose_exec + run capability_check --agent devcontainer-agent + assert_success + assert_output --partial "capability.check" +} + +@test "capability check forwards agent and context to compose exec" { + capability_compose_exec() { echo "EXEC $*"; } + export -f capability_compose_exec + run capability_check --agent devcontainer-agent --context '{"task":"x"}' + assert_success + assert_output --partial "--agent-id devcontainer-agent" + assert_output --partial '--context {"task":"x"}' +} + +@test "capability lease requires --ref and --justification" { + run bash "$CAPABILITY_CMD" lease --ref cap-reach-api + assert_failure + assert_output --partial "justification" +} + +@test "capability lease execs into capability-runtime container" { + capability_compose_exec() { echo "EXEC $*"; } + export -f capability_compose_exec + run capability_lease --ref cap-reach-api --justification "smoke test" + assert_success + assert_output --partial "capability.lease" +} + +@test "capability revoke execs into capability-runtime container" { + capability_compose_exec() { echo "EXEC $*"; } + export -f capability_compose_exec + run capability_revoke --ref cap-reach-api --reason policy + assert_success + assert_output --partial "capability.revoke" +} + +@test "capability watch execs into capability-runtime container" { + capability_compose_exec() { echo "EXEC $*"; } + export -f capability_compose_exec + run capability_watch + assert_success + assert_output --partial "watch" +} + +@test "capability with unknown subcommand prints usage and fails" { + run bash "$CAPABILITY_CMD" bogus + assert_failure + assert_output --partial "Usage" +} + +@test "sandcat capability check routes subcommand through module dispatcher" { + printf 'test\n' > "$SCT_ROOT/.version" + stub docker \ + "compose -f $TEST_REPO/.devcontainer/compose-all.yml exec -T capability-runtime python -m capability_runtime.cli capability.check --agent-id devcontainer-agent --context {} : echo 'EXEC capability.check'" + run bash "$SCT_ROOT/bin/sandcat" capability check --agent devcontainer-agent + assert_success + assert_output --partial "capability.check" +} diff --git a/cli/test/capability/test_helper.bash b/cli/test/capability/test_helper.bash new file mode 100644 index 00000000..e5c389a0 --- /dev/null +++ b/cli/test/capability/test_helper.bash @@ -0,0 +1,21 @@ +#!/bin/bash + +bats_require_minimum_version 1.5.0 + +if shopt -s compat32 2>/dev/null; then + export BASH_COMPAT=3.2 +fi +set -uo pipefail +export SHELLOPTS + +SCT_ROOT="$BATS_TEST_DIRNAME/../.." +BATS_LIB_PATH="$SCT_ROOT/support":${BATS_LIB_PATH-} + +bats_load_library bats-ext +bats_load_library bats-support +bats_load_library bats-assert +bats_load_library bats-mock-ext + +export SCT_ROOT +export SCT_LIBDIR="$SCT_ROOT/lib" +export SCT_LIBEXECDIR="$SCT_ROOT/libexec" From e865645715711f6efd9ad975b33ca1bb15fa4051 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 29 Jun 2026 10:24:46 +0000 Subject: [PATCH 044/138] docs: document Phase 3b capability sidecar and MCP bridge --- CONTEXT.md | 85 ++++++++++++++++++ capability-runtime/README.md | 76 +++++++++++++++- .../src/capability_runtime/agent_loop.py | 29 +++--- .../src/capability_runtime/runtime.py | 2 + capability-runtime/tests/test_rpc_unix.py | 10 ++- cli/README.md | 88 +++++++++++++++++++ 6 files changed, 269 insertions(+), 21 deletions(-) create mode 100644 CONTEXT.md diff --git a/CONTEXT.md b/CONTEXT.md new file mode 100644 index 00000000..209b6023 --- /dev/null +++ b/CONTEXT.md @@ -0,0 +1,85 @@ +# Sandcat Capability Networking + +Capability-oriented networking for autonomous agents in sandcat devcontainers. Agents reason only from their current `CapabilityBundle`; reachability to network endpoints is a leased/revocable capability, not ambient permission. + +## Language + +**Capability Runtime**: +The authoritative control plane that owns catalog state, leases, revocations, and observability. Issues `CapabilityBundle` snapshots to agents. + +_Avoid_: capability library, agent runtime (when meaning the control plane) + +**Capability Bundle**: +The typed snapshot of what an agent may use right now — tools, network routes, budgets — returned by `check_current_capabilities`. + +_Avoid_: permission set, tool list + +**Phase 3 (Networking Realization)**: +Python `capability-runtime/` bridge proving `Reachability == Capability`: logical revoke removes NetBird peer/route; physical route disappearance reconciles back into runtime state. + +_Avoid_: NetBird CLI work (that is a separate sandcat track) + +**Phase 3b (Sandcat Integration)**: +Compose sidecar wiring: `capability-runtime` service, Capability MCP for agents, `sandcat capability` for operators. **Implemented** — plan: `docs/superpowers/plans/2026-06-23-capability-sandcat-phase3b.md`. + +_Avoid_: Phase 4 + +**Phase 3c (Dynamic L7 Policy)**: +mitmproxy ↔ capability-runtime loop: dynamic HTTP allowlist from bundle, L7 execution events, quota-driven revoke. **DRAFT spec:** `docs/superpowers/specs/2026-06-23-capability-dynamic-l7-policy-phase3c-design.md`. + +_Avoid_: conflating with Phase 3b sidecar work + +**Phase 4 (Comparative Evaluation)**: +Controlled experiments baseline vs treatment; metrics and ablations. **DRAFT spec:** `docs/superpowers/specs/2026-06-23-capability-comparative-evaluation-phase4-design.md`. Does not implement new enforcement — measures Phases 0–3c. + +**Physical Revocation**: +Removing reachability by deleting a NetBird peer or route, causing `wg-client` to drop the WireGuard route via `wg syncconf`. + +_Avoid_: network kill switch (too vague) + +**Logical Revocation**: +Runtime catalog transition to `Revoked` — capability disappears from the bundle regardless of physical path. + +**Network Binding**: +The link between a network capability and concrete NetBird identifiers (`peer_id`, `network` CIDR, optional `route_id`). + +## Relationships + +- A **Capability Bundle** includes zero or more network capabilities when their lifecycle state is Visible or Leased +- Each network capability has exactly one **Network Binding** +- **Logical Revocation** triggers **Physical Revocation** via `NetBirdRevocationBackend` +- **Physical Revocation** outside the runtime triggers **Logical Revocation** via `RouteDisappearanceWatcher` + +## Example dialogue + +> **Dev:** "When the agent's `reach_api` lease expires, does the route disappear?" +> **Domain expert:** "Only if revocation or expiry also drives **Logical Revocation**, which then calls NetBird to remove the peer/route. Expiry alone today returns the capability to Visible or Declared per base policy — physical removal is explicit." + +**Capability Control Plane**: +The trusted process that owns `CapabilityRuntime` state, NetBird revocation credentials, and the route watcher. Runs in a dedicated compose sidecar (`capability-runtime` service), not inside the agent or `wg-client`. + +_Avoid_: capability daemon (when meaning the agent), in-process runtime (PoC only) + +**Capability RPC**: +Internal JSON-RPC 2.0 dispatcher inside the capability sidecar. Used for operator CLI (`sandcat capability` via exec) and tests. Not the primary agent-facing protocol. + +_Avoid_: exposing revoke/register on this surface + +**Capability MCP**: +MCP server exposed by the capability sidecar to the agent. Meta-tools: `capability_check`, `capability_lease`, `capability_discover`. Workload tools (`write_note`, `create_pr`, etc.) remain separate MCP servers gated by bundle visibility. + +_Avoid_: using MCP for NetBird admin, conflating control-plane MCP with workload MCP + +**Agent Identity**: +Opaque runtime-assigned id for the single agent in a devcontainer. Fixed as `SANDCAT_AGENT_ID` by compose; injected by the MCP bridge; not accepted from agent-supplied parameters. + +_Avoid_: client-provided agent_id, multi-agent per container (Phase 3b) + +> **Dev:** "Should the agent call the runtime over REST?" +> **Domain expert:** "No. The agent speaks **Capability MCP** for check/lease/discover. Work tools speak their own MCP servers. REST is only for NetBird management behind the trusted sidecar." + +## Flagged ambiguities + +- NetBird **implementation** runs inside `wg-client` (`wt0` overlay); the original plan described a separate `netbird` sync sidecar for `wg0` — these are different layers. +- Phase 3b plan is formal and **implemented** (Tasks 1–8): `docs/superpowers/plans/2026-06-23-capability-sandcat-phase3b.md`. +- Phase 3c and Phase 4 are **DRAFT specs** only (not approved for implementation): `docs/superpowers/specs/2026-06-23-capability-dynamic-l7-policy-phase3c-design.md`, `docs/superpowers/specs/2026-06-23-capability-comparative-evaluation-phase4-design.md`. diff --git a/capability-runtime/README.md b/capability-runtime/README.md index 97acd230..85007fde 100644 --- a/capability-runtime/README.md +++ b/capability-runtime/README.md @@ -27,6 +27,16 @@ Phase 3 realizes this thesis by binding each network capability to a NetBird pee - PoC 3: `reach_api` network route lifecycle demo - Observability events with `physical_revocation` and `physical_trigger` flags +**In scope (Phase 3b — sandcat sidecar integration):** + +- `capability-runtime` compose sidecar: daemon, dual Unix socket RPC, route watcher, settings-backed `RestNetBirdClient` +- **Agent surface** (`agent.sock`): `capability.check`, `capability.lease`, `capability.discover` only +- **Admin surface** (`admin.sock`): agent methods plus `capability.revoke`, `capability.watch.poll` +- **MCP bridge** (`capability-mcp-bridge`): stdio MCP in agent container → JSON-RPC over `agent.sock` +- Fixed `SANDCAT_AGENT_ID` per devcontainer; bridge injects identity — agent-supplied `agent_id` ignored +- Operator CLI: `sandcat capability` via `docker compose exec capability-runtime` +- Catalog loaded at sidecar startup from `CAPABILITY_CATALOG_JSON` — no `register_*` over RPC + **Out of scope / known limitations:** - Token budget enforcement (`token_budget` is stored but not decremented) @@ -34,7 +44,7 @@ Phase 3 realizes this thesis by binding each network capability to a NetBird pee - `TaskContext`-driven visibility rules - Non-tool capability kinds (`rules`, `skills`, etc.) in bundles - Multi-agent leasing on the same capability (global `LEASED` catalog state) -- Production sandcat wiring (`RestNetBirdClient` tokens, `sandcat capability` subcommand) — injectable client only in PoC +- MCP workload gateway, NetBird ACL/reverse-proxy bindings, host-published HTTP RPC (Phase 3c+) ## NetBird bridge (logical ↔ physical) @@ -73,11 +83,64 @@ RouteDisappearanceWatcher.poll_once() The watcher ensures the runtime catalog stays consistent when reachability disappears outside the runtime — the spec's physical `Revoked` trigger (Phase 3 / idea7). -## Security (Phase 0+1) +## Sidecar architecture (Phase 3b) + +``` +┌──────────────────────── agent container ────────────────────────┐ +│ Cursor / Claude agent │ +│ │ stdio MCP │ +│ ▼ │ +│ capability-mcp-bridge ──JSON-RPC──► agent.sock (ro volume) │ +│ (SANDCAT_AGENT_ID injected) │ +└─────────────────────────────────────────────────────────────────┘ + │ + capability-socket volume + │ +┌──────────────────────── capability-runtime sidecar ───────────┐ +│ CapabilityRuntime + RouteDisappearanceWatcher │ +│ RestNetBirdClient ← settings.json (netbird_api_token) │ +│ │ │ │ +│ agent.sock (check/lease/discover) admin.sock (revoke/watch) │ +└─────────────────────────────────────────────────────────────────┘ + │ + operator: sandcat capability + (docker compose exec → admin.sock) +``` + +The agent container mounts `capability-socket:/run/sandcat/capability:ro` and receives `CAPABILITY_AGENT_SOCKET` plus `SANDCAT_AGENT_ID`. It does **not** mount `admin.sock` write access, does **not** receive `NB_API_TOKEN`, and does **not** import `CapabilityRuntime`. NetBird credentials live only in the sidecar via read-only `settings.json`. + +Cursor MCP config (`.cursor/mcp.json` in devcontainer): + +```json +{ + "mcpServers": { + "sandcat-capability": { + "command": "capability-mcp-bridge", + "args": [] + } + } +} +``` + +MCP meta-tools (`capability_check`, `capability_lease`, `capability_discover`) map to the agent RPC surface. Workload tools (`write_note`, `create_pr`, etc.) remain separate MCP servers gated by bundle visibility. + +## Security (Phase 0+1 + 3b) Mutating APIs require `caller` to match the lease-bound `agent_id` (`CallerIdentityMismatch` on impersonation). Leased tool execution is serialized per `lease_id` in `AgentExecutionLoop` to prevent quota races. Observability events are runtime-authored (`source: runtime`) or agent-loop-bound (`source: agent_loop` with enforced `agent_id`); public `emit_*` APIs are not exposed on `CapabilityRuntime`. -**Not yet addressed:** cryptographic trace signing, network-authenticated control plane, cross-process trust boundaries. +**Phase 3b boundary:** + +| Surface | Socket | Methods | Who | +|---------|--------|---------|-----| +| Agent | `agent.sock` | check, lease, discover | MCP bridge in agent container | +| Admin | `admin.sock` | check, lease, discover, revoke, watch.poll | `sandcat capability` operator CLI | + +- `capability.revoke` requires `caller=operator` on the runtime — agents cannot self-revoke or revoke others +- RPC dispatcher allowlists reject unknown methods and admin-only methods on the agent socket +- `agent_id` in RPC/MCP params is overwritten with `SANDCAT_AGENT_ID` on the agent surface +- Catalog registration happens at sidecar startup only — not over RPC + +**Not yet addressed:** cryptographic trace signing, network-authenticated control plane, cross-process cryptographic auth (Unix permissions + container split sufficient for 3b). ## Quick start @@ -104,3 +167,10 @@ PYTHONPATH=src:. python poc/network_route_demo.py | `observability.py` | JSONL trace + replay | | `agent_loop.py` | Check-then-act harness | | `mcp_adapter.py` | Transport-agnostic MCP tool wrapper | +| `daemon.py` | Sidecar main: runtime, watcher, dual Unix sockets | +| `rpc/dispatcher.py` | JSON-RPC routing with agent/admin allowlists | +| `rpc/transports/unix.py` | AF_UNIX JSON-RPC server/client | +| `mcp/server.py` | Minimal MCP meta-tools server | +| `mcp/bridge.py` | Stdio MCP ↔ agent.sock forwarder | +| `settings.py` | Sandcat settings JSON layers for NetBird tokens | +| `cli.py` | Operator admin-socket CLI (used by `sandcat capability`) | diff --git a/capability-runtime/src/capability_runtime/agent_loop.py b/capability-runtime/src/capability_runtime/agent_loop.py index 8cd06d2b..2185bd29 100644 --- a/capability-runtime/src/capability_runtime/agent_loop.py +++ b/capability-runtime/src/capability_runtime/agent_loop.py @@ -16,14 +16,15 @@ class AgentExecutionLoop: def __init__(self, runtime: CapabilityRuntime): self._runtime = runtime - self._lease_locks: dict[LeaseId, threading.Lock] = {} - self._lease_locks_guard = threading.Lock() + self._tool_locks: dict[tuple[str, str], threading.Lock] = {} + self._tool_locks_guard = threading.Lock() - def _lock_for_lease(self, lease_id: LeaseId) -> threading.Lock: - with self._lease_locks_guard: - if lease_id not in self._lease_locks: - self._lease_locks[lease_id] = threading.Lock() - return self._lease_locks[lease_id] + def _lock_for_tool(self, agent_id: AgentIdentity, tool_name: str) -> threading.Lock: + key = (agent_id.value, tool_name) + with self._tool_locks_guard: + if key not in self._tool_locks: + self._tool_locks[key] = threading.Lock() + return self._tool_locks[key] def run_step( self, @@ -34,14 +35,11 @@ def run_step( now: datetime | None = None, ) -> Any: effective_now = now or datetime.now(timezone.utc) + lock = self._lock_for_tool(agent_id, tool_name) - bundle = self._runtime.check_current_capabilities(agent_id, context) - lease_id = _lease_id_for_tool(bundle.tools, tool_name) - lock = self._lock_for_lease(lease_id) if lease_id is not None else None - - if lock is not None: - lock.acquire() - try: + with lock: + bundle = self._runtime.check_current_capabilities(agent_id, context) + lease_id = _lease_id_for_tool(bundle.tools, tool_name) self._runtime.enforce_action( agent_id, tool_name, bundle.version, effective_now ) @@ -51,9 +49,6 @@ def run_step( agent_id, agent_id, lease_id, effective_now ) return result - finally: - if lock is not None: - lock.release() def _lease_id_for_tool(tools, tool_name: str) -> LeaseId | None: diff --git a/capability-runtime/src/capability_runtime/runtime.py b/capability-runtime/src/capability_runtime/runtime.py index 827ba712..8c1ccfab 100644 --- a/capability-runtime/src/capability_runtime/runtime.py +++ b/capability-runtime/src/capability_runtime/runtime.py @@ -363,6 +363,8 @@ def record_action( if lease is not None: self.catalog.set_state(lease.capability_ref, LifecycleState.EXPIRED) self.revocation_manager.revoke_by_lease(lease_id, "quota exhausted") + self._bundle_version += 1 + self._current_bundles[agent_id] = self._bundle_version def enforce_action( self, agent_id: AgentIdentity, tool_name: str, bundle_version: int, now: datetime diff --git a/capability-runtime/tests/test_rpc_unix.py b/capability-runtime/tests/test_rpc_unix.py index d75d7d4f..ccef30de 100644 --- a/capability-runtime/tests/test_rpc_unix.py +++ b/capability-runtime/tests/test_rpc_unix.py @@ -24,10 +24,18 @@ def mock_dispatcher(): def _wait_for_socket(path: Path, timeout: float = 2.0) -> None: + import socket + deadline = time.time() + timeout while time.time() < deadline: if path.exists(): - return + try: + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock: + sock.settimeout(0.1) + sock.connect(str(path)) + return + except (ConnectionRefusedError, OSError): + pass time.sleep(0.01) raise TimeoutError(f"socket not ready: {path}") diff --git a/cli/README.md b/cli/README.md index ac035e56..9ce04b1c 100644 --- a/cli/README.md +++ b/cli/README.md @@ -27,6 +27,10 @@ Options: a second interface `wt0` for the NetBird overlay mesh. `wg0` (the mitmproxy inspection tunnel) is untouched. Seeds `netbird_enrollment_key` and `netbird_api_token` in `~/.config/sandcat/settings.json`. +- `--capability` - Enable the capability-runtime sidecar (requires `--netbird`). + Adds a `capability-runtime` compose service, mounts a shared Unix socket volume + into the agent container, and installs `capability-mcp-bridge` for Cursor MCP. + NetBird API credentials stay in the sidecar — they are not injected into the agent. - `--netbird-server` - NetBird management server mode (requires `--netbird`): `cloud` | `new` | `quickstart` | ``. `new` provisions a local localhost template; `quickstart` prints the official NetBird install command for @@ -55,6 +59,9 @@ sandcat init --agent claude --ide vscode --secret-provider protonpass --name myp # With NetBird dynamic WireGuard sandcat init --agent claude --ide vscode --netbird --name myproject + +# With NetBird + capability sidecar (reachability == capability) +sandcat init --agent cursor --ide vscode --netbird --capability --name myproject ``` #### Proton Pass setup (scoped Personal Access Token) @@ -426,6 +433,87 @@ sandcat netbird route add --network 10.8.0.0/24 --peer-id sandcat netbird route remove --route-id ``` +## Capability sidecar (Phase 3b) + +When initialized with `--netbird --capability`, sandcat deploys a trusted +`capability-runtime` compose sidecar alongside the agent. The sidecar owns +`CapabilityRuntime` state, NetBird revocation credentials, and the route watcher. +The agent container talks to the runtime only through a thin MCP bridge over a +read-only Unix socket — it never receives `NB_API_TOKEN` or direct NetBird access. + +``` +Agent (Cursor/Claude) ──stdio MCP──► capability-mcp-bridge ──► agent.sock +Operator (host) ──compose exec──► admin.sock +Sidecar ──RestNetBirdClient──► NetBird management API +``` + +### Setup + +Requires NetBird (`--netbird`) and both credentials in user settings (see +[Dynamic networking](#dynamic-networking-netbird)). The sidecar reads +`netbird_api_token` and `netbird_management_url` from mounted `settings.json`; +the agent container does not receive these values. + +```bash +sandcat init --agent cursor --ide vscode --netbird --capability --name myproject +sandcat compose up -d +``` + +### Cursor MCP config + +Add to `.cursor/mcp.json` in the devcontainer (the bridge is installed at +`/usr/local/bin/capability-mcp-bridge`): + +```json +{ + "mcpServers": { + "sandcat-capability": { + "command": "capability-mcp-bridge", + "args": [] + } + } +} +``` + +MCP meta-tools: `capability_check`, `capability_lease`, `capability_discover`. +Workload tools remain on their own MCP servers and appear in the bundle only +when leased or visible. + +### Operator commands + +`sandcat capability` runs inside the `capability-runtime` container via +`docker compose exec` — no host-published ports. + +```bash +# Show current capability bundle +sandcat capability check --context '{}' + +# Lease a network capability (triggers NetBird peer/route via sidecar) +sandcat capability lease --ref cap-reach-api --justification "need API access" + +# Revoke (operator-only; removes NetBird peer/route) +sandcat capability revoke --ref cap-reach-api --reason policy + +# Foreground route-watcher poll loop (debugging) +sandcat capability watch + +# End-to-end smoke demo +sandcat capability demo +``` + +### Security boundary + +| Path | Socket | Who | Can revoke? | +|------|--------|-----|-------------| +| Agent MCP bridge | `agent.sock` (ro mount) | Agent in container | No | +| `sandcat capability` | `admin.sock` | Operator on host | Yes | + +- `admin.sock` is not mounted in the agent service +- Agent RPC surface rejects `capability.revoke` and unknown methods +- `SANDCAT_AGENT_ID` is fixed per devcontainer and injected by the bridge; + agent-supplied `agent_id` parameters are ignored +- Catalog is loaded at sidecar startup from config — not registerable over RPC + ## Directory Structure Each module is contained in its own directory under `cli/libexec/`. From eeadf0de1d9c3eb88d0443482056fb12a3f6f4bf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Mon, 29 Jun 2026 13:13:56 +0000 Subject: [PATCH 045/138] fix(cli): mount capability socket outside read-only wg-runtime path Agent already mounts wg-runtime at /run/sandcat:ro; nesting capability-socket there fails at container start. Use /run/sandcat-capability instead. --- capability-runtime/README.md | 2 +- .../src/capability_runtime/cli.py | 2 +- .../src/capability_runtime/daemon.py | 4 ++-- .../src/capability_runtime/mcp/bridge.py | 2 +- cli/lib/composefile.bash | 22 +++++++++++++++---- cli/templates/devcontainer/Dockerfile.app | 2 ++ .../sandcat/Dockerfile.capability-runtime | 1 + .../sandcat/compose-capability.yml | 7 +++--- cli/test/composefile/capability.bats | 6 +++++ 9 files changed, 36 insertions(+), 12 deletions(-) diff --git a/capability-runtime/README.md b/capability-runtime/README.md index 85007fde..80fc72f6 100644 --- a/capability-runtime/README.md +++ b/capability-runtime/README.md @@ -107,7 +107,7 @@ The watcher ensures the runtime catalog stays consistent when reachability disap (docker compose exec → admin.sock) ``` -The agent container mounts `capability-socket:/run/sandcat/capability:ro` and receives `CAPABILITY_AGENT_SOCKET` plus `SANDCAT_AGENT_ID`. It does **not** mount `admin.sock` write access, does **not** receive `NB_API_TOKEN`, and does **not** import `CapabilityRuntime`. NetBird credentials live only in the sidecar via read-only `settings.json`. +The agent container mounts `capability-socket:/run/sandcat-capability:ro` (separate from read-only `wg-runtime:/run/sandcat`) and receives `CAPABILITY_AGENT_SOCKET` plus `SANDCAT_AGENT_ID`. It does **not** mount `admin.sock` write access, does **not** receive `NB_API_TOKEN`, and does **not** import `CapabilityRuntime`. NetBird credentials live only in the sidecar via read-only `settings.json`. Cursor MCP config (`.cursor/mcp.json` in devcontainer): diff --git a/capability-runtime/src/capability_runtime/cli.py b/capability-runtime/src/capability_runtime/cli.py index f4a147af..406b46ee 100644 --- a/capability-runtime/src/capability_runtime/cli.py +++ b/capability-runtime/src/capability_runtime/cli.py @@ -12,7 +12,7 @@ from capability_runtime.rpc.transports.unix import UnixRpcClient DEFAULT_ADMIN_SOCKET = Path( - os.environ.get("CAPABILITY_ADMIN_SOCKET", "/run/sandcat/capability/admin.sock") + os.environ.get("CAPABILITY_ADMIN_SOCKET", "/run/sandcat-capability/admin.sock") ) SUBCOMMAND_TO_METHOD = { diff --git a/capability-runtime/src/capability_runtime/daemon.py b/capability-runtime/src/capability_runtime/daemon.py index 488ab5fd..96e79ac4 100644 --- a/capability-runtime/src/capability_runtime/daemon.py +++ b/capability-runtime/src/capability_runtime/daemon.py @@ -42,13 +42,13 @@ def from_env(cls) -> DaemonConfig: agent_socket=Path( os.environ.get( "CAPABILITY_AGENT_SOCKET", - "/run/sandcat/capability/agent.sock", + "/run/sandcat-capability/agent.sock", ) ), admin_socket=Path( os.environ.get( "CAPABILITY_ADMIN_SOCKET", - "/run/sandcat/capability/admin.sock", + "/run/sandcat-capability/admin.sock", ) ), trace_file=Path( diff --git a/capability-runtime/src/capability_runtime/mcp/bridge.py b/capability-runtime/src/capability_runtime/mcp/bridge.py index 6e1e18be..f6d8c49c 100644 --- a/capability-runtime/src/capability_runtime/mcp/bridge.py +++ b/capability-runtime/src/capability_runtime/mcp/bridge.py @@ -11,7 +11,7 @@ from capability_runtime.mcp.server import McpServer from capability_runtime.rpc.transports.stdio_bridge import BridgeRpcClient -DEFAULT_AGENT_SOCKET = "/run/sandcat/capability/agent.sock" +DEFAULT_AGENT_SOCKET = "/run/sandcat-capability/agent.sock" def run_mcp_bridge( diff --git a/cli/lib/composefile.bash b/cli/lib/composefile.bash index 8a0be78b..4cd2b975 100644 --- a/cli/lib/composefile.bash +++ b/cli/lib/composefile.bash @@ -680,15 +680,29 @@ enable_capability() { yq -i '.include += [{"path": "sandcat/compose-capability.yml"}]' "$compose_file" fi + # Legacy path nested under wg-runtime:/run/sandcat:ro — Docker cannot mount there. + yq -i ' + .services.agent.volumes = ( + (.services.agent.volumes // []) + | map(select(. != "capability-socket:/run/sandcat/capability:ro")) + ) + ' "$compose_file" + yq -i ' + .services.agent.environment = ( + (.services.agent.environment // []) + | map(select(. != "CAPABILITY_AGENT_SOCKET=/run/sandcat/capability/agent.sock")) + ) + ' "$compose_file" + local has_volume - has_volume=$(yq '[.services.agent.volumes[]? | select(. == "capability-socket:/run/sandcat/capability:ro")] | length' "$compose_file") + has_volume=$(yq '[.services.agent.volumes[]? | select(. == "capability-socket:/run/sandcat-capability:ro")] | length' "$compose_file") if [[ "$has_volume" -eq 0 ]]; then - yq -i '.services.agent.volumes += ["capability-socket:/run/sandcat/capability:ro"]' "$compose_file" + yq -i '.services.agent.volumes += ["capability-socket:/run/sandcat-capability:ro"]' "$compose_file" fi local has_agent_id has_socket has_agent_id=$(yq '[.services.agent.environment[]? | select(. == "SANDCAT_AGENT_ID=devcontainer-agent")] | length' "$compose_file") - has_socket=$(yq '[.services.agent.environment[]? | select(. == "CAPABILITY_AGENT_SOCKET=/run/sandcat/capability/agent.sock")] | length' "$compose_file") + has_socket=$(yq '[.services.agent.environment[]? | select(. == "CAPABILITY_AGENT_SOCKET=/run/sandcat-capability/agent.sock")] | length' "$compose_file") if [[ "$has_agent_id" -eq 0 || "$has_socket" -eq 0 ]]; then local env_additions=() @@ -696,7 +710,7 @@ enable_capability() { env_additions+=("SANDCAT_AGENT_ID=devcontainer-agent") fi if [[ "$has_socket" -eq 0 ]]; then - env_additions+=("CAPABILITY_AGENT_SOCKET=/run/sandcat/capability/agent.sock") + env_additions+=("CAPABILITY_AGENT_SOCKET=/run/sandcat-capability/agent.sock") fi local entry yq_array="" for entry in "${env_additions[@]}"; do diff --git a/cli/templates/devcontainer/Dockerfile.app b/cli/templates/devcontainer/Dockerfile.app index e90ee505..268e3947 100644 --- a/cli/templates/devcontainer/Dockerfile.app +++ b/cli/templates/devcontainer/Dockerfile.app @@ -32,6 +32,8 @@ COPY --chmod=755 sandcat/scripts/app-user-init.sh /usr/local/bin/app-user-init.s COPY --chmod=644 sandcat/scripts/java-env.sh /etc/profile.d/sandcat-java.sh COPY --chmod=755 sandcat/scripts/capability-mcp-bridge.sh /usr/local/bin/capability-mcp-bridge COPY --chown=vscode:vscode sandcat/tmux.conf /home/vscode/.tmux.conf +# Mountpoint for capability-socket volume (must not nest under wg-runtime:/run/sandcat:ro). +RUN mkdir -p /run/sandcat-capability # __CAPABILITY_RUNTIME_INSTALL__ diff --git a/cli/templates/devcontainer/sandcat/Dockerfile.capability-runtime b/cli/templates/devcontainer/sandcat/Dockerfile.capability-runtime index 6a9eb37a..2518bbd7 100644 --- a/cli/templates/devcontainer/sandcat/Dockerfile.capability-runtime +++ b/cli/templates/devcontainer/sandcat/Dockerfile.capability-runtime @@ -1,4 +1,5 @@ FROM python:3.12-slim +RUN mkdir -p /run/sandcat-capability WORKDIR /app COPY capability-runtime/pyproject.toml capability-runtime/src ./ ENV PYTHONPATH=/app/src diff --git a/cli/templates/devcontainer/sandcat/compose-capability.yml b/cli/templates/devcontainer/sandcat/compose-capability.yml index c5a34162..8ff33e4f 100644 --- a/cli/templates/devcontainer/sandcat/compose-capability.yml +++ b/cli/templates/devcontainer/sandcat/compose-capability.yml @@ -4,13 +4,14 @@ services: context: . dockerfile: Dockerfile.capability-runtime volumes: - - capability-socket:/run/sandcat/capability + # Separate from wg-runtime:/run/sandcat:ro on agent — nested mounts fail there. + - capability-socket:/run/sandcat-capability - ~/.config/sandcat/settings.json:/config/settings.json:ro environment: - SANDCAT_SETTINGS_USER=/config/settings.json - CAPABILITY_CATALOG_JSON=/etc/sandcat/capability-catalog.json - - CAPABILITY_AGENT_SOCKET=/run/sandcat/capability/agent.sock - - CAPABILITY_ADMIN_SOCKET=/run/sandcat/capability/admin.sock + - CAPABILITY_AGENT_SOCKET=/run/sandcat-capability/agent.sock + - CAPABILITY_ADMIN_SOCKET=/run/sandcat-capability/admin.sock restart: unless-stopped volumes: diff --git a/cli/test/composefile/capability.bats b/cli/test/composefile/capability.bats index d87b8edc..966d22b1 100644 --- a/cli/test/composefile/capability.bats +++ b/cli/test/composefile/capability.bats @@ -26,6 +26,12 @@ teardown() { [ "$status" -eq 0 ] } +@test "enable_capability mounts capability socket outside wg-runtime path" { + enable_capability "$COMPOSE_DIR" + run yq '.services.agent.volumes[] | select(. == "capability-socket:/run/sandcat-capability:ro")' "$COMPOSE_DIR/compose-all.yml" + [ "$status" -eq 0 ] +} + @test "enable_capability is idempotent" { enable_capability "$COMPOSE_DIR" enable_capability "$COMPOSE_DIR" From 2ee2aa6fc9adde6c0d54860a6480b307a5bfd038 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 30 Jun 2026 13:22:21 +0000 Subject: [PATCH 046/138] feat(capability-runtime): add NetworkBinding sync_mode and grant protocol Co-authored-by: Cursor --- .../src/capability_runtime/network.py | 23 ++++- capability-runtime/tests/test_network.py | 91 ++++++++++++++++++- 2 files changed, 111 insertions(+), 3 deletions(-) diff --git a/capability-runtime/src/capability_runtime/network.py b/capability-runtime/src/capability_runtime/network.py index 461ed6fe..1fbd31bf 100644 --- a/capability-runtime/src/capability_runtime/network.py +++ b/capability-runtime/src/capability_runtime/network.py @@ -1,20 +1,39 @@ from __future__ import annotations -from dataclasses import dataclass -from typing import Protocol +from dataclasses import dataclass, field +from enum import StrEnum +from typing import Any, Protocol from capability_runtime.types import CapabilityRef +class SyncMode(StrEnum): + ROUTE_ENABLE = "route_enable" + ACL_POLICY = "acl_policy" + PEER_REMOVE = "peer_remove" # break-glass; Phase 3 compat + + @dataclass class NetworkBinding: capability_ref: CapabilityRef peer_id: str network: str route_id: str | None + sync_mode: SyncMode = field(default=SyncMode.ROUTE_ENABLE) + + +def sync_mode_from_catalog(entry: dict[str, Any]) -> SyncMode: + """Parse sync_mode from a catalog capability entry (flat or nested binding).""" + binding = entry.get("binding", entry) + raw = binding.get("sync_mode", SyncMode.ROUTE_ENABLE) + return SyncMode(raw) class PhysicalRevocationBackend(Protocol): def revoke_peer(self, peer_id: str, reason: str) -> None: ... def revoke_route(self, route_id: str, reason: str) -> None: ... + + def grant_binding(self, binding: NetworkBinding) -> None: ... + + def revoke_binding(self, binding: NetworkBinding, reason: str) -> None: ... diff --git a/capability-runtime/tests/test_network.py b/capability-runtime/tests/test_network.py index 5d70af95..74ca8cdd 100644 --- a/capability-runtime/tests/test_network.py +++ b/capability-runtime/tests/test_network.py @@ -1,4 +1,9 @@ -from capability_runtime.network import NetworkBinding, PhysicalRevocationBackend +from capability_runtime.network import ( + NetworkBinding, + PhysicalRevocationBackend, + SyncMode, + sync_mode_from_catalog, +) from capability_runtime.types import CapabilityRef, NetworkCapability @@ -23,3 +28,87 @@ def test_network_binding_dataclass(): route_id="route-1", ) assert binding.route_id == "route-1" + + +def test_network_binding_default_sync_mode(): + binding = NetworkBinding( + CapabilityRef("cap-reach-api"), + "peer-abc", + "10.8.0.0/24", + "route-1", + ) + assert binding.sync_mode is SyncMode.ROUTE_ENABLE + + +def test_network_binding_explicit_sync_mode(): + binding = NetworkBinding( + capability_ref=CapabilityRef("cap-reach-api"), + peer_id="peer-abc", + network="10.8.0.0/24", + route_id=None, + sync_mode=SyncMode.ACL_POLICY, + ) + assert binding.sync_mode is SyncMode.ACL_POLICY + + +def test_sync_mode_from_catalog_flat_entry(): + entry = { + "peer_id": "peer-abc", + "network": "10.8.0.0/24", + "route_id": None, + "sync_mode": "acl_policy", + } + assert sync_mode_from_catalog(entry) is SyncMode.ACL_POLICY + + +def test_sync_mode_from_catalog_nested_binding(): + entry = { + "ref": "cap-reach-api", + "name": "reach_api", + "binding": { + "peer_id": "peer-abc", + "network": "10.8.0.0/24", + "route_id": None, + "sync_mode": "route_enable", + }, + } + assert sync_mode_from_catalog(entry) is SyncMode.ROUTE_ENABLE + + +def test_sync_mode_from_catalog_defaults_to_route_enable(): + assert sync_mode_from_catalog({"peer_id": "peer-abc"}) is SyncMode.ROUTE_ENABLE + assert ( + sync_mode_from_catalog({"binding": {"peer_id": "peer-abc"}}) + is SyncMode.ROUTE_ENABLE + ) + + +def test_sync_mode_from_catalog_peer_remove(): + entry = {"sync_mode": "peer_remove"} + assert sync_mode_from_catalog(entry) is SyncMode.PEER_REMOVE + + +class _StubBackend: + def revoke_peer(self, peer_id: str, reason: str) -> None: + pass + + def revoke_route(self, route_id: str, reason: str) -> None: + pass + + def grant_binding(self, binding: NetworkBinding) -> None: + pass + + def revoke_binding(self, binding: NetworkBinding, reason: str) -> None: + pass + + +def test_physical_revocation_backend_protocol_includes_grant_and_revoke_binding(): + backend: PhysicalRevocationBackend = _StubBackend() + binding = NetworkBinding( + CapabilityRef("cap-reach-api"), + "peer-abc", + "10.8.0.0/24", + None, + ) + backend.grant_binding(binding) + backend.revoke_binding(binding, reason="test") From 10a8e0bd38655631b4d626cbbaa112b8be6df166 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 30 Jun 2026 13:27:01 +0000 Subject: [PATCH 047/138] feat(capability-runtime): add NetBirdClient enable/disable binding --- .../src/capability_runtime/netbird_client.py | 123 +++++++++- capability-runtime/tests/test_netbird_sync.py | 218 ++++++++++++++++++ cli/lib/netbird.bash | 18 +- 3 files changed, 350 insertions(+), 9 deletions(-) create mode 100644 capability-runtime/tests/test_netbird_sync.py diff --git a/capability-runtime/src/capability_runtime/netbird_client.py b/capability-runtime/src/capability_runtime/netbird_client.py index 3d366ffa..4983b67e 100644 --- a/capability-runtime/src/capability_runtime/netbird_client.py +++ b/capability-runtime/src/capability_runtime/netbird_client.py @@ -2,10 +2,13 @@ import json import os -from typing import Protocol +from dataclasses import replace +from typing import Any, Protocol from urllib.error import HTTPError from urllib.request import Request, urlopen +from capability_runtime.network import NetworkBinding, SyncMode + class NetBirdClient(Protocol): def list_peers(self) -> list[dict]: ... @@ -20,6 +23,10 @@ def peer_exists(self, peer_id: str) -> bool: ... def route_exists(self, route_id: str) -> bool: ... + def enable_binding(self, binding: NetworkBinding) -> NetworkBinding: ... + + def disable_binding(self, binding: NetworkBinding) -> None: ... + class MockNetBirdClient: def __init__( @@ -29,6 +36,7 @@ def __init__( ) -> None: self._peers = list(peers or []) self._routes = list(routes or []) + self._next_route_id = 1 def list_peers(self) -> list[dict]: return list(self._peers) @@ -48,6 +56,51 @@ def peer_exists(self, peer_id: str) -> bool: def route_exists(self, route_id: str) -> bool: return any(route.get("id") == route_id for route in self._routes) + def enable_binding(self, binding: NetworkBinding) -> NetworkBinding: + if binding.sync_mode is SyncMode.PEER_REMOVE: + return binding + if binding.sync_mode is SyncMode.ACL_POLICY: + # ACL policy sync deferred to a future phase. + return binding + if binding.sync_mode is not SyncMode.ROUTE_ENABLE: + return binding + + if binding.route_id: + for route in self._routes: + if route.get("id") == binding.route_id: + route["enabled"] = True + return binding + + route_id = f"route-{self._next_route_id}" + self._next_route_id += 1 + self._routes.append( + { + "id": route_id, + "network": binding.network, + "peer": binding.peer_id, + "enabled": True, + } + ) + return replace(binding, route_id=route_id) + + def disable_binding(self, binding: NetworkBinding) -> None: + if binding.sync_mode is SyncMode.PEER_REMOVE: + if binding.peer_id: + self.remove_peer(binding.peer_id) + return + if binding.sync_mode is SyncMode.ACL_POLICY: + # ACL policy sync deferred to a future phase. + return + if binding.sync_mode is not SyncMode.ROUTE_ENABLE: + return + + if not binding.route_id: + return + for route in self._routes: + if route.get("id") == binding.route_id: + route["enabled"] = False + return + class RestNetBirdClient: def __init__( @@ -88,6 +141,53 @@ def peer_exists(self, peer_id: str) -> bool: def route_exists(self, route_id: str) -> bool: return any(route.get("id") == route_id for route in self.list_routes()) + def enable_binding(self, binding: NetworkBinding) -> NetworkBinding: + if binding.sync_mode is SyncMode.PEER_REMOVE: + return binding + if binding.sync_mode is SyncMode.ACL_POLICY: + # ACL policy sync deferred to a future phase. + return binding + if binding.sync_mode is not SyncMode.ROUTE_ENABLE: + return binding + + if binding.route_id: + self._request( + "PATCH", + f"/api/routes/{binding.route_id}", + {"enabled": True}, + ) + return binding + + created = self._request( + "POST", + "/api/routes", + { + "network": binding.network, + "peer": binding.peer_id, + "enabled": True, + }, + )[0] + return replace(binding, route_id=created["id"]) + + def disable_binding(self, binding: NetworkBinding) -> None: + if binding.sync_mode is SyncMode.PEER_REMOVE: + if binding.peer_id: + self.remove_peer(binding.peer_id) + return + if binding.sync_mode is SyncMode.ACL_POLICY: + # ACL policy sync deferred to a future phase. + return + if binding.sync_mode is not SyncMode.ROUTE_ENABLE: + return + + if not binding.route_id: + return + self._request( + "PATCH", + f"/api/routes/{binding.route_id}", + {"enabled": False}, + ) + def _management_base_url(self) -> str: base = self._management_url.rstrip("/") if base.endswith("/api"): @@ -99,10 +199,17 @@ def _require_token(self) -> str: raise RuntimeError("NB_API_TOKEN is required for NetBird REST API calls") return self._token - def _request(self, method: str, path: str) -> list[dict]: + def _request( + self, + method: str, + path: str, + body: dict[str, Any] | None = None, + ) -> list[dict]: url = f"{self._management_base_url()}{path}" + data = json.dumps(body).encode() if body is not None else None request = Request( url, + data=data, method=method, headers={ "Authorization": f"Token {self._require_token()}", @@ -112,13 +219,13 @@ def _request(self, method: str, path: str) -> list[dict]: ) try: with urlopen(request) as response: - body = response.read().decode() + raw = response.read().decode() except HTTPError: raise - if not body: + if not raw: return [] - data = json.loads(body) - if isinstance(data, list): - return data - return [data] + parsed = json.loads(raw) + if isinstance(parsed, list): + return parsed + return [parsed] diff --git a/capability-runtime/tests/test_netbird_sync.py b/capability-runtime/tests/test_netbird_sync.py new file mode 100644 index 00000000..338320cd --- /dev/null +++ b/capability-runtime/tests/test_netbird_sync.py @@ -0,0 +1,218 @@ +"""Tests for NetBirdClient enable_binding / disable_binding.""" + +from __future__ import annotations + +import json +from io import BytesIO + +from capability_runtime.netbird_client import MockNetBirdClient, RestNetBirdClient +from capability_runtime.network import NetworkBinding, SyncMode +from capability_runtime.types import CapabilityRef + + +def _binding( + *, + route_id: str | None = None, + sync_mode: SyncMode = SyncMode.ROUTE_ENABLE, +) -> NetworkBinding: + return NetworkBinding( + capability_ref=CapabilityRef("cap-reach-api"), + peer_id="peer-abc", + network="10.8.0.0/24", + route_id=route_id, + sync_mode=sync_mode, + ) + + +def test_mock_enable_binding_creates_route_when_missing_route_id(): + client = MockNetBirdClient(peers=[{"id": "peer-abc"}]) + binding = _binding() + + updated = client.enable_binding(binding) + + assert updated.route_id is not None + routes = client.list_routes() + assert len(routes) == 1 + assert routes[0]["network"] == "10.8.0.0/24" + assert routes[0]["peer"] == "peer-abc" + assert routes[0]["enabled"] is True + + +def test_mock_enable_binding_enables_existing_route(): + client = MockNetBirdClient( + routes=[ + { + "id": "route-1", + "network": "10.8.0.0/24", + "peer": "peer-abc", + "enabled": False, + } + ] + ) + binding = _binding(route_id="route-1") + + client.enable_binding(binding) + + assert client.list_routes()[0]["enabled"] is True + + +def test_mock_enable_binding_peer_remove_is_noop(): + client = MockNetBirdClient(peers=[{"id": "peer-abc"}]) + binding = _binding(sync_mode=SyncMode.PEER_REMOVE) + + updated = client.enable_binding(binding) + + assert updated == binding + assert client.peer_exists("peer-abc") + assert client.list_routes() == [] + + +def test_mock_enable_binding_acl_policy_is_noop(): + client = MockNetBirdClient() + binding = _binding(sync_mode=SyncMode.ACL_POLICY) + + updated = client.enable_binding(binding) + + assert updated == binding + assert client.list_routes() == [] + + +def test_mock_disable_binding_disables_route(): + client = MockNetBirdClient( + routes=[ + { + "id": "route-1", + "network": "10.8.0.0/24", + "peer": "peer-abc", + "enabled": True, + } + ] + ) + binding = _binding(route_id="route-1") + + client.disable_binding(binding) + + assert client.list_routes()[0]["enabled"] is False + assert client.route_exists("route-1") + + +def test_mock_disable_binding_peer_remove_deletes_peer(): + client = MockNetBirdClient(peers=[{"id": "peer-abc"}]) + binding = _binding(sync_mode=SyncMode.PEER_REMOVE) + + client.disable_binding(binding) + + assert not client.peer_exists("peer-abc") + + +def test_mock_enable_after_disable_is_idempotent(): + client = MockNetBirdClient( + routes=[ + { + "id": "route-1", + "network": "10.8.0.0/24", + "peer": "peer-abc", + "enabled": True, + } + ] + ) + binding = _binding(route_id="route-1") + + client.disable_binding(binding) + client.enable_binding(binding) + + assert client.list_routes()[0]["enabled"] is True + + +def test_rest_enable_binding_posts_route_when_no_route_id(monkeypatch): + captured: dict = {} + + def fake_urlopen(request): + captured["method"] = request.get_method() + captured["url"] = request.full_url + captured["body"] = request.data.decode() if request.data else None + return BytesIO( + json.dumps( + { + "id": "route-new", + "network": "10.8.0.0/24", + "peer": "peer-abc", + "enabled": True, + } + ).encode() + ) + + monkeypatch.setenv("NB_API_TOKEN", "test-token") + monkeypatch.setattr("capability_runtime.netbird_client.urlopen", fake_urlopen) + + client = RestNetBirdClient() + updated = client.enable_binding(_binding()) + + assert captured["method"] == "POST" + assert captured["url"] == "https://api.netbird.io/api/routes" + assert json.loads(captured["body"]) == { + "network": "10.8.0.0/24", + "peer": "peer-abc", + "enabled": True, + } + assert updated.route_id == "route-new" + + +def test_rest_enable_binding_patches_existing_route(monkeypatch): + captured: dict = {} + + def fake_urlopen(request): + captured["method"] = request.get_method() + captured["url"] = request.full_url + captured["body"] = request.data.decode() if request.data else None + return BytesIO(b"") + + monkeypatch.setenv("NB_API_TOKEN", "test-token") + monkeypatch.setattr("capability_runtime.netbird_client.urlopen", fake_urlopen) + + client = RestNetBirdClient() + binding = _binding(route_id="route-1") + updated = client.enable_binding(binding) + + assert captured["method"] == "PATCH" + assert captured["url"] == "https://api.netbird.io/api/routes/route-1" + assert json.loads(captured["body"]) == {"enabled": True} + assert updated.route_id == "route-1" + + +def test_rest_disable_binding_patches_route_disabled(monkeypatch): + captured: dict = {} + + def fake_urlopen(request): + captured["method"] = request.get_method() + captured["url"] = request.full_url + captured["body"] = request.data.decode() if request.data else None + return BytesIO(b"") + + monkeypatch.setenv("NB_API_TOKEN", "test-token") + monkeypatch.setattr("capability_runtime.netbird_client.urlopen", fake_urlopen) + + client = RestNetBirdClient() + client.disable_binding(_binding(route_id="route-1")) + + assert captured["method"] == "PATCH" + assert captured["url"] == "https://api.netbird.io/api/routes/route-1" + assert json.loads(captured["body"]) == {"enabled": False} + + +def test_rest_disable_binding_peer_remove_deletes_peer(monkeypatch): + captured: dict = {} + + def fake_urlopen(request): + captured["method"] = request.get_method() + captured["url"] = request.full_url + return BytesIO(b"") + + monkeypatch.setenv("NB_API_TOKEN", "test-token") + monkeypatch.setattr("capability_runtime.netbird_client.urlopen", fake_urlopen) + + client = RestNetBirdClient() + client.disable_binding(_binding(sync_mode=SyncMode.PEER_REMOVE)) + + assert captured["method"] == "DELETE" + assert captured["url"] == "https://api.netbird.io/api/peers/peer-abc" diff --git a/cli/lib/netbird.bash b/cli/lib/netbird.bash index f5d7aefc..68dc17cb 100644 --- a/cli/lib/netbird.bash +++ b/cli/lib/netbird.bash @@ -384,7 +384,7 @@ _netbird_api_print_error() { # NB_MANAGEMENT_URL defaults to https://api.netbird.io. # Prints the response body on success; writes curl/API errors to stderr. # Args: -# $1 - HTTP method (GET, POST, DELETE) +# $1 - HTTP method (GET, POST, PATCH, DELETE) # $2 - API path (e.g. /api/peers) # $3 - Optional JSON body netbird_api() { @@ -445,6 +445,22 @@ netbird_route_add() { "{\"network\":\"$network\",\"peer\":\"$peer_id\",\"enabled\":true}" } +# Enables an existing network route by ID. +# Args: +# $1 - Route ID +netbird_route_enable() { + local route_id=$1 + netbird_api "PATCH" "/api/routes/$route_id" '{"enabled":true}' +} + +# Disables an existing network route by ID without deleting it. +# Args: +# $1 - Route ID +netbird_route_disable() { + local route_id=$1 + netbird_api "PATCH" "/api/routes/$route_id" '{"enabled":false}' +} + # Removes a network route by ID. # Args: # $1 - Route ID (returned by netbird_route_add) From a791f6a51455c590ad3bae9d50bc2f497051d7fc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 30 Jun 2026 13:31:44 +0000 Subject: [PATCH 048/138] feat(capability-runtime): wire grant path to NetBird enable_binding Task 3: Grant path in CapabilityRuntime - Add grant_binding method to NetBirdRevocationBackend that delegates to client.enable_binding(binding) and returns updated binding - Create netbird_sync.py orchestration helper for grant_network_binding - Modify request_capability_lease to call grant_binding for network capabilities after successful lease grant - Persist updated binding (route_id) in catalog via set_network_binding - Emit capability_leased event with physical_sync: "enabled" metadata for network capabilities - Implement fail-closed rollback on enable_binding failure: revoke lease and revert catalog state to pre-lease state - Add comprehensive integration tests in test_grant_revoke_integration.py All 117 tests pass. --- .../src/capability_runtime/netbird_backend.py | 7 + .../src/capability_runtime/netbird_sync.py | 28 +++ .../src/capability_runtime/runtime.py | 61 ++++- .../tests/test_grant_revoke_integration.py | 233 ++++++++++++++++++ 4 files changed, 319 insertions(+), 10 deletions(-) create mode 100644 capability-runtime/src/capability_runtime/netbird_sync.py create mode 100644 capability-runtime/tests/test_grant_revoke_integration.py diff --git a/capability-runtime/src/capability_runtime/netbird_backend.py b/capability-runtime/src/capability_runtime/netbird_backend.py index a831ff84..ee533b66 100644 --- a/capability-runtime/src/capability_runtime/netbird_backend.py +++ b/capability-runtime/src/capability_runtime/netbird_backend.py @@ -8,6 +8,13 @@ class NetBirdRevocationBackend: def __init__(self, client: NetBirdClient) -> None: self._client = client + def grant_binding(self, binding: NetworkBinding) -> NetworkBinding: + """Enable a network binding via NetBird client. + + Returns the updated binding (may have a new route_id if one was created). + """ + return self._client.enable_binding(binding) + def revoke_binding(self, binding: NetworkBinding, reason: str) -> None: if binding.route_id: self.revoke_route(binding.route_id, reason) diff --git a/capability-runtime/src/capability_runtime/netbird_sync.py b/capability-runtime/src/capability_runtime/netbird_sync.py new file mode 100644 index 00000000..595c9552 --- /dev/null +++ b/capability-runtime/src/capability_runtime/netbird_sync.py @@ -0,0 +1,28 @@ +"""Orchestration helpers for NetBird physical sync during capability grant.""" + +from __future__ import annotations + +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from capability_runtime.network import NetworkBinding + from capability_runtime.netbird_backend import NetBirdRevocationBackend + + +def grant_network_binding( + backend: NetBirdRevocationBackend, + binding: NetworkBinding, +) -> NetworkBinding: + """Grant a network binding by enabling it via the NetBird backend. + + Args: + backend: The NetBird revocation backend + binding: The network binding to grant + + Returns: + The updated binding (may have a new route_id if one was created) + + Raises: + Any exception from enable_binding (caller should handle rollback) + """ + return backend.grant_binding(binding) diff --git a/capability-runtime/src/capability_runtime/runtime.py b/capability-runtime/src/capability_runtime/runtime.py index 8c1ccfab..07950649 100644 --- a/capability-runtime/src/capability_runtime/runtime.py +++ b/capability-runtime/src/capability_runtime/runtime.py @@ -23,6 +23,7 @@ from capability_runtime.network import NetworkBinding from capability_runtime.netbird_backend import NetBirdRevocationBackend from capability_runtime.netbird_client import NetBirdClient +from capability_runtime.netbird_sync import grant_network_binding from capability_runtime.observability import ObservabilityCollector from capability_runtime.revoke import RevocationManager from capability_runtime.types import ( @@ -217,6 +218,8 @@ def request_capability_lease( if state not in (LifecycleState.DECLARED, LifecycleState.DISCOVERABLE, LifecycleState.VISIBLE): raise CapabilityUnknown(capability_ref) + # Save the original state for rollback in case of failure + original_state = state now = datetime.now(timezone.utc) capability_name = self.catalog.get_name(capability_ref) @@ -245,16 +248,54 @@ def request_capability_lease( # Set catalog state to LEASED self.catalog.set_state(capability_ref, LifecycleState.LEASED) - self.observability.emit_capability_event( - { - "event": "lease_granted", - "agent_id": agent_id.value, - "capability_ref": capability_ref.value, - "lease_id": decision.lease_id.value, - "quota": quota, - "justification": justification, - } - ) + # Check if this is a network capability that needs physical sync + binding = self.catalog.get_network_binding(capability_ref) + physical_sync_status = None + + if binding is not None and self._netbird_backend is not None: + # Attempt to enable the binding via NetBird + try: + updated_binding = grant_network_binding(self._netbird_backend, binding) + # Update the binding in catalog if route_id changed + if updated_binding.route_id != binding.route_id: + self.catalog.set_network_binding(capability_ref, updated_binding) + physical_sync_status = "enabled" + except Exception as e: + # Rollback on failure (fail closed) + # 1. Revoke the lease + self.lease_manager._leases.pop(decision.lease_id, None) + self.lease_manager._remaining_quota.pop(decision.lease_id, None) + # 2. Revert catalog state + self.catalog.set_state(capability_ref, original_state) + # Re-raise the exception + raise + + # Emit appropriate event + if physical_sync_status == "enabled": + # Emit capability_leased event for network capabilities + self.observability.emit_capability_event( + { + "event": "capability_leased", + "agent_id": agent_id.value, + "capability_ref": capability_ref.value, + "lease_id": decision.lease_id.value, + "quota": quota, + "justification": justification, + "physical_sync": physical_sync_status, + } + ) + else: + # Emit lease_granted event for tool capabilities + self.observability.emit_capability_event( + { + "event": "lease_granted", + "agent_id": agent_id.value, + "capability_ref": capability_ref.value, + "lease_id": decision.lease_id.value, + "quota": quota, + "justification": justification, + } + ) return decision diff --git a/capability-runtime/tests/test_grant_revoke_integration.py b/capability-runtime/tests/test_grant_revoke_integration.py new file mode 100644 index 00000000..8b1db42e --- /dev/null +++ b/capability-runtime/tests/test_grant_revoke_integration.py @@ -0,0 +1,233 @@ +"""Integration tests for grant → enable_binding → revoke flow.""" + +from __future__ import annotations + +from datetime import datetime, timezone +from pathlib import Path + +import pytest + +from capability_runtime.catalog import LifecycleState +from capability_runtime.netbird_client import MockNetBirdClient +from capability_runtime.network import NetworkBinding, SyncMode +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity, CapabilityRef + + +@pytest.fixture +def temp_trace_file(tmp_path: Path) -> Path: + """Create a temporary trace file.""" + return tmp_path / "trace.jsonl" + + +@pytest.fixture +def netbird_client() -> MockNetBirdClient: + """Create a mock NetBird client.""" + return MockNetBirdClient( + peers=[{"id": "peer-123", "name": "test-peer"}], + routes=[], + ) + + +@pytest.fixture +def runtime_with_netbird(temp_trace_file: Path, netbird_client: MockNetBirdClient) -> CapabilityRuntime: + """Create a runtime with NetBird backend enabled.""" + return CapabilityRuntime( + trace_file=temp_trace_file, + trace_id="test-trace", + seed=42, + netbird_client=netbird_client, + ) + + +def test_grant_network_capability_enables_binding( + runtime_with_netbird: CapabilityRuntime, + netbird_client: MockNetBirdClient, +) -> None: + """Test that granting a network capability calls enable_binding and emits capability_leased.""" + # Register a network capability in VISIBLE state + ref = CapabilityRef("cap-network-test") + binding = NetworkBinding( + capability_ref=ref, + peer_id="peer-123", + network="10.0.0.0/24", + route_id=None, # No route yet + sync_mode=SyncMode.ROUTE_ENABLE, + ) + runtime_with_netbird.register_network_capability( + "network_test", + ref, + binding, + initial_state=LifecycleState.VISIBLE, + ) + + # Grant lease + agent_id = AgentIdentity("test-agent") + decision = runtime_with_netbird.request_capability_lease( + caller=agent_id, + agent_id=agent_id, + capability_ref=ref, + justification="testing network grant", + ) + + # Verify lease granted + assert decision.lease_id is not None + + # Verify binding was enabled (route created) + routes = netbird_client.list_routes() + assert len(routes) == 1 + assert routes[0]["network"] == "10.0.0.0/24" + assert routes[0]["peer"] == "peer-123" + assert routes[0]["enabled"] is True + + # Verify route_id was persisted in catalog + updated_binding = runtime_with_netbird.catalog.get_network_binding(ref) + assert updated_binding is not None + assert updated_binding.route_id is not None + assert updated_binding.route_id == routes[0]["id"] + + # Verify capability_leased event was emitted with physical_sync metadata + events = runtime_with_netbird.observability._events + leased_events = [e for e in events if e.get("event") == "capability_leased"] + assert len(leased_events) >= 1 + last_leased = leased_events[-1] + assert last_leased.get("physical_sync") == "enabled" + assert last_leased.get("capability_ref") == ref.value + + +def test_grant_network_capability_rolls_back_on_enable_failure( + runtime_with_netbird: CapabilityRuntime, + netbird_client: MockNetBirdClient, +) -> None: + """Test that enable_binding failure rolls back the lease grant (fail closed).""" + # Register a network capability + ref = CapabilityRef("cap-network-fail") + binding = NetworkBinding( + capability_ref=ref, + peer_id="nonexistent-peer", # This will cause enable_binding to fail + network="10.0.0.0/24", + route_id=None, + sync_mode=SyncMode.ROUTE_ENABLE, + ) + runtime_with_netbird.register_network_capability( + "network_fail", + ref, + binding, + initial_state=LifecycleState.VISIBLE, + ) + + # Mock enable_binding to raise an error + original_enable = netbird_client.enable_binding + + def failing_enable(binding: NetworkBinding) -> NetworkBinding: + raise RuntimeError("NetBird API failure") + + netbird_client.enable_binding = failing_enable # type: ignore + + # Attempt to grant lease should fail + agent_id = AgentIdentity("test-agent") + with pytest.raises(RuntimeError, match="NetBird API failure"): + runtime_with_netbird.request_capability_lease( + caller=agent_id, + agent_id=agent_id, + capability_ref=ref, + justification="testing rollback", + ) + + # Verify lease was NOT granted (rolled back) + assert len(runtime_with_netbird.lease_manager._leases) == 0 + + # Verify catalog state was rolled back (should be VISIBLE, not LEASED) + state = runtime_with_netbird.catalog.get_state(ref) + assert state == LifecycleState.VISIBLE + + # Restore original function + netbird_client.enable_binding = original_enable # type: ignore + + +def test_grant_tool_capability_skips_physical_sync( + runtime_with_netbird: CapabilityRuntime, + netbird_client: MockNetBirdClient, +) -> None: + """Test that granting a non-network (tool) capability does not call enable_binding.""" + # Use the built-in create_pr capability (registered in __init__) + ref = CapabilityRef("cap-create-pr") + runtime_with_netbird.catalog.set_state(ref, LifecycleState.VISIBLE) + + # Grant lease + agent_id = AgentIdentity("test-agent") + decision = runtime_with_netbird.request_capability_lease( + caller=agent_id, + agent_id=agent_id, + capability_ref=ref, + justification="testing tool grant", + ) + + # Verify lease granted + assert decision.lease_id is not None + + # Verify no routes were created (no physical sync for tool capabilities) + routes = netbird_client.list_routes() + assert len(routes) == 0 + + # Verify lease_granted event was emitted (not capability_leased with physical_sync) + events = runtime_with_netbird.observability._events + granted_events = [e for e in events if e.get("event") == "lease_granted"] + assert len(granted_events) >= 1 + + +def test_grant_network_capability_with_existing_route( + runtime_with_netbird: CapabilityRuntime, + netbird_client: MockNetBirdClient, +) -> None: + """Test that granting a network capability with an existing route_id enables it.""" + # Pre-create a disabled route + netbird_client._routes.append( + { + "id": "route-existing", + "network": "10.0.0.0/24", + "peer": "peer-123", + "enabled": False, + } + ) + + # Register a network capability with existing route_id + ref = CapabilityRef("cap-network-existing") + binding = NetworkBinding( + capability_ref=ref, + peer_id="peer-123", + network="10.0.0.0/24", + route_id="route-existing", # Existing route + sync_mode=SyncMode.ROUTE_ENABLE, + ) + runtime_with_netbird.register_network_capability( + "network_existing", + ref, + binding, + initial_state=LifecycleState.VISIBLE, + ) + + # Grant lease + agent_id = AgentIdentity("test-agent") + decision = runtime_with_netbird.request_capability_lease( + caller=agent_id, + agent_id=agent_id, + capability_ref=ref, + justification="testing existing route", + ) + + # Verify lease granted + assert decision.lease_id is not None + + # Verify existing route was enabled (not a new route created) + routes = netbird_client.list_routes() + assert len(routes) == 1 + assert routes[0]["id"] == "route-existing" + assert routes[0]["enabled"] is True + + # Verify capability_leased event + events = runtime_with_netbird.observability._events + leased_events = [e for e in events if e.get("event") == "capability_leased"] + assert len(leased_events) >= 1 + last_leased = leased_events[-1] + assert last_leased.get("physical_sync") == "enabled" From 109ce26dd50be14bf5784e071e15fa6510e9d425 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 30 Jun 2026 13:38:38 +0000 Subject: [PATCH 049/138] feat(capability-runtime): prefer disable_binding on revoke Implement Task 4 (Phase 3c): change revoke path to disable routes instead of deleting peers. Changes: - NetBirdRevocationBackend.revoke_binding now calls client.disable_binding - Default sync_mode (ROUTE_ENABLE): disables route, keeps peer - PEER_REMOVE mode: deletes peer (Phase 3 compat maintained) - Quota exhaustion triggers disable_binding for network leases - TTL expiry triggers disable_binding via check_current_capabilities - Added revoke_network_binding helper to netbird_sync module Tests: - Updated test_netbird_backend to expect disable behavior - Updated test_runtime_network to verify peer stays, route disabled - Added test_quota_exhaustion_disables_network_binding - Added test_ttl_expiry_disables_network_binding - Updated POC network route demo and tests All 120 tests pass. Co-authored-by: Cursor --- capability-runtime/poc/network_route_demo.py | 14 +++- .../src/capability_runtime.egg-info/PKG-INFO | 4 + .../capability_runtime.egg-info/SOURCES.txt | 59 ++++++++++++++ .../dependency_links.txt | 1 + .../capability_runtime.egg-info/top_level.txt | 1 + .../src/capability_runtime/netbird_backend.py | 10 ++- .../src/capability_runtime/netbird_sync.py | 15 ++++ .../src/capability_runtime/runtime.py | 24 ++++++ .../tests/test_netbird_backend.py | 34 +++++++- .../tests/test_poc_network_route.py | 4 +- .../tests/test_runtime_network.py | 79 ++++++++++++++++++- 11 files changed, 230 insertions(+), 15 deletions(-) create mode 100644 capability-runtime/src/capability_runtime.egg-info/PKG-INFO create mode 100644 capability-runtime/src/capability_runtime.egg-info/SOURCES.txt create mode 100644 capability-runtime/src/capability_runtime.egg-info/dependency_links.txt create mode 100644 capability-runtime/src/capability_runtime.egg-info/top_level.txt diff --git a/capability-runtime/poc/network_route_demo.py b/capability-runtime/poc/network_route_demo.py index 1c02dd17..fb23fcf3 100644 --- a/capability-runtime/poc/network_route_demo.py +++ b/capability-runtime/poc/network_route_demo.py @@ -24,7 +24,8 @@ class DemoResult: lease_decision: LeaseDecision networks_after_lease: list[str] peer_id_after_lease: str | None - peer_removed_by_revoke: bool + peer_still_exists_after_revoke: bool + route_disabled_by_revoke: bool networks_after_revoke: list[str] networks_after_external_removal: list[str] external_removal_peer_gone: bool @@ -94,11 +95,15 @@ def run_poc3_demo(trace_path: Path, *, quiet: bool = False) -> DemoResult: ) runtime.revoke_capability(AgentIdentity("operator"), ref, "security policy") - peer_removed = not client.peer_exists("peer-abc") + peer_still_exists = client.peer_exists("peer-abc") + route_disabled = False + if client.route_exists("route-1"): + routes = [r for r in client.list_routes() if r["id"] == "route-1"] + route_disabled = routes[0]["enabled"] is False if routes else False _print_step( quiet, 4, - f"revoke_capability → mock client peer removed (peer_exists={client.peer_exists('peer-abc')})", + f"revoke_capability → peer still exists={peer_still_exists}, route disabled={route_disabled}", ) bundle3 = runtime.check_current_capabilities(agent, context) @@ -127,7 +132,8 @@ def run_poc3_demo(trace_path: Path, *, quiet: bool = False) -> DemoResult: lease_decision=decision, networks_after_lease=networks_after_lease, peer_id_after_lease=peer_id, - peer_removed_by_revoke=peer_removed, + peer_still_exists_after_revoke=peer_still_exists, + route_disabled_by_revoke=route_disabled, networks_after_revoke=networks_after_revoke, networks_after_external_removal=networks_after_external, external_removal_peer_gone=not client.peer_exists("peer-abc"), diff --git a/capability-runtime/src/capability_runtime.egg-info/PKG-INFO b/capability-runtime/src/capability_runtime.egg-info/PKG-INFO new file mode 100644 index 00000000..aeb081ce --- /dev/null +++ b/capability-runtime/src/capability_runtime.egg-info/PKG-INFO @@ -0,0 +1,4 @@ +Metadata-Version: 2.4 +Name: capability-runtime +Version: 0.1.0 +Requires-Python: >=3.12 diff --git a/capability-runtime/src/capability_runtime.egg-info/SOURCES.txt b/capability-runtime/src/capability_runtime.egg-info/SOURCES.txt new file mode 100644 index 00000000..178c738e --- /dev/null +++ b/capability-runtime/src/capability_runtime.egg-info/SOURCES.txt @@ -0,0 +1,59 @@ +README.md +pyproject.toml +src/capability_runtime/__init__.py +src/capability_runtime/agent_loop.py +src/capability_runtime/catalog.py +src/capability_runtime/cli.py +src/capability_runtime/daemon.py +src/capability_runtime/discover.py +src/capability_runtime/errors.py +src/capability_runtime/lease.py +src/capability_runtime/mcp_adapter.py +src/capability_runtime/netbird_backend.py +src/capability_runtime/netbird_client.py +src/capability_runtime/network.py +src/capability_runtime/observability.py +src/capability_runtime/policy.py +src/capability_runtime/revoke.py +src/capability_runtime/route_watcher.py +src/capability_runtime/runtime.py +src/capability_runtime/settings.py +src/capability_runtime/types.py +src/capability_runtime.egg-info/PKG-INFO +src/capability_runtime.egg-info/SOURCES.txt +src/capability_runtime.egg-info/dependency_links.txt +src/capability_runtime.egg-info/top_level.txt +src/capability_runtime/mcp/__init__.py +src/capability_runtime/mcp/bridge.py +src/capability_runtime/mcp/server.py +src/capability_runtime/rpc/__init__.py +src/capability_runtime/rpc/dispatcher.py +src/capability_runtime/rpc/errors.py +src/capability_runtime/rpc/transports/__init__.py +src/capability_runtime/rpc/transports/stdio_bridge.py +src/capability_runtime/rpc/transports/unix.py +tests/test_agent_loop.py +tests/test_catalog.py +tests/test_daemon_integration.py +tests/test_discover.py +tests/test_errors.py +tests/test_lease.py +tests/test_mcp_adapter.py +tests/test_mcp_bridge.py +tests/test_mcp_server.py +tests/test_netbird_backend.py +tests/test_netbird_client.py +tests/test_network.py +tests/test_observability.py +tests/test_poc_create_pr.py +tests/test_poc_network_route.py +tests/test_policy.py +tests/test_revoke.py +tests/test_route_watcher.py +tests/test_rpc_dispatcher.py +tests/test_rpc_unix.py +tests/test_runtime.py +tests/test_runtime_network.py +tests/test_security.py +tests/test_settings.py +tests/test_types.py \ No newline at end of file diff --git a/capability-runtime/src/capability_runtime.egg-info/dependency_links.txt b/capability-runtime/src/capability_runtime.egg-info/dependency_links.txt new file mode 100644 index 00000000..8b137891 --- /dev/null +++ b/capability-runtime/src/capability_runtime.egg-info/dependency_links.txt @@ -0,0 +1 @@ + diff --git a/capability-runtime/src/capability_runtime.egg-info/top_level.txt b/capability-runtime/src/capability_runtime.egg-info/top_level.txt new file mode 100644 index 00000000..49a0ab9a --- /dev/null +++ b/capability-runtime/src/capability_runtime.egg-info/top_level.txt @@ -0,0 +1 @@ +capability_runtime diff --git a/capability-runtime/src/capability_runtime/netbird_backend.py b/capability-runtime/src/capability_runtime/netbird_backend.py index ee533b66..58745285 100644 --- a/capability-runtime/src/capability_runtime/netbird_backend.py +++ b/capability-runtime/src/capability_runtime/netbird_backend.py @@ -16,10 +16,12 @@ def grant_binding(self, binding: NetworkBinding) -> NetworkBinding: return self._client.enable_binding(binding) def revoke_binding(self, binding: NetworkBinding, reason: str) -> None: - if binding.route_id: - self.revoke_route(binding.route_id, reason) - if binding.peer_id: - self.revoke_peer(binding.peer_id, reason) + """Revoke a network binding via NetBird client. + + Default behavior (ROUTE_ENABLE): disables route, keeps peer. + PEER_REMOVE mode: deletes peer (Phase 3 compat). + """ + self._client.disable_binding(binding) def revoke_peer(self, peer_id: str, reason: str) -> None: self._client.remove_peer(peer_id) diff --git a/capability-runtime/src/capability_runtime/netbird_sync.py b/capability-runtime/src/capability_runtime/netbird_sync.py index 595c9552..e63538e8 100644 --- a/capability-runtime/src/capability_runtime/netbird_sync.py +++ b/capability-runtime/src/capability_runtime/netbird_sync.py @@ -26,3 +26,18 @@ def grant_network_binding( Any exception from enable_binding (caller should handle rollback) """ return backend.grant_binding(binding) + + +def revoke_network_binding( + backend: NetBirdRevocationBackend, + binding: NetworkBinding, + reason: str, +) -> None: + """Revoke a network binding by disabling it via the NetBird backend. + + Args: + backend: The NetBird revocation backend + binding: The network binding to revoke + reason: Reason for revocation (for observability) + """ + backend.revoke_binding(binding, reason) diff --git a/capability-runtime/src/capability_runtime/runtime.py b/capability-runtime/src/capability_runtime/runtime.py index 07950649..1695a7d2 100644 --- a/capability-runtime/src/capability_runtime/runtime.py +++ b/capability-runtime/src/capability_runtime/runtime.py @@ -170,6 +170,23 @@ def check_current_capabilities( earliest_expiry = lease.expires_at break + # Disable expired network leases (TTL expiry hook) + if self._netbird_backend is not None: + from capability_runtime.netbird_sync import revoke_network_binding + + for lease_id, lease in list(self.lease_manager._leases.items()): + if ( + self.lease_manager.is_expired(lease_id, now) + and not self.revocation_manager.is_lease_revoked(lease_id) + and not self.lease_manager.is_exhausted(lease_id) + ): + binding = self.catalog.get_network_binding(lease.capability_ref) + if binding is not None: + # Disable the binding and revoke the lease + revoke_network_binding(self._netbird_backend, binding, "TTL expired") + self.revocation_manager.revoke_by_lease(lease_id, "TTL expired") + self.catalog.set_state(lease.capability_ref, LifecycleState.EXPIRED) + bundle = CapabilityBundle( agent_id=agent_id, issued_at=now, @@ -402,6 +419,13 @@ def record_action( if remaining == 0: lease = self.lease_manager.get_lease(lease_id) if lease is not None: + # Check if this is a network capability that needs physical sync + binding = self.catalog.get_network_binding(lease.capability_ref) + if binding is not None and self._netbird_backend is not None: + # Disable the binding via NetBird + from capability_runtime.netbird_sync import revoke_network_binding + revoke_network_binding(self._netbird_backend, binding, "quota exhausted") + self.catalog.set_state(lease.capability_ref, LifecycleState.EXPIRED) self.revocation_manager.revoke_by_lease(lease_id, "quota exhausted") self._bundle_version += 1 diff --git a/capability-runtime/tests/test_netbird_backend.py b/capability-runtime/tests/test_netbird_backend.py index 0fbd1140..0b7a1c58 100644 --- a/capability-runtime/tests/test_netbird_backend.py +++ b/capability-runtime/tests/test_netbird_backend.py @@ -2,14 +2,15 @@ from capability_runtime.netbird_backend import NetBirdRevocationBackend from capability_runtime.netbird_client import MockNetBirdClient -from capability_runtime.network import NetworkBinding +from capability_runtime.network import NetworkBinding, SyncMode from capability_runtime.types import CapabilityRef -def test_backend_revokes_route_then_peer(): +def test_backend_disables_route_keeps_peer_by_default(): + """Default sync_mode (ROUTE_ENABLE) disables route, keeps peer.""" client = MockNetBirdClient( peers=[{"id": "peer-abc", "connected": True}], - routes=[{"id": "route-1", "network": "10.8.0.0/24", "peer": "peer-abc"}], + routes=[{"id": "route-1", "network": "10.8.0.0/24", "peer": "peer-abc", "enabled": True}], ) backend = NetBirdRevocationBackend(client) binding = NetworkBinding( @@ -17,7 +18,32 @@ def test_backend_revokes_route_then_peer(): peer_id="peer-abc", network="10.8.0.0/24", route_id="route-1", + sync_mode=SyncMode.ROUTE_ENABLE, ) backend.revoke_binding(binding, reason="policy") - assert not client.route_exists("route-1") + # Route should still exist but be disabled + assert client.route_exists("route-1") + routes = [r for r in client.list_routes() if r["id"] == "route-1"] + assert len(routes) == 1 + assert routes[0]["enabled"] is False + # Peer should still exist + assert client.peer_exists("peer-abc") + + +def test_backend_removes_peer_when_peer_remove_mode(): + """sync_mode=PEER_REMOVE deletes peer (Phase 3 compat).""" + client = MockNetBirdClient( + peers=[{"id": "peer-abc", "connected": True}], + routes=[{"id": "route-1", "network": "10.8.0.0/24", "peer": "peer-abc", "enabled": True}], + ) + backend = NetBirdRevocationBackend(client) + binding = NetworkBinding( + capability_ref=CapabilityRef("cap-reach-api"), + peer_id="peer-abc", + network="10.8.0.0/24", + route_id="route-1", + sync_mode=SyncMode.PEER_REMOVE, + ) + backend.revoke_binding(binding, reason="policy") + # Peer should be deleted in PEER_REMOVE mode assert not client.peer_exists("peer-abc") diff --git a/capability-runtime/tests/test_poc_network_route.py b/capability-runtime/tests/test_poc_network_route.py index eea63916..e24b2943 100644 --- a/capability-runtime/tests/test_poc_network_route.py +++ b/capability-runtime/tests/test_poc_network_route.py @@ -16,7 +16,9 @@ def test_poc3_network_route_lifecycle(tmp_path: Path) -> None: assert result.lease_decision.capability_ref == CapabilityRef("cap-reach-api") assert "reach_api" in result.networks_after_lease assert result.peer_id_after_lease == "peer-abc" - assert result.peer_removed_by_revoke is True + # Phase 3c: prefer disable over peer delete + assert result.peer_still_exists_after_revoke is True + assert result.route_disabled_by_revoke is True assert "reach_api" not in result.networks_after_revoke assert "reach_api" not in result.networks_after_external_removal assert result.external_removal_peer_gone is True diff --git a/capability-runtime/tests/test_runtime_network.py b/capability-runtime/tests/test_runtime_network.py index 00030671..f8014f64 100644 --- a/capability-runtime/tests/test_runtime_network.py +++ b/capability-runtime/tests/test_runtime_network.py @@ -30,7 +30,7 @@ def test_revoke_network_capability_calls_netbird_backend(tmp_path): client = MockNetBirdClient( peers=[{"id": "peer-abc", "connected": True}], - routes=[{"id": "route-1", "network": "10.8.0.0/24"}], + routes=[{"id": "route-1", "network": "10.8.0.0/24", "enabled": True}], ) runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-n2", 2, netbird_client=client) agent = AgentIdentity("agent-1") @@ -38,7 +38,12 @@ def test_revoke_network_capability_calls_netbird_backend(tmp_path): binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", "route-1") runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) runtime.revoke_capability(operator, ref, "security") - assert not client.peer_exists("peer-abc") + # Default sync_mode (ROUTE_ENABLE) keeps peer, disables route + assert client.peer_exists("peer-abc") + assert client.route_exists("route-1") + routes = [r for r in client.list_routes() if r["id"] == "route-1"] + assert routes[0]["enabled"] is False + # Capability should be logically revoked bundle = runtime.check_current_capabilities(agent, {}) assert "reach_api" not in [n.name for n in bundle.networks] @@ -63,3 +68,73 @@ def test_revoke_non_network_capability_does_not_call_netbird(tmp_path): assert client.peer_exists("peer-xyz") bundle = runtime.check_current_capabilities(agent, {}) assert "create_pr_tool" not in [t.name for t in bundle.tools] + + +def test_quota_exhaustion_disables_network_binding(tmp_path): + """When quota is exhausted for a network lease, disable the binding.""" + from datetime import datetime, timezone + from capability_runtime.catalog import LifecycleState + from capability_runtime.netbird_client import MockNetBirdClient + from capability_runtime.network import NetworkBinding + from capability_runtime.runtime import CapabilityRuntime + from capability_runtime.types import AgentIdentity, CapabilityRef + + client = MockNetBirdClient( + peers=[{"id": "peer-quota", "connected": True}], + routes=[{"id": "route-quota", "network": "192.168.1.0/24", "enabled": True}], + ) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-quota", 1, netbird_client=client) + agent = AgentIdentity("agent-quota") + ref = CapabilityRef("cap-quota-test") + binding = NetworkBinding(ref, "peer-quota", "192.168.1.0/24", "route-quota") + runtime.register_network_capability("quota_net", ref, binding, LifecycleState.DECLARED) + + # Lease with quota=1 + decision = runtime.request_capability_lease(agent, agent, ref, "testing quota") + + # Exhaust quota + now = datetime.now(timezone.utc) + runtime.record_action(agent, agent, decision.lease_id, now) + + # Peer should still exist, route should be disabled + assert client.peer_exists("peer-quota") + assert client.route_exists("route-quota") + routes = [r for r in client.list_routes() if r["id"] == "route-quota"] + assert routes[0]["enabled"] is False + + +def test_ttl_expiry_disables_network_binding(tmp_path): + """When a lease expires (TTL), disable the binding on next bundle check.""" + from datetime import datetime, timedelta, timezone + from capability_runtime.catalog import LifecycleState + from capability_runtime.netbird_client import MockNetBirdClient + from capability_runtime.network import NetworkBinding + from capability_runtime.runtime import CapabilityRuntime + from capability_runtime.types import AgentIdentity, CapabilityRef, LeaseId + + client = MockNetBirdClient( + peers=[{"id": "peer-ttl", "connected": True}], + routes=[{"id": "route-ttl", "network": "172.16.0.0/24", "enabled": True}], + ) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-ttl", 2, netbird_client=client) + agent = AgentIdentity("agent-ttl") + ref = CapabilityRef("cap-ttl-test") + binding = NetworkBinding(ref, "peer-ttl", "172.16.0.0/24", "route-ttl") + runtime.register_network_capability("ttl_net", ref, binding, LifecycleState.DECLARED) + + # Lease with short TTL + decision = runtime.request_capability_lease(agent, agent, ref, "testing ttl") + + # Force expiry by manipulating lease expiry time + runtime.lease_manager._leases[decision.lease_id].expires_at = datetime.now(timezone.utc) - timedelta(seconds=1) + + # Check capabilities - should detect expired lease and disable binding + bundle = runtime.check_current_capabilities(agent, {}) + + # Peer should still exist, route should be disabled + assert client.peer_exists("peer-ttl") + assert client.route_exists("route-ttl") + routes = [r for r in client.list_routes() if r["id"] == "route-ttl"] + assert routes[0]["enabled"] is False + # Capability should not be in bundle + assert "ttl_net" not in [n.name for n in bundle.networks] From 60cc94119cdd0c4db58234ffba08fac1d112bd90 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 30 Jun 2026 13:42:23 +0000 Subject: [PATCH 050/138] feat(cli): catalog sync_mode and capability watch physical sync events - Add sync_mode: route_enable to cap-reach-api binding in catalog - Update daemon.py load_catalog_into_runtime to use sync_mode_from_catalog - Enhance capability watch to tail JSONL trace events and print capability_leased, capability_revoked events with physical_sync status - Add bats test for lease then revoke flow sequence Co-authored-by: Cursor --- .../src/capability_runtime/cli.py | 41 ++++++++++++++++++- .../src/capability_runtime/daemon.py | 3 +- .../sandcat/capability-catalog.json | 3 +- cli/test/capability/capability.bats | 19 +++++++++ 4 files changed, 63 insertions(+), 3 deletions(-) diff --git a/capability-runtime/src/capability_runtime/cli.py b/capability-runtime/src/capability_runtime/cli.py index 406b46ee..9872fede 100644 --- a/capability-runtime/src/capability_runtime/cli.py +++ b/capability-runtime/src/capability_runtime/cli.py @@ -14,6 +14,9 @@ DEFAULT_ADMIN_SOCKET = Path( os.environ.get("CAPABILITY_ADMIN_SOCKET", "/run/sandcat-capability/admin.sock") ) +DEFAULT_TRACE_FILE = Path( + os.environ.get("CAPABILITY_TRACE_FILE", "/var/lib/sandcat/capability/trace.jsonl") +) SUBCOMMAND_TO_METHOD = { "check": "capability.check", @@ -90,12 +93,48 @@ def cmd_revoke(args: argparse.Namespace) -> None: def cmd_watch(args: argparse.Namespace) -> None: print( - f"Watching capability route watcher (poll every {args.interval}s)...", + f"Watching capability events and route watcher (poll every {args.interval}s)...", file=sys.stderr, ) + trace_file = DEFAULT_TRACE_FILE + trace_offset = 0 + if trace_file.exists(): + trace_offset = trace_file.stat().st_size + while True: result = _call("capability.watch.poll", {}) _print_json({"polled": result.get("polled", True), "ts": time.time()}) + + if trace_file.exists(): + with trace_file.open("r") as fp: + fp.seek(trace_offset) + for line in fp: + trace_offset = fp.tell() + try: + event = json.loads(line) + if event.get("kind") == "capability": + event_type = event.get("event") + if event_type in ( + "capability_leased", + "capability_revoked", + "lease_granted", + "physical_revocation", + ): + summary = {"event": event_type, "ts": event.get("ts")} + if "physical_sync" in event: + summary["physical_sync"] = event["physical_sync"] + if "physical_revocation" in event: + summary["physical_revocation"] = event[ + "physical_revocation" + ] + if "capability_ref" in event: + summary["capability_ref"] = event["capability_ref"] + if "lease_id" in event: + summary["lease_id"] = event["lease_id"] + _print_json(summary) + except (json.JSONDecodeError, KeyError): + continue + sys.stdout.flush() time.sleep(args.interval) diff --git a/capability-runtime/src/capability_runtime/daemon.py b/capability-runtime/src/capability_runtime/daemon.py index 96e79ac4..990d644b 100644 --- a/capability-runtime/src/capability_runtime/daemon.py +++ b/capability-runtime/src/capability_runtime/daemon.py @@ -12,7 +12,7 @@ from capability_runtime.catalog import LifecycleState from capability_runtime.netbird_client import MockNetBirdClient, NetBirdClient, RestNetBirdClient -from capability_runtime.network import NetworkBinding +from capability_runtime.network import NetworkBinding, sync_mode_from_catalog from capability_runtime.rpc.dispatcher import RpcDispatcher from capability_runtime.rpc.transports.unix import UnixRpcServer from capability_runtime.route_watcher import RouteDisappearanceWatcher @@ -93,6 +93,7 @@ def load_catalog_into_runtime(runtime: CapabilityRuntime, catalog_path: Path) -> entry["peer_id"], entry["network"], entry.get("route_id"), + sync_mode_from_catalog(entry), ) runtime.register_network_capability( name, diff --git a/cli/templates/devcontainer/sandcat/capability-catalog.json b/cli/templates/devcontainer/sandcat/capability-catalog.json index 219ca43c..5610295b 100644 --- a/cli/templates/devcontainer/sandcat/capability-catalog.json +++ b/cli/templates/devcontainer/sandcat/capability-catalog.json @@ -11,7 +11,8 @@ "type": "network", "peer_id": "peer-placeholder", "network": "10.8.0.0/24", - "route_id": "route-placeholder" + "route_id": "route-placeholder", + "sync_mode": "route_enable" } ] } diff --git a/cli/test/capability/capability.bats b/cli/test/capability/capability.bats index a85b3b5d..12fb2c21 100644 --- a/cli/test/capability/capability.bats +++ b/cli/test/capability/capability.bats @@ -77,3 +77,22 @@ teardown() { assert_success assert_output --partial "capability.check" } + +@test "capability lease then revoke flow execs both commands" { + capability_compose_exec() { + echo "EXEC $*" + } + export -f capability_compose_exec + + # Lease + run capability_lease --ref cap-reach-api --justification "test flow" + assert_success + assert_output --partial "capability.lease" + assert_output --partial "cap-reach-api" + + # Revoke + run capability_revoke --ref cap-reach-api --reason "test complete" + assert_success + assert_output --partial "capability.revoke" + assert_output --partial "cap-reach-api" +} From 8491d223065007abe8ad8e78a167642df248121e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Tue, 30 Jun 2026 13:43:36 +0000 Subject: [PATCH 051/138] docs: Phase 3c NetBird policy sync grant/revoke Co-authored-by: Cursor --- CONTEXT.md | 13 ++++++---- capability-runtime/README.md | 37 ++++++++++++++++++++------ cli/README.md | 50 +++++++++++++++++++++++++++++++++++- 3 files changed, 86 insertions(+), 14 deletions(-) diff --git a/CONTEXT.md b/CONTEXT.md index 209b6023..1eaa6705 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -24,10 +24,10 @@ Compose sidecar wiring: `capability-runtime` service, Capability MCP for agents, _Avoid_: Phase 4 -**Phase 3c (Dynamic L7 Policy)**: -mitmproxy ↔ capability-runtime loop: dynamic HTTP allowlist from bundle, L7 execution events, quota-driven revoke. **DRAFT spec:** `docs/superpowers/specs/2026-06-23-capability-dynamic-l7-policy-phase3c-design.md`. +**Phase 3c (NetBird Policy Sync)**: +On lease grant/revoke, `CapabilityRuntime` synchronizes NetBird physical bindings (routes, future ACL) via `enable_binding` / `disable_binding`. Default revoke disables the route and keeps the peer. mitmproxy remains egress inspection only. **Spec:** `docs/superpowers/specs/2026-06-30-capability-netbird-policy-sync-phase3c-design.md`. Plan: `docs/superpowers/plans/2026-06-30-capability-netbird-policy-sync-phase3c.md`. -_Avoid_: conflating with Phase 3b sidecar work +_Avoid_: conflating with Phase 3b sidecar work; per-request mitmproxy L7 allowlist (see deprecated spec below) **Phase 4 (Comparative Evaluation)**: Controlled experiments baseline vs treatment; metrics and ablations. **DRAFT spec:** `docs/superpowers/specs/2026-06-23-capability-comparative-evaluation-phase4-design.md`. Does not implement new enforcement — measures Phases 0–3c. @@ -47,7 +47,8 @@ The link between a network capability and concrete NetBird identifiers (`peer_id - A **Capability Bundle** includes zero or more network capabilities when their lifecycle state is Visible or Leased - Each network capability has exactly one **Network Binding** -- **Logical Revocation** triggers **Physical Revocation** via `NetBirdRevocationBackend` +- **Logical Revocation** triggers **Physical Revocation** via `NetBirdRevocationBackend` (`disable_binding` by default; `peer_remove` when catalog `sync_mode` requires) +- **Logical Grant (lease)** triggers **Physical Enable** via `grant_binding` → `enable_binding` - **Physical Revocation** outside the runtime triggers **Logical Revocation** via `RouteDisappearanceWatcher` ## Example dialogue @@ -82,4 +83,6 @@ _Avoid_: client-provided agent_id, multi-agent per container (Phase 3b) - NetBird **implementation** runs inside `wg-client` (`wt0` overlay); the original plan described a separate `netbird` sync sidecar for `wg0` — these are different layers. - Phase 3b plan is formal and **implemented** (Tasks 1–8): `docs/superpowers/plans/2026-06-23-capability-sandcat-phase3b.md`. -- Phase 3c and Phase 4 are **DRAFT specs** only (not approved for implementation): `docs/superpowers/specs/2026-06-23-capability-dynamic-l7-policy-phase3c-design.md`, `docs/superpowers/specs/2026-06-23-capability-comparative-evaluation-phase4-design.md`. +- Phase 3c NetBird policy sync is **implemented** — supersedes the deprecated L7 policy design. +- **DEPRECATED:** `docs/superpowers/specs/2026-06-23-capability-dynamic-l7-policy-phase3c-design.md` — per-request mitmproxy L7 allowlist from bundle; replaced by NetBird policy sync spec above. +- Phase 4 is a **DRAFT spec** only (not approved for implementation): `docs/superpowers/specs/2026-06-23-capability-comparative-evaluation-phase4-design.md`. diff --git a/capability-runtime/README.md b/capability-runtime/README.md index 80fc72f6..8066ba22 100644 --- a/capability-runtime/README.md +++ b/capability-runtime/README.md @@ -37,6 +37,14 @@ Phase 3 realizes this thesis by binding each network capability to a NetBird pee - Operator CLI: `sandcat capability` via `docker compose exec capability-runtime` - Catalog loaded at sidecar startup from `CAPABILITY_CATALOG_JSON` — no `register_*` over RPC +**In scope (Phase 3c — NetBird policy sync):** + +- **Grant sync on lease:** `request_capability_lease` calls `NetBirdClient.enable_binding()` via `grant_binding` when the capability has a `NetworkBinding` +- **Revoke sync on revoke / quota / TTL:** `revoke_capability` and lease exhaustion call `disable_binding()` — default `sync_mode=route_enable` disables the route and keeps the peer enrolled +- **`sync_mode` on `NetworkBinding`:** `route_enable` (default), `acl_policy` (stub), `peer_remove` (Phase 3 break-glass — deletes peer on revoke) +- Grant failure rolls back the lease (fail closed); observability emits `physical_sync: enabled` on successful network lease +- mitmproxy remains egress inspection only — no per-request L7 allowlist from bundle + **Out of scope / known limitations:** - Token budget enforcement (`token_budget` is stored but not decremented) @@ -44,24 +52,36 @@ Phase 3 realizes this thesis by binding each network capability to a NetBird pee - `TaskContext`-driven visibility rules - Non-tool capability kinds (`rules`, `skills`, etc.) in bundles - Multi-agent leasing on the same capability (global `LEASED` catalog state) -- MCP workload gateway, NetBird ACL/reverse-proxy bindings, host-published HTTP RPC (Phase 3c+) +- Full NetBird ACL/group API (`acl_policy` sync_mode is stubbed) +- MCP workload gateway, host-published HTTP RPC (Phase 3d+) ## NetBird bridge (logical ↔ physical) -Phase 3 connects the capability runtime to the sandcat NetBird deployment model described in [NetBird dynamic WireGuard plan](../../docs/superpowers/plans/2026-06-15-netbird-dynamic-wireguard.md): +Phase 3 connects the capability runtime to the sandcat NetBird deployment model described in [NetBird dynamic WireGuard plan](../../docs/superpowers/plans/2026-06-15-netbird-dynamic-wireguard.md). Phase 3c adds bidirectional sync on grant and revoke via `enable_binding` / `disable_binding`. ``` +Grant (lease) Physical path (sandcat) +───────────── ─────────────────────── +request_capability_lease(ref) + → grant_network_binding + → enable_binding(sync_mode) → NetBird management API + → catalog LEASED → route enabled / created + → emit physical_sync: enabled → wg-client: route on wt0 + │ + │ (on enable_binding failure: rollback lease, revert catalog) + ▼ + Logical revoke (runtime) Physical path (sandcat) ───────────────────────── ─────────────────────── revoke_capability(ref, reason) → NetBirdRevocationBackend - → remove_peer / remove_route → NetBird management API - → catalog REVOKED → netbird container writes peers.conf - → emit physical_revocation → wg-client: wg syncconf wg0 - → agent loses route to endpoint + → disable_binding(sync_mode) → NetBird management API + → catalog REVOKED → route disabled (default) + → emit physical_revocation → wg-client drops route on wt0 + → peer remains enrolled (route_enable) ``` -When revocation originates from the runtime, `NetBirdRevocationBackend.revoke_binding()` removes the bound peer and/or route through the injected `NetBirdClient`. In production this maps to NetBird API calls that eventually rewrite `peers.conf` and trigger `wg syncconf` in `wg-client` — the agent loses routing without a container restart. +When revocation originates from the runtime, `NetBirdRevocationBackend.revoke_binding()` calls `disable_binding()` through the injected `NetBirdClient`. Default `sync_mode=route_enable` disables the route but keeps the peer — the agent loses routing without unenrolling the mesh peer. Set `sync_mode=peer_remove` in the catalog for Phase 3 break-glass behavior (delete peer on revoke). ## RouteDisappearanceWatcher (physical → logical) @@ -160,7 +180,8 @@ PYTHONPATH=src:. python poc/network_route_demo.py | `catalog.py` | Lifecycle states, network binding storage | | `network.py` | `NetworkBinding`, `PhysicalRevocationBackend` protocol | | `netbird_client.py` | `NetBirdClient` protocol, mock and REST implementations | -| `netbird_backend.py` | `NetBirdRevocationBackend` — logical revoke → peer/route removal | +| `netbird_backend.py` | `NetBirdRevocationBackend` — grant/revoke via `enable_binding` / `disable_binding` | +| `netbird_sync.py` | Grant/revoke orchestration helpers for network bindings | | `route_watcher.py` | `RouteDisappearanceWatcher` — physical disappearance → logical revoke | | `lease.py` / `revoke.py` | Grant, quota, revocation | | `policy.py` | PoC lease parameters (not in core runtime) | diff --git a/cli/README.md b/cli/README.md index 9ce04b1c..92d43180 100644 --- a/cli/README.md +++ b/cli/README.md @@ -491,7 +491,7 @@ sandcat capability check --context '{}' # Lease a network capability (triggers NetBird peer/route via sidecar) sandcat capability lease --ref cap-reach-api --justification "need API access" -# Revoke (operator-only; removes NetBird peer/route) +# Revoke (operator-only; disables NetBird route by default, keeps peer) sandcat capability revoke --ref cap-reach-api --reason policy # Foreground route-watcher poll loop (debugging) @@ -514,6 +514,54 @@ sandcat capability demo agent-supplied `agent_id` parameters are ignored - Catalog is loaded at sidecar startup from config — not registerable over RPC +### Phase 3c grant/revoke flow + +When the sidecar loads a network capability from the catalog, each entry may include a `sync_mode` that controls how NetBird physical state is synchronized on lease and revoke: + +| `sync_mode` | On lease (`enable_binding`) | On revoke (`disable_binding`) | +|-------------|----------------------------|-------------------------------| +| `route_enable` (default) | Enable or create NetBird route for the binding | Disable route; peer stays enrolled | +| `peer_remove` | No-op (peer already enrolled) | Delete peer (Phase 3 break-glass) | +| `acl_policy` | Stub — future ACL/group sync | Stub | + +```bash +# Lease triggers enable_binding → route visible on wt0 +sandcat capability lease --ref cap-reach-api --justification "need API access" +sandcat netbird status # route enabled + +# Revoke triggers disable_binding → route gone, peer remains +sandcat capability revoke --ref cap-reach-api --reason done +sandcat netbird status # route disabled; peer still listed +``` + +Grant failure rolls back the lease (fail closed). Only the operator admin socket can revoke; agents cannot trigger `enable_binding` or `disable_binding`. + +#### Capability catalog schema (network entries) + +Network capabilities in `capability-catalog.json` (mounted as `CAPABILITY_CATALOG_JSON` at sidecar startup): + +```json +{ + "capabilities": [ + { + "name": "reach_api", + "ref": "cap-reach-api", + "type": "network", + "peer_id": "", + "network": "10.8.0.0/24", + "route_id": "", + "sync_mode": "route_enable" + } + ] +} +``` + +- `peer_id`, `network` — required NetBird identifiers for the binding +- `route_id` — optional; if omitted, `enable_binding` creates a route via the NetBird API and stores the returned id +- `sync_mode` — optional; defaults to `route_enable`. Use `peer_remove` only when revoke must delete the peer (legacy Phase 3 behavior) + +Tool capabilities (`type: "tool"`) do not use `sync_mode` or binding fields. + ## Directory Structure Each module is contained in its own directory under `cli/libexec/`. From 636e6a2bfe0f3f5bfa99cc31bc689bcc01b7fabc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Thu, 2 Jul 2026 13:21:04 +0000 Subject: [PATCH 052/138] fix(cli): fix capability sidecar startup and agent MCP connectivity --- capability-runtime/README.md | 2 +- .../capability_runtime.egg-info/SOURCES.txt | 3 ++ .../src/capability_runtime/daemon.py | 3 ++ .../capability_runtime/rpc/transports/unix.py | 13 ++++- capability-runtime/tests/test_rpc_unix.py | 14 ++++++ cli/lib/composefile.bash | 47 +++++++++++++++++++ cli/templates/devcontainer/devcontainer.json | 4 +- .../sandcat/scripts/capability-mcp-bridge.sh | 2 +- cli/test/composefile/capability.bats | 7 +++ 9 files changed, 91 insertions(+), 4 deletions(-) diff --git a/capability-runtime/README.md b/capability-runtime/README.md index 8066ba22..22dd4867 100644 --- a/capability-runtime/README.md +++ b/capability-runtime/README.md @@ -160,7 +160,7 @@ Mutating APIs require `caller` to match the lease-bound `agent_id` (`CallerIdent - `agent_id` in RPC/MCP params is overwritten with `SANDCAT_AGENT_ID` on the agent surface - Catalog registration happens at sidecar startup only — not over RPC -**Not yet addressed:** cryptographic trace signing, network-authenticated control plane, cross-process cryptographic auth (Unix permissions + container split sufficient for 3b). +**Not yet addressed:** cryptographic trace signing, network-authenticated control plane, cross-process cryptographic auth (Unix permissions + container split sufficient for 3b), and replacing the Python-based agent MCP bridge with a native binary to remove Python from non-Python agent images. ## Quick start diff --git a/capability-runtime/src/capability_runtime.egg-info/SOURCES.txt b/capability-runtime/src/capability_runtime.egg-info/SOURCES.txt index 178c738e..9a8d1492 100644 --- a/capability-runtime/src/capability_runtime.egg-info/SOURCES.txt +++ b/capability-runtime/src/capability_runtime.egg-info/SOURCES.txt @@ -11,6 +11,7 @@ src/capability_runtime/lease.py src/capability_runtime/mcp_adapter.py src/capability_runtime/netbird_backend.py src/capability_runtime/netbird_client.py +src/capability_runtime/netbird_sync.py src/capability_runtime/network.py src/capability_runtime/observability.py src/capability_runtime/policy.py @@ -37,12 +38,14 @@ tests/test_catalog.py tests/test_daemon_integration.py tests/test_discover.py tests/test_errors.py +tests/test_grant_revoke_integration.py tests/test_lease.py tests/test_mcp_adapter.py tests/test_mcp_bridge.py tests/test_mcp_server.py tests/test_netbird_backend.py tests/test_netbird_client.py +tests/test_netbird_sync.py tests/test_network.py tests/test_observability.py tests/test_poc_create_pr.py diff --git a/capability-runtime/src/capability_runtime/daemon.py b/capability-runtime/src/capability_runtime/daemon.py index 990d644b..f317dba5 100644 --- a/capability-runtime/src/capability_runtime/daemon.py +++ b/capability-runtime/src/capability_runtime/daemon.py @@ -153,6 +153,8 @@ def __init__(self, config: DaemonConfig) -> None: surface="agent", bound_agent_id=config.agent_id, ), + # Sidecar runs as root; agent container connects as vscode on shared volume. + socket_mode=0o666, ) self._admin_server = UnixRpcServer( config.admin_socket, @@ -162,6 +164,7 @@ def __init__(self, config: DaemonConfig) -> None: bound_agent_id="operator", watcher=self._watcher, ), + socket_mode=0o600, ) self._running = False diff --git a/capability-runtime/src/capability_runtime/rpc/transports/unix.py b/capability-runtime/src/capability_runtime/rpc/transports/unix.py index af98e56f..0dc1b4a8 100644 --- a/capability-runtime/src/capability_runtime/rpc/transports/unix.py +++ b/capability-runtime/src/capability_runtime/rpc/transports/unix.py @@ -3,6 +3,7 @@ from __future__ import annotations import json +import os import socket import threading from pathlib import Path @@ -15,9 +16,16 @@ class UnixRpcServer: """Accept AF_UNIX connections, handle one JSON-RPC request per connection.""" - def __init__(self, socket_path: Path, dispatcher: RpcDispatcher) -> None: + def __init__( + self, + socket_path: Path, + dispatcher: RpcDispatcher, + *, + socket_mode: int | None = None, + ) -> None: self._socket_path = socket_path self._dispatcher = dispatcher + self._socket_mode = socket_mode self._stop_event = threading.Event() self._thread: threading.Thread | None = None self._server: socket.socket | None = None @@ -51,12 +59,15 @@ def stop(self) -> None: def _serve(self) -> None: self._socket_path.parent.mkdir(parents=True, exist_ok=True) + os.chmod(self._socket_path.parent, 0o755) if self._socket_path.exists(): self._socket_path.unlink() server = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) self._server = server server.bind(str(self._socket_path)) + if self._socket_mode is not None: + os.chmod(self._socket_path, self._socket_mode) server.listen(5) server.settimeout(1.0) diff --git a/capability-runtime/tests/test_rpc_unix.py b/capability-runtime/tests/test_rpc_unix.py index ccef30de..d8597eb6 100644 --- a/capability-runtime/tests/test_rpc_unix.py +++ b/capability-runtime/tests/test_rpc_unix.py @@ -89,3 +89,17 @@ def test_invalid_json_returns_parse_error(tmp_path, mock_dispatcher): finally: server.stop() + +def test_socket_mode_allows_non_owner_connect(tmp_path, mock_dispatcher): + sock_path = tmp_path / "agent.sock" + server = UnixRpcServer(sock_path, mock_dispatcher, socket_mode=0o666) + server.start() + try: + _wait_for_socket(sock_path) + assert oct(sock_path.stat().st_mode & 0o777) == "0o666" + client = UnixRpcClient(sock_path) + response = client.call("capability.check", {"context": {}}) + assert response["result"] == {"ok": True} + finally: + server.stop() + diff --git a/cli/lib/composefile.bash b/cli/lib/composefile.bash index 4cd2b975..78f6d369 100644 --- a/cli/lib/composefile.bash +++ b/cli/lib/composefile.bash @@ -719,4 +719,51 @@ enable_capability() { yq_array="[${yq_array%,}]" yq -i ".services.agent.environment = ((.services.agent.environment // []) + ${yq_array})" "$compose_file" fi + + # Start capability-runtime whenever agent starts (devcontainer reopen, sandcat run). + local has_dep + has_dep=$(yq '.services.agent.depends_on.capability-runtime.condition // ""' "$compose_file") + if [[ "$has_dep" != "service_started" ]]; then + yq -i '.services.agent.depends_on.capability-runtime.condition = "service_started"' "$compose_file" + fi + + _enable_capability_mcp_config "$(dirname "$compose_dir")" "$compose_dir/devcontainer.json" +} + +# Writes .cursor/mcp.json and patches devcontainer remoteEnv for capability MCP. +# Args: +# $1 - Project root (parent of .devcontainer) +# $2 - Path to devcontainer.json +_enable_capability_mcp_config() { + local project_path=$1 + local devcontainer_json=$2 + local mcp_dir="$project_path/.cursor" + + mkdir -p "$mcp_dir" + cat >"$mcp_dir/mcp.json" <<'EOF' +{ + "mcpServers": { + "sandcat-capability": { + "command": "capability-mcp-bridge", + "args": [], + "env": { + "SANDCAT_AGENT_ID": "devcontainer-agent", + "CAPABILITY_AGENT_SOCKET": "/run/sandcat-capability/agent.sock" + } + } + } +} +EOF + + if [[ ! -f "$devcontainer_json" ]] || grep -q 'SANDCAT_AGENT_ID' "$devcontainer_json"; then + return 0 + fi + + # JSONC devcontainer.json — inject remoteEnv keys after GIT_ASKPASS. + sed -i.bak \ + 's|"GIT_ASKPASS": ""|"GIT_ASKPASS": "",\ + "SANDCAT_AGENT_ID": "devcontainer-agent",\ + "CAPABILITY_AGENT_SOCKET": "/run/sandcat-capability/agent.sock"|' \ + "$devcontainer_json" + rm -f "${devcontainer_json}.bak" } diff --git a/cli/templates/devcontainer/devcontainer.json b/cli/templates/devcontainer/devcontainer.json index 4f9bac46..df1f77b5 100644 --- a/cli/templates/devcontainer/devcontainer.json +++ b/cli/templates/devcontainer/devcontainer.json @@ -23,7 +23,9 @@ "remoteEnv": { "SSH_AUTH_SOCK": "", "GPG_AGENT_INFO": "", - "GIT_ASKPASS": "" + "GIT_ASKPASS": "", + "SANDCAT_AGENT_ID": "devcontainer-agent", + "CAPABILITY_AGENT_SOCKET": "/run/sandcat-capability/agent.sock" }, // __CUSTOMIZATIONS_START__ "customizations": { diff --git a/cli/templates/devcontainer/sandcat/scripts/capability-mcp-bridge.sh b/cli/templates/devcontainer/sandcat/scripts/capability-mcp-bridge.sh index a20d16df..12414286 100644 --- a/cli/templates/devcontainer/sandcat/scripts/capability-mcp-bridge.sh +++ b/cli/templates/devcontainer/sandcat/scripts/capability-mcp-bridge.sh @@ -1,2 +1,2 @@ #!/bin/bash -exec python -m capability_runtime.mcp.bridge +exec /usr/local/bin/python -m capability_runtime.mcp.bridge diff --git a/cli/test/composefile/capability.bats b/cli/test/composefile/capability.bats index 966d22b1..e7465c28 100644 --- a/cli/test/composefile/capability.bats +++ b/cli/test/composefile/capability.bats @@ -32,6 +32,13 @@ teardown() { [ "$status" -eq 0 ] } +@test "enable_capability adds capability-runtime to agent depends_on" { + enable_capability "$COMPOSE_DIR" + run yq '.services.agent.depends_on.capability-runtime.condition' "$COMPOSE_DIR/compose-all.yml" + assert_success + assert_output "service_started" +} + @test "enable_capability is idempotent" { enable_capability "$COMPOSE_DIR" enable_capability "$COMPOSE_DIR" From 95ab2d6bec8227e55f4b4243073b0dab88552607 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Fri, 3 Jul 2026 06:31:06 +0000 Subject: [PATCH 053/138] fix(capability-runtime): disable NetBird binding on revoke by lease ID --- .../src/capability_runtime/runtime.py | 9 ++++ .../tests/test_revoke_by_lease_network.py | 50 +++++++++++++++++++ 2 files changed, 59 insertions(+) create mode 100644 capability-runtime/tests/test_revoke_by_lease_network.py diff --git a/capability-runtime/src/capability_runtime/runtime.py b/capability-runtime/src/capability_runtime/runtime.py index 1695a7d2..d782a1a1 100644 --- a/capability-runtime/src/capability_runtime/runtime.py +++ b/capability-runtime/src/capability_runtime/runtime.py @@ -327,12 +327,21 @@ def revoke_capability( physical_revocation = False if isinstance(target, LeaseId): + lease = self.lease_manager.get_lease(target) + if lease is not None: + binding = self.catalog.get_network_binding(lease.capability_ref) + if binding is not None and self._netbird_backend is not None: + from capability_runtime.netbird_sync import revoke_network_binding + revoke_network_binding(self._netbird_backend, binding, reason) + physical_revocation = True self.revocation_manager.revoke_by_lease(target, reason) + self._bundle_version += 1 self.observability.emit_capability_event( { "event": "capability_revoked", "lease_id": target.value, "reason": reason, + "physical_revocation": physical_revocation, } ) else: diff --git a/capability-runtime/tests/test_revoke_by_lease_network.py b/capability-runtime/tests/test_revoke_by_lease_network.py new file mode 100644 index 00000000..61226e2f --- /dev/null +++ b/capability-runtime/tests/test_revoke_by_lease_network.py @@ -0,0 +1,50 @@ +# capability-runtime/tests/test_revoke_by_lease_network.py +"""Revoke by lease ID must disable network bindings.""" + +from capability_runtime.catalog import LifecycleState +from capability_runtime.netbird_client import MockNetBirdClient +from capability_runtime.network import NetworkBinding, SyncMode +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity, CapabilityRef + +_OPERATOR = AgentIdentity("operator") + + +def test_revoke_by_lease_id_disables_network_route(tmp_path): + client = MockNetBirdClient( + peers=[{"id": "peer-abc"}], + routes=[{"id": "route-1", "network": "10.8.0.0/24", "peer": "peer-abc", "enabled": True}], + ) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-rl", 1, netbird_client=client) + agent = AgentIdentity("agent-1") + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", "route-1", sync_mode=SyncMode.ROUTE_ENABLE) + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + + decision = runtime.request_capability_lease(agent, agent, ref, "need api") + version_before = runtime._bundle_version + runtime.revoke_capability(_OPERATOR, decision.lease_id, "operator revoke") + + routes = [r for r in client.list_routes() if r["id"] == "route-1"] + assert routes[0]["enabled"] is False + assert client.peer_exists("peer-abc") + bundle = runtime.check_current_capabilities(agent, {}) + assert "reach_api" not in [n.name for n in bundle.networks] + assert runtime._bundle_version > version_before + + +def test_revoke_by_lease_id_skips_netbird_for_tool_capability(tmp_path): + """Tool leases must not call NetBird on revoke-by-lease-ID.""" + client = MockNetBirdClient(peers=[{"id": "peer-xyz"}], routes=[]) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-rl-tool", 2, netbird_client=client) + agent = AgentIdentity("agent-1") + ref = CapabilityRef("cap-create-pr") + runtime.catalog.register("create_pr", ref, LifecycleState.VISIBLE) + + decision = runtime.request_capability_lease(agent, agent, ref, "need pr") + runtime.revoke_capability(_OPERATOR, decision.lease_id, "done") + + assert client.list_routes() == [] + assert client.peer_exists("peer-xyz") + bundle = runtime.check_current_capabilities(agent, {}) + assert "create_pr" not in [t.name for t in bundle.tools] From 558c1ce8f92c0c9f70610a688f12ce15a9994a20 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Fri, 3 Jul 2026 06:35:08 +0000 Subject: [PATCH 054/138] test(cli): use yq -e for capability compose existence checks --- cli/test/composefile/capability.bats | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/cli/test/composefile/capability.bats b/cli/test/composefile/capability.bats index e7465c28..b8beef61 100644 --- a/cli/test/composefile/capability.bats +++ b/cli/test/composefile/capability.bats @@ -16,20 +16,17 @@ teardown() { @test "enable_capability adds capability-runtime service include" { enable_capability "$COMPOSE_DIR" - run yq '.include[] | select(.path == "sandcat/compose-capability.yml")' "$COMPOSE_DIR/compose-all.yml" - [ "$status" -eq 0 ] + yq -e '.include[] | select(.path == "sandcat/compose-capability.yml")' "$COMPOSE_DIR/compose-all.yml" } @test "enable_capability sets SANDCAT_AGENT_ID on agent service" { enable_capability "$COMPOSE_DIR" - run yq '.services.agent.environment[] | select(. == "SANDCAT_AGENT_ID=devcontainer-agent")' "$COMPOSE_DIR/compose-all.yml" - [ "$status" -eq 0 ] + yq -e '.services.agent.environment[] | select(. == "SANDCAT_AGENT_ID=devcontainer-agent")' "$COMPOSE_DIR/compose-all.yml" } @test "enable_capability mounts capability socket outside wg-runtime path" { enable_capability "$COMPOSE_DIR" - run yq '.services.agent.volumes[] | select(. == "capability-socket:/run/sandcat-capability:ro")' "$COMPOSE_DIR/compose-all.yml" - [ "$status" -eq 0 ] + yq -e '.services.agent.volumes[] | select(. == "capability-socket:/run/sandcat-capability:ro")' "$COMPOSE_DIR/compose-all.yml" } @test "enable_capability adds capability-runtime to agent depends_on" { From ef7a7fbcc3f8bea51c101dedf85534306b924fa3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Fri, 3 Jul 2026 06:36:09 +0000 Subject: [PATCH 055/138] fix(capability-runtime): allow re-lease after revoke or expiry --- .../src/capability_runtime/runtime.py | 9 ++++++- .../tests/test_re_lease_after_revoke.py | 27 +++++++++++++++++++ 2 files changed, 35 insertions(+), 1 deletion(-) create mode 100644 capability-runtime/tests/test_re_lease_after_revoke.py diff --git a/capability-runtime/src/capability_runtime/runtime.py b/capability-runtime/src/capability_runtime/runtime.py index d782a1a1..7e21e11c 100644 --- a/capability-runtime/src/capability_runtime/runtime.py +++ b/capability-runtime/src/capability_runtime/runtime.py @@ -232,7 +232,14 @@ def request_capability_lease( _assert_caller(caller, agent_id) # Check capability exists state = self.catalog.get_state(capability_ref) - if state not in (LifecycleState.DECLARED, LifecycleState.DISCOVERABLE, LifecycleState.VISIBLE): + leasable = { + LifecycleState.DECLARED, + LifecycleState.DISCOVERABLE, + LifecycleState.VISIBLE, + LifecycleState.REVOKED, + LifecycleState.EXPIRED, + } + if state not in leasable: raise CapabilityUnknown(capability_ref) # Save the original state for rollback in case of failure diff --git a/capability-runtime/tests/test_re_lease_after_revoke.py b/capability-runtime/tests/test_re_lease_after_revoke.py new file mode 100644 index 00000000..c3d1f68b --- /dev/null +++ b/capability-runtime/tests/test_re_lease_after_revoke.py @@ -0,0 +1,27 @@ +# capability-runtime/tests/test_re_lease_after_revoke.py +from capability_runtime.catalog import LifecycleState +from capability_runtime.netbird_client import MockNetBirdClient +from capability_runtime.network import NetworkBinding, SyncMode +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity, CapabilityRef + +_OPERATOR = AgentIdentity("operator") + + +def test_re_lease_after_revoke_by_ref(tmp_path): + client = MockNetBirdClient(peers=[{"id": "peer-abc"}], routes=[]) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-re", 2, netbird_client=client) + agent = AgentIdentity("agent-1") + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", None, sync_mode=SyncMode.ROUTE_ENABLE) + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + + runtime.request_capability_lease(agent, agent, ref, "first lease") + runtime.revoke_capability(_OPERATOR, ref, "done") + assert runtime.catalog.get_state(ref) == LifecycleState.REVOKED + + decision2 = runtime.request_capability_lease(agent, agent, ref, "second lease") + assert decision2.lease_id is not None + assert len(client.list_routes()) == 1 + bundle = runtime.check_current_capabilities(agent, {}) + assert "reach_api" in [n.name for n in bundle.networks] From 4edbca5c0a6d9bcf1094ce2a7a65a8bb4fe4289b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Fri, 3 Jul 2026 06:39:04 +0000 Subject: [PATCH 056/138] fix(capability-runtime): mark leases revoked on ref-based revoke --- capability-runtime/src/capability_runtime/revoke.py | 3 +++ capability-runtime/tests/test_re_lease_after_revoke.py | 1 + 2 files changed, 4 insertions(+) diff --git a/capability-runtime/src/capability_runtime/revoke.py b/capability-runtime/src/capability_runtime/revoke.py index 9b91912e..75b0104e 100644 --- a/capability-runtime/src/capability_runtime/revoke.py +++ b/capability-runtime/src/capability_runtime/revoke.py @@ -20,6 +20,9 @@ def revoke_by_lease(self, lease_id: LeaseId, reason: str) -> None: def revoke_by_ref(self, capability_ref: CapabilityRef, reason: str) -> None: self._catalog.set_state(capability_ref, LifecycleState.REVOKED) + for lease_id, lease in self._lease_manager._leases.items(): + if lease.capability_ref == capability_ref: + self._revoked_leases.add(lease_id) def is_revoked(self, capability_ref: CapabilityRef) -> bool: try: diff --git a/capability-runtime/tests/test_re_lease_after_revoke.py b/capability-runtime/tests/test_re_lease_after_revoke.py index c3d1f68b..1b7ec50a 100644 --- a/capability-runtime/tests/test_re_lease_after_revoke.py +++ b/capability-runtime/tests/test_re_lease_after_revoke.py @@ -25,3 +25,4 @@ def test_re_lease_after_revoke_by_ref(tmp_path): assert len(client.list_routes()) == 1 bundle = runtime.check_current_capabilities(agent, {}) assert "reach_api" in [n.name for n in bundle.networks] + assert bundle.networks[0].lease_id == decision2.lease_id From 2876774bb626990eaa2b13d3e36cf99c8f0d8ca2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Fri, 3 Jul 2026 06:42:45 +0000 Subject: [PATCH 057/138] feat(capability-runtime): reconcile route disappearance in watcher --- .../src/capability_runtime/netbird_client.py | 18 +++++ .../src/capability_runtime/route_watcher.py | 71 ++++++++++++++++++- .../tests/test_route_watcher_routes.py | 69 ++++++++++++++++++ 3 files changed, 157 insertions(+), 1 deletion(-) create mode 100644 capability-runtime/tests/test_route_watcher_routes.py diff --git a/capability-runtime/src/capability_runtime/netbird_client.py b/capability-runtime/src/capability_runtime/netbird_client.py index 4983b67e..9a2f3194 100644 --- a/capability-runtime/src/capability_runtime/netbird_client.py +++ b/capability-runtime/src/capability_runtime/netbird_client.py @@ -23,6 +23,8 @@ def peer_exists(self, peer_id: str) -> bool: ... def route_exists(self, route_id: str) -> bool: ... + def get_route_state(self, binding: NetworkBinding) -> str: ... + def enable_binding(self, binding: NetworkBinding) -> NetworkBinding: ... def disable_binding(self, binding: NetworkBinding) -> None: ... @@ -56,6 +58,14 @@ def peer_exists(self, peer_id: str) -> bool: def route_exists(self, route_id: str) -> bool: return any(route.get("id") == route_id for route in self._routes) + def get_route_state(self, binding: NetworkBinding) -> str: + if not binding.route_id: + return "missing" + for route in self._routes: + if route.get("id") == binding.route_id: + return "disabled" if route.get("enabled") is False else "enabled" + return "missing" + def enable_binding(self, binding: NetworkBinding) -> NetworkBinding: if binding.sync_mode is SyncMode.PEER_REMOVE: return binding @@ -141,6 +151,14 @@ def peer_exists(self, peer_id: str) -> bool: def route_exists(self, route_id: str) -> bool: return any(route.get("id") == route_id for route in self.list_routes()) + def get_route_state(self, binding: NetworkBinding) -> str: + if not binding.route_id: + return "missing" + for route in self.list_routes(): + if route.get("id") == binding.route_id: + return "disabled" if route.get("enabled") is False else "enabled" + return "missing" + def enable_binding(self, binding: NetworkBinding) -> NetworkBinding: if binding.sync_mode is SyncMode.PEER_REMOVE: return binding diff --git a/capability-runtime/src/capability_runtime/route_watcher.py b/capability-runtime/src/capability_runtime/route_watcher.py index 0b8ff53e..3b66dea7 100644 --- a/capability-runtime/src/capability_runtime/route_watcher.py +++ b/capability-runtime/src/capability_runtime/route_watcher.py @@ -6,8 +6,12 @@ if TYPE_CHECKING: from capability_runtime.netbird_client import NetBirdClient + from capability_runtime.network import NetworkBinding from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.catalog import LifecycleState +from capability_runtime.network import SyncMode + class RouteDisappearanceWatcher: """Monitor NetBird peers and trigger logical revoke when they disappear. @@ -28,12 +32,53 @@ def __init__(self, runtime: CapabilityRuntime, client: NetBirdClient): self._runtime = runtime self._client = client + def _should_watch_binding(self, binding: NetworkBinding) -> bool: + """Determine if a binding should be watched for route disappearance. + + A binding should be watched if: + - Its catalog state is VISIBLE or LEASED, OR + - It has an active lease (non-revoked, non-expired, non-exhausted) + + DECLARED and DISCOVERABLE bindings are not watched. + + Args: + binding: The network binding to check + + Returns: + True if the binding should be watched + """ + from datetime import datetime, timezone + + # Check catalog state + try: + state = self._runtime.catalog.get_state(binding.capability_ref) + if state in (LifecycleState.VISIBLE, LifecycleState.LEASED): + return True + except Exception: + pass + + # Check for active leases + now = datetime.now(timezone.utc) + for lease_id, lease in self._runtime.lease_manager._leases.items(): + if lease.capability_ref == binding.capability_ref: + if ( + not self._runtime.lease_manager.is_expired(lease_id, now) + and not self._runtime.revocation_manager.is_lease_revoked(lease_id) + and not self._runtime.lease_manager.is_exhausted(lease_id) + ): + return True + + return False + def poll_once(self) -> None: """Poll once for disappeared peers and revoke their capabilities. For each network binding registered in the catalog, checks if the associated peer_id still exists in NetBird. If not, performs a logical-only revocation via runtime.revoke_from_physical(). + + For ROUTE_ENABLE bindings with a route_id, also checks if the route is + disabled or missing. If so, performs logical-only revocation. """ # Collect all network bindings from catalog bindings_to_revoke = [] @@ -41,11 +86,35 @@ def poll_once(self) -> None: for ref in self._runtime.catalog._by_ref: binding = self._runtime.catalog.get_network_binding(ref) if binding is not None: + # Only watch bindings that are VISIBLE/LEASED or have active leases + if not self._should_watch_binding(binding): + continue + # Check if peer still exists if not self._client.peer_exists(binding.peer_id): bindings_to_revoke.append(binding) + continue + + # For ROUTE_ENABLE mode with a route_id, check route state + if binding.sync_mode is SyncMode.ROUTE_ENABLE and binding.route_id: + route_state = self._client.get_route_state(binding) + if route_state in ("disabled", "missing"): + bindings_to_revoke.append(binding) - # Revoke all disappeared bindings + # Deduplicate bindings + seen = set() + unique_bindings = [] for binding in bindings_to_revoke: + key = (binding.capability_ref, binding.peer_id, binding.network) + if key not in seen: + seen.add(key) + unique_bindings.append(binding) + + # Revoke all disappeared bindings + for binding in unique_bindings: reason = f"peer {binding.peer_id} disappeared from NetBird" + if binding.sync_mode is SyncMode.ROUTE_ENABLE and binding.route_id: + route_state = self._client.get_route_state(binding) + if route_state in ("disabled", "missing"): + reason = f"route {binding.route_id} is {route_state}" self._runtime.revoke_from_physical(binding, reason) diff --git a/capability-runtime/tests/test_route_watcher_routes.py b/capability-runtime/tests/test_route_watcher_routes.py new file mode 100644 index 00000000..eca9c955 --- /dev/null +++ b/capability-runtime/tests/test_route_watcher_routes.py @@ -0,0 +1,69 @@ +from capability_runtime.netbird_client import MockNetBirdClient +from capability_runtime.network import NetworkBinding, SyncMode +from capability_runtime.types import CapabilityRef + + +def test_get_route_state_enabled(): + client = MockNetBirdClient( + routes=[{"id": "route-1", "network": "10.8.0.0/24", "peer": "peer-abc", "enabled": True}], + ) + binding = NetworkBinding(CapabilityRef("cap-reach-api"), "peer-abc", "10.8.0.0/24", "route-1", SyncMode.ROUTE_ENABLE) + assert client.get_route_state(binding) == "enabled" + + +def test_get_route_state_disabled(): + client = MockNetBirdClient( + routes=[{"id": "route-1", "network": "10.8.0.0/24", "peer": "peer-abc", "enabled": False}], + ) + binding = NetworkBinding(CapabilityRef("cap-reach-api"), "peer-abc", "10.8.0.0/24", "route-1", SyncMode.ROUTE_ENABLE) + assert client.get_route_state(binding) == "disabled" + + +def test_get_route_state_missing(): + client = MockNetBirdClient(routes=[]) + binding = NetworkBinding(CapabilityRef("cap-reach-api"), "peer-abc", "10.8.0.0/24", "route-1", SyncMode.ROUTE_ENABLE) + assert client.get_route_state(binding) == "missing" + + +def test_watcher_revokes_when_route_disabled_but_peer_exists(tmp_path): + from capability_runtime.catalog import LifecycleState + from capability_runtime.route_watcher import RouteDisappearanceWatcher + from capability_runtime.runtime import CapabilityRuntime + from capability_runtime.types import AgentIdentity, CapabilityRef + + client = MockNetBirdClient( + peers=[{"id": "peer-abc"}], + routes=[{"id": "route-1", "network": "10.8.0.0/24", "peer": "peer-abc", "enabled": True}], + ) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-rw", 3, netbird_client=client) + agent = AgentIdentity("agent-1") + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", "route-1", SyncMode.ROUTE_ENABLE) + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + + client._routes[0]["enabled"] = False + + RouteDisappearanceWatcher(runtime, client).poll_once() + + bundle = runtime.check_current_capabilities(agent, {}) + assert "reach_api" not in [n.name for n in bundle.networks] + + +def test_watcher_ignores_declared_binding_when_route_disabled(tmp_path): + from capability_runtime.catalog import LifecycleState + from capability_runtime.route_watcher import RouteDisappearanceWatcher + from capability_runtime.runtime import CapabilityRuntime + from capability_runtime.types import CapabilityRef + + client = MockNetBirdClient( + peers=[{"id": "peer-abc"}], + routes=[{"id": "route-1", "network": "10.8.0.0/24", "peer": "peer-abc", "enabled": False}], + ) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-rw2", 4, netbird_client=client) + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", "route-1", SyncMode.ROUTE_ENABLE) + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.DECLARED) + + RouteDisappearanceWatcher(runtime, client).poll_once() + + assert runtime.catalog.get_state(ref) == LifecycleState.DECLARED From 4139ace8c5c3a041e3f3eaba0fdc549276df1295 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Fri, 3 Jul 2026 06:45:47 +0000 Subject: [PATCH 058/138] fix(capability-runtime): tolerate NetBird errors during TTL expiry --- .../src/capability_runtime/runtime.py | 16 ++++++++-- .../tests/test_runtime_network.py | 29 +++++++++++++++++++ 2 files changed, 43 insertions(+), 2 deletions(-) diff --git a/capability-runtime/src/capability_runtime/runtime.py b/capability-runtime/src/capability_runtime/runtime.py index 7e21e11c..50f33ec4 100644 --- a/capability-runtime/src/capability_runtime/runtime.py +++ b/capability-runtime/src/capability_runtime/runtime.py @@ -182,10 +182,22 @@ def check_current_capabilities( ): binding = self.catalog.get_network_binding(lease.capability_ref) if binding is not None: - # Disable the binding and revoke the lease - revoke_network_binding(self._netbird_backend, binding, "TTL expired") + physical_sync = "disabled" + try: + revoke_network_binding(self._netbird_backend, binding, "TTL expired") + except Exception: + physical_sync = "failed" self.revocation_manager.revoke_by_lease(lease_id, "TTL expired") self.catalog.set_state(lease.capability_ref, LifecycleState.EXPIRED) + self.observability.emit_capability_event( + { + "event": "capability_revoked", + "lease_id": lease_id.value, + "capability_ref": lease.capability_ref.value, + "reason": "TTL expired", + "physical_sync": physical_sync, + } + ) bundle = CapabilityBundle( agent_id=agent_id, diff --git a/capability-runtime/tests/test_runtime_network.py b/capability-runtime/tests/test_runtime_network.py index f8014f64..a7e8dab9 100644 --- a/capability-runtime/tests/test_runtime_network.py +++ b/capability-runtime/tests/test_runtime_network.py @@ -138,3 +138,32 @@ def test_ttl_expiry_disables_network_binding(tmp_path): assert routes[0]["enabled"] is False # Capability should not be in bundle assert "ttl_net" not in [n.name for n in bundle.networks] + + +def test_ttl_expiry_continues_when_netbird_disable_fails(tmp_path, monkeypatch): + from datetime import datetime, timedelta, timezone + from capability_runtime.catalog import LifecycleState + from capability_runtime.netbird_client import MockNetBirdClient + from capability_runtime.network import NetworkBinding, SyncMode + from capability_runtime.runtime import CapabilityRuntime + from capability_runtime.types import AgentIdentity, CapabilityRef + + client = MockNetBirdClient(peers=[{"id": "peer-abc"}], routes=[{"id": "route-1", "enabled": True}]) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-ttl", 4, netbird_client=client) + agent = AgentIdentity("agent-1") + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", "route-1", SyncMode.ROUTE_ENABLE) + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + decision = runtime.request_capability_lease(agent, agent, ref, "ttl test") + + runtime.lease_manager._leases[decision.lease_id].expires_at = ( + datetime.now(timezone.utc) - timedelta(seconds=1) + ) + + def boom(*_a, **_kw): + raise RuntimeError("netbird down") + + monkeypatch.setattr(client, "disable_binding", boom) + + bundle = runtime.check_current_capabilities(agent, {}) + assert "reach_api" not in [n.name for n in bundle.networks] From 587bf0558c312125ebe75c85d89ac5c84c1d33a2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Fri, 3 Jul 2026 06:48:12 +0000 Subject: [PATCH 059/138] docs: add Phase 3c engineering gate script and live smoke steps --- capability-runtime/README.md | 12 ++++++++- .../scripts/phase3c_engineering_gate.sh | 26 +++++++++++++++++++ cli/README.md | 9 +++++++ 3 files changed, 46 insertions(+), 1 deletion(-) create mode 100755 capability-runtime/scripts/phase3c_engineering_gate.sh diff --git a/capability-runtime/README.md b/capability-runtime/README.md index 22dd4867..bb9e8b9d 100644 --- a/capability-runtime/README.md +++ b/capability-runtime/README.md @@ -101,7 +101,7 @@ RouteDisappearanceWatcher.poll_once() → emit physical_trigger: true ``` -The watcher ensures the runtime catalog stays consistent when reachability disappears outside the runtime — the spec's physical `Revoked` trigger (Phase 3 / idea7). +The watcher ensures the runtime catalog stays consistent when reachability disappears outside the runtime — the spec's physical `Revoked` trigger (Phase 3 / idea7). In watcher JSONL events, `physical_trigger` is an alias for the spec's `physical_revocation_reconciled` flag. ## Sidecar architecture (Phase 3b) @@ -172,6 +172,16 @@ PYTHONPATH=src:. python poc/mcp_tool_demo.py PYTHONPATH=src:. python poc/network_route_demo.py ``` +## Engineering gate (Phase 3c) + +Automated unit tests and PoC demo, plus printed manual live-smoke steps (NetBird enrollment required): + +```bash +bash scripts/phase3c_engineering_gate.sh +``` + +Manual steps map to the [Phase 3c spec §11 success criteria](../../docs/superpowers/specs/2026-06-30-capability-netbird-policy-sync-phase3c-design.md#11-success-criteria-engineering-gate-before-phase-4). For catalog ID setup before live smoke, see [Catalog IDs for live smoke](../../cli/README.md#catalog-ids-for-live-smoke) in the CLI README. + ## Layout | Module | Role | diff --git a/capability-runtime/scripts/phase3c_engineering_gate.sh b/capability-runtime/scripts/phase3c_engineering_gate.sh new file mode 100755 index 00000000..c79c348d --- /dev/null +++ b/capability-runtime/scripts/phase3c_engineering_gate.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +# capability-runtime/scripts/phase3c_engineering_gate.sh +set -euo pipefail + +echo "== Phase 3c unit gate ==" +cd "$(dirname "$0")/.." +pytest -q + +echo "== Phase 3c PoC demo ==" +PYTHONPATH=src python poc/network_route_demo.py >/dev/null +echo "PoC demo: OK" + +echo "== Manual steps (require enrolled NetBird + NB_API_TOKEN) ==" +cat <<'EOF' +1. sandcat init --netbird --capability --name +2. Set real peer_id/route_id in .devcontainer/sandcat/capability-catalog.json +3. sandcat compose up -d +4. sandcat capability lease --ref cap-reach-api --justification "gate" +5. sandcat capability check # reach_api in bundle +6. sandcat netbird status # route enabled within 30s +7. ping mesh target via wt0 # reachable while leased +8. sandcat capability revoke --ref cap-reach-api --reason "gate" +9. sandcat capability check # reach_api absent +10. sandcat netbird status # route disabled; peer remains +11. sandcat capability watch # JSONL shows grant/revoke + physical_sync +EOF diff --git a/cli/README.md b/cli/README.md index 92d43180..fb16f0b2 100644 --- a/cli/README.md +++ b/cli/README.md @@ -562,6 +562,15 @@ Network capabilities in `capability-catalog.json` (mounted as `CAPABILITY_CATALO Tool capabilities (`type: "tool"`) do not use `sync_mode` or binding fields. +#### Catalog IDs for live smoke + +Replace placeholders in `capability-catalog.json` before leasing `reach_api`: + +1. `sandcat netbird status` — copy peer ID serving your test network +2. `sandcat netbird route list` (or NetBird dashboard) — copy route ID if pre-provisioned +3. Re-init or edit `.devcontainer/sandcat/capability-catalog.json` +4. Restart capability-runtime: `docker compose restart capability-runtime` + ## Directory Structure Each module is contained in its own directory under `cli/libexec/`. From f59863140c5e4fd7a56d1bc928b3403d4f6e09d2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Fri, 3 Jul 2026 09:58:17 +0000 Subject: [PATCH 060/138] ix(capability-runtime): tolerate NetBird errors on operator revoke --- .../src/capability_runtime/lease.py | 18 +++++ .../src/capability_runtime/revoke.py | 5 +- .../src/capability_runtime/route_watcher.py | 62 ++++++++++++----- .../src/capability_runtime/runtime.py | 66 +++++++++++------- .../tests/test_re_lease_after_revoke.py | 24 +++++++ .../tests/test_revoke_by_lease_network.py | 37 ++++++++++ .../tests/test_route_watcher_routes.py | 67 +++++++++++++++++++ .../tests/test_runtime_network.py | 41 ++++++++++++ 8 files changed, 277 insertions(+), 43 deletions(-) diff --git a/capability-runtime/src/capability_runtime/lease.py b/capability-runtime/src/capability_runtime/lease.py index f435cb41..d194efb1 100644 --- a/capability-runtime/src/capability_runtime/lease.py +++ b/capability-runtime/src/capability_runtime/lease.py @@ -62,3 +62,21 @@ def is_expired(self, lease_id: LeaseId, now: datetime) -> bool: def get_lease(self, lease_id: LeaseId) -> LeaseDecision | None: return self._leases.get(lease_id) + + def iter_leases_for_ref(self, capability_ref: CapabilityRef): + """Yield (lease_id, lease) pairs for a capability ref.""" + for lease_id, lease in self._leases.items(): + if lease.capability_ref == capability_ref: + yield lease_id, lease + + def iter_active_leases_for_ref( + self, capability_ref: CapabilityRef, now: datetime, *, is_revoked + ): + """Yield active leases for a capability ref at a point in time.""" + for lease_id, lease in self.iter_leases_for_ref(capability_ref): + if ( + not self.is_expired(lease_id, now) + and not is_revoked(lease_id) + and not self.is_exhausted(lease_id) + ): + yield lease_id, lease diff --git a/capability-runtime/src/capability_runtime/revoke.py b/capability-runtime/src/capability_runtime/revoke.py index 75b0104e..e6ec1edd 100644 --- a/capability-runtime/src/capability_runtime/revoke.py +++ b/capability-runtime/src/capability_runtime/revoke.py @@ -20,9 +20,8 @@ def revoke_by_lease(self, lease_id: LeaseId, reason: str) -> None: def revoke_by_ref(self, capability_ref: CapabilityRef, reason: str) -> None: self._catalog.set_state(capability_ref, LifecycleState.REVOKED) - for lease_id, lease in self._lease_manager._leases.items(): - if lease.capability_ref == capability_ref: - self._revoked_leases.add(lease_id) + for lease_id, _lease in self._lease_manager.iter_leases_for_ref(capability_ref): + self._revoked_leases.add(lease_id) def is_revoked(self, capability_ref: CapabilityRef) -> bool: try: diff --git a/capability-runtime/src/capability_runtime/route_watcher.py b/capability-runtime/src/capability_runtime/route_watcher.py index 3b66dea7..70f1c8d8 100644 --- a/capability-runtime/src/capability_runtime/route_watcher.py +++ b/capability-runtime/src/capability_runtime/route_watcher.py @@ -10,6 +10,7 @@ from capability_runtime.runtime import CapabilityRuntime from capability_runtime.catalog import LifecycleState +from capability_runtime.errors import CapabilityUnknown from capability_runtime.network import SyncMode @@ -48,27 +49,51 @@ def _should_watch_binding(self, binding: NetworkBinding) -> bool: True if the binding should be watched """ from datetime import datetime, timezone - - # Check catalog state + try: state = self._runtime.catalog.get_state(binding.capability_ref) if state in (LifecycleState.VISIBLE, LifecycleState.LEASED): return True - except Exception: - pass - - # Check for active leases + except CapabilityUnknown: + return False + now = datetime.now(timezone.utc) - for lease_id, lease in self._runtime.lease_manager._leases.items(): - if lease.capability_ref == binding.capability_ref: - if ( - not self._runtime.lease_manager.is_expired(lease_id, now) - and not self._runtime.revocation_manager.is_lease_revoked(lease_id) - and not self._runtime.lease_manager.is_exhausted(lease_id) - ): - return True - - return False + is_revoked = self._runtime.revocation_manager.is_lease_revoked + return any( + True + for _lease_id, _lease in self._runtime.lease_manager.iter_active_leases_for_ref( + binding.capability_ref, now, is_revoked=is_revoked + ) + ) + + def _reconcile_stale_physical_routes(self) -> None: + """Retry NetBird disable when logical revoke succeeded but route stayed enabled.""" + backend = self._runtime._netbird_backend + if backend is None: + return + + from capability_runtime.netbird_sync import revoke_network_binding + + for ref in self._runtime.catalog._by_ref: + binding = self._runtime.catalog.get_network_binding(ref) + if ( + binding is None + or binding.sync_mode is not SyncMode.ROUTE_ENABLE + or not binding.route_id + ): + continue + try: + state = self._runtime.catalog.get_state(ref) + except CapabilityUnknown: + continue + if state not in (LifecycleState.REVOKED, LifecycleState.EXPIRED): + continue + if self._client.get_route_state(binding) != "enabled": + continue + try: + revoke_network_binding(backend, binding, "physical reconcile") + except Exception: + pass def poll_once(self) -> None: """Poll once for disappeared peers and revoke their capabilities. @@ -79,7 +104,12 @@ def poll_once(self) -> None: For ROUTE_ENABLE bindings with a route_id, also checks if the route is disabled or missing. If so, performs logical-only revocation. + + Also retries physical disable for REVOKED/EXPIRED bindings whose route + is still enabled (e.g. after a transient NetBird API failure). """ + self._reconcile_stale_physical_routes() + # Collect all network bindings from catalog bindings_to_revoke = [] diff --git a/capability-runtime/src/capability_runtime/runtime.py b/capability-runtime/src/capability_runtime/runtime.py index 50f33ec4..32b58033 100644 --- a/capability-runtime/src/capability_runtime/runtime.py +++ b/capability-runtime/src/capability_runtime/runtime.py @@ -347,44 +347,62 @@ def revoke_capability( if isinstance(target, LeaseId): lease = self.lease_manager.get_lease(target) + physical_sync: str | None = None + capability_ref_value: str | None = None if lease is not None: + capability_ref_value = lease.capability_ref.value binding = self.catalog.get_network_binding(lease.capability_ref) if binding is not None and self._netbird_backend is not None: from capability_runtime.netbird_sync import revoke_network_binding - revoke_network_binding(self._netbird_backend, binding, reason) - physical_revocation = True + + physical_sync = "disabled" + try: + revoke_network_binding(self._netbird_backend, binding, reason) + physical_revocation = True + except Exception: + physical_sync = "failed" self.revocation_manager.revoke_by_lease(target, reason) self._bundle_version += 1 - self.observability.emit_capability_event( - { - "event": "capability_revoked", - "lease_id": target.value, - "reason": reason, - "physical_revocation": physical_revocation, - } - ) + event: dict[str, object] = { + "event": "capability_revoked", + "lease_id": target.value, + "reason": reason, + "physical_revocation": physical_revocation, + } + if capability_ref_value is not None: + event["capability_ref"] = capability_ref_value + if physical_sync is not None: + event["physical_sync"] = physical_sync + self.observability.emit_capability_event(event) else: # Check if this is a network capability binding = self.catalog.get_network_binding(target) + physical_sync: str | None = None if binding is not None and self._netbird_backend is not None: - # Perform physical revocation via NetBird backend - self._netbird_backend.revoke_binding(binding, reason) - physical_revocation = True - + from capability_runtime.netbird_sync import revoke_network_binding + + physical_sync = "disabled" + try: + revoke_network_binding(self._netbird_backend, binding, reason) + physical_revocation = True + except Exception: + physical_sync = "failed" + # Perform logical revocation self.revocation_manager.revoke_by_ref(target, reason) - + # Invalidate agent bundle cache self._bundle_version += 1 - - self.observability.emit_capability_event( - { - "event": "capability_revoked", - "capability_ref": target.value, - "reason": reason, - "physical_revocation": physical_revocation, - } - ) + + event = { + "event": "capability_revoked", + "capability_ref": target.value, + "reason": reason, + "physical_revocation": physical_revocation, + } + if physical_sync is not None: + event["physical_sync"] = physical_sync + self.observability.emit_capability_event(event) def revoke_from_physical(self, binding: NetworkBinding, reason: str) -> None: """Perform logical revoke when physical route already disappeared. diff --git a/capability-runtime/tests/test_re_lease_after_revoke.py b/capability-runtime/tests/test_re_lease_after_revoke.py index 1b7ec50a..588e7512 100644 --- a/capability-runtime/tests/test_re_lease_after_revoke.py +++ b/capability-runtime/tests/test_re_lease_after_revoke.py @@ -1,4 +1,6 @@ # capability-runtime/tests/test_re_lease_after_revoke.py +from datetime import datetime, timedelta, timezone + from capability_runtime.catalog import LifecycleState from capability_runtime.netbird_client import MockNetBirdClient from capability_runtime.network import NetworkBinding, SyncMode @@ -26,3 +28,25 @@ def test_re_lease_after_revoke_by_ref(tmp_path): bundle = runtime.check_current_capabilities(agent, {}) assert "reach_api" in [n.name for n in bundle.networks] assert bundle.networks[0].lease_id == decision2.lease_id + + +def test_re_lease_after_expiry(tmp_path): + client = MockNetBirdClient(peers=[{"id": "peer-abc"}], routes=[]) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-exp", 3, netbird_client=client) + agent = AgentIdentity("agent-1") + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", None, sync_mode=SyncMode.ROUTE_ENABLE) + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + + decision1 = runtime.request_capability_lease(agent, agent, ref, "first lease") + runtime.lease_manager._leases[decision1.lease_id].expires_at = ( + datetime.now(timezone.utc) - timedelta(seconds=1) + ) + runtime.check_current_capabilities(agent, {}) + assert runtime.catalog.get_state(ref) == LifecycleState.EXPIRED + + decision2 = runtime.request_capability_lease(agent, agent, ref, "second lease") + assert decision2.lease_id is not None + bundle = runtime.check_current_capabilities(agent, {}) + assert "reach_api" in [n.name for n in bundle.networks] + assert bundle.networks[0].lease_id == decision2.lease_id diff --git a/capability-runtime/tests/test_revoke_by_lease_network.py b/capability-runtime/tests/test_revoke_by_lease_network.py index 61226e2f..32ff9390 100644 --- a/capability-runtime/tests/test_revoke_by_lease_network.py +++ b/capability-runtime/tests/test_revoke_by_lease_network.py @@ -1,6 +1,8 @@ # capability-runtime/tests/test_revoke_by_lease_network.py """Revoke by lease ID must disable network bindings.""" +import json + from capability_runtime.catalog import LifecycleState from capability_runtime.netbird_client import MockNetBirdClient from capability_runtime.network import NetworkBinding, SyncMode @@ -33,6 +35,41 @@ def test_revoke_by_lease_id_disables_network_route(tmp_path): assert runtime._bundle_version > version_before +def test_revoke_by_lease_id_continues_when_netbird_disable_fails(tmp_path, monkeypatch): + client = MockNetBirdClient( + peers=[{"id": "peer-abc"}], + routes=[{"id": "route-1", "network": "10.8.0.0/24", "peer": "peer-abc", "enabled": True}], + ) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-rl-fail", 3, netbird_client=client) + agent = AgentIdentity("agent-1") + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", "route-1", sync_mode=SyncMode.ROUTE_ENABLE) + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + + decision = runtime.request_capability_lease(agent, agent, ref, "need api") + + def boom(*_a, **_kw): + raise RuntimeError("netbird down") + + monkeypatch.setattr(client, "disable_binding", boom) + + runtime.revoke_capability(_OPERATOR, decision.lease_id, "operator revoke") + + bundle = runtime.check_current_capabilities(agent, {}) + assert "reach_api" not in [n.name for n in bundle.networks] + routes = [r for r in client.list_routes() if r["id"] == "route-1"] + assert routes[0]["enabled"] is True + + events = [ + json.loads(line) + for line in (tmp_path / "t.jsonl").read_text().strip().split("\n") + if line + ] + revoke_events = [e for e in events if e.get("event") == "capability_revoked"] + assert revoke_events[-1]["physical_sync"] == "failed" + assert revoke_events[-1]["capability_ref"] == ref.value + + def test_revoke_by_lease_id_skips_netbird_for_tool_capability(tmp_path): """Tool leases must not call NetBird on revoke-by-lease-ID.""" client = MockNetBirdClient(peers=[{"id": "peer-xyz"}], routes=[]) diff --git a/capability-runtime/tests/test_route_watcher_routes.py b/capability-runtime/tests/test_route_watcher_routes.py index eca9c955..35bd58f1 100644 --- a/capability-runtime/tests/test_route_watcher_routes.py +++ b/capability-runtime/tests/test_route_watcher_routes.py @@ -47,6 +47,73 @@ def test_watcher_revokes_when_route_disabled_but_peer_exists(tmp_path): bundle = runtime.check_current_capabilities(agent, {}) assert "reach_api" not in [n.name for n in bundle.networks] + assert runtime.catalog.get_state(ref) == LifecycleState.REVOKED + + +def test_watcher_revokes_when_route_disabled_with_active_lease(tmp_path): + from capability_runtime.catalog import LifecycleState + from capability_runtime.route_watcher import RouteDisappearanceWatcher + from capability_runtime.runtime import CapabilityRuntime + from capability_runtime.types import AgentIdentity, CapabilityRef + + client = MockNetBirdClient( + peers=[{"id": "peer-abc"}], + routes=[{"id": "route-1", "network": "10.8.0.0/24", "peer": "peer-abc", "enabled": True}], + ) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-rw-lease", 5, netbird_client=client) + agent = AgentIdentity("agent-1") + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", "route-1", SyncMode.ROUTE_ENABLE) + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + runtime.request_capability_lease(agent, agent, ref, "need api") + + client._routes[0]["enabled"] = False + + RouteDisappearanceWatcher(runtime, client).poll_once() + + bundle = runtime.check_current_capabilities(agent, {}) + assert "reach_api" not in [n.name for n in bundle.networks] + assert runtime.catalog.get_state(ref) == LifecycleState.REVOKED + + +def test_watcher_reconciles_route_after_failed_physical_revoke(tmp_path, monkeypatch): + from capability_runtime.catalog import LifecycleState + from capability_runtime.route_watcher import RouteDisappearanceWatcher + from capability_runtime.runtime import CapabilityRuntime + from capability_runtime.types import AgentIdentity, CapabilityRef + + client = MockNetBirdClient( + peers=[{"id": "peer-abc"}], + routes=[{"id": "route-1", "network": "10.8.0.0/24", "peer": "peer-abc", "enabled": True}], + ) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-rw-reconcile", 6, netbird_client=client) + agent = AgentIdentity("agent-1") + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", "route-1", SyncMode.ROUTE_ENABLE) + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + runtime.request_capability_lease(agent, agent, ref, "need api") + + fail_once = [True] + original_disable = client.disable_binding + + def flaky_disable(binding): + if fail_once[0]: + fail_once[0] = False + raise RuntimeError("netbird down") + return original_disable(binding) + + monkeypatch.setattr(client, "disable_binding", flaky_disable) + runtime.revoke_capability(AgentIdentity("operator"), ref, "security") + assert runtime.catalog.get_state(ref) == LifecycleState.REVOKED + routes = [r for r in client.list_routes() if r["id"] == "route-1"] + assert routes[0]["enabled"] is True + + RouteDisappearanceWatcher(runtime, client).poll_once() + + routes = [r for r in client.list_routes() if r["id"] == "route-1"] + assert routes[0]["enabled"] is False + bundle = runtime.check_current_capabilities(agent, {}) + assert "reach_api" not in [n.name for n in bundle.networks] def test_watcher_ignores_declared_binding_when_route_disabled(tmp_path): diff --git a/capability-runtime/tests/test_runtime_network.py b/capability-runtime/tests/test_runtime_network.py index a7e8dab9..53011416 100644 --- a/capability-runtime/tests/test_runtime_network.py +++ b/capability-runtime/tests/test_runtime_network.py @@ -48,6 +48,47 @@ def test_revoke_network_capability_calls_netbird_backend(tmp_path): assert "reach_api" not in [n.name for n in bundle.networks] +def test_revoke_by_ref_continues_when_netbird_disable_fails(tmp_path, monkeypatch): + import json + + from capability_runtime.catalog import LifecycleState + from capability_runtime.netbird_client import MockNetBirdClient + from capability_runtime.network import NetworkBinding, SyncMode + from capability_runtime.runtime import CapabilityRuntime + from capability_runtime.types import AgentIdentity, CapabilityRef + + operator = AgentIdentity("operator") + client = MockNetBirdClient( + peers=[{"id": "peer-abc"}], + routes=[{"id": "route-1", "network": "10.8.0.0/24", "peer": "peer-abc", "enabled": True}], + ) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-n2-fail", 8, netbird_client=client) + agent = AgentIdentity("agent-1") + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", "route-1", sync_mode=SyncMode.ROUTE_ENABLE) + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + runtime.request_capability_lease(agent, agent, ref, "need api") + + def boom(*_a, **_kw): + raise RuntimeError("netbird down") + + monkeypatch.setattr(client, "disable_binding", boom) + runtime.revoke_capability(operator, ref, "security") + + bundle = runtime.check_current_capabilities(agent, {}) + assert "reach_api" not in [n.name for n in bundle.networks] + routes = [r for r in client.list_routes() if r["id"] == "route-1"] + assert routes[0]["enabled"] is True + + events = [ + json.loads(line) + for line in (tmp_path / "t.jsonl").read_text().strip().split("\n") + if line + ] + revoke_events = [e for e in events if e.get("event") == "capability_revoked"] + assert revoke_events[-1]["physical_sync"] == "failed" + + def test_revoke_non_network_capability_does_not_call_netbird(tmp_path): from capability_runtime.catalog import LifecycleState from capability_runtime.netbird_client import MockNetBirdClient From b49b3a220de034c140079a9cf6682a64985c0e9e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Wed, 8 Jul 2026 21:40:39 +0000 Subject: [PATCH 061/138] fixes --- .../src/capability_runtime/daemon.py | 11 +- .../src/capability_runtime/netbird_client.py | 222 ++++++++++++++++-- .../src/capability_runtime/route_watcher.py | 11 +- .../src/capability_runtime/rpc/dispatcher.py | 2 + .../src/capability_runtime/runtime.py | 68 +++--- .../src/capability_runtime/settings.py | 60 ++++- .../tests/test_netbird_client.py | 8 +- capability-runtime/tests/test_netbird_sync.py | 111 ++++++++- .../tests/test_route_watcher_routes.py | 38 ++- .../tests/test_rpc_dispatcher.py | 32 +++ .../tests/test_runtime_network.py | 50 +++- capability-runtime/tests/test_settings.py | 71 +++++- cli/lib/netbird.bash | 143 ++++++++++- cli/libexec/netbird/netbird | 30 ++- .../sandcat/capability-catalog.json | 3 +- cli/test/composefile/netbird_contract.bats | 10 + cli/test/netbird/netbird.bats | 25 +- cli/test/netbird/netbird_api.bats | 42 +++- 18 files changed, 842 insertions(+), 95 deletions(-) diff --git a/capability-runtime/src/capability_runtime/daemon.py b/capability-runtime/src/capability_runtime/daemon.py index f317dba5..5649af1d 100644 --- a/capability-runtime/src/capability_runtime/daemon.py +++ b/capability-runtime/src/capability_runtime/daemon.py @@ -88,11 +88,20 @@ def load_catalog_into_runtime(runtime: CapabilityRuntime, catalog_path: Path) -> ref = CapabilityRef(entry["ref"]) cap_type = entry.get("type", "tool") if cap_type == "network": + route_id = entry.get("route_id") + if isinstance(route_id, str): + route_id = route_id.strip() or None + if route_id and route_id.lower() in { + "route-placeholder", + "peer-placeholder", + "placeholder", + }: + route_id = None binding = NetworkBinding( ref, entry["peer_id"], entry["network"], - entry.get("route_id"), + route_id, sync_mode_from_catalog(entry), ) runtime.register_network_capability( diff --git a/capability-runtime/src/capability_runtime/netbird_client.py b/capability-runtime/src/capability_runtime/netbird_client.py index 9a2f3194..dbfd2496 100644 --- a/capability-runtime/src/capability_runtime/netbird_client.py +++ b/capability-runtime/src/capability_runtime/netbird_client.py @@ -10,6 +10,100 @@ from capability_runtime.network import NetworkBinding, SyncMode +def _default_network_id(network: str) -> str: + """NetBird network_id (1-40 chars) derived from a CIDR.""" + return network.replace(".", "-").replace("/", "-")[:40] + + +DEFAULT_ROUTE_METRIC = 9999 +DEFAULT_ROUTE_MASQUERADE = True +DEFAULT_ROUTE_KEEP_ROUTE = False + + +class NetBirdApiError(RuntimeError): + def __init__(self, method: str, path: str, code: int, detail: str) -> None: + self.method = method + self.path = path + self.code = code + self.detail = detail + super().__init__( + f"NetBird API {method} {path} failed (HTTP {code}): {detail}" + ) + + +def _network_id_for_binding(binding: NetworkBinding) -> str: + ref = binding.capability_ref.value + if ref.startswith("cap-"): + ref = ref[4:] + slug = ref.replace("_", "-")[:40] + if slug: + return slug + return _default_network_id(binding.network) + + +def _effective_route_id(route_id: str | None) -> str | None: + """Treat template placeholders and empty strings as no route id.""" + if route_id is None: + return None + normalized = route_id.strip().lower() + if normalized in {"", "route-placeholder", "peer-placeholder", "placeholder"}: + return None + return route_id.strip() + + +def build_route_create_payload( + *, + network: str, + peer_id: str, + network_id: str, + groups: list[str], + metric: int = DEFAULT_ROUTE_METRIC, + masquerade: bool = DEFAULT_ROUTE_MASQUERADE, + keep_route: bool = DEFAULT_ROUTE_KEEP_ROUTE, +) -> dict[str, Any]: + """NetBird POST /api/routes body with all required fields.""" + if not groups: + raise ValueError("route distribution groups must not be empty") + return { + "description": f"sandcat route {network_id}", + "network_id": network_id, + "enabled": True, + "peer": peer_id, + "network": network, + "metric": metric, + "masquerade": masquerade, + "groups": groups, + "keep_route": keep_route, + } + + +_ROUTE_PUT_KEYS = ( + "description", + "network_id", + "enabled", + "peer", + "peer_groups", + "network", + "domains", + "metric", + "masquerade", + "groups", + "keep_route", + "access_control_groups", + "skip_auto_apply", +) + + +def route_put_body(route: dict[str, Any], *, enabled: bool | None = None) -> dict[str, Any]: + """Build PUT /api/routes/{id} body from a GET route (NetBird has no PATCH).""" + body: dict[str, Any] = { + key: route[key] for key in _ROUTE_PUT_KEYS if key in route and route[key] is not None + } + if enabled is not None: + body["enabled"] = enabled + return body + + class NetBirdClient(Protocol): def list_peers(self) -> list[dict]: ... @@ -75,11 +169,20 @@ def enable_binding(self, binding: NetworkBinding) -> NetworkBinding: if binding.sync_mode is not SyncMode.ROUTE_ENABLE: return binding - if binding.route_id: + route_id = _effective_route_id(binding.route_id) + if route_id: for route in self._routes: - if route.get("id") == binding.route_id: + if route.get("id") == route_id: route["enabled"] = True - return binding + return binding + + for route in self._routes: + if ( + route.get("peer") == binding.peer_id + and route.get("network") == binding.network + ): + route["enabled"] = True + return replace(binding, route_id=route["id"]) route_id = f"route-{self._next_route_id}" self._next_route_id += 1 @@ -139,6 +242,12 @@ def list_peers(self) -> list[dict]: def list_routes(self) -> list[dict]: return self._request("GET", "/api/routes") + def list_groups(self, *, name: str | None = None) -> list[dict]: + path = "/api/groups" + if name: + path = f"{path}?name={name}" + return self._request("GET", path) + def remove_peer(self, peer_id: str) -> None: self._request("DELETE", f"/api/peers/{peer_id}") @@ -168,24 +277,92 @@ def enable_binding(self, binding: NetworkBinding) -> NetworkBinding: if binding.sync_mode is not SyncMode.ROUTE_ENABLE: return binding - if binding.route_id: - self._request( - "PATCH", - f"/api/routes/{binding.route_id}", - {"enabled": True}, - ) + route_id = _effective_route_id(binding.route_id) + if route_id and self.route_exists(route_id): + self._enable_route(route_id) return binding - created = self._request( + existing = self._find_route_for_binding(binding) + if existing is not None: + resolved_id = str(existing["id"]) + self._enable_route(resolved_id) + return replace(binding, route_id=resolved_id) + + try: + created = self._create_route(binding) + except NetBirdApiError as exc: + if exc.code == 422: + existing = self._find_route_for_binding(binding) + if existing is not None: + resolved_id = str(existing["id"]) + self._enable_route(resolved_id) + return replace(binding, route_id=resolved_id) + raise + + return replace(binding, route_id=created["id"]) + + def _enable_route(self, route_id: str) -> None: + self._set_route_enabled(route_id, True) + + def _set_route_enabled(self, route_id: str, enabled: bool) -> None: + route = self._get_route(route_id) + self._request( + "PUT", + f"/api/routes/{route_id}", + route_put_body(route, enabled=enabled), + ) + + def _get_route(self, route_id: str) -> dict: + return self._request("GET", f"/api/routes/{route_id}")[0] + + def _create_route(self, binding: NetworkBinding) -> dict: + return self._request( "POST", "/api/routes", - { - "network": binding.network, - "peer": binding.peer_id, - "enabled": True, - }, + build_route_create_payload( + network=binding.network, + peer_id=binding.peer_id, + network_id=_network_id_for_binding(binding), + groups=self._resolve_route_distribution_groups(), + ), )[0] - return replace(binding, route_id=created["id"]) + + def _find_route_for_binding(self, binding: NetworkBinding) -> dict | None: + network_id = _network_id_for_binding(binding) + for route in self.list_routes(): + if route.get("peer") != binding.peer_id: + continue + if route.get("network") == binding.network: + return route + if route.get("network_id") == network_id: + return route + return None + + def _http_error(self, method: str, path: str, exc: HTTPError) -> NetBirdApiError: + detail = exc.read().decode() if exc.fp else "" + return NetBirdApiError(method, path, exc.code, detail) + + def _resolve_route_distribution_groups(self) -> list[str]: + from capability_runtime.settings import load_netbird_route_groups + + configured = load_netbird_route_groups() + if configured: + return configured + + groups = self.list_groups(name="All") + for group in groups: + if group.get("name") == "All" and group.get("id"): + return [str(group["id"])] + + if not groups: + groups = self.list_groups() + if groups and groups[0].get("id"): + return [str(groups[0]["id"])] + + raise RuntimeError( + "No NetBird distribution groups found; set netbird_route_groups in " + "settings or NB_ROUTE_GROUPS" + ) def disable_binding(self, binding: NetworkBinding) -> None: if binding.sync_mode is SyncMode.PEER_REMOVE: @@ -200,11 +377,10 @@ def disable_binding(self, binding: NetworkBinding) -> None: if not binding.route_id: return - self._request( - "PATCH", - f"/api/routes/{binding.route_id}", - {"enabled": False}, - ) + route_id = _effective_route_id(binding.route_id) + if not route_id: + return + self._set_route_enabled(route_id, False) def _management_base_url(self) -> str: base = self._management_url.rstrip("/") @@ -238,8 +414,8 @@ def _request( try: with urlopen(request) as response: raw = response.read().decode() - except HTTPError: - raise + except HTTPError as exc: + raise self._http_error(method, path, exc) from exc if not raw: return [] diff --git a/capability-runtime/src/capability_runtime/route_watcher.py b/capability-runtime/src/capability_runtime/route_watcher.py index 70f1c8d8..ed77275f 100644 --- a/capability-runtime/src/capability_runtime/route_watcher.py +++ b/capability-runtime/src/capability_runtime/route_watcher.py @@ -93,7 +93,15 @@ def _reconcile_stale_physical_routes(self) -> None: try: revoke_network_binding(backend, binding, "physical reconcile") except Exception: - pass + self._runtime.observability.emit_capability_event( + { + "event": "physical_reconcile_failed", + "capability_ref": ref.value, + "peer_id": binding.peer_id, + "route_id": binding.route_id, + "reason": "physical reconcile", + } + ) def poll_once(self) -> None: """Poll once for disappeared peers and revoke their capabilities. @@ -108,6 +116,7 @@ def poll_once(self) -> None: Also retries physical disable for REVOKED/EXPIRED bindings whose route is still enabled (e.g. after a transient NetBird API failure). """ + self._runtime.process_expired_network_leases() self._reconcile_stale_physical_routes() # Collect all network bindings from catalog diff --git a/capability-runtime/src/capability_runtime/rpc/dispatcher.py b/capability-runtime/src/capability_runtime/rpc/dispatcher.py index 7f3f0f5f..be192808 100644 --- a/capability-runtime/src/capability_runtime/rpc/dispatcher.py +++ b/capability-runtime/src/capability_runtime/rpc/dispatcher.py @@ -76,6 +76,8 @@ def handle(self, request: dict) -> dict: return self._error_response(request_id, rpc_errors.INTERNAL_ERROR, str(exc)) except (KeyError, TypeError, ValueError) as exc: return self._error_response(request_id, rpc_errors.INVALID_PARAMS, str(exc)) + except Exception as exc: + return self._error_response(request_id, rpc_errors.INTERNAL_ERROR, str(exc)) return {"jsonrpc": "2.0", "result": result, "id": request_id} diff --git a/capability-runtime/src/capability_runtime/runtime.py b/capability-runtime/src/capability_runtime/runtime.py index 32b58033..a66320c8 100644 --- a/capability-runtime/src/capability_runtime/runtime.py +++ b/capability-runtime/src/capability_runtime/runtime.py @@ -73,6 +73,42 @@ def __init__( # Register create_pr as DECLARED (invisible until leased) self.catalog.register("create_pr", CapabilityRef("cap-create-pr"), LifecycleState.DECLARED) + def process_expired_network_leases(self, now: datetime | None = None) -> None: + """Disable NetBird bindings for expired network leases (TTL hook).""" + if self._netbird_backend is None: + return + + from capability_runtime.netbird_sync import revoke_network_binding + + if now is None: + now = datetime.now(timezone.utc) + + for lease_id, lease in list(self.lease_manager._leases.items()): + if ( + self.lease_manager.is_expired(lease_id, now) + and not self.revocation_manager.is_lease_revoked(lease_id) + and not self.lease_manager.is_exhausted(lease_id) + ): + binding = self.catalog.get_network_binding(lease.capability_ref) + if binding is None: + continue + physical_sync = "disabled" + try: + revoke_network_binding(self._netbird_backend, binding, "TTL expired") + except Exception: + physical_sync = "failed" + self.revocation_manager.revoke_by_lease(lease_id, "TTL expired") + self.catalog.set_state(lease.capability_ref, LifecycleState.EXPIRED) + self.observability.emit_capability_event( + { + "event": "capability_revoked", + "lease_id": lease_id.value, + "capability_ref": lease.capability_ref.value, + "reason": "TTL expired", + "physical_sync": physical_sync, + } + ) + def register_network_capability( self, name: str, @@ -170,34 +206,7 @@ def check_current_capabilities( earliest_expiry = lease.expires_at break - # Disable expired network leases (TTL expiry hook) - if self._netbird_backend is not None: - from capability_runtime.netbird_sync import revoke_network_binding - - for lease_id, lease in list(self.lease_manager._leases.items()): - if ( - self.lease_manager.is_expired(lease_id, now) - and not self.revocation_manager.is_lease_revoked(lease_id) - and not self.lease_manager.is_exhausted(lease_id) - ): - binding = self.catalog.get_network_binding(lease.capability_ref) - if binding is not None: - physical_sync = "disabled" - try: - revoke_network_binding(self._netbird_backend, binding, "TTL expired") - except Exception: - physical_sync = "failed" - self.revocation_manager.revoke_by_lease(lease_id, "TTL expired") - self.catalog.set_state(lease.capability_ref, LifecycleState.EXPIRED) - self.observability.emit_capability_event( - { - "event": "capability_revoked", - "lease_id": lease_id.value, - "capability_ref": lease.capability_ref.value, - "reason": "TTL expired", - "physical_sync": physical_sync, - } - ) + self.process_expired_network_leases(now) bundle = CapabilityBundle( agent_id=agent_id, @@ -244,6 +253,9 @@ def request_capability_lease( _assert_caller(caller, agent_id) # Check capability exists state = self.catalog.get_state(capability_ref) + # REVOKED/EXPIRED remain leasable so operators can re-grant after revoke/TTL + # without sidecar restart. Physical routes are disabled on revoke/expiry; + # a new lease re-enables via NetBird (see test_re_lease_after_revoke.py). leasable = { LifecycleState.DECLARED, LifecycleState.DISCOVERABLE, diff --git a/capability-runtime/src/capability_runtime/settings.py b/capability-runtime/src/capability_runtime/settings.py index 31f22a74..94b8a67c 100644 --- a/capability-runtime/src/capability_runtime/settings.py +++ b/capability-runtime/src/capability_runtime/settings.py @@ -2,6 +2,7 @@ import json import os +import re from dataclasses import dataclass from pathlib import Path @@ -33,9 +34,66 @@ def _read_setting(key: str) -> str | None: return value +_LOOPBACK_MANAGEMENT_URL = re.compile( + r"^https?://(localhost|127\.0\.0\.1)([:/]|$)", + re.IGNORECASE, +) + + +def _management_url_is_loopback(url: str) -> bool: + return _LOOPBACK_MANAGEMENT_URL.match(url) is not None + + +def _resolve_management_url(url: str | None) -> str | None: + """Use enrollment URL when management URL targets host loopback (container-safe). + + Matches sandcat CLI ``netbird_enrollment_management_url_from``: wg-client and + capability-runtime run in Docker and cannot reach the host via localhost. + """ + if url is None or not _management_url_is_loopback(url): + return url + enrollment = _read_setting("netbird_enrollment_management_url") + return enrollment if enrollment else url + + +def load_netbird_route_groups() -> list[str] | None: + """Distribution group IDs for new routes (NB_ROUTE_GROUPS or netbird_route_groups).""" + raw = os.environ.get("NB_ROUTE_GROUPS") + if raw: + groups = [part.strip() for part in raw.split(",") if part.strip()] + return groups or None + + for env_var in ("SANDCAT_SETTINGS_USER", "SANDCAT_SETTINGS_PROJECT"): + path_str = os.environ.get(env_var) + if not path_str: + continue + path = Path(path_str) + if not path.is_file(): + continue + with path.open(encoding="utf-8") as handle: + data = json.load(handle) + value = data.get("netbird_route_groups") + if not value: + continue + if isinstance(value, list): + groups = [str(item).strip() for item in value if str(item).strip()] + if groups: + return groups + setting = str(value) + if setting.startswith("["): + parsed = json.loads(setting) + groups = [str(item).strip() for item in parsed if str(item).strip()] + if groups: + return groups + groups = [part.strip() for part in setting.split(",") if part.strip()] + if groups: + return groups + return None + + def load_netbird_credentials() -> NetBirdCredentials: token = os.environ.get("NB_API_TOKEN") or _read_setting("netbird_api_token") - url = ( + url = _resolve_management_url( os.environ.get("NB_MANAGEMENT_URL") or _read_setting("netbird_management_url") or None diff --git a/capability-runtime/tests/test_netbird_client.py b/capability-runtime/tests/test_netbird_client.py index 27c0c841..bf02f545 100644 --- a/capability-runtime/tests/test_netbird_client.py +++ b/capability-runtime/tests/test_netbird_client.py @@ -8,7 +8,11 @@ import pytest -from capability_runtime.netbird_client import MockNetBirdClient, RestNetBirdClient +from capability_runtime.netbird_client import ( + MockNetBirdClient, + NetBirdApiError, + RestNetBirdClient, +) def test_mock_client_tracks_peers_and_routes(): @@ -142,5 +146,5 @@ def fake_urlopen(request): monkeypatch.setenv("NB_API_TOKEN", "test-token") monkeypatch.setattr("capability_runtime.netbird_client.urlopen", fake_urlopen) - with pytest.raises(HTTPError): + with pytest.raises(NetBirdApiError, match="HTTP 404"): RestNetBirdClient().remove_peer("peer-missing") diff --git a/capability-runtime/tests/test_netbird_sync.py b/capability-runtime/tests/test_netbird_sync.py index 338320cd..02a55ffe 100644 --- a/capability-runtime/tests/test_netbird_sync.py +++ b/capability-runtime/tests/test_netbird_sync.py @@ -4,6 +4,7 @@ import json from io import BytesIO +from urllib.error import HTTPError from capability_runtime.netbird_client import MockNetBirdClient, RestNetBirdClient from capability_runtime.network import NetworkBinding, SyncMode @@ -24,6 +25,24 @@ def _binding( ) +def _route_record(**overrides: object) -> dict: + route = { + "id": "route-1", + "network_type": "IPv4", + "description": "sandcat route reach-api", + "network_id": "reach-api", + "enabled": False, + "peer": "peer-abc", + "network": "10.8.0.0/24", + "metric": 9999, + "masquerade": True, + "groups": ["grp-test"], + "keep_route": False, + } + route.update(overrides) + return route + + def test_mock_enable_binding_creates_route_when_missing_route_id(): client = MockNetBirdClient(peers=[{"id": "peer-abc"}]) binding = _binding() @@ -131,6 +150,8 @@ def fake_urlopen(request): captured["method"] = request.get_method() captured["url"] = request.full_url captured["body"] = request.data.decode() if request.data else None + if request.get_method() == "GET" and request.full_url.endswith("/api/routes"): + return BytesIO(b"[]") return BytesIO( json.dumps( { @@ -143,6 +164,7 @@ def fake_urlopen(request): ) monkeypatch.setenv("NB_API_TOKEN", "test-token") + monkeypatch.setenv("NB_ROUTE_GROUPS", "grp-test") monkeypatch.setattr("capability_runtime.netbird_client.urlopen", fake_urlopen) client = RestNetBirdClient() @@ -151,20 +173,31 @@ def fake_urlopen(request): assert captured["method"] == "POST" assert captured["url"] == "https://api.netbird.io/api/routes" assert json.loads(captured["body"]) == { + "description": "sandcat route reach-api", + "network_id": "reach-api", "network": "10.8.0.0/24", "peer": "peer-abc", "enabled": True, + "metric": 9999, + "masquerade": True, + "groups": ["grp-test"], + "keep_route": False, } assert updated.route_id == "route-new" -def test_rest_enable_binding_patches_existing_route(monkeypatch): +def test_rest_enable_binding_puts_existing_route(monkeypatch): captured: dict = {} + routes = [_route_record(id="route-1")] def fake_urlopen(request): captured["method"] = request.get_method() captured["url"] = request.full_url captured["body"] = request.data.decode() if request.data else None + if request.get_method() == "GET" and request.full_url.endswith("/api/routes"): + return BytesIO(json.dumps(routes).encode()) + if request.get_method() == "GET" and request.full_url.endswith("/api/routes/route-1"): + return BytesIO(json.dumps(_route_record(id="route-1")).encode()) return BytesIO(b"") monkeypatch.setenv("NB_API_TOKEN", "test-token") @@ -174,19 +207,85 @@ def fake_urlopen(request): binding = _binding(route_id="route-1") updated = client.enable_binding(binding) - assert captured["method"] == "PATCH" + assert captured["method"] == "PUT" assert captured["url"] == "https://api.netbird.io/api/routes/route-1" - assert json.loads(captured["body"]) == {"enabled": True} + assert json.loads(captured["body"])["enabled"] is True assert updated.route_id == "route-1" -def test_rest_disable_binding_patches_route_disabled(monkeypatch): +def test_rest_enable_binding_reuses_existing_route_without_route_id(monkeypatch): + calls: list[tuple[str, str]] = [] + routes = [_route_record(id="route-existing")] + + def fake_urlopen(request): + method = request.get_method() + url = request.full_url + calls.append((method, url)) + if method == "GET" and url.endswith("/api/routes"): + return BytesIO(json.dumps(routes).encode()) + if method == "GET" and url.endswith("/api/routes/route-existing"): + return BytesIO(json.dumps(_route_record(id="route-existing")).encode()) + return BytesIO(b"") + + monkeypatch.setenv("NB_API_TOKEN", "test-token") + monkeypatch.setattr("capability_runtime.netbird_client.urlopen", fake_urlopen) + + client = RestNetBirdClient() + updated = client.enable_binding(_binding()) + + assert ("PUT", "https://api.netbird.io/api/routes/route-existing") in calls + assert updated.route_id == "route-existing" + assert all(method != "POST" for method, _ in calls) + + +def test_rest_enable_binding_retries_put_after_duplicate_post(monkeypatch): + calls: list[tuple[str, str]] = [] + route_list_calls = 0 + routes = [_route_record(id="route-existing")] + + def fake_urlopen(request): + nonlocal route_list_calls + method = request.get_method() + url = request.full_url + calls.append((method, url)) + if method == "GET" and url.endswith("/api/routes"): + route_list_calls += 1 + if route_list_calls == 1: + return BytesIO(b"[]") + return BytesIO(json.dumps(routes).encode()) + if method == "GET" and url.endswith("/api/routes/route-existing"): + return BytesIO(json.dumps(_route_record(id="route-existing")).encode()) + if method == "POST" and url.endswith("/api/routes"): + raise HTTPError( + url, + 422, + "Unprocessable Entity", + hdrs=None, + fp=BytesIO(b'{"message":"route already exists","code":422}'), + ) + return BytesIO(b"") + + monkeypatch.setenv("NB_API_TOKEN", "test-token") + monkeypatch.setenv("NB_ROUTE_GROUPS", "grp-test") + monkeypatch.setattr("capability_runtime.netbird_client.urlopen", fake_urlopen) + + client = RestNetBirdClient() + updated = client.enable_binding(_binding()) + + assert ("POST", "https://api.netbird.io/api/routes") in calls + assert ("PUT", "https://api.netbird.io/api/routes/route-existing") in calls + assert updated.route_id == "route-existing" + + +def test_rest_disable_binding_puts_route_disabled(monkeypatch): captured: dict = {} def fake_urlopen(request): captured["method"] = request.get_method() captured["url"] = request.full_url captured["body"] = request.data.decode() if request.data else None + if request.get_method() == "GET": + return BytesIO(json.dumps(_route_record(id="route-1", enabled=True)).encode()) return BytesIO(b"") monkeypatch.setenv("NB_API_TOKEN", "test-token") @@ -195,9 +294,9 @@ def fake_urlopen(request): client = RestNetBirdClient() client.disable_binding(_binding(route_id="route-1")) - assert captured["method"] == "PATCH" + assert captured["method"] == "PUT" assert captured["url"] == "https://api.netbird.io/api/routes/route-1" - assert json.loads(captured["body"]) == {"enabled": False} + assert json.loads(captured["body"])["enabled"] is False def test_rest_disable_binding_peer_remove_deletes_peer(monkeypatch): diff --git a/capability-runtime/tests/test_route_watcher_routes.py b/capability-runtime/tests/test_route_watcher_routes.py index 35bd58f1..c56bfc1e 100644 --- a/capability-runtime/tests/test_route_watcher_routes.py +++ b/capability-runtime/tests/test_route_watcher_routes.py @@ -1,6 +1,11 @@ from capability_runtime.netbird_client import MockNetBirdClient from capability_runtime.network import NetworkBinding, SyncMode +from capability_runtime.catalog import LifecycleState +from capability_runtime.route_watcher import RouteDisappearanceWatcher +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity from capability_runtime.types import CapabilityRef +import json def test_get_route_state_enabled(): @@ -117,11 +122,6 @@ def flaky_disable(binding): def test_watcher_ignores_declared_binding_when_route_disabled(tmp_path): - from capability_runtime.catalog import LifecycleState - from capability_runtime.route_watcher import RouteDisappearanceWatcher - from capability_runtime.runtime import CapabilityRuntime - from capability_runtime.types import CapabilityRef - client = MockNetBirdClient( peers=[{"id": "peer-abc"}], routes=[{"id": "route-1", "network": "10.8.0.0/24", "peer": "peer-abc", "enabled": False}], @@ -134,3 +134,31 @@ def test_watcher_ignores_declared_binding_when_route_disabled(tmp_path): RouteDisappearanceWatcher(runtime, client).poll_once() assert runtime.catalog.get_state(ref) == LifecycleState.DECLARED + + +def test_watcher_emits_event_when_reconcile_disable_fails(tmp_path, monkeypatch): + client = MockNetBirdClient( + peers=[{"id": "peer-abc"}], + routes=[{"id": "route-1", "network": "10.8.0.0/24", "peer": "peer-abc", "enabled": True}], + ) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-rw-fail", 7, netbird_client=client) + agent = AgentIdentity("agent-1") + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", "route-1", SyncMode.ROUTE_ENABLE) + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + runtime.request_capability_lease(agent, agent, ref, "need api") + runtime.revoke_capability(AgentIdentity("operator"), ref, "operator revoke") + client._routes[0]["enabled"] = True + + def always_fail(_binding): + raise RuntimeError("netbird still down") + + monkeypatch.setattr(client, "disable_binding", always_fail) + + RouteDisappearanceWatcher(runtime, client).poll_once() + + events = [json.loads(line) for line in (tmp_path / "t.jsonl").read_text().splitlines() if line] + reconcile_failures = [e for e in events if e.get("event") == "physical_reconcile_failed"] + assert reconcile_failures + assert reconcile_failures[-1]["capability_ref"] == "cap-reach-api" + assert reconcile_failures[-1]["route_id"] == "route-1" diff --git a/capability-runtime/tests/test_rpc_dispatcher.py b/capability-runtime/tests/test_rpc_dispatcher.py index a8c17524..594d28a4 100644 --- a/capability-runtime/tests/test_rpc_dispatcher.py +++ b/capability-runtime/tests/test_rpc_dispatcher.py @@ -149,3 +149,35 @@ def test_invalid_request_missing_method(runtime): ) response = dispatcher.handle({"jsonrpc": "2.0", "id": 1}) assert response["error"]["code"] == -32600 + + +def test_lease_returns_rpc_error_when_netbird_grant_raises(tmp_path): + from capability_runtime.netbird_client import MockNetBirdClient + from capability_runtime.network import NetworkBinding, SyncMode + + client = MockNetBirdClient(peers=[{"id": "peer-abc"}], routes=[]) + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-rpc", 401, netbird_client=client) + ref = CapabilityRef("cap-reach-api") + binding = NetworkBinding(ref, "peer-abc", "10.8.0.0/24", "route-bad", SyncMode.ROUTE_ENABLE) + runtime.register_network_capability("reach_api", ref, binding, LifecycleState.VISIBLE) + + def boom(*_a, **_kw): + raise RuntimeError("HTTP Error 404: Not Found") + + client.enable_binding = boom # type: ignore[method-assign] + + dispatcher = RpcDispatcher(runtime, surface="admin", bound_agent_id="operator") + response = dispatcher.handle( + { + "jsonrpc": "2.0", + "id": 1, + "method": "capability.lease", + "params": { + "agent_id": "devcontainer-agent", + "capability_ref": "cap-reach-api", + "justification": "gate", + }, + } + ) + assert "error" in response + assert "404" in response["error"]["message"] diff --git a/capability-runtime/tests/test_runtime_network.py b/capability-runtime/tests/test_runtime_network.py index 53011416..4d73b67d 100644 --- a/capability-runtime/tests/test_runtime_network.py +++ b/capability-runtime/tests/test_runtime_network.py @@ -151,7 +151,7 @@ def test_ttl_expiry_disables_network_binding(tmp_path): from capability_runtime.netbird_client import MockNetBirdClient from capability_runtime.network import NetworkBinding from capability_runtime.runtime import CapabilityRuntime - from capability_runtime.types import AgentIdentity, CapabilityRef, LeaseId + from capability_runtime.types import AgentIdentity, CapabilityRef client = MockNetBirdClient( peers=[{"id": "peer-ttl", "connected": True}], @@ -162,25 +162,51 @@ def test_ttl_expiry_disables_network_binding(tmp_path): ref = CapabilityRef("cap-ttl-test") binding = NetworkBinding(ref, "peer-ttl", "172.16.0.0/24", "route-ttl") runtime.register_network_capability("ttl_net", ref, binding, LifecycleState.DECLARED) - - # Lease with short TTL + decision = runtime.request_capability_lease(agent, agent, ref, "testing ttl") - - # Force expiry by manipulating lease expiry time - runtime.lease_manager._leases[decision.lease_id].expires_at = datetime.now(timezone.utc) - timedelta(seconds=1) - - # Check capabilities - should detect expired lease and disable binding + runtime.lease_manager._leases[decision.lease_id].expires_at = ( + datetime.now(timezone.utc) - timedelta(seconds=1) + ) + bundle = runtime.check_current_capabilities(agent, {}) - - # Peer should still exist, route should be disabled + assert client.peer_exists("peer-ttl") - assert client.route_exists("route-ttl") routes = [r for r in client.list_routes() if r["id"] == "route-ttl"] assert routes[0]["enabled"] is False - # Capability should not be in bundle assert "ttl_net" not in [n.name for n in bundle.networks] +def test_ttl_expiry_via_watcher_poll(tmp_path): + """Watcher poll processes TTL expiry without requiring capability.check.""" + from datetime import datetime, timedelta, timezone + from capability_runtime.catalog import LifecycleState + from capability_runtime.netbird_client import MockNetBirdClient + from capability_runtime.network import NetworkBinding, SyncMode + from capability_runtime.route_watcher import RouteDisappearanceWatcher + from capability_runtime.runtime import CapabilityRuntime + from capability_runtime.types import AgentIdentity, CapabilityRef + + client = MockNetBirdClient( + peers=[{"id": "peer-ttl", "connected": True}], + routes=[{"id": "route-ttl", "network": "172.16.0.0/24", "enabled": True}], + ) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace-ttl-w", 3, netbird_client=client) + agent = AgentIdentity("agent-ttl") + ref = CapabilityRef("cap-ttl-watcher") + binding = NetworkBinding(ref, "peer-ttl", "172.16.0.0/24", "route-ttl", SyncMode.ROUTE_ENABLE) + runtime.register_network_capability("ttl_net", ref, binding, LifecycleState.VISIBLE) + decision = runtime.request_capability_lease(agent, agent, ref, "testing ttl") + runtime.lease_manager._leases[decision.lease_id].expires_at = ( + datetime.now(timezone.utc) - timedelta(seconds=1) + ) + + RouteDisappearanceWatcher(runtime, client).poll_once() + + routes = [r for r in client.list_routes() if r["id"] == "route-ttl"] + assert routes[0]["enabled"] is False + assert runtime.catalog.get_state(ref) == LifecycleState.EXPIRED + + def test_ttl_expiry_continues_when_netbird_disable_fails(tmp_path, monkeypatch): from datetime import datetime, timedelta, timezone from capability_runtime.catalog import LifecycleState diff --git a/capability-runtime/tests/test_settings.py b/capability-runtime/tests/test_settings.py index 3a2b58ec..474e19e4 100644 --- a/capability-runtime/tests/test_settings.py +++ b/capability-runtime/tests/test_settings.py @@ -1,6 +1,6 @@ import json -from capability_runtime.settings import load_netbird_credentials +from capability_runtime.settings import load_netbird_credentials, load_netbird_route_groups def test_load_credentials_project_over_user(tmp_path, monkeypatch): @@ -22,3 +22,72 @@ def test_load_credentials_project_over_user(tmp_path, monkeypatch): creds = load_netbird_credentials() assert creds.api_token == "proj-tok" assert creds.management_url == "https://mgmt.example.com" + + +def test_load_credentials_uses_enrollment_url_when_management_is_localhost( + tmp_path, monkeypatch +): + settings = tmp_path / "settings.json" + settings.write_text( + json.dumps( + { + "netbird_api_token": "tok", + "netbird_management_url": "http://localhost:33073", + "netbird_enrollment_management_url": "http://192.168.5.2:33073", + } + ) + ) + monkeypatch.setenv("SANDCAT_SETTINGS_USER", str(settings)) + monkeypatch.delenv("SANDCAT_SETTINGS_PROJECT", raising=False) + monkeypatch.delenv("NB_MANAGEMENT_URL", raising=False) + + creds = load_netbird_credentials() + assert creds.management_url == "http://192.168.5.2:33073" + + +def test_load_credentials_keeps_remote_management_url(tmp_path, monkeypatch): + settings = tmp_path / "settings.json" + settings.write_text( + json.dumps( + { + "netbird_api_token": "tok", + "netbird_management_url": "https://netbird.example.com", + "netbird_enrollment_management_url": "http://192.168.5.2:33073", + } + ) + ) + monkeypatch.setenv("SANDCAT_SETTINGS_USER", str(settings)) + + creds = load_netbird_credentials() + assert creds.management_url == "https://netbird.example.com" + + +def test_load_credentials_loopback_management_without_enrollment_unchanged( + tmp_path, monkeypatch +): + settings = tmp_path / "settings.json" + settings.write_text( + json.dumps( + { + "netbird_api_token": "tok", + "netbird_management_url": "http://127.0.0.1:33073", + } + ) + ) + monkeypatch.setenv("SANDCAT_SETTINGS_USER", str(settings)) + + creds = load_netbird_credentials() + assert creds.management_url == "http://127.0.0.1:33073" + + +def test_load_route_groups_from_env(monkeypatch): + monkeypatch.setenv("NB_ROUTE_GROUPS", "grp-a, grp-b") + assert load_netbird_route_groups() == ["grp-a", "grp-b"] + + +def test_load_route_groups_from_settings_json_array(tmp_path, monkeypatch): + settings = tmp_path / "settings.json" + settings.write_text(json.dumps({"netbird_route_groups": ["grp-x", "grp-y"]})) + monkeypatch.setenv("SANDCAT_SETTINGS_USER", str(settings)) + monkeypatch.delenv("NB_ROUTE_GROUPS", raising=False) + assert load_netbird_route_groups() == ["grp-x", "grp-y"] diff --git a/cli/lib/netbird.bash b/cli/lib/netbird.bash index 68dc17cb..22a9c09d 100644 --- a/cli/lib/netbird.bash +++ b/cli/lib/netbird.bash @@ -434,23 +434,142 @@ netbird_status() { netbird_api "GET" "/api/peers" } +# Derives a NetBird network_id (1-40 chars) from a CIDR when none is supplied. +netbird_default_network_id() { + local network=$1 + local id + id=${network//./-} + id=${id//\//-} + printf '%.40s' "$id" +} + +# URL-encode a path segment for safe use in API paths. +# Args: +# $1 - Raw path segment +netbird_urlencode_path_segment() { + local raw=$1 + local encoded="" + local i ch hex + for ((i = 0; i < ${#raw}; i++)); do + ch=${raw:i:1} + case "$ch" in + [a-zA-Z0-9.~_-]) encoded+="$ch" ;; + *) + printf -v hex '%02X' "'$ch" + encoded+="%${hex}" + ;; + esac + done + printf '%s' "$encoded" +} + +# Resolves NetBird route distribution group IDs (non-empty). +# Order: NB_ROUTE_GROUPS, netbird_route_groups setting, GET /api/groups (All). +# Prints one group ID per line. +netbird_route_distribution_group_ids() { + local raw id + + if [[ -n "${NB_ROUTE_GROUPS:-}" ]]; then + local IFS=, + for id in $NB_ROUTE_GROUPS; do + id=${id//[[:space:]]/} + [[ -n "$id" ]] && printf '%s\n' "$id" + done + return 0 + fi + + raw=$(netbird_read_setting netbird_route_groups) + if [[ -n "$raw" ]]; then + if [[ "$raw" == \[* ]]; then + require jq + jq -r '.[]' <<<"$raw" + return 0 + fi + local IFS=, + for id in $raw; do + id=${id//[[:space:]]/} + [[ -n "$id" ]] && printf '%s\n' "$id" + done + return 0 + fi + + require jq + local groups + groups=$(netbird_api "GET" "/api/groups?name=All") || return 1 + id=$(jq -r '.[] | select(.name == "All") | .id' <<<"$groups" | head -n1) + if [[ -z "$id" ]]; then + groups=$(netbird_api "GET" "/api/groups") || return 1 + id=$(jq -r '.[0].id // empty' <<<"$groups") + fi + if [[ -z "$id" ]]; then + echo "No NetBird distribution groups found; set netbird_route_groups in settings or NB_ROUTE_GROUPS" >&2 + return 1 + fi + printf '%s\n' "$id" +} + +# Builds the JSON body for POST /api/routes (all NetBird-required fields). +# Args: +# $1 - Network CIDR +# $2 - Peer ID +# $3 - network_id +# $4 - metric (optional, default 9999) +netbird_route_create_body() { + local network=$1 peer_id=$2 network_id=$3 metric=${4:-9999} + require jq + + local -a group_ids=() + local gid + while IFS= read -r gid; do + [[ -n "$gid" ]] && group_ids+=("$gid") + done < <(netbird_route_distribution_group_ids) || return 1 + if [[ ${#group_ids[@]} -eq 0 ]]; then + echo "NetBird route distribution groups list is empty" >&2 + return 1 + fi + + local groups_json + groups_json=$(printf '%s\n' "${group_ids[@]}" | jq -R . | jq -s .) + + jq -nc \ + --arg description "sandcat route ${network_id}" \ + --arg network_id "$network_id" \ + --arg network "$network" \ + --arg peer "$peer_id" \ + --argjson metric "$metric" \ + --argjson groups "$groups_json" \ + '{description: $description, network_id: $network_id, enabled: true, peer: $peer, network: $network, metric: $metric, masquerade: true, groups: $groups, keep_route: false}' +} + # Adds a network route served by a peer. # Args: # $1 - Network CIDR (e.g. 10.8.0.0/24) # $2 - Peer ID that serves the route +# $3 - Optional network_id (defaults to a slug derived from the CIDR) +# $4 - Optional route metric 1-9999 (default 9999; lower = higher priority) netbird_route_add() { local network=$1 local peer_id=$2 - netbird_api "POST" "/api/routes" \ - "{\"network\":\"$network\",\"peer\":\"$peer_id\",\"enabled\":true}" + local network_id=${3:-$(netbird_default_network_id "$network")} + local metric=${4:-9999} + local body + + body=$(netbird_route_create_body "$network" "$peer_id" "$network_id" "$metric") || return 1 + netbird_api "POST" "/api/routes" "$body" } -# Enables an existing network route by ID. +# Enables an existing network route by ID (GET + PUT; NetBird has no PATCH). # Args: # $1 - Route ID netbird_route_enable() { local route_id=$1 - netbird_api "PATCH" "/api/routes/$route_id" '{"enabled":true}' + local route_id_escaped + route_id_escaped=$(netbird_urlencode_path_segment "$route_id") + require jq + local route body + route=$(netbird_api "GET" "/api/routes/$route_id_escaped") || return 1 + body=$(jq '.enabled = true | del(.id, .network_type)' <<<"$route") + netbird_api "PUT" "/api/routes/$route_id_escaped" "$body" } # Disables an existing network route by ID without deleting it. @@ -458,7 +577,13 @@ netbird_route_enable() { # $1 - Route ID netbird_route_disable() { local route_id=$1 - netbird_api "PATCH" "/api/routes/$route_id" '{"enabled":false}' + local route_id_escaped + route_id_escaped=$(netbird_urlencode_path_segment "$route_id") + require jq + local route body + route=$(netbird_api "GET" "/api/routes/$route_id_escaped") || return 1 + body=$(jq '.enabled = false | del(.id, .network_type)' <<<"$route") + netbird_api "PUT" "/api/routes/$route_id_escaped" "$body" } # Removes a network route by ID. @@ -466,7 +591,9 @@ netbird_route_disable() { # $1 - Route ID (returned by netbird_route_add) netbird_route_remove() { local route_id=$1 - netbird_api "DELETE" "/api/routes/$route_id" + local route_id_escaped + route_id_escaped=$(netbird_urlencode_path_segment "$route_id") + netbird_api "DELETE" "/api/routes/$route_id_escaped" } # Removes a peer from the NetBird management server. @@ -476,7 +603,9 @@ netbird_route_remove() { # $1 - Peer ID netbird_peer_remove() { local peer_id=$1 - netbird_api "DELETE" "/api/peers/$peer_id" + local peer_id_escaped + peer_id_escaped=$(netbird_urlencode_path_segment "$peer_id") + netbird_api "DELETE" "/api/peers/$peer_id_escaped" } # Returns the provisioned self-hosted NetBird server directory. diff --git a/cli/libexec/netbird/netbird b/cli/libexec/netbird/netbird index 16a35d87..7c4e1edc 100755 --- a/cli/libexec/netbird/netbird +++ b/cli/libexec/netbird/netbird @@ -16,7 +16,7 @@ Subcommands: server stop Stop the local self-hosted NetBird server stack server status Show docker compose status for the local server peer remove --peer-id Remove a peer; wg-client drops the route from wt0 - route add --network --peer-id Add a network route + route add --network --peer-id [--network-id ] [--metric ] Add a network route route remove --route-id Remove a route by ID EOF } @@ -99,7 +99,7 @@ cmd_route() { shift || true case "$subcmd" in add) - local network="" peer_id="" + local network="" peer_id="" network_id="" metric="" while [[ $# -gt 0 ]]; do case $1 in --network) @@ -118,6 +118,22 @@ cmd_route() { peer_id="$2" shift 2 ;; + --network-id) + if [[ $# -lt 2 || "$2" == --* ]]; then + echo "Option --network-id requires a value" | error + return 1 + fi + network_id="$2" + shift 2 + ;; + --metric) + if [[ $# -lt 2 || "$2" == --* ]]; then + echo "Option --metric requires a value" | error + return 1 + fi + metric="$2" + shift 2 + ;; *) echo "Unknown option: $1" | error; return 1 ;; esac done @@ -131,8 +147,16 @@ cmd_route() { echo "--network must be a CIDR (e.g. 10.8.0.0/24), not an interface name" | error return 1 fi + if [[ -n "$metric" ]]; then + if [[ ! "$metric" =~ ^[0-9]+$ ]] || (( metric < 1 || metric > 9999 )); then + echo "Option --metric must be an integer between 1 and 9999" | error + return 1 + fi + fi local response - response=$(netbird_route_add "$network" "$peer_id") || return 1 + local rid="${network_id:-$(netbird_default_network_id "$network")}" + local route_metric="${metric:-9999}" + response=$(netbird_route_add "$network" "$peer_id" "$rid" "$route_metric") || return 1 if command -v jq &>/dev/null; then echo "$response" | jq . else diff --git a/cli/templates/devcontainer/sandcat/capability-catalog.json b/cli/templates/devcontainer/sandcat/capability-catalog.json index 5610295b..2708f6b8 100644 --- a/cli/templates/devcontainer/sandcat/capability-catalog.json +++ b/cli/templates/devcontainer/sandcat/capability-catalog.json @@ -9,9 +9,8 @@ "name": "reach_api", "ref": "cap-reach-api", "type": "network", - "peer_id": "peer-placeholder", + "peer_id": "REPLACE_WITH_NETBIRD_PEER_ID", "network": "10.8.0.0/24", - "route_id": "route-placeholder", "sync_mode": "route_enable" } ] diff --git a/cli/test/composefile/netbird_contract.bats b/cli/test/composefile/netbird_contract.bats index fea8d8c0..e084266e 100644 --- a/cli/test/composefile/netbird_contract.bats +++ b/cli/test/composefile/netbird_contract.bats @@ -37,3 +37,13 @@ setup() { [[ "$NETBIRD_SHA256_AMD64" =~ ^[0-9a-f]{64}$ ]] [[ "$NETBIRD_SHA256_ARM64" =~ ^[0-9a-f]{64}$ ]] } + +@test "Dockerfile.wg-client verifies netbird tarball with sha256sum" { + local dockerfile="$SCT_TEMPLATEDIR/devcontainer/sandcat/Dockerfile.wg-client" + run grep -F 'sha256sum -c -' "$dockerfile" + assert_success + run grep -F 'NETBIRD_SHA256_AMD64' "$dockerfile" + assert_success + run grep -F 'NETBIRD_SHA256_ARM64' "$dockerfile" + assert_success +} diff --git a/cli/test/netbird/netbird.bats b/cli/test/netbird/netbird.bats index cc8e1b5a..4e26ce05 100644 --- a/cli/test/netbird/netbird.bats +++ b/cli/test/netbird/netbird.bats @@ -6,6 +6,7 @@ setup() { NETBIRD_CMD="$SCT_LIBEXECDIR/netbird/netbird" export NB_MANAGEMENT_URL="https://api.netbird.io" export NB_API_TOKEN="test-token" + export NB_ROUTE_GROUPS="grp-test" } teardown() { @@ -74,12 +75,20 @@ teardown() { @test "netbird route add calls netbird_route_add" { stub curl \ - "-sS -X POST -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"abc123\",\"enabled\":true}' -w * https://api.netbird.io/api/routes : printf '%s\n200' '{\"id\":\"route1\"}'" + "-sS -X POST -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -d '{\"description\":\"sandcat route 10-8-0-0-24\",\"network_id\":\"10-8-0-0-24\",\"enabled\":true,\"peer\":\"abc123\",\"network\":\"10.8.0.0/24\",\"metric\":9999,\"masquerade\":true,\"groups\":[\"grp-test\"],\"keep_route\":false}' -w * https://api.netbird.io/api/routes : printf '%s\n200' '{\"id\":\"route1\"}'" run bash "$NETBIRD_CMD" route add --network 10.8.0.0/24 --peer-id abc123 assert_success assert_output --partial "route1" } +@test "netbird route add passes explicit --network-id" { + stub curl \ + "-sS -X POST -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -d '{\"description\":\"sandcat route reach-api\",\"network_id\":\"reach-api\",\"enabled\":true,\"peer\":\"abc123\",\"network\":\"100.79.107.115/32\",\"metric\":9999,\"masquerade\":true,\"groups\":[\"grp-test\"],\"keep_route\":false}' -w * https://api.netbird.io/api/routes : printf '%s\n200' '{\"id\":\"route1\"}'" + run bash "$NETBIRD_CMD" route add --network 100.79.107.115/32 --peer-id abc123 --network-id reach-api + assert_success + assert_output --partial "route1" +} + @test "netbird route remove requires --route-id" { run bash "$NETBIRD_CMD" route remove assert_failure @@ -99,6 +108,20 @@ teardown() { assert_success } +@test "netbird route remove URL-encodes route id path segment" { + stub curl \ + "-sS -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://api.netbird.io/api/routes/route%2Fone%3Fx%3D1 : printf '\n200'" + run bash "$NETBIRD_CMD" route remove --route-id "route/one?x=1" + assert_success +} + +@test "netbird peer remove URL-encodes peer id path segment" { + stub curl \ + "-sS -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://api.netbird.io/api/peers/peer%2Fabc%3Ffoo%3Dbar : printf '\n200'" + run bash "$NETBIRD_CMD" peer remove --peer-id "peer/abc?foo=bar" + assert_success +} + @test "netbird with unknown subcommand prints usage and fails" { run bash "$NETBIRD_CMD" bogus assert_failure diff --git a/cli/test/netbird/netbird_api.bats b/cli/test/netbird/netbird_api.bats index acedb18e..8cfb704f 100644 --- a/cli/test/netbird/netbird_api.bats +++ b/cli/test/netbird/netbird_api.bats @@ -8,6 +8,7 @@ setup() { mkdir -p "$HOME/.config/sandcat" export NB_MANAGEMENT_URL="https://api.netbird.io" export NB_API_TOKEN="test-token" + export NB_ROUTE_GROUPS="grp-test" } teardown() { @@ -29,9 +30,26 @@ teardown() { assert_output --partial "peer1" } -@test "netbird_route_add calls POST /api/routes with network and peer" { +@test "netbird_route_add calls POST /api/routes with required fields" { stub curl \ - "-sS -X POST -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -d '{\"network\":\"10.8.0.0/24\",\"peer\":\"peer1\",\"enabled\":true}' -w * https://api.netbird.io/api/routes : printf '%s\n200' '{\"id\":\"route1\"}'" + "-sS -X POST -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -d '{\"description\":\"sandcat route 10-8-0-0-24\",\"network_id\":\"10-8-0-0-24\",\"enabled\":true,\"peer\":\"peer1\",\"network\":\"10.8.0.0/24\",\"metric\":9999,\"masquerade\":true,\"groups\":[\"grp-test\"],\"keep_route\":false}' -w * https://api.netbird.io/api/routes : printf '%s\n200' '{\"id\":\"route1\"}'" + run netbird_route_add "10.8.0.0/24" "peer1" + assert_success +} + +@test "netbird_route_add accepts explicit network_id" { + stub curl \ + "-sS -X POST -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -d '{\"description\":\"sandcat route reach-api\",\"network_id\":\"reach-api\",\"enabled\":true,\"peer\":\"peer1\",\"network\":\"10.8.0.0/24\",\"metric\":9999,\"masquerade\":true,\"groups\":[\"grp-test\"],\"keep_route\":false}' -w * https://api.netbird.io/api/routes : printf '%s\n200' '{\"id\":\"route1\"}'" + run netbird_route_add "10.8.0.0/24" "peer1" "reach-api" + assert_success +} + +@test "netbird_route_add discovers All group when groups unset" { + unset NB_ROUTE_GROUPS + stub curl \ + "-sS -X GET -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://api.netbird.io/api/groups?name=All : printf '%s\n200' '[{\"id\":\"grp-all\",\"name\":\"All\"}]'" + stub curl \ + "-sS -X POST -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -d '{\"description\":\"sandcat route 10-8-0-0-24\",\"network_id\":\"10-8-0-0-24\",\"enabled\":true,\"peer\":\"peer1\",\"network\":\"10.8.0.0/24\",\"metric\":9999,\"masquerade\":true,\"groups\":[\"grp-all\"],\"keep_route\":false}' -w * https://api.netbird.io/api/routes : printf '%s\n200' '{\"id\":\"route1\"}'" run netbird_route_add "10.8.0.0/24" "peer1" assert_success } @@ -85,3 +103,23 @@ teardown() { run netbird_management_base_url assert_output "http://localhost:33073" } + +@test "netbird_urlencode_path_segment encodes reserved characters" { + run netbird_urlencode_path_segment "route/one?x=1" + assert_success + assert_output "route%2Fone%3Fx%3D1" +} + +@test "netbird_route_add escapes peer_id in JSON body" { + stub curl \ + "-sS -X POST -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -d '{\"description\":\"sandcat route 10-8-0-0-24\",\"network_id\":\"10-8-0-0-24\",\"enabled\":true,\"peer\":\"peer\\\"evil\",\"network\":\"10.8.0.0/24\",\"metric\":9999,\"masquerade\":true,\"groups\":[\"grp-test\"],\"keep_route\":false}' -w * https://api.netbird.io/api/routes : printf '%s\n200' '{\"id\":\"route1\"}'" + run netbird_route_add "10.8.0.0/24" 'peer"evil' + assert_success +} + +@test "netbird_route_remove URL-encodes route id path segment" { + stub curl \ + "-sS -X DELETE -H 'Authorization: Token test-token' -H 'Accept: application/json' -H 'Content-Type: application/json' -w * https://api.netbird.io/api/routes/route%2Fone%3Fx%3D1 : printf '\n200'" + run netbird_route_remove "route/one?x=1" + assert_success +} From 2006c6646d42b5132898b958c89b93b884a36e96 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Wed, 8 Jul 2026 21:23:51 +0000 Subject: [PATCH 062/138] feat(cli): add proxy-peer hello HTTP server for mesh smoke --- .../sandcat/scripts/proxy-peer-hello.py | 36 +++++++++++++++++++ cli/test/proxy_peer/proxy_peer_hello.bats | 18 ++++++++++ cli/test/proxy_peer/test_helper.bash | 24 +++++++++++++ 3 files changed, 78 insertions(+) create mode 100755 cli/templates/devcontainer/sandcat/scripts/proxy-peer-hello.py create mode 100644 cli/test/proxy_peer/proxy_peer_hello.bats create mode 100644 cli/test/proxy_peer/test_helper.bash diff --git a/cli/templates/devcontainer/sandcat/scripts/proxy-peer-hello.py b/cli/templates/devcontainer/sandcat/scripts/proxy-peer-hello.py new file mode 100755 index 00000000..6983d4bf --- /dev/null +++ b/cli/templates/devcontainer/sandcat/scripts/proxy-peer-hello.py @@ -0,0 +1,36 @@ +#!/usr/bin/env python3 +"""Minimal HTTP hello for proxy-peer mesh smoke tests.""" +from __future__ import annotations + +import argparse +import json +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer + + +class Handler(BaseHTTPRequestHandler): + def do_GET(self) -> None: # noqa: N802 + if self.path not in ("/hello", "/hello/"): + self.send_error(404) + return + body = json.dumps({"service": "proxy-peer", "ok": True}).encode() + self.send_response(200) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def log_message(self, format: str, *args: object) -> None: + return + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument("--host", default="0.0.0.0") + parser.add_argument("--port", type=int, default=8080) + args = parser.parse_args() + server = ThreadingHTTPServer((args.host, args.port), Handler) + server.serve_forever() + + +if __name__ == "__main__": + main() diff --git a/cli/test/proxy_peer/proxy_peer_hello.bats b/cli/test/proxy_peer/proxy_peer_hello.bats new file mode 100644 index 00000000..41aac308 --- /dev/null +++ b/cli/test/proxy_peer/proxy_peer_hello.bats @@ -0,0 +1,18 @@ +#!/usr/bin/env bats + +setup() { + load test_helper + export HELLO="$SCT_TEMPLATEDIR/devcontainer/sandcat/scripts/proxy-peer-hello.py" +} + +teardown() { + pkill -f "proxy-peer-hello.py --port 18080" || true +} + +@test "proxy-peer-hello responds with JSON on /hello" { + python3 "$HELLO" --port 18080 & + sleep 0.5 + run curl -sf http://127.0.0.1:18080/hello + assert_success + assert_output --partial '"service": "proxy-peer"' +} diff --git a/cli/test/proxy_peer/test_helper.bash b/cli/test/proxy_peer/test_helper.bash new file mode 100644 index 00000000..bf20e08c --- /dev/null +++ b/cli/test/proxy_peer/test_helper.bash @@ -0,0 +1,24 @@ +#!/bin/bash + +bats_require_minimum_version 1.5.0 + +# Enable Bash 3.2 compat mode when running on Bash 4.4+ +# On actual Bash 3.2 (macOS default), these options don't exist and aren't needed. +if shopt -s compat32 2>/dev/null; then + export BASH_COMPAT=3.2 +fi +set -uo pipefail +export SHELLOPTS + +SCT_ROOT="$BATS_TEST_DIRNAME/../.." + +BATS_LIB_PATH="$SCT_ROOT/support":${BATS_LIB_PATH-} + +bats_load_library bats-ext +bats_load_library bats-support +bats_load_library bats-assert +bats_load_library bats-mock-ext + +export SCT_ROOT +export SCT_LIBDIR="$SCT_ROOT/lib" +export SCT_TEMPLATEDIR="$SCT_ROOT/templates" From 086aebc781c3af4cd2b6f3a96f78e4e9331cdfcb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Wed, 8 Jul 2026 21:28:44 +0000 Subject: [PATCH 063/138] test(cli): harden proxy-peer hello bats harness --- cli/test/proxy_peer/proxy_peer_hello.bats | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/cli/test/proxy_peer/proxy_peer_hello.bats b/cli/test/proxy_peer/proxy_peer_hello.bats index 41aac308..fd96b4ed 100644 --- a/cli/test/proxy_peer/proxy_peer_hello.bats +++ b/cli/test/proxy_peer/proxy_peer_hello.bats @@ -6,13 +6,25 @@ setup() { } teardown() { - pkill -f "proxy-peer-hello.py --port 18080" || true + if [[ -n "${HELLO_PID:-}" ]]; then + kill "$HELLO_PID" 2>/dev/null || true + wait "$HELLO_PID" 2>/dev/null || true + fi } @test "proxy-peer-hello responds with JSON on /hello" { python3 "$HELLO" --port 18080 & - sleep 0.5 + HELLO_PID=$! + export HELLO_PID + + for _ in $(seq 1 20); do + if curl -sf http://127.0.0.1:18080/hello >/dev/null 2>&1; then + break + fi + sleep 0.1 + done + run curl -sf http://127.0.0.1:18080/hello assert_success - assert_output --partial '"service": "proxy-peer"' + assert_output '{"service": "proxy-peer", "ok": true}' } From ad596769997eac6d3d3ab707045bd55bfa859a3e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Wed, 8 Jul 2026 21:29:49 +0000 Subject: [PATCH 064/138] feat(cli): add proxy-peer NetBird enrollment init script Co-authored-by: Cursor --- .../sandcat/scripts/proxy-peer-init.sh | 15 +++++++++++++++ cli/test/proxy_peer/proxy_peer_init.bats | 11 +++++++++++ 2 files changed, 26 insertions(+) create mode 100755 cli/templates/devcontainer/sandcat/scripts/proxy-peer-init.sh create mode 100644 cli/test/proxy_peer/proxy_peer_init.bats diff --git a/cli/templates/devcontainer/sandcat/scripts/proxy-peer-init.sh b/cli/templates/devcontainer/sandcat/scripts/proxy-peer-init.sh new file mode 100755 index 00000000..00cca922 --- /dev/null +++ b/cli/templates/devcontainer/sandcat/scripts/proxy-peer-init.sh @@ -0,0 +1,15 @@ +#!/usr/bin/env bash +# cli/templates/devcontainer/sandcat/scripts/proxy-peer-init.sh +set -euo pipefail + +NB_SETUP_KEY="${NB_SETUP_KEY:?NB_SETUP_KEY is required}" +NB_MANAGEMENT_URL="${NB_MANAGEMENT_URL:-}" +HELLO_PORT="${PROXY_PEER_PORT:-8080}" + +if [[ -n "$NB_MANAGEMENT_URL" ]]; then + export NB_MANAGEMENT_URL +fi + +netbird up --setup-key "$NB_SETUP_KEY" + +exec python3 /usr/local/bin/proxy-peer-hello.py --port "$HELLO_PORT" diff --git a/cli/test/proxy_peer/proxy_peer_init.bats b/cli/test/proxy_peer/proxy_peer_init.bats new file mode 100644 index 00000000..ca5842c8 --- /dev/null +++ b/cli/test/proxy_peer/proxy_peer_init.bats @@ -0,0 +1,11 @@ +#!/usr/bin/env bats + +setup() { + load test_helper +} + +@test "proxy-peer-init fails without NB_SETUP_KEY" { + run bash "$SCT_TEMPLATEDIR/devcontainer/sandcat/scripts/proxy-peer-init.sh" + assert_failure + assert_output --partial "NB_SETUP_KEY is required" +} From 492f3dc439f5103ec90a90536945ffce443aa6c2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Wed, 8 Jul 2026 21:31:33 +0000 Subject: [PATCH 065/138] feat(cli): add proxy-peer Docker image and compose stack --- .../sandcat/Dockerfile.proxy-peer | 36 +++++++++++++++++++ .../sandcat/compose-proxy-peer.yml | 16 +++++++++ cli/test/composefile/proxy_peer.bats | 10 ++++++ 3 files changed, 62 insertions(+) create mode 100644 cli/templates/devcontainer/sandcat/Dockerfile.proxy-peer create mode 100644 cli/templates/devcontainer/sandcat/compose-proxy-peer.yml create mode 100644 cli/test/composefile/proxy_peer.bats diff --git a/cli/templates/devcontainer/sandcat/Dockerfile.proxy-peer b/cli/templates/devcontainer/sandcat/Dockerfile.proxy-peer new file mode 100644 index 00000000..194493b6 --- /dev/null +++ b/cli/templates/devcontainer/sandcat/Dockerfile.proxy-peer @@ -0,0 +1,36 @@ +FROM debian:trixie-slim + +RUN apt-get update \ + && apt-get install -y --no-install-recommends ca-certificates curl python3 \ + && rm -rf /var/lib/apt/lists/* + +# Install the NetBird client daemon. The daemon manages wt0 (the NetBird +# overlay mesh) inside this container, which already holds NET_ADMIN. +# +# Version + per-arch checksums are the single source of truth in netbird.env +# (sibling to this Dockerfile) and MUST be supplied as build args. No defaults +# here on purpose, so the pin lives in exactly one place. +ARG NETBIRD_VERSION +ARG NETBIRD_SHA256_AMD64 +ARG NETBIRD_SHA256_ARM64 +RUN test -n "$NETBIRD_VERSION" || { echo "NETBIRD_VERSION build arg is required (source netbird.env)" >&2; exit 1; } \ + && ARCH=$(dpkg --print-architecture) \ + && case "$ARCH" in \ + amd64) NETBIRD_SHA256="$NETBIRD_SHA256_AMD64" ;; \ + arm64) NETBIRD_SHA256="$NETBIRD_SHA256_ARM64" ;; \ + *) echo "Unsupported architecture: $ARCH" >&2; exit 1 ;; \ + esac \ + && test -n "$NETBIRD_SHA256" \ + && curl -sSLf -o /tmp/netbird.tar.gz \ + "https://github.com/netbirdio/netbird/releases/download/v${NETBIRD_VERSION}/netbird_${NETBIRD_VERSION}_linux_${ARCH}.tar.gz" \ + && echo "${NETBIRD_SHA256} /tmp/netbird.tar.gz" | sha256sum -c - \ + && tar xzf /tmp/netbird.tar.gz -C /usr/local/bin netbird \ + && chmod +x /usr/local/bin/netbird \ + && rm /tmp/netbird.tar.gz \ + && netbird version + +COPY scripts/proxy-peer-init.sh /usr/local/bin/proxy-peer-init.sh +COPY scripts/proxy-peer-hello.py /usr/local/bin/proxy-peer-hello.py +RUN chmod +x /usr/local/bin/proxy-peer-init.sh + +ENTRYPOINT ["/usr/local/bin/proxy-peer-init.sh"] diff --git a/cli/templates/devcontainer/sandcat/compose-proxy-peer.yml b/cli/templates/devcontainer/sandcat/compose-proxy-peer.yml new file mode 100644 index 00000000..8e5a6a4b --- /dev/null +++ b/cli/templates/devcontainer/sandcat/compose-proxy-peer.yml @@ -0,0 +1,16 @@ +services: + proxy-peer: + build: + context: . + dockerfile: Dockerfile.proxy-peer + args: + NETBIRD_VERSION: ${NETBIRD_VERSION} + NETBIRD_SHA256_AMD64: ${NETBIRD_SHA256_AMD64} + NETBIRD_SHA256_ARM64: ${NETBIRD_SHA256_ARM64} + cap_add: + - NET_ADMIN + environment: + - NB_SETUP_KEY + - NB_MANAGEMENT_URL + - PROXY_PEER_PORT=8080 + restart: unless-stopped diff --git a/cli/test/composefile/proxy_peer.bats b/cli/test/composefile/proxy_peer.bats new file mode 100644 index 00000000..8fd0909d --- /dev/null +++ b/cli/test/composefile/proxy_peer.bats @@ -0,0 +1,10 @@ +#!/usr/bin/env bats + +setup() { + load test_helper +} + +@test "compose-proxy-peer defines proxy-peer service with NET_ADMIN" { + yq -e '.services["proxy-peer"].cap_add[] | select(. == "NET_ADMIN")' \ + "$SCT_TEMPLATEDIR/devcontainer/sandcat/compose-proxy-peer.yml" +} From 3a79ef1215a59f7d4e55fac97b542e43b521f88b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Wed, 8 Jul 2026 21:35:51 +0000 Subject: [PATCH 066/138] feat(cli): add sandcat init --proxy-peer template wiring --- cli/lib/composefile.bash | 29 +++++++++--- cli/libexec/init/devcontainer | 17 +++++++ cli/libexec/init/init | 13 ++++++ cli/templates/settings-proxy-peer.json | 10 ++++ cli/test/init/init_proxy_peer.bats | 63 ++++++++++++++++++++++++++ 5 files changed, 126 insertions(+), 6 deletions(-) create mode 100644 cli/templates/settings-proxy-peer.json create mode 100644 cli/test/init/init_proxy_peer.bats diff --git a/cli/lib/composefile.bash b/cli/lib/composefile.bash index 78f6d369..31f97fad 100644 --- a/cli/lib/composefile.bash +++ b/cli/lib/composefile.bash @@ -591,13 +591,15 @@ apply_upstream_ca_bundles() { "$compose_file" } -# Injects NetBird version and per-arch checksum build args into wg-client's -# compose build section, sourced from netbird.env (sibling to compose-proxy.yml). +# Injects NetBird version and per-arch checksum build args into a service's +# compose build section, sourced from netbird.env (sibling to the compose file). # Args: -# $1 - Path to compose-proxy.yml +# $1 - Path to compose file +# $2 - Service name (default: wg-client) apply_netbird_build_args() { require yq local compose_file=$1 + local service_name=${2:-wg-client} local netbird_env netbird_env="$(dirname "$compose_file")/netbird.env" @@ -614,12 +616,27 @@ apply_netbird_build_args() { : "${NETBIRD_SHA256_ARM64:?NETBIRD_SHA256_ARM64 missing from $netbird_env}" yq -i " - .services.\"wg-client\".build.args.NETBIRD_VERSION = \"${NETBIRD_VERSION}\" | - .services.\"wg-client\".build.args.NETBIRD_SHA256_AMD64 = \"${NETBIRD_SHA256_AMD64}\" | - .services.\"wg-client\".build.args.NETBIRD_SHA256_ARM64 = \"${NETBIRD_SHA256_ARM64}\" + .services.\"${service_name}\".build.args.NETBIRD_VERSION = \"${NETBIRD_VERSION}\" | + .services.\"${service_name}\".build.args.NETBIRD_SHA256_AMD64 = \"${NETBIRD_SHA256_AMD64}\" | + .services.\"${service_name}\".build.args.NETBIRD_SHA256_ARM64 = \"${NETBIRD_SHA256_ARM64}\" " "$compose_file" } +# Copies proxy-peer compose stack and injects NetBird build args. +# Args: +# $1 - Path to the devcontainer directory (parent of sandcat/) +enable_proxy_peer() { + require yq + local compose_dir=$1 + local src="$SCT_TEMPLATEDIR/devcontainer/sandcat/compose-proxy-peer.yml" + local dst="$compose_dir/sandcat/compose-proxy-peer.yml" + cp "$src" "$dst" + cp "$SCT_TEMPLATEDIR/devcontainer/sandcat/Dockerfile.proxy-peer" "$compose_dir/sandcat/" + cp "$SCT_TEMPLATEDIR/devcontainer/sandcat/scripts/proxy-peer-init.sh" "$compose_dir/sandcat/scripts/" + cp "$SCT_TEMPLATEDIR/devcontainer/sandcat/scripts/proxy-peer-hello.py" "$compose_dir/sandcat/scripts/" + apply_netbird_build_args "$dst" "proxy-peer" +} + # Adds NB_SETUP_KEY to the wg-client service's environment in the deployed # compose-proxy.yml. wg-client-init.sh reads this at startup to enroll the # container as a NetBird peer and start the daemon on wt0. diff --git a/cli/libexec/init/devcontainer b/cli/libexec/init/devcontainer index 49e2d291..79a2d8ad 100755 --- a/cli/libexec/init/devcontainer +++ b/cli/libexec/init/devcontainer @@ -17,6 +17,7 @@ source "$SCT_LIBDIR/devcontainer.bash" # --ide - The IDE name (e.g., "vscode", "jetbrains", "none") (optional) # --netbird-management-url - NetBird management server URL for wg-client (optional) # --capability - Enable capability-runtime sidecar (requires --netbird) +# --proxy-peer - Enable proxy-peer gateway stack (requires --netbird) devcontainer() { local settings_file="" local project_path="" @@ -29,6 +30,7 @@ devcontainer() { local netbird="false" local netbird_management_url="" local capability="false" + local proxy_peer="false" while [[ $# -gt 0 ]] do @@ -85,6 +87,10 @@ devcontainer() { capability="true" shift 1 ;; + --proxy-peer) + proxy_peer="true" + shift 1 + ;; --netbird-management-url) netbird_management_url="$2" shift 2 @@ -104,6 +110,10 @@ devcontainer() { echo "--capability requires --netbird" | error return 1 fi + if [[ "$proxy_peer" == "true" && "$netbird" != "true" ]]; then + echo "--proxy-peer requires --netbird" | error + return 1 + fi local devcontainer_dir="$project_path/.devcontainer" local compose_file="$devcontainer_dir/compose-all.yml" @@ -167,6 +177,13 @@ RUN pip install --no-cache-dir /tmp/capability-runtime && rm -rf /tmp/capability enable_capability "$devcontainer_dir" fi + if [[ "$proxy_peer" == "true" ]]; then + enable_proxy_peer "$devcontainer_dir" + mkdir -p "$project_path/.sandcat" + cp "$SCT_TEMPLATEDIR/settings-proxy-peer.json" \ + "$project_path/.sandcat/settings.proxy-peer.example.json" + fi + customize_compose_file "$rel_settings_file" "$compose_file" "$agent" "$ide" "$project_name" "$stacks" set_project_name "$compose_file" "$project_name" diff --git a/cli/libexec/init/init b/cli/libexec/init/init index e4f64617..73a9ad25 100755 --- a/cli/libexec/init/init +++ b/cli/libexec/init/init @@ -164,6 +164,7 @@ add_secret_provider_tokens_to_user_settings() { # --1password - Deprecated; same as --secret-provider 1password # --features - Comma-separated optional non-provider features (tui) # --capability - Enable capability-runtime sidecar (requires --netbird) +# --proxy-peer - Enable proxy-peer gateway stack (requires --netbird) init() { require yq @@ -186,6 +187,7 @@ init() { local provisioned_netbird_server="false" local netbird_selfhosted_quickstart="false" local capability="false" + local proxy_peer="false" while [[ $# -gt 0 ]] do @@ -245,6 +247,10 @@ init() { capability="true" shift 1 ;; + --proxy-peer) + proxy_peer="true" + shift 1 + ;; --netbird-server) netbird_server="$2" netbird_server_provided=true @@ -273,6 +279,10 @@ init() { echo "--capability requires --netbird" | error return 1 fi + if [[ "$proxy_peer" == "true" && "$netbird" != "true" ]]; then + echo "--proxy-peer requires --netbird" | error + return 1 + fi if [[ "$netbird_server_provided" == "true" ]]; then case "$netbird_server" in cloud|new|quickstart|http://*|https://*) @@ -585,6 +595,9 @@ init() { if [[ "$capability" == "true" ]]; then devcontainer_args+=(--capability) fi + if [[ "$proxy_peer" == "true" ]]; then + devcontainer_args+=(--proxy-peer) + fi devcontainer "${devcontainer_args[@]}" local gitignore_status="skipped" diff --git a/cli/templates/settings-proxy-peer.json b/cli/templates/settings-proxy-peer.json new file mode 100644 index 00000000..11413511 --- /dev/null +++ b/cli/templates/settings-proxy-peer.json @@ -0,0 +1,10 @@ +{ + "network": [ + { + "action": "allow", + "host": "REPLACE_PROXY_PEER_MESH_IP", + "port": 8080, + "comment": "Layer 1: only proxy-peer gateway; replace IP after enrollment" + } + ] +} diff --git a/cli/test/init/init_proxy_peer.bats b/cli/test/init/init_proxy_peer.bats new file mode 100644 index 00000000..30ee7054 --- /dev/null +++ b/cli/test/init/init_proxy_peer.bats @@ -0,0 +1,63 @@ +#!/usr/bin/env bats +# shellcheck disable=SC2030,SC2031 + +setup() { + load test_helper + # shellcheck source=../../libexec/init/init + source "$SCT_LIBEXECDIR/init/init" + # shellcheck source=../../libexec/init/devcontainer + source "$SCT_LIBEXECDIR/init/devcontainer" + + PROJECT_DIR="$BATS_TEST_TMPDIR/project" + mkdir -p "$PROJECT_DIR" + + SCT_HOME_DIR="$BATS_TEST_TMPDIR/config/sandcat" + mkdir -p "$SCT_HOME_DIR" + sct_home() { echo "$SCT_HOME_DIR"; } + export -f sct_home + + export HOME="$BATS_TEST_TMPDIR/home" + mkdir -p "$HOME" +} + +teardown() { + unstub_all +} + +@test "init rejects --proxy-peer without --netbird" { + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" \ + --stacks "" --proxy web --features "" --secret-provider none --proxy-peer + assert_failure + assert_output --partial "--proxy-peer requires --netbird" +} + +@test "init --netbird --capability --proxy-peer copies compose-proxy-peer.yml" { + mkdir -p "$PROJECT_DIR/.sandcat" + touch "$PROJECT_DIR/.sandcat/settings.json" + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" \ + --stacks "" --proxy web --features "" --secret-provider none \ + --netbird --netbird-server cloud --capability --proxy-peer + assert_success + + [[ -f "$PROJECT_DIR/.devcontainer/sandcat/compose-proxy-peer.yml" ]] + [[ -f "$PROJECT_DIR/.devcontainer/sandcat/Dockerfile.proxy-peer" ]] + [[ -f "$PROJECT_DIR/.devcontainer/sandcat/scripts/proxy-peer-init.sh" ]] + [[ -f "$PROJECT_DIR/.devcontainer/sandcat/scripts/proxy-peer-hello.py" ]] +} + +@test "init --netbird --proxy-peer copies settings.proxy-peer.example.json" { + mkdir -p "$PROJECT_DIR/.sandcat" + touch "$PROJECT_DIR/.sandcat/settings.json" + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" \ + --stacks "" --proxy web --features "" --secret-provider none \ + --netbird --netbird-server cloud --proxy-peer + assert_success + + [[ -f "$PROJECT_DIR/.sandcat/settings.proxy-peer.example.json" ]] + run yq -r '.network[0].host' "$PROJECT_DIR/.sandcat/settings.proxy-peer.example.json" + assert_output "REPLACE_PROXY_PEER_MESH_IP" +} From 75624bdfea720465c20fd6f64725e618e495d96a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Wed, 8 Jul 2026 21:37:36 +0000 Subject: [PATCH 067/138] docs(cli): add Layer 1 mitmproxy profile for proxy-peer gateway --- cli/README.md | 80 +++++++++++++++++++++ cli/test/mitmproxy/settings_proxy_peer.bats | 19 +++++ 2 files changed, 99 insertions(+) create mode 100644 cli/test/mitmproxy/settings_proxy_peer.bats diff --git a/cli/README.md b/cli/README.md index fb16f0b2..056d454c 100644 --- a/cli/README.md +++ b/cli/README.md @@ -31,6 +31,10 @@ Options: Adds a `capability-runtime` compose service, mounts a shared Unix socket volume into the agent container, and installs `capability-mcp-bridge` for Cursor MCP. NetBird API credentials stay in the sidecar — they are not injected into the agent. +- `--proxy-peer` - Enable the proxy-peer gateway stack (requires `--netbird`). + Deploys a dedicated NetBird `proxy-peer` compose service and copies a Layer 1 + mitmproxy settings example (`.sandcat/settings.proxy-peer.example.json`) for + deny-by-default egress. Pair with `--capability` for Layer 2 lease/revoke control. - `--netbird-server` - NetBird management server mode (requires `--netbird`): `cloud` | `new` | `quickstart` | ``. `new` provisions a local localhost template; `quickstart` prints the official NetBird install command for @@ -62,6 +66,9 @@ sandcat init --agent claude --ide vscode --netbird --name myproject # With NetBird + capability sidecar (reachability == capability) sandcat init --agent cursor --ide vscode --netbird --capability --name myproject + +# With NetBird + proxy-peer gateway (two-layer control) +sandcat init --agent cursor --ide vscode --netbird --capability --proxy-peer --name myproject ``` #### Proton Pass setup (scoped Personal Access Token) @@ -571,6 +578,79 @@ Replace placeholders in `capability-catalog.json` before leasing `reach_api`: 3. Re-init or edit `.devcontainer/sandcat/capability-catalog.json` 4. Restart capability-runtime: `docker compose restart capability-runtime` +## Proxy-peer gateway (Phase 3e) + +When initialized with `--proxy-peer` (requires `--netbird`), sandcat deploys a +dedicated NetBird `proxy-peer` gateway peer and operationalizes a **two-layer** +control model for agent egress: + +``` +Layer 1 — static mitmproxy baseline (always on) + deny-by-default; only proxy-peer mesh IP:8080 allowed + +Layer 2 — dynamic NetBird lease/revoke (capability-runtime) + lease enables route to proxy-peer; revoke or quota exhaustion disables it +``` + +| Layer | Mechanism | What it controls | +|-------|-----------|------------------| +| **Layer 1** | mitmproxy `network` rules in `.sandcat/settings.json` | Static egress menu — agent can only reach the proxy-peer gateway IP on port 8080 | +| **Layer 2** | `capability-runtime` + NetBird route sync (Phase 3c) | Dynamic reachability — route to proxy-peer exists only while leased | + +Layer 1 blocks direct egress (e.g. `curl https://example.com`) even when Layer 2 +has no active lease. Layer 2 gates whether the agent can actually reach the +proxy-peer mesh endpoint at all. + +### Setup + +```bash +sandcat init --agent cursor --ide vscode --netbird --capability --proxy-peer --name myproject +docker compose -f .devcontainer/sandcat/compose-proxy-peer.yml up -d --build +sandcat compose up -d +``` + +Init copies the Layer 1 template from +[`templates/settings-proxy-peer.json`](templates/settings-proxy-peer.json) to +`.sandcat/settings.proxy-peer.example.json`. The template is deny-by-default with +a single allow rule for the proxy-peer gateway: + +```json +{ + "network": [ + { + "action": "allow", + "host": "REPLACE_PROXY_PEER_MESH_IP", + "port": 8080, + "comment": "Layer 1: only proxy-peer gateway; replace IP after enrollment" + } + ] +} +``` + +### Operator merge workflow + +After `proxy-peer` enrolls, apply Layer 1 to the live mitmproxy profile: + +1. `sandcat netbird status` — note the `proxy-peer` peer's mesh IP (e.g. `100.64.0.5`). +2. Merge the example into project settings — either: + - **Copy:** `cp .sandcat/settings.proxy-peer.example.json .sandcat/settings.json` + - **Merge:** add the `network` array (or individual allow rule) from the example into your existing `.sandcat/settings.json` +3. Replace `REPLACE_PROXY_PEER_MESH_IP` with the mesh IP from step 1. +4. `sandcat restart-proxy` — reload mitmproxy with the Layer 1 profile. + +Layer 2 uses the existing capability sidecar paths. Configure `cap-reach-proxy` +in `capability-catalog.json` with the proxy-peer `peer_id` and +`/32`, then lease/revoke as usual: + +```bash +sandcat capability lease --ref cap-reach-proxy --justification "need gateway access" +sandcat run curl -sf http://:8080/hello # succeeds while leased +sandcat capability revoke --ref cap-reach-proxy --reason done +``` + +Without an active lease, traffic to the proxy-peer mesh IP times out even though +Layer 1 allows the host:port in mitmproxy. + ## Directory Structure Each module is contained in its own directory under `cli/libexec/`. diff --git a/cli/test/mitmproxy/settings_proxy_peer.bats b/cli/test/mitmproxy/settings_proxy_peer.bats new file mode 100644 index 00000000..caf60c29 --- /dev/null +++ b/cli/test/mitmproxy/settings_proxy_peer.bats @@ -0,0 +1,19 @@ +#!/usr/bin/env bats + +setup() { + load "$BATS_TEST_DIRNAME/../composefile/test_helper" +} + +@test "settings-proxy-peer.json allows proxy-peer mesh IP on port 8080" { + local template="$SCT_TEMPLATEDIR/settings-proxy-peer.json" + + [[ -f "$template" ]] + + run yq -r '.network[] | select(.action == "allow") | .port' "$template" + assert_success + assert_output "8080" + + run yq -r '.network[] | select(.action == "allow") | .host' "$template" + assert_success + assert_output "REPLACE_PROXY_PEER_MESH_IP" +} From 123a6d95fbd84cde50543b960ef75465634710b4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Wed, 8 Jul 2026 21:39:02 +0000 Subject: [PATCH 068/138] feat(capability-runtime): catalog lease_policy for network capabilities --- .../src/capability_runtime/daemon.py | 14 ++++++++ .../src/capability_runtime/policy.py | 4 +++ .../src/capability_runtime/runtime.py | 33 ++++++++++++------- .../tests/test_catalog_lease_policy.py | 27 +++++++++++++++ .../sandcat/capability-catalog.json | 10 ++++++ 5 files changed, 77 insertions(+), 11 deletions(-) create mode 100644 capability-runtime/tests/test_catalog_lease_policy.py diff --git a/capability-runtime/src/capability_runtime/daemon.py b/capability-runtime/src/capability_runtime/daemon.py index 5649af1d..e13ddcd4 100644 --- a/capability-runtime/src/capability_runtime/daemon.py +++ b/capability-runtime/src/capability_runtime/daemon.py @@ -8,9 +8,11 @@ import threading import time from dataclasses import dataclass +from datetime import timedelta from pathlib import Path from capability_runtime.catalog import LifecycleState +from capability_runtime.policy import LeasePolicy from capability_runtime.netbird_client import MockNetBirdClient, NetBirdClient, RestNetBirdClient from capability_runtime.network import NetworkBinding, sync_mode_from_catalog from capability_runtime.rpc.dispatcher import RpcDispatcher @@ -114,6 +116,18 @@ def load_catalog_into_runtime(runtime: CapabilityRuntime, catalog_path: Path) -> if runtime.catalog.get_by_name(name) is None: runtime.catalog.register(name, ref, LifecycleState.DECLARED) + if "lease_policy" in entry: + lp = entry["lease_policy"] + runtime.register_lease_policy( + name, + LeasePolicy( + quota=lp["quota"], + ttl=timedelta(minutes=lp["ttl_minutes"]), + token_budget=lp["token_budget"], + risk_envelope=lp.get("risk_envelope", "medium"), + ), + ) + class _WatcherPollThread: def __init__(self, watcher: RouteDisappearanceWatcher, interval: float) -> None: diff --git a/capability-runtime/src/capability_runtime/policy.py b/capability-runtime/src/capability_runtime/policy.py index ae70620f..6bf1a144 100644 --- a/capability-runtime/src/capability_runtime/policy.py +++ b/capability-runtime/src/capability_runtime/policy.py @@ -34,6 +34,10 @@ class LeasePolicy: } +def register_lease_policy(capability_name: str, policy: LeasePolicy) -> None: + _POC_POLICIES[capability_name] = policy + + def lease_policy_for(capability_name: str | None) -> LeasePolicy: if capability_name is None: raise LeasePolicyNotFound("missing capability name") diff --git a/capability-runtime/src/capability_runtime/runtime.py b/capability-runtime/src/capability_runtime/runtime.py index a66320c8..eea4e883 100644 --- a/capability-runtime/src/capability_runtime/runtime.py +++ b/capability-runtime/src/capability_runtime/runtime.py @@ -25,6 +25,7 @@ from capability_runtime.netbird_client import NetBirdClient from capability_runtime.netbird_sync import grant_network_binding from capability_runtime.observability import ObservabilityCollector +from capability_runtime.policy import LeasePolicy, LeasePolicyNotFound, lease_policy_for, register_lease_policy from capability_runtime.revoke import RevocationManager from capability_runtime.types import ( AgentIdentity, @@ -120,6 +121,10 @@ def register_network_capability( self.catalog.register(name, ref, initial_state) self.catalog.set_network_binding(ref, binding) + def register_lease_policy(self, capability_name: str, policy: LeasePolicy) -> None: + """Register a lease policy for a capability by name.""" + register_lease_policy(capability_name, policy) + def check_current_capabilities( self, agent_id: AgentIdentity, context: dict ) -> CapabilityBundle: @@ -271,17 +276,23 @@ def request_capability_lease( now = datetime.now(timezone.utc) capability_name = self.catalog.get_name(capability_ref) - if capability_name == "write_note": - quota = 3 - ttl = timedelta(minutes=5) - token_budget = 10_000 - risk_envelope = "medium" - else: - # PoC 1 params for create_pr (default) - quota = 1 - ttl = timedelta(minutes=10) - token_budget = 25_000 - risk_envelope = "high" + try: + policy = lease_policy_for(capability_name) + quota = policy.quota + ttl = policy.ttl + token_budget = policy.token_budget + risk_envelope = policy.risk_envelope + except LeasePolicyNotFound: + if capability_name == "write_note": + quota = 3 + ttl = timedelta(minutes=5) + token_budget = 10_000 + risk_envelope = "medium" + else: + quota = 1 + ttl = timedelta(minutes=10) + token_budget = 25_000 + risk_envelope = "high" decision = self.lease_manager.grant( agent_id=agent_id, diff --git a/capability-runtime/tests/test_catalog_lease_policy.py b/capability-runtime/tests/test_catalog_lease_policy.py new file mode 100644 index 00000000..f396b997 --- /dev/null +++ b/capability-runtime/tests/test_catalog_lease_policy.py @@ -0,0 +1,27 @@ +from pathlib import Path +import json +from capability_runtime.daemon import load_catalog_into_runtime +from capability_runtime.netbird_client import MockNetBirdClient +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity, CapabilityRef + + +def test_network_lease_uses_catalog_quota(tmp_path): + catalog = tmp_path / "catalog.json" + catalog.write_text(json.dumps({ + "capabilities": [{ + "name": "reach_proxy", + "ref": "cap-reach-proxy", + "type": "network", + "peer_id": "peer-pp", + "network": "100.64.0.5/32", + "sync_mode": "route_enable", + "lease_policy": {"quota": 5, "ttl_minutes": 15, "token_budget": 10000}, + }] + })) + client = MockNetBirdClient(peers=[{"id": "peer-pp"}], routes=[]) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace", 1, netbird_client=client) + load_catalog_into_runtime(runtime, catalog) + agent = AgentIdentity("agent-1") + decision = runtime.request_capability_lease(agent, agent, CapabilityRef("cap-reach-proxy"), "gate") + assert decision.quota == 5 diff --git a/cli/templates/devcontainer/sandcat/capability-catalog.json b/cli/templates/devcontainer/sandcat/capability-catalog.json index 2708f6b8..8ed9baf5 100644 --- a/cli/templates/devcontainer/sandcat/capability-catalog.json +++ b/cli/templates/devcontainer/sandcat/capability-catalog.json @@ -12,6 +12,16 @@ "peer_id": "REPLACE_WITH_NETBIRD_PEER_ID", "network": "10.8.0.0/24", "sync_mode": "route_enable" + }, + { + "name": "reach_proxy", + "ref": "cap-reach-proxy", + "type": "network", + "peer_id": "peer-placeholder", + "network": "100.64.0.0/32", + "sync_mode": "route_enable", + "proxy": {"port": 8080, "path_prefix": "/hello"}, + "lease_policy": {"quota": 5, "ttl_minutes": 15, "token_budget": 10000} } ] } From dc3da7dafd5487cbf5eab9a2f18671497c4704a9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Wed, 8 Jul 2026 21:41:02 +0000 Subject: [PATCH 069/138] feat(capability-runtime): admin capability.l7.record for flow quota --- .../src/capability_runtime/l7_record.py | 88 +++++++++++++++++ .../src/capability_runtime/rpc/dispatcher.py | 20 ++++ capability-runtime/tests/test_l7_record.py | 96 +++++++++++++++++++ 3 files changed, 204 insertions(+) create mode 100644 capability-runtime/src/capability_runtime/l7_record.py create mode 100644 capability-runtime/tests/test_l7_record.py diff --git a/capability-runtime/src/capability_runtime/l7_record.py b/capability-runtime/src/capability_runtime/l7_record.py new file mode 100644 index 00000000..be256179 --- /dev/null +++ b/capability-runtime/src/capability_runtime/l7_record.py @@ -0,0 +1,88 @@ +"""Map L7 flow observations to network lease quota decrements.""" + +from __future__ import annotations + +import ipaddress +from datetime import datetime, timezone +from typing import Any + +from capability_runtime.catalog import LifecycleState +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity, LeaseId + + +def _host_in_binding_network(host: str, network: str) -> bool: + try: + addr = ipaddress.ip_address(host) + net = ipaddress.ip_network(network, strict=False) + return addr in net + except ValueError: + return False + + +def _find_active_network_lease_for_host( + runtime: CapabilityRuntime, + agent_id: AgentIdentity, + host: str, + now: datetime, +) -> LeaseId | None: + for lease_id, lease in runtime.lease_manager._leases.items(): + if lease.agent_id != agent_id: + continue + if runtime.lease_manager.is_expired(lease_id, now): + continue + if runtime.revocation_manager.is_lease_revoked(lease_id): + continue + if runtime.lease_manager.is_exhausted(lease_id): + continue + + binding = runtime.catalog.get_network_binding(lease.capability_ref) + if binding is None: + continue + if not _host_in_binding_network(host, binding.network): + continue + + state = runtime.catalog.get_state(lease.capability_ref) + if state != LifecycleState.LEASED: + continue + + return lease_id + + return None + + +def record_l7_flow( + runtime: CapabilityRuntime, + agent: AgentIdentity, + *, + host: str, + method: str, + status: int, + trace_id: str | None = None, +) -> bool: + """Record an L7 flow against a matching network lease quota. + + Returns True when a matching active lease was found and quota decremented. + """ + now = datetime.now(timezone.utc) + lease_id = _find_active_network_lease_for_host(runtime, agent, host, now) + + event: dict[str, Any] = { + "event": "l7_flow", + "agent_id": agent.value, + "host": host, + "method": method, + "status": status, + } + if trace_id is not None: + event["trace_id"] = trace_id + if lease_id is not None: + event["lease_id"] = lease_id.value + + runtime.observability.emit_capability_event(event) + + if lease_id is None: + return False + + runtime.record_action(agent, agent, lease_id, now) + return True diff --git a/capability-runtime/src/capability_runtime/rpc/dispatcher.py b/capability-runtime/src/capability_runtime/rpc/dispatcher.py index be192808..a9f4112a 100644 --- a/capability-runtime/src/capability_runtime/rpc/dispatcher.py +++ b/capability-runtime/src/capability_runtime/rpc/dispatcher.py @@ -7,6 +7,7 @@ from capability_runtime.discover import DiscoveryIntent from capability_runtime.errors import CapabilityRuntimeError +from capability_runtime.l7_record import record_l7_flow from capability_runtime.rpc import errors as rpc_errors from capability_runtime.runtime import CapabilityRuntime from capability_runtime.route_watcher import RouteDisappearanceWatcher @@ -30,6 +31,7 @@ { "capability.revoke", "capability.watch.poll", + "capability.l7.record", } ) @@ -101,6 +103,8 @@ def _dispatch(self, method: str, params: dict) -> dict: return self._handle_revoke(params) if method == "capability.watch.poll": return self._handle_watch_poll(params) + if method == "capability.l7.record": + return self._handle_l7_record(params) raise RuntimeError(f"unhandled allowed method: {method}") def _resolve_agent_id(self, params: dict) -> AgentIdentity: @@ -147,6 +151,22 @@ def _handle_watch_poll(self, params: dict) -> dict: self._watcher.poll_once() return {"polled": True} + def _handle_l7_record(self, params: dict) -> dict: + agent_id = self._resolve_agent_id(params) + host = params["host"] + method = params["method"] + status = params["status"] + trace_id = params.get("trace_id") + recorded = record_l7_flow( + self._runtime, + agent_id, + host=host, + method=method, + status=status, + trace_id=trace_id, + ) + return {"recorded": recorded} + def _parse_revoke_target(runtime: CapabilityRuntime, target: str) -> LeaseId | CapabilityRef: lease_id = LeaseId(target) diff --git a/capability-runtime/tests/test_l7_record.py b/capability-runtime/tests/test_l7_record.py new file mode 100644 index 00000000..4ab918c9 --- /dev/null +++ b/capability-runtime/tests/test_l7_record.py @@ -0,0 +1,96 @@ +"""Tests for L7 flow recording and admin capability.l7.record RPC.""" + +from __future__ import annotations + +from datetime import timedelta + +from capability_runtime.catalog import LifecycleState +from capability_runtime.l7_record import record_l7_flow +from capability_runtime.netbird_client import MockNetBirdClient +from capability_runtime.network import NetworkBinding, SyncMode +from capability_runtime.policy import LeasePolicy, register_lease_policy +from capability_runtime.rpc.dispatcher import RpcDispatcher +from capability_runtime.runtime import CapabilityRuntime +from capability_runtime.types import AgentIdentity, CapabilityRef + + +def test_l7_record_decrements_network_lease_quota(tmp_path): + client = MockNetBirdClient(peers=[{"id": "peer-pp"}], routes=[]) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace", 1, netbird_client=client) + agent = AgentIdentity("devcontainer-agent") + ref = CapabilityRef("cap-reach-proxy") + binding = NetworkBinding(ref, "peer-pp", "100.64.0.5/32", None, SyncMode.ROUTE_ENABLE) + runtime.register_network_capability("reach_proxy", ref, binding, LifecycleState.VISIBLE) + register_lease_policy( + "reach_proxy", + LeasePolicy( + quota=2, + ttl=timedelta(minutes=15), + token_budget=10000, + risk_envelope="medium", + ), + ) + decision = runtime.request_capability_lease(agent, agent, ref, "test") + record_l7_flow(runtime, agent, host="100.64.0.5", method="GET", status=200) + record_l7_flow(runtime, agent, host="100.64.0.5", method="GET", status=200) + bundle = runtime.check_current_capabilities(agent, {}) + assert "reach_proxy" not in [n.name for n in bundle.networks] + assert decision.lease_id is not None + + +def test_agent_surface_rejects_l7_record(tmp_path): + runtime = CapabilityRuntime(tmp_path / "trace.jsonl", "trace-sec-l7", 500) + dispatcher = RpcDispatcher( + runtime, surface="agent", bound_agent_id="devcontainer-agent" + ) + response = dispatcher.handle( + { + "jsonrpc": "2.0", + "id": 1, + "method": "capability.l7.record", + "params": { + "host": "100.64.0.5", + "method": "GET", + "status": 200, + }, + } + ) + assert "error" in response + assert response["error"]["code"] == -32601 + + +def test_admin_surface_records_l7_flow(tmp_path): + client = MockNetBirdClient(peers=[{"id": "peer-pp"}], routes=[]) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace", 1, netbird_client=client) + agent = AgentIdentity("devcontainer-agent") + ref = CapabilityRef("cap-reach-proxy") + binding = NetworkBinding(ref, "peer-pp", "100.64.0.5/32", None, SyncMode.ROUTE_ENABLE) + runtime.register_network_capability("reach_proxy", ref, binding, LifecycleState.VISIBLE) + register_lease_policy( + "reach_proxy", + LeasePolicy( + quota=2, + ttl=timedelta(minutes=15), + token_budget=10000, + risk_envelope="medium", + ), + ) + runtime.request_capability_lease(agent, agent, ref, "test") + + dispatcher = RpcDispatcher(runtime, surface="admin", bound_agent_id="operator") + response = dispatcher.handle( + { + "jsonrpc": "2.0", + "id": 1, + "method": "capability.l7.record", + "params": { + "agent_id": "devcontainer-agent", + "host": "100.64.0.5", + "method": "GET", + "status": 200, + "trace_id": "trace-abc", + }, + } + ) + assert "result" in response + assert response["result"]["recorded"] is True From da6e98ebd6677253853f1dd54e463a3b182c9798 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Wed, 8 Jul 2026 21:45:25 +0000 Subject: [PATCH 070/138] feat(mitmproxy): post-hoc l7 flow record to capability sidecar --- cli/lib/composefile.bash | 30 +++++++++ .../sandcat/scripts/l7_record_client.py | 28 ++++++++ .../sandcat/scripts/mitmproxy_addon_common.py | 44 +++++++++++++ cli/test/composefile/capability.bats | 18 ++++++ cli/test/mitmproxy/l7_record.bats | 64 +++++++++++++++++++ 5 files changed, 184 insertions(+) create mode 100644 cli/templates/devcontainer/sandcat/scripts/l7_record_client.py create mode 100644 cli/test/mitmproxy/l7_record.bats diff --git a/cli/lib/composefile.bash b/cli/lib/composefile.bash index 31f97fad..7a19e1d3 100644 --- a/cli/lib/composefile.bash +++ b/cli/lib/composefile.bash @@ -744,6 +744,36 @@ enable_capability() { yq -i '.services.agent.depends_on.capability-runtime.condition = "service_started"' "$compose_file" fi + local proxy_compose="$compose_dir/sandcat/compose-proxy.yml" + if [[ -f "$proxy_compose" ]]; then + local has_cap_vol + has_cap_vol=$(yq '[.volumes | keys[]? | select(. == "capability-socket")] | length' "$proxy_compose") + if [[ "$has_cap_vol" -eq 0 ]]; then + yq -i '.volumes.capability-socket = {}' "$proxy_compose" + fi + + local has_mitm_cap_vol has_l7_client has_l7_record has_mitm_agent_id + has_mitm_cap_vol=$(yq '[.services.mitmproxy.volumes[]? | select(. == "capability-socket:/run/sandcat-capability:ro")] | length' "$proxy_compose") + if [[ "$has_mitm_cap_vol" -eq 0 ]]; then + yq -i '.services.mitmproxy.volumes += ["capability-socket:/run/sandcat-capability:ro"]' "$proxy_compose" + fi + + has_l7_client=$(yq '[.services.mitmproxy.volumes[]? | select(. == "./scripts/l7_record_client.py:/scripts/l7_record_client.py:ro")] | length' "$proxy_compose") + if [[ "$has_l7_client" -eq 0 ]]; then + yq -i '.services.mitmproxy.volumes += ["./scripts/l7_record_client.py:/scripts/l7_record_client.py:ro"]' "$proxy_compose" + fi + + has_l7_record=$(yq '[.services.mitmproxy.environment[]? | select(. == "CAPABILITY_L7_RECORD")] | length' "$proxy_compose") + if [[ "$has_l7_record" -eq 0 ]]; then + yq -i '.services.mitmproxy.environment = ((.services.mitmproxy.environment // []) + ["CAPABILITY_L7_RECORD"])' "$proxy_compose" + fi + + has_mitm_agent_id=$(yq '[.services.mitmproxy.environment[]? | select(. == "SANDCAT_AGENT_ID=devcontainer-agent")] | length' "$proxy_compose") + if [[ "$has_mitm_agent_id" -eq 0 ]]; then + yq -i '.services.mitmproxy.environment = ((.services.mitmproxy.environment // []) + ["SANDCAT_AGENT_ID=devcontainer-agent"])' "$proxy_compose" + fi + fi + _enable_capability_mcp_config "$(dirname "$compose_dir")" "$compose_dir/devcontainer.json" } diff --git a/cli/templates/devcontainer/sandcat/scripts/l7_record_client.py b/cli/templates/devcontainer/sandcat/scripts/l7_record_client.py new file mode 100644 index 00000000..76e98520 --- /dev/null +++ b/cli/templates/devcontainer/sandcat/scripts/l7_record_client.py @@ -0,0 +1,28 @@ +"""Best-effort post-hoc flow record to capability-runtime admin socket.""" +import json +import os +import socket + +ADMIN_SOCKET = os.environ.get( + "CAPABILITY_ADMIN_SOCKET", "/run/sandcat-capability/admin.sock" +) + +def record_flow(*, agent_id: str, host: str, method: str, status: int) -> None: + payload = { + "jsonrpc": "2.0", + "id": 1, + "method": "capability.l7.record", + "params": { + "agent_id": agent_id, + "host": host, + "method": method, + "status": status, + }, + } + try: + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock: + sock.settimeout(0.2) + sock.connect(ADMIN_SOCKET) + sock.sendall((json.dumps(payload) + "\n").encode()) + except OSError: + return # best-effort; never block mitmproxy diff --git a/cli/templates/devcontainer/sandcat/scripts/mitmproxy_addon_common.py b/cli/templates/devcontainer/sandcat/scripts/mitmproxy_addon_common.py index b5a81ab9..baf394da 100644 --- a/cli/templates/devcontainer/sandcat/scripts/mitmproxy_addon_common.py +++ b/cli/templates/devcontainer/sandcat/scripts/mitmproxy_addon_common.py @@ -811,6 +811,30 @@ def _is_request_allowed(self, method: str | None, host: str) -> bool: rule = self._find_matching_rule(method, host) return rule is not None and rule.get("action") == "allow" + _MESH_CGNAT = ipaddress.ip_network("100.64.0.0/10") + + def _host_matches_network_allow_rule(self, host: str) -> bool: + host = host.lower().rstrip(".") + for rule in self.network_rules: + if rule.get("action") != "allow": + continue + if fnmatch(host, rule["host"].lower()): + return True + return False + + @classmethod + def _host_in_mesh_cgnat(cls, host: str) -> bool: + try: + return ipaddress.ip_address(host) in cls._MESH_CGNAT + except ValueError: + return False + + def _should_record_l7_flow(self, host: str) -> bool: + return ( + self._host_matches_network_allow_rule(host) + or self._host_in_mesh_cgnat(host) + ) + # ----------------------------------------------------------- env writer @staticmethod @@ -1079,6 +1103,26 @@ def responseheaders(self, flow: http.HTTPFlow): if self._is_streaming_request(flow): flow.response.stream = True + def response(self, flow: http.HTTPFlow): + if os.environ.get("CAPABILITY_L7_RECORD") != "1": + return + if not flow.response or flow.response.status_code is None: + return + + host = flow.request.pretty_host + if not self._should_record_l7_flow(host): + return + + from l7_record_client import record_flow + + agent_id = os.environ.get("SANDCAT_AGENT_ID", "devcontainer-agent") + record_flow( + agent_id=agent_id, + host=host, + method=flow.request.method, + status=flow.response.status_code, + ) + def dns_request(self, flow: dns.DNSFlow): question = flow.request.question if question is None: diff --git a/cli/test/composefile/capability.bats b/cli/test/composefile/capability.bats index b8beef61..ddc1c59f 100644 --- a/cli/test/composefile/capability.bats +++ b/cli/test/composefile/capability.bats @@ -8,6 +8,7 @@ setup() { mkdir -p "$COMPOSE_DIR/sandcat" cp "$SCT_TEMPLATEDIR/devcontainer/compose-all.yml" "$COMPOSE_DIR/compose-all.yml" cp "$SCT_TEMPLATEDIR/devcontainer/sandcat/compose-capability.yml" "$COMPOSE_DIR/sandcat/compose-capability.yml" + cp "$SCT_TEMPLATEDIR/devcontainer/sandcat/compose-proxy.yml" "$COMPOSE_DIR/sandcat/compose-proxy.yml" } teardown() { @@ -46,3 +47,20 @@ teardown() { run yq '[.services.agent.environment[] | select(. == "SANDCAT_AGENT_ID=devcontainer-agent")] | length' "$COMPOSE_DIR/compose-all.yml" assert_output "1" } + +@test "enable_capability mounts capability admin socket into mitmproxy" { + enable_capability "$COMPOSE_DIR" + yq -e '.services.mitmproxy.volumes[] | select(. == "capability-socket:/run/sandcat-capability:ro")' \ + "$COMPOSE_DIR/sandcat/compose-proxy.yml" + yq -e '.services.mitmproxy.volumes[] | select(. == "./scripts/l7_record_client.py:/scripts/l7_record_client.py:ro")' \ + "$COMPOSE_DIR/sandcat/compose-proxy.yml" + yq -e '.volumes.capability-socket' "$COMPOSE_DIR/sandcat/compose-proxy.yml" +} + +@test "enable_capability passes CAPABILITY_L7_RECORD through mitmproxy" { + enable_capability "$COMPOSE_DIR" + yq -e '.services.mitmproxy.environment[] | select(. == "CAPABILITY_L7_RECORD")' \ + "$COMPOSE_DIR/sandcat/compose-proxy.yml" + yq -e '.services.mitmproxy.environment[] | select(. == "SANDCAT_AGENT_ID=devcontainer-agent")' \ + "$COMPOSE_DIR/sandcat/compose-proxy.yml" +} diff --git a/cli/test/mitmproxy/l7_record.bats b/cli/test/mitmproxy/l7_record.bats new file mode 100644 index 00000000..64f82fb8 --- /dev/null +++ b/cli/test/mitmproxy/l7_record.bats @@ -0,0 +1,64 @@ +#!/usr/bin/env bats + +setup() { + load "$BATS_TEST_DIRNAME/../composefile/test_helper" +} + +@test "l7_record_client builds valid JSON-RPC payload" { + local script="$SCT_TEMPLATEDIR/devcontainer/sandcat/scripts/l7_record_client.py" + + [[ -f "$script" ]] + + run python3 - "$script" <<'PY' +import importlib.util +import json +import socket +import sys +from unittest.mock import patch + +script = sys.argv[1] +spec = importlib.util.spec_from_file_location("l7_record_client", script) +mod = importlib.util.module_from_spec(spec) +spec.loader.exec_module(mod) + +sent = [] + +class FakeSock: + def __enter__(self): + return self + + def __exit__(self, *args): + return False + + def settimeout(self, _timeout): + pass + + def connect(self, _addr): + pass + + def sendall(self, data): + sent.append(data) + +with patch.object(socket, "socket", return_value=FakeSock()): + mod.record_flow( + agent_id="test-agent", + host="100.64.0.5", + method="GET", + status=200, + ) + +payload = json.loads(sent[0].decode().strip()) +assert payload["jsonrpc"] == "2.0" +assert payload["id"] == 1 +assert payload["method"] == "capability.l7.record" +assert payload["params"] == { + "agent_id": "test-agent", + "host": "100.64.0.5", + "method": "GET", + "status": 200, +} +print("ok") +PY + assert_success + assert_output "ok" +} From 1ce0eda9c32e939d53358ea9ea519fdc299aabc5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Wed, 8 Jul 2026 21:47:26 +0000 Subject: [PATCH 071/138] docs: add Phase 3e proxy-peer engineering gate --- capability-runtime/README.md | 13 ++++++++++-- .../scripts/phase3e_engineering_gate.sh | 21 +++++++++++++++++++ cli/README.md | 10 +++++++++ 3 files changed, 42 insertions(+), 2 deletions(-) create mode 100755 capability-runtime/scripts/phase3e_engineering_gate.sh diff --git a/capability-runtime/README.md b/capability-runtime/README.md index bb9e8b9d..c1201118 100644 --- a/capability-runtime/README.md +++ b/capability-runtime/README.md @@ -45,6 +45,13 @@ Phase 3 realizes this thesis by binding each network capability to a NetBird pee - Grant failure rolls back the lease (fail closed); observability emits `physical_sync: enabled` on successful network lease - mitmproxy remains egress inspection only — no per-request L7 allowlist from bundle +**In scope (Phase 3e — proxy-peer gateway):** + +- Catalog `lease_policy` on network capabilities overrides PoC hardcoded quotas in `request_capability_lease` +- Admin-only `capability.l7.record` RPC decrements network lease quota from post-hoc mitmproxy flow records +- `l7_record.py` matches flow host to active network binding CIDR and calls `record_action` +- mitmproxy addon emits `l7_flow` events when `CAPABILITY_L7_RECORD=1` (best-effort Unix socket to admin surface) + **Out of scope / known limitations:** - Token budget enforcement (`token_budget` is stored but not decremented) @@ -178,9 +185,10 @@ Automated unit tests and PoC demo, plus printed manual live-smoke steps (NetBird ```bash bash scripts/phase3c_engineering_gate.sh +bash scripts/phase3e_engineering_gate.sh ``` -Manual steps map to the [Phase 3c spec §11 success criteria](../../docs/superpowers/specs/2026-06-30-capability-netbird-policy-sync-phase3c-design.md#11-success-criteria-engineering-gate-before-phase-4). For catalog ID setup before live smoke, see [Catalog IDs for live smoke](../../cli/README.md#catalog-ids-for-live-smoke) in the CLI README. +Manual steps map to the [Phase 3c spec §11 success criteria](../../docs/superpowers/specs/2026-06-30-capability-netbird-policy-sync-phase3c-design.md#11-success-criteria-engineering-gate-before-phase-4) and [Phase 3e spec §9](../../docs/superpowers/specs/2026-07-08-capability-proxy-peer-gateway-phase3e-design.md). For catalog ID setup before live smoke, see [Catalog IDs for live smoke](../../cli/README.md#catalog-ids-for-live-smoke) in the CLI README. ## Layout @@ -194,7 +202,8 @@ Manual steps map to the [Phase 3c spec §11 success criteria](../../docs/superpo | `netbird_sync.py` | Grant/revoke orchestration helpers for network bindings | | `route_watcher.py` | `RouteDisappearanceWatcher` — physical disappearance → logical revoke | | `lease.py` / `revoke.py` | Grant, quota, revocation | -| `policy.py` | PoC lease parameters (not in core runtime) | +| `policy.py` | Lease parameters (PoC defaults + catalog `lease_policy` registration) | +| `l7_record.py` | Post-hoc L7 flow → network lease quota decrement | | `observability.py` | JSONL trace + replay | | `agent_loop.py` | Check-then-act harness | | `mcp_adapter.py` | Transport-agnostic MCP tool wrapper | diff --git a/capability-runtime/scripts/phase3e_engineering_gate.sh b/capability-runtime/scripts/phase3e_engineering_gate.sh new file mode 100755 index 00000000..4de55ee0 --- /dev/null +++ b/capability-runtime/scripts/phase3e_engineering_gate.sh @@ -0,0 +1,21 @@ +#!/usr/bin/env bash +# capability-runtime/scripts/phase3e_engineering_gate.sh +set -euo pipefail +cd "$(dirname "$0")/.." +pytest -q +PYTHONPATH=src python poc/network_route_demo.py >/dev/null +echo "== Manual proxy-peer gate ==" +cat <<'EOF' +1. sandcat init --netbird --capability --proxy-peer --name +2. docker compose -f .devcontainer/sandcat/compose-proxy-peer.yml up -d --build +3. sandcat netbird status # note proxy-peer peer_id + mesh IP +4. Set capability-catalog.json: cap-reach-proxy peer_id + network /32 +5. Merge settings-proxy-peer.json into .sandcat/settings.json (replace mesh IP) +6. sandcat restart-proxy && docker compose build capability-runtime && sandcat compose up -d +7. sandcat run curl -sf --connect-timeout 3 http://:8080/hello # FAIL without lease +8. sandcat capability lease --ref cap-reach-proxy --justification "gate" +9. sandcat run curl -sf http://:8080/hello # PASS +10. sandcat capability revoke --ref cap-reach-proxy --reason "gate" +11. sandcat run curl -sf --connect-timeout 3 http://:8080/hello # FAIL +12. Lease with quota=2; three curls → auto-revoke (with CAPABILITY_L7_RECORD=1) +EOF diff --git a/cli/README.md b/cli/README.md index 056d454c..ed3b2e2d 100644 --- a/cli/README.md +++ b/cli/README.md @@ -651,6 +651,16 @@ sandcat capability revoke --ref cap-reach-proxy --reason done Without an active lease, traffic to the proxy-peer mesh IP times out even though Layer 1 allows the host:port in mitmproxy. +### Usage-metered quota (L7 record) + +When `--capability` is enabled, mitmproxy can decrement network lease quota from +post-hoc flow records. Set `CAPABILITY_L7_RECORD=1` in the mitmproxy service +environment (compose passes the variable through when capability is enabled; set +the value in your shell or `.env` before `sandcat compose up`). Each successful +HTTP response to a mesh or Layer-1-allowed host emits `capability.l7.record` on +the admin socket; quota exhaustion triggers the same auto-revoke path as tool +quota. + ## Directory Structure Each module is contained in its own directory under `cli/libexec/`. From 7bd6ca5f1c70e09aa1a59909cc81e540dc2afbe4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Thu, 9 Jul 2026 06:38:13 +0000 Subject: [PATCH 072/138] fix(phase3e): address code review findings for L7 quota loop - Open admin.sock to 0o666 so mitmproxy can connect for L7 records - Mount capability-socket read-write on mitmproxy (not :ro) - Record L7 flows only for Layer-1 allow-rule hosts and 2xx responses - Use LeaseManager.iter_active_leases_for_agent instead of private _leases - Require --capability when --proxy-peer is set --- .../src/capability_runtime/daemon.py | 3 +- .../src/capability_runtime/l7_record.py | 28 ++++++++------ .../src/capability_runtime/lease.py | 14 +++++++ capability-runtime/tests/test_l7_record.py | 27 +++++++++++++ capability-runtime/tests/test_rpc_unix.py | 38 +++++++++++++++++++ cli/.version | 1 + cli/README.md | 4 +- cli/lib/composefile.bash | 4 +- cli/libexec/init/devcontainer | 6 ++- cli/libexec/init/init | 6 ++- .../sandcat/scripts/mitmproxy_addon_common.py | 22 +++-------- cli/test/composefile/capability.bats | 2 +- cli/test/init/init_proxy_peer.bats | 10 ++++- 13 files changed, 127 insertions(+), 38 deletions(-) create mode 100644 cli/.version diff --git a/capability-runtime/src/capability_runtime/daemon.py b/capability-runtime/src/capability_runtime/daemon.py index e13ddcd4..d0f127a7 100644 --- a/capability-runtime/src/capability_runtime/daemon.py +++ b/capability-runtime/src/capability_runtime/daemon.py @@ -187,7 +187,8 @@ def __init__(self, config: DaemonConfig) -> None: bound_agent_id="operator", watcher=self._watcher, ), - socket_mode=0o600, + # mitmproxy runs as non-root and must connect for L7 quota (phase 3e). + socket_mode=0o666, ) self._running = False diff --git a/capability-runtime/src/capability_runtime/l7_record.py b/capability-runtime/src/capability_runtime/l7_record.py index be256179..0f125030 100644 --- a/capability-runtime/src/capability_runtime/l7_record.py +++ b/capability-runtime/src/capability_runtime/l7_record.py @@ -20,22 +20,20 @@ def _host_in_binding_network(host: str, network: str) -> bool: return False +def _is_billable_l7_status(status: int) -> bool: + return 200 <= status < 300 + + def _find_active_network_lease_for_host( runtime: CapabilityRuntime, agent_id: AgentIdentity, host: str, now: datetime, ) -> LeaseId | None: - for lease_id, lease in runtime.lease_manager._leases.items(): - if lease.agent_id != agent_id: - continue - if runtime.lease_manager.is_expired(lease_id, now): - continue - if runtime.revocation_manager.is_lease_revoked(lease_id): - continue - if runtime.lease_manager.is_exhausted(lease_id): - continue - + is_revoked = runtime.revocation_manager.is_lease_revoked + for lease_id, lease in runtime.lease_manager.iter_active_leases_for_agent( + agent_id, now, is_revoked=is_revoked + ): binding = runtime.catalog.get_network_binding(lease.capability_ref) if binding is None: continue @@ -63,9 +61,15 @@ def record_l7_flow( """Record an L7 flow against a matching network lease quota. Returns True when a matching active lease was found and quota decremented. + Successful HTTP responses (2xx) decrement quota; other statuses are logged only. """ now = datetime.now(timezone.utc) - lease_id = _find_active_network_lease_for_host(runtime, agent, host, now) + billable = _is_billable_l7_status(status) + lease_id = ( + _find_active_network_lease_for_host(runtime, agent, host, now) + if billable + else None + ) event: dict[str, Any] = { "event": "l7_flow", @@ -81,7 +85,7 @@ def record_l7_flow( runtime.observability.emit_capability_event(event) - if lease_id is None: + if not billable or lease_id is None: return False runtime.record_action(agent, agent, lease_id, now) diff --git a/capability-runtime/src/capability_runtime/lease.py b/capability-runtime/src/capability_runtime/lease.py index d194efb1..335096e3 100644 --- a/capability-runtime/src/capability_runtime/lease.py +++ b/capability-runtime/src/capability_runtime/lease.py @@ -80,3 +80,17 @@ def iter_active_leases_for_ref( and not self.is_exhausted(lease_id) ): yield lease_id, lease + + def iter_active_leases_for_agent( + self, agent_id: AgentIdentity, now: datetime, *, is_revoked + ): + """Yield active leases held by an agent at a point in time.""" + for lease_id, lease in self._leases.items(): + if lease.agent_id != agent_id: + continue + if ( + not self.is_expired(lease_id, now) + and not is_revoked(lease_id) + and not self.is_exhausted(lease_id) + ): + yield lease_id, lease diff --git a/capability-runtime/tests/test_l7_record.py b/capability-runtime/tests/test_l7_record.py index 4ab918c9..03ff312b 100644 --- a/capability-runtime/tests/test_l7_record.py +++ b/capability-runtime/tests/test_l7_record.py @@ -94,3 +94,30 @@ def test_admin_surface_records_l7_flow(tmp_path): ) assert "result" in response assert response["result"]["recorded"] is True + + +def test_l7_record_does_not_decrement_on_error_status(tmp_path): + client = MockNetBirdClient(peers=[{"id": "peer-pp"}], routes=[]) + runtime = CapabilityRuntime(tmp_path / "t.jsonl", "trace", 1, netbird_client=client) + agent = AgentIdentity("devcontainer-agent") + ref = CapabilityRef("cap-reach-proxy") + binding = NetworkBinding(ref, "peer-pp", "100.64.0.5/32", None, SyncMode.ROUTE_ENABLE) + runtime.register_network_capability("reach_proxy", ref, binding, LifecycleState.VISIBLE) + register_lease_policy( + "reach_proxy", + LeasePolicy( + quota=2, + ttl=timedelta(minutes=15), + token_budget=10000, + risk_envelope="medium", + ), + ) + runtime.request_capability_lease(agent, agent, ref, "test") + + recorded = record_l7_flow( + runtime, agent, host="100.64.0.5", method="GET", status=503 + ) + assert recorded is False + + bundle = runtime.check_current_capabilities(agent, {}) + assert "reach_proxy" in [n.name for n in bundle.networks] diff --git a/capability-runtime/tests/test_rpc_unix.py b/capability-runtime/tests/test_rpc_unix.py index d8597eb6..420d7ad6 100644 --- a/capability-runtime/tests/test_rpc_unix.py +++ b/capability-runtime/tests/test_rpc_unix.py @@ -90,6 +90,44 @@ def test_invalid_json_returns_parse_error(tmp_path, mock_dispatcher): server.stop() +def test_l7_record_client_connects_to_world_writable_admin_socket(tmp_path, mock_dispatcher): + """mitmproxy runs non-root; admin.sock must be group/world connectable.""" + sock_path = tmp_path / "admin.sock" + server = UnixRpcServer(sock_path, mock_dispatcher, socket_mode=0o666) + server.start() + try: + _wait_for_socket(sock_path) + assert oct(sock_path.stat().st_mode & 0o777) == "0o666" + + import importlib.util + import os + + os.environ["CAPABILITY_ADMIN_SOCKET"] = str(sock_path) + + script = ( + Path(__file__).resolve().parents[2] + / "cli/templates/devcontainer/sandcat/scripts/l7_record_client.py" + ) + spec = importlib.util.spec_from_file_location("l7_record_client", script) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + + mod.record_flow( + agent_id="devcontainer-agent", + host="100.64.0.5", + method="GET", + status=200, + ) + deadline = time.time() + 2.0 + while time.time() < deadline and mock_dispatcher.handle.call_count == 0: + time.sleep(0.01) + mock_dispatcher.handle.assert_called_once() + request = mock_dispatcher.handle.call_args[0][0] + assert request["method"] == "capability.l7.record" + finally: + server.stop() + + def test_socket_mode_allows_non_owner_connect(tmp_path, mock_dispatcher): sock_path = tmp_path / "agent.sock" server = UnixRpcServer(sock_path, mock_dispatcher, socket_mode=0o666) diff --git a/cli/.version b/cli/.version new file mode 100644 index 00000000..9daeafb9 --- /dev/null +++ b/cli/.version @@ -0,0 +1 @@ +test diff --git a/cli/README.md b/cli/README.md index ed3b2e2d..ff11e34f 100644 --- a/cli/README.md +++ b/cli/README.md @@ -31,7 +31,7 @@ Options: Adds a `capability-runtime` compose service, mounts a shared Unix socket volume into the agent container, and installs `capability-mcp-bridge` for Cursor MCP. NetBird API credentials stay in the sidecar — they are not injected into the agent. -- `--proxy-peer` - Enable the proxy-peer gateway stack (requires `--netbird`). +- `--proxy-peer` - Enable the proxy-peer gateway stack (requires `--netbird` and `--capability`). Deploys a dedicated NetBird `proxy-peer` compose service and copies a Layer 1 mitmproxy settings example (`.sandcat/settings.proxy-peer.example.json`) for deny-by-default egress. Pair with `--capability` for Layer 2 lease/revoke control. @@ -580,7 +580,7 @@ Replace placeholders in `capability-catalog.json` before leasing `reach_api`: ## Proxy-peer gateway (Phase 3e) -When initialized with `--proxy-peer` (requires `--netbird`), sandcat deploys a +When initialized with `--proxy-peer` (requires `--netbird` and `--capability`), sandcat deploys a dedicated NetBird `proxy-peer` gateway peer and operationalizes a **two-layer** control model for agent egress: diff --git a/cli/lib/composefile.bash b/cli/lib/composefile.bash index 7a19e1d3..149b05d0 100644 --- a/cli/lib/composefile.bash +++ b/cli/lib/composefile.bash @@ -753,9 +753,9 @@ enable_capability() { fi local has_mitm_cap_vol has_l7_client has_l7_record has_mitm_agent_id - has_mitm_cap_vol=$(yq '[.services.mitmproxy.volumes[]? | select(. == "capability-socket:/run/sandcat-capability:ro")] | length' "$proxy_compose") + has_mitm_cap_vol=$(yq '[.services.mitmproxy.volumes[]? | select(. == "capability-socket:/run/sandcat-capability")] | length' "$proxy_compose") if [[ "$has_mitm_cap_vol" -eq 0 ]]; then - yq -i '.services.mitmproxy.volumes += ["capability-socket:/run/sandcat-capability:ro"]' "$proxy_compose" + yq -i '.services.mitmproxy.volumes += ["capability-socket:/run/sandcat-capability"]' "$proxy_compose" fi has_l7_client=$(yq '[.services.mitmproxy.volumes[]? | select(. == "./scripts/l7_record_client.py:/scripts/l7_record_client.py:ro")] | length' "$proxy_compose") diff --git a/cli/libexec/init/devcontainer b/cli/libexec/init/devcontainer index 79a2d8ad..dd59c4ff 100755 --- a/cli/libexec/init/devcontainer +++ b/cli/libexec/init/devcontainer @@ -17,7 +17,7 @@ source "$SCT_LIBDIR/devcontainer.bash" # --ide - The IDE name (e.g., "vscode", "jetbrains", "none") (optional) # --netbird-management-url - NetBird management server URL for wg-client (optional) # --capability - Enable capability-runtime sidecar (requires --netbird) -# --proxy-peer - Enable proxy-peer gateway stack (requires --netbird) +# --proxy-peer - Enable proxy-peer gateway stack (requires --netbird and --capability) devcontainer() { local settings_file="" local project_path="" @@ -114,6 +114,10 @@ devcontainer() { echo "--proxy-peer requires --netbird" | error return 1 fi + if [[ "$proxy_peer" == "true" && "$capability" != "true" ]]; then + echo "--proxy-peer requires --capability" | error + return 1 + fi local devcontainer_dir="$project_path/.devcontainer" local compose_file="$devcontainer_dir/compose-all.yml" diff --git a/cli/libexec/init/init b/cli/libexec/init/init index 73a9ad25..f044d0a3 100755 --- a/cli/libexec/init/init +++ b/cli/libexec/init/init @@ -164,7 +164,7 @@ add_secret_provider_tokens_to_user_settings() { # --1password - Deprecated; same as --secret-provider 1password # --features - Comma-separated optional non-provider features (tui) # --capability - Enable capability-runtime sidecar (requires --netbird) -# --proxy-peer - Enable proxy-peer gateway stack (requires --netbird) +# --proxy-peer - Enable proxy-peer gateway stack (requires --netbird and --capability) init() { require yq @@ -283,6 +283,10 @@ init() { echo "--proxy-peer requires --netbird" | error return 1 fi + if [[ "$proxy_peer" == "true" && "$capability" != "true" ]]; then + echo "--proxy-peer requires --capability" | error + return 1 + fi if [[ "$netbird_server_provided" == "true" ]]; then case "$netbird_server" in cloud|new|quickstart|http://*|https://*) diff --git a/cli/templates/devcontainer/sandcat/scripts/mitmproxy_addon_common.py b/cli/templates/devcontainer/sandcat/scripts/mitmproxy_addon_common.py index baf394da..57333673 100644 --- a/cli/templates/devcontainer/sandcat/scripts/mitmproxy_addon_common.py +++ b/cli/templates/devcontainer/sandcat/scripts/mitmproxy_addon_common.py @@ -811,8 +811,6 @@ def _is_request_allowed(self, method: str | None, host: str) -> bool: rule = self._find_matching_rule(method, host) return rule is not None and rule.get("action") == "allow" - _MESH_CGNAT = ipaddress.ip_network("100.64.0.0/10") - def _host_matches_network_allow_rule(self, host: str) -> bool: host = host.lower().rstrip(".") for rule in self.network_rules: @@ -822,19 +820,6 @@ def _host_matches_network_allow_rule(self, host: str) -> bool: return True return False - @classmethod - def _host_in_mesh_cgnat(cls, host: str) -> bool: - try: - return ipaddress.ip_address(host) in cls._MESH_CGNAT - except ValueError: - return False - - def _should_record_l7_flow(self, host: str) -> bool: - return ( - self._host_matches_network_allow_rule(host) - or self._host_in_mesh_cgnat(host) - ) - # ----------------------------------------------------------- env writer @staticmethod @@ -1108,9 +1093,12 @@ def response(self, flow: http.HTTPFlow): return if not flow.response or flow.response.status_code is None: return + status = flow.response.status_code + if not (200 <= status < 300): + return host = flow.request.pretty_host - if not self._should_record_l7_flow(host): + if not self._host_matches_network_allow_rule(host): return from l7_record_client import record_flow @@ -1120,7 +1108,7 @@ def response(self, flow: http.HTTPFlow): agent_id=agent_id, host=host, method=flow.request.method, - status=flow.response.status_code, + status=status, ) def dns_request(self, flow: dns.DNSFlow): diff --git a/cli/test/composefile/capability.bats b/cli/test/composefile/capability.bats index ddc1c59f..78b80900 100644 --- a/cli/test/composefile/capability.bats +++ b/cli/test/composefile/capability.bats @@ -50,7 +50,7 @@ teardown() { @test "enable_capability mounts capability admin socket into mitmproxy" { enable_capability "$COMPOSE_DIR" - yq -e '.services.mitmproxy.volumes[] | select(. == "capability-socket:/run/sandcat-capability:ro")' \ + yq -e '.services.mitmproxy.volumes[] | select(. == "capability-socket:/run/sandcat-capability")' \ "$COMPOSE_DIR/sandcat/compose-proxy.yml" yq -e '.services.mitmproxy.volumes[] | select(. == "./scripts/l7_record_client.py:/scripts/l7_record_client.py:ro")' \ "$COMPOSE_DIR/sandcat/compose-proxy.yml" diff --git a/cli/test/init/init_proxy_peer.bats b/cli/test/init/init_proxy_peer.bats index 30ee7054..2497502f 100644 --- a/cli/test/init/init_proxy_peer.bats +++ b/cli/test/init/init_proxy_peer.bats @@ -31,6 +31,14 @@ teardown() { assert_output --partial "--proxy-peer requires --netbird" } +@test "init rejects --proxy-peer without --capability" { + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" \ + --stacks "" --proxy web --features "" --secret-provider none \ + --netbird --netbird-server cloud --proxy-peer + assert_failure + assert_output --partial "--proxy-peer requires --capability" +} + @test "init --netbird --capability --proxy-peer copies compose-proxy-peer.yml" { mkdir -p "$PROJECT_DIR/.sandcat" touch "$PROJECT_DIR/.sandcat/settings.json" @@ -54,7 +62,7 @@ teardown() { run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" \ --stacks "" --proxy web --features "" --secret-provider none \ - --netbird --netbird-server cloud --proxy-peer + --netbird --netbird-server cloud --capability --proxy-peer assert_success [[ -f "$PROJECT_DIR/.sandcat/settings.proxy-peer.example.json" ]] From c04b46cf364d6d564dcaefc5786e162a2b098e97 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Thu, 9 Jul 2026 06:38:26 +0000 Subject: [PATCH 073/138] chore: drop accidental cli/.version from commit --- cli/.version | 1 - 1 file changed, 1 deletion(-) delete mode 100644 cli/.version diff --git a/cli/.version b/cli/.version deleted file mode 100644 index 9daeafb9..00000000 --- a/cli/.version +++ /dev/null @@ -1 +0,0 @@ -test From fd46105f4d35173f9facd2611bf06dae472bc288 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Thu, 9 Jul 2026 08:49:52 +0000 Subject: [PATCH 074/138] fix(capability-runtime): fail closed on revoke and schedule TTL expiry Enforce durable operator revoke semantics and prevent logical revocation when NetBird disable fails. Add an expiry scheduler thread for prompt TTL processing and URL-encode REST path segments for NetBird client parity. --- .../src/capability_runtime/daemon.py | 41 +++++++++++++++ .../src/capability_runtime/netbird_client.py | 13 +++-- .../src/capability_runtime/runtime.py | 51 ++++++++++++------- .../tests/test_netbird_client.py | 8 +-- .../tests/test_re_lease_after_revoke.py | 10 ++-- .../tests/test_revoke_by_lease_network.py | 20 +++----- .../tests/test_route_watcher_routes.py | 11 ++-- .../tests/test_runtime_network.py | 24 ++++----- 8 files changed, 118 insertions(+), 60 deletions(-) diff --git a/capability-runtime/src/capability_runtime/daemon.py b/capability-runtime/src/capability_runtime/daemon.py index d0f127a7..73a7d8f2 100644 --- a/capability-runtime/src/capability_runtime/daemon.py +++ b/capability-runtime/src/capability_runtime/daemon.py @@ -159,6 +159,44 @@ def _run(self) -> None: self._stop.wait(self._interval) +class _LeaseExpiryThread: + """Wake at nearest lease expiry to process TTL promptly.""" + + def __init__(self, runtime: CapabilityRuntime) -> None: + self._runtime = runtime + self._stop = threading.Event() + self._thread: threading.Thread | None = None + + def start(self) -> None: + if self._thread is not None and self._thread.is_alive(): + return + self._stop.clear() + self._thread = threading.Thread( + target=self._run, + name="lease-expiry", + daemon=True, + ) + self._thread.start() + + def stop(self) -> None: + self._stop.set() + if self._thread is not None: + self._thread.join(timeout=2.0) + self._thread = None + + def _run(self) -> None: + while not self._stop.is_set(): + now = time.time() + expiry = self._runtime.next_network_lease_expiry() + if expiry is None: + self._stop.wait(1.0) + continue + wait_s = max(0.0, expiry.timestamp() - now) + if self._stop.wait(wait_s): + break + self._runtime.process_expired_network_leases() + + class CapabilityDaemon: """Run runtime, route watcher, and agent/admin RPC sockets.""" @@ -169,6 +207,7 @@ def __init__(self, config: DaemonConfig) -> None: load_catalog_into_runtime(self._runtime, config.catalog_path) self._watcher = RouteDisappearanceWatcher(self._runtime, self._netbird_client) self._watcher_thread = _WatcherPollThread(self._watcher, config.watch_interval) + self._lease_expiry_thread = _LeaseExpiryThread(self._runtime) self._agent_server = UnixRpcServer( config.agent_socket, RpcDispatcher( @@ -199,6 +238,7 @@ def runtime(self) -> CapabilityRuntime: def start(self) -> None: if self._running: return + self._lease_expiry_thread.start() self._watcher_thread.start() self._agent_server.start() self._admin_server.start() @@ -209,6 +249,7 @@ def stop(self) -> None: return self._admin_server.stop() self._agent_server.stop() + self._lease_expiry_thread.stop() self._watcher_thread.stop() self._running = False diff --git a/capability-runtime/src/capability_runtime/netbird_client.py b/capability-runtime/src/capability_runtime/netbird_client.py index dbfd2496..0dc48643 100644 --- a/capability-runtime/src/capability_runtime/netbird_client.py +++ b/capability-runtime/src/capability_runtime/netbird_client.py @@ -5,6 +5,7 @@ from dataclasses import replace from typing import Any, Protocol from urllib.error import HTTPError +from urllib.parse import quote from urllib.request import Request, urlopen from capability_runtime.network import NetworkBinding, SyncMode @@ -249,10 +250,10 @@ def list_groups(self, *, name: str | None = None) -> list[dict]: return self._request("GET", path) def remove_peer(self, peer_id: str) -> None: - self._request("DELETE", f"/api/peers/{peer_id}") + self._request("DELETE", f"/api/peers/{_urlencode_path_segment(peer_id)}") def remove_route(self, route_id: str) -> None: - self._request("DELETE", f"/api/routes/{route_id}") + self._request("DELETE", f"/api/routes/{_urlencode_path_segment(route_id)}") def peer_exists(self, peer_id: str) -> bool: return any(peer.get("id") == peer_id for peer in self.list_peers()) @@ -308,12 +309,12 @@ def _set_route_enabled(self, route_id: str, enabled: bool) -> None: route = self._get_route(route_id) self._request( "PUT", - f"/api/routes/{route_id}", + f"/api/routes/{_urlencode_path_segment(route_id)}", route_put_body(route, enabled=enabled), ) def _get_route(self, route_id: str) -> dict: - return self._request("GET", f"/api/routes/{route_id}")[0] + return self._request("GET", f"/api/routes/{_urlencode_path_segment(route_id)}")[0] def _create_route(self, binding: NetworkBinding) -> dict: return self._request( @@ -423,3 +424,7 @@ def _request( if isinstance(parsed, list): return parsed return [parsed] + + +def _urlencode_path_segment(value: str) -> str: + return quote(value, safe="") diff --git a/capability-runtime/src/capability_runtime/runtime.py b/capability-runtime/src/capability_runtime/runtime.py index eea4e883..677313c8 100644 --- a/capability-runtime/src/capability_runtime/runtime.py +++ b/capability-runtime/src/capability_runtime/runtime.py @@ -93,11 +93,18 @@ def process_expired_network_leases(self, now: datetime | None = None) -> None: binding = self.catalog.get_network_binding(lease.capability_ref) if binding is None: continue - physical_sync = "disabled" try: revoke_network_binding(self._netbird_backend, binding, "TTL expired") except Exception: - physical_sync = "failed" + self.observability.emit_capability_event( + { + "event": "physical_sync_failed", + "lease_id": lease_id.value, + "capability_ref": lease.capability_ref.value, + "reason": "TTL expired", + } + ) + continue self.revocation_manager.revoke_by_lease(lease_id, "TTL expired") self.catalog.set_state(lease.capability_ref, LifecycleState.EXPIRED) self.observability.emit_capability_event( @@ -106,10 +113,28 @@ def process_expired_network_leases(self, now: datetime | None = None) -> None: "lease_id": lease_id.value, "capability_ref": lease.capability_ref.value, "reason": "TTL expired", - "physical_sync": physical_sync, + "physical_sync": "disabled", } ) + def next_network_lease_expiry(self, now: datetime | None = None) -> datetime | None: + """Return earliest expiry among active network leases.""" + if now is None: + now = datetime.now(timezone.utc) + + next_expiry: datetime | None = None + for lease_id, lease in self.lease_manager._leases.items(): + if self.revocation_manager.is_lease_revoked(lease_id): + continue + if self.lease_manager.is_exhausted(lease_id): + continue + if self.catalog.get_network_binding(lease.capability_ref) is None: + continue + expiry = lease.expires_at + if next_expiry is None or expiry < next_expiry: + next_expiry = expiry + return next_expiry + def register_network_capability( self, name: str, @@ -258,14 +283,12 @@ def request_capability_lease( _assert_caller(caller, agent_id) # Check capability exists state = self.catalog.get_state(capability_ref) - # REVOKED/EXPIRED remain leasable so operators can re-grant after revoke/TTL - # without sidecar restart. Physical routes are disabled on revoke/expiry; - # a new lease re-enables via NetBird (see test_re_lease_after_revoke.py). + # REVOKED is intentionally not leasable from the agent surface: operator + # revoke acts as a durable stop until lifecycle state changes externally. leasable = { LifecycleState.DECLARED, LifecycleState.DISCOVERABLE, LifecycleState.VISIBLE, - LifecycleState.REVOKED, LifecycleState.EXPIRED, } if state not in leasable: @@ -378,12 +401,9 @@ def revoke_capability( if binding is not None and self._netbird_backend is not None: from capability_runtime.netbird_sync import revoke_network_binding + revoke_network_binding(self._netbird_backend, binding, reason) physical_sync = "disabled" - try: - revoke_network_binding(self._netbird_backend, binding, reason) - physical_revocation = True - except Exception: - physical_sync = "failed" + physical_revocation = True self.revocation_manager.revoke_by_lease(target, reason) self._bundle_version += 1 event: dict[str, object] = { @@ -404,12 +424,9 @@ def revoke_capability( if binding is not None and self._netbird_backend is not None: from capability_runtime.netbird_sync import revoke_network_binding + revoke_network_binding(self._netbird_backend, binding, reason) physical_sync = "disabled" - try: - revoke_network_binding(self._netbird_backend, binding, reason) - physical_revocation = True - except Exception: - physical_sync = "failed" + physical_revocation = True # Perform logical revocation self.revocation_manager.revoke_by_ref(target, reason) diff --git a/capability-runtime/tests/test_netbird_client.py b/capability-runtime/tests/test_netbird_client.py index bf02f545..3287c615 100644 --- a/capability-runtime/tests/test_netbird_client.py +++ b/capability-runtime/tests/test_netbird_client.py @@ -58,10 +58,10 @@ def fake_urlopen(request): monkeypatch.setattr("capability_runtime.netbird_client.urlopen", fake_urlopen) client = RestNetBirdClient() - client.remove_peer("peer-abc") + client.remove_peer("peer/abc 1") assert captured["method"] == "DELETE" - assert captured["url"] == "https://api.netbird.io/api/peers/peer-abc" + assert captured["url"] == "https://api.netbird.io/api/peers/peer%2Fabc%201" assert captured["headers"]["Authorization"] == "Token test-token" @@ -77,10 +77,10 @@ def fake_urlopen(request): monkeypatch.setattr("capability_runtime.netbird_client.urlopen", fake_urlopen) client = RestNetBirdClient() - client.remove_route("route-1") + client.remove_route("route/1 a") assert captured["method"] == "DELETE" - assert captured["url"] == "https://api.netbird.io/api/routes/route-1" + assert captured["url"] == "https://api.netbird.io/api/routes/route%2F1%20a" def test_rest_client_list_peers(monkeypatch): diff --git a/capability-runtime/tests/test_re_lease_after_revoke.py b/capability-runtime/tests/test_re_lease_after_revoke.py index 588e7512..ca3e913d 100644 --- a/capability-runtime/tests/test_re_lease_after_revoke.py +++ b/capability-runtime/tests/test_re_lease_after_revoke.py @@ -1,7 +1,10 @@ # capability-runtime/tests/test_re_lease_after_revoke.py from datetime import datetime, timedelta, timezone +import pytest + from capability_runtime.catalog import LifecycleState +from capability_runtime.errors import CapabilityUnknown from capability_runtime.netbird_client import MockNetBirdClient from capability_runtime.network import NetworkBinding, SyncMode from capability_runtime.runtime import CapabilityRuntime @@ -22,12 +25,11 @@ def test_re_lease_after_revoke_by_ref(tmp_path): runtime.revoke_capability(_OPERATOR, ref, "done") assert runtime.catalog.get_state(ref) == LifecycleState.REVOKED - decision2 = runtime.request_capability_lease(agent, agent, ref, "second lease") - assert decision2.lease_id is not None + with pytest.raises(CapabilityUnknown): + runtime.request_capability_lease(agent, agent, ref, "second lease") assert len(client.list_routes()) == 1 bundle = runtime.check_current_capabilities(agent, {}) - assert "reach_api" in [n.name for n in bundle.networks] - assert bundle.networks[0].lease_id == decision2.lease_id + assert "reach_api" not in [n.name for n in bundle.networks] def test_re_lease_after_expiry(tmp_path): diff --git a/capability-runtime/tests/test_revoke_by_lease_network.py b/capability-runtime/tests/test_revoke_by_lease_network.py index 32ff9390..7d26645c 100644 --- a/capability-runtime/tests/test_revoke_by_lease_network.py +++ b/capability-runtime/tests/test_revoke_by_lease_network.py @@ -1,8 +1,6 @@ # capability-runtime/tests/test_revoke_by_lease_network.py """Revoke by lease ID must disable network bindings.""" -import json - from capability_runtime.catalog import LifecycleState from capability_runtime.netbird_client import MockNetBirdClient from capability_runtime.network import NetworkBinding, SyncMode @@ -35,7 +33,9 @@ def test_revoke_by_lease_id_disables_network_route(tmp_path): assert runtime._bundle_version > version_before -def test_revoke_by_lease_id_continues_when_netbird_disable_fails(tmp_path, monkeypatch): +def test_revoke_by_lease_id_fails_closed_when_netbird_disable_fails(tmp_path, monkeypatch): + import pytest + client = MockNetBirdClient( peers=[{"id": "peer-abc"}], routes=[{"id": "route-1", "network": "10.8.0.0/24", "peer": "peer-abc", "enabled": True}], @@ -53,22 +53,14 @@ def boom(*_a, **_kw): monkeypatch.setattr(client, "disable_binding", boom) - runtime.revoke_capability(_OPERATOR, decision.lease_id, "operator revoke") + with pytest.raises(RuntimeError, match="netbird down"): + runtime.revoke_capability(_OPERATOR, decision.lease_id, "operator revoke") bundle = runtime.check_current_capabilities(agent, {}) - assert "reach_api" not in [n.name for n in bundle.networks] + assert "reach_api" in [n.name for n in bundle.networks] routes = [r for r in client.list_routes() if r["id"] == "route-1"] assert routes[0]["enabled"] is True - events = [ - json.loads(line) - for line in (tmp_path / "t.jsonl").read_text().strip().split("\n") - if line - ] - revoke_events = [e for e in events if e.get("event") == "capability_revoked"] - assert revoke_events[-1]["physical_sync"] == "failed" - assert revoke_events[-1]["capability_ref"] == ref.value - def test_revoke_by_lease_id_skips_netbird_for_tool_capability(tmp_path): """Tool leases must not call NetBird on revoke-by-lease-ID.""" diff --git a/capability-runtime/tests/test_route_watcher_routes.py b/capability-runtime/tests/test_route_watcher_routes.py index c56bfc1e..0ef7151c 100644 --- a/capability-runtime/tests/test_route_watcher_routes.py +++ b/capability-runtime/tests/test_route_watcher_routes.py @@ -81,7 +81,7 @@ def test_watcher_revokes_when_route_disabled_with_active_lease(tmp_path): assert runtime.catalog.get_state(ref) == LifecycleState.REVOKED -def test_watcher_reconciles_route_after_failed_physical_revoke(tmp_path, monkeypatch): +def test_operator_revoke_succeeds_after_retry_when_disable_recovers(tmp_path, monkeypatch): from capability_runtime.catalog import LifecycleState from capability_runtime.route_watcher import RouteDisappearanceWatcher from capability_runtime.runtime import CapabilityRuntime @@ -108,12 +108,15 @@ def flaky_disable(binding): return original_disable(binding) monkeypatch.setattr(client, "disable_binding", flaky_disable) - runtime.revoke_capability(AgentIdentity("operator"), ref, "security") - assert runtime.catalog.get_state(ref) == LifecycleState.REVOKED + import pytest + + with pytest.raises(RuntimeError, match="netbird down"): + runtime.revoke_capability(AgentIdentity("operator"), ref, "security") + assert runtime.catalog.get_state(ref) == LifecycleState.LEASED routes = [r for r in client.list_routes() if r["id"] == "route-1"] assert routes[0]["enabled"] is True - RouteDisappearanceWatcher(runtime, client).poll_once() + runtime.revoke_capability(AgentIdentity("operator"), ref, "security") routes = [r for r in client.list_routes() if r["id"] == "route-1"] assert routes[0]["enabled"] is False diff --git a/capability-runtime/tests/test_runtime_network.py b/capability-runtime/tests/test_runtime_network.py index 4d73b67d..3fc70818 100644 --- a/capability-runtime/tests/test_runtime_network.py +++ b/capability-runtime/tests/test_runtime_network.py @@ -48,8 +48,9 @@ def test_revoke_network_capability_calls_netbird_backend(tmp_path): assert "reach_api" not in [n.name for n in bundle.networks] -def test_revoke_by_ref_continues_when_netbird_disable_fails(tmp_path, monkeypatch): - import json +def test_revoke_by_ref_fails_closed_when_netbird_disable_fails(tmp_path, monkeypatch): + + import pytest from capability_runtime.catalog import LifecycleState from capability_runtime.netbird_client import MockNetBirdClient @@ -73,21 +74,15 @@ def boom(*_a, **_kw): raise RuntimeError("netbird down") monkeypatch.setattr(client, "disable_binding", boom) - runtime.revoke_capability(operator, ref, "security") + with pytest.raises(RuntimeError, match="netbird down"): + runtime.revoke_capability(operator, ref, "security") bundle = runtime.check_current_capabilities(agent, {}) - assert "reach_api" not in [n.name for n in bundle.networks] + assert "reach_api" in [n.name for n in bundle.networks] + assert runtime.catalog.get_state(ref) == LifecycleState.LEASED routes = [r for r in client.list_routes() if r["id"] == "route-1"] assert routes[0]["enabled"] is True - events = [ - json.loads(line) - for line in (tmp_path / "t.jsonl").read_text().strip().split("\n") - if line - ] - revoke_events = [e for e in events if e.get("event") == "capability_revoked"] - assert revoke_events[-1]["physical_sync"] == "failed" - def test_revoke_non_network_capability_does_not_call_netbird(tmp_path): from capability_runtime.catalog import LifecycleState @@ -207,7 +202,7 @@ def test_ttl_expiry_via_watcher_poll(tmp_path): assert runtime.catalog.get_state(ref) == LifecycleState.EXPIRED -def test_ttl_expiry_continues_when_netbird_disable_fails(tmp_path, monkeypatch): +def test_ttl_expiry_fails_closed_when_netbird_disable_fails(tmp_path, monkeypatch): from datetime import datetime, timedelta, timezone from capability_runtime.catalog import LifecycleState from capability_runtime.netbird_client import MockNetBirdClient @@ -234,3 +229,6 @@ def boom(*_a, **_kw): bundle = runtime.check_current_capabilities(agent, {}) assert "reach_api" not in [n.name for n in bundle.networks] + assert runtime.catalog.get_state(ref) == LifecycleState.LEASED + routes = [r for r in client.list_routes() if r["id"] == "route-1"] + assert routes[0]["enabled"] is True From e764e19b4e302be7423f4b50a139c2787428b3ef Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Thu, 9 Jul 2026 14:48:51 +0000 Subject: [PATCH 075/138] test(cli): harden proxy-peer hello bats against port flakes --- cli/test/proxy_peer/proxy_peer_hello.bats | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/cli/test/proxy_peer/proxy_peer_hello.bats b/cli/test/proxy_peer/proxy_peer_hello.bats index fd96b4ed..f6fae169 100644 --- a/cli/test/proxy_peer/proxy_peer_hello.bats +++ b/cli/test/proxy_peer/proxy_peer_hello.bats @@ -13,18 +13,26 @@ teardown() { } @test "proxy-peer-hello responds with JSON on /hello" { - python3 "$HELLO" --port 18080 & + # Avoid fixed-port flakes when another test/process already uses 18080. + local port=$((18080 + (BATS_SUITE_TEST_NUMBER % 1000))) + local hello_log="$BATS_TEST_TMPDIR/proxy-peer-hello.log" + + python3 "$HELLO" --port "$port" >"$hello_log" 2>&1 & HELLO_PID=$! export HELLO_PID - for _ in $(seq 1 20); do - if curl -sf http://127.0.0.1:18080/hello >/dev/null 2>&1; then + for _ in $(seq 1 30); do + if ! kill -0 "$HELLO_PID" 2>/dev/null; then + run cat "$hello_log" + assert_failure + fi + if curl -sf "http://127.0.0.1:${port}/hello" >/dev/null 2>&1; then break fi sleep 0.1 done - run curl -sf http://127.0.0.1:18080/hello + run curl -sf "http://127.0.0.1:${port}/hello" assert_success assert_output '{"service": "proxy-peer", "ok": true}' } From fff385e03fc2fe352acc3212689d36d8d902cabb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Wi=C4=85cek?= Date: Wed, 22 Jul 2026 10:31:42 +0000 Subject: [PATCH 076/138] docs(spec): add proxy-peer NetBird DNS targeting design Co-authored-by: Cursor --- ...ility-proxy-peer-gateway-phase3e-design.md | 210 ++++++++++++++++++ ...026-07-20-proxy-peer-netbird-dns-design.md | 139 ++++++++++++ 2 files changed, 349 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-08-capability-proxy-peer-gateway-phase3e-design.md create mode 100644 docs/superpowers/specs/2026-07-20-proxy-peer-netbird-dns-design.md diff --git a/docs/superpowers/specs/2026-07-08-capability-proxy-peer-gateway-phase3e-design.md b/docs/superpowers/specs/2026-07-08-capability-proxy-peer-gateway-phase3e-design.md new file mode 100644 index 00000000..ccc0cb60 --- /dev/null +++ b/docs/superpowers/specs/2026-07-08-capability-proxy-peer-gateway-phase3e-design.md @@ -0,0 +1,210 @@ +# Capability Proxy-Peer Gateway (Phase 3e) — Design Spec + +> **Date:** 2026-07-08 +> **Status:** **DRAFT** — approved direction; pending implementation +> **Scope:** Deploy a dedicated NetBird **proxy-peer** as a controlled gateway to protected upstreams (HTTP APIs, MCP proxies, paid SaaS). Sandcat uses a **two-layer control model**: static egress baseline (mitmproxy) + dynamic lease/revoke/quota (CapabilityRuntime ↔ NetBird Routes API). + +**Prerequisite:** [Phase 3c — NetBird Policy Sync](./2026-06-30-capability-netbird-policy-sync-phase3c-design.md) (engineering gate complete) + +**Follow-up:** [Phase 3d — Gateway metering slice](./2026-06-30-capability-agent-network-phase3d-design.md) (Agent Network / token budgets on proxy-peer upstreams) + +**Ergonomics amendment:** [Proxy-Peer NetBird DNS Targeting](./2026-07-20-proxy-peer-netbird-dns-design.md) — stable peer hostnames replace per-recreate IP edits in catalog + Layer 1 + +**Consumer:** [Phase 4 Comparative Evaluation](./2026-06-23-capability-comparative-evaluation-phase4-design.md) — benchmark tasks E2, E7, E8 + +--- + +## 1. Problem + +Phase 3c proves lease/revoke drives NetBird route enable/disable for a mesh target. Real deployments need: + +| Gap | Today | +|-----|-------| +| **Trust zones** | Catalog points at arbitrary peers/CIDRs; no dedicated gateway pattern | +| **Bypass risk** | mitmproxy may allow broad hosts; agent could reach SaaS directly if static rules permit | +| **Paid / sensitive upstreams** | No standard place to hold API keys (context7, internal APIs) away from agent | +| **Usage throttling** | `record_action` + quota exists logically but is not wired from egress flows | +| **Two-layer story** | Layer 1 (static menu) vs Layer 2 (dynamic lease) is implicit, not operationalized | + +--- + +## 2. Two-layer control model + +```text +Layer 1 — Static baseline (always on) + iptables kill switch + mitmproxy allowlist + → defines what destinations can EVER leave the agent stack + → deny-by-default; allow proxy-peer mesh IP:port only (+ infra mirrors) + +Layer 2 — Dynamic lease (operator/agent MCP) + CapabilityRuntime bundle + NetBird route enable/disable + → defines WHEN a declared capability is reachable + → revoke / quota exhaustion / TTL closes the route (circuit breaker) +``` + +```mermaid +flowchart TB + subgraph L1["Layer 1 — static"] + MITM["mitmproxy allowlist"] + IPT["iptables kill switch"] + end + + subgraph L2["Layer 2 — dynamic"] + RT["CapabilityRuntime"] + NB["NetBird Routes API"] + end + + subgraph GW["proxy-peer (NetBird peer)"] + PX["forward proxy / hello"] + UP["context7 / internal API / MCP"] + PX --> UP + end + + AG["agent"] --> WG0["wg0"] --> MITM --> WT0["wt0"] + RT --> NB --> WT0 + WT0 -->|"leased"| PX +``` + +**Invariant (unchanged):** agent egress still flows **wg0 → mitmproxy → wt0**. proxy-peer is a **mesh peer**, not a bypass around mitmproxy. + +--- + +## 3. Goals + +1. **proxy-peer container** — enrolls as NetBird peer (`NET_ADMIN`), exposes minimal HTTP surface on mesh IP. +2. **Layer 1 template** — sandcat settings profile allowing only proxy-peer mesh IP (and required infra). +3. **Layer 2 catalog** — network capabilities map to `proxy-peer` `/32`; lease/revoke uses existing `route_enable` sync. +4. **Catalog-driven lease policy** — per-cap `quota`, `ttl` for network caps (e.g. context7 `action_quota: 5`). +5. **Flow → quota** — mitmproxy post-hoc `l7_flow` record decrements quota via admin RPC (closes Phase 3c Task 6 gap). +6. **Engineering gate** — scripted smoke: lease → curl proxy-peer → revoke → unreachable; quota → auto-revoke. + +--- + +## 4. Non-Goals (Phase 3e) + +- NetBird **Networks** policy objects (migrate in Phase 3f; 3e stays Routes API) +- Per-request mitmproxy ↔ bundle RPC deny path (deprecated) +- Full MCP workload gateway on proxy-peer (stub path prefix only) +- Agent Network LLM proxy (Phase 3d) +- Proxy-peer holding production secrets for all SaaS (hello + one upstream stub sufficient for gate) + +--- + +## 5. proxy-peer deployment + +### 5.1 Topology + +- **Separate compose stack** (`compose-proxy-peer.yml`) started by operator alongside sandcat devcontainer — not in agent network namespace. +- Same `NB_SETUP_KEY` and management URL as wg-client. +- Container name: `proxy-peer`; publishes no host ports (mesh-only reachability). + +### 5.2 Runtime + +1. Install NetBird (reuse `netbird.env` pin from wg-client). +2. `netbird up` with setup key; management URL from env (host IP for self-hosted). +3. Run `proxy-peer-hello.py` — HTTP server on `0.0.0.0:8080` returning JSON with peer mesh IP and capability path stubs. + +### 5.3 Operator workflow + +```bash +sandcat init --netbird --capability --proxy-peer --name demo +# edit capability-catalog.json with proxy-peer peer_id + mesh /32 +docker compose -f .devcontainer/sandcat/compose-proxy-peer.yml up -d +sandcat compose up -d +``` + +--- + +## 6. Catalog extension + +```json +{ + "name": "reach_proxy", + "ref": "cap-reach-proxy", + "type": "network", + "peer_id": "", + "network": "/32", + "sync_mode": "route_enable", + "proxy": { + "port": 8080, + "path_prefix": "/hello" + }, + "lease_policy": { + "quota": 5, + "ttl_minutes": 15, + "token_budget": 10000 + } +} +``` + +| Field | Purpose | +|-------|---------| +| `proxy.port` | Documented target for mitm allowlist + smoke curl | +| `proxy.path_prefix` | Future upstream routing on proxy-peer | +| `lease_policy` | Overrides PoC hardcoded quotas in `runtime.request_capability_lease` | + +--- + +## 7. Layer 1 — mitmproxy profile + +Template `cli/templates/settings-proxy-peer.json`: + +```json +{ + "network": [ + { + "action": "allow", + "host": "REPLACE_PROXY_PEER_MESH_IP", + "port": 8080 + } + ] +} +``` + +`sandcat init --proxy-peer` copies this to project settings guidance; operator replaces mesh IP after first `proxy-peer` enrollment. + +--- + +## 8. Layer 2 — lease / quota / revoke + +Uses existing Phase 3c paths: + +| Event | Bundle | NetBird | +|-------|--------|---------| +| Lease | Cap visible | Route enabled to `/32` | +| Revoke | Cap removed | Route disabled | +| Quota exhausted | Revoke + cap removed | Route disabled | +| TTL expired | Expired state | Route disabled (best-effort) | + +**New in 3e:** mitmproxy emits `l7_flow` → admin `capability.l7.record` → `record_action` when `host` matches leased network capability's proxy peer. + +--- + +## 9. Success criteria (engineering gate) + +- [ ] `proxy-peer` enrolls; `sandcat netbird status` lists second peer with mesh IP +- [ ] Layer 1 mitm profile blocks direct `curl https://example.com` from agent +- [ ] Without lease: `curl http://:8080/hello` fails (timeout / no route) +- [ ] With lease: hello JSON succeeds through mitmproxy +- [ ] Revoke: hello unreachable within 30s +- [ ] Quota=2: third flow triggers revoke + route disable +- [ ] JSONL shows `capability_leased`, `quota_decremented`, `capability_revoked`, `physical_sync` + +--- + +## 10. Relationship to other phases + +| Phase | Relationship | +|-------|--------------| +| 3c | **Prerequisite** — route enable/disable on lease lifecycle | +| 3d | **Extends** proxy-peer with upstream metering / Agent Network for LLM | +| 3f (future) | NetBird Networks + deny-by-default policies | +| 4 | **Measures** two-layer model in E2, E7, E8 | + +--- + +## 11. Further reading + +- [Phase 3c spec](./2026-06-30-capability-netbird-policy-sync-phase3c-design.md) +- [Phase 3e implementation plan](../plans/2026-07-08-capability-proxy-peer-gateway-phase3e.md) +- [CONTEXT.md](../../../CONTEXT.md) diff --git a/docs/superpowers/specs/2026-07-20-proxy-peer-netbird-dns-design.md b/docs/superpowers/specs/2026-07-20-proxy-peer-netbird-dns-design.md new file mode 100644 index 00000000..19967273 --- /dev/null +++ b/docs/superpowers/specs/2026-07-20-proxy-peer-netbird-dns-design.md @@ -0,0 +1,139 @@ +# Proxy-Peer NetBird DNS Targeting — Design Spec + +> **Date:** 2026-07-20 +> **Status:** DRAFT +> **Scope:** Amendment to [Phase 3e](./2026-07-08-capability-proxy-peer-gateway-phase3e-design.md). Stop forcing operators to update hardcoded mesh IPs in `capability-catalog.json` and Layer 1 settings after every proxy-peer container recreate. + +**Parent:** [Phase 3e — Proxy-Peer Gateway](./2026-07-08-capability-proxy-peer-gateway-phase3e-design.md) + +**Plan:** [Proxy-Peer NetBird DNS Targeting Plan](../plans/2026-07-20-proxy-peer-netbird-dns.md) + +--- + +## 1. Problem + +Phase 3e requires the operator to manually update three files after recreating the `proxy-peer` container: + +| File | What changes | Why | +|------|-------------|-----| +| `capability-catalog.json` | `peer_id` + `network` IP | New container = new NetBird enrollment | +| `.sandcat/settings.json` | `host` IP in network allow rule | Layer 1 mitm allowlist keys on IP | +| Engineering gate smoke curl | Target IP in curl URL | Manual step in gate script | + +Each recreate produces a new peer ID and a new mesh IP, breaking all three locations simultaneously. This is a PoC-level pain that surfaces immediately during development (container rebuild, `--force-recreate`, `up --build`). + +--- + +## 2. Root causes + +1. **NetBird peer identity is container-ephemeral.** `/var/lib/netbird` is not persisted across container recreates, so each `netbird up` registers a fresh peer with a new ID and a new IP assignment from the mesh CIDR pool. +2. **Catalog stores physical `peer_id`/`network` directly.** There is no layer of indirection to a stable name. +3. **Layer 1 template uses literal IP.** `settings-proxy-peer.json` has `REPLACE_PROXY_PEER_MESH_IP` which is then pasted literally into settings — no stable hostname. + +--- + +## 3. Decision: hostname-first, IP-physical + +Physical NetBird routes remain `route_enable` on `/32` — the Routes API is CIDR-based and that does not change. + +**New indirection layer:** the operator sets a stable `dns_label` in the catalog (matching the NetBird peer name, which is set at enrollment time and does not change on recreate if the same hostname/`--name` is used). At lease time, capability-runtime resolves `dns_label` → current `peer_id` + mesh IP via the NetBird peers API, then calls `enable_binding` with the resolved values. + +**Agent DNS:** wg-client dnsmasq is extended to forward the NetBird mesh domain (`netbird.selfhosted` or the configured domain) to NetBird's internal nameserver when available. The agent can then `curl http://peer-proxy.netbird.selfhosted:8080/hello` and the FQDN resolves to the current mesh IP without operator intervention. + +**Layer 1 mitm:** `settings-proxy-peer.json` switches from a literal IP placeholder to `peer-proxy.netbird.selfhosted`. The mitmproxy addon already uses `fnmatch` host matching, so this requires no addon changes — just the settings value. + +**Fallback:** IP-only mode remains fully supported. If `dns_label` is absent, behavior is identical to Phase 3e today. If NetBird DNS is not enabled, the operator falls back to editing IPs as before. + +--- + +## 4. Catalog schema extension + +```json +{ + "ref": "cap-reach-proxy", + "type": "network", + "dns_label": "peer-proxy.netbird.selfhosted", + "peer_id": "peer-placeholder", + "network": "0.0.0.0/32", + "sync_mode": "route_enable", + "proxy": {"port": 8080, "path_prefix": "/hello"}, + "lease_policy": {"quota": 5, "ttl_minutes": 15, "token_budget": 10000} +} +``` + +| Field | When `dns_label` set | When not set | +|-------|---------------------|--------------| +| `dns_label` | Stable NetBird peer name (FQDN or hostname) | Absent | +| `peer_id` | Any placeholder; overwritten at resolve | Real peer ID (Phase 3e behavior) | +| `network` | Any placeholder CIDR; overwritten at resolve | Real mesh `/32` (Phase 3e behavior) | + +Resolution rules: +- Exact match on `dns_label` field of NetBird peer. +- Prefix match: if label contains no dot, match peers whose `dns_label` starts with `