diff --git a/docs/notes/development/buzz/self-hosting.md b/docs/notes/development/buzz/self-hosting.md index 09c869ea3..6c4b47cf0 100644 --- a/docs/notes/development/buzz/self-hosting.md +++ b/docs/notes/development/buzz/self-hosting.md @@ -426,3 +426,25 @@ The mulatta relay package labels version `0.2.0` from `relayVersion` while build Above all, every relay claim was read at `desktop-v0.5.4`, which is not the relay's release axis, while `crates/buzz-relay` ships at 0.2.0 on `relay-v*`. The spike labeled this caveat throughout and then reasoned as though it did not apply; it applies, and it applies to every citation in this document. + +## Addendum, 2026-08-19: the R2 gate is settled, and the relay has landed disabled + +This section is appended; nothing above it has been altered, and its record stands as written on 2026-08-04. + +The R2 conformance gate above has now been settled at the source, against upstream `relay-v0.2.1` (`6e5c462`) and against Cloudflare documentation fetched live on 2026-08-19. +The verdict of this note is upheld — R2 fails, and it fails both the startup probe and the steady-state push path — but three of the arguments above are wrong and are corrected here. +First, this note missed the documented off-switch: `BUZZ_GIT_CONFORMANCE_PROBE=false` exists at `main.rs:496-499` and is used by upstream's own tests, so the judgement that disabling hides rather than fixes the problem was right while the mechanism went unnamed. +Second, this note asserted the 429 mechanism without checking the S3 client layer: the client is `rust-s3` v0.37.2 with `fail-on-err`, and it does retry blindly — once, after one second, retrying 412 as well — which is insufficient against 32 writers on a 1-write/sec key but was never verified before the conclusion was drawn. +Third, this note overstated R2's consistency weakness: R2 documents strong read-after-write and strong list consistency, and would likely pass the probe's phases 1, 3 and 4, so the real gap is Cloudflare's silence on conditional-PUT CAS atomicity rather than documented weakness, and the disqualifying mechanism is throttling. +Two further corrections of detail: the probe issues 192 conditional writes per boot rather than roughly 96, and the probe's `DeleteObject` calls are error-ignored, so a 403 leaks scratch keys rather than failing the probe as the token-scope guidance above implies. +The recommendation carried forward is Garage single-node on magnetite, with MinIO as the known-good fallback because upstream verifies empirically against MinIO; Garage's conditional-PUT CAS was not documentation-verified in that pass, so the probe itself is the intended adjudicator. + +Packaging and a NixOS module now exist on branch `fm/vx-buzz-relay-magnetite`. +The relay is pinned separately from the desktop train at `pkgs/by-name/buzz/relay-source` and built at `pkgs/by-name/buzz/relay`, so a server upgrade is not coupled to a client upgrade. +The module is `modules/nixos/buzz-relay.nix`, imported by magnetite and NOT enabled. +The full change is recorded under `openspec/changes/buzz-relay-module/`, whose `design.md` carries the object-storage finding with its citations. + +The "if it is built anyway" section above, and its Phase 4 instruction to deploy to magnetite with the relay running, are superseded by the binding deploy-scope decision of 2026-08-19. +That decision lands the packaging, the module and the change reviewed and buildable with `services.buzz-relay.enable` defaulting to false and no host setting it, and keeps the operational go/no-go as a separate later change. +Three prerequisites remain unmet and gate that later change: an object store that passes the conformance probe, a decision on Redis as a new daemon class on this fleet, and a backup story that does not exist fleet-wide. +The hostname question in the open questions above is deliberately still unanswered, because the Host is a durable tenant key and the choice belongs to the change that actually turns the relay on. diff --git a/modules/machines/nixos/magnetite/default.nix b/modules/machines/nixos/magnetite/default.nix index a246361bc..899de2de8 100644 --- a/modules/machines/nixos/magnetite/default.nix +++ b/modules/machines/nixos/magnetite/default.nix @@ -48,6 +48,11 @@ in kanidm matrix omnigraph + # Imported but deliberately not enabled: services.buzz-relay.enable + # defaults to false and is not set here. See the header comment in + # modules/nixos/buzz-relay.nix for the object-store, Redis and backup + # prerequisites that gate switching it on. + buzz-relay effects-vanixiets-secrets effects-ironstar-secrets ]); diff --git a/modules/nixos/buzz-relay.nix b/modules/nixos/buzz-relay.nix new file mode 100644 index 000000000..ae6085a98 --- /dev/null +++ b/modules/nixos/buzz-relay.nix @@ -0,0 +1,806 @@ +# buzz-relay — the Buzz Nostr relay, git server and media host. +# +# DELIBERATELY SHIPPED DISABLED. Importing this module changes nothing; every +# unit, firewall hole and secret below sits behind `services.buzz-relay.enable`, +# which defaults to false and is not set on any host. The module exists so the +# configuration surface is reviewable and the credential slots are declared +# before anything is switched on, not because the relay is ready to run. +# +# Three operational prerequisites are unmet, and each is its own change: +# +# 1. Object storage. The relay hard-requires an S3-compatible store — media +# storage init is fatal at startup (`main.rs:444-448` at upstream +# 6e5c462: `config.media.validate()` then `MediaStorage::new`, both +# `?`-propagating) and there is no feature flag that disables media. Worse, +# the git object store runs a *conformance probe* at boot +# (`BUZZ_GIT_CONFORMANCE_PROBE`, default ON, `main.rs:498-516`) that races +# 32 writers for 3 rounds against the bucket to prove compare-and-swap +# semantics. A store that merely speaks S3 is not enough; it has to pass +# that probe. Which bucket, on which provider, with what consistency +# guarantees, is undecided. +# 2. Redis. Required (`config.rs:515`, and the relay's pubsub/registry paths +# assume it). This fleet has never run Redis as a daemon class — there is +# no precedent, no hardening baseline and no backup story for it here. +# 3. Backups. There is no backup system on this fleet at all: magnetite has +# local ZFS snapshots and explicitly no off-site replication. The relay's +# Postgres holds the community, membership and signed-auth history, which +# is exactly the class of state that must not live on one disk. +# +# Postgres, Redis and object storage are therefore modelled here as +# DEPENDENCIES AND OPTIONS. This module provisions none of them: no +# `services.postgresql` colonisation, no `services.redis`, no bucket. It states +# what it needs and asserts when enabled without it. +# +# There is also no nginx vhost, no DNS record and no tenant hostname in this +# change. That is deliberate and load-bearing rather than an omission: the Host +# derived from `RELAY_URL` is the durable tenant key. It is persisted as +# `communities.host` in Postgres and signed into every NIP-42/NIP-98 auth event, +# so it cannot be renamed later without invalidating history. `relayUrl` below +# consequently has NO DEFAULT and is required at enable time. +# +# See docs/notes/development/buzz/self-hosting.md. +{ ... }: +{ + flake.modules.nixos.buzz-relay = + { + config, + lib, + pkgs, + ... + }: + let + cfg = config.services.buzz-relay; + + # IPv6 literals must be bracketed before they are joined to a port, and + # the mesh addresses this fleet binds are IPv6 (modules/lib/hosts.nix). + bindTarget = + if lib.hasInfix ":" cfg.bindAddress then + "[${cfg.bindAddress}]:${toString cfg.port}" + else + "${cfg.bindAddress}:${toString cfg.port}"; + + # The /64 ZeroTier prefix of this fleet's mesh, mirroring + # modules/nixos/cognee.nix. The no-public-bind assertion admits loopback + # or any address inside this prefix. + ztPrefix = "fddb:4344:343b:14b9:"; + isLoopbackOrMesh = + addr: addr == "127.0.0.1" || addr == "::1" || addr == "localhost" || lib.hasPrefix ztPrefix addr; + + # Typed options are rendered first, then `settings` last, so an operator + # can override any of them without a module edit — the escape hatch + # modules/nixos/cognee.nix:144-156 uses. Values are stringified here + # because the relay reads everything through `std::env::var`. + # + # Booleans are emitted as exactly "true"/"false". The relay has at least + # five distinct boolean dialects (`BUZZ_AUTO_MIGRATE` takes true/1/yes/on + # at main.rs:29-36; `Config::parse_bool` takes true/1/on plus + # false/0/off/""; many vars are a bare `== "true" || == "1"`; + # `BUZZ_GIT_CONFORMANCE_PROBE` is `!= "false"`; `BUZZ_HUDDLE_AUDIO_AVAILABLE` + # is `!(== "false" || == "0")`). Only the literal strings "true" and + # "false" are read identically by all five. + renderValue = + value: + if lib.isBool value then + (if value then "true" else "false") + else if lib.isInt value then + toString value + else + value; + + typedEnvironment = { + BUZZ_BIND_ADDR = bindTarget; + BUZZ_HEALTH_PORT = toString cfg.healthPort; + BUZZ_METRICS_PORT = toString cfg.metricsPort; + RELAY_URL = cfg.relayUrl; + DATABASE_URL = cfg.database.url; + REDIS_URL = cfg.redis.url; + BUZZ_S3_ENDPOINT = cfg.objectStore.endpoint; + BUZZ_S3_BUCKET = cfg.objectStore.bucket; + BUZZ_S3_ADDRESSING_STYLE = cfg.objectStore.addressingStyle; + BUZZ_AUTO_MIGRATE = if cfg.autoMigrate then "true" else "false"; + # Forced on. Upstream defaults this to false, and false is the branch + # that selects a hardcoded, published dev keypair (0000…0001) when no + # private key is supplied (main.rs:419-440). The identity generator + # below always supplies one, so this only converts a silent dev-key + # fallback into a loud startup failure. Overridable through `settings`, + # which renders last. + BUZZ_REQUIRE_AUTH_TOKEN = "true"; + # `create_dir_all`'d by the relay at startup (config.rs:414-432), and + # relative to CWD if left unset — the "./repos" default at + # config.rs:820-822 — which under systemd would be `/`. + BUZZ_GIT_REPO_PATH = "${cfg.stateDir}/git"; + } + // lib.optionalAttrs (cfg.objectStore.region != null) { + BUZZ_S3_REGION = cfg.objectStore.region; + } + // lib.optionalAttrs (cfg.adminHost != null) { BUZZ_ADMIN_HOST = cfg.adminHost; }; + + environment = typedEnvironment // (lib.mapAttrs (_: renderValue) cfg.settings); + + # Copied from modules/nixos/omnigraph.nix. AF_UNIX is retained because the + # relay can serve an additional listener on a Unix domain socket + # (`BUZZ_UDS_PATH`), and because Postgres over a local socket needs it. + hardening = { + CapabilityBoundingSet = [ "" ]; + DeviceAllow = ""; + DevicePolicy = "closed"; + LockPersonality = true; + MemoryDenyWriteExecute = true; + NoNewPrivileges = true; + PrivateDevices = true; + PrivateTmp = true; + PrivateUsers = true; + ProcSubset = "pid"; + ProtectClock = true; + ProtectControlGroups = true; + ProtectHome = true; + ProtectHostname = true; + ProtectKernelLogs = true; + ProtectKernelModules = true; + ProtectKernelTunables = true; + ProtectProc = "invisible"; + ProtectSystem = "strict"; + RemoveIPC = true; + RestrictAddressFamilies = [ + "AF_INET" + "AF_INET6" + "AF_UNIX" + ]; + RestrictNamespaces = true; + RestrictRealtime = true; + RestrictSUIDSGID = true; + SystemCallArchitectures = "native"; + SystemCallFilter = [ + "@system-service" + "~@resources" + "~@privileged" + ]; + UMask = "0077"; + }; + in + { + options.services.buzz-relay = { + # The typed set below is chosen on one rule: an option is typed if + # getting it wrong is either unrecoverable or silent. That yields three + # groups. + # + # Unrecoverable — `relayUrl` (the tenant key, signed into auth events + # and persisted as communities.host, so effectively immutable after + # first boot) and `database.*` (the relay's only durable store). + # + # Silently wrong — `bindAddress`/`port` (a public bind succeeds + # silently under ip_nonlocal_bind=1, so it needs a build-time gate, + # which needs a typed value to gate on), `objectStore.*` (dev defaults + # `buzz_dev`/`buzz_dev_secret` against localhost:9000 mean an + # unconfigured relay fails at the probe rather than at config parse), + # and `autoMigrate` (off by default, so a fresh database silently + # serves against an unmigrated schema). + # + # Interface contract — `healthPort`/`metricsPort` (both bind 0.0.0.0 + # unconditionally, so the firewall must know them), `adminHost` (the + # admin surface is inert until set, so its presence is a security + # decision), `stateDir`, and `openFirewall`. + # + # Everything else — the ~90 remaining variables: rate limits, media size + # caps, pool sizes, background-task intervals, mesh, push, join policy, + # OTEL — is reachable through `settings`, which renders last and can + # override any of the above. + + enable = lib.mkEnableOption '' + the Buzz relay. + + Off by default and not enabled on any host. See the header comment in + modules/nixos/buzz-relay.nix for the object-store, Redis and backup + prerequisites that must land before this is switched on + ''; + + package = lib.mkPackageOption pkgs "buzz-relay" { }; + + relayUrl = lib.mkOption { + type = lib.types.str; + example = "wss://buzz.example.net"; + description = '' + Public WebSocket URL clients reach this relay on. Required: there is + deliberately no default. + + This is the single most consequential setting in the module, and it + is effectively immutable after first boot. The relay derives the + deployment community's host from it and seeds it into Postgres as + `communities.host`, and that host is then embedded in every signed + NIP-42 and NIP-98 auth event. Changing it later does not rename a + deployment; it orphans the existing community row and invalidates + the signed history against it. + + The scheme is load-bearing too, not cosmetic: it drives the + expected-URL reconstruction both auth schemes verify against, so a + TLS-terminated deployment must say `wss://`, never `ws://`. + + Upstream's default is `ws://localhost:3000`, which is why this + option refuses to default — inheriting that default would produce a + relay that starts, serves, and writes a permanent `localhost` + tenant row. + ''; + }; + + bindAddress = lib.mkOption { + type = lib.types.str; + default = "127.0.0.1"; + description = '' + Address the application listener (WebSocket + REST) binds. An IPv6 + literal is bracketed automatically when `BUZZ_BIND_ADDR` is + assembled. + + Constrained by an assertion to loopback or the ZeroTier mesh prefix. + Upstream defaults to `0.0.0.0`, which this module does not inherit. + ''; + }; + + port = lib.mkOption { + type = lib.types.port; + default = 3000; + description = "Port the application listener binds."; + }; + + healthPort = lib.mkOption { + type = lib.types.port; + default = 8080; + description = '' + Port of the separate health router, serving `/_liveness`, + `/_readiness` and `/_status`. + + Note this listener binds `0.0.0.0` unconditionally — the address is + hardcoded, not derived from + {option}`services.buzz-relay.bindAddress`. Containment is therefore + a firewall matter, which is why this port is typed: the mesh-scoped + firewall rule below has to name it. + ''; + }; + + metricsPort = lib.mkOption { + type = lib.types.port; + default = 9102; + description = '' + Port of the Prometheus `/metrics` listener. Like the health + listener, this binds `0.0.0.0` unconditionally regardless of + {option}`services.buzz-relay.bindAddress`. + ''; + }; + + adminHost = lib.mkOption { + type = lib.types.nullOr lib.types.str; + default = null; + example = "admin.buzz.example.net"; + description = '' + Authority (host, optionally `host:port`, with no scheme, path, `@` + or backslash) that serves the bundled admin SPA and the admin API. + + Left null the entire admin surface is inert and unrouted, which is + the default posture. Setting it is a security decision, not a + cosmetic one: it exposes the operator API, gated by NIP-98 + signatures from the configured operator pubkeys rather than by any + shared secret this module could hold. + ''; + }; + + autoMigrate = lib.mkOption { + type = lib.types.bool; + default = false; + description = '' + Whether to run the embedded sqlx migrations at startup + (`BUZZ_AUTO_MIGRATE`). + + Upstream defaults this off and then *continues starting anyway*, + logging only "Skipping database migrations because BUZZ_AUTO_MIGRATE + is not enabled". A fresh database therefore does not fail loudly; it + serves against a schema that does not exist yet and fails later, in + request paths. The alternative is running `buzz-admin migrate` out + of band before first start. + + Left off here to match upstream rather than to recommend it: turning + it on couples schema change to unit restart, which is a deployment + policy decision for whoever enables this relay. + ''; + }; + + stateDir = lib.mkOption { + type = lib.types.str; + default = "/var/lib/buzz-relay"; + description = '' + Writable state directory, created by systemd via `StateDirectory=`. + + Holds the git scratch tree (`BUZZ_GIT_REPO_PATH`, set to `git/` + beneath this) and its pack cache. Upstream is explicit that this + tree "need not be persistent or shared across replicas" — repository + name uniqueness lives in Postgres, not here — so it is a cache in + the durability sense even though it is stored under + `/var/lib`. Backups do not need to cover it; Postgres and the object + store are the authoritative state. + ''; + }; + + openFirewall = lib.mkOption { + type = lib.types.bool; + default = true; + description = '' + Whether to open the application, health and metrics ports on the + ZeroTier mesh interface (`zt+`) only, never globally. + + Health and metrics are included because both listeners bind + `0.0.0.0` regardless of + {option}`services.buzz-relay.bindAddress`, so the interface-scoped + rule is the only thing containing them. + ''; + }; + + database = { + url = lib.mkOption { + type = lib.types.str; + example = "postgres://buzz@localhost:5432/buzz"; + description = '' + libpq connection URL for the relay's PostgreSQL database + (`DATABASE_URL`). Required: there is deliberately no default, + because upstream's is `postgres://buzz:buzz_dev@localhost:5432/buzz` + — a real URL with a dev password that would be silently inherited. + + This module provisions no database. Postgres must already exist, + with the `buzz` role and database created and the `pgcrypto` + extension available, before the relay is enabled. Connecting is + fatal at startup with no retry: `Db::new` failing aborts the + process, which is why the unit below disables systemd's start + rate limiter. + + Do not put the password in this URL — it would land in the Nix + store, world-readable. The `buzz-relay-db-password` generator + below supplies it as `PGPASSWORD` through an `EnvironmentFile` + instead; a peer-authenticated local socket also works. + ''; + }; + + host = lib.mkOption { + type = lib.types.str; + default = "localhost"; + description = '' + Hostname of the PostgreSQL server, used to order the unit after a + local `postgresql.service` and to document the dependency. Purely + declarative — the relay itself reads only + {option}`services.buzz-relay.database.url`. + ''; + }; + }; + + redis = { + url = lib.mkOption { + type = lib.types.str; + example = "redis://localhost:6379"; + description = '' + Redis connection URL (`REDIS_URL`). Redis is a hard runtime + requirement, not an optional cache. + + This module provisions nothing: it neither enables + `services.redis` nor asserts that a local instance exists, because + the URL may legitimately point off-host. Required: there is + deliberately no default, because this fleet runs no Redis at all, + so an operator enabling the relay must supply a reachable one + rather than inherit a localhost address nothing is listening on. + + A password belongs in an `EnvironmentFile` override of `REDIS_URL` + rather than here, since this value reaches the Nix store. + ''; + }; + }; + + objectStore = { + endpoint = lib.mkOption { + type = lib.types.str; + example = "https://accountid.r2.cloudflarestorage.com"; + description = '' + Endpoint URL of the S3-compatible object store + (`BUZZ_S3_ENDPOINT`). Required: no default, because upstream's is + `http://localhost:9000` — a MinIO dev endpoint that an + unconfigured relay would silently address. + + The backend is deliberately pluggable rather than pinned to a + provider. Whatever is chosen must survive the git object-store + conformance probe the relay runs at boot, which races concurrent + writers to prove compare-and-swap semantics; eventual-consistency + stores fail it. + ''; + }; + + bucket = lib.mkOption { + type = lib.types.str; + example = "buzz-media"; + description = '' + Bucket holding media and git objects (`BUZZ_S3_BUCKET`). + Required: upstream defaults to `buzz-media`, which would be + inherited silently. + ''; + }; + + region = lib.mkOption { + type = lib.types.nullOr lib.types.str; + default = null; + example = "auto"; + description = '' + Value of `BUZZ_S3_REGION`. Left null the relay falls back to + `AWS_REGION` and then to `us-east-1`, which most non-AWS + S3-compatible stores reject. + ''; + }; + + addressingStyle = lib.mkOption { + type = lib.types.enum [ + "path" + "virtual" + ]; + default = "path"; + description = '' + Bucket addressing style (`BUZZ_S3_ADDRESSING_STYLE`). Path-style + matches MinIO and most self-hosted stores; AWS S3 proper and + several CDN-fronted providers want virtual-host style. + ''; + }; + }; + + settings = lib.mkOption { + type = lib.types.attrsOf ( + lib.types.oneOf [ + lib.types.str + lib.types.int + lib.types.bool + ] + ); + default = { }; + example = { + BUZZ_REQUIRE_AUTH_TOKEN = true; + BUZZ_REQUIRE_RELAY_MEMBERSHIP = true; + RELAY_OWNER_PUBKEY = "0000…"; + BUZZ_PUSH_GATEWAY_DELIVERY_URL = ""; + BUZZ_MAX_CONNECTIONS = 10000; + RUST_LOG = "buzz_relay=info"; + }; + description = '' + Free-form environment variables for the relay, rendered last into + the unit environment and therefore able to override any value the + typed options above produce. + + The relay is configured exclusively through environment variables — + no config file, no CLI flags, roughly 110 variables in total. The + typed options cover the dozen or so that are unrecoverable or + silently wrong when misconfigured; everything else (rate limits, + media size caps, pool sizes, background-task intervals, join policy, + mesh, push, OTEL) belongs here. + + Booleans are rendered as the literal strings `true` and `false`, + which is the only form every one of the relay's several boolean + parsers agrees on. Integers are rendered with `toString`. + + Nothing secret may go here: these values are rendered into the unit + file in the Nix store, which is world-readable. Secrets belong in an + `EnvironmentFile`. + + Two entries worth knowing about when this relay is eventually + switched on. `BUZZ_PUSH_GATEWAY_DELIVERY_URL` defaults to a live + third-party endpoint at `https://push.buzz.xyz`, so a self-hosted + deployment that does not want outbound calls there must set it to + the empty string. And `BUZZ_REPLICA_HEAD_MAX_AGE_SECS` is a renamed + variable whose mere presence is a hard startup error — setting it + here is a way to make the relay refuse to boot. + ''; + }; + }; + + config = lib.mkIf cfg.enable { + assertions = [ + { + assertion = cfg.relayUrl != ""; + message = '' + services.buzz-relay.relayUrl is empty. It is the relay's public + WebSocket URL, and the Host derived from it becomes the durable + tenant key: it is persisted as communities.host and signed into + every NIP-42/NIP-98 auth event, so it cannot be changed later + without orphaning that community and invalidating its signed + history. Choose it before first boot. + ''; + } + { + assertion = lib.hasPrefix "wss://" cfg.relayUrl || lib.hasPrefix "ws://" cfg.relayUrl; + message = '' + services.buzz-relay.relayUrl (${cfg.relayUrl}) must be a ws:// or + wss:// URL. Its scheme drives the expected-URL reconstruction that + NIP-42 and NIP-98 verify signatures against, so an http:// or + bare-host value does not merely look wrong, it fails + authentication at runtime. Any TLS-terminated deployment must say + wss://. + ''; + } + { + # Mirrors the cognee no-public-bind gate. Load-bearing because + # magnetite retains net.ipv6.ip_nonlocal_bind=1, under which + # binding an address the host does not hold succeeds silently + # instead of failing at startup. + assertion = isLoopbackOrMesh cfg.bindAddress; + message = '' + buzz-relay no-public-bind invariant violated: bindAddress must be + loopback (127.0.0.1/::1) or inside the ZeroTier prefix + ${ztPrefix}*/64. Resolved bindAddress = ${cfg.bindAddress}. + + A public bind would be silent under the retained + ip_nonlocal_bind=1 rather than failing at startup. Public reach + belongs behind the nginx reverse proxy, which is a separate + change from this one. + + Note that the health (${toString cfg.healthPort}) and metrics + (${toString cfg.metricsPort}) listeners bind 0.0.0.0 + unconditionally and cannot be constrained this way; the + mesh-scoped firewall rule is what contains them. + ''; + } + { + assertion = cfg.objectStore.endpoint != "" && cfg.objectStore.bucket != ""; + message = '' + services.buzz-relay.objectStore.endpoint and .bucket must both be + set. The relay hard-requires an S3-compatible object store — + media storage initialisation is fatal at startup and there is no + flag that disables media — and upstream's defaults are a MinIO + dev endpoint (http://localhost:9000, bucket buzz-media) that would + otherwise be inherited silently. + ''; + } + { + assertion = cfg.database.url != ""; + message = '' + services.buzz-relay.database.url is empty. This module provisions + no PostgreSQL: the database, the role and the pgcrypto extension + must already exist. Connecting is fatal at startup with no retry, + so an unreachable database means the unit fails rather than + degrades. + ''; + } + { + assertion = !(lib.hasInfix "buzz_dev" cfg.database.url); + message = '' + services.buzz-relay.database.url appears to contain upstream's dev + password (buzz_dev). Beyond being the published default, any + password written into this option is rendered into the Nix store + and is world-readable on the host. The password is supplied as + PGPASSWORD by the buzz-relay-db-password generator below; a + peer-authenticated local socket also works. + ''; + } + { + assertion = cfg.redis.url != ""; + message = '' + services.buzz-relay.redis.url is empty. Redis is a hard runtime + requirement of the relay, not an optional cache, and this module + provisions none — no services.redis is enabled anywhere in this + fleet. A reachable Redis must be supplied before the relay is + enabled. + ''; + } + ]; + + # Credential slots. Generator names are effectively immutable once + # minted: renaming one orphans the encrypted material committed under + # vars/per-machine///. They are therefore named + # `buzz-relay-`, matching the service name rather than the + # upstream variable name, so that a rename or re-prefix upstream does + # not strand this fleet's sops material. + + # Relay Nostr identity: a 32-byte secp256k1 secret key as 64 lowercase + # hex characters. Auto-generated because upstream NEVER generates or + # persists one. With the variable unset the relay either falls back to a + # hardcoded, published dev key (0000…0001, when BUZZ_REQUIRE_AUTH_TOKEN + # is false) or panics outright (when it is true) — there is no keyfile + # path option and no random-and-save branch, despite a stale doc comment + # upstream claiming "a fresh keypair is generated at startup". + # + # The key must be stable across restarts: it signs the NIP-43 events + # that membership mode verifies, and a rotated key makes every + # previously signed event unverifiable. `openssl rand -hex 32` follows + # the niks3-api-token / sso-cookie-secret pattern, and is uniform over + # the secp256k1 key space for all practical purposes. + clan.core.vars.generators.buzz-relay-identity = { + files."env" = { + restartUnits = [ "buzz-relay.service" ]; + }; + runtimeInputs = [ pkgs.openssl ]; + script = '' + printf 'BUZZ_RELAY_PRIVATE_KEY=%s\n' "$(openssl rand -hex 32)" > "$out/env" + ''; + }; + + # Git hook HMAC secret. Auto-generated to fix a fail-silent upstream + # default: with the variable unset the relay mints a random 32-byte + # secret on every boot (`rand::random()` then hex-encoded) and logs + # nothing at all about having done so — unlike the relay keypair, which + # at least warns. Every restart therefore silently invalidates + # outstanding hook signatures. + # + # Upstream validates a *supplied* value at >= 32 characters. 32 random + # bytes rendered as 64 hex characters clears that and matches the length + # of the value upstream generates for itself. + clan.core.vars.generators.buzz-relay-git-hook-hmac = { + files."env" = { + restartUnits = [ "buzz-relay.service" ]; + }; + runtimeInputs = [ pkgs.openssl ]; + script = '' + printf 'BUZZ_GIT_HOOK_HMAC_SECRET=%s\n' "$(openssl rand -hex 32)" > "$out/env" + ''; + }; + + # PostgreSQL role password, in cognee-db-password's dual shape: one + # generated value emitted twice. The `password` file exists so that the + # later provisioning change can set the role password from it (an ALTER + # ROLE in a postgresql-setup ExecStartPost, owned by whichever change + # actually creates the database); the `env` fragment is what the relay + # reads, and it has no *_FILE variant, so its copy must be a KEY=value + # line. + # + # PGPASSWORD rather than an interpolated DATABASE_URL: sqlx populates + # the password from it when the URL does not carry one, and it keeps the + # password out of the store-resident URL entirely instead of requiring + # the URL to be assembled at runtime. + # + # restartUnits names only buzz-relay.service: this module creates no + # postgresql.service, so naming one here would be a restart request for + # a unit it does not own. + clan.core.vars.generators.buzz-relay-db-password = { + files."password" = { + owner = "postgres"; + restartUnits = [ "buzz-relay.service" ]; + }; + files."env" = { + restartUnits = [ "buzz-relay.service" ]; + }; + runtimeInputs = [ pkgs.openssl ]; + script = '' + password=$(openssl rand -hex 32) + printf '%s' "$password" > "$out/password" + printf 'PGPASSWORD=%s\n' "$password" > "$out/env" + ''; + }; + + # Object store credentials, following the omnigraph-r2 prompt pattern + # (modules/machines/nixos/magnetite/default.nix:264-302): these are + # minted in a provider's dashboard, not derivable on the host, so the + # generator prompts for them once rather than inventing a value. + # + # Emitted as a single `env` file rather than separate + # access-key/secret-key files because the relay has no *_FILE variants + # and reads both only from the process environment. That `env` file is + # wired into EnvironmentFile below, so this prompt is the only place an + # operator supplies these — there is deliberately no second, manual + # credentials-path option competing with it. + clan.core.vars.generators.buzz-relay-object-store = { + prompts.access-key = { + description = '' + Access key ID for the S3-compatible object store backing the buzz + relay's media and git objects, scoped read/write on the relay's + bucket. + ''; + type = "hidden"; + persist = true; + display = { + group = "buzz-relay"; + label = "BUZZ_S3_ACCESS_KEY"; + }; + }; + + prompts.secret-key = { + description = "Secret access key paired with the access key ID above."; + type = "hidden"; + persist = true; + display = { + group = "buzz-relay"; + label = "BUZZ_S3_SECRET_KEY"; + }; + }; + + files.access-key.deploy = false; + files.secret-key.deploy = false; + + files."env" = { + restartUnits = [ "buzz-relay.service" ]; + }; + + script = '' + { + printf 'BUZZ_S3_ACCESS_KEY=%s\n' "$(cat "$prompts/access-key")" + printf 'BUZZ_S3_SECRET_KEY=%s\n' "$(cat "$prompts/secret-key")" + } > "$out/env" + ''; + }; + + systemd.services.buzz-relay = { + description = "Buzz relay"; + wantedBy = [ "multi-user.target" ]; + after = [ + "network-online.target" + ] + ++ lib.optional ( + cfg.database.host == "localhost" || cfg.database.host == "127.0.0.1" + ) "postgresql.service"; + wants = [ "network-online.target" ]; + + inherit environment; + + # The relay shells out to the git binary in roughly nineteen places + # across its CAS-publish, hydrate and transport paths, and resolves it + # by name. Without git on PATH those paths fail at runtime rather than + # at startup. + path = [ pkgs.git ]; + + serviceConfig = hardening // { + # `exec`, not `notify`: the relay implements no sd_notify handshake + # (nothing in the workspace links libsystemd or reads NOTIFY_SOCKET) + # and takes no arguments. `exec` at least defers "started" until + # after a successful execve, which `simple` does not. + Type = "exec"; + ExecStart = lib.getExe' cfg.package "buzz-relay"; + + # Every secret the relay needs arrives here, and each one comes from + # a clan-vars generator rather than an operator-managed path: the + # sops-backed lane is the only one that keeps this material out of + # the Nix store and out of a committed file. + EnvironmentFile = [ + config.clan.core.vars.generators.buzz-relay-identity.files."env".path + config.clan.core.vars.generators.buzz-relay-git-hook-hmac.files."env".path + config.clan.core.vars.generators.buzz-relay-db-password.files."env".path + config.clan.core.vars.generators.buzz-relay-object-store.files."env".path + ]; + + StateDirectory = "buzz-relay"; + WorkingDirectory = cfg.stateDir; + + # No LoadCredential here, deliberately. It is this fleet's usual + # pairing with DynamicUser — staging happens root-side before + # privilege drop, so a credential's ownership does not constrain the + # service — but the relay cannot consume one: it reads no *_FILE + # variant for any setting and never looks at CREDENTIALS_DIRECTORY. + # Every secret therefore arrives through EnvironmentFile above, + # which systemd also reads as root before the privilege drop, so the + # DynamicUser-safety property is preserved. EnvironmentFile is read + # once at unit start, which is why each generator above names its + # restartUnits. + + DynamicUser = true; + Restart = "on-failure"; + + # Postgres being unreachable at startup is fatal with no retry: the + # relay aborts rather than backing off. Combined with systemd's + # default 5-restarts-in-10s cap that turns a slow database into a + # permanently failed unit needing manual reset-failed — exactly the + # 2026-05-22 ENOSPC incident, where postgres PANICked and gitea was + # marked permanently failed. The fix is the same one applied to + # gitea then: disable the rate cap, lengthen the backoff, let the + # relay keep retrying while postgres recovers. Startup is idempotent. + RestartSec = "30s"; + + # Above the relay's 35s worst-case teardown, measured from SIGTERM: + # a fixed 5s grace during which readiness returns 503 and no + # listener has closed yet, then a 30s hard drain that force-exits + # with status 1 if exceeded. A TimeoutStopSec at or below 35s would + # race that self-imposed budget and turn an orderly drain into a + # SIGKILL. The margin covers the exit path after the drain returns. + TimeoutStopSec = "45s"; + }; + + unitConfig = { + StartLimitIntervalSec = 0; + }; + }; + + # Mesh-scoped, never global — the fleet precedent (cognee, mosh). The + # health and metrics listeners are included because both bind 0.0.0.0 + # unconditionally, independent of bindAddress, so this rule is the only + # thing containing them. + networking.firewall.interfaces."zt+".allowedTCPPorts = lib.mkIf cfg.openFirewall [ + cfg.port + cfg.healthPort + cfg.metricsPort + ]; + }; + }; +} diff --git a/openspec/changes/buzz-relay-module/design.md b/openspec/changes/buzz-relay-module/design.md new file mode 100644 index 000000000..fbe0baf90 --- /dev/null +++ b/openspec/changes/buzz-relay-module/design.md @@ -0,0 +1,347 @@ +## Context + +The Buzz relay is the server half of the Buzz communications platform, whose client half this fleet already packages and consumes. +A 2026-08-04 working note (`docs/notes/development/buzz/self-hosting.md`) assembled a synthesis and a refutation into a single record and left the object-storage question as the gate on everything else. +The binding human decision of 2026-08-19 (`decision-deploy-scope`) fixed the scope of this change: land the packaging derivation, the NixOS module, and the full OpenSpec change reviewed and buildable, with the module's `enable` defaulting to false and magnetite NOT enabled, and keep the operational go/no-go as a separate later change. + +Confirmed facts, verified at upstream `6e5c462ac524de60d7edb46c66130fd779cc9006` (tag `relay-v0.2.1`, committed 2026-08-08), load-bearing and not open questions: + +- The relay hard-requires an S3-compatible object store. + Media storage initialisation is fatal at startup (`main.rs:444-448`: `config.media.validate()` then `MediaStorage::new`, both `?`-propagating) and no feature flag disables media. +- The git object store additionally runs a four-phase conformance probe at boot (`main.rs:496-521`), default on, which races 32 concurrent conditional PUTs against a *single* key per phase (`store.rs:591`/`644` phase 2; `store.rs:731`/`738` phase 3) dispatched via `join_all`. + A failing probe propagates with `?` and the relay refuses to start. +- Redis is a hard runtime requirement (`config.rs:515`), not an optional cache. + This fleet has never run Redis as a daemon class: there is no precedent, no hardening baseline, and no backup story for it here. +- There is no backup system on this fleet at all. + magnetite has local ZFS snapshots and explicitly no off-site replication, while the relay's Postgres would hold community, membership, and signed-auth history. +- The Host derived from `RELAY_URL` is the durable tenant key. + It is persisted as `communities.host` in Postgres and signed into every NIP-42 and NIP-98 auth event, so it cannot be renamed later without orphaning the community row and invalidating the signed history against it. +- The relay releases on its own `relay-v*` tag line with its own crate version, explicitly not inheriting the workspace version (`crates/buzz-relay/Cargo.toml`), while the client packages this fleet already ships track `desktop-v*`. +- The relay exposes no `*_FILE` configuration variant anywhere, so every secret must arrive as a `KEY=value` line through `EnvironmentFile`. +- `web/` and `admin-web/` are TypeScript sources, not prebuilt assets: 65 and 14 files respectively, both Vite/React apps whose `build` script is `tsc && vite build`, with a 7118-line `pnpm-lock.yaml` and two pnpm patches. + +This design covers the disabled landing only. +The operational go/no-go — standing up the object store, introducing Redis, establishing backups, choosing the tenant hostname, opening the vhost — is a separate later change that inherits the prerequisites stated here. + +## Goals / Non-Goals + +**Goals:** + +Deliver two package derivations for the relay on a `relay-v*` source pin held separate from the shared `desktop-v*` pin, so a server upgrade is not coupled to a client upgrade. +Deliver a `flake.modules.nixos.buzz-relay` module whose full configuration surface and credential slots are reviewable, and which is inert because `enable` defaults to false and is set on no host. +Import that module on magnetite without enabling it, so the module stays inside the evaluated configuration and cannot silently rot, while the live host is unchanged. +Settle the object-storage question at the source with citations, in both directions, and record the finding as the input to the later go/no-go change. +State the enable path's three unmet prerequisites explicitly, so the later change inherits them rather than rediscovering them. +Assert at evaluation time on every option whose absence or wrong value would be silently wrong at runtime rather than loudly wrong at startup. + +**Non-Goals (out of scope):** + +Do not build or provision an object store, in any implementation, in this change. +Do not provision PostgreSQL or Redis on magnetite; model all three backing services as dependencies and options only. +Do not implement backups; carry the prerequisite explicitly instead. +Do not spend a vhost, a DNS record, or a tenant hostname, because those are the irreversible naming choices reserved for the change that turns the relay on. +Do not enable the relay on any host, and do not set `services.buzz-relay.enable = true` anywhere. +Do not package the `web` and `admin-web` bundles, which would add a second JavaScript dependency closure and a second hash-churn surface to every version bump. + +## Decisions + +### D1: the object store question is settled at source, and R2 fails on both the startup probe and steady state + +The binding decision required R2 fitness to be established at the source rather than inherited from the 2026-08-04 note as a premise, and named the finding as one of the more valuable things this change can produce. +It was verified against upstream `6e5c462` and against live Cloudflare documentation fetched on 2026-08-19. + +**Verdict: R2 fails, and it fails twice over.** + +**(i) It fails the startup probe, deterministically, at defaults.** +The probe fires 32 concurrent conditional PUTs at *one* key per race phase (`store.rs:591` and `644` for phase 2's `If-Match` race on `probe/pointer-`; `store.rs:731` and `738` for phase 3's `If-None-Match: *` race on one content-addressed key), dispatched simultaneously via `futures_util::future::join_all` (`store.rs:649`, `742`). +At defaults that is 3 rounds × 32 racers × 2 race phases = **192** concurrent single-key conditional PUTs per boot. +R2 documents, verbatim: "Maximum concurrent writes to the same object name (key) | 1 per second", with footnote 5 stating "Concurrent writes to the same object name (key) at a higher rate return HTTP 429 (rate limited) responses" (). +A second Cloudflare page corroborates it: error code 10058 `TooManyRequests` 429, "Rate limit exceeded. Often caused by multiple concurrent requests to the same object key (limit: 1 write/second per key)" (). +The relay's classifier has no 429 arm: `classify_cas` maps only `HttpFailWithBody(412, _)` to `LostRace` (`store.rs:546`), the transport-drop escape hatch matches only `S3Error::Reqwest | Http | Io`, and a 429 therefore falls to the catch-all at `store.rs:686` in phase 2 and `store.rs:781` in phase 3, producing `ProbeFailure` and a `?` at `main.rs:521` that refuses the boot. + +**(ii) It is unsafe in steady state, and this is the disqualifying finding, because it survives disabling the probe.** +The pointer key is the *sole* writer-serialization primitive for a repository: `cas_publish.rs:61-63` states outright that there is no advisory lock and that writer serialization is the CAS, and that adding a per-repo mutex would hide the exact contention the design's `Inv_NoFork` proves safe. +Every push to a repository writes that same pointer key, so two pushes within one second earn a 429 by the citation above. +`classify_cas` is shared by probe and production and turns a 429 into `StoreError::Backend` rather than `Conflict`, and `finalize_push` maps only `Conflict` to a 409 while routing everything else to the catch-all `(StatusCode::INTERNAL_SERVER_ERROR, "git error")` (`transport.rs:851`ff). +The net effect is that concurrent pushes to one repository return an opaque 500 instead of the `409 non-fast-forward` that tells the client's git to pull and rebase, intermittently and load-dependently. + +**The off-switch exists, and using it is worse than the failure it hides.** +`BUZZ_GIT_CONFORMANCE_PROBE=false` skips the probe entirely (`main.rs:496-499`, default on, any value other than the exact string `"false"` leaves it on), and upstream uses it themselves at `crates/buzz-test-client/tests/nip42_host_binding_live.rs:15`. +Disabling removes only the *admission check*, never the *dependency*, so on R2 it converts a loud startup failure into the silent, load-dependent 500 described above. +That is a correctness hazard rather than an inconvenience, and it is the strictly worse failure mode. + +**Lowering the race width risks a false pass, which is worse than failing.** +The minimum is 2 (`store.rs:578`), and two concurrent writes to one key is already twice the documented ceiling. +At width 2 the probe becomes a coin-flip that may pass when the two writes happen to straddle a second boundary, producing a *false admission* that hides the steady-state defect underneath. +`BUZZ_GIT_PROBE_WRITERS=2` is therefore not a workaround and recommending it would be harmful. + +**Recommendation: Garage, single-node, on magnetite, with MinIO as the known-good fallback.** +The binding requirement is narrow and unusually demanding: `PutObject` honouring `If-Match`/`If-None-Match` with linearizable CAS, returning 412 on loss and an `ETag` on the winning response that chains into the next `If-Match`. +Multipart, checksums, and listing are never exercised by the probe. +Garage is preferred on fleet fit: roughly 100–200 MB RSS against seven co-tenant services on a host with a documented 2026-06-10 disk-starvation incident and a 250 GiB `/nix` quota, plain-file storage that backs up by `zfs snapshot` and `zfs send` with no cluster-aware dance, and real nixpkgs packaging (`pkgs.garage`, `services.garage`). +Ceph is disqualified on that same incident (multi-daemon cluster, GBs of RAM, wants raw devices) and SeaweedFS on the requirement that matters, since documented conditional-PUT CAS could not be established for it. + +**Recorded honestly: Garage's conditional-PUT CAS was not doc-verified in this pass.** +That claim rests on prior knowledge and is the thinnest evidence in the finding; Garage's documentation was not fetched. +Upstream has empirically verified **MinIO**, not Garage (`store.rs:14-15`, `store.rs:1174-1188`), which makes MinIO the lowest-conformance-risk choice and a defensible answer if the operator's priority is certainty. +Since the probe *is* the decision procedure and is cheap to run, the recommendation is to stand up Garage, point the relay at it, and let the probe adjudicate, falling back to MinIO if it fails. +The probe, not this document, is the final authority. + +**This verification corrected three claims of the 2026-08-04 spike.** + +1. **It missed the off-switch.** + The note judged correctly that disabling the probe hides the problem rather than fixing it, but it never named `BUZZ_GIT_CONFORMANCE_PROBE`, so an operator reading it would not know the flag exists. +2. **It asserted the 429 mechanism without checking the client layer.** + The client is `rust-s3` v0.37.2, not `aws-sdk-s3`, built with `fail-on-err` (`crates/buzz-relay/Cargo.toml:65`), so every non-2xx becomes `Err(HttpFailWithBody(..))` (`tokio_backend.rs:111-117`), and PUTs *are* wrapped in a `retry!` macro (`tokio_backend.rs:171`, `bucket.rs:2348-2363`). + The retry does not rescue R2 for three reasons: the budget is exactly 1 because `RETRIES` defaults to `AtomicU8::new(1)` (`lib.rs:38`) and `set_retries` is never called anywhere in the buzz workspace; the retry is status-blind, matching `Err(e)` with no inspection; and it retries 412 too, so every losing CAS racer wastes a second and issues a duplicate write on *every* backend. + One 1-second retry cannot absorb 32 writers against a 1-write/sec ceiling, so the verdict stands — but the note reached it without checking. +3. **It overstated R2's consistency weakness.** + The note framed R2 as last-write-wins and implied weak consistency. + R2 in fact documents itself as "strongly consistent", with read-after-write "Strongly consistent: readers will immediately see the latest object globally" and object listing "Strongly consistent" (). + The last-write-wins line is real but describes *unconditional* concurrent PUTs, which is S3's own behaviour and not in itself a defect. + R2 would likely *pass* the probe's phases 1, 3, and 4 on consistency grounds. + The real gap is **silence on CAS atomicity, not documented weakness**: Cloudflare's error-codes page links to a `#conditional-operations-in-putobject` anchor that does not exist on the extensions page, and the only atomicity language there is a disclaimer about `cf-copy-destination-if-*` not being atomic relative to `x-amz-copy-source-if-*`. + Cloudflare is willing to call out non-atomicity where it applies, which makes the silence on PutObject-CAS a genuine gap rather than a charitable oversight. + The corrected reading is that **R2 fails on throttling, and separately lacks a documented CAS-atomicity guarantee** — it is not "eventually consistent" in the way the note's framing suggested. + +Nothing relevant changed upstream or at Cloudflare since 2026-08-04: `store.rs` is byte-identical to its only prior commit (2026-08-03), and a grep for rate-limit, 429, conditional-write, and consistency terms across the full R2 changelog returns zero matches after that date. + +### D2: the module ships disabled, and the three unmet prerequisites are named as the enable path's inheritance + +`services.buzz-relay.enable` is `mkEnableOption` defaulting to false, every unit, firewall opening and generator sits inside `lib.mkIf cfg.enable`, and no host sets it. +The binding decision is the proximate reason, but the substantive reason is that three operational prerequisites are unmet, and each is its own change. + +1. **Object storage.** + The relay hard-requires an S3-compatible store, media storage initialisation is fatal at startup with no flag to disable media, and the git store's boot probe raises the bar above "speaks S3" to "passes the conformance probe" (D1). + Which bucket, on which provider, is undecided, and D1 rules out the provider the decision preferred. +2. **Redis as a new daemon class.** + Redis is a hard runtime requirement (`config.rs:515`), and this fleet has never run it: no precedent, no hardening baseline, no backup story. + Adopting a new daemon class is a fleet-level decision, not a side effect of enabling a service. +3. **Backups, which do not exist fleet-wide.** + There is no backup system on this fleet at all — magnetite has local ZFS snapshots and explicitly no off-site replication. + The relay's Postgres would hold community, membership, and signed-auth history, which is exactly the class of state that must not live on one disk. + +Rationale: shipping the surface disabled makes the configuration and the credential slots reviewable before anything is switched on, which is precisely what a change that cannot yet meet its prerequisites should deliver. +Naming the three prerequisites here is a direct requirement of the binding decision, so the later go/no-go change inherits them rather than rediscovering them. +Alternatives considered: enabling on magnetite (rejected — the binding decision forbids it, and all three prerequisites are unmet); provisioning Postgres, Redis and a bucket here (rejected — the binding decision forbids provisioning new daemons in this change, and D1 shows the object-store choice is not settled); not landing the module at all until the prerequisites are met (rejected — the decision explicitly wants the surface reviewed and buildable now, and an unlanded module cannot be reviewed). + +### D3: no vhost, no DNS record, and no tenant hostname are spent in this change + +This change opens no nginx vhost, adds no Cloudflare DNS record, and chooses no hostname. +`relayUrl` is a typed option with **no default**, required at enable time. + +The reason is that the Host derived from `RELAY_URL` is the durable tenant key, not a display name. +The relay derives the deployment community's host from it and seeds it into Postgres as `communities.host`, and that host is then embedded in every signed NIP-42 and NIP-98 auth event. +Changing it later does not rename a deployment; it orphans the existing community row and invalidates the signed history against it. +The binding decision states this consequence directly and reserves the irreversible naming choice for the change that actually turns the relay on, adding that a placeholder must be obviously a placeholder and the option mandatory at enable time rather than defaulted. +There is no placeholder here at all, which is the strongest form of that instruction: `relayUrl` has no default, so an enabled configuration that omits it fails at the option layer. + +The scheme is load-bearing too, not cosmetic: it drives the expected-URL reconstruction that both auth schemes verify against, so a TLS-terminated deployment must say `wss://` and an `https://` value fails authentication at *runtime* rather than at parse. +An assertion therefore constrains `relayUrl` to a `ws://` or `wss://` prefix, and a second asserts it is non-empty with the tenant-key immutability spelled out in the message. +Refusing upstream's `ws://localhost:3000` default is part of the same decision: inheriting it would produce a relay that starts, serves, and writes a permanent `localhost` tenant row. + +Rationale: the one setting that cannot be corrected later is the one this change most carefully declines to choose. +Alternatives considered: defaulting `relayUrl` to `wss://buzz.scientistexperience.net` (rejected — that is the irreversible choice the decision reserves, and a default is exactly how it would get made by accident); defaulting to an obvious placeholder such as `wss://placeholder.invalid` (rejected — it would satisfy the assertions and let an enabled configuration evaluate, converting a build-time refusal into a runtime tenant-row mistake); opening the vhost now and enabling later (rejected — the vhost's server name *is* the hostname choice). + +### D4: a separate `relay-v*` source pin rather than repinning the shared `desktop-v*` source + +`pkgs/by-name/buzz/relay-source` is a second, independent pin of the same upstream repository as the existing `pkgs/by-name/buzz/source`. +`source` tracks `desktop-v*` and feeds the desktop and home-side CLI and git helpers; `relay-source` tracks `relay-v*` and feeds the server. + +The separation is deliberate rather than incidental. +A single shared pin would couple a server upgrade to a client upgrade: bumping the relay to pick up a server fix would simultaneously move the CLI and the credential helper that the desktop configuration installs on every machine, and bumping the desktop train would silently redeploy the relay. +The binding decision took this recommendation directly, instructing that the shared buzz source the desktop configuration consumes must not be repinned. +The cost is a second `fetchCargoVendor` of the same roughly 1000-package workspace lockfile; the benefit is that the two upgrade decisions stay independent. + +Unlike `source`, the relay's `version` is a real crate version rather than a release-train number: `crates/buzz-relay/Cargo.toml` declares `version = "0.2.1"` with an explicit comment that it does not inherit the frozen workspace `0.1.0`, because the relay ships as a pinnable artifact released on its own cadence. +The tag, the derivation attribute, and the crate version therefore all agree, which is not true of the client packages. + +One forced deviation is recorded here because it will otherwise look like carelessness. +Upstream publishes **no GitHub Release object for `relay-v*` tags** — `gh api repos/block/buzz/releases/tags/relay-v0.2.1` returns 404, and the full releases listing contains `desktop-v*`, `mobile-v*`, `chart-v*` and bare `v0.x.y` entries but zero `relay-v*` entries. +`source/update.sh` filters the *releases* API, so a verbatim copy with the prefix swapped could never bump the pin and would always take its loud-failure path. +`relay-source/update.sh` therefore reads `GET /repos/block/buzz/git/matching-refs/tags/relay-v` instead, with prerelease exclusion done lexically because a bare tag carries no `draft`/`prerelease` flag, and with an assertion that the ref object is a commit rather than an annotated tag. + +Rationale: independent upgrade decisions are worth one duplicated vendor fetch, and the decision explicitly required it. +Alternatives considered: repinning `source` to `relay-v0.2.1` (rejected — couples server and client upgrades in both directions and would move the desktop CLI on every relay fix); a single pin with per-package `rev` overrides (rejected — the vendored dependency set is per-pin, so this is the same duplicate fetch with less clarity). + +### D5: the `web` and `admin-web` bundles are deliberately unset, so there is no admin UI and no invite landing page + +`BUZZ_WEB_DIR` and `BUZZ_ADMIN_WEB_DIR` are not set by the package and not set by the module. + +They are not prebuilt static assets. +At `relay-v0.2.1`, `web/` is 65 files of `.tsx`/`.ts`/config and `admin-web/` is 14, with no `dist/` and no built `index.html`; both are Vite/React apps whose `build` script is `tsc && vite build`. +Upstream's container sets the two variables to `/srv/buzz/web` and `/srv/buzz/admin-web` (`Dockerfile:151-152`), but only after `Dockerfile:117-118` runs `pnpm install --frozen-lockfile` and then `pnpm -C web build && pnpm -C admin-web build`. +Packaging them would mean vendoring a second, JavaScript dependency closure — a 7118-line `pnpm-lock.yaml` plus two pnpm patches — alongside the cargo one, and adding a second hash-churn surface to every version bump. + +**The asymmetry is the sharp part and is why "optional" is the wrong word.** +Setting either variable to a directory that lacks `index.html` is a **hard startup failure**: `Config::from_env` returns `ConfigError::InvalidValue` (`config.rs:968-975`, and `:947-955` for the admin path) and the relay refuses to start. +Leaving them **unset** is safe and is upstream's own source-tree default (`TESTING.md:285` lists `BUZZ_WEB_DIR` as "unset (source)"), leaving `web_dir = None` (`config.rs:960-964`) with the relay running normally. +`BUZZ_ADMIN_WEB_DIR` is read only when `BUZZ_ADMIN_HOST` is set (`config.rs:930-942`), so with that host unset the admin surface is absent entirely and the web dir is never consulted. +A wrapper that "helpfully" pointed at a plausible path would therefore convert a working relay into one that cannot boot, which is why unset is the only safe value until the bundles are actually built. + +**What that costs, stated plainly:** no read-only admin dashboard, no bundled invite landing page at `/invite/{code}`, and no optional git repository browser (`BUZZ_SERVE_GIT_WEB_GUI`). +**What is unaffected:** the WebSocket relay, the REST surface, git push and pull over HTTP, and NIP-42/NIP-98 authentication. + +This gap is recorded as a known gap rather than a defect, and the package's install check pins the semantics it depends on by asserting that a `BUZZ_WEB_DIR` without `index.html` is fatal. +Rationale: the JavaScript closure is a real cost with a real maintenance tail, the relay's core surfaces do not need it, and the failure mode of getting it wrong is a relay that will not start. +Alternatives considered: vendoring the pnpm closure now (rejected — out of scope, and it doubles the hash surface of every bump for a surface nothing yet consumes); setting the variables to a path the module creates empty (rejected — an empty directory has no `index.html`, so this is exactly the hard startup failure); leaving them settable by the operator (the `settings` escape hatch already permits this, with the failure mode documented). + +### D6: the typed option surface, a `settings` escape hatch rendered last, and seven assertions + +An option is typed if getting it wrong is either unrecoverable or silent; everything else routes through a free-form `settings` attribute set rendered *last*, so it can override any typed value without a module edit. +That is the escape-hatch pattern `modules/nixos/cognee.nix` already uses, and it keeps roughly 90 further environment variables reachable without enumerating them. + +`settings` renders booleans as the literal strings `"true"`/`"false"` because the relay has at least five distinct boolean dialects, and those two literals are the only forms all five read identically. + +No option in this module names a path to a secret, because secrets arrive from the clan-vars generators themselves (D9). + +The seven assertions each guard a failure that would otherwise be silent rather than loud: + +- `relayUrl` non-empty, and `relayUrl` matching `ws://` or `wss://` — the tenant-key and expected-URL-reconstruction reasons in D3. +- No public bind: `bindAddress` must be loopback or inside the fleet's ZeroTier `/64`. + This is load-bearing because magnetite retains `net.ipv6.ip_nonlocal_bind=1`, under which binding an address the host does not hold *succeeds silently* rather than failing at startup. + The message records that the health and metrics listeners bind `0.0.0.0` unconditionally regardless of `bindAddress`, so the mesh-scoped firewall rule is their only containment. +- `objectStore.endpoint` and `.bucket` both set — upstream's defaults are a live MinIO dev target (`http://localhost:9000`, bucket `buzz-media`) that an unconfigured relay would address without complaint. + Neither carries a default here, so omitting one fails at the option layer before its assertion is reached; the object store's credentials are not asserted on at all, because they are not an option — the generator supplies them (D9). +- `database.url` non-empty — this module provisions no PostgreSQL, and connecting is fatal at startup with no retry. +- `database.url` must not contain `buzz_dev` — this catches both the published default and the more general hazard that any password written into that option is rendered into the Nix store and is world-readable on the host. +- `redis.url` non-empty — Redis is a hard requirement and this fleet provisions none. + This option carries no default, so the requirement is genuinely reachable: an enabled configuration that omits it fails at the option layer, rather than silently inheriting a `redis://localhost:6379` that points at a Redis no host on this fleet runs. + +Rationale: the module cannot prevent a bad deployment, but it can convert every silent runtime wrongness into a build-time refusal, which is the only leverage a disabled module has. +Alternatives considered: typing all ~106 variables (rejected — unmaintainable, and most are numeric tunables whose wrong values are loud); typing none and using `settings` alone (rejected — the unrecoverable and silent classes are exactly what needs names and assertions); defaulting the object-store and database options to upstream's values (rejected — every one of those defaults is a live dev target that fails late and confusingly rather than early). + +### D7: four clan-vars credential slots, named after the service and declared before use + +Four generators are declared, all inside the `enable` guard, named `buzz-relay-` after the *service* rather than the upstream environment variable, because generator names are effectively immutable once minted — renaming one orphans the encrypted material committed under `vars/per-machine///` — and an upstream variable rename must not be able to strand this fleet's sops material. + +- `buzz-relay-identity` emits the relay's Nostr secret key. + Auto-generated because upstream **never generates or persists one**: with the variable unset the relay falls back to a hardcoded, published dev key or panics outright, despite a stale upstream doc comment claiming a fresh keypair is generated at startup. + The key must be stable across restarts because it signs the events membership mode verifies. +- `buzz-relay-git-hook-hmac` fixes a fail-silent upstream default: with the variable unset the relay mints a fresh random secret on every boot and logs nothing at all about having done so, so every restart silently invalidates outstanding hook signatures. +- `buzz-relay-db-password` follows the `cognee-db-password` dual shape, emitting one value twice — a bare `password` for whatever provisions the role, and an `env` fragment carrying `PGPASSWORD=` for the relay — which keeps the password out of the store-resident `DATABASE_URL` entirely. +- `buzz-relay-object-store` follows the niks3-s3 prompt pattern, because provider-minted credentials are not derivable on the host, and emits a single `env` file because the relay reads both keys only from the process environment. + +All four generators' `env` files are wired into the unit's `EnvironmentFile`, so every declared slot is one the relay actually reads (D9). +A generator whose output nothing consumes is worse than no generator, because it still prompts the operator and still commits encrypted material, while the service it was minted for never receives it. + +`LoadCredential` is deliberately **not** used, unlike the sso-gateway precedent. +The relay reads no `*_FILE` variant and never consults `CREDENTIALS_DIRECTORY`, so `LoadCredential` would be dead config; `EnvironmentFile` preserves the same DynamicUser-safety property because systemd reads it as root before privilege drop. + +Rationale: declaring the credential slots before the service is enabled is the other half of "reviewable surface" — the slots are the part a reviewer most needs to see, and the immutability of their names makes minting them a decision rather than a detail. +Alternatives considered: naming generators after the upstream variables (rejected — an upstream rename would strand sops material behind an immutable name); deferring the generators to the enable change (rejected — the names are the immutable part, so choosing them under review now is strictly better than choosing them under deployment pressure later); delivering secrets by `LoadCredential` (rejected — verified dead config for this binary). + +### D8: magnetite imports the module and does not enable it + +`modules/machines/nixos/magnetite/default.nix` gains the `buzz-relay` import alongside its neighbours, with a comment recording that `services.buzz-relay.enable` defaults to false and is not set there, and pointing at the module header for the prerequisites that gate switching it on. + +Importing without enabling keeps the module inside the evaluated configuration, so it is type-checked, formatted, and dead-code-checked on every evaluation and cannot silently rot between now and the go/no-go change. +Semantic inertness was verified rather than assumed: with the module imported and disabled, the configuration reports `enable = false`, no `buzz-relay` clan-vars generators, no `systemd.services.buzz-relay` unit, and an unchanged ZeroTier-scoped firewall port list. + +One investigation is recorded because it would otherwise read as a contradiction. +The magnetite `drvPath` *does* differ between the with-import and without-import evaluations, which would appear to contradict "changes nothing". +It was chased rather than accepted: `nix-diff` showed the only difference in the entire secrets manifest was the `sopsFile` store path for an unrelated age key — that is, `inputs.self`, the whole-flake source hash — with zero occurrences of "buzz" in either derivation, and a control experiment appending a newline to an unrelated docs file with *no module import at all* shifted the `drvPath` identically. +The shift is `inputs.self` source-hashing, which any file addition triggers, and is not attributable to the module. + +Rationale: an unimported module is unreviewed and unevaluated, while an imported and disabled one is fully checked and operationally inert, which is exactly the posture this change wants. +Alternatives considered: leaving the module unimported until the enable change (rejected — it would not be evaluated, so nothing would catch it rotting); importing and enabling with a placeholder configuration (rejected — D2 and D3 both forbid it, and the placeholder hostname is the irreversible choice). + +### D9: secrets reach the relay from the clan-vars generators themselves, not from an operator-managed file path + +The generators' `env` files are wired directly into the unit's `EnvironmentFile`, and the `objectStore.credentialsFile` option is deleted along with any `database.passwordFile` option. +There is no option anywhere in this module that names a path to a secret. + +The binding constraint is where secrets are allowed to live on this fleet. +Every secret travels the sops-backed clan-vars lane, encrypted at rest under `vars/per-machine///` and decrypted onto the host at activation, and it must never reach the Nix store or a file committed in this repository. +A hand-managed `credentialsFile` path cannot satisfy that constraint, because the file it names is by definition outside the lane: somebody has to put it there by hand, keep it there, and remember it exists. +So the generator was always the real mechanism and the path option was a second, weaker one standing beside it. + +Carrying both was worse than carrying either alone, and this was the shape review found. +`buzz-relay-object-store` prompts the operator for an access key and a secret key, encrypts them, and commits them — and then the unit never read that file, while an assertion separately demanded `objectStore.credentialsFile` be set. +An operator flipping `enable` would therefore be asked for the same credentials twice by two different mechanisms, and the one that looked most like the answer was the inert one. +`buzz-relay-db-password` had the same defect: it minted a password into an `env` file that nothing consumed. +Wiring the generator files in and deleting the options resolves both halves at once — the operator is prompted exactly once per secret, and the relay actually receives what they supplied. + +Rationale: a credential path option is not a neutral convenience when a sops-backed lane already exists, because it invites a secret to be managed outside the only mechanism that keeps it encrypted. +Deleting the option removes the duplication, removes the inert prompt, and removes the possibility of the weaker path being chosen. +Alternatives considered: keeping the path options as the real mechanism and deleting the two generators as premature (rejected — it would delete the sops lane and leave a hand-managed plaintext file as the only way to supply a secret, which is the wrong direction on the constraint that actually binds); keeping both and documenting the precedence (rejected — the duplication was itself the defect, and a comment cannot stop an operator from populating the inert one); leaving the generators unconsumed with a comment saying so (rejected — a prompt-bearing generator is not inert at enable time, so this would keep the double prompt). + +### D10: `BUZZ_REQUIRE_AUTH_TOKEN` is defaulted to true, departing from upstream + +The module sets `BUZZ_REQUIRE_AUTH_TOKEN=true` in its typed environment rather than inheriting upstream's default of false. +This is a deliberate departure from an upstream default, recorded here because departures from upstream defaults should be visible rather than discovered. + +The reason is that upstream's default selects a published hardcoded development key. +With `BUZZ_RELAY_PRIVATE_KEY` unset the relay falls back to a dev private key of `0000…0001` when `BUZZ_REQUIRE_AUTH_TOKEN` is false, and panics outright when it is true (`main.rs:419-440`). +Upstream's default therefore fails open: the unsafe configuration is the one that starts. +Defaulting the variable to true inverts that, so the same misconfiguration becomes a loud startup failure instead of a relay signing with a key that anybody can read from a public repository. +The identity generator (D7) supplies a real key, so on the configured path this default costs nothing and changes no behaviour; it only closes the failure mode that appears when the key is missing. + +Setting it in `typedEnvironment` rather than asserting on it is the stronger of the two options review offered, because an assertion can only reject a wrong value an operator wrote, whereas a default also covers the operator who wrote nothing. +The `settings` escape hatch still renders last, so an operator who genuinely needs the upstream behaviour can override it explicitly — which is the right shape for a security default: on unless deliberately turned off. + +Rationale: a security default that fails open is worth departing from upstream to close, and closing it by default rather than by assertion covers the silent case as well as the wrong-value case. +Alternatives considered: asserting on it instead of defaulting it (rejected — an assertion cannot fire on an operator who never set the variable, which is exactly the case upstream's default makes dangerous); leaving it at upstream's default and documenting the hazard (rejected — the module's whole posture is converting silent runtime wrongness into loud refusal, and this is the clearest instance of it); making it a typed option (rejected — it is a security invariant of this deployment rather than a knob, and `settings` already provides the escape hatch). + +## Risks / Trade-offs + +[Risk] R2 was the preferred backend and it fails, so the enable path now needs a self-hosted object store the fleet does not have. → Mitigation: the finding is recorded with citations in both directions (D1) and a fleet-fit recommendation is handed to the later go/no-go change rather than a store being built here. + +[Risk] Garage's conditional-PUT CAS conformance is documented but was not doc-verified in this pass, making it the thinnest claim in the finding. → Mitigation: recorded honestly as such (D1), with MinIO named as the known-good fallback because upstream verifies against it empirically, and with the cheap startup probe designated as the final adjudicator rather than either document. + +[Risk] An operator who reads only the R2 rate-limit line might reach for `BUZZ_GIT_CONFORMANCE_PROBE=false` or `BUZZ_GIT_PROBE_WRITERS=2`. → Mitigation: D1 records that the off-switch converts a loud startup failure into a silent correctness hazard, and that width 2 risks a false pass which is worse than failing. + +[Risk] A disabled module is easy to mistake for a working deployment on a later reading. → Mitigation: the module header, the `enable` option description, the magnetite import comment, and this design all state the disabled posture and the three prerequisites, and no host sets `enable`. + +[Risk] The tenant hostname is irreversible and a future author might default it for convenience. → Mitigation: `relayUrl` has no default at all and no placeholder, so an enabled configuration that omits it fails at the option layer (D3). + +[Risk] A public bind would succeed silently on magnetite under the retained `ip_nonlocal_bind=1`. → Mitigation: the no-public-bind assertion constrains `bindAddress` to loopback or the mesh prefix, and its message records that health and metrics bind `0.0.0.0` regardless so the firewall is their only containment (D6). + +[Trade-off] A second source pin duplicates a roughly 1000-package vendor fetch of the same lockfile. → Accepted as the cost of keeping server and client upgrade decisions independent, which the binding decision required (D4). + +[Trade-off] No admin UI and no invite landing page ship, because the `web` and `admin-web` bundles are unset. → Accepted rather than vendoring a second JavaScript dependency closure; the alternative of pointing the variables at a directory lacking `index.html` is a hard startup failure, while unset is upstream's own source-tree default (D5). + +[Trade-off] Redis would be a new daemon class on this fleet, with no precedent, hardening baseline, or backup story. → Not resolved here; carried explicitly as prerequisite 2 of the enable path (D2). + +[Trade-off] The relay has no `*_FILE` variants, so every secret must reach it as a `KEY=value` line through `EnvironmentFile` rather than `LoadCredential`. → Accepted; systemd reads `EnvironmentFile` as root before privilege drop, preserving the DynamicUser-safety property (D7). + +[Risk] A declared credential slot that nothing consumes is not inert at enable time: it prompts the operator, commits encrypted material, and still leaves the service without the secret. → Mitigation: every generator's `env` file is wired into the unit's `EnvironmentFile` and no path-to-a-secret option exists beside it, so each secret is asked for once and arrives where it was asked for (D9). + +[Trade-off] Forcing `BUZZ_REQUIRE_AUTH_TOKEN` on departs from an upstream default, so a deployment expecting upstream's behaviour will find this one stricter. → Accepted, because upstream's default fails open onto a published development key; the `settings` escape hatch renders last, so the upstream behaviour stays reachable by explicit intent rather than by silence (D10). + +## Migration Plan + +Land order: + +1. Add `pkgs/by-name/buzz/relay-source` pinned to `relay-v0.2.1` with its tag-refs-based `update.sh`, leaving the shared `source` derivation untouched (D4). +2. Add `pkgs/by-name/buzz/relay` building from that pin, with the compiled-in hook shebang rewritten to a store path, the six-tool runtime PATH prefix, and the config-validator install check (D5). +3. Author `modules/nixos/buzz-relay.nix` as `flake.modules.nixos.buzz-relay`: the sixteen typed options, the `settings` escape hatch rendered last, the seven assertions, the four clan-vars generator declarations wired into the unit's `EnvironmentFile`, and the hardened systemd unit, all behind `enable` defaulting to false (D2, D6, D7, D9). +4. Import the module on magnetite without enabling it, and verify semantic inertness at the configuration level rather than by `drvPath` (D8). +5. Record the object-storage finding with citations and its three corrections to the 2026-08-04 note (D1), and append a dated addendum to that note so its readers are not left with the superseded guidance. + +There is no deploy step, no `clan vars generate`, and no terranix apply in this change, because nothing is enabled and no secret material is needed by a module that emits no unit. + +Rollback: remove the magnetite import, the module file, and the two package directories. +No secret material exists to delete, because the generators are inside the `enable` guard and have never been realised; no DNS record, vhost, or ACME certificate exists to withdraw; no database, bucket, or Redis instance was created. +This is a substantially cheaper rollback than any enabled deployment would offer, which is part of the argument for landing disabled. + +Enable path, inherited by the later go/no-go change and stated here so it is not rediscovered: + +1. Choose and stand up an object store that passes the relay's conformance probe — Garage single-node recommended, MinIO the known-good fallback — and let the probe adjudicate (D1). +2. Decide whether Redis is accepted as a new daemon class on this fleet, and if so provision it with a hardening baseline and a backup story. +3. Establish a backup story for the relay's PostgreSQL, which requires a fleet-wide backup capability that does not currently exist in any form. +4. Choose the tenant hostname, understanding that it is persisted as `communities.host` and signed into every auth event and is therefore not a free rename, then set `relayUrl` with a `wss://` scheme (D3). +5. Open the nginx vhost and the Cloudflare DNS record for that hostname, correcting the body-size, proxy-timeout, and websocket-upgrade defaults, none of which suit a git transport that pushes large packs over long-lived upgraded connections. +6. Provision the PostgreSQL database, role, and extensions, which this module does not do. +7. Run `clan vars generate` so the four generators emit their material, and populate the object-store prompts with `clan vars set`. +8. Only then set `services.buzz-relay.enable = true`. + +## Open Questions + +Whether Garage's conditional-PUT CAS actually satisfies the probe's phases 2 and 4 is unresolved by documentation and is answered by running the probe against a real Garage instance, which is the first task of the go/no-go change (D1). + +Whether Redis is accepted as a new daemon class on this fleet is a fleet-level decision that this change deliberately does not make (D2). + +Whether the conformance probe should remain enabled in steady state, given that it couples relay startup to object-store reachability and spends 192 conditional writes per boot, is deferred until a backend that passes it exists; on is upstream's default and the honest posture (D1). + +Which hostname the relay would take, and whether that interacts with retiring Gitea's `git.` name, is reserved for the change that turns the relay on because the choice is irreversible (D3). diff --git a/openspec/changes/buzz-relay-module/proposal.md b/openspec/changes/buzz-relay-module/proposal.md new file mode 100644 index 000000000..2fa76951c --- /dev/null +++ b/openspec/changes/buzz-relay-module/proposal.md @@ -0,0 +1,71 @@ +## Why + +The Buzz relay is the server half of the Buzz communications platform, and this fleet already ships the client half: `buzz-source`, `buzz-cli`, `buzz-git-credential-nostr`, and `buzz-git-sign-nostr` are packaged and consumed by the desktop configuration. +Self-hosting the relay was investigated in a 2026-08-04 working note (`docs/notes/development/buzz/self-hosting.md`) which reached no verdict on the one question that gates everything: whether Cloudflare R2 can serve as the relay's object store. +The binding human decision of 2026-08-19 (`decision-deploy-scope`) resolved the scope question directly: land the packaging, the NixOS module, and the OpenSpec change reviewed and buildable, with the module's `enable` defaulting to false and magnetite not enabled, and keep the operational go/no-go as a separate later change. +That decision also required the R2 incompatibility to be established at the source rather than inherited from the note as a premise, and named that finding as one of the more valuable things this change can produce. +This change is the land-it-disabled half. +It delivers a reviewable configuration surface and declared credential slots, it records the verified object-storage finding with citations, and it states the enable path's prerequisites explicitly so the later go/no-go change inherits them rather than rediscovering them. +Nothing on the live host changes: the module is imported by magnetite and not enabled, so every unit, firewall opening, and secret sits behind an `enable` that no host sets. + +## What Changes + +This change packages the relay, declares its configuration surface as a NixOS module, and imports that module on magnetite without enabling it. +It provisions no daemons, opens no vhost, spends no DNS record, and chooses no tenant hostname, because the binding decision reserves every irreversible naming choice for the change that actually turns the relay on. +The object-storage question is answered here as a finding rather than as an implementation: R2 was verified at source and fails, Garage single-node is recommended with MinIO as the known-good fallback, and no object store is built in this change. + +**A separate `relay-v*` source pin, independent of the shared desktop pin** +- From: one `pkgs/by-name/buzz/source` derivation pinned to the `desktop-v*` tag line, feeding every Buzz package this fleet builds. +- To: a second `pkgs/by-name/buzz/relay-source` derivation pinned to `relay-v0.2.1` (`6e5c462ac524de60d7edb46c66130fd779cc9006`, 2026-08-08) with its own `update.sh`, alongside the untouched `source` derivation. +- Reason: the relay releases on its own tag line and its own cadence, so a single shared pin would couple a server upgrade to a client upgrade — bumping the relay to pick up a server fix would simultaneously move the CLI and the credential helper the desktop configuration installs. +- Impact: a second `fetchCargoVendor` of the same workspace lockfile, in exchange for two upgrade decisions that stay independent. + +**A `buzz-relay` package derivation built from that pin** +- From: no relay binary is buildable from this repository. +- To: a `pkgs/by-name/buzz/relay` derivation building the `buzz-relay` binary, with the compiled-in pre-receive hook shebang rewritten to a store path, a runtime PATH prefix carrying the six tools the fail-closed hook and the git transport both need, and an install check that drives the relay's own configuration validator. +- Reason: the hook is a compiled-in `const &str` that `patchShebangs` cannot reach, and a missing tool does not degrade the hook but rejects every push, so both facts are asserted at build time rather than assumed. +- Impact: `buzz-relay` builds on aarch64-darwin and x86_64-linux, and the two new package checks register automatically. + +**A NixOS module shipped with `enable = false`** +- From: no relay configuration surface exists in this repository. +- To: a `flake.modules.nixos.buzz-relay` module with sixteen typed options, a free-form `settings` escape hatch rendered last, seven assertions, four clan-vars generator declarations wired into the unit's `EnvironmentFile`, and a hardened systemd unit — all behind `services.buzz-relay.enable`, which defaults to false. +- Reason: the binding decision is to land the surface reviewed and buildable while the operational go/no-go stays a separate change, so the configuration and the credential slots are declared before anything is switched on. +- Impact: the module is inert; importing it changes no unit, no firewall rule, and no secret on any host. + +**Object storage answered as a verified finding, not an implementation** +- From: the 2026-08-04 note asserted R2 incompatibility with a shaky consistency argument, an unchecked S3 client layer, and no named off-switch. +- To: a source-verified finding recorded in design.md — R2 fails both the startup probe and steady state on the documented per-key 1-write/sec throttle, with Garage single-node recommended and MinIO named as the known-good fallback because upstream verifies against MinIO — together with three explicit corrections to the note's own claims. +- Reason: the binding decision required the incompatibility to be established at source rather than taken as a premise, and required a recommendation reasoning about fleet fit rather than features in the abstract. +- Impact: no object store is built here; the recommendation becomes the input to the later go/no-go change. + +**Postgres, Redis, and backups modelled as dependencies rather than provisioned** +- From: the working note framed Postgres, Redis, and object storage as things the module would wire up, and its Phase 4 prescribes deploying with `enable = true`. +- To: all three modelled as typed options and assertions with no provisioning — no `services.postgresql` colonisation, no `services.redis`, no bucket — and the three unmet prerequisites stated explicitly as the enable path's inheritance. +- Reason: the binding decision forbids provisioning new daemons on magnetite in this change and requires the change to carry the prerequisites explicitly so the later go/no-go inherits them. +- Impact: enabling the relay without an object store, a Redis, and a backup story fails at the option layer or the assertion layer rather than starting a relay that cannot work. + +**A magnetite import that is deliberately not an enablement** +- From: magnetite imports its service modules and enables them in the same place. +- To: magnetite imports `buzz-relay` with a comment recording that `services.buzz-relay.enable` defaults to false and is not set here, pointing at the module header for the prerequisites that gate switching it on. +- Reason: importing keeps the module inside the evaluated configuration so it is reviewable and cannot silently rot, while the unset `enable` keeps the live host unchanged. +- Impact: magnetite's evaluated configuration is semantically unchanged — no new unit, no new firewall port, no new generator. + +## Capabilities + +### New Capabilities +- `buzz-relay`: the Buzz relay's packaging and NixOS configuration surface as a deliberately inert deployment — two package derivations on a `relay-v*` source pin held separate from the shared `desktop-v*` pin, and a `flake.modules.nixos.buzz-relay` module that ships `enable = false`, provisions no daemons, refuses a public bind, asserts on every option whose absence would be silently wrong, declares four clan-vars credential slots and wires each one into the unit that consumes it, forces auth-token enforcement on against an upstream default that would otherwise select a published development key, and is imported by magnetite without being enabled; together with the recorded object-storage finding (R2 fails the relay's conformance probe and its steady-state push path, Garage single-node recommended, MinIO the known-good fallback) and the three explicit prerequisites the enable path inherits. + +### Modified Capabilities + + +## Impact + +Files added: `pkgs/by-name/buzz/relay-source/package.nix`, `pkgs/by-name/buzz/relay-source/update.sh`, `pkgs/by-name/buzz/relay/package.nix`, and `modules/nixos/buzz-relay.nix`. +Files updated: `modules/machines/nixos/magnetite/default.nix` gains the module import and a comment recording that it is deliberately not enabled. +`modules/checks/packages.nix` is unchanged, because both new packages build on aarch64-darwin and need no blacklist entry. +The four existing `pkgs/by-name/buzz/{source,cli,git-credential-nostr,git-sign-nostr}` packages are verified untouched, which is the point of the separate source pin. +Live-host impact is nil: magnetite's evaluated configuration gains no systemd unit, no firewall opening, and no secret, because every one of them is behind an `enable` that defaults to false and is set on no host. +Out of scope, and each named as an explicit prerequisite the later go/no-go change inherits: standing up the object store, introducing Redis as a new daemon class on this fleet, and establishing a backup story that does not exist fleet-wide. +Also out of scope: the nginx vhost, the Cloudflare DNS record, and the tenant hostname, because the Host derived from `RELAY_URL` is a durable tenant key persisted in Postgres and signed into every auth event, so it is not a free rename and belongs to the change that turns the relay on. +Also out of scope: building the `web` and `admin-web` Vite/React bundles, which would add a second JavaScript dependency closure; both variables are left unset, which is upstream's own source-tree default. +Verification here is confined to what a disabled module can prove: both packages build, the magnetite configuration evaluates with the module imported and stays semantically inert, and the enabled path evaluates with no failed assertions when given a complete configuration. diff --git a/openspec/changes/buzz-relay-module/retrospective.md b/openspec/changes/buzz-relay-module/retrospective.md new file mode 100644 index 000000000..13597e559 --- /dev/null +++ b/openspec/changes/buzz-relay-module/retrospective.md @@ -0,0 +1,76 @@ +# Retrospective: buzz-relay-module + +> STUB: written after verify passed. Complete this post-implementation. +> Written: (after verify passed) +> Commit range: `..` +> Worktree: + +--- + +## 0. Evidence + +> Up-front quantified data — later Wins / Misses bullets reference it directly. + +- **Commit range**: `..` ( commits) +- **Diff size**: <+X / -Y lines across N files> +- **Tasks done**: / +- **Active hours**: +- **Subagent dispatches**: +- **New external dependencies**: +- **Bugs encountered post-merge**: +- **OpenSpec validate state at archive**: +- **Test coverage signal**: + +Commit chain (chronological): + +``` + +... + +``` + +--- + +## 1. Wins + +- [evidence: ] + +## 2. Misses + +- [high] [blocking | evidence: ...] +- [med] [painful | evidence: ...] +- [low] [nit | evidence: ...] + +## 3. Plan deviations + +| Plan task | What changed | Why | +|-----------|--------------|-----| +| — | — | — | + +## 4. Skill / workflow compliance + +| Skill | Used | +|--------------------------------------------------|------| +| superpowers:brainstorming | | +| superpowers:writing-plans | | +| superpowers:using-git-worktrees | | +| superpowers:subagent-driven-development | | +| (transitive) superpowers:test-driven-development | | +| (transitive) superpowers:requesting-code-review | | +| superpowers:finishing-a-development-branch | | + +### Deliberately Skipped Skills + +> Each `no` above must answer: what was skipped, why this cycle, how to prevent recurrence. + +- + +## 5. Surprises + +- + +## 6. Promote candidates → long-term learning + +- [ ] [high/med/low] **** → **Promote to** + > **Why**: + > **How to apply**: diff --git a/openspec/changes/buzz-relay-module/specs/buzz-relay/spec.md b/openspec/changes/buzz-relay-module/specs/buzz-relay/spec.md new file mode 100644 index 000000000..aa1ca1425 --- /dev/null +++ b/openspec/changes/buzz-relay-module/specs/buzz-relay/spec.md @@ -0,0 +1,204 @@ +## ADDED Requirements + +### Requirement: The relay module ships disabled and is inert when imported + +The configuration SHALL provide the Buzz relay as a `flake.modules.nixos.buzz-relay` module whose `services.buzz-relay.enable` option defaults to false. +Every systemd unit, firewall opening, and clan-vars generator the module declares SHALL sit behind that `enable` option, so that importing the module changes nothing. +No host SHALL set `services.buzz-relay.enable` in this change. +The module SHALL be imported by magnetite so that it stays inside the evaluated configuration and is type-checked on every evaluation, and that import SHALL NOT be an enablement. + +#### Scenario: the module is imported without being enabled + +- **WHEN** the magnetite NixOS configuration is evaluated with `flake.modules.nixos.buzz-relay` imported +- **THEN** `services.buzz-relay.enable` resolves to false because no host sets it, and the evaluation succeeds + +#### Scenario: an imported but disabled module emits nothing + +- **WHEN** the magnetite configuration is evaluated with the module imported and `enable` left at its default +- **THEN** no `systemd.services.buzz-relay` unit exists, no `buzz-relay` clan-vars generator is declared, and the ZeroTier-scoped firewall port list is unchanged from before the import + +### Requirement: Missing required options fail at evaluation rather than at runtime + +The module SHALL declare `relayUrl`, `objectStore.endpoint`, `objectStore.bucket`, `database.url`, and `redis.url` with no defaults, because upstream's defaults for each are a live development target that an unconfigured relay would address without complaint. +An enabled configuration that omits a required option SHALL fail at evaluation time. +The module SHALL assert that `relayUrl` is non-empty, that `objectStore.endpoint` and `objectStore.bucket` are both set, that `database.url` is non-empty, and that `redis.url` is non-empty. +The module SHALL declare no option naming a path to a secret, because the object-store credentials and the database password reach the relay from their clan-vars generators rather than from a file an operator manages by hand. +The module SHALL assert that `database.url` does not contain the string `buzz_dev`, because that is both upstream's published development password and a signal that a password has been written into an option that is rendered world-readable into the Nix store. +The module SHALL assert that `relayUrl` carries a `ws://` or `wss://` scheme, because the scheme drives the expected-URL reconstruction that NIP-42 and NIP-98 verify signatures against, so a wrong scheme fails authentication at runtime rather than at parse. + +#### Scenario: an enabled configuration omits a required option + +- **WHEN** `services.buzz-relay.enable` is set to true with `objectStore.endpoint` left undefined +- **THEN** evaluation fails at the option layer reporting that the option was accessed but has no value defined, rather than producing a relay that would start against a development default + +#### Scenario: an enabled configuration supplies wrong values + +- **WHEN** an enabled configuration sets a non-websocket `relayUrl` scheme, a public `bindAddress`, and a `database.url` containing `buzz_dev` +- **THEN** the corresponding assertions fire at evaluation time, each with a message naming the silent runtime failure it prevents + +#### Scenario: a complete enabled configuration evaluates cleanly + +- **WHEN** an enabled configuration supplies every required option with valid values +- **THEN** evaluation succeeds with no failed assertions and the unit's `ExecStart` resolves to the `buzz-relay` package's binary + +### Requirement: The module refuses a public bind + +The module SHALL constrain `bindAddress` to loopback or to the fleet's ZeroTier mesh prefix, and SHALL default it to `127.0.0.1` rather than inheriting upstream's `0.0.0.0`. +This constraint SHALL be an assertion rather than a convention, because magnetite retains `net.ipv6.ip_nonlocal_bind=1`, under which binding an address the host does not hold succeeds silently instead of failing at startup. +The assertion message SHALL record that the health and metrics listeners bind `0.0.0.0` unconditionally and independently of `bindAddress`, so a mesh-scoped firewall rule is their only containment. + +#### Scenario: a public bind address is rejected + +- **WHEN** an enabled configuration sets `bindAddress` to an address that is neither loopback nor inside the ZeroTier mesh prefix +- **THEN** the no-public-bind assertion fires at evaluation time and its message names the retained `ip_nonlocal_bind=1` behaviour that would otherwise make the mistake silent + +#### Scenario: the health and metrics containment is stated rather than assumed + +- **WHEN** the no-public-bind assertion message is read +- **THEN** it records that the health and metrics listeners bind `0.0.0.0` regardless of `bindAddress` and that the mesh-scoped firewall rule is what contains them + +### Requirement: The module provisions no daemons and no backing services + +The module SHALL provision no PostgreSQL, no Redis, and no object store, and SHALL model all three as dependencies expressed through typed options and assertions. +`redis.url` SHALL carry no default, so that an enabled configuration which omits it is refused at evaluation rather than silently directed at a `redis://localhost:6379` that no host on this fleet serves. +The module SHALL NOT enable `services.postgresql`, SHALL NOT enable `services.redis`, and SHALL NOT create a bucket. +The module SHALL NOT declare an nginx vhost, a DNS record, or an ACME certificate. + +#### Scenario: enabling the relay does not colonise the host's backing services + +- **WHEN** the module is evaluated with `enable` set to true +- **THEN** it enables no `services.postgresql` and no `services.redis`, creates no bucket, and declares no nginx vhost, DNS record, or ACME certificate + +#### Scenario: an absent backing service is reported as a configuration error + +- **WHEN** an enabled configuration supplies no `redis.url` +- **THEN** evaluation fails because the option has no default and no value, rather than the relay inheriting a localhost Redis address that resolves to nothing on this fleet + +### Requirement: The relay's tenant hostname is not chosen in this change + +The module SHALL declare `relayUrl` with no default and no placeholder value, so that the tenant hostname is a mandatory choice at enable time rather than an inherited one. +The `relayUrl` option's documentation and its assertion messages SHALL record that the Host derived from it is persisted as `communities.host` in PostgreSQL and signed into every NIP-42 and NIP-98 auth event, so changing it later orphans the existing community row and invalidates the signed history against it. +This change SHALL spend no vhost, no DNS record, and no hostname, because those are the irreversible naming choices reserved for the change that turns the relay on. + +#### Scenario: no hostname is committed by this change + +- **WHEN** this change's package derivations, module, and magnetite import are inspected for a relay hostname +- **THEN** none is present, because `relayUrl` has no default and no vhost or DNS record is declared + +#### Scenario: the tenant-key consequence is recorded where an operator will meet it + +- **WHEN** an operator reads the `relayUrl` option description or triggers its assertions +- **THEN** the durable-tenant-key consequence is stated explicitly, including that the Host is persisted as `communities.host` and signed into every auth event + +### Requirement: The relay is built from a source pin independent of the shared desktop pin + +The configuration SHALL package the relay from a `relay-source` derivation pinned to the upstream `relay-v*` tag line, held separate from the existing `source` derivation that tracks `desktop-v*` and feeds the client-side packages. +The shared `source` derivation and the four existing client packages built from it SHALL remain untouched by this change. +The separation SHALL exist so that a server upgrade is not coupled to a client upgrade in either direction. +The relay source's update script SHALL read the upstream tag-refs API rather than the releases API, because upstream publishes no GitHub Release object for `relay-v*` tags, and SHALL exclude prereleases lexically because a bare tag carries no prerelease flag. + +#### Scenario: the relay pin moves without moving the client packages + +- **WHEN** the `relay-source` pin is advanced to a newer `relay-v*` tag +- **THEN** the shared `source` derivation and the client packages built from it are unchanged, so the desktop CLI and git credential helper do not move with the server + +#### Scenario: the update script can actually resolve a relay tag + +- **WHEN** the relay source's update script queries upstream for the newest non-prerelease `relay-v*` version +- **THEN** it resolves the tag through the tag-refs API and asserts the ref object is a commit, rather than filtering a releases API that contains no `relay-v*` entries + +### Requirement: The relay package asserts its fail-closed runtime dependencies at build time + +The relay package SHALL rewrite the compiled-in pre-receive hook shebang from `#!/usr/bin/env bash` to a store path, using a substitution that fails when its pattern is absent, because the hook text is a compiled-in string constant that shebang patching cannot reach and the file only appears on disk at runtime. +The relay package SHALL wrap the binary with a PATH prefix carrying the tools the fail-closed hook and the git transport both require, because a missing tool does not degrade the hook but causes it to reject every push. +The relay package SHALL verify both the shebang rewrite and the wrapper contents in an install check, because a dropped wrapper or a missed substitution yields a relay that starts and serves traffic normally and then rejects every git push. +The relay package's install check SHALL drive the relay's own configuration validator rather than a flag probe, because the relay registers no argument parser and any flag invocation proceeds to open a database connection. + +#### Scenario: the hook shebang is rewritten and the rewrite is asserted + +- **WHEN** the relay package is built +- **THEN** the compiled-in pre-receive hook shebang resolves to a store-path bash and no `#!/usr/bin/env bash` remains in the wrapped binary, with the substitution failing the build if the upstream pattern is absent + +#### Scenario: the install check exercises real relay code without a database + +- **WHEN** the relay package's install check runs +- **THEN** it points `BUZZ_WEB_DIR` at a directory without `index.html` and asserts both a non-zero exit and the validator's exact message, needing no network and no database + +### Requirement: The web and admin bundles are deliberately unset + +The relay package and the module SHALL leave `BUZZ_WEB_DIR` and `BUZZ_ADMIN_WEB_DIR` unset, because both upstream directories are TypeScript sources requiring a pnpm build rather than prebuilt static assets, and packaging them would vendor a second JavaScript dependency closure. +Leaving them unset SHALL be recognised as upstream's own source-tree default and as the only safe value, because setting either to a directory lacking `index.html` returns a configuration error that makes the relay refuse to start. +The resulting gap SHALL be recorded explicitly: no read-only admin dashboard, no bundled invite landing page, and no optional git repository browser, while the WebSocket relay, the REST surface, git push and pull over HTTP, and NIP-42 and NIP-98 authentication are unaffected. + +#### Scenario: the bundles are absent rather than misconfigured + +- **WHEN** the relay package and the module are inspected for `BUZZ_WEB_DIR` and `BUZZ_ADMIN_WEB_DIR` +- **THEN** neither variable is set, rather than being pointed at a directory that lacks `index.html` and would make the relay refuse to start + +#### Scenario: the missing admin surface is documented as a known gap + +- **WHEN** a reader asks what the unset bundles cost +- **THEN** the change records that there is no admin dashboard, no invite landing page, and no git repository browser, and that the relay, REST, git transport, and auth surfaces are unaffected + +### Requirement: Credential slots are declared before the service is enabled + +The module SHALL declare four clan-vars generators for the relay's Nostr identity key, its git hook HMAC secret, its PostgreSQL role password, and its object-store credentials, all inside the `enable` guard so that none is realised while the module is disabled. +Generator names SHALL be derived from the service name rather than from upstream environment variable names, because a generator name is effectively immutable once minted and renaming one orphans the encrypted material committed for that host. +The identity and hook-HMAC generators SHALL exist because upstream's unset behaviour is unsafe rather than merely absent: an unset identity key falls back to a published development key or panics, and an unset hook HMAC secret is regenerated on every boot with no log line at all. +The module SHALL deliver secrets through `EnvironmentFile` rather than `LoadCredential`, because the relay exposes no `*_FILE` configuration variant and never consults the systemd credentials directory. +Every generator the module declares SHALL have its `env` file wired into the unit's `EnvironmentFile`, so that no generator prompts an operator or commits encrypted material for a secret the relay never receives. +The module SHALL NOT additionally declare an option naming a path to the same secret, because a secret on this fleet travels the sops-backed clan-vars lane and a hand-managed file would place it outside that lane while asking the operator for it a second time. + +#### Scenario: the generators are declared but not realised while disabled + +- **WHEN** the magnetite configuration is evaluated with the module imported and disabled +- **THEN** no `buzz-relay` clan-vars generator is present in the configuration, because all four sit inside the `enable` guard + +#### Scenario: generator names survive an upstream variable rename + +- **WHEN** upstream renames the environment variable a generator feeds +- **THEN** the generator name is unaffected because it is derived from the service name, so the encrypted material committed under that name is not orphaned + +#### Scenario: each secret is supplied once and actually reaches the relay + +- **WHEN** an operator enables the relay and supplies the object-store credentials through the prompt its generator raises +- **THEN** those credentials reach the unit through its `EnvironmentFile`, and no separate option demands the same credentials by a file path + +### Requirement: Auth-token enforcement is on by default, departing from upstream + +The module SHALL set `BUZZ_REQUIRE_AUTH_TOKEN` to true in the environment it renders, rather than inheriting upstream's default of false. +This departure SHALL exist because upstream's default is what selects the published hardcoded development private key when no relay identity key is supplied, so the upstream default fails open and the configuration that starts is the unsafe one. +With enforcement on, that same missing-key case SHALL become a startup failure rather than a relay signing events with a key that anybody can read from a public repository. +The value SHALL remain overridable through the `settings` escape hatch, which renders last, so an operator who needs upstream's behaviour states that intent explicitly. + +#### Scenario: the published development key cannot be selected silently + +- **WHEN** the relay is enabled and its identity key is absent from the environment +- **THEN** the relay refuses to start rather than falling back to the published development key, because the module has already forced auth-token enforcement on + +#### Scenario: the departure from upstream is recorded rather than discovered + +- **WHEN** a reader asks why this module's auth-token default differs from upstream's +- **THEN** the change records it as a deliberate security decision, naming the published development key that upstream's default selects + +### Requirement: The object-store finding and the enable path's prerequisites are recorded by this change + +This change SHALL record the object-storage verification as a decision with its citations intact, establishing that Cloudflare R2 fails both the startup conformance probe and the steady-state push path, that the probe's off-switch converts a loud startup failure into a silent correctness hazard, that lowering the probe's race width risks a false pass which is worse than failing, and that Garage single-node is recommended with MinIO as the known-good fallback because upstream verifies against MinIO. +This change SHALL record honestly that Garage's conditional-PUT compare-and-swap conformance was not documentation-verified in this pass and that the startup probe is the cheap adjudicator of the question. +This change SHALL record the three corrections the verification made to the prior working note's claims: that the note missed the documented probe off-switch, that it asserted the rate-limit mechanism without checking the S3 client layer's retry behaviour, and that it overstated the backend's consistency weakness when the documented gap is silence on compare-and-swap atomicity rather than weak consistency. +This change SHALL state the enable path's unmet prerequisites explicitly — an object store that passes the probe, Redis as a new daemon class on this fleet, and a backup story that does not exist fleet-wide — so that the later go/no-go change inherits them rather than rediscovering them. + +#### Scenario: the later change inherits the prerequisites rather than rediscovering them + +- **WHEN** the later go/no-go change reads this change's artifacts to determine what enabling the relay requires +- **THEN** it finds the object store, Redis, and backup prerequisites stated explicitly, together with the ordered enable path and the tenant-hostname consequence + +#### Scenario: the object-storage verdict is auditable in both directions + +- **WHEN** a reader checks the object-storage decision +- **THEN** it carries the probe's concurrency shape, the documented per-key write limit and its 429 response, the classifier's absent 429 arm, the steady-state push consequence, the recommendation, and the explicitly flagged thinness of the Garage claim + +#### Scenario: the prior note's superseded claims are corrected rather than left standing + +- **WHEN** a reader arrives at the object-storage question through the earlier working note +- **THEN** the note carries a dated addendum recording that the gate has been settled at source, the three corrections to its own claims, and that its instruction to enable the relay is superseded by the ship-disabled decision diff --git a/openspec/changes/buzz-relay-module/tasks.md b/openspec/changes/buzz-relay-module/tasks.md new file mode 100644 index 000000000..93e19e344 --- /dev/null +++ b/openspec/changes/buzz-relay-module/tasks.md @@ -0,0 +1,173 @@ +## 1. Object-storage verification at source (D1) + +This section precedes the packaging and module work because the binding decision made it the gate, and because its outcome determines what the enable path must provide. +No object store is built here; the finding is the deliverable. + +- [x] 1.1 Identify the exact conformance probe and the exact S3 semantics it depends on, phase by phase, at the pinned upstream revision + - Four phases at `store.rs:576-883`, entered from `main.rs:496-521`. Phase 1 sequential create-only plus read-after-write; phase 2 `If-Match` race; phase 3 `If-None-Match: *` race; phase 4 ETag-token consistency. + - Established what the probe does NOT exercise, which narrows the question: no multipart, no `x-amz-checksum-*`, no `ListObjectsV2`. Integrity is application-level SHA-256. + - `DeleteObject` is used at `store.rs:623`, `733`, `876`, but all three are `let _ =` ignored, so a 403 leaks scratch keys rather than failing the probe. This corrects the prior note's token-scoping claim. +- [x] 1.2 Establish the probe's concurrency shape, which is the decisive fact + - `race_width` defaults to 32 and is floor-clamped at 2 (`store.rs:114-121`, `578`); `race_rounds` defaults to 3. + - All concurrency targets ONE key per phase: `store.rs:591`/`644` for phase 2's pointer key, `store.rs:731`/`738` for phase 3's content-addressed key. Dispatch is simultaneous via `join_all` (`649`, `742`). + - 3 rounds × 32 racers × 2 race phases = 192 concurrent single-key conditional PUTs per boot, correcting the prior note's figure of roughly 96. +- [x] 1.3 Check whether upstream changed the probe, its classifier, its defaults, or the storage layer since the 2026-08-04 note + - No. `store.rs` has exactly one commit in its history and is byte-identical at the pinned revision; `git diff --stat` against that commit produces no output. +- [x] 1.4 Check R2's current documented support for precisely those semantics, by live fetch rather than recall + - Per-key write limit still 1/sec returning HTTP 429, now corroborated by a second page (error code 10058) that the prior note did not cite. + - Conditional `If-Match` and `If-None-Match` on `PutObject` are implemented, and 412 is documented; but the error-codes page links to a `#conditional-operations-in-putobject` section that does not exist, so CAS atomicity is undocumented. + - No R2 changelog entry after 2026-08-04 touches rate limits, conditional writes, or consistency. +- [x] 1.5 Check the S3 client layer the prior note did not check + - Client is `rust-s3` v0.37.2 with `fail-on-err`, so every non-2xx becomes `Err(HttpFailWithBody(..))`. PUTs are wrapped in a `retry!` macro, but the budget is 1, the backoff is 1s, and the retry is status-blind — it retries 412 as well, wasting a second per losing racer on every backend. + - One 1-second retry cannot absorb 32 writers against a 1-write/sec ceiling, so the verdict stands; the prior note simply reached it without checking. +- [x] 1.6 Record the verdict with citations in both directions, including where the prior note was wrong + - R2 fails twice: deterministically at the startup probe, and in steady state where the pointer key is the sole writer-serialization primitive (`cas_publish.rs:61-63`) so a 429 surfaces as an opaque 500 instead of a `409 non-fast-forward` (`transport.rs:851`ff). + - Recorded that `BUZZ_GIT_CONFORMANCE_PROBE=false` exists and that using it converts a loud startup failure into a silent correctness hazard, and that `BUZZ_GIT_PROBE_WRITERS=2` risks a false pass which is worse than failing. + - Three corrections to the prior note recorded as such: the missed off-switch, the unchecked client layer, and the overstated consistency weakness where the real gap is silence on CAS atomicity. +- [x] 1.7 Write the self-hosted recommendation reasoning about fleet fit rather than features in the abstract + - Garage single-node on magnetite, on RSS footprint against seven co-tenant services and a documented disk-starvation incident, plain-file storage that backs up by ZFS snapshot and send, and real nixpkgs packaging. Ceph disqualified on footprint, SeaweedFS on unestablished conditional-PUT CAS. + - Recorded honestly that Garage's conditional-PUT CAS was not doc-verified in this pass and is the thinnest claim, that upstream verifies empirically against MinIO which is therefore the known-good fallback, and that the probe itself is the cheap final adjudicator. + +## 2. Independent relay source pin (D4) + +- [x] 2.1 Add `pkgs/by-name/buzz/relay-source/package.nix` pinned to the newest non-prerelease `relay-v*` tag, with `passthru.cargoDeps` vendored for that pin + - Pinned `relay-v0.2.1` at `6e5c462ac524de60d7edb46c66130fd779cc9006` (2026-08-08), version attribute `0.2.1`. + - The header records that unlike the shared `source` derivation, `version` here is a real crate version rather than a release-train number, because the relay crate declares its version explicitly and does not inherit the frozen workspace version. +- [x] 2.2 Leave the shared `pkgs/by-name/buzz/source` derivation and the four client packages built from it untouched + - Verified: `nix build .#buzz-source .#buzz-cli` still succeeds unchanged, and no file under the four existing buzz package directories was modified. +- [x] 2.3 Add `pkgs/by-name/buzz/relay-source/update.sh` modelled on its sibling, deviating only where the sibling's mechanism cannot work + - DEVIATION, forced: upstream publishes no GitHub Release object for `relay-v*` tags, so the sibling's releases-API filter could never bump the pin. The script reads the tag-refs API instead, with the reason documented in its header. + - Prerelease exclusion is lexical because a bare tag carries no `draft`/`prerelease` flag, and the rev extraction asserts the ref object is a commit so a future annotated tag fails loudly instead of recording a tag-object sha. + - Verified live end to end: the filter yields the three relay tags, selects the newest, resolves the pinned rev, and a re-run exits 0 with no diff. + +## 3. Relay package derivation (D5) + +- [x] 3.1 Add `pkgs/by-name/buzz/relay/package.nix` building the relay binary from the relay-source pin + - Inherits `version` and `cargoDeps` from the pin so a bump moves both together; `cargoBuildFlags` names the binary explicitly so an upstream second binary changes the install set visibly. +- [x] 3.2 Rewrite the compiled-in pre-receive hook shebang to a store path, asserting the substitution rather than assuming it + - Done in `postPatch` with `--replace-fail`, because the hook is a compiled-in string constant with no file on disk at build time and plain substitution does not fail on an absent pattern. + - Verified in the built artifact: the wrapped binary contains a store-path bash shebang and zero occurrences of `/usr/bin/env bash`. +- [x] 3.3 Wrap the binary with the runtime PATH prefix the fail-closed hook and the git transport both need + - The hook is fail-closed by construction and upstream encodes the same requirement as a compiled-in test asserting its own container installs curl and openssl. + - The prefix is load-bearing on a second independent path: the relay shells out to a bare `git` in its git transport, so git must be on PATH for fetch and push to work at all. +- [x] 3.4 Leave `BUZZ_WEB_DIR` and `BUZZ_ADMIN_WEB_DIR` unset, with the asymmetry documented + - Both upstream directories are Vite/React TypeScript sources, not prebuilt assets; packaging them would vendor a 7118-line pnpm lockfile plus two patches as a second hash-churn surface. + - Unset is upstream's own source-tree default and is safe; setting either to a directory lacking `index.html` is a HARD STARTUP FAILURE. Recorded cost: no admin dashboard, no invite landing page, no git repo browser. Unaffected: relay, REST, git transport, auth. +- [x] 3.5 Add an install check that reaches real relay code without a network or a database + - DEVIATION, forced: a `--help` probe fails, because the relay registers no argument parser at all and any invocation proceeds to open a database pool and time out. + - Replaced with a check that drives the configuration validator, which runs before any database work, and asserts both the non-zero exit and the exact message. This pins the very semantics 3.4 relies on. + - The check also asserts the wrapper contents and the shebang rewrite both positively and negatively, because either failure yields a relay that serves traffic normally and then rejects every git push. +- [x] 3.6 Build on both supported systems and confirm no package-check blacklist entry is needed + - Builds on aarch64-darwin and on x86_64-linux; aarch64-linux evaluates to a valid derivation path and was not built (builder unreachable, expected). + - `modules/checks/packages.nix` deliberately unchanged, because both new packages build on darwin. The two new package checks auto-registered and pass. + - Confirmed empirically that the TLS backend is `aws-lc-sys` via the cmake backend and that `openssl-sys` is never compiled, so native dependencies are cmake only, with no bindgen and no perl. + +## 4. NixOS module option surface (D6) + +- [x] 4.1 Author `modules/nixos/buzz-relay.nix` as `flake.modules.nixos.buzz-relay` using the plain deferred-module pattern + - Chosen over a clan service because the relay needs values only visible in `config.*`, which the statically evaluated clan inventory layer cannot see; this matches every other service module on the host. +- [x] 4.2 Type the options whose wrong value is unrecoverable or silent, and route everything else through `settings` + - Sixteen typed options across three groups: unrecoverable (`relayUrl`, the database identity), silently wrong (bind address and port, the object-store endpoint, bucket, region and addressing style, `redis.url`, `autoMigrate`), and interface contract (health and metrics ports, `adminHost`, `stateDir`, `openFirewall`). + - No option names a path to a secret, and no option is declared that nothing reads: `database.name` and `database.user` were dropped on review because the relay reads only `DATABASE_URL`, so both were typed, documented, and referenced nowhere. + - Roughly 90 further variables reach the unit through free-form `settings`, rendered LAST so it can override any typed value without a module edit. + - `settings` renders booleans as the literal strings `"true"`/`"false"`, because the relay has five distinct boolean dialects and those two literals are the only forms all five read identically. +- [x] 4.3 Give the required options no defaults rather than inheriting upstream's + - `relayUrl`, `objectStore.endpoint`, `objectStore.bucket`, `database.url`, and `redis.url` have no defaults. Upstream's values for each are a live development target that an unconfigured relay would address without complaint. + - Verified: enabling with a required option missing fails at the OPTION layer first, reporting the option was accessed but has no value defined. +- [x] 4.4 Default `bindAddress` to loopback rather than upstream's public bind + - Defaults to `127.0.0.1`, not `0.0.0.0`, with IPv6 literals bracketed automatically when the bind target is assembled. +- [x] 4.5 Force `BUZZ_REQUIRE_AUTH_TOKEN` on rather than inheriting upstream's default of false (D10) + - Upstream's default is what selects the published hardcoded dev private key when no identity key is supplied, so it fails open: the configuration that starts is the unsafe one. + - Set in the rendered environment rather than asserted on, because an assertion cannot fire on an operator who never wrote the variable, which is exactly the dangerous case. `settings` renders last, so the upstream behaviour remains reachable by explicit intent. + +## 5. Assertions that convert silent runtime wrongness into build-time refusal (D3, D6) + +- [x] 5.1 Assert `relayUrl` is non-empty, with the durable-tenant-key consequence in the message + - The message states that the Host derived from it is persisted as `communities.host` and signed into every auth event, so it cannot be changed later without orphaning that community and invalidating its signed history. +- [x] 5.2 Assert `relayUrl` carries a `ws://` or `wss://` scheme + - The scheme drives the expected-URL reconstruction both auth schemes verify against, so a wrong scheme fails authentication at runtime rather than at parse. +- [x] 5.3 Assert the no-public-bind invariant, admitting only loopback or the fleet's ZeroTier prefix + - Load-bearing because the host retains `ip_nonlocal_bind=1`, under which binding an address the host does not hold succeeds SILENTLY instead of failing at startup. + - The message records that the health and metrics listeners bind `0.0.0.0` unconditionally regardless of this option, so the mesh-scoped firewall rule is their only containment. +- [x] 5.4 Assert the object-store endpoint and bucket are both set + - Both have no default, so omitting one fails at the option layer before the assertion is reached; the assertion covers the empty-string case that a default would not. + - No assertion demands a credentials file, because there is no such option: the object-store credentials arrive from their generator through `EnvironmentFile`. +- [x] 5.5 Assert `database.url` is non-empty and does not contain the published dev password + - The dev-password assertion catches both upstream's default and the more general hazard that any password written into that option is rendered world-readable into the Nix store. +- [x] 5.6 Assert `redis.url` is non-empty, stating that this fleet provisions no Redis + - The option carries no default. It briefly did, and that made the assertion unreachable on every documented path while pointing an unconfigured relay at a `redis://localhost:6379` that no host here serves; removing the default makes the requirement fail at eval instead. +- [x] 5.7 Confirm the assertions actually fire together on a wrong configuration + - Verified: an enabled configuration with a bad scheme, a public bind, and a dev password in the database URL fires each of the corresponding assertions with its full message. + +## 6. Credential slots declared before enablement (D7) + +- [x] 6.1 Declare the four clan-vars generators inside the `enable` guard, named after the service rather than the upstream variables + - Generator names are effectively immutable once minted, because renaming one orphans the encrypted material committed for that host; naming them after the service means an upstream variable rename cannot strand this fleet's material. +- [x] 6.2 Declare the relay identity generator, because upstream never generates or persists one + - With the variable unset the relay falls back to a hardcoded published dev key, or panics outright when auth tokens are required. Upstream's own doc comment claiming a fresh keypair is generated at startup is stale, and the code behaviour was encoded rather than the comment. + - The key must be stable across restarts because it signs the events membership mode verifies. +- [x] 6.3 Declare the git hook HMAC generator, fixing a fail-silent upstream default + - With the variable unset the relay mints a fresh secret on every boot and logs NOTHING about having done so, unlike the identity key which at least warns, so every restart silently invalidates outstanding hook signatures. +- [x] 6.4 Declare the database password generator in the established dual shape + - One generated value emitted twice: a bare password for whatever provisions the role, and an env fragment for the relay, which keeps the password out of the store-resident database URL entirely. +- [x] 6.5 Declare the object-store credentials generator in the prompt-and-fail-loudly shape + - Provider-minted credentials are not derivable on the host, so the generator prompts rather than inventing a value. Emitted as a single env file because the relay reads both keys only from the process environment. +- [x] 6.6 Deliver secrets via `EnvironmentFile` rather than `LoadCredential`, and record why + - The database password was initially staged through `LoadCredential` and then removed on review: the relay reads no `*_FILE` variant and never consults the credentials directory, so it was dead config. + - A comment records that `EnvironmentFile` preserves the same DynamicUser-safety property, because systemd reads it as root before privilege drop. +- [x] 6.7 Wire every generator's env file into the unit's `EnvironmentFile` so no declared slot is inert + - Found on review: the object-store and database-password generators were declared but unconsumed, so enabling would have prompted the operator for S3 credentials, committed them encrypted, and then still failed an assertion demanding the same credentials by a separate file path. + - Resolved by wiring both generators in and deleting `objectStore.credentialsFile` and the database password-file option, so each secret is prompted for exactly once and actually reaches the relay (D9). + - The binding constraint is that a secret on this fleet must travel the sops-backed clan-vars lane and never reach the Nix store or a committed file, which a hand-managed path option cannot satisfy. + +## 7. Hardened systemd unit, each fact verified before encoding (D6) + +- [x] 7.1 Use `Type = "exec"`, verified rather than assumed + - Verified there is no readiness-notification support anywhere in the upstream workspace, so a notify type was never an option. +- [x] 7.2 Set a stop timeout above the relay's real shutdown budget + - Verified upstream: a fixed grace period plus a drain timeout that force-exits, giving a 35-second worst case. The configured timeout leaves margin without racing it. +- [x] 7.3 Apply the restart-loop configuration from the prior host incident precedent + - Justified because database connection failure at start aborts with no retry, so the relay would otherwise exhaust its start limit against a temporarily unavailable dependency. +- [x] 7.4 Put git on the unit's PATH, verified against the relay's actual subprocess use + - Verified nineteen git subprocess spawns across the relay's git modules at the pinned revision, correcting the prior note's count of roughly sixteen. +- [x] 7.5 Apply the fleet's standard hardening block, dynamic user, state directory, and mesh-scoped firewall + - The firewall rule is scoped to the mesh interface only and covers the application, health, and metrics ports, because the latter two bind `0.0.0.0` unconditionally. + +## 8. magnetite import without enablement (D8) + +- [x] 8.1 Import the module on magnetite alongside its neighbours, with a comment recording that it is deliberately not enabled + - The comment states that `enable` defaults to false and is not set there, and points at the module header for the object-store, Redis, and backup prerequisites that gate switching it on. +- [x] 8.2 Verify the disabled path evaluates and is semantically inert + - Verified at the configuration level: `enable` false, no buzz generators, no relay unit, and an unchanged mesh firewall port list. +- [x] 8.3 Verify the enabled path evaluates cleanly when given a complete configuration + - Verified: with the real relay package and every required option supplied, evaluation succeeds with no failed assertions and the unit's `ExecStart` resolves to the packaged binary. + - Spot-checked that IPv6 bind targets are bracketed, that `settings` overrides typed values because it renders last, and that git is first on the unit's PATH. +- [x] 8.4 Investigate rather than accept the derivation-path shift, which would otherwise contradict inertness + - The only difference in the entire secrets manifest was the source-path hash of an unrelated key, with zero occurrences of "buzz" in either derivation. + - Control experiment: appending a newline to an unrelated docs file with NO module import at all shifted the derivation path identically, so the shift is whole-flake source hashing rather than anything attributable to the module. +- [x] 8.5 Confirm `services.buzz-relay.enable = true` appears on no host + - Confirmed: it is set nowhere in the repository, which is the binding decision's central constraint. + +## 9. Formatting and repository checks + +- [x] 9.1 Format every new and modified Nix file and confirm re-running is a no-op +- [x] 9.2 Confirm the dead-code check and the shell lint pass on the new files +- [x] 9.3 Confirm the repository's formatting check passes + +## 10. Change artifacts and the superseded working note (D1, D2) + +- [x] 10.1 Author the OpenSpec change artifacts for `buzz-relay-module` under the repository's schema + - `proposal.md`, `design.md`, `specs/buzz-relay/spec.md`, `tasks.md`, plus the `verify.md` and `retrospective.md` stubs, matching the artifact set the `sso-gateway` change carries at this stage. + - Declares a NEW `buzz-relay` capability, because no archived capability covers self-hosted service deployment and the precedent for a service change declaring its own is the cognee endpoint change. +- [x] 10.2 Carry the object-storage finding into `design.md` as a decision with its citations intact + - Includes the verdict on both axes, the off-switch hazard, the false-pass hazard at reduced race width, the recommendation, the honestly flagged thinness of the Garage claim, and the three corrections to the prior note. +- [x] 10.3 State the enable path's prerequisites explicitly so the later go/no-go change inherits them + - The three unmet prerequisites are named in the proposal, in the module header, and as a decision in the design, and the design's Migration Plan carries an ordered eight-step enable path ending at setting `enable = true`. + - That Migration Plan is the single forward pointer, deliberately: this ledger records only work this change completed, so an enable-path step carried here as an unchecked task would block archiving a change that is itself finished. +- [x] 10.4 Append a dated addendum to the existing self-hosting working note, append-only + - Records that the object-store gate has been settled at source and how, the three corrections to the note's own claims, where the packaging and the disabled module now live, and that the note's own instruction to deploy with the relay enabled is superseded by the ship-disabled decision. + - Append-only by construction: no existing line of that dated working note was reworded or removed, because its record stands as written. +- [x] 10.5 Validate the change strictly and keep the repository's totals at zero failed +- [x] 10.6 Reconcile the artifacts with the module fixes that landed after adversarial review + - Review found no blocking defect and four should-fix items; the module was changed for all four and the artifacts were corrected to stay true of what ships. + - Recorded the credential-path resolution as D9 and the auth-token default as D10, and rewrote every spec scenario that asserted on `credentialsFile`, on the dead `database.name`/`database.user`, or on a defaulted `redis.url`. diff --git a/openspec/changes/buzz-relay-module/verify.md b/openspec/changes/buzz-relay-module/verify.md new file mode 100644 index 000000000..2e55211be --- /dev/null +++ b/openspec/changes/buzz-relay-module/verify.md @@ -0,0 +1,122 @@ +# Verification Report + +> This file is produced by the `openspec-verify-change` skill after apply completes, to confirm that the +> implementation is consistent with the specs / design / tasks. Any failed check must be returned to its +> corresponding artifact for correction before re-running verify. +> +> STUB: not yet filled. Complete this report post-implementation, after apply. + +**Change**: `buzz-relay-module` +**Verified at**: `YYYY-MM-DD HH:mm` +**Verifier**: `` + +--- + +## 1. Structural Validation (`openspec validate --all --json`) + +- [ ] All items report `"valid": true` + +**Result**: + +```text + +``` + +If any items fail, list their id and issues: + +| Item | Type | Issues | +|---|---|---| +| — | — | — | + +--- + +## 2. Task Completion (`tasks.md`) + +- [ ] All `- [ ]` have been changed to `- [x]` + +**Incomplete tasks** (if any): + +| Task | Reason incomplete | Blocks archive? | +|---|---|---| +| — | — | — | + +--- + +## 3. Delta Spec Sync State + +For each delta spec file reported by the CLI +(`openspec status --change "buzz-relay-module" --json | jq -r '.artifactPaths.specs.existingOutputPaths[]'`), +compare against the corresponding main capability spec: + +| Capability | Sync status | Notes | +|---|---|---| +| buzz-relay | synced / pending sync / N/A | — | + +--- + +## 4. Design / Specs Coherence Spot Check + +Spot-check whether the decisions in `design.md` are reflected in the Requirements and +Scenarios of `specs/*.md`: + +| Sampled item | design description | specs correspondence | Gap | +|---|---|---|---| +| — | — | — | — | + +**Drift warnings** (non-blocking): + +- + +--- + +## 5. Implementation Signal + +- [ ] No unstaged files in the worktree +- [ ] All related commits have been pushed + +**Commit range** (if known): `..` + +--- + +## 6. Front-Door Routing Leak Detector (warning, non-blocking) + +Design output should not land in `docs/superpowers/specs/`. + +Detect: + +```bash +ls docs/superpowers/specs/*.md 2>/dev/null +``` + +- [ ] No files, or any existing files are legitimate residue from before schema installation + +**Leak list** (if any): + +| File | Content captured into change? | Recommended action | +|---|---|---| +| — | — | — | + +--- + +## 7. Deferred Manual Dogfood vs Automated Test Equivalence + +For each manual dogfood / smoke task in plan.md marked `[~]` deferred, list the +equivalent automated test coverage item by item. + +| Deferred dogfood (plan §) | Equivalent automated test | Coverage assessment | Real gap? | +|---|---|---|---| +| — | — | — | — | + +> When plan.md has no rows marked `[~]`, this section may be left blank (blank means PASS). + +--- + +## Overall Decision + +- [ ] (pass) PASS — may proceed to finishing-a-development-branch and archive +- [ ] (warn) PASS WITH WARNINGS — may proceed but note: `` +- [ ] (fail) FAIL — return to the failed artifact, correct it, then re-run verify + +**Next step**: + + diff --git a/pkgs/by-name/buzz/relay-source/package.nix b/pkgs/by-name/buzz/relay-source/package.nix new file mode 100644 index 000000000..d853914ba --- /dev/null +++ b/pkgs/by-name/buzz/relay-source/package.nix @@ -0,0 +1,66 @@ +# buzz-relay - source tree and vendored cargo dependencies for the relay. +# +# This is a second, independent pin of the same upstream repository as the +# sibling `source` derivation. It exists because the relay releases on its own +# tag line. `source` tracks desktop-v* and feeds the desktop/home-side CLI and +# git helpers; this tracks relay-v* and feeds the server. +# +# The separation is deliberate rather than incidental. A single shared pin +# would couple a server upgrade to a client upgrade: bumping the relay to pick +# up a server fix would simultaneously move the CLI and the credential helper +# that the desktop configuration installs, and bumping the desktop train would +# silently redeploy the relay. The cost is a second fetchCargoVendor of the +# same ~1000-package workspace lockfile; the benefit is that the two upgrade +# decisions stay independent. +# +# Unlike `source`, `version` here is a real crate version rather than a release +# train number. crates/buzz-relay/Cargo.toml declares `version = "0.2.1"` with +# an explicit comment that it does NOT inherit the workspace version (which is +# frozen at 0.1.0), because the relay ships as a pinnable artifact released on +# its own cadence. So the tag, this attribute, and what the binary reports all +# agree. +# +# Upstream publishes no GitHub Release object for relay-v* tags — only the +# annotated-less git tags themselves. update.sh therefore reads the tag refs +# API rather than the releases API; see the comment there. +# +# passthru.cargoDeps is the vendored dependency set for this pin. A consumer +# inherits it and must never also set cargoHash: the precedence chain in +# build-rust-package/default.nix:104-113 tests cargoVendorDir, then cargoDeps, +# then cargoLock, then cargoHash, so a non-null cargoDeps short-circuits before +# cargoHash is read and a stale or fabricated hash sitting beside it would +# never produce an error. +# +# Source: https://github.com/block/buzz +{ + fetchFromGitHub, + rustPlatform, +}: +let + version = "0.2.1"; + + # Self-reference is safe because `passthru` never becomes a derivation input: + # fetchFromGitHub forwards it to the fetcher (fetchgithub/default.nix:210-213) + # and mkDerivation excludes it from the derivation proper, so forcing + # `self.passthru.cargoDeps` does not force a cycle through `self`. + self = fetchFromGitHub { + name = "buzz-relay-source-${version}"; + owner = "block"; + repo = "buzz"; + tag = "relay-v${version}"; + hash = "sha256-vc9vMTQzL1NCJTIYasoGq+KCu2Lbdu8Wz7scsyyoiJ8="; + + passthru = { + inherit version; + rev = "6e5c462ac524de60d7edb46c66130fd779cc9006"; + + cargoDeps = rustPlatform.fetchCargoVendor { + src = self; + hash = "sha256-XWKN73l+tPw5p7uEg192wu5kndqXPVhDiGqM9N9bJnk="; + }; + + updateScript = ./update.sh; + }; + }; +in +self diff --git a/pkgs/by-name/buzz/relay-source/update.sh b/pkgs/by-name/buzz/relay-source/update.sh new file mode 100755 index 000000000..750fce7b7 --- /dev/null +++ b/pkgs/by-name/buzz/relay-source/update.sh @@ -0,0 +1,125 @@ +#!/usr/bin/env nix-shell +#!nix-shell --pure -i bash -p curl jq cacert git nix-prefetch-github gnused coreutils +# shellcheck shell=bash +# +# Bumps pkgs/by-name/buzz/relay-source to the newest relay-v tag: rewrites the +# version, the src hash and the recorded rev in package.nix, and blanks the +# vendor hash. +# Invoked via `nix run .#update-buzz-relay-source` (passthru.updateScript). +# +# This deliberately reads the *tag refs* API rather than the releases API that +# the sibling source/update.sh uses. Upstream publishes GitHub Release objects +# for desktop-v*, mobile-v* and chart-v*, but not for relay-v*: as of +# relay-v0.2.1, `GET /releases/tags/relay-v0.2.1` returns 404 and no relay-v +# entry appears anywhere in `GET /releases?per_page=100`. A releases-based +# filter would therefore always take the empty-set failure path and could never +# bump this pin. git/matching-refs/tags/relay-v returns all relay tags and +# resolves each to a commit sha in one call. +# +# The draft/prerelease filtering that source/update.sh performs has no analogue +# here, since a bare tag carries no such flags. Release-candidate tags are +# excluded lexically instead: sort -V would order relay-v0.3.0-rc.1 after +# relay-v0.2.1 and pin a prerelease. +# +# No lockfile is generated: upstream ships Cargo.lock, and the vendored +# dependency set is derived from it by rustPlatform.fetchCargoVendor. + +set -euo pipefail + +owner="block" +repo="buzz" +fake_sri="sha256-0000000000000000000000000000000000000000000=" + +repo_root="$(git rev-parse --show-toplevel)" +pkg_dir="${repo_root}/pkgs/by-name/buzz/relay-source" +pkg_nix="${pkg_dir}/package.nix" + +current_version="$(sed -n 's/^ version = "\(.*\)";$/\1/p' "$pkg_nix" | head -1)" +if [[ -z "$current_version" ]]; then + echo "error: could not read the current version from ${pkg_nix}" >&2 + exit 1 +fi + +refs_json="$(curl -fsSL \ + -H "Accept: application/vnd.github+json" \ + "https://api.github.com/repos/${owner}/${repo}/git/matching-refs/tags/relay-v")" + +mapfile -t relay_versions < <( + printf '%s' "$refs_json" \ + | jq -r '.[].ref' \ + | sed -n 's|^refs/tags/relay-v||p' \ + | grep -v -- '-' \ + | sort -V +) + +# The relay-v namespace is young and the buzz tag namespace has already been +# renamed twice, so an empty filtered set means the naming moved rather than +# that no release exists. Fail loudly with the evidence instead of silently +# reporting no-op. +if [[ ${#relay_versions[@]} -eq 0 ]]; then + echo "error: no tag matching relay-v* was found" >&2 + echo "observed tag refs:" >&2 + printf '%s' "$refs_json" | jq -r '.[].ref' | sed 's/^/ /' >&2 + exit 1 +fi + +latest_version="${relay_versions[-1]}" +latest_tag="relay-v${latest_version}" + +if [[ "$current_version" == "$latest_version" ]]; then + echo "buzz relay source is already at version ${current_version}" + exit 0 +fi + +echo "Updating buzz relay source: ${current_version} -> ${latest_version}" + +echo "Computing source hash for tag ${latest_tag}..." +new_sri="$(nix-prefetch-github "$owner" "$repo" --rev "$latest_tag" | jq -r '.hash')" +if [[ -z "$new_sri" || "$new_sri" == "null" ]]; then + echo "error: nix-prefetch-github did not return a hash" >&2 + exit 1 +fi + +# Resolved from the same payload the tag list came from. A tag object rather +# than a direct commit ref would report type "tag" and a sha that is not the +# commit, so the type is asserted rather than assumed. +new_rev="$(printf '%s' "$refs_json" \ + | jq -r --arg ref "refs/tags/${latest_tag}" \ + '.[] | select(.ref == $ref) | select(.object.type == "commit") | .object.sha')" +if [[ -z "$new_rev" || "$new_rev" == "null" ]]; then + echo "error: could not resolve ${latest_tag} to a commit sha" >&2 + echo "note: an annotated tag needs one further dereference through git/tags/" >&2 + exit 1 +fi + +# package.nix carries two `hash =` lines. Scope each rewrite to one side of the +# passthru block so the src hash and the vendor hash cannot be confused. +passthru_line="$(grep -n '^ passthru = {$' "$pkg_nix" | head -1 | cut -d: -f1)" +if [[ -z "$passthru_line" ]]; then + echo "error: could not locate the passthru block in ${pkg_nix}" >&2 + exit 1 +fi + +sed -i'' -e "s|^ version = \"${current_version}\";\$| version = \"${latest_version}\";|" "$pkg_nix" +sed -i'' -e "1,${passthru_line}s|hash = \"sha256-[^\"]*\"|hash = \"${new_sri}\"|" "$pkg_nix" +sed -i'' -e "${passthru_line},\$s|hash = \"sha256-[^\"]*\"|hash = \"${fake_sri}\"|" "$pkg_nix" +sed -i'' -e "s|rev = \"[0-9a-f]\{40\}\"|rev = \"${new_rev}\"|" "$pkg_nix" + +# Fail loudly if any rewrite did not take, rather than reporting success on a no-op. +grep -q "version = \"${latest_version}\"" "$pkg_nix" \ + || { echo "error: version was not updated in package.nix" >&2; exit 1; } +grep -q "hash = \"${new_sri}\"" "$pkg_nix" \ + || { echo "error: src hash was not updated in package.nix" >&2; exit 1; } +grep -q "hash = \"${fake_sri}\"" "$pkg_nix" \ + || { echo "error: vendor hash was not reset in package.nix" >&2; exit 1; } +grep -q "rev = \"${new_rev}\"" "$pkg_nix" \ + || { echo "error: rev was not updated in package.nix" >&2; exit 1; } + +echo "Updated buzz relay source to ${latest_version}" +echo " tag: ${latest_tag}" +echo " rev: ${new_rev}" +echo " src hash: ${new_sri}" +echo +echo "The vendor hash was reset to the placeholder and must be recomputed:" +echo " nix build .#buzz-relay-source.cargoDeps.vendorStaging" +echo "then copy the reported got: hash into passthru.cargoDeps.hash." diff --git a/pkgs/by-name/buzz/relay/package.nix b/pkgs/by-name/buzz/relay/package.nix new file mode 100644 index 000000000..3d257208b --- /dev/null +++ b/pkgs/by-name/buzz/relay/package.nix @@ -0,0 +1,218 @@ +# buzz-relay - WebSocket relay server for the Buzz communications platform. +# +# Built from the relay-source sibling rather than the shared `source` sibling. +# `source` tracks the desktop-v* train and feeds the client-side packages; +# relay-source tracks relay-v* and is bumped on the server's own schedule, so a +# relay upgrade does not drag the desktop CLI and git helpers with it. +# +# Unlike the client packages, `version` here is the crate's real version: +# crates/buzz-relay/Cargo.toml pins 0.2.1 explicitly and documents that it does +# not inherit the workspace 0.1.0, because the relay ships as an independently +# released artifact. +# +# reqwest is configured workspace-wide as +# `{ version = "0.13", features = ["json", "rustls"], default-features = false }` +# (Cargo.toml:102), and 0.13's `rustls` feature implies `__rustls-aws-lc-rs`, +# so aws-lc-sys is compiled from source exactly as in buzz-cli. openssl-sys is +# also in the lockfile but is not reached on this feature resolution. +# aws-lc-sys defaults to its CcBuilder backend; AWS_LC_SYS_CMAKE_BUILDER forces +# the cmake backend instead, matching nixpkgs' own unconditional aws-lc-sys +# crate override (default-crate-overrides.nix:58-62), whose comment notes the +# cc backend fails at least on Darwin. cmake is therefore a build tool here +# rather than the build system, hence dontUseCmakeConfigure. +# +# The wrapper's PATH prefix is load-bearing on two independent paths, and both +# fail closed. +# +# The first is the pre-receive hook. The relay writes a bash script into every +# bare repository it creates (crates/buzz-relay/src/api/git/hook.rs:32-146, +# installed by install_hook at :148-180) whose first line is +# `#!/usr/bin/env bash`. That script is FAIL-CLOSED by construction: it runs +# `set -eo pipefail`, computes an HMAC with `openssl dgst`, POSTs to the +# relay's internal policy endpoint with `curl`, and exits 1 on any non-200, +# timeout or network error. It additionally uses git, sed, date, mktemp and +# sort. A missing tool therefore does not degrade the hook, it rejects every +# push. Upstream's own container encodes the same requirement as a compiled-in +# test: hook.rs:186-206 asserts that the Dockerfile runtime stage installs curl +# and openssl, with the message "relay runtime image must install {tool}; the +# git pre-receive hook uses it and fails closed without it". +# +# `#!/usr/bin/env bash` is rewritten to a store path rather than left alone. +# Under systemd with a hardened unit there is no ambient PATH to find `env`'s +# bash through, and the hook is spawned by git rather than by the wrapper, so +# it does not inherit the wrapper's PATH prefix at the point the shebang is +# resolved. bashNonInteractive is the correct choice over bash: the hook is a +# non-interactive script and the interactive variant only adds readline. +# +# The second path is the relay process itself, which shells out to a bare +# `git` by name (api/git/transport.rs:866, :1266, :1670), so git must be on +# PATH for fetch/push to work at all. +# +# BUZZ_WEB_DIR and BUZZ_ADMIN_WEB_DIR are deliberately NOT set. Upstream's +# container sets them to /srv/buzz/web and /srv/buzz/admin-web +# (Dockerfile:151-152), but those directories hold the *built output* of two +# Vite/React applications: web/ and admin-web/ in the source tree are +# TypeScript sources, and Dockerfile:117-118 runs +# `pnpm install --frozen-lockfile` then `pnpm -C web build && pnpm -C admin-web build` +# to produce web/dist and admin-web/dist. Packaging those would mean vendoring +# a second, JavaScript dependency closure (a 7118-line pnpm-lock.yaml plus two +# pnpm patches) alongside the cargo one, which is out of scope here. +# +# Leaving them unset is safe and is upstream's own source-tree default +# (TESTING.md:285 lists BUZZ_WEB_DIR as "unset (source)"). Setting them to a +# path that does not contain index.html is NOT safe: config.rs:968-975 and +# :947-955 return ConfigError::InvalidValue in that case and the relay refuses +# to start. Unset leaves web_dir = None (config.rs:960-964) and the relay runs +# normally. +# +# What that costs: the bundled invite landing page at /invite/{code} and the +# optional git repository browser (BUZZ_SERVE_GIT_WEB_GUI) are not served, and +# the read-only admin dashboard is unavailable. The admin dashboard is gated on +# BUZZ_ADMIN_HOST regardless (config.rs:930-942), so BUZZ_ADMIN_WEB_DIR is +# never even consulted unless that host is configured. Everything else — the +# WebSocket relay, the REST surface, git push/pull over HTTP, NIP-42/NIP-98 +# auth — is unaffected. +# +# Source: https://github.com/block/buzz +{ + lib, + rustPlatform, + relay-source, + cmake, + makeWrapper, + bashNonInteractive, + cacert, + coreutils, + curl, + gitMinimal, + gnused, + openssl, +}: +rustPlatform.buildRustPackage (finalAttrs: { + pname = "buzz-relay"; + inherit (relay-source) version cargoDeps; + src = relay-source; + + # --bin names the expected binary explicitly. Redundant at this pin, where + # the crate declares exactly one [[bin]] and also sets default-run, but it + # makes an upstream addition of a second binary change the install set + # visibly instead of silently. + cargoBuildFlags = [ + "-p" + "buzz-relay" + "--bin" + "buzz-relay" + ]; + + nativeBuildInputs = [ + cmake + makeWrapper + ]; + dontUseCmakeConfigure = true; + + env.AWS_LC_SYS_CMAKE_BUILDER = "1"; + + # The shebang rewrite is applied to the Rust string literal in the source + # tree, before compilation, because the hook text is compiled into the binary + # as a `const &str`. patchShebangs cannot reach it: there is no such file on + # disk at build time, and the file the relay eventually writes is created at + # runtime, long after any fixup phase. + # + # The substitution is asserted rather than assumed. substituteInPlace does + # not fail when its pattern is absent, so an upstream change to the shebang + # line would otherwise leave a relay that emits `/usr/bin/env bash` hooks and + # fails every push on a machine without that path. + postPatch = '' + substituteInPlace crates/buzz-relay/src/api/git/hook.rs \ + --replace-fail \ + '#!/usr/bin/env bash' \ + '#!${lib.getExe bashNonInteractive}' + ''; + + postInstall = '' + wrapProgram $out/bin/buzz-relay \ + --set-default SSL_CERT_FILE ${cacert}/etc/ssl/certs/ca-bundle.crt \ + --prefix PATH : ${ + lib.makeBinPath [ + bashNonInteractive + coreutils + curl + gitMinimal + gnused + openssl + ] + } + ''; + + doCheck = false; + + doInstallCheck = true; + # There is no flag probe available here. crates/buzz-relay/src/main.rs + # registers no argument parser at all — not clap, not anything — so + # `--help` is silently ignored and the process proceeds to open a Postgres + # pool and eventually fails on a connection timeout. That is exactly the + # situation source/package.nix already documents for the client binaries. + # + # Instead the check drives the configuration validator, which runs at + # main.rs:142 before any database work. Pointing BUZZ_WEB_DIR at a directory + # with no index.html makes Config::from_env return + # ConfigError::InvalidValue (config.rs:968-975), which main.rs turns into a + # non-zero exit with "Invalid configuration" on stderr. This reaches real + # relay code, terminates deterministically, needs no network and no DB, and + # asserts the precise semantics this package's comment relies on when it + # declines to set BUZZ_WEB_DIR: a bad web dir is fatal at startup, so unset + # is the only safe value until the bundles are actually built. + # + # The wrapper and the shebang rewrite are asserted separately, because a + # dropped wrapper or a missed substitution yields a relay that starts and + # serves traffic normally and then rejects every git push at the first hook + # invocation — a failure no startup probe would surface. + # + # The shebang assertion greps the binary, not a file on disk. The hook is a + # compiled-in `const &str` (hook.rs:32) that only reaches the filesystem when + # the relay creates a repository at runtime. + installCheckPhase = '' + runHook preInstallCheck + + grep -qF '${gitMinimal}' "$out/bin/buzz-relay" + grep -qF '${openssl}' "$out/bin/buzz-relay" + grep -qF '${curl}' "$out/bin/buzz-relay" + + grep -qF '#!${lib.getExe bashNonInteractive}' "$out/bin/.buzz-relay-wrapped" + if grep -qF '#!/usr/bin/env bash' "$out/bin/.buzz-relay-wrapped"; then + echo "error: pre-receive hook shebang was not rewritten to a store path" >&2 + exit 1 + fi + + checkdir=$(mktemp -d) + mkdir -p "$checkdir/empty-web" + + # Asserting both the non-zero exit and the message pins the outcome from + # both directions: a non-zero exit alone would be satisfied by a missing + # library or a sandbox permission problem rather than by the validator. + # + # `if ... then exit 1; fi` rather than `! cmd`, because under set -e the + # shell does not exit when a command's status is inverted with `!`, so + # `! "$out/bin/buzz-relay"` would pass silently in the one case this + # exists to catch. + if BUZZ_WEB_DIR="$checkdir/empty-web" \ + "$out/bin/buzz-relay" > "$checkdir/out" 2>&1; then + echo "error: relay accepted a BUZZ_WEB_DIR with no index.html" >&2 + cat "$checkdir/out" >&2 + exit 1 + fi + grep -qF 'does not contain index.html' "$checkdir/out" + + runHook postInstallCheck + ''; + + meta = { + homepage = "https://github.com/block/buzz"; + description = "WebSocket relay server for the Buzz communications platform"; + changelog = "https://github.com/block/buzz/releases/tag/relay-v${finalAttrs.version}"; + license = lib.licenses.asl20; + mainProgram = "buzz-relay"; + maintainers = with lib.maintainers; [ cameronraysmith ]; + platforms = lib.platforms.linux ++ lib.platforms.darwin; + }; +})