Repository navigation
feat: Run Datum Connect as a headless tunnel endpoint - #192
Merged
Merged
Conversation
Datum Compute wants to run `datum-connect serve` in a container as a credential-less tunnel endpoint. As PID 1 it never received Ctrl-C, so a pod stop hung until SIGKILL and relays kept routing to a dead endpoint; its key also had to be copied into the writable repo, and adding the proxy needed a separate `add tcp-proxy` step before `serve`. Key changes: - Exit on SIGTERM as well as SIGINT and shut down the router, closing the endpoint so relays drop it at once - Add --listen-key-file (DATUM_CONNECT_LISTEN_KEY_FILE) to read the 32-byte identity key from a read-only path, such as a mounted Secret - Add --tcp-proxy (DATUM_CONNECT_TCP_PROXIES) to enable a proxy at startup; repeated starts reuse the existing entry instead of duplicating it - Default CLI logging to info with RUST_LOG overrides, matching the desktop app
The repo had no container image for the CLI after the stale gateway image was removed in #185. Compute cells need one to run `serve` as a tunnel endpoint. Key changes: - Add a Dockerfile that builds only the CLI and runs `serve` as the non-root user 65532, with its repo at /var/lib/datum-connect - Publish ghcr.io/datum-cloud/datum-connect:<tag> on release tag pushes and from the manual release, whose GITHUB_TOKEN-created tag does not trigger the tag push workflow - Build the image without pushing on PRs that touch the CLI or lib The image never uses the iroh-gateway name, which datum-cloud/iroh-gateway publishes on its own.
3 of 4 tasks
scotwells
added a commit
to datum-cloud/compute
that referenced
this pull request
Sep 30, 2026
A cell opts into shell sessions by deploying the agent beside Datum Connect tunnel endpoints, so the agent is reachable only through its paired endpoint and the endpoint holds no cluster credential. Key changes: - Two two-replica StatefulSets in compute-shell-system, paired by ordinal through a headless Service - Least-privilege RBAC for the agent; the endpoint mounts no service account token and reads its key read-only - Portable NetworkPolicies, and Cilium policies in their own component that also police traffic to the node's apiserver - An opt-in overlay that keeps the namespace intact, since the cell overlay renames every namespace it renders The datum-connect image tag is a placeholder until datum-cloud/app#192 is released.
The CLI now runs as the internet-facing tunnel endpoint for shell sessions, so it should not attach default PII to Sentry events, and cells on arm64 need an image too. Key changes: - Set send_default_pii to false for the CLI; it never signs a user in, and no build bakes a Sentry DSN into it today - Build linux/amd64 and linux/arm64 by cross-compiling on the build platform rather than compiling Rust under QEMU; QEMU only runs the runtime stage's package install - Stamp the release tag into the CLI and lib versions, so the binary reports its real version instead of 0.1.0
scotwells
marked this pull request as ready for review
September 30, 2026 01:33
scotwells
added a commit
to datum-cloud/compute
that referenced
this pull request
Sep 30, 2026
A cell opts into shell sessions by deploying the agent beside Datum Connect tunnel endpoints, so the agent is reachable only through its paired endpoint and the endpoint holds no cluster credential. Key changes: - Two two-replica StatefulSets in compute-shell-system, paired by ordinal through a headless Service - Least-privilege RBAC for the agent; the endpoint mounts no service account token and reads its key read-only - Portable NetworkPolicies, and Cilium policies in their own component that also police traffic to the node's apiserver - An opt-in overlay that keeps the namespace intact, since the cell overlay renames every namespace it renders The datum-connect image tag is a placeholder until datum-cloud/app#192 is released.
scotwells
added a commit
to datum-cloud/compute
that referenced
this pull request
Sep 30, 2026
A cell opts into shell sessions by deploying the agent beside Datum Connect tunnel endpoints, so the agent is reachable only through its paired endpoint and the endpoint holds no cluster credential. Key changes: - Two two-replica StatefulSets in compute-shell-system, paired by ordinal through a headless Service - Least-privilege RBAC for the agent; the endpoint mounts no service account token and reads its key read-only - Portable NetworkPolicies, and Cilium policies in their own component that also police traffic to the node's apiserver - An opt-in overlay that keeps the namespace intact, since the cell overlay renames every namespace it renders The datum-connect image tag is a placeholder until datum-cloud/app#192 is released.
scotwells
added a commit
to datum-cloud/compute
that referenced
this pull request
Sep 30, 2026
A cell opts into shell sessions by deploying the agent beside Datum Connect tunnel endpoints, so the agent is reachable only through its paired endpoint and the endpoint holds no cluster credential. Key changes: - Two two-replica StatefulSets in compute-shell-system, paired by ordinal through a headless Service - Least-privilege RBAC for the agent; the endpoint mounts no service account token and reads its key read-only - Portable NetworkPolicies, and Cilium policies in their own component that also police traffic to the node's apiserver - An opt-in overlay that keeps the namespace intact, since the cell overlay renames every namespace it renders The datum-connect image tag is a placeholder until datum-cloud/app#192 is released.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Datum Connect can now run as a headless tunnel endpoint in a container, so Datum Compute can reuse it for instance shell sessions (design: datum-cloud/compute#384). The endpoint stops cleanly the moment its pod is told to, so relays stop routing to it right away instead of after a kill timeout. It can use an identity key mounted read-only from a secret and be told which service to expose at startup, so a pod needs no setup step. Each release now also publishes the command line tool as its own container image.
What changed
serveexits on SIGTERM as well as Ctrl-C and shuts down the endpoint, so relays drop it at once. As PID 1 in a container it previously ignored SIGTERM and was killed after the grace period.serve --listen-key-file(orDATUM_CONNECT_LISTEN_KEY_FILE) reads the 32-byte identity key from a read-only path instead of the writable repo.serve --tcp-proxy HOST:PORT(orDATUM_CONNECT_TCP_PROXIES, comma-separated) enables a proxy at startup; restarts reuse the existing entry.infoand honoursRUST_LOG, as the desktop app already does.Dockerfilebuilds only the CLI for linux/amd64 and linux/arm64 and runsserveas non-root user 65532, repo at/var/lib/datum-connect. Rust is cross-compiled on the build host; QEMU only runs the runtime stage's package install. Release builds stamp the tag into the binary's version. No secrets or relay lists are baked in; relays come fromDATUM_CONNECT_RELAY_URLSat runtime.ghcr.io/datum-cloud/datum-connect:<tag>: on tag pushes, and from the manual release (its tag is created withGITHUB_TOKEN, which does not trigger tag-push workflows). PRs touching the CLI build the image without pushing.Image name
The stale image this repo used to push as
ghcr.io/datum-cloud/iroh-gatewaywas already removed in #185;datum-cloud/iroh-gatewayis its only publisher. This PR publishes under a new name,datum-connect, and leaves that name alone.Test plan
cargo fmt --check --all,cargo check --locked, CI's clippy invocation (no new warnings) andcargo test -p lib --locked(58 passed, including a new idempotency test)serveran as uid 65532 on a read-only root filesystem with a read-only mounted key, and kept the same endpoint ID across runs with that keydocker stop(SIGTERM) exited 0 in 0.2s; SIGINT also exited 0; a 31-byte key fails with a clear errorghcr.io/datum-cloud/datum-connectand the package is visible to compute cellsRelated to datum-cloud/enhancements#900