Repository navigation
fix(router): Serve the GoBGP API read-only for the gobgp CLI - #1167
Conversation
galactic-router started its embedded GoBGP servers without a gRPC API, so the gobgp CLI in the galactic-debug image had nothing to connect to. A new GALACTIC_ROUTER_GOBGP_API_DIR setting (--gobgp-api-dir), empty by default, serves each BGPRouter's API on a unix socket named <namespace>_<name>.sock in that directory. A gRPC interceptor passes only an explicit list of read methods (GetBgp, GetTable, WatchEvent and the List* methods) and answers every other method with PermissionDenied, so the socket can't add paths or change peers or policy. A test fails when a GoBGP bump adds a method neither list classifies. The router makes the directory mode 0700 and removes a stale socket before each server starts, including after a crash. GoBGP listens in its own goroutine, which Stop does not wait for. A server stopped by Reconfigure before it listened would bind the path late, taking it from the replacement or unlinking its socket. The router therefore waits for each new server's socket to accept a connection before returning it, and logs a socket that never comes up, since GoBGP's own listen error goes to its discarded logger. A socket that can't be bound at all skips the wait, so BGP start is never delayed by it. The base DaemonSet sets /run/galactic-router and mounts it from the host, so every galactic-router and galactic-router-rr pod restarts when this rolls out. A galactic-debug node pod can then run gobgp --target unix:///host/run/galactic-router/<socket>. The galactic-debug README moves gobgp out of "What doesn't work" and shows how to use it. Fixes #1157
The galactic-debug image built the gobgp CLI at v4.9.0 while go.mod has github.com/osrg/gobgp/v4 v4.10.0. Nothing tracked the Dockerfile's GOBGP_VERSION build arg, so it drifted on every go.mod bump. GOBGP_VERSION moves to v4.10.0. A Renovate regex custom manager now reads the arg through a "# renovate:" comment, and a package rule puts it in the same "Go dependencies" group as go.mod's gobgp, so both bump in one pull request and the CLI keeps speaking the embedded server's API. Refs #1157
3cd2226 to
7feeecc
Compare
|
Both review passes ran on head 7feeecc and agreed, so this PR is marked ready and set to auto-merge. First pass, at 3cd2226: pr-adversary returned merge, with one warning and one decision. The warning was a fixed 5s server-start wait when the API socket could not bind; the fix pass skipped that wait when no API is served and added a test for it. The decision was that the reviewer could not run the real gobgp CLI or the Kind e2e itself. The author ran both locally, and a person chose to proceed on that basis. pr-conventions-reviewer returned merge, with three warnings and two nits: split the commit, state the shared-base rollout, check the renovate.json overlap with #1114, plus identifiers in the test plan and a title that did not match the subject. The fix pass split the work into two signed commits, stated that the router pods restart, reworded the test plan, and matched the title to the commit. The #1114 overlap is in a different hunk, so both can land. Second pass, at 7feeecc: pr-rereviewer returned merge and confirmed all five fixes. It reverted the server-start guard in a scratch worktree, and the new test failed after the 5s wait. It found nothing new. pr-conventions-reviewer returned merge with no findings. It rendered the base and both router overlays and found no collision with #1114 or #1109. CI on 7feeecc: all required checks passed, including Build, Lint, Manifests, Unit Tests, Unit Tests (root), Documentation, and E2E Tests. Merge method: merge, the only method the main ruleset allows. Auto-merge waits for one approving review from a code owner, and the push dismisses stale approvals. Code owners were requested. |
ecv
left a comment
There was a problem hiding this comment.
Approve.
Read-only is enforced by an allowlist of 16 Get*/List*/WatchEvent methods in both a unary and a stream interceptor, so every other RPC, including any a GoBGP bump adds, gets PermissionDenied. The socket is a unix socket only, in a root-only 0700 directory, with no TCP listener.
api.goreadOnlyMethods:TestReadOnlyMethodCoversServicewalksGoBgpService_ServiceDescMethods and Streams and fails on any method not classified, so a bump of the pinned v4.10.0 forces a decision.TestAPISocketRefusesWritescovers a unary write and the client-streamingAddPathStream.root.goprepareGoBGPAPIDir:MkdirAllplus an explicitChmod(0700)handles the world-readable dir thatDirectoryOrCreatemakes. The router runs as uid 0 withdrop: ALL, which is enough to own and chmod a dir it created. Only host root (or a privileged pod mounting the host) reaches the sockets.- Without
GALACTIC_ROUTER_GOBGP_API_DIRset, no gRPC listener starts, so no API appears on any other deployment. - Rollout risk, plainly: the DaemonSet gains a hostPath volume, so every galactic-router pod, plain and rr, restarts on the rollout. The DaemonSet sets no
updateStrategy, so the default RollingUpdate withmaxUnavailable: 1applies per DaemonSet. Each restart drops that node's BGP sessions, and nothing ininternal/runtime/gobgpconfigures graceful restart, so routes withdraw until the sessions re-establish. When the bundle version carrying this reaches edge sites via infra, roll one site at a time and watch peer-state metrics between sites. Do not let it ride a tag with other edge changes. - Version match is right:
go.modhasgithub.com/osrg/gobgp/v4 v4.10.0and the debug imageGOBGP_VERSION=v4.10.0. The renovate custom manager regex matches the# renovate:comment followed by theARGline, and the packageRule groups both into "Go dependencies". - The API returns full routing state, including peer passwords and TCP-AO keys, as the docs already say. Root on the node could read these anyway, so acceptable.
Follow-ups:
- Add
updateStrategywith an explicitmaxUnavailableand aminReadySecondsto the router DaemonSet so a rollout paces itself by session re-establishment rather than pod readiness. - Configure BGP graceful restart on the embedded GoBGP peers so a router restart stops being a route withdrawal.
- Confirm the Renovate regex with a dry run (
renovate --platform=localor the Dependency Dashboard) after merge. Thegodatasource lookup for a/v4module path is not exercised by CI here.
The gobgp CLI in the debug image couldn't reach galactic-router, because the router served no API for it. The router now serves a read-only API to root on each node, so on-call can read the live BGP tables from a debug pod and can't write to them. Every router pod restarts on rollout, since the DaemonSet gains a host directory for the socket. The commit messages have the design.
Test plan
Fixes #1157