Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/fixed-18992-pin-server-side-fetch-address.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- Server-side fetches of user- or spoke-chosen URLs (dashboard agent import and workflow spec URLs, the hub's hive config proxy and webhook relay to a spoke's dashboard URL, knowledge document imports) now resolve each connection once, refuse it when any DNS answer is private, loopback, link-local or otherwise not globally routable, and dial the address that was checked. A name that rebinds between the SSRF check and the dial is refused instead of reached, and every redirect hop is checked at dial time ([#18992](https://github.com/hivecommons/hive/issues/18992), part of [#18986](https://github.com/hivecommons/hive/issues/18986)).
16 changes: 16 additions & 0 deletions src/docs/security-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -364,6 +364,22 @@ on the same evidence as a literal IP. Set
destination is intended; it lifts the guard for all three legs at once, and the
proxy warns at boot when it is set.

**Server-side fetches of user- or spoke-chosen URLs.** The dashboard's agent
import and workflow spec fetches, the hub's hive config proxy and webhook
relay to a spoke's self-reported dashboard URL, and knowledge document imports
check the URL before fetching and then dial through a pinned transport
(`netguard.PinnedDialer`): the name is resolved once per connection, every
answer must pass the same classifier, and the connection goes to an address
that was checked, never to the name. A name that rebinds between the check and
the dial is refused, and because the check runs per connection it covers
every redirect hop as well. Two limits are deliberate. When an
`HTTP(S)_PROXY` applies, the connection is to the operator's proxy and the
proxy resolves the target, the same limit as the agent proxy's chained
upstream leg, so the pre-fetch check and the redirect policy are the controls
there. Operator-configured gateway endpoints and the fixed-host `/api/style`
fetch are not pinned: a gateway may legitimately sit on a private address, and
redirects off either are still checked.

**Hub-to-spoke channel.** Heartbeats carry a per-hive bearer key derived from
the hub master secret and are compared in constant time. The payload is
telemetry only: health, version, agent and governor summaries and 24-hour
Expand Down
1 change: 1 addition & 0 deletions src/pkg/dashboard/api_agents.go
Original file line number Diff line number Diff line change
Expand Up @@ -346,6 +346,7 @@ func (s *Server) handleAgentImport(w http.ResponseWriter, r *http.Request) {
client := &http.Client{
Timeout: importHTTPTimeoutS * 1e9, // nanoseconds
CheckRedirect: noRedirectToPrivate,
Transport: ssrfPinnedTransport,
}
resp, err := client.Get(body.URL)
if err != nil {
Expand Down
2 changes: 1 addition & 1 deletion src/pkg/dashboard/api_workflows_ingest_sources.go
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,7 @@ func (s *Server) loadWorkflowSpecText(ctx context.Context, req workflowIngestSpe
}

// workflowSpecHTTP is the client for URL specs; a seam for tests.
var workflowSpecHTTP = &http.Client{Timeout: workflowSpecURLTimeout, CheckRedirect: noRedirectToPrivate}
var workflowSpecHTTP = &http.Client{Timeout: workflowSpecURLTimeout, CheckRedirect: noRedirectToPrivate, Transport: ssrfPinnedTransport}

func fetchWorkflowSpecURL(ctx context.Context, rawURL string) (string, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, rawURL, nil)
Expand Down
12 changes: 10 additions & 2 deletions src/pkg/dashboard/import_test_transport_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,14 +37,22 @@ func routeImportFetchesToServer(t *testing.T, ts *httptest.Server) {
"gist.github.com": {"140.82.121.3"},
})
previous := http.DefaultTransport
http.DefaultTransport = roundTripFunc(func(req *http.Request) (*http.Response, error) {
rewrite := roundTripFunc(func(req *http.Request) (*http.Response, error) {
rewritten := req.Clone(req.Context())
rewritten.URL.Scheme = target.Scheme
rewritten.URL.Host = target.Host
rewritten.Host = req.URL.Host
return previous.RoundTrip(rewritten)
})
t.Cleanup(func() { http.DefaultTransport = previous })
http.DefaultTransport = rewrite
// The import handler dials through the pinned transport (#18992), not
// http.DefaultTransport, so route that seam the same way.
previousPinned := ssrfPinnedTransport
ssrfPinnedTransport = rewrite
t.Cleanup(func() {
http.DefaultTransport = previous
ssrfPinnedTransport = previousPinned
})
}

// Guard against the helper above being mistaken for a production bypass: the
Expand Down
21 changes: 21 additions & 0 deletions src/pkg/dashboard/ssrf_guard.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,31 @@ import (
"fmt"
"net/http"
"strings"

"github.com/hivecommons/hive/pkg/netguard"
)

const ssrfMaxRedirects = 3

// ssrfPinnedTransport is the transport for every fetch whose URL a user or a
// remote party chooses and that isPrivateURL already checked (#18992). The
// pre-fetch check resolves the name, and a default http.Client resolves it
// again when it dials, so a rebinding name passed the check as public and was
// connected to as private. This transport resolves each dial once through
// privateURLResolver, runs isPrivateURL on every answer (so the dial-time
// check is the pre-fetch check, test exemptions included), and connects to
// an address it checked. It runs for every redirect hop's connection too.
//
// Not used for operator-configured gateway endpoints (governor probes, model
// discovery), which may legitimately be private, nor for /api/style, whose
// host is the fixed raw.githubusercontent.com; a redirect off either is still
// bounded by noRedirectToPrivate. A package var so import tests can route the
// same seam to an httptest server.
var ssrfPinnedTransport http.RoundTripper = netguard.PinnedDialer{
Resolve: func(ctx context.Context, host string) ([]string, error) { return privateURLResolver(ctx, host) },
Refuse: func(ctx context.Context, target string) bool { return isPrivateURL(ctx, "http://"+target) },
}.Transport()

// noRedirectToPrivate returns an http.Client CheckRedirect function that
// rejects any redirect whose target resolves to a private/internal host.
// It also limits the redirect chain depth to ssrfMaxRedirects. Use this
Expand Down
83 changes: 83 additions & 0 deletions src/pkg/dashboard/ssrf_pinned_transport_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
package dashboard

import (
"context"
"errors"
"net"
"net/http"
"net/http/httptest"
"strings"
"sync"
"sync/atomic"
"testing"

"github.com/hivecommons/hive/pkg/netguard"
)

// stubRebindingImportResolver answers host with a public address on the first
// lookup and loopback afterwards: the pre-fetch guard sees public, a client
// that resolves again sees 127.0.0.1 (#18992).
func stubRebindingImportResolver(t *testing.T, host string) {
t.Helper()
var mu sync.Mutex
n := 0
orig := privateURLResolver
privateURLResolver = func(_ context.Context, h string) ([]string, error) {
if !strings.EqualFold(h, host) {
return nil, errors.New("no such host")
}
mu.Lock()
defer mu.Unlock()
n++
if n == 1 {
return []string{"93.184.216.34"}, nil
}
return []string{"127.0.0.1"}, nil
}
t.Cleanup(func() { privateURLResolver = orig })
}

func TestSSRFPinnedTransport_RefusesRebindingAfterPreflight(t *testing.T) {
var reached atomic.Bool
srv := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) { reached.Store(true) }))
defer srv.Close()
_, port, _ := net.SplitHostPort(strings.TrimPrefix(srv.URL, "http://"))
stubRebindingImportResolver(t, "agents.rebind.example")

target := "http://agents.rebind.example:" + port + "/agent.yaml"
if err := validateImportFetchURL(context.Background(), target); err != nil {
t.Fatalf("preflight should pass on the first (public) answer: %v", err)
}
client := &http.Client{CheckRedirect: noRedirectToPrivate, Transport: ssrfPinnedTransport}
resp, err := client.Get(target)
if err == nil {
_ = resp.Body.Close()
}
if !errors.Is(err, netguard.ErrRefused) || reached.Load() {
t.Fatalf("rebound import fetch: err=%v reached=%v; want netguard.ErrRefused and no fetch", err, reached.Load())
}
}

// The pinned transport's dial-time check is isPrivateURL itself, so the
// exact host:port test exemption reaches an httptest server and nothing else.
func TestSSRFPinnedTransport_UsesIsPrivateURLAtDialTime(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(http.StatusNoContent) }))
defer srv.Close()
client := &http.Client{Transport: ssrfPinnedTransport}

if resp, err := client.Get(srv.URL); err == nil {
_ = resp.Body.Close()
t.Fatal("pinned transport dialled loopback without an exemption")
} else if !errors.Is(err, netguard.ErrRefused) {
t.Fatalf("err = %v, want netguard.ErrRefused", err)
}

prev := privateURLTestExemptHostPorts
privateURLTestExemptHostPorts = map[string]struct{}{strings.TrimPrefix(srv.URL, "http://"): {}}
t.Cleanup(func() { privateURLTestExemptHostPorts = prev })
resp, err := client.Get(srv.URL)
if err != nil {
t.Fatalf("exempted loopback test server: %v", err)
}
_ = resp.Body.Close()
}
128 changes: 128 additions & 0 deletions src/pkg/hub/pinned_fetch_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
package hub

import (
"bytes"
"context"
"errors"
"net"
"net/http"
"net/http/httptest"
"strings"
"sync"
"sync/atomic"
"testing"

"github.com/hivecommons/hive/pkg/netguard"
)

// stubRebindingResolver answers host with public on the first lookup and
// loopback on every later one, the DNS rebinding shape #18992 closes: the
// SSRF guard's lookup sees a public address, the client's own lookup sees
// 127.0.0.1. It returns a lookup counter for host.
func stubRebindingResolver(t *testing.T, host string) func() int {
t.Helper()
var mu sync.Mutex
n := 0
orig := privateURLResolver
privateURLResolver = func(_ context.Context, h string) ([]string, error) {
if !strings.EqualFold(h, host) {
return nil, errors.New("no such host")
}
mu.Lock()
defer mu.Unlock()
n++
if n == 1 {
return []string{publicTestResolvedAddr}, nil
}
return []string{"127.0.0.1"}, nil
}
t.Cleanup(func() { privateURLResolver = orig })
return func() int {
mu.Lock()
defer mu.Unlock()
return n
}
}

func loopbackPort(t *testing.T, srv *httptest.Server) string {
t.Helper()
_, port, err := net.SplitHostPort(strings.TrimPrefix(srv.URL, "http://"))
if err != nil {
t.Fatal(err)
}
return port
}

// The hive config proxy checks the self-reported DashboardURL with the real
// guard and then fetches it. The fetch must resolve through the same seam and
// dial only a checked address, so the rebound loopback answer is refused.
func TestHandleProxyHiveConfigPinsResolvedAddress(t *testing.T) {
var reached atomic.Bool
upstream := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) { reached.Store(true) }))
defer upstream.Close()
lookups := stubRebindingResolver(t, "spoke.rebind.example")

mux := proxyConfigServer(t, "alice", "http://spoke.rebind.example:"+loopbackPort(t, upstream))
req := httptest.NewRequest("GET", "/api/saas/hive-config/cfg-hive", nil)
req.AddCookie(testAuthCookie("alice"))
w := httptest.NewRecorder()
mux.ServeHTTP(w, req)

if reached.Load() {
t.Fatal("hive config proxy reached the rebound loopback address")
}
if w.Code != http.StatusBadGateway {
t.Fatalf("rebound fetch: expected 502, got %d (body: %s)", w.Code, w.Body.String())
}
if got := lookups(); got < 2 {
t.Fatalf("resolver consulted %d times, want the guard's lookup and the pinned dial's own", got)
}
}

func TestSpokeWebhookRelayClientPinsResolvedAddress(t *testing.T) {
var reached atomic.Bool
spoke := httptest.NewServer(http.HandlerFunc(func(http.ResponseWriter, *http.Request) { reached.Store(true) }))
defer spoke.Close()
stubRebindingResolver(t, "relay.rebind.example")

dashboardURL := "http://relay.rebind.example:" + loopbackPort(t, spoke)
if spokeWebhookRelayGuard(context.Background(), dashboardURL) {
t.Fatal("the guard's lookup should see the first, public answer")
}
req, err := http.NewRequest(http.MethodPost, dashboardURL+spokeWebhookPath, bytes.NewReader([]byte("{}")))
if err != nil {
t.Fatal(err)
}
resp, err := spokeWebhookRelayClient.Do(req)
if err == nil {
_ = resp.Body.Close()
}
if !errors.Is(err, netguard.ErrRefused) || reached.Load() {
t.Fatalf("relay: err=%v reached=%v; want netguard.ErrRefused and no delivery", err, reached.Load())
}
}

// The relay tests swap spokeWebhookRelayGuard to reach an httptest spoke; the
// pinned transport must honour that seam rather than capture the original.
func TestSpokeWebhookRelayClientHonoursGuardSeam(t *testing.T) {
var reached atomic.Bool
spoke := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
reached.Store(true)
w.WriteHeader(http.StatusOK)
}))
defer spoke.Close()
allowLoopbackRelay(t)

req, err := http.NewRequest(http.MethodPost, spoke.URL+spokeWebhookPath, bytes.NewReader([]byte("{}")))
if err != nil {
t.Fatal(err)
}
resp, err := spokeWebhookRelayClient.Do(req)
if err != nil {
t.Fatalf("relay with guard overridden: %v", err)
}
_ = resp.Body.Close()
if !reached.Load() {
t.Fatal("relay did not reach the allowed spoke")
}
}
9 changes: 9 additions & 0 deletions src/pkg/hub/saas_hives_api.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package hub

import (
"context"
_ "embed"
"encoding/json"
"fmt"
Expand Down Expand Up @@ -2127,6 +2128,13 @@ func (s *HubServer) pushVisibilityToSpoke(id string, isPublic bool) {
// loopback httptest server; production always uses isPrivateURL.
var hiveConfigSSRFGuard = isPrivateURL

// hiveConfigTransport dials only an address hiveConfigSSRFGuard passed, so a
// DashboardURL whose name rebinds after the guard ran is refused at dial time
// (#18992).
var hiveConfigTransport = pinnedFetchTransport(func(ctx context.Context, rawURL string) bool {
return hiveConfigSSRFGuard(ctx, rawURL)
})

func (s *HubServer) handleProxyHiveConfig(w http.ResponseWriter, r *http.Request) {
hiveID := r.PathValue("hiveID")
caller := s.getAuthUser(r)
Expand Down Expand Up @@ -2173,6 +2181,7 @@ func (s *HubServer) handleProxyHiveConfig(w http.ResponseWriter, r *http.Request
// Do NOT follow redirects — a 30x could send us from a public
// DashboardURL to an internal host, re-opening the SSRF the guard closes.
CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse },
Transport: hiveConfigTransport,
}
resp, err := client.Get(strings.TrimRight(dashURL, "/") + "/api/config/download")
if err != nil {
Expand Down
16 changes: 16 additions & 0 deletions src/pkg/hub/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -5165,6 +5165,22 @@ func isPrivateURL(ctx context.Context, rawURL string) bool {
return netguard.IsPrivateURL(ctx, rawURL, netguard.Resolver(privateURLResolver))
}

// pinnedFetchTransport returns the transport for a fetch whose URL a remote
// party chose and that guard already checked (#18992). The pre-fetch guard
// resolves the name, and a default http.Client resolves it again when it
// dials, so a self-reported DashboardURL whose name rebinds passed the guard
// as public and was connected to as private. This transport resolves each
// dial once through privateURLResolver, runs guard on every answer, and
// connects to an address it checked, for every connection including redirect
// hops. guard is called on each dial, not captured, so a test that swaps the
// guard var behind it reaches its loopback server through the same path.
func pinnedFetchTransport(guard func(ctx context.Context, rawURL string) bool) *http.Transport {
return netguard.PinnedDialer{
Resolve: func(ctx context.Context, host string) ([]string, error) { return privateURLResolver(ctx, host) },
Refuse: func(ctx context.Context, target string) bool { return guard(ctx, "http://"+target) },
}.Transport()
}

func clampInt(v, min, max int) int {
if v < min {
return min
Expand Down
7 changes: 7 additions & 0 deletions src/pkg/hub/webhook.go
Original file line number Diff line number Diff line change
Expand Up @@ -123,8 +123,15 @@ var spokeWebhookRelayGuard = isPrivateURL
// answers 30x to an internal host would otherwise re-open the SSRF the guard
// above closes. The spoke receiver is a fixed path, so a redirect is never
// legitimate here.
//
// Its transport dials only an address spokeWebhookRelayGuard passed, so a
// DashboardURL whose name rebinds between the guard and the dial is refused
// instead of reached (#18992).
var spokeWebhookRelayClient = &http.Client{
CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse },
Transport: pinnedFetchTransport(func(ctx context.Context, rawURL string) bool {
return spokeWebhookRelayGuard(ctx, rawURL)
}),
}

// relayWebhookToSpoke forwards a verified PR-shaped delivery, byte for byte
Expand Down
1 change: 1 addition & 0 deletions src/pkg/knowledge/connector_support_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,7 @@ func TestFetchDocumentFacts_URL(t *testing.T) {
_, _ = w.Write([]byte("<html><head><title>Page Title</title></head><body><p>Body text here.</p></body></html>"))
}))
defer srv.Close()
allowDocFetchTo(t, srv)

facts, title, err := FetchDocumentFacts(context.Background(), DocSourceConfig{URL: srv.URL + "/doc"}, "", "", nil)
if err != nil {
Expand Down
Loading
Loading