Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- **Egress: git knowledge sources and the GitHub token check connect only to the address they checked** ([#19221](https://github.com/hivecommons/hive/issues/19221), part of [#18986](https://github.com/hivecommons/hive/issues/18986)). A git source's host was validated with one DNS lookup and then resolved again by `git` itself on every clone, `ls-remote` and pull, so a name that rebinds between the two was checked as public and fetched as private. Each git command now resolves the host once, refuses the fetch when any answer is private, loopback, link-local or otherwise barred, or when the lookup fails or comes back empty, and pins git to the checked addresses with `-c http.curloptResolve` (git 2.37 or later). Periodic syncs are checked the same way, so a source whose name later moves inward stops syncing. `HIVE_ALLOW_PRIVATE_GIT_SOURCE=true` still allows an internal Git server, but link-local (cloud metadata), multicast and unspecified answers stay refused. The dashboard's GitHub token check now dials through a pinned transport that refuses barred addresses for `api.github.com`, allows a configured API URL to be private while still refusing metadata answers, and refuses cross-host redirects. The hub's mail relay to a hive's self-reported dashboard URL now dials through the same pinned transport as the webhook relay.
17 changes: 17 additions & 0 deletions src/docs/security-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -419,6 +419,23 @@ there. Operator-configured gateway endpoints and the fixed-host `/api/style`
fetch are not pinned: a gateway may legitimately sit on a private address, and
redirects off either are still checked.

The hub's contribute proxies, lite-enrollment repo access checks and mail
relay to a spoke's dashboard, and the dashboard's GitHub token check, dial
through the same pinned transport. The
token check refuses every barred class for public `api.github.com`; for a
configured API URL, which is intentionally chosen and may be private, it still
pins every dial but refuses only link-local (cloud metadata), multicast and
unspecified answers, and it refuses any cross-host redirect so that allowance
never reaches another host. Git knowledge sources run `git` as a subprocess, so
they pin through git instead: each clone, `ls-remote` and pull resolves the
host once, refuses the fetch when any answer is barred or the lookup fails or
comes back empty, and passes the checked addresses to git as
`-c http.curloptResolve=host:port:addr`, so git connects to them and never
looks the name up itself (git 2.37 or later; redirects are already refused with
`http.followRedirects=false`). `HIVE_ALLOW_PRIVATE_GIT_SOURCE=true` allows a
private or loopback Git server; link-local, multicast and unspecified answers
stay refused even then.

**Proxy decisions fail closed.** The egress audit
([#18986](https://github.com/hivecommons/hive/issues/18986), item 5) walked
every branch of the proxy's per-request decision path that can fail and
Expand Down
16 changes: 12 additions & 4 deletions src/pkg/dashboard/api_contribute.go
Original file line number Diff line number Diff line change
Expand Up @@ -1737,14 +1737,22 @@ func validateGitHubToken(token, apiURL string) string {
}
ghTokenCacheMu.RUnlock()

userEndpoint := "https://api.github.com/user"
if apiURL != "" && apiURL != "https://api.github.com" {
userEndpoint := config.DefaultGitHubAPIURL + "/user"
ctx := context.Background()
if apiURL != "" && apiURL != config.DefaultGitHubAPIURL {
userEndpoint = apiURL + "/user"
// A configured API URL is intentionally chosen and may be private;
// githubTokenTransport still pins and checks every dial to it.
ctx = context.WithValue(ctx, githubTokenConfiguredTargetKey{}, true)
}

const tokenValidateTimeout = 10 * time.Second
client := &http.Client{Timeout: tokenValidateTimeout}
req, err := http.NewRequest("GET", userEndpoint, nil)
client := &http.Client{
Timeout: tokenValidateTimeout,
Transport: githubTokenRoundTripper(),
CheckRedirect: githubTokenCheckRedirect,
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, userEndpoint, nil)
if err != nil {
return ""
}
Expand Down
107 changes: 107 additions & 0 deletions src/pkg/dashboard/github_token_transport.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
package dashboard

import (
"context"
"fmt"
"net"
"net/http"
"strings"
"time"

"github.com/hivecommons/hive/pkg/netguard"
)

const (
// githubTokenDialTimeout and githubTokenDialKeepAlive match the
// net.Dialer http.DefaultTransport uses.
githubTokenDialTimeout = 30 * time.Second
githubTokenDialKeepAlive = 30 * time.Second

// githubTokenMaxRedirects bounds a token check's same-host redirects.
githubTokenMaxRedirects = 3
)

// githubTokenConfiguredTargetKey marks a token-check request whose API URL is
// a configured override rather than public api.github.com.
type githubTokenConfiguredTargetKey struct{}

// githubTokenConfiguredRefusedReasons are the address classes refused even
// for a configured API URL. A configured target is intentionally chosen and
// may be private or loopback (an httptest server, an internal mirror), but no
// GitHub API lives on link-local (cloud instance metadata), multicast or an
// unspecified address.
var githubTokenConfiguredRefusedReasons = map[string]bool{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: this map has the same five reasons as gitSourceOptInRefusedReasons (src/pkg/knowledge/gitsource.go:568). Two copies of a security allowlist can drift. Consider exporting one set from netguard (e.g. a DenyReasonIsMetadataClass helper) and using it in both places.

netguard.ReasonLinkLocal: true,
netguard.ReasonMulticast: true,
netguard.ReasonUnspecified: true,
netguard.ReasonThisNetwork: true,
netguard.ReasonBroadcast: true,
}

// githubTokenDial connects to an address githubTokenTransport already
// checked. A package var so TestMain routes it through the suite's
// loopback-only guard and tests can observe where a dial would go.
var githubTokenDial netguard.DialFunc = (&net.Dialer{
Timeout: githubTokenDialTimeout,
KeepAlive: githubTokenDialKeepAlive,
}).DialContext

// githubTokenTransport is the transport for validateGitHubToken (#19221,
// part of #18986). The token check sends a user's bearer token to the API
// host, and a default http.Client resolved the name at dial time with nothing
// checking the answer. This transport resolves each dial once through
// privateURLResolver, checks every answer, fails closed when the lookup
// errors or returns nothing, and connects only to an address it checked.
//
// For public api.github.com every class isPrivateURL bars is refused. For a
// configured API URL (githubTokenConfiguredTargetKey on the request context)
// the target is intentionally chosen, so only
// githubTokenConfiguredRefusedReasons are refused; githubTokenCheckRedirect
// refuses a cross-host redirect, so that allowance never reaches another host.
var githubTokenTransport http.RoundTripper = netguard.PinnedDialer{
Resolve: func(ctx context.Context, host string) ([]string, error) { return privateURLResolver(ctx, host) },
Refuse: githubTokenRefuse,
Dial: func(ctx context.Context, network, addr string) (net.Conn, error) {
return githubTokenDial(ctx, network, addr)
},
}.Transport()

func githubTokenRefuse(ctx context.Context, target string) bool {
configured, _ := ctx.Value(githubTokenConfiguredTargetKey{}).(bool)
if !configured {
return isPrivateURL(ctx, "http://"+target)
}
host, _, err := net.SplitHostPort(target)
if err != nil {
return true
}
addr, ok := netguard.ParseHostLiteral(host)
if !ok {
return true
}
return githubTokenConfiguredRefusedReasons[netguard.DenyReason(addr)]
}

// githubTokenRoundTripper is githubTokenTransport, unless http.DefaultTransport
// was replaced with a non-*http.Transport RoundTripper (tests and
// instrumented builds inject one there, the same convention as the hub's
// pinnedOrReplacedDefault); then that replacement is used.
func githubTokenRoundTripper() http.RoundTripper {
if _, ok := http.DefaultTransport.(*http.Transport); ok {
return githubTokenTransport
}
return http.DefaultTransport
}

// githubTokenCheckRedirect allows only a bounded number of same-host
// redirects. A cross-host hop would carry the configured-target allowance
// and the bearer token to a host nobody configured.
func githubTokenCheckRedirect(req *http.Request, via []*http.Request) error {
if len(via) >= githubTokenMaxRedirects {
return fmt.Errorf("stopped after %d redirects", githubTokenMaxRedirects)
}
if len(via) > 0 && !strings.EqualFold(req.URL.Host, via[0].URL.Host) {
return fmt.Errorf("%w: cross-host redirect to %s", netguard.ErrRefused, req.URL.Host)
}
return nil
}
194 changes: 194 additions & 0 deletions src/pkg/dashboard/github_token_transport_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,194 @@
package dashboard

import (
"context"
"errors"
"net"
"net/http"
"net/http/httptest"
"net/url"
"sync"
"sync/atomic"
"testing"

"github.com/hivecommons/hive/pkg/netguard"
)

// Resolve-once and dial pinning for validateGitHubToken (#19221, part of
// #18986). These tests drive a resolver that answers differently on later
// lookups and a recording dialer, so they never leave the machine.

const (
tokenPinPublicIP = "140.82.112.6"
tokenPinHost = "rebind.example"
)

// stubTokenResolver answers first on the first lookup and later afterwards.
func stubTokenResolver(t *testing.T, first, later []string, err error) *int32 {
t.Helper()
var calls int32
orig := privateURLResolver
privateURLResolver = func(_ context.Context, _ string) ([]string, error) {
if err != nil {
atomic.AddInt32(&calls, 1)
return nil, err
}
if atomic.AddInt32(&calls, 1) == 1 {
return first, nil
}
return later, nil
}
t.Cleanup(func() { privateURLResolver = orig })
return &calls
}

// recordTokenDials replaces githubTokenDial with one that records each
// target. When to is set, the dial connects there instead (an httptest
// server standing in for the checked public address).
func recordTokenDials(t *testing.T, to string) *[]string {
t.Helper()
var mu sync.Mutex
var got []string
orig := githubTokenDial
githubTokenDial = func(ctx context.Context, network, addr string) (net.Conn, error) {
mu.Lock()
got = append(got, addr)
mu.Unlock()
if to == "" {
return nil, errors.New("test dialer: no connection")
}
return (&net.Dialer{}).DialContext(ctx, network, to)
}
t.Cleanup(func() { githubTokenDial = orig })
return &got
}

// The invariant: a name that rebinds between the check and the dial cannot
// reach a private address. The pre-fetch check sees the public answer; the
// dial's own single lookup sees loopback and refuses, and the loopback server
// receives nothing.
func TestGitHubTokenTransport_RebindBetweenCheckAndDialRefused(t *testing.T) {
var hits int32
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
atomic.AddInt32(&hits, 1)
}))
t.Cleanup(srv.Close)
u, err := url.Parse(srv.URL)
if err != nil {
t.Fatal(err)
}
stubTokenResolver(t, []string{tokenPinPublicIP}, []string{"127.0.0.1"}, nil)
// Real dialer, so a dial to the rebound address would reach srv.
orig := githubTokenDial
githubTokenDial = (&net.Dialer{}).DialContext
t.Cleanup(func() { githubTokenDial = orig })

target := "http://" + tokenPinHost + ":" + u.Port() + "/user"
if isPrivateURL(context.Background(), target) {
t.Fatal("pre-fetch check should see the public first answer")
}
req, err := http.NewRequest(http.MethodGet, target, nil)
if err != nil {
t.Fatal(err)
}
resp, err := (&http.Client{Transport: githubTokenTransport}).Do(req)
if err == nil {
resp.Body.Close()
t.Fatal("dial after rebind succeeded")
}
if !errors.Is(err, netguard.ErrRefused) {
t.Fatalf("dial after rebind = %v; want netguard.ErrRefused", err)
}
if n := atomic.LoadInt32(&hits); n != 0 {
t.Fatalf("rebound loopback server received %d request(s)", n)
}
}

// Default api.github.com: the dial connects to the address it checked, from
// one lookup, never to the name.
func TestValidateGitHubToken_DialsCheckedAddressOnce(t *testing.T) {
calls := stubTokenResolver(t, []string{tokenPinPublicIP}, []string{"127.0.0.1"}, nil)
dials := recordTokenDials(t, "")

_ = validateGitHubToken("ghp_pin_checked_once", "")
if got := atomic.LoadInt32(calls); got != 1 {
t.Fatalf("resolver called %d times, want 1", got)
}
if len(*dials) != 1 || (*dials)[0] != net.JoinHostPort(tokenPinPublicIP, "443") {
t.Fatalf("dials = %v, want only the checked %s:443", *dials, tokenPinPublicIP)
}
}

// Default api.github.com: a private answer (any of several), an empty answer
// or a lookup error refuses the dial before any connection is attempted.
func TestValidateGitHubToken_DefaultHostRefusesPrivateAndFailsClosed(t *testing.T) {
cases := []struct {
name string
answers []string
err error
}{
{"loopback", []string{"127.0.0.1"}, nil},
{"any answer private", []string{tokenPinPublicIP, "10.1.2.3"}, nil},
{"metadata", []string{"169.254.169.254"}, nil},
{"empty", nil, nil},
{"lookup error", nil, errors.New("dns unavailable")},
}
for i, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
stubTokenResolver(t, tc.answers, tc.answers, tc.err)
dials := recordTokenDials(t, "")
if got := validateGitHubToken("ghp_pin_refused_"+string(rune('a'+i)), ""); got != "" {
t.Fatalf("username = %q, want refusal", got)
}
if len(*dials) != 0 {
t.Fatalf("dialled %v despite a barred or failed lookup", *dials)
}
})
}
}

// A configured API URL is intentionally chosen and may be private: an
// httptest server on loopback answers. Link-local metadata stays refused.
func TestValidateGitHubToken_ConfiguredTargetMayBePrivateButNotMetadata(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte(`{"login":"octo-pin"}`))
}))
t.Cleanup(srv.Close)
if got := validateGitHubToken("ghp_pin_configured_ok", srv.URL); got != "octo-pin" {
t.Fatalf("configured loopback API: username = %q, want octo-pin", got)
}

stubTokenResolver(t, []string{"169.254.169.254"}, []string{"169.254.169.254"}, nil)
dials := recordTokenDials(t, "")
if got := validateGitHubToken("ghp_pin_configured_md", "http://ghe.internal.example"); got != "" {
t.Fatalf("configured host resolving to metadata: username = %q, want refusal", got)
}
if len(*dials) != 0 {
t.Fatalf("dialled %v for a metadata answer", *dials)
}
}

// The configured-target allowance must not follow a cross-host redirect.
func TestValidateGitHubToken_CrossHostRedirectRefused(t *testing.T) {
var otherHits int32
other := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
atomic.AddInt32(&otherHits, 1)
_, _ = w.Write([]byte(`{"login":"elsewhere"}`))
}))
t.Cleanup(other.Close)
ou, err := url.Parse(other.URL)
if err != nil {
t.Fatal(err)
}
front := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "http://localhost:"+ou.Port()+"/user", http.StatusFound)
}))
t.Cleanup(front.Close)

if got := validateGitHubToken("ghp_pin_redirect", front.URL); got != "" {
t.Fatalf("username = %q after a cross-host redirect, want refusal", got)
}
if n := atomic.LoadInt32(&otherHits); n != 0 {
t.Fatalf("redirect target received %d request(s)", n)
}
}
3 changes: 3 additions & 0 deletions src/pkg/dashboard/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,9 @@ func TestMain(m *testing.M) {
}
http.DefaultTransport = guarded
http.DefaultClient.Transport = guarded
// validateGitHubToken dials through its own pinned transport (#19221);
// route its final connect through the same guard.
githubTokenDial = guarded.DialContext

// Filesystem counterpart of the network guard above: newAuditLog wires a
// lumberjack writer at the package-level auditLogPath (/data/audit.jsonl)
Expand Down
8 changes: 7 additions & 1 deletion src/pkg/hub/mail_relay.go
Original file line number Diff line number Diff line change
Expand Up @@ -90,10 +90,16 @@ func mailChunkBytes() int64 {
var mailDeliveryIDRE = regexp.MustCompile(`^[0-9a-f]{32}$`)

// mailRelayClient never follows redirects, for the reason documented on
// spokeWebhookRelayClient.
// spokeWebhookRelayClient. Like that client, it dials only an address
// spokeWebhookRelayGuard passed, so a self-reported DashboardURL whose name
// rebinds between the guard in handleMailInbound and the dial is refused
// instead of reached (#19221).
var mailRelayClient = &http.Client{
Timeout: mailRelayTimeout,
CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse },
Transport: pinnedFetchTransport(func(ctx context.Context, rawURL string) bool {
return spokeWebhookRelayGuard(ctx, rawURL)
}),
}

func (s *HubServer) registerMailRoutes() {
Expand Down
Loading
Loading