Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/added-hub-spoke-repo-sync.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- Hubs can opt in to spoke repository sync: a registration key updates only its own repo list, at least one proven repo must remain active, and new repos stay pending until their owner proves them ([#19236](https://github.com/hivecommons/hive/issues/19236)).
3 changes: 2 additions & 1 deletion src/docs/api-reference/hub-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,8 @@ Part of the [Dashboard REST API reference](../api-reference.md); see that page f
| `POST` | `/api/hub/v1/registrations` | Hub auth | Register a self-hosted hive as a resource ([#18642](https://github.com/hivecommons/hive/issues/18642)): the same body, proof and refusals as `POST /api/hub/selfservice/hives` (including `429` over `HIVE_HUB_SELFSERVICE_MAX_PER_USER`), except that a repo list over `HIVE_HUB_SELFSERVICE_MAX_REPOS` answers `413 {error, limit: "max_repos", max_repos}` instead of `400`. `201 {registration, key, hub_url, snippets}` with `Location` and a strong `ETag` header; `registration` is the legacy view plus `etag` and `repo_proofs[{repo, checked_at}]` (when the owner's admin permission on each repo was last proven). The raw key appears once. `404` while `HIVE_HUB_SELFSERVICE_REGISTRATION` is off | `pkg/hub/selfservice_v1_api.go` |
| `GET` | `/api/hub/v1/registrations` | Hub auth | The caller's own registrations: `{registrations[]}`, each with its `etag` and `repo_proofs`; never a key or a hash | `pkg/hub/selfservice_v1_api.go` |
| `GET` | `/api/hub/v1/registrations/{id}` | Hub auth | One of the caller's registrations: `{registration}` with a strong `ETag` header; `If-None-Match` with the current ETag answers `304`. Another user's hive, a missing hive and a hub admin who does not own it all get `404`. The ETag comes from a revision counter that heartbeats do not bump, and reveals nothing about the key | `pkg/hub/selfservice_v1_api.go` |
| `PATCH` | `/api/hub/v1/registrations/{id}` | Hub auth | Owner only, by hub session (a registration key is not accepted): change `repos` and/or `dashboard_url` in place with the heartbeat key kept valid ([#18642](https://github.com/hivecommons/hive/issues/18642)). Body: `repos` is a full list (replacement) or `{add[], remove[]}`; `dashboard_url` is an `https` URL or `""` to clear it. `If-Match` is required: `428` when absent, `412` carrying the current `ETag` when stale (checked before any GitHub call and again under the store lock). Only added repos are proven (`403` naming the first refused repo, nothing applied); kept repos keep their proof time and removed repos lose theirs. `org`, the owner, the ID and the keys are not editable: a body naming `org` or any other field is `400`, as is a list that would leave no repo. Over `HIVE_HUB_SELFSERVICE_MAX_REPOS` is `413 {error, limit: "max_repos", max_repos}`. Removing a repo the registration does not list is ignored; an edit that changes nothing answers `200` with the current `ETag` and does not bump the revision. `200 {registration}` with the new `ETag`; the key never appears. `404` for a missing hive, another user's hive and a hub admin who does not own it; `404` while `HIVE_HUB_SELFSERVICE_REGISTRATION` is off | `pkg/hub/selfservice_v1_patch.go` |
| `PATCH` | `/api/hub/v1/registrations/{id}` | Hub auth | Owner only, by hub session: change `repos` and/or `dashboard_url` in place with the heartbeat key kept valid ([#18642](https://github.com/hivecommons/hive/issues/18642)). Body: `repos` is a full list (replacement) or `{add[], remove[]}`; `dashboard_url` is an `https` URL or `""` to clear it. `If-Match` is required: `428` when absent, `412` carrying the current `ETag` when stale (checked before any GitHub call and again under the store lock). Only added repos are proven (`403` naming the first refused repo, nothing applied); kept repos keep their proof time and removed repos lose theirs. Owners prove repos in `pending_proof` by adding them here. `org`, the owner, the ID and the keys are not editable: a body naming `org` or any other field is `400`, as is a list that would leave no repo. Over `HIVE_HUB_SELFSERVICE_MAX_REPOS` is `413 {error, limit: "max_repos", max_repos}`. Removing a repo the registration does not list is ignored; an edit that changes nothing answers `200` with the current `ETag` and does not bump the revision. `200 {registration}` with the new `ETag`; the key never appears. `404` for a missing hive, another user's hive and a hub admin who does not own it; `404` while `HIVE_HUB_SELFSERVICE_REGISTRATION` is off | `pkg/hub/selfservice_v1_patch.go` |
| `PATCH` | `/api/hub/v1/registrations/{id}` | Own registration bearer | `{"op":"sync-repos","repos":["owner/name"]}` replaces the desired list with the supplied nonempty list, up to `max_repos`. Retained proven repos stay active; other requested repos appear only in `registration.pending_proof` until the owner proves them. Omitted repos are removed. No GitHub calls. Requires `If-Match` (send `*` for the first sync or use the returned `ETag`; `428` missing, `412` stale). `200 {registration}` with its resulting `ETag`; `400` for a list that keeps no proven repo or for other fields or operations; `404` for a registration key used on a foreign or missing ID. Discovery advertises `sync-repos` only when `HIVE_HUB_SELFSERVICE_SYNC_REPOS` is on (off by default; also requires self-service registration to be enabled) | `pkg/hub/selfservice_v1_sync.go` |
| `POST` | `/api/hub/v1/registrations/{id}/keys` | Hub auth | Owner only: rotate the heartbeat key; the previous key keeps authenticating for the 10 minute overlap. `201 {id, key, hub_url, overlap, snippets}` (the legacy rotate answers `200`) with the new `ETag`; the raw key appears once. Optional `If-Match` (`412` when stale). A body asking for a grace window is refused with `400`: caller-chosen grace is a later rollout step | `pkg/hub/selfservice_v1_api.go` |
| `DELETE` | `/api/hub/v1/registrations/{id}` | Hub auth | Owner or hub admin: revoke the key and remove the registry entry. `200 {removed, id}`. Optional `If-Match` (`412` when stale, carrying the current `ETag`) | `pkg/hub/selfservice_v1_api.go` |
| `POST` | `/api/hub/selfservice/hives/{id}/rotate` | Hub auth | Owner only: mint a new heartbeat key for `{id}`; the previous key keeps authenticating for `overlap` (10 minutes). `200 {id, key, hub_url, overlap, snippets}`, raw key once. Another user's hive, a missing hive and a hub admin who does not own it all get `404` | `pkg/hub/server.go:1794` |
Expand Down
11 changes: 10 additions & 1 deletion src/pkg/hub/selfservice_registry.go
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,9 @@ type SelfServiceHive struct {
// removes. Records from before the field have no entries; their repos
// were all proven at OwnershipCheckedAt.
RepoProofs map[string]time.Time `json:"repo_proofs,omitempty"`
// PendingProof is the spoke's requested but unproven repos. Only Repos
// grants registry visibility; pending entries are kept for owner review.
PendingProof []string `json:"pending_proof,omitempty"`
// LastAcceptedHeartbeatAt is the last heartbeat that passed the bearer
// check. Zero until the first one; the expiry sweep (selfservice_expiry.go)
// counts the window from here, or from CreatedAt while it is zero.
Expand Down Expand Up @@ -393,6 +396,7 @@ func (st *selfServiceStore) update(id, ifMatch string, mutate func(*SelfServiceH
before := *h
next := *h
next.Repos = append([]string(nil), h.Repos...)
next.PendingProof = append([]string(nil), h.PendingProof...)
next.RepoProofs = make(map[string]time.Time, len(h.RepoProofs))
for k, v := range h.RepoProofs {
next.RepoProofs[k] = v
Expand Down Expand Up @@ -456,7 +460,12 @@ func (st *selfServiceStore) verify(id, presented string) bool {
if !ok {
return false
}
hash := hashSelfServiceKey(presented)
return st.matchesKey(*h, hashSelfServiceKey(presented))
}

// matchesKey applies the heartbeat key policy to a record already loaded
// under the store lock. Mutations must check the credential under that lock.
func (st *selfServiceStore) matchesKey(h SelfServiceHive, hash string) bool {
if h.KeyHash != "" && secureCompareHub(hash, h.KeyHash) {
return true
}
Expand Down
26 changes: 18 additions & 8 deletions src/pkg/hub/selfservice_v1_api.go
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ type hubV1Operation struct {
IfMatch string `json:"if_match,omitempty"`
}

// hubV1Operations is what this hub implements. Discovery lists exactly this.
// hubV1Operations are the operations available on every enabled hub.
var hubV1Operations = []hubV1Operation{
{Name: "discovery", Method: http.MethodGet, Path: hubV1BasePath},
{Name: "create", Method: http.MethodPost, Path: hubV1BasePath + "/registrations"},
Expand All @@ -68,15 +68,16 @@ var hubV1Operations = []hubV1Operation{
var hubV1PatchFields = []string{"repos", "dashboard_url"}

// registerHubV1Routes wires the v1 routes onto the hub mux. Discovery is
// public (a spoke reads it before it has a session); every other route is
// behind the hub session like the v6 routes. All of them answer 404 while
// HIVE_HUB_SELFSERVICE_REGISTRATION is off.
// public (a spoke reads it before it has a session); the registration-key
// PATCH lane is opt-in, and the remaining routes require hub auth.
// All routes answer 404 while HIVE_HUB_SELFSERVICE_REGISTRATION is off
// for callers with the appropriate credential.
func (s *HubServer) registerHubV1Routes() {
s.mux.HandleFunc("GET "+hubV1BasePath, s.handleHubV1Discovery)
s.mux.HandleFunc("POST "+hubV1BasePath+"/registrations", s.requireAuth(s.handleHubV1Create))
s.mux.HandleFunc("GET "+hubV1BasePath+"/registrations", s.requireAuth(s.handleHubV1List))
s.mux.HandleFunc("GET "+hubV1BasePath+"/registrations/{id}", s.requireAuth(s.handleHubV1Get))
s.mux.HandleFunc("PATCH "+hubV1BasePath+"/registrations/{id}", s.requireAuth(s.handleHubV1Patch))
s.mux.HandleFunc("PATCH "+hubV1BasePath+"/registrations/{id}", s.routeHubV1Patch)
s.mux.HandleFunc("DELETE "+hubV1BasePath+"/registrations/{id}", s.requireAuth(s.handleHubV1Delete))
s.mux.HandleFunc("POST "+hubV1BasePath+"/registrations/{id}/keys", s.requireAuth(s.handleHubV1RotateKey))
}
Expand All @@ -86,8 +87,9 @@ func (s *HubServer) registerHubV1Routes() {
// proof times.
type hubV1RegistrationView struct {
selfServiceHiveView
ETag string `json:"etag"`
RepoProofs []hubV1RepoProof `json:"repo_proofs"`
ETag string `json:"etag"`
RepoProofs []hubV1RepoProof `json:"repo_proofs"`
PendingProof []string `json:"pending_proof,omitempty"`
}

// hubV1RepoProof is when the owner's admin permission on one repo was last
Expand All @@ -112,6 +114,7 @@ func (s *HubServer) hubV1View(h SelfServiceHive) hubV1RegistrationView {
selfServiceHiveView: selfServiceView(h, s.selfServiceOnline(h.ID)),
ETag: selfServiceETag(h),
RepoProofs: proofs,
PendingProof: h.PendingProof,
}
}

Expand Down Expand Up @@ -139,11 +142,18 @@ func (s *HubServer) handleHubV1Discovery(w http.ResponseWriter, r *http.Request)
if !s.requireSelfServiceEnabled(w) {
return
}
operations := hubV1Operations
if hubV1SyncReposEnabled() {
operations = append(operations, hubV1Operation{
Name: hubV1SyncReposOp, Method: http.MethodPatch,
Path: hubV1BasePath + "/registrations/{id}", IfMatch: "required",
})
}
writeHubV1JSON(w, http.StatusOK, "", map[string]any{
"protocol": hubV1Protocol,
"version": hubV1ProtocolVersion,
"base_path": hubV1BasePath,
"operations": hubV1Operations,
"operations": operations,
"patch_fields": hubV1PatchFields,
"limits": map[string]any{
"max_repos": selfServiceMaxRepos(),
Expand Down
12 changes: 9 additions & 3 deletions src/pkg/hub/selfservice_v1_patch.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ package hub
// registering again.
//
// Invariants (in addition to selfservice_v1_api.go's):
// - The hub session owner is the only caller. The registration key is not a
// credential here, and a hub admin who does not own the hive gets 404, the
// same answer as for a missing hive.
// - The ordinary edit requires the hub session owner. A hub admin who does
// not own the hive gets 404, the same answer as for a missing hive. The
// separately gated sync-repos operation uses the registration key.
// - If-Match is required (428 when absent, 412 when stale). The precondition
// is checked before any GitHub call and again inside the store lock.
// - Only repos the edit ADDS are proven (the same read-only
Expand All @@ -31,6 +31,7 @@ import (
"fmt"
"io"
"net/http"
"slices"
"sort"
"strings"
)
Expand Down Expand Up @@ -301,6 +302,11 @@ func (s *HubServer) handleHubV1Patch(w http.ResponseWriter, r *http.Request) {
}
n.Repos = plan.repos
n.RepoProofs = selfServiceProofTimes(n.RepoProofs, plan.repos, n.OwnershipCheckedAt, now, plan.added)
if len(n.PendingProof) > 0 && len(plan.added) > 0 {
n.PendingProof = slices.DeleteFunc(n.PendingProof, func(repo string) bool {
return hubV1ContainsFold(plan.added, repo)
})
}
n.DashboardURL = plan.dashboardURL
return nil
})
Expand Down
168 changes: 168 additions & 0 deletions src/pkg/hub/selfservice_v1_sync.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
package hub

import (
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"strings"
"time"
)

const (
hubV1SyncReposEnv = "HIVE_HUB_SELFSERVICE_SYNC_REPOS"
hubV1SyncReposOp = "sync-repos"
)

var errHubV1SyncNoWrite = errors.New("sync-repos did not change the registration")

func hubV1SyncReposEnabled() bool {
switch strings.ToLower(strings.TrimSpace(os.Getenv(hubV1SyncReposEnv))) {
case "1", "true", "yes", "on":
return true
}
return false
}

func (s *HubServer) routeHubV1Patch(w http.ResponseWriter, r *http.Request) {
// Never send a registration key through the hub auth middleware, which
// validates bearer credentials against GitHub. Key shape selects the
// lane, not authority: only the addressed record's hashes authenticate it.
if key, ok := strings.CutPrefix(r.Header.Get("Authorization"), "Bearer "); ok &&
len(key) == base64.RawURLEncoding.EncodedLen(selfServiceKeyBytes) {
var decoded [selfServiceKeyBytes]byte
if _, err := base64.RawURLEncoding.Strict().Decode(decoded[:], []byte(key)); err == nil {
s.handleHubV1SyncRepos(w, r, key)
return
}
}
s.requireAuth(s.handleHubV1Patch)(w, r)
}

func (s *HubServer) handleHubV1SyncRepos(w http.ResponseWriter, r *http.Request, key string) {
if !s.requireSelfServiceEnabled(w) {
return
}
if !hubV1SyncReposEnabled() {
writeJSONError(w, http.StatusNotFound, "sync-repos is not enabled")
return
}
if !isNonBrowserAPIRequest(r) || !isCSRFSafe(r) {
writeJSONError(w, http.StatusForbidden, "sync-repos requires a registration bearer without a hub session cookie and must pass the CSRF check")
return
}
limit := selfServiceRequestBodyLimit()
data, err := io.ReadAll(io.LimitReader(r.Body, limit+1))
if err != nil {
writeJSONError(w, http.StatusBadRequest, "could not read the request body")
return
}
if int64(len(data)) > limit {
writeJSONError(w, http.StatusRequestEntityTooLarge, "the request body is too large")
return
}

id := strings.ToLower(strings.TrimSpace(r.PathValue("id")))
hash := hashSelfServiceKey(key)
ifMatch := strings.TrimSpace(r.Header.Get("If-Match"))
var current SelfServiceHive
var refusal *hubV1PatchRefusal
stored, err := s.selfService.update(id, "", func(h *SelfServiceHive) error {
// Authenticate before exposing the current ETag. Check against the
// live record under the write lock, including after key rotation.
if !s.selfService.matchesKey(*h, hash) {
return errSelfServiceNotFound
}
current = *h
switch {
case ifMatch == "":
refusal = hubV1Refuse(http.StatusPreconditionRequired, "If-Match is required: send an ETag or '*' for the current registration")
case !selfServiceETagMatches(*h, ifMatch):
refusal = hubV1Refuse(http.StatusPreconditionFailed, "the registration changed; retry with its current ETag")
}
if refusal != nil {
return errHubV1SyncNoWrite
}
refusal = hubV1ApplySyncRepos(h, data)
if refusal != nil || (hubV1SameRepos(h.Repos, current.Repos) && hubV1SameRepos(h.PendingProof, current.PendingProof)) {
return errHubV1SyncNoWrite
}
return nil
})
switch {
case errors.Is(err, errSelfServiceNotFound):
writeJSONError(w, http.StatusNotFound, "self-hosted hive not found")
case errors.Is(err, errHubV1SyncNoWrite):
if refusal != nil {
etag := ""
if refusal.status == http.StatusPreconditionRequired || refusal.status == http.StatusPreconditionFailed {
etag = selfServiceETag(current)
}
writeHubV1JSON(w, refusal.status, etag, refusal.body)
return
}
writeHubV1JSON(w, http.StatusOK, selfServiceETag(current), map[string]any{"registration": s.hubV1View(current)})
case err != nil:
s.logger.Error("self-service sync-repos: store write failed", "hive", id, "error", err)
writeJSONError(w, http.StatusInternalServerError, "could not persist the change")
default:
s.logger.Info("audit: selfservice_hive_repos_synced", "hive", id, "repos", len(stored.Repos),
"pending_proof", len(stored.PendingProof), "revision", stored.Revision)
writeHubV1JSON(w, http.StatusOK, selfServiceETag(stored), map[string]any{"registration": s.hubV1View(stored)})
}
}

func hubV1ApplySyncRepos(h *SelfServiceHive, data []byte) *hubV1PatchRefusal {
var fields map[string]json.RawMessage
if err := json.Unmarshal(data, &fields); err != nil || fields == nil {
return hubV1Refuse(http.StatusBadRequest, "sync-repos takes a JSON object with op and repos")
}
for field := range fields {
if field != "op" && field != "repos" {
return hubV1Refuse(http.StatusBadRequest, "field %q cannot be changed by sync-repos", field)
}
}
var op string
if json.Unmarshal(fields["op"], &op) != nil || op != hubV1SyncReposOp {
return hubV1Refuse(http.StatusBadRequest, "a registration key may only PATCH with op %q", hubV1SyncReposOp)
}
var requested []string
if json.Unmarshal(fields["repos"], &requested) != nil || requested == nil {
return hubV1Refuse(http.StatusBadRequest, "repos must be a list of owner/name repos")
}
if maxRepos := selfServiceMaxRepos(); len(requested) > maxRepos {
return &hubV1PatchRefusal{
status: http.StatusRequestEntityTooLarge,
body: hubV1RepoCapBody(fmt.Errorf("at most %d repos per registration", maxRepos), maxRepos),
}
}
repos, err := selfServiceValidateRepos(requested)
if err != nil {
return hubV1Refuse(http.StatusBadRequest, "%s", err.Error())
}

// Both lists are replaced. A previously active repo keeps its proof;
// everything else must go through the owner's ordinary proof path.
activeByName := make(map[string]string, len(h.Repos))
for _, repo := range h.Repos {
activeByName[strings.ToLower(repo)] = repo
}
var active, pending []string
for _, repo := range repos {
if existing, ok := activeByName[strings.ToLower(repo)]; ok {
active = append(active, existing)
} else {
pending = append(pending, repo)
}
}
if len(active) == 0 {
return hubV1Refuse(http.StatusBadRequest, "sync-repos must keep at least one already-proven repo active")
}
h.Repos = active

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A nonempty request made only of new repos leaves h.Repos empty (all go to pending). Owner PATCH refuses a list that leaves no repo (400), and server.go:4100 treats a repo-less hive as a placeholder. Suggested fix:

Suggested change
h.Repos = active
if len(active) == 0 {
return hubV1Refuse(http.StatusBadRequest, "sync-repos must keep at least one already-proven repo active")
}
h.Repos = active

Please add a test for an all-new request.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 2df1a6da80: sync-repos now returns 400 for an all-new list without changing the registration or its ETag. The API docs and changelog describe the rule, and go test ./pkg/hub -run "^TestHubV1SyncRepos" -count=1 passed. The previous head’s coverage check failed because the agent shard failed two kick lifecycle tests, not because of Hub coverage.

h.PendingProof = pending
h.RepoProofs = selfServiceProofTimes(h.RepoProofs, active, h.OwnershipCheckedAt, time.Time{}, nil)
return nil
}
Loading
Loading