Repository navigation
✨ feature: let spokes sync their own registration repos #19252
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
hivecommons-hive
wants to merge
7
commits into
v5
Choose a base branch
from
hive/adopt-19243
base: v5
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+324
−13
Open
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
a1581f4
feat(hub): allow spokes to sync their own registration repos
Karibhou df4bb4e
fix(hub): keep a proven repo active during spoke sync
Karibhou e2dc67b
fix(ci): retry gotestsum installation in v2 tests
Karibhou 74b63b6
ci: drop out-of-scope gotestsum workflow change so integration-candid…
hivecommons-hive[bot] f63e35c
ci: re-trigger checks on a fresh head for the adopted PR
hivecommons-hive[bot] cd4f9c9
Merge v5 into hive/adopt-19243
hivecommons-hive[bot] 912a74c
🐛 fix: clear stale pending_proof entries when owner PATCH removes repos
hivecommons-hive[bot] File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| - Hubs can opt in to spoke repository sync: a registration key updates only its own repo list, at least one proven repo must remain active, and new repos stay pending until their owner proves them ([#19236](https://github.com/hivecommons/hive/issues/19236)). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,168 @@ | ||
| package hub | ||
|
|
||
| import ( | ||
| "encoding/base64" | ||
| "encoding/json" | ||
| "errors" | ||
| "fmt" | ||
| "io" | ||
| "net/http" | ||
| "os" | ||
| "strings" | ||
| "time" | ||
| ) | ||
|
|
||
| const ( | ||
| hubV1SyncReposEnv = "HIVE_HUB_SELFSERVICE_SYNC_REPOS" | ||
| hubV1SyncReposOp = "sync-repos" | ||
| ) | ||
|
|
||
| var errHubV1SyncNoWrite = errors.New("sync-repos did not change the registration") | ||
|
|
||
| func hubV1SyncReposEnabled() bool { | ||
| switch strings.ToLower(strings.TrimSpace(os.Getenv(hubV1SyncReposEnv))) { | ||
| case "1", "true", "yes", "on": | ||
| return true | ||
| } | ||
| return false | ||
| } | ||
|
|
||
| func (s *HubServer) routeHubV1Patch(w http.ResponseWriter, r *http.Request) { | ||
| // Never send a registration key through the hub auth middleware, which | ||
| // validates bearer credentials against GitHub. Key shape selects the | ||
| // lane, not authority: only the addressed record's hashes authenticate it. | ||
| if key, ok := strings.CutPrefix(r.Header.Get("Authorization"), "Bearer "); ok && | ||
| len(key) == base64.RawURLEncoding.EncodedLen(selfServiceKeyBytes) { | ||
| var decoded [selfServiceKeyBytes]byte | ||
| if _, err := base64.RawURLEncoding.Strict().Decode(decoded[:], []byte(key)); err == nil { | ||
| s.handleHubV1SyncRepos(w, r, key) | ||
| return | ||
| } | ||
| } | ||
| s.requireAuth(s.handleHubV1Patch)(w, r) | ||
| } | ||
|
|
||
| func (s *HubServer) handleHubV1SyncRepos(w http.ResponseWriter, r *http.Request, key string) { | ||
| if !s.requireSelfServiceEnabled(w) { | ||
| return | ||
| } | ||
| if !hubV1SyncReposEnabled() { | ||
| writeJSONError(w, http.StatusNotFound, "sync-repos is not enabled") | ||
| return | ||
| } | ||
| if !isNonBrowserAPIRequest(r) || !isCSRFSafe(r) { | ||
| writeJSONError(w, http.StatusForbidden, "sync-repos requires a registration bearer without a hub session cookie and must pass the CSRF check") | ||
| return | ||
| } | ||
| limit := selfServiceRequestBodyLimit() | ||
| data, err := io.ReadAll(io.LimitReader(r.Body, limit+1)) | ||
| if err != nil { | ||
| writeJSONError(w, http.StatusBadRequest, "could not read the request body") | ||
| return | ||
| } | ||
| if int64(len(data)) > limit { | ||
| writeJSONError(w, http.StatusRequestEntityTooLarge, "the request body is too large") | ||
| return | ||
| } | ||
|
|
||
| id := strings.ToLower(strings.TrimSpace(r.PathValue("id"))) | ||
| hash := hashSelfServiceKey(key) | ||
| ifMatch := strings.TrimSpace(r.Header.Get("If-Match")) | ||
| var current SelfServiceHive | ||
| var refusal *hubV1PatchRefusal | ||
| stored, err := s.selfService.update(id, "", func(h *SelfServiceHive) error { | ||
| // Authenticate before exposing the current ETag. Check against the | ||
| // live record under the write lock, including after key rotation. | ||
| if !s.selfService.matchesKey(*h, hash) { | ||
| return errSelfServiceNotFound | ||
| } | ||
| current = *h | ||
| switch { | ||
| case ifMatch == "": | ||
| refusal = hubV1Refuse(http.StatusPreconditionRequired, "If-Match is required: send an ETag or '*' for the current registration") | ||
| case !selfServiceETagMatches(*h, ifMatch): | ||
| refusal = hubV1Refuse(http.StatusPreconditionFailed, "the registration changed; retry with its current ETag") | ||
| } | ||
| if refusal != nil { | ||
| return errHubV1SyncNoWrite | ||
| } | ||
| refusal = hubV1ApplySyncRepos(h, data) | ||
| if refusal != nil || (hubV1SameRepos(h.Repos, current.Repos) && hubV1SameRepos(h.PendingProof, current.PendingProof)) { | ||
| return errHubV1SyncNoWrite | ||
| } | ||
| return nil | ||
| }) | ||
| switch { | ||
| case errors.Is(err, errSelfServiceNotFound): | ||
| writeJSONError(w, http.StatusNotFound, "self-hosted hive not found") | ||
| case errors.Is(err, errHubV1SyncNoWrite): | ||
| if refusal != nil { | ||
| etag := "" | ||
| if refusal.status == http.StatusPreconditionRequired || refusal.status == http.StatusPreconditionFailed { | ||
| etag = selfServiceETag(current) | ||
| } | ||
| writeHubV1JSON(w, refusal.status, etag, refusal.body) | ||
| return | ||
| } | ||
| writeHubV1JSON(w, http.StatusOK, selfServiceETag(current), map[string]any{"registration": s.hubV1View(current)}) | ||
| case err != nil: | ||
| s.logger.Error("self-service sync-repos: store write failed", "hive", id, "error", err) | ||
| writeJSONError(w, http.StatusInternalServerError, "could not persist the change") | ||
| default: | ||
| s.logger.Info("audit: selfservice_hive_repos_synced", "hive", id, "repos", len(stored.Repos), | ||
| "pending_proof", len(stored.PendingProof), "revision", stored.Revision) | ||
| writeHubV1JSON(w, http.StatusOK, selfServiceETag(stored), map[string]any{"registration": s.hubV1View(stored)}) | ||
| } | ||
| } | ||
|
|
||
| func hubV1ApplySyncRepos(h *SelfServiceHive, data []byte) *hubV1PatchRefusal { | ||
| var fields map[string]json.RawMessage | ||
| if err := json.Unmarshal(data, &fields); err != nil || fields == nil { | ||
| return hubV1Refuse(http.StatusBadRequest, "sync-repos takes a JSON object with op and repos") | ||
| } | ||
| for field := range fields { | ||
| if field != "op" && field != "repos" { | ||
| return hubV1Refuse(http.StatusBadRequest, "field %q cannot be changed by sync-repos", field) | ||
| } | ||
| } | ||
| var op string | ||
| if json.Unmarshal(fields["op"], &op) != nil || op != hubV1SyncReposOp { | ||
| return hubV1Refuse(http.StatusBadRequest, "a registration key may only PATCH with op %q", hubV1SyncReposOp) | ||
| } | ||
| var requested []string | ||
| if json.Unmarshal(fields["repos"], &requested) != nil || requested == nil { | ||
| return hubV1Refuse(http.StatusBadRequest, "repos must be a list of owner/name repos") | ||
| } | ||
| if maxRepos := selfServiceMaxRepos(); len(requested) > maxRepos { | ||
| return &hubV1PatchRefusal{ | ||
| status: http.StatusRequestEntityTooLarge, | ||
| body: hubV1RepoCapBody(fmt.Errorf("at most %d repos per registration", maxRepos), maxRepos), | ||
| } | ||
| } | ||
| repos, err := selfServiceValidateRepos(requested) | ||
| if err != nil { | ||
| return hubV1Refuse(http.StatusBadRequest, "%s", err.Error()) | ||
| } | ||
|
|
||
| // Both lists are replaced. A previously active repo keeps its proof; | ||
| // everything else must go through the owner's ordinary proof path. | ||
| activeByName := make(map[string]string, len(h.Repos)) | ||
| for _, repo := range h.Repos { | ||
| activeByName[strings.ToLower(repo)] = repo | ||
| } | ||
| var active, pending []string | ||
| for _, repo := range repos { | ||
| if existing, ok := activeByName[strings.ToLower(repo)]; ok { | ||
| active = append(active, existing) | ||
| } else { | ||
| pending = append(pending, repo) | ||
| } | ||
| } | ||
| if len(active) == 0 { | ||
| return hubV1Refuse(http.StatusBadRequest, "sync-repos must keep at least one already-proven repo active") | ||
| } | ||
| h.Repos = active | ||
| h.PendingProof = pending | ||
| h.RepoProofs = selfServiceProofTimes(h.RepoProofs, active, h.OwnershipCheckedAt, time.Time{}, nil) | ||
| return nil | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Low, out-of-scope: a key-shaped bearer skips requireAuth, so the body is read (up to the limit) and the global store write lock is taken in update() before the key is checked. Wrong keys return 404 without a save, but each request still takes the lock. Consider a read-lock key check before the body read, or confirm that a rate limiter covers this route.