Repository navigation
✨ feature: let spokes sync their own registration repos #19243
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
+324
−13
Closed
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
d65cd25
feat(hub): allow spokes to sync their own registration repos
Karibhou 9b8c7a0
Merge branch 'v5' into scanner/fix-19236
hivecommons-hive[bot] 2df1a6d
fix(hub): keep a proven repo active during spoke sync
Karibhou 374fc45
fix(ci): retry gotestsum installation in v2 tests
Karibhou 1d11c31
Merge v5 into scanner/fix-19236 to sync generated integration workflow
hivecommons-hive[bot] bc209fa
ci: drop out-of-scope gotestsum workflow change so integration-candid…
hivecommons-hive[bot] File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| - Hubs can opt in to spoke repository sync: a registration key updates only its own repo list, at least one proven repo must remain active, and new repos stay pending until their owner proves them ([#19236](https://github.com/hivecommons/hive/issues/19236)). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,168 @@ | ||
| package hub | ||
|
|
||
| import ( | ||
| "encoding/base64" | ||
| "encoding/json" | ||
| "errors" | ||
| "fmt" | ||
| "io" | ||
| "net/http" | ||
| "os" | ||
| "strings" | ||
| "time" | ||
| ) | ||
|
|
||
| const ( | ||
| hubV1SyncReposEnv = "HIVE_HUB_SELFSERVICE_SYNC_REPOS" | ||
| hubV1SyncReposOp = "sync-repos" | ||
| ) | ||
|
|
||
| var errHubV1SyncNoWrite = errors.New("sync-repos did not change the registration") | ||
|
|
||
| func hubV1SyncReposEnabled() bool { | ||
| switch strings.ToLower(strings.TrimSpace(os.Getenv(hubV1SyncReposEnv))) { | ||
| case "1", "true", "yes", "on": | ||
| return true | ||
| } | ||
| return false | ||
| } | ||
|
|
||
| func (s *HubServer) routeHubV1Patch(w http.ResponseWriter, r *http.Request) { | ||
| // Never send a registration key through the hub auth middleware, which | ||
| // validates bearer credentials against GitHub. Key shape selects the | ||
| // lane, not authority: only the addressed record's hashes authenticate it. | ||
| if key, ok := strings.CutPrefix(r.Header.Get("Authorization"), "Bearer "); ok && | ||
| len(key) == base64.RawURLEncoding.EncodedLen(selfServiceKeyBytes) { | ||
| var decoded [selfServiceKeyBytes]byte | ||
| if _, err := base64.RawURLEncoding.Strict().Decode(decoded[:], []byte(key)); err == nil { | ||
| s.handleHubV1SyncRepos(w, r, key) | ||
| return | ||
| } | ||
| } | ||
| s.requireAuth(s.handleHubV1Patch)(w, r) | ||
| } | ||
|
|
||
| func (s *HubServer) handleHubV1SyncRepos(w http.ResponseWriter, r *http.Request, key string) { | ||
| if !s.requireSelfServiceEnabled(w) { | ||
| return | ||
| } | ||
| if !hubV1SyncReposEnabled() { | ||
| writeJSONError(w, http.StatusNotFound, "sync-repos is not enabled") | ||
| return | ||
| } | ||
| if !isNonBrowserAPIRequest(r) || !isCSRFSafe(r) { | ||
| writeJSONError(w, http.StatusForbidden, "sync-repos requires a registration bearer without a hub session cookie and must pass the CSRF check") | ||
| return | ||
| } | ||
| limit := selfServiceRequestBodyLimit() | ||
| data, err := io.ReadAll(io.LimitReader(r.Body, limit+1)) | ||
| if err != nil { | ||
| writeJSONError(w, http.StatusBadRequest, "could not read the request body") | ||
| return | ||
| } | ||
| if int64(len(data)) > limit { | ||
| writeJSONError(w, http.StatusRequestEntityTooLarge, "the request body is too large") | ||
| return | ||
| } | ||
|
|
||
| id := strings.ToLower(strings.TrimSpace(r.PathValue("id"))) | ||
| hash := hashSelfServiceKey(key) | ||
| ifMatch := strings.TrimSpace(r.Header.Get("If-Match")) | ||
| var current SelfServiceHive | ||
| var refusal *hubV1PatchRefusal | ||
| stored, err := s.selfService.update(id, "", func(h *SelfServiceHive) error { | ||
| // Authenticate before exposing the current ETag. Check against the | ||
| // live record under the write lock, including after key rotation. | ||
| if !s.selfService.matchesKey(*h, hash) { | ||
| return errSelfServiceNotFound | ||
| } | ||
| current = *h | ||
| switch { | ||
| case ifMatch == "": | ||
| refusal = hubV1Refuse(http.StatusPreconditionRequired, "If-Match is required: send an ETag or '*' for the current registration") | ||
| case !selfServiceETagMatches(*h, ifMatch): | ||
| refusal = hubV1Refuse(http.StatusPreconditionFailed, "the registration changed; retry with its current ETag") | ||
| } | ||
| if refusal != nil { | ||
| return errHubV1SyncNoWrite | ||
| } | ||
| refusal = hubV1ApplySyncRepos(h, data) | ||
| if refusal != nil || (hubV1SameRepos(h.Repos, current.Repos) && hubV1SameRepos(h.PendingProof, current.PendingProof)) { | ||
| return errHubV1SyncNoWrite | ||
| } | ||
| return nil | ||
| }) | ||
| switch { | ||
| case errors.Is(err, errSelfServiceNotFound): | ||
| writeJSONError(w, http.StatusNotFound, "self-hosted hive not found") | ||
| case errors.Is(err, errHubV1SyncNoWrite): | ||
| if refusal != nil { | ||
| etag := "" | ||
| if refusal.status == http.StatusPreconditionRequired || refusal.status == http.StatusPreconditionFailed { | ||
| etag = selfServiceETag(current) | ||
| } | ||
| writeHubV1JSON(w, refusal.status, etag, refusal.body) | ||
| return | ||
| } | ||
| writeHubV1JSON(w, http.StatusOK, selfServiceETag(current), map[string]any{"registration": s.hubV1View(current)}) | ||
| case err != nil: | ||
| s.logger.Error("self-service sync-repos: store write failed", "hive", id, "error", err) | ||
| writeJSONError(w, http.StatusInternalServerError, "could not persist the change") | ||
| default: | ||
| s.logger.Info("audit: selfservice_hive_repos_synced", "hive", id, "repos", len(stored.Repos), | ||
| "pending_proof", len(stored.PendingProof), "revision", stored.Revision) | ||
| writeHubV1JSON(w, http.StatusOK, selfServiceETag(stored), map[string]any{"registration": s.hubV1View(stored)}) | ||
| } | ||
| } | ||
|
|
||
| func hubV1ApplySyncRepos(h *SelfServiceHive, data []byte) *hubV1PatchRefusal { | ||
| var fields map[string]json.RawMessage | ||
| if err := json.Unmarshal(data, &fields); err != nil || fields == nil { | ||
| return hubV1Refuse(http.StatusBadRequest, "sync-repos takes a JSON object with op and repos") | ||
| } | ||
| for field := range fields { | ||
| if field != "op" && field != "repos" { | ||
| return hubV1Refuse(http.StatusBadRequest, "field %q cannot be changed by sync-repos", field) | ||
| } | ||
| } | ||
| var op string | ||
| if json.Unmarshal(fields["op"], &op) != nil || op != hubV1SyncReposOp { | ||
| return hubV1Refuse(http.StatusBadRequest, "a registration key may only PATCH with op %q", hubV1SyncReposOp) | ||
| } | ||
| var requested []string | ||
| if json.Unmarshal(fields["repos"], &requested) != nil || requested == nil { | ||
| return hubV1Refuse(http.StatusBadRequest, "repos must be a list of owner/name repos") | ||
| } | ||
| if maxRepos := selfServiceMaxRepos(); len(requested) > maxRepos { | ||
| return &hubV1PatchRefusal{ | ||
| status: http.StatusRequestEntityTooLarge, | ||
| body: hubV1RepoCapBody(fmt.Errorf("at most %d repos per registration", maxRepos), maxRepos), | ||
| } | ||
| } | ||
| repos, err := selfServiceValidateRepos(requested) | ||
| if err != nil { | ||
| return hubV1Refuse(http.StatusBadRequest, "%s", err.Error()) | ||
| } | ||
|
|
||
| // Both lists are replaced. A previously active repo keeps its proof; | ||
| // everything else must go through the owner's ordinary proof path. | ||
| activeByName := make(map[string]string, len(h.Repos)) | ||
| for _, repo := range h.Repos { | ||
| activeByName[strings.ToLower(repo)] = repo | ||
| } | ||
| var active, pending []string | ||
| for _, repo := range repos { | ||
| if existing, ok := activeByName[strings.ToLower(repo)]; ok { | ||
| active = append(active, existing) | ||
| } else { | ||
| pending = append(pending, repo) | ||
| } | ||
| } | ||
| if len(active) == 0 { | ||
| return hubV1Refuse(http.StatusBadRequest, "sync-repos must keep at least one already-proven repo active") | ||
| } | ||
| h.Repos = active | ||
| h.PendingProof = pending | ||
| h.RepoProofs = selfServiceProofTimes(h.RepoProofs, active, h.OwnershipCheckedAt, time.Time{}, nil) | ||
| return nil | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A nonempty request made only of new repos leaves
h.Reposempty (all go to pending). Owner PATCH refuses a list that leaves no repo (400), andserver.go:4100treats a repo-less hive as a placeholder. Suggested fix:Please add a test for an all-new request.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in
2df1a6da80:sync-reposnow returns 400 for an all-new list without changing the registration or its ETag. The API docs and changelog describe the rule, andgo test ./pkg/hub -run "^TestHubV1SyncRepos" -count=1passed. The previous head’s coverage check failed because theagentshard failed two kick lifecycle tests, not because of Hub coverage.