Skip to content

✨ hub: registration protocol v1 discovery and in-place PATCH (prototype) - #189

Merged
tuna-reef-hive[bot] merged 10 commits into
tuna/v6from
feat/hub-registration-v1
Nov 17, 2026
Merged

tuna-reef-hive[bot] merged 10 commits into
tuna/v6from
feat/hub-registration-v1

Conversation

@hanthor

@hanthor hanthor commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Summary

Prototype of rollout step 1 of the hub registration RFC, hivecommons#18642. It adds discovery, plus a PATCH that edits a registration's repos and dashboard_url without re-keying the spoke.

Backport first: the self-service API was not on tuna/v6

tuna/v6 forked from upstream v6 on 2026-10-29. The self-service API landed upstream on 2026-11-14, so selfservice_api.go, selfservice_registry.go and the heartbeat fence did not exist here, and the open sync PR #179 (dea69b350) predates them too. The first six commits are cherry-picks of the upstream self-service stack. Each one names its upstream SHA. They are re-signed by me, with the original authors credited as Co-authored-by, so the squash sign-off check passes:

Upstream Change
eee2c6107 credential fence for byo- heartbeats (hivecommons#18081)
6df6262e3 registry with hashed, rotatable keys (hivecommons#18150)
ba875a349 registration API with the GitHub admin proof (hivecommons#18155)
40a9f3d73 expiry sweep (hivecommons#18167)
436bf8599 hub UI register form (hivecommons#18168)
644af5925 hardening: cap race, hyphen IDs, dot segments (hivecommons#18170)

Conflicts: src/docs/api-reference/hub-server.md does not exist on tuna/v6, so it was dropped. In background_pollers.go, only the expiry-sweep start(...) was kept, because tuna/v6 does not have the benchmarks or marketplace pollers. The fence file matches upstream except for UpgradeChannelTag, which tuna/v6's heartbeat response doesn't have. A full upstream merge (#179 and later) will fold these commits in.

The RFC work is the commit ✨ hub: registration protocol v1 discovery and in-place PATCH. Review that one. Two test-only commits follow it to get v2-tests green on this base:

  • heartbeat_selfservice_fence_test.go skips three fence entries (ActivityETag, ActivityEventsAck, UpgradeAnnounce) for fields that tuna/v6's HeartbeatResponse doesn't have yet. heartbeat_selfservice_fence.go stays identical to upstream.
  • The claim-ledger reload tests take the upstream clock-pinning fix. Their fixtures are dated 2026-10-14, so 30-day retention now prunes the claims on reload. That breaks tuna/v6 itself, whatever the change under test.

v2-tests (dispatched): https://github.com/tuna-os/hive/actions/runs/44027795730 passed before the owner decisions. After them, https://github.com/tuna-os/hive/actions/runs/44028690500 passed. After the re-sign (same tree), https://github.com/tuna-os/hive/actions/runs/44029619635 passed.

Routes added

Method Path Notes
GET /api/hub/v1 Public discovery: protocol, version, operations, patch_fields, limits {max_repos, max_per_user, expiry, key_rotation_overlap}, proof_methods: ["hub-session-token"] (the token stored at hub sign-in; github-user-token is step 2)
POST /api/hub/v1/registrations Same logic as the v6 register. Returns 201 with Location and ETag. Over the repo cap it returns 413 with limit: max_repos; the v6 create keeps 400
GET /api/hub/v1/registrations The caller's own records. Each item carries etag and repo_proofs [{repo, checked_at}]
GET /api/hub/v1/registrations/{id} ETag. A matching If-None-Match gets 304
PATCH /api/hub/v1/registrations/{id} repos as {add, remove} or as a full array, and/or dashboard_url. The key is not touched. If-Match is required (428 when missing, 412 when stale). org gets 400: "org is not editable; delete and re-register"
POST /api/hub/v1/registrations/{id}/keys Rotate. If-Match is optional. A grace body gets a 400, because grace is step 3
DELETE /api/hub/v1/registrations/{id} Owner or hub admin. If-Match is optional

All of them return 404 while HIVE_HUB_SELFSERVICE_REGISTRATION is off, the same as the v6 family.

Design points

  • The key stays valid. selfServiceStore.update puts back the key hashes, owner, ID, CreatedAt and heartbeat stamp after the edit, so a PATCH can't touch the credential.
  • Per-repo proof times. SelfServiceHive.RepoProofs is set for every repo on create and for added repos on PATCH. Other repos keep their times, and a removed repo's entry is dropped. Records from before the field report OwnershipCheckedAt, which stays unchanged for the v6 routes.
  • Only added repos get proved. The read-only GET /repos/{o}/{r} check runs for repos that are new to the record. Removals and URL edits make no GitHub calls. A refused proof leaves the record byte-for-byte unchanged.
  • Concurrency. SelfServiceHive.Revision goes up on register, PATCH and rotate, but not on heartbeats, so a live spoke doesn't invalidate an operator's ETag. The ETag is a hash of ID, CreatedAt and revision, so it reveals nothing about the key. If-Match uses strong comparison (a W/ tag never matches). It is checked once before the GitHub calls (to fail fast) and again inside the store lock.
  • Configurable cap. HIVE_HUB_SELFSERVICE_MAX_REPOS defaults to 20, so nothing changes unless it is set. Values above 500 are clamped to 500. The request body limit grows with the cap. The v6 routes read the same cap.
  • The v6 routes are unchanged. handleSelfServiceRegister now calls a shared selfServiceRegisterCore and selfServiceProveOwnership, but its body, status codes and headers stay the same. A test pins this.
  • No GitHub writes. heartbeat_selfservice_fence.go is not changed in the last commit.

Tests (selfservice_v1_api_test.go, table-driven, using the existing self-service env)

  • Discovery shape, the cap default, configured, clamped and invalid values, and 404 with the flag off.
  • The v1 routes are registered on the mux.
  • Create, get, list and 304. The v1-created key authenticates.
  • PATCH add, add+remove, remove-only, full replacement, URL set and clear. After each one, the old key still passes verifyHeartbeatBearerGeneration, and only the added repos hit the GitHub stub.
  • PATCH refusals with the record unchanged: stale ETag 412, made-up ETag 412, weak ETag 412, missing If-Match 428, someone else's registration 404, hub admin 404, not admin of an added repo 403, added repo the token can't see 403, org or key_hash fields 400, empty patch, removing every repo, the same repo in add and remove, a bad ref, an http URL.
  • Configurable cap: the default refuses 21 on create, a raised cap accepts 21, a PATCH past the cap gets 413 with limit: max_repos.
  • Rotate and delete with and without If-Match, as owner, admin and another user.
  • The v6 route bodies (register, list, status, rotate, delete) keep their key sets, and a v1 PATCH shows up in the v6 list.

The shared env gained a GitHub-call recorder, plus the admin repos alice/docs, alice/site and alice/bulk-*.

RFC questions: resolved by the owner (also on hivecommons#18642)

  1. Step 1 advertises hub-session-token. github-user-token comes with step 2.
  2. Proof times are kept per repo (repo_proofs). OwnershipCheckedAt stays for the v6 routes.
  3. A v1 create over the cap returns 413 with limit: max_repos, the same as PATCH. The v6 create keeps 400.
  4. org stays non-editable and gets a clear error.
  5. Rotate returns 201.

Links hivecommons#18642.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Open PR overlap check

Base branch: tuna/v6

Conflicts are checked with every PR merged onto the current tip of tuna/v6, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#179 🔀 sync: merge upstream hivecommons/hive v6 (dea69b3) into tuna/v6 CONFLICT src/pkg/github/prclaims_reopen_test.go
src/pkg/hub/background_pollers.go

@github-actions github-actions Bot added the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 17, 2026
@hanthor
hanthor force-pushed the feat/hub-registration-v1 branch from ab581d2 to 091b903 Compare November 17, 2026 05:57
@tuna-reef-hive

tuna-reef-hive Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

@hanthor this PR is held at stage/071-needs-author and the hive will not merge it yet.

Failing checks:

  • check-failure

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

Evidence bundle `#189f5611b1` sha256:39a82f42d602d58e204204e7452dc83c717689c51ff7351a19f1cf73c3c054fb — download from the dashboard or `hivectl review evidence #189`.

hanthor and others added 10 commits November 17, 2026 06:13
…commons#18081)

* ✨ hub: credential fence for self-service (byo-) hive heartbeats

Step 1 of the hivecommons#18060 RFC. Hive IDs in the reserved byo- namespace are
listed as directory metadata only, so their heartbeat response must not
carry hub-held credentials or management actions. Several delivery paths
resolve what to send from spoke-reported identity (appKeySyncForHeartbeat
falls back to the payload cluster_id for a hive with no SaaS record), so a
per-site check is fragile.

One guard in signHeartbeatResponse, the single point every heartbeat
response passes through, zeroes each field that
heartbeatResponseFieldFence classifies as fenced, and fails closed for any
unclassified field. TestHeartbeatResponseFieldFenceInventory fails when a
HeartbeatResponse field is added without a classification, following the
TestNotificationWriteSitesInventory pattern.

Hive-Run: hivecommons#18060
Hive-Plan: RFC hivecommons#18060 suggested order of work, step 1 (fence and inventory test)
Hive-Spec: hivecommons#18060#proposed-solution-4-credential-fence

* 🚨 hub: use reflect.Pointer in self-service fence test (govet inline)

---------

(backported from hivecommons/hive commit eee2c61)

Co-authored-by: Danathar <doug.baggett@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Danathar <Danathar@users.noreply.github.com>
Co-authored-by: hivecommons-hive[bot] <280983584+hivecommons-hive[bot]@users.noreply.github.com>
Signed-off-by: James Reilly <jreilly1821@gmail.com>
…nd bearer verification (hivecommons#18150)

* ✨ hub: self-service (byo-) hive registry with hashed rotatable keys and bearer verification

Step 2 of the hivecommons#18060 RFC. A self-service hive's heartbeat bearer is a
random key the hub mints per hive and stores only as a SHA-256 hash in
/data/hub-selfservice.json. verifyHeartbeatBearerGeneration checks a
byo- ID against that hash and nothing else: the master-derived paths
are never tried for a byo- ID, and a self-service key is only ever
compared against the record for the ID the caller claims (audit F2).
Rotation keeps the previous hash live for a 10-minute overlap; revoke
removes the record. Accepted heartbeats stamp the record for the
expiry sweep that follows in a later child.

Fixes hivecommons#18142
Part of hivecommons#18060

* 📖 docs: resync api-reference citations after hub server.go line shift

* 🐛 hub: debounce self-service heartbeat stamp against the last persisted value

---------

(backported from hivecommons/hive commit 6df6262)

Co-authored-by: Andy Anderson <andy@clubanderson.com>
Signed-off-by: James Reilly <jreilly1821@gmail.com>
…nership proof, rotate and revoke (hivecommons#18155)

Behind HIVE_HUB_SELFSERVICE_REGISTRATION (default off), a signed-in user can
register a hive they run themselves. The hub builds the ID as
byo-<github login>-<name>, refuses reserved and duplicate IDs, proves
permissions.admin on every owner/name repo with the caller's own stored
GitHub token (refusing the whole registration by repo name otherwise),
enforces HIVE_HUB_SELFSERVICE_MAX_PER_USER (default 5), and returns the raw
heartbeat key once with Docker Compose, Podman and Kubernetes snippets. The
owner can list, rotate and revoke; a hub admin can revoke. Another user's
hive and the whole family while the flag is off answer 404. Every decision
writes an audit: line that never carries key material.

Part of hivecommons#18060
Fixes hivecommons#18144

(backported from hivecommons/hive commit ba875a3)

Co-authored-by: Andy Anderson <andy@clubanderson.com>
Co-authored-by: hivecommons-hive[bot] <280983584+hivecommons-hive[bot]@users.noreply.github.com>
Signed-off-by: James Reilly <jreilly1821@gmail.com>
…pe (hivecommons#18167)

* ✨ hub: self-service hive expiry sweep and hub-assigned self-hosted type

A background sweep (HIVE_HUB_SELFSERVICE_SWEEP_INTERVAL, default 10m) revokes
the key and removes the registry entry of every byo- registration that went
longer than HIVE_HUB_SELFSERVICE_EXPIRY (default 24h) without an accepted
heartbeat, counted from LastAcceptedHeartbeatAt or from CreatedAt while there
has been none. Removal takes the same path as the owner DELETE
(selfService.revoke + removeRegistryEntry) and writes
audit: selfservice_hive_expired with id, owner, last_heartbeat.

handleHeartbeat now types byo- registry entries as self-hosted from the ID
prefix; a hive_type the payload claims is ignored for that lane. Hosted and
operator-listed entries keep their existing rule.

Fixes hivecommons#18146
Part of hivecommons#18060

* 🌱 test: drop hosted- heartbeat case that needs provisioning state

A hosted- heartbeat answers 403 on a bare test hub before the type is
assigned; the table test covers that branch of hubAssignedHiveType.

---------

(backported from hivecommons/hive commit 40a9f3d)

Co-authored-by: Andy Anderson <andy@clubanderson.com>
Signed-off-by: James Reilly <jreilly1821@gmail.com>
…tate and remove (hivecommons#18168)

* ✨ hub UI: Register your own hive form with one-time key, snippets, rotate and remove

Behind HIVE_HUB_SELFSERVICE_REGISTRATION (GET /api/hub/selfservice/status
on load). Both Register-your-own-hive CTAs open a modal when enabled and
keep linking to /get-started#self-host otherwise. 201 shows the key once
with Docker Compose, Podman and Kubernetes snippets; a Self-hosted hives
section lists the owner's entries with Rotate key and Remove.

Fixes hivecommons#18145
Part of hivecommons#18060

* 🐛 selfservice form: classes instead of style attributes, camelCase rows, test fixes

Keeps the dashboard style ratchet at its baseline (no new style=,
raw font sizes, rgba colours, px padding or radius), avoids the
h.created_at hive-table guard by normalising list rows, and fixes two
assertions in the new UX test.

---------

(backported from hivecommons/hive commit 436bf85)

Co-authored-by: Andy Anderson <andy@clubanderson.com>
Signed-off-by: James Reilly <jreilly1821@gmail.com>
…gments) (hivecommons#18170)

- Re-check HIVE_HUB_SELFSERVICE_MAX_PER_USER inside the store under the
  write lock (registerCapped); the handler's count runs before up to 20
  GitHub calls and raced for concurrent POSTs from one user.
- Build self-service IDs as byo-<login>--<name>. GitHub logins never
  contain "--" or end in "-", so the boundary is unambiguous; refuse
  logins that break that rule and names starting with "-".
- isValidRepoRef refuses "." and ".." segments before url.JoinPath.
- Tests: concurrent registers at cap-1 give exactly one 201; hyphen
  logins a-b/c and a/b-c get distinct IDs; owner/.. and owner/. are 400.

Closes hivecommons#18166

Hive-Run: hivecommons#18166
Hive-Plan: hivecommons#18060 self-service registration follow-ups
Hive-Spec: hub-selfservice-registration#18166

(backported from hivecommons/hive commit 644af59)

Co-authored-by: Danathar <doug.baggett@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Danathar <Danathar@users.noreply.github.com>
Co-authored-by: clubanderson <407614+clubanderson@users.noreply.github.com>
Signed-off-by: James Reilly <jreilly1821@gmail.com>
Rollout step 1 of the hub registration RFC (hivecommons#18642):

- GET /api/hub/v1 discovery: protocol version, operations, limits
  (max_repos, max_per_user, expiry, key_rotation_overlap) and proof
  methods (github-user-token).
- /api/hub/v1/registrations resources over the existing self-service
  store: create, list, get, delete, and rotate as POST .../keys.
- PATCH /api/hub/v1/registrations/{id} changes repos (add/remove lists or
  a full replacement) and dashboard_url in place. The heartbeat key is not
  touched. Only added repos get the read-only GitHub ownership check.
  Writes carry an ETag; PATCH requires If-Match (428 without, 412 stale).
- HIVE_HUB_SELFSERVICE_MAX_REPOS makes the repo cap configurable
  (default 20, clamped to 500).

The v6 /api/hub/selfservice/* routes keep their bodies and status codes.
No GitHub writes; the heartbeat credential fence is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: James Reilly <jreilly1821@gmail.com>
The heartbeat credential fence backported from hivecommons/hive
(eee2c61) classifies three HeartbeatResponse fields that tuna/v6 does
not have yet (ActivityETag, ActivityEventsAck, UpgradeAnnounce), so the
stale-entry check in TestHeartbeatResponseFieldFenceInventory failed.
Keep heartbeat_selfservice_fence.go identical to upstream and have the
test skip those three names; an upstream sync that adds the fields makes
the skip list removable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: James Reilly <jreilly1821@gmail.com>
TestReopenedClaimPersistsAcrossLoad and
TestCloseOnMergeLedgerSweepSettledClaimsSkipAPI reload a ledger under the
wall clock. Their fixtures are dated 2026-10-14, so settledClaimRetention
(30 days) now prunes the claims on reload and both tests fail on tuna/v6
regardless of the change under test. Take the upstream fix (hivecommons/hive
v6): load with loadClaimLedgerAt and the test clock. Test-only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: James Reilly <jreilly1821@gmail.com>
…tion API

- Discovery advertises proof_methods ["hub-session-token"] (the token
  stored at hub sign-in). github-user-token is rollout step 2.
- Per-repo proof times: SelfServiceHive.RepoProofs, set for every repo on
  create, for added repos on PATCH, kept for the others, dropped for
  removed repos. The v1 view exposes repo_proofs [{repo, checked_at}];
  records from before the field report OwnershipCheckedAt, which stays as
  it was for the v6 routes.
- A v1 create over the repo cap answers 413 with limit: max_repos, like
  PATCH. The v6 create keeps its 400.
- A PATCH naming org gets "org is not editable; delete and re-register".
- Rotate keeps answering 201.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: James Reilly <jreilly1821@gmail.com>
@hanthor
hanthor force-pushed the feat/hub-registration-v1 branch from ec2507d to f5611b1 Compare November 17, 2026 06:13
@tuna-reef-hive
tuna-reef-hive Bot merged commit bac23e9 into tuna/v6 Nov 17, 2026
53 checks passed
@github-actions

Copy link
Copy Markdown

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

kubestellar-prow Bot pushed a commit to hivecommons/hive that referenced this pull request Nov 18, 2026
Rollout step 1, part A of the hub registration protocol
(#18642), ported from the tuna-os#189 prototype:

- GET /api/hub/v1 discovery: protocol version, operations, limits
  (max_repos, max_per_user, expiry, key_rotation_overlap) and the proof
  method the hub actually offers, hub-session-token. Readable without
  signing in.
- HIVE_HUB_SELFSERVICE_MAX_REPOS makes the per-registration repo cap
  configurable (default 20, clamped to 500). The legacy create keeps its
  400 body; the request body limit scales with the cap.

Everything is behind HIVE_HUB_SELFSERVICE_REGISTRATION. No GitHub writes;
heartbeat_signing.go and the credential fence are untouched.

Signed-off-by: James Reilly <jreilly1821@gmail.com>
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Co-authored-by: James Reilly <5840441+hanthor@users.noreply.github.com>
clubanderson added a commit to hivecommons/hive that referenced this pull request Nov 18, 2026
Rollout step 1, part B of the hub registration protocol
(#18642), ported from the tuna-os#189 prototype:

- Per-repo proof times (SelfServiceHive.RepoProofs) and a Revision
  counter. Both are additive JSON fields, so older binaries ignore them;
  a record without a revision loads as 0. Heartbeats do not bump the
  revision, so a live spoke never invalidates an operator's ETag.
- Store update() that restores the key hashes, owner, ID, CreatedAt and
  the heartbeat stamp after any mutation, plus rotate and revoke guarded
  by an ETag precondition.
- selfServiceRegisterCore and selfServiceProveOwnership split out of the
  legacy register handler. Tests pin the legacy /api/hub/selfservice/*
  responses byte for byte.
- v1 POST/GET/list/DELETE and POST .../keys: an ETag on every record,
  optional If-Match on delete and rotate, 413 with limit: max_repos over
  the repo cap, 201 on rotate, and a 400 for a caller-chosen grace window.

Behind HIVE_HUB_SELFSERVICE_REGISTRATION. No GitHub writes;
heartbeat_signing.go and the credential fence are untouched.

Co-authored-by: James Reilly <5840441+hanthor@users.noreply.github.com>
Signed-off-by: James Reilly <jreilly1821@gmail.com>
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
kubestellar-prow Bot pushed a commit to hivecommons/hive that referenced this pull request Nov 18, 2026
…18896)

* ✨ hub: v1 registration resources, per-repo proofs and revision

Rollout step 1, part B of the hub registration protocol
(#18642), ported from the tuna-os#189 prototype:

- Per-repo proof times (SelfServiceHive.RepoProofs) and a Revision
  counter. Both are additive JSON fields, so older binaries ignore them;
  a record without a revision loads as 0. Heartbeats do not bump the
  revision, so a live spoke never invalidates an operator's ETag.
- Store update() that restores the key hashes, owner, ID, CreatedAt and
  the heartbeat stamp after any mutation, plus rotate and revoke guarded
  by an ETag precondition.
- selfServiceRegisterCore and selfServiceProveOwnership split out of the
  legacy register handler. Tests pin the legacy /api/hub/selfservice/*
  responses byte for byte.
- v1 POST/GET/list/DELETE and POST .../keys: an ETag on every record,
  optional If-Match on delete and rotate, 413 with limit: max_repos over
  the repo cap, 201 on rotate, and a 400 for a caller-chosen grace window.

Behind HIVE_HUB_SELFSERVICE_REGISTRATION. No GitHub writes;
heartbeat_signing.go and the credential fence are untouched.

Co-authored-by: James Reilly <5840441+hanthor@users.noreply.github.com>
Signed-off-by: James Reilly <jreilly1821@gmail.com>
Signed-off-by: Andrew Anderson <andy@clubanderson.com>

* 🐛 hub: fix v1 registry test fixture and drop an unused helper

Co-authored-by: James Reilly <5840441+hanthor@users.noreply.github.com>
Signed-off-by: Andrew Anderson <andy@clubanderson.com>

---------

Signed-off-by: James Reilly <jreilly1821@gmail.com>
Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Co-authored-by: James Reilly <5840441+hanthor@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conflicts-with-open-pr Open PR has a merge conflict with another open PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant