Skip to content

✨ Customers: import hub-consented fields with hub provenance - #19247

Merged
hivecommons-hive[bot] merged 3 commits into
v5from
scanner/fix-19163
Nov 19, 2026
Merged

hivecommons-hive[bot] merged 3 commits into
v5from
scanner/fix-19163

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Phase 3 of the decided Customers hub consent design (src/docs/design/customers-hub-consent.md, #19088). This is the first spoke code. Phases 1 and 2 let a person consent per hive and let the hub answer a match request. Nothing used that answer yet. This PR adds the consumer, and it is off by default.

Why it is shaped this way

  • hub provenance sits between auto and manual. A value the person stated about themselves beats public GitHub profile text, so hub outranks auto. A value someone typed on this hive is final, so manual is never overwritten. The GitHub profile fetch, the location-derived country and the affiliation suggestion all leave a hub field alone. Editing it on the profile makes it manual, and export carries the provenance as stored.
  • Import only fills, never creates. The import asks only about confirmed customers by a confirmed GitHub login. A suggestion, a claimed identity, a bot or a tombstoned login is never hashed. It writes only real name, organization and country on a person who still holds that login. It never touches identities, tags, goals, notes or satisfaction.
  • The reply is untrusted text. Only match_hash, the three values and cleared are read. Each value is stripped of control and format characters (including bidi and zero-width marks), whitespace-collapsed and capped to MaxProfileFieldLen. A country must be a known ISO code. An entry for a hash the hive did not ask about is ignored.
  • Revocation reaches the spoke. A cleared entry empties only the hub-provenance fields it names, and each one returns to automatic. A hub value not refreshed for 72h is cleared the same way, so a hive that loses its hub link does not keep a withdrawn value. Expiry runs on every pass, even with the switch off, because it makes no request.
  • The wire contract matches ✨ feat: hub per-hive consent match endpoint #19219. POST /api/customers/match sends {hive_id, hashes} with the per-hive heartbeat bearer, never follows a redirect and sends no cookie. The hashes are HMAC-SHA256 under derivePerHiveKey(master, "hive-customer-match-v1", hive_id). Batches hold at most 200 hashes, a pass sends at most 4 requests (under the hub's 6/min limit), and a pass follows truncated. A parity test in pkg/hub pins the spoke's key and hash to the hub's.
  • The switch is necessary, never sufficient. The import runs only with customers.hub_import.enabled set (or HIVE_CUSTOMERS_HUB_IMPORT_ENABLED, or the new Settings, Features switch beside Customer avatars, with a help mark and an env-override line) and a hub registration. With either missing, a pass makes zero requests. The person's consent on the hub still decides what arrives.
  • Audit: each import or clear logs one audit: line with the person id and field names only. A pass logs counts only.

Match key

The spoke reads HIVE_CUSTOMER_MATCH_KEY, or derives the key from HIVE_HUB_SECRET and the hive id. Hub-hosted spokes do not hold the master. Until the hub injects the key, their import has no key and asks nothing, so it is safe but inert. The hub-side provisioning (and the byo- case) is split into #19246.

Tests

pkg/customers/hub_import_test.go covers each acceptance bullet:

  • manual is never overwritten
  • hub overwrites auto
  • no person is created or promoted
  • switch off makes zero requests
  • cleared and staleness both revert to automatic
  • a forgotten person is not rebuilt
  • only the three fields are read, sanitised and capped

It also covers batching and truncation, and the accept-time single-person pull. cmd/hive tests cover the HTTP matcher and "off means no request" end to end. Dashboard tests pin the switch markup, its round trip and the "from the hub" mark. Tests were not run locally (per AGENTS.md). CI is the verdict.

Closes #19163

🤖 Generated with Claude Code

-- hive: agent=scanner backend=claude model=claude-sonnet-5-5 claude=2.1.328 requested_by=@clubanderson

Phase 3 of src/docs/design/customers-hub-consent.md (#19088). A third
profile Source, hub, sits between auto and manual. An opt-in import posts
salted hashes of confirmed customers' GitHub logins to the hub match
endpoint (#19162). It applies only the real name, organization and country
a person consented to share, and it never creates a person or overwrites a
manual field. A cleared reply or the 72h staleness expiry hands a hub field
back to automatic, and a forgotten login is never asked about.

customers.hub_import.enabled (default off), HIVE_CUSTOMERS_HUB_IMPORT_ENABLED
and a Settings, Features switch beside Customer avatars gate the import.
The import also needs a hub registration. Hub-side provisioning of the
match key is #19246.

Closes #19163

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
Co-authored-by: clubanderson <407614+clubanderson@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 19, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign clubanderson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Nov 19, 2026
@hivecommons-hive

hivecommons-hive Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor Author

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

hive/awaiting-ci: CI pending - fixed the outbound-site inventory entry for the new hub match sender and merged v5 in. Sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: 781c5a3d

Review findings from reviewer (folded here because this head already carries its hive review):

I read the import logic (src/pkg/customers/hub_import.go), the HTTP matcher (src/cmd/hive/customers_hub_import.go) and the new key derivation (src/pkg/hub/spoke/keys.go) at head 970528e. The code reads as correct to me and matches the decided design in the PR body (#19088 phases). I am routing it only because it is a trust and personal-data surface: a new outbound flow of identifier hashes, the heartbeat bearer on a new route, and a new derived key. It is default off, so merging it is inert until an operator opts in.

Verified:

  • Off means no request: Run returns before Matcher.Match unless Enabled() and a non-empty key (hub_import.go, Run). Expiry still runs when off, which makes no request.
  • Only confirmed, unclaimed, non-bot, non-forgotten GitHub identities are hashed (candidates). Replies for hashes that were not asked are dropped (pos[e.MatchHash]).
  • ApplyHubMatch never overwrites SourceManual, writes only the three HubFields, and rechecks status, identity and tombstone inside changePerson.
  • The client never follows redirects, so the bearer goes only to the configured hub (customersHubClient); a non-200 is an error.
  • Reply text is stripped of control and format characters and capped; country must be a known ISO code.

One low-severity finding (not a blocker): when more than 800 confirmed logins exist (HubImportMaxRequests 4 x HubImportBatch 200), Run sorts the hashes and asks only the first 800 every pass. The same people are asked every hour and the rest never are, because the order is deterministic and nothing rotates the window. Fine for the current scale; a rotating offset would fix it if customer counts grow.

Open question, not verified: how the hub orders matches when truncated is true. The resume logic uses the index of the last returned asked hash (last+1), which is only right if the hub answers in request order. I did not read the hub handler in this pass.

Decision for the maintainer: the PR body says hub-hosted spokes have no match key until #19246 lands, so this ships inert for them. Please confirm that landing the consumer before the provisioning is intended.

Review findings from ci-maintainer (folded here because this head already carries its hive review):

Delta at 781c5a3 is mainly a merge of main (hub heartbeat/docs/changelog), no scope creep from this PR; the prosefilter outbound-site entry for customersHubMatcher.Match is correct.

Previous finding "only first 800 sorted hashes are ever asked": still present — see src/pkg/customers/hub_import.go:363-366. Sorted hashes plus a 4-request cap means customers past the first 800 are never imported or cleared (#19163 scope 2). Requested change: rotate through the hash list across passes with a persisted cursor.

Evidence bundle `#19247781c5a3` sha256:4ed711f3b291264baeb12d3faa9dccd9a00b08c8c89330767ad99d908ca52cd4 — download from the dashboard or `hivectl review evidence #19247`.

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#19240 ✨ hub: registration key rotation with a grace window same files src/docs/env-vars.md
#19202 ✨ ask: hub panel for optional Ask AI help same files src/docs/env-vars.md
src/pkg/prosefilter/outbound_sites_test.go
#19204 ✨ ask: hive backend and hub relay for optional Ask AI help same files src/docs/env-vars.md
src/pkg/config/testdata/agent_runtime_golden.json
src/pkg/dashboard/api_governor_features.go
src/pkg/dashboard/static-src/js/config-features.js
src/pkg/dashboard/static/index.html
src/pkg/prosefilter/outbound_sites_test.go
#19206 ✨ ask: hive panel and rollout verification for optional Ask AI help same files src/docs/env-vars.md
src/pkg/config/testdata/agent_runtime_golden.json
src/pkg/dashboard/api_governor_features.go
src/pkg/dashboard/static-src/js/config-features.js
src/pkg/dashboard/static/index.html
src/pkg/prosefilter/outbound_sites_test.go
#19241 ✨ hub: remaining ownership proof methods with per-request scopes same files src/docs/env-vars.md

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

UI render evidence

Render evidence captured.

Built from 781c5a3dedbf.
Screenshots: download the artifact (sign-in to GitHub required).
Run: https://github.com/hivecommons/hive/actions/runs/44516472027

Page Section Viewport Theme File Size
hive overview desktop light hive-overview-desktop-light.png 326 KB
hive overview desktop dark hive-overview-desktop-dark.png 307 KB
hive overview phone light hive-overview-phone-light.png 467 KB
hive overview phone dark hive-overview-phone-dark.png 464 KB
hub-fleet overview desktop light hub-fleet-overview-desktop-light.png 113 KB
hub-fleet overview desktop dark hub-fleet-overview-desktop-dark.png 115 KB
hub-fleet overview phone light hub-fleet-overview-phone-light.png 265 KB
hub-fleet overview phone dark hub-fleet-overview-phone-dark.png 270 KB
hub-dashboard overview desktop light hub-dashboard-overview-desktop-light.png 338 KB
hub-dashboard overview desktop dark hub-dashboard-overview-desktop-dark.png 344 KB
hub-dashboard overview phone light hub-dashboard-overview-phone-light.png 624 KB
hub-dashboard overview phone dark hub-dashboard-overview-phone-dark.png 656 KB

These are fixture renders, not live data: the page was built from the PR head and filled with synthetic data from src/scripts/fixtures/render-evidence/. They show layout and theming, not how real hives behave. How to read them: src/docs/ui-render-evidence.md.

@hivecommons-hive hivecommons-hive Bot added author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 19, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correctness review at 970528e. Provenance precedence, sanitising, the identity and tombstone re-check, expiry and truncation resume look right. One in-scope finding, inline: the 4-request cap with a fixed hash order means customers beyond the first 800 are never asked about (#19163 scopes 2 and 3). Tests not run.

-- hive: agent=adjudicator backend=claude model=claude-sonnet-5-5 claude=2.1.328

hashes := make([]string, 0, len(byHash))
for hash := range byHash {
hashes = append(hashes, hash)
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Starvation with more than 800 candidates: hashes are sorted here and only the first 4x200 are sent per pass, always the same ones. Customers whose hashes sort later are never imported or cleared, and their existing hub values expire at 72h. Rotate the start offset or order by least-recently-asked, and add a test with more than 800 candidates.

@hivecommons-hive hivecommons-hive Bot added stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Mutation testing

gremlins found 18 surviving mutant(s) in changed files: the code ran under tests, but changing it did not fail any test.

File Line Mutation
src/pkg/customers/hub_import.go 158 CONDITIONALS_BOUNDARY
src/pkg/customers/hub_import.go 162 CONDITIONALS_BOUNDARY
src/pkg/customers/hub_import.go 162 CONDITIONALS_NEGATION
src/pkg/customers/hub_import.go 228 CONDITIONALS_NEGATION
src/pkg/customers/hub_import.go 228 CONDITIONALS_NEGATION
src/pkg/customers/hub_import.go 267 CONDITIONALS_BOUNDARY
src/pkg/customers/hub_import.go 365 CONDITIONALS_BOUNDARY
src/pkg/customers/hub_import.go 367 CONDITIONALS_BOUNDARY
src/pkg/customers/hub_import.go 376 ARITHMETIC_BASE
src/pkg/customers/hub_import.go 376 INVERT_NEGATIVES
src/pkg/customers/hub_import.go 386 CONDITIONALS_BOUNDARY
src/pkg/customers/hub_import.go 394 CONDITIONALS_BOUNDARY
src/pkg/customers/hub_import.go 398 CONDITIONALS_BOUNDARY
src/pkg/customers/hub_import.go 404 ARITHMETIC_BASE
src/pkg/customers/hub_import.go 404 CONDITIONALS_BOUNDARY
src/pkg/customers/hub_import.go 404 CONDITIONALS_BOUNDARY
src/pkg/customers/hub_import.go 407 ARITHMETIC_BASE
src/pkg/customers/hub_import.go 437 CONDITIONALS_BOUNDARY

18 mutant(s) in changed files are not reached by any test (NOT COVERED). A maintainer may apply the mutation/acknowledged label to accept this.

File Line Mutation
src/cmd/hive/customers_hub_import.go 32 ARITHMETIC_BASE
src/cmd/hive/customers_hub_import.go 34 ARITHMETIC_BASE
src/cmd/hive/customers_hub_import.go 117 CONDITIONALS_NEGATION
src/cmd/hive/customers_hub_import.go 118 ARITHMETIC_BASE
src/cmd/hive/customers_hub_import.go 128 CONDITIONALS_NEGATION
src/cmd/hive/customers_hub_import.go 128 CONDITIONALS_NEGATION
src/cmd/hive/customers_hub_import.go 134 CONDITIONALS_BOUNDARY
src/cmd/hive/customers_hub_import.go 134 CONDITIONALS_BOUNDARY
src/cmd/hive/customers_hub_import.go 134 CONDITIONALS_NEGATION
src/cmd/hive/customers_hub_import.go 134 CONDITIONALS_NEGATION
src/cmd/hive/customers_hub_import.go 134 CONDITIONALS_NEGATION
src/cmd/hive/customers_hub_import.go 147 CONDITIONALS_NEGATION
src/pkg/customers/hub_import.go 47 ARITHMETIC_BASE
src/pkg/customers/hub_import.go 163 INCREMENT_DECREMENT
src/pkg/customers/hub_import.go 290 CONDITIONALS_BOUNDARY
src/pkg/customers/hub_import.go 290 CONDITIONALS_NEGATION
src/pkg/hub/spoke/keys.go 102 CONDITIONALS_NEGATION
src/pkg/hub/spoke/keys.go 105 CONDITIONALS_NEGATION

ℹ️ 7 mutant(s) on added lines were SKIPPED because gremlins v0.6.0 only honours the first lines added lines of each default-context hunk, so non-contiguous edits in one hunk are partly missed. This does not affect the verdict.

File Line Mutation
src/pkg/customers/store.go 577 CONDITIONALS_NEGATION
src/pkg/customers/store.go 577 CONDITIONALS_NEGATION
src/pkg/customers/store.go 626 CONDITIONALS_NEGATION
src/pkg/customers/store.go 626 CONDITIONALS_NEGATION
src/pkg/customers/store.go 626 CONDITIONALS_NEGATION
src/pkg/customers/store.go 626 CONDITIONALS_NEGATION
src/pkg/customers/store.go 626 CONDITIONALS_NEGATION

Verdict: lived (#17349).

…nd-site test

Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it labels Nov 19, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delta only merges unrelated hub heartbeat and docs changes. The prior correctness finding is still present: hub_import.go:363-368 sorts hashes and caps requests, so customers whose hash sorts late are never asked, and their hub fields can then age out via staleness expiry. Rotate the starting offset between runs or order by oldest refresh.

-- hive: agent=adjudicator backend=claude model=claude-sonnet-5-5 claude=2.1.328

@hivecommons-hive hivecommons-hive Bot added stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 19, 2026
@hivecommons-hive
hivecommons-hive Bot merged commit e5f47d1 into v5 Nov 19, 2026
106 of 107 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the scanner/fix-19163 branch November 19, 2026 11:51
@hivecommons-hive hivecommons-hive Bot removed role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) labels Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/hive This repository's own hive (the App or one of its agent lanes) authored this PR copilot dco All commits are DCO signed-off dco-signoff: yes Indicates the PR's author has signed the DCO. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

✨ Customers: import hub-consented fields with hub provenance, tombstones and a Settings switch (phase 3 of #19088)

0 participants