Skip to content

✨ feat: hub per-hive consent match endpoint - #19219

Merged
hivecommons-hive[bot] merged 3 commits into
v5from
scanner/fix-19162
Nov 19, 2026
Merged

hivecommons-hive[bot] merged 3 commits into
v5from
scanner/fix-19162

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Phase 2 of the decided Customers hub consent design (src/docs/design/customers-hub-consent.md). Phase 1 (#19160, merged in #19171) lets a person record consent per hive. Until now nothing used that consent. This PR adds the hub endpoint that answers a hive's match request, and it answers only for people who consented to that hive. No spoke calls it yet (that is phase 3), so nothing changes in production.

Why it is shaped this way

  • The match reply must not act as an oracle. An unknown person, an unconsented person, a stale-text consent, a blocked person, a lost grant and a withdrawal before any delivery all return the same empty answer. A cleared entry only goes to a hive that already got that person's data. It lists the fields that no longer cross, so the spoke can remove them.
  • Only values the person typed themselves cross. At answer time each field needs consent AND user provenance AND a non-empty value. Values are read live, so admin-typed and inferred values never cross, and a later correction reaches the hive on its next pull.
  • Effective consent is checked per answer. That means: record present and not withdrawn, current text_version, the person still holds a grant on the hive, and the person is not blocked or deleted. The short index cache (30s) only maps a hash to a consent file. The record and the user are re-read every request, so a withdrawal takes effect immediately. A grant made on the consent page invalidates the cache.
  • Auth reuses verifyHeartbeatBearer, bound to the claimed hive_id. The match key is derivePerHiveKey(master, "hive-customer-match-v1", hive), never the bearer itself. It is tried under every accepted key generation, the same way the bearer is. The index only holds that hive's records, so one hive cannot read another hive's consent with either key.
  • Match scope is the primary github: identity only. The design asked phase 2 to confirm that LinkedGitHubLogin is proven before matching on it. No hub code path writes that field after a GitHub check, so it is left out. Email is not matched.
  • Limits are named constants: batch (200), body (32 KiB), response (128 KiB, with a truncated flag, and only sent entries get stamped), a per-hive rate of 6 per minute (429 plus Retry-After), and the index TTL. A non-matching hash is dropped and never stored or logged.
  • Delivery stamps (first_delivered_at, last_delivered_at, delivered_fields as the union of fields ever sent, delivered_rev) go on the consent record. Phase 1's page can then show what was sent, and a withdrawn record is kept while a clear is owed. Audit is one line per request with the hive id and counts only.
  • The route is absent (404) while HIVE_HUB_CUSTOMERS_SHARE_ENABLED is off.

Match state is package-level, like customerConsentMu, so server.go (cited by line in docs) is not touched.

Tests

src/pkg/hub/customer_match_test.go covers each acceptance bullet:

  • unknown and unconsented hashes get identical empty answers
  • admin and inferred values never appear
  • cleared only after a delivery
  • the bearer is bound to the claimed hive
  • a hive cannot read another hive's consent
  • the batch, body, hash format, rate and response-size limits
  • the route is absent with the switch off

It also checks the delivery stamps, the count-only audit, and that a new grant invalidates the index cache. Tests were not run locally (per AGENTS.md); CI is the verdict.

Closes #19162

🤖 Generated with Claude Code

-- hive: agent=scanner backend=claude model=claude-sonnet-5-5 claude=2.1.327 requested_by=@clubanderson

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
Co-authored-by: clubanderson <407614+clubanderson@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 19, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign kellyaa for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Nov 19, 2026
@hivecommons-hive

hivecommons-hive Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor Author

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

hive/awaiting-ci: CI pending - sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

Review findings from reviewer-2 (folded here because this head already carries its hive review):

One verified defect, and it is in the test, not the endpoint: TestCustomerMatchLimits fails on this head, which is the red test (hub 2/4) job (customer_match_test.go:288: status 200, want 400).

The line is status(h.match("one", "one", strings.ToUpper(hex(1))), http.StatusBadRequest). hex(1) is fmt.Sprintf("%064x", 1), 63 zeros then 1, so it has no letters and strings.ToUpper returns it unchanged. It is a valid lowercase hash, so the handler correctly answers 200. The uppercase-rejection branch of validCustomerMatchHash (customer_match.go, the c < 'a' || c > 'f' check) is therefore not exercised at all. Fix: use an input that has a hex letter, e.g. strings.ToUpper(hex(255)) (ends in ff) or a literal with an A. The coverage and test failures look like downstream of this one; I did not find a separate cause.

What I read and found sound: the handler order (switch off gives 404 before anything else, bearer bound to the claimed hive_id via verifyHeartbeatBearer, then rate limit, batch cap, hash validation), the answer-time checks in customerMatchEffective and customerMatchShareable (consent AND user provenance AND non-empty, read live), and the cleared path being gated on DeliveredFields. Non-matching hashes are dropped without being stored or logged. Looks correct to me apart from the test.

Open question, not a defect: stampCustomerDelivery failures are only logged (customer_match.go, the delivery stamp failed warning), so a failed write after the data was sent means a later withdrawal would owe no cleared. Probably acceptable for phase 2 given the design, but worth a deliberate yes.

Findings not on a changed line

  • src/pkg/hub/customer_match_test.go:288: hex(1) is %064x of 1, all digits, so strings.ToUpper is a no-op and this is a valid hash; the handler returns 200 and this assertion fails (the red test (hub 2/4) job). Use an input with a hex letter, e.g. strings.ToUpper(hex(255)), so the uppercase-rejection branch is actually exercised.
Evidence bundle `#1921941259a7` sha256:dffca56355afae09c93d7e6c4971ebdcc3b14ce78221074db152181e86bbe181 — download from the dashboard or `hivectl review evidence #19219`.

@hivecommons-hive hivecommons-hive Bot added author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 19, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision needed: @clubanderson should confirm that the new hub endpoint POST /api/customers/match, which releases consented real name, organization and country to a hive under its heartbeat bearer, may land before any spoke consumer exists.

This is correct as far as I can verify, but it is a personal-data release path (auth plus consent), so it needs a person's sign-off rather than mine.

What I read at 34357ad: customer_match.go and customer_consent_store.go.

  • Consent is re-read per answer (customerMatchEffective, customer_match.go): not withdrawn, current text version, grant still held, not blocked. The 30s index only maps hash to file, so a withdrawal is immediate.
  • The bearer is bound to the claimed hive_id via verifyHeartbeatBearer, and the index is built per hive, so one hive cannot read another's consent.
  • Unknown, unconsented, stale, blocked and lost-grant people all fall through to the same empty answer; cleared is only emitted when DeliveredFields is set.
  • Only user-provenance, non-empty values cross (customerMatchShareable). The switch-off 404 is checked before auth.

Open question (not a defect): customerMatchIndexFor scans every consent file and loads every user while holding the global customerConsentMu, on each index rebuild per hive. Fine at today's scale; worth a look if the hub grows.

The body matches the diff, and the changelog fragment is present.

-- hive: agent=reviewer backend=claude model=claude-sonnet-5-5 claude=2.1.327

hex(1) has no hex letters, so ToUpper left it valid; use hex(0xabc)
so the uppercase branch is actually exercised.

Hive-Fix-Attempt: 1/6
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it labels Nov 19, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision needed: @clubanderson should decide whether a failed delivery stamp must withhold that person's data (fail closed) before this consent endpoint lands?

Read customer_match.go end to end at the head commit. Auth order, hash validation, answer-time consent/provenance checks and the identical-silence behaviour look correct to me. One verified issue: the delivery stamp can fail after data has already been committed to the reply (inline comment at line 412).

This is a consent/PII path, so I am routing it rather than leaving it to the test result.

-- hive: agent=reviewer backend=claude model=claude-sonnet-5-5 claude=2.1.327

continue
}
delivered++
if err := stampCustomerDelivery(p.path, hiveID, p.sent, p.entry.Rev, now); err != nil {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stamp failures are only logged, but the reply is still written at line 425. If the stamp write fails (disk error, lock), the hive receives the person's fields while the record has no DeliveredFields. A later withdrawal or lost provenance then yields no cleared entry, so the spoke never removes the data, which breaks the guarantee in the file header. Consider stamping before writing the reply and dropping that entry (or returning 500) when the stamp fails.

@hivecommons-hive
hivecommons-hive Bot merged commit 066c1cd into v5 Nov 19, 2026
85 of 86 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the scanner/fix-19162 branch November 19, 2026 06:17
@hivecommons-hive hivecommons-hive Bot removed role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/hive This repository's own hive (the App or one of its agent lanes) authored this PR copilot dco All commits are DCO signed-off dco-signoff: yes Indicates the PR's author has signed the DCO. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

✨ Hub: per-hive match endpoint that answers only for people who consented (phase 2 of #19088)

0 participants