Skip to content

✨ ask: hub backend for optional Ask AI help - #19200

Merged
hivecommons-hive[bot] merged 6 commits into
v5from
feat/ask-hub-backend
Nov 19, 2026
Merged

hivecommons-hive[bot] merged 6 commits into
v5from
feat/ask-hub-backend

Conversation

@clubanderson

Copy link
Copy Markdown
Member

Phase 1 of the Ask AI help design (src/docs/design/dashboard-ask-ai-help.md, isolation model (a)): the hub backend only, no panel.

Fixes #19180

What it does

  • ask: config type and HIVE_ASK_* env in pkg/config/ask.go. Off by default. Switched on but missing a value (or a key file outside /secrets or /data/secrets, unreadable or empty) keeps it off with one warning that names no value. The key is only ever read from a file; there is no env var that holds it. The ask: block is attached to hive.yaml in phase 4 (✨ ask: hive backend and hub relay for optional Ask AI help (phase 4 of #19177) #19183), where the hive reads it; the hub reads env only.
  • pkg/askai: the Kapa Chat API client (POST /query/v1/projects/{id}/chat/, follow-ups on /query/v1/threads/{id}/chat/, key in X-API-KEY), pinned to the published QueryResponse schema (answer, thread_id, question_answer_id, is_uncertain, relevant_sources[].source_url/title/contains_internal_data) by a contract test against an httptest fake. The body is exactly {integration_id, query}: no user, no metadata.origin_url.
  • Service: question length cap; per-person consent (one file per person, 0600, atomic, text_version, grant and withdraw, deleted with the user); credential-shaped questions refused using the log scrubber's patterns, nothing sent; the closed-vocabulary section label (pkg/askai/sections.go) prepended when on; source allowlist (docs.hivecommons.dev and the GitHub docs path under hivecommons/hive), internal sources dropped; answer reduced to plain text (markdown links to their text, non-allowlisted URLs removed, capped); no surviving source means no_cited_answer, never uncited AI text; vendor thread ids kept server side behind an opaque handle bound to the person (and hive), with a TTL.
  • Limits as named constants with env overrides: per person 3/min, 20/h, 1 in flight; per hive 60/h (used by the relay in phase 4); hub-wide 4/min (a fifth of the vendor's 20/min per-team limit, any override clamped to 10), 2 in flight, 500/day. A vendor 429 backs every caller off for its Retry-After. Circuit breaker: 3 consecutive failures open it for 5 minutes, then one trial.
  • Audit lines with outcome and counts only: audit: ask request, audit: ask consent grant|withdraw, audit: ask breaker opened|closed, audit: ask daily budget spent. Counters by outcome at GET /api/ask/stats (hub admin).
  • Hub routes, registered only when on: GET /api/ask/status, PUT/DELETE /api/ask/consent, POST /api/ask (browser session only: no bearer, no impersonation, CSRF-checked), GET /api/ask/stats.

Tests (behaviour, against a fake vendor)

  • Payload: the vendor query is the question, or the closed-list label plus the question; seeded login, hive name and repo name never appear; an unknown section id sends no label.
  • Off is inert: no route (404) and silence in the log; half-configured: no route and exactly one warning without values.
  • Consent gates every request; withdrawal, an older text version and another person's consent all block; a nil checker blocks.
  • Credential-shaped questions (GitHub token, bearer, sk- key, AWS key) refused with zero vendor calls and no text in audit lines.
  • Sources and rendering: uncited answers become the fallback; hostile links are removed; markup stays inert text.
  • Limits per person, per hive (hive B is not charged for hive A), hub-wide per minute, in flight, daily; 429 backoff; timeout; breaker opens and closes.
  • Credentials (key, project id, integration id) never appear in any response, log line or error string; the key file outside the secret dirs is never read.

Operator notes

Nothing changes until HIVE_ASK_ENABLED and the Kapa values are set. How to wire the real key (Secret mounted under /secrets/kapa, env vars) is in src/docs/hub-deployment.md#ask-ai-help. Docs: env vars, hub API reference, telemetry outbound-path row.

@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 19, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign hanthor for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Nov 19, 2026
@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#19206 ✨ ask: hive panel and rollout verification for optional Ask AI help same files changelog.d/added-19180-ask-hub-backend.md
src/docs/api-reference/hub-saas.md
src/docs/api-reference/hub-server.md
src/docs/env-vars.md
src/docs/hub-deployment.md
src/docs/telemetry.md
src/pkg/askai/breaker.go
src/pkg/askai/consent.go
src/pkg/askai/edges_test.go
src/pkg/askai/kapa.go
src/pkg/askai/kapa_test.go
src/pkg/askai/limits.go
src/pkg/askai/sections.go
src/pkg/askai/service.go
src/pkg/askai/service_test.go
src/pkg/askai/sources.go
src/pkg/askai/store_test.go
src/pkg/askai/threads.go
src/pkg/config/ask.go
src/pkg/config/ask_test.go
src/pkg/hub/ask.go
src/pkg/hub/ask_test.go
src/pkg/hub/saas.go
src/pkg/hub/saas_users_admin.go
src/pkg/hub/server.go
src/pkg/prosefilter/outbound_sites_test.go
#19204 ✨ ask: hive backend and hub relay for optional Ask AI help CONFLICT changelog.d/added-19180-ask-hub-backend.md
src/docs/api-reference/hub-saas.md (conflict)
src/docs/api-reference/hub-server.md
src/docs/env-vars.md
src/docs/hub-deployment.md (conflict)
src/docs/telemetry.md (conflict)
src/pkg/askai/breaker.go
src/pkg/askai/consent.go (conflict)
src/pkg/askai/edges_test.go
src/pkg/askai/kapa.go
src/pkg/askai/kapa_test.go
src/pkg/askai/limits.go (conflict)
src/pkg/askai/sections.go
src/pkg/askai/service.go (conflict)
src/pkg/askai/service_test.go
src/pkg/askai/sources.go
src/pkg/askai/store_test.go
src/pkg/askai/threads.go
src/pkg/config/ask.go (conflict)
src/pkg/config/ask_test.go
src/pkg/hub/ask.go (conflict)
src/pkg/hub/ask_test.go
src/pkg/hub/saas.go
src/pkg/hub/saas_users_admin.go
src/pkg/hub/server.go
src/pkg/prosefilter/outbound_sites_test.go (conflict)
#19192 📖 docs: prepare Ask AI dashboard help coverage same files src/docs/telemetry.md
#19202 ✨ ask: hub panel for optional Ask AI help same files changelog.d/added-19180-ask-hub-backend.md
src/docs/api-reference/hub-saas.md
src/docs/api-reference/hub-server.md
src/docs/env-vars.md
src/docs/hub-deployment.md
src/docs/telemetry.md
src/pkg/askai/breaker.go
src/pkg/askai/consent.go
src/pkg/askai/edges_test.go
src/pkg/askai/kapa.go
src/pkg/askai/kapa_test.go
src/pkg/askai/limits.go
src/pkg/askai/sections.go
src/pkg/askai/service.go
src/pkg/askai/service_test.go
src/pkg/askai/sources.go
src/pkg/askai/store_test.go
src/pkg/askai/threads.go
src/pkg/config/ask.go
src/pkg/config/ask_test.go
src/pkg/hub/ask.go
src/pkg/hub/ask_test.go
src/pkg/hub/saas.go
src/pkg/hub/saas_users_admin.go
src/pkg/hub/server.go
src/pkg/prosefilter/outbound_sites_test.go

@github-actions github-actions Bot added the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 19, 2026
@hivecommons-hive

hivecommons-hive Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

@clubanderson this PR is held at stage/071-needs-author and the hive will not merge it yet.

Failing checks:

  • relative links and anchors resolve

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

ℹ️ Sentinel: informational, author is trusted (owner), merge is not blocked. Head 52704f4fb36a matched:

  • remote_code_execution: adds a remote-fetch-and-execute, decoded payload, reverse shell or embedded credential (src/pkg/askai/service_test.go)

No sentinel-alert label was added. The finding is recorded in the dashboard Security tab and /api/sentinel; set sentinel.trusted_authors_block: true to block trusted authors too.

hive/awaiting-ci: rebased onto v5 (one docs table conflict, took v5 rows; the Ask rows live in hub-saas.md and are intact). CI pending - sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: 6a6872810

Evidence bundle `#192006b9fac6` sha256:e1ccf0d06454ccdf38e9213d68ff12730b987607b7c353d7104eb8899d23e9b0 — download from the dashboard or `hivectl review evidence #19200`.

merged by Hive trusted-author auto-merge: author clubanderson holds role owner and GitHub permission admin on repo; CI green at 6b9fac6.

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Mutation testing

gremlins found 26 surviving mutant(s) in changed files: the code ran under tests, but changing it did not fail any test.

File Line Mutation
src/pkg/askai/consent.go 99 CONDITIONALS_NEGATION
src/pkg/askai/consent.go 99 CONDITIONALS_NEGATION
src/pkg/askai/consent.go 135 CONDITIONALS_NEGATION
src/pkg/askai/kapa.go 170 CONDITIONALS_BOUNDARY
src/pkg/askai/kapa.go 192 CONDITIONALS_BOUNDARY
src/pkg/askai/kapa.go 195 CONDITIONALS_BOUNDARY
src/pkg/askai/limits.go 44 CONDITIONALS_BOUNDARY
src/pkg/askai/limits.go 44 CONDITIONALS_NEGATION
src/pkg/askai/limits.go 122 INCREMENT_DECREMENT
src/pkg/askai/limits.go 137 CONDITIONALS_BOUNDARY
src/pkg/askai/service.go 168 CONDITIONALS_BOUNDARY
src/pkg/askai/service.go 236 CONDITIONALS_NEGATION
src/pkg/askai/service.go 264 INCREMENT_DECREMENT
src/pkg/askai/service.go 266 CONDITIONALS_NEGATION
src/pkg/askai/service.go 269 CONDITIONALS_BOUNDARY
src/pkg/askai/service.go 269 CONDITIONALS_NEGATION
src/pkg/askai/service.go 272 CONDITIONALS_BOUNDARY
src/pkg/askai/service.go 272 CONDITIONALS_NEGATION
src/pkg/askai/service.go 282 CONDITIONALS_NEGATION
src/pkg/askai/service.go 319 CONDITIONALS_BOUNDARY
src/pkg/askai/sources.go 130 CONDITIONALS_NEGATION
src/pkg/askai/sources.go 146 CONDITIONALS_BOUNDARY
src/pkg/askai/threads.go 68 CONDITIONALS_NEGATION
src/pkg/askai/threads.go 68 CONDITIONALS_NEGATION
src/pkg/hub/ask.go 141 CONDITIONALS_NEGATION
src/pkg/hub/ask.go 284 CONDITIONALS_NEGATION

32 mutant(s) in changed files are not reached by any test (NOT COVERED). A maintainer may apply the mutation/acknowledged label to accept this.

File Line Mutation
src/pkg/askai/breaker.go 13 ARITHMETIC_BASE
src/pkg/askai/consent.go 19 ARITHMETIC_BASE
src/pkg/askai/consent.go 20 ARITHMETIC_BASE
src/pkg/askai/consent.go 21 ARITHMETIC_BASE
src/pkg/askai/kapa.go 38 ARITHMETIC_BASE
src/pkg/askai/kapa.go 43 ARITHMETIC_BASE
src/pkg/askai/kapa.go 45 ARITHMETIC_BASE
src/pkg/askai/limits.go 11 ARITHMETIC_BASE
src/pkg/askai/limits.go 105 CONDITIONALS_BOUNDARY
src/pkg/askai/limits.go 105 CONDITIONALS_NEGATION
src/pkg/askai/limits.go 107 CONDITIONALS_BOUNDARY
src/pkg/askai/limits.go 107 CONDITIONALS_NEGATION
src/pkg/askai/limits.go 109 CONDITIONALS_BOUNDARY
src/pkg/askai/limits.go 109 CONDITIONALS_NEGATION
src/pkg/askai/limits.go 111 CONDITIONALS_BOUNDARY
src/pkg/askai/limits.go 111 CONDITIONALS_NEGATION
src/pkg/askai/limits.go 111 CONDITIONALS_NEGATION
src/pkg/askai/limits.go 113 CONDITIONALS_BOUNDARY
src/pkg/askai/limits.go 113 CONDITIONALS_NEGATION
src/pkg/askai/limits.go 115 CONDITIONALS_BOUNDARY
src/pkg/askai/limits.go 115 CONDITIONALS_NEGATION
src/pkg/askai/limits.go 117 CONDITIONALS_BOUNDARY
src/pkg/askai/limits.go 117 CONDITIONALS_NEGATION
src/pkg/askai/service.go 52 ARITHMETIC_BASE
src/pkg/askai/service.go 52 ARITHMETIC_BASE
src/pkg/askai/sources.go 15 ARITHMETIC_BASE
src/pkg/askai/sources.go 15 ARITHMETIC_BASE
src/pkg/askai/threads.go 13 ARITHMETIC_BASE
src/pkg/config/ask.go 88 ARITHMETIC_BASE
src/pkg/config/ask.go 89 ARITHMETIC_BASE
src/pkg/hub/ask.go 75 CONDITIONALS_NEGATION
src/pkg/hub/ask.go 80 CONDITIONALS_NEGATION

Verdict: lived (#17349).

@kubestellar-prow kubestellar-prow Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 19, 2026
Adds the ask config block and HIVE_ASK_* env (off by default, inert when
unconfigured, key only from a file under the allowed secret dirs), the
pkg/askai service (Kapa Chat API client pinned to the published
QueryResponse schema, per-person consent store, credential-shape refusal,
source allowlist, plain-text answers, per-person/per-hive/hub-wide limits,
circuit breaker, content-free audit lines) and the hub routes
GET /api/ask/status, PUT/DELETE /api/ask/consent, POST /api/ask and
GET /api/ask/stats. Routes are registered only when the feature is on.

Fixes #19180

Signed-off-by: Andy Anderson <andy@clubanderson.com>
…sleep

The Kapa Chat request is a model request (machine, reasonModelPrompt) in
the new outbound-sites inventory from v5. The in-flight test now waits on
a channel instead of polling with time.Sleep, keeping the sleep ratchet.

Signed-off-by: Andy Anderson <andy@clubanderson.com>
Lifts pkg/askai over the 90% package floor: bad vendor JSON, an invalid
thread id dropped, cancellation, the half-open breaker's single trial,
handle eviction, idle-counter pruning, corrupt consent records and the
service defaults.

Signed-off-by: Andy Anderson <andy@clubanderson.com>
@kubestellar-prow kubestellar-prow Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 19, 2026
@github-actions github-actions Bot added conflicts-with-open-pr Open PR has a merge conflict with another open PR and removed conflicts-with-open-pr Open PR has a merge conflict with another open PR labels Nov 19, 2026
Signed-off-by: Andy Anderson <andy@clubanderson.com>

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correctness review at head c0ad209 (read the diff plus the contract in #19180). Scope is met: routes are gated, consent gates every call, and the limiter reserves all-or-nothing. I found two edge cases in the failure paths, both small.

  1. A client disconnect counts as a vendor failure. service.go:236 calls breaker.failure for any error from Upstream.Chat. The call context derives from r.Context() (hub/ask.go handleAsk), and kapa_test.go shows context.Canceled is returned as a plain error. Three users who close the tab mid-request open the breaker for 5 minutes for everyone. Suggested fix, before the breaker call: if ctx.Err() != nil (the caller's context, not callCtx), release the trial with s.breaker.cancelTrial() and skip the failure count.
+		if ctx.Err() != nil {
+			s.breaker.cancelTrial()
+			return s.finish(req, s.result(OutcomeUpstreamError, msgUnavailable), questionLen, start)
+		}
 		if s.breaker.failure(s.now()) == breakerOpened {
  1. cancelTrial clears the trial flag unconditionally (breaker.go:66). A request admitted while the breaker was closed, then rate limited or given a 429 after the breaker has opened, clears a different request's half-open trial. A second trial call can then go out. The impact is low, because it needs a concurrent open/close race. Suggested fix: have allow return whether this caller took the trial slot, and call cancelTrial only when it did.

-- hive: agent=adjudicator backend=claude model=claude-sonnet-5-5 claude=2.1.327

Comment thread src/pkg/askai/service.go
r.RetryAfterSeconds = seconds(status.RetryAfter)
return s.finish(req, r, questionLen, start)
}
if s.breaker.failure(s.now()) == breakerOpened {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A caller-cancelled request (browser disconnect, ctx from r.Context()) is counted as a vendor failure; three tab closes open the breaker for 5 min. Skip the failure count when ctx.Err() != nil and release the trial.

Comment thread src/pkg/askai/breaker.go

// cancelTrial returns an admitted trial slot that never reached the vendor
// (for example, the request was rate limited after allow).
func (b *breaker) cancelTrial() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cancelTrial clears trial unconditionally, so a request that never took the trial slot can free another request's half-open trial. Return a 'took trial' bool from allow and cancel only then.

@github-actions github-actions Bot added the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 19, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delta re-review at 6b9fac6. The compare against c0ad209 touches none of this PR's files: src/pkg/askai, src/pkg/hub/ask*.go and src/pkg/config/ask*.go are all absent. The 45 changed files are merge-from-base noise, so the scope against #19180 is unchanged.

All three prior findings are still present, with fixes in the inline threads:

  • service.go:236: client cancellation counts as an upstream failure and can open the breaker.
  • breaker.go:66: cancelTrial clears the trial flag unconditionally.
  • kapa.go:160: redirects are followed with X-API-KEY set.

-- hive: agent=adjudicator-3 backend=claude model=claude-sonnet-5-5 claude=2.1.327

Comment thread src/pkg/askai/service.go
r.RetryAfterSeconds = seconds(status.RetryAfter)
return s.finish(req, r, questionLen, start)
}
if s.breaker.failure(s.now()) == breakerOpened {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A caller disconnect or ctx cancel returns ask: vendor call cancelled: %w from kapa.go:165, which reaches breaker.failure here as if the vendor failed. Three abandoned requests open the breaker for 5 minutes. Suggested fix:

Suggested change
if s.breaker.failure(s.now()) == breakerOpened {
if ctx.Err() != nil && !errors.Is(callCtx.Err(), context.DeadlineExceeded) {
s.breaker.cancelTrial()
return s.finish(req, s.result(OutcomeUpstreamError, msgUnavailable), questionLen, start)
}
if s.breaker.failure(s.now()) == breakerOpened {

(callCtx is cancelled by then, so capture callCtx.Err() before cancel() at line 226 if you use it.)

Comment thread src/pkg/askai/breaker.go

// cancelTrial returns an admitted trial slot that never reached the vendor
// (for example, the request was rate limited after allow).
func (b *breaker) cancelTrial() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cancelTrial clears trial without checking that this caller holds the slot. When the breaker is closed, or a different caller holds the trial, a rate-limited request calling it lets a second trial through. Suggested fix: have allow return whether it took the trial slot, and call cancelTrial only in that case. cancelTrial can also check b.open as a minimum.

Comment thread src/pkg/askai/kapa.go
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
req.Header.Set(kapaAPIKeyHeader, c.apiKey)
resp, err := c.http.Do(req)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

http.Client follows redirects by default, and Go strips only Authorization, WWW-Authenticate and Cookie on a cross-host hop. X-API-KEY would be forwarded to a redirect target. Suggested fix after the nil check at line 105: set CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse } on the default client. Apply the same to a caller-supplied client, or wrap it so a 3xx is treated as an upstream error.

@hivecommons-hive
hivecommons-hive Bot merged commit eaaaea3 into v5 Nov 19, 2026
84 of 85 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the feat/ask-hub-backend branch November 19, 2026 02:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conflicts-with-open-pr Open PR has a merge conflict with another open PR dco-signoff: yes Indicates the PR's author has signed the DCO. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

✨ ask: hub backend for optional Ask AI help (phase 1 of #19177)

1 participant