Repository navigation
✨ ask: hub backend for optional Ask AI help - #19200
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
|
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor. @clubanderson this PR is held at Failing checks:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. ℹ️ Sentinel: informational, author is trusted (owner), merge is not blocked. Head
No hive/awaiting-ci: rebased onto v5 (one docs table conflict, took v5 rows; the Ask rows live in hub-saas.md and are intact). CI pending - sweep will merge when green. 🐝 Hive Agent: merged by Hive trusted-author auto-merge: author |
Mutation testinggremlins found 26 surviving mutant(s) in changed files: the code ran under tests, but changing it did not fail any test.
32 mutant(s) in changed files are not reached by any test (NOT COVERED). A maintainer may apply the
Verdict: |
Adds the ask config block and HIVE_ASK_* env (off by default, inert when unconfigured, key only from a file under the allowed secret dirs), the pkg/askai service (Kapa Chat API client pinned to the published QueryResponse schema, per-person consent store, credential-shape refusal, source allowlist, plain-text answers, per-person/per-hive/hub-wide limits, circuit breaker, content-free audit lines) and the hub routes GET /api/ask/status, PUT/DELETE /api/ask/consent, POST /api/ask and GET /api/ask/stats. Routes are registered only when the feature is on. Fixes #19180 Signed-off-by: Andy Anderson <andy@clubanderson.com>
…sleep The Kapa Chat request is a model request (machine, reasonModelPrompt) in the new outbound-sites inventory from v5. The in-flight test now waits on a channel instead of polling with time.Sleep, keeping the sleep ratchet. Signed-off-by: Andy Anderson <andy@clubanderson.com>
Lifts pkg/askai over the 90% package floor: bad vendor JSON, an invalid thread id dropped, cancellation, the half-open breaker's single trial, handle eviction, idle-counter pruning, corrupt consent records and the service defaults. Signed-off-by: Andy Anderson <andy@clubanderson.com>
142019f to
04994d3
Compare
Signed-off-by: Andy Anderson <andy@clubanderson.com>
There was a problem hiding this comment.
Correctness review at head c0ad209 (read the diff plus the contract in #19180). Scope is met: routes are gated, consent gates every call, and the limiter reserves all-or-nothing. I found two edge cases in the failure paths, both small.
- A client disconnect counts as a vendor failure.
service.go:236callsbreaker.failurefor any error fromUpstream.Chat. The call context derives fromr.Context()(hub/ask.gohandleAsk), andkapa_test.goshowscontext.Canceledis returned as a plain error. Three users who close the tab mid-request open the breaker for 5 minutes for everyone. Suggested fix, before the breaker call: ifctx.Err() != nil(the caller's context, notcallCtx), release the trial withs.breaker.cancelTrial()and skip the failure count.
+ if ctx.Err() != nil {
+ s.breaker.cancelTrial()
+ return s.finish(req, s.result(OutcomeUpstreamError, msgUnavailable), questionLen, start)
+ }
if s.breaker.failure(s.now()) == breakerOpened {cancelTrialclears the trial flag unconditionally (breaker.go:66). A request admitted while the breaker was closed, then rate limited or given a 429 after the breaker has opened, clears a different request's half-open trial. A second trial call can then go out. The impact is low, because it needs a concurrent open/close race. Suggested fix: haveallowreturn whether this caller took the trial slot, and callcancelTrialonly when it did.
-- hive: agent=adjudicator backend=claude model=claude-sonnet-5-5 claude=2.1.327
| r.RetryAfterSeconds = seconds(status.RetryAfter) | ||
| return s.finish(req, r, questionLen, start) | ||
| } | ||
| if s.breaker.failure(s.now()) == breakerOpened { |
There was a problem hiding this comment.
A caller-cancelled request (browser disconnect, ctx from r.Context()) is counted as a vendor failure; three tab closes open the breaker for 5 min. Skip the failure count when ctx.Err() != nil and release the trial.
|
|
||
| // cancelTrial returns an admitted trial slot that never reached the vendor | ||
| // (for example, the request was rate limited after allow). | ||
| func (b *breaker) cancelTrial() { |
There was a problem hiding this comment.
cancelTrial clears trial unconditionally, so a request that never took the trial slot can free another request's half-open trial. Return a 'took trial' bool from allow and cancel only then.
There was a problem hiding this comment.
Delta re-review at 6b9fac6. The compare against c0ad209 touches none of this PR's files: src/pkg/askai, src/pkg/hub/ask*.go and src/pkg/config/ask*.go are all absent. The 45 changed files are merge-from-base noise, so the scope against #19180 is unchanged.
All three prior findings are still present, with fixes in the inline threads:
service.go:236: client cancellation counts as an upstream failure and can open the breaker.breaker.go:66:cancelTrialclears the trial flag unconditionally.kapa.go:160: redirects are followed withX-API-KEYset.
-- hive: agent=adjudicator-3 backend=claude model=claude-sonnet-5-5 claude=2.1.327
| r.RetryAfterSeconds = seconds(status.RetryAfter) | ||
| return s.finish(req, r, questionLen, start) | ||
| } | ||
| if s.breaker.failure(s.now()) == breakerOpened { |
There was a problem hiding this comment.
A caller disconnect or ctx cancel returns ask: vendor call cancelled: %w from kapa.go:165, which reaches breaker.failure here as if the vendor failed. Three abandoned requests open the breaker for 5 minutes. Suggested fix:
| if s.breaker.failure(s.now()) == breakerOpened { | |
| if ctx.Err() != nil && !errors.Is(callCtx.Err(), context.DeadlineExceeded) { | |
| s.breaker.cancelTrial() | |
| return s.finish(req, s.result(OutcomeUpstreamError, msgUnavailable), questionLen, start) | |
| } | |
| if s.breaker.failure(s.now()) == breakerOpened { |
(callCtx is cancelled by then, so capture callCtx.Err() before cancel() at line 226 if you use it.)
|
|
||
| // cancelTrial returns an admitted trial slot that never reached the vendor | ||
| // (for example, the request was rate limited after allow). | ||
| func (b *breaker) cancelTrial() { |
There was a problem hiding this comment.
cancelTrial clears trial without checking that this caller holds the slot. When the breaker is closed, or a different caller holds the trial, a rate-limited request calling it lets a second trial through. Suggested fix: have allow return whether it took the trial slot, and call cancelTrial only in that case. cancelTrial can also check b.open as a minimum.
| req.Header.Set("Content-Type", "application/json") | ||
| req.Header.Set("Accept", "application/json") | ||
| req.Header.Set(kapaAPIKeyHeader, c.apiKey) | ||
| resp, err := c.http.Do(req) |
There was a problem hiding this comment.
http.Client follows redirects by default, and Go strips only Authorization, WWW-Authenticate and Cookie on a cross-host hop. X-API-KEY would be forwarded to a redirect target. Suggested fix after the nil check at line 105: set CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse } on the default client. Apply the same to a caller-supplied client, or wrap it so a 3xx is treated as an upstream error.
Phase 1 of the Ask AI help design (
src/docs/design/dashboard-ask-ai-help.md, isolation model (a)): the hub backend only, no panel.Fixes #19180
What it does
ask:config type andHIVE_ASK_*env inpkg/config/ask.go. Off by default. Switched on but missing a value (or a key file outside/secretsor/data/secrets, unreadable or empty) keeps it off with one warning that names no value. The key is only ever read from a file; there is no env var that holds it. Theask:block is attached tohive.yamlin phase 4 (✨ ask: hive backend and hub relay for optional Ask AI help (phase 4 of #19177) #19183), where the hive reads it; the hub reads env only.pkg/askai: the Kapa Chat API client (POST /query/v1/projects/{id}/chat/, follow-ups on/query/v1/threads/{id}/chat/, key inX-API-KEY), pinned to the publishedQueryResponseschema (answer,thread_id,question_answer_id,is_uncertain,relevant_sources[].source_url/title/contains_internal_data) by a contract test against an httptest fake. The body is exactly{integration_id, query}: nouser, nometadata.origin_url.0600, atomic,text_version, grant and withdraw, deleted with the user); credential-shaped questions refused using the log scrubber's patterns, nothing sent; the closed-vocabulary section label (pkg/askai/sections.go) prepended when on; source allowlist (docs.hivecommons.devand the GitHub docs path underhivecommons/hive), internal sources dropped; answer reduced to plain text (markdown links to their text, non-allowlisted URLs removed, capped); no surviving source meansno_cited_answer, never uncited AI text; vendor thread ids kept server side behind an opaque handle bound to the person (and hive), with a TTL.audit: ask request,audit: ask consent grant|withdraw,audit: ask breaker opened|closed,audit: ask daily budget spent. Counters by outcome atGET /api/ask/stats(hub admin).GET /api/ask/status,PUT/DELETE /api/ask/consent,POST /api/ask(browser session only: no bearer, no impersonation, CSRF-checked),GET /api/ask/stats.Tests (behaviour, against a fake vendor)
sk-key, AWS key) refused with zero vendor calls and no text in audit lines.Operator notes
Nothing changes until
HIVE_ASK_ENABLEDand the Kapa values are set. How to wire the real key (Secret mounted under/secrets/kapa, env vars) is insrc/docs/hub-deployment.md#ask-ai-help. Docs: env vars, hub API reference, telemetry outbound-path row.