Skip to content

✨ knowledge: advisor_llm config block with hard budget ceilings - #17167

Merged
hivecommons-hive[bot] merged 3 commits into
v5from
scanner/fix-17154
Nov 8, 2026
Merged

hivecommons-hive[bot] merged 3 commits into
v5from
scanner/fix-17154

Conversation

@hivecommons-hive

Copy link
Copy Markdown
Contributor

The model-ranked advisor needs its own model, endpoint and spend limits; the existing advisor tier is a contributor trust tier and routes nothing. Landing a typed, validated config surface first lets the ranking decorator build on it, and the hard call ceilings (overridable only explicitly) bound cost before any model call exists. Misconfigured shadow/rank without a model degrades to off with a warning instead of blocking the hive from loading.

Closes #17154
Part of #17054 (slice 2 of the advisor phase 2 plan; later slices remain open)

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 requested_by=@clubanderson

@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 8, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign danathar for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Nov 8, 2026
@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#17170 ✨ crux: asynchronous KnowledgeAdviser with scope Action and scrubbed PaneTail same files src/docs/agent-knowledge-and-memory.md
src/docs/knowledge-curator.md
#17123 ✨ knowledge: finish CLI memory drain phase 1 — dedup tags, spool, draft create path same files src/docs/agent-knowledge-and-memory.md

@hivecommons-hive hivecommons-hive Bot added author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 8, 2026
@hivecommons-hive

hivecommons-hive Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor Author

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

hive/awaiting-ci: CI pending — sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: ad80df003

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

Review findings from reviewer-2 (folded here because this head already carries its hive review):

Read at head 824386a against the v5 tree, then against #17154. Looks correct to me; nothing blocking.

What I verified

  • src/pkg/config/knowledge_advisor_llm.go:150-160 — KnowledgeAdvisorLLM() resolves endpoint/key/model in the same order as GovernorConfig.ResolveReviewer (config.go:2700-2710): explicit → governor.litellm → unset. Reuses the existing resolveKeyFromFileThenEnv (config.go:2727) rather than re-implementing it.
  • knowledge_advisor_llm.go:70-123 — every rule in ✨ knowledge: knowledge.advisor_llm config block with hard budget ceilings (advisor phase 2, slice 2) #17154 is enforced: mode / on_error enums are load errors, negative counts are load errors, holdout_percent is range-checked, the 6/300 ceilings are load errors naming the key unless budget_override: true, and shadow/rank without a model is a warning that KnowledgeAdvisorLLM() then folds to off (:174). The warning surfaces via log.Printf("WARNING: …") in validate.go:198-204, which is the idiom config.go already uses for tolerated misconfig.
  • Table test knowledge_advisor_llm_test.go:14-54 covers each of those branches including the two the issue calls out by name.
  • Golden (testdata/agent_runtime_golden.json) and the settings-parity allowlist are both updated, so the dashboard-parity and golden guards will not trip.
  • No vendor/model names in code defaults, fixtures or docs (model: "<your-small-model>").

One note, not a defect

advisorLLMModel() (:60-65) falls back to governor.litellm.default_model, so mode: shadow with an empty model but a configured litellm default does not resolve to off. #17154 says both "follow ResolveReviewer" and "empty model forces off"; the PR picked the former and the docs (knowledge-curator.md) describe that behaviour accurately. Flagging only so the decorator slice knows which reading landed.

Evidence bundle `#17167824386a` sha256:fef2a41c959f45e7a58d2fdc53c00335d855157d6fe9076f39706ff8a6926b9a — download from the dashboard or `hivectl review evidence #17167`.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The config block itself reads correctly to me: validateKnowledgeAdvisorLLM rejects bad enums and negatives, enforces the two ceilings unless budget_override, and KnowledgeAdvisorLLM() falls back to governor.litellm for endpoint/key/model the same way ResolveReviewer does; the key is resolved from file/env and never kept on the YAML struct. Validation is wired into ValidateWithOptions (validate.go:198) before the connectors check.

What is blocking CI is caused by this diff and is a one-file fix:

  • src/pkg/config/testdata/agent_runtime_golden.json — TestAgentRuntimeConfigGolden fails in test (rest 2/4), test (dashboard 1/4), test (dashboard shuffle 1/3), coverage and test with .Knowledge.AdvisorLLM: unexpected. Adding AdvisorLLM AdvisorLLMConfig to KnowledgeConfig (knowledge_config.go:61) changes the parsed shape the golden pins. Per agent_runtime_golden_test.go:20, regenerate with HIVE_UPDATE_GOLDEN=1 go test ./pkg/config -run TestAgentRuntimeConfigGolden and commit the updated JSON; the schema change is intentional here.

Minor, not blocking: the negative-value error at knowledge_advisor_llm.go:110 says "must be at least 1" while the check is < 0 and 0 is accepted as "use the default"; "must not be negative" would match the behaviour.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it labels Nov 8, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Pushed f944cc6 for the remaining review note: the negative-value error now reads "must not be negative (omit or 0 for the default)", matching the < 0 check. The golden regeneration from the blocking finding landed earlier in ad80df0.

hive/awaiting-ci: CI pending — sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: unknown

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Nov 8, 2026
hivecommons-hive Bot and others added 3 commits November 7, 2026 22:12
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
Co-authored-by: clubanderson <407614+clubanderson@users.noreply.github.com>
The settings parity test requires every hive.yaml leaf to be either in
the Settings UI or allowlisted; advisor_llm is yaml-only while it soaks,
like knowledge.advisor. The agent-runtime golden gains the new zero-value
AdvisorLLM block.

Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
Zero is accepted as 'use the default'; only negatives are rejected, so say so.

Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

mobile-shots failed on the tracker-phone-chrome activity probe, which this config-only PR doesn't touch. v5 has since landed the probe fix (fa453b5, #17189) — rebased onto v5 to pick it up; no code changes.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: 824386ae9

— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it labels Nov 8, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read against v5 (src/pkg/config/config.go, validate.go, knowledge_config.go). Looks correct to me; no defects found.

  • KnowledgeAdvisorLLM() (knowledge_advisor_llm.go:118-160) follows GovernorConfig.ResolveReviewer (v5:config.go:2697-2712) field for field: trimmed explicit endpoint → LiteLLM.ResolveEndpoint(), resolveKeyFromFileThenEnv → LiteLLM.ResolveAPIKey(), explicit model → LiteLLM.DefaultModel. The "same precedence as the reviewer" claim in the doc comment is accurate.
  • Validation returns errors for negative counts, bad enum values, holdout_percent outside 0–100, and ceilings exceeded without budget_override (:76-113); shadow/rank without a model is a warning that resolves to off (:109-111, :152-154), not an error. The log.Printf("WARNING: …") path from validate.go:202 is the idiom config.go:6474/:6624 already use.
  • The block is a no-op unless knowledge.advisor is true (:119), so existing configs are unaffected; the 13 new keys are allowlisted for settings parity with the same soak note as knowledge.advisor (#17052).

One thing worth knowing rather than fixing: with budget_override: true both ceilings are unbounded (:97-104 is skipped entirely) rather than raised to a higher cap. That matches the body's "overridable only explicitly" and the docs, so it is a design choice, not a bug.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive
hivecommons-hive Bot merged commit 3f1177a into v5 Nov 8, 2026
87 of 90 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the scanner/fix-17154 branch November 8, 2026 03:45
@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/hive This repository's own hive (the App or one of its agent lanes) authored this PR copilot dco All commits are DCO signed-off dco-signoff: yes Indicates the PR's author has signed the DCO. role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) size/L Denotes a PR that changes 100-499 lines, ignoring generated files. stage/050-review-pending Review swarm requested; awaiting a verdict for the current head

Projects

None yet

Development

Successfully merging this pull request may close these issues.

✨ knowledge: knowledge.advisor_llm config block with hard budget ceilings (advisor phase 2, slice 2)

0 participants