Repository navigation
✨ knowledge: advisor_llm config block with hard budget ceilings - #17167
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of
|
|
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor. hive/awaiting-ci: CI pending — sweep will merge when green. 🐝 Hive Agent: — hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 Review findings from Read at head What I verified
One note, not a defect
|
There was a problem hiding this comment.
The config block itself reads correctly to me: validateKnowledgeAdvisorLLM rejects bad enums and negatives, enforces the two ceilings unless budget_override, and KnowledgeAdvisorLLM() falls back to governor.litellm for endpoint/key/model the same way ResolveReviewer does; the key is resolved from file/env and never kept on the YAML struct. Validation is wired into ValidateWithOptions (validate.go:198) before the connectors check.
What is blocking CI is caused by this diff and is a one-file fix:
src/pkg/config/testdata/agent_runtime_golden.json—TestAgentRuntimeConfigGoldenfails intest (rest 2/4),test (dashboard 1/4),test (dashboard shuffle 1/3),coverageandtestwith.Knowledge.AdvisorLLM: unexpected. AddingAdvisorLLM AdvisorLLMConfigtoKnowledgeConfig(knowledge_config.go:61) changes the parsed shape the golden pins. Peragent_runtime_golden_test.go:20, regenerate withHIVE_UPDATE_GOLDEN=1 go test ./pkg/config -run TestAgentRuntimeConfigGoldenand commit the updated JSON; the schema change is intentional here.
Minor, not blocking: the negative-value error at knowledge_advisor_llm.go:110 says "must be at least 1" while the check is < 0 and 0 is accepted as "use the default"; "must not be negative" would match the behaviour.
— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Pushed f944cc6 for the remaining review note: the negative-value error now reads "must not be negative (omit or 0 for the default)", matching the hive/awaiting-ci: CI pending — sweep will merge when green. 🐝 Hive Agent: — hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com> Co-authored-by: clubanderson <407614+clubanderson@users.noreply.github.com>
The settings parity test requires every hive.yaml leaf to be either in the Settings UI or allowlisted; advisor_llm is yaml-only while it soaks, like knowledge.advisor. The agent-runtime golden gains the new zero-value AdvisorLLM block. Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
Zero is accepted as 'use the default'; only negatives are rejected, so say so. Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
f944cc6 to
824386a
Compare
|
mobile-shots failed on the 🐝 Hive Agent: — hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 |
There was a problem hiding this comment.
Read against v5 (src/pkg/config/config.go, validate.go, knowledge_config.go). Looks correct to me; no defects found.
KnowledgeAdvisorLLM()(knowledge_advisor_llm.go:118-160) followsGovernorConfig.ResolveReviewer(v5:config.go:2697-2712) field for field: trimmed explicit endpoint →LiteLLM.ResolveEndpoint(),resolveKeyFromFileThenEnv→LiteLLM.ResolveAPIKey(), explicit model →LiteLLM.DefaultModel. The "same precedence as the reviewer" claim in the doc comment is accurate.- Validation returns errors for negative counts, bad enum values,
holdout_percentoutside 0–100, and ceilings exceeded withoutbudget_override(:76-113); shadow/rank without a model is a warning that resolves tooff(:109-111,:152-154), not an error. Thelog.Printf("WARNING: …")path fromvalidate.go:202is the idiomconfig.go:6474/:6624already use. - The block is a no-op unless
knowledge.advisoris true (:119), so existing configs are unaffected; the 13 new keys are allowlisted for settings parity with the same soak note asknowledge.advisor(#17052).
One thing worth knowing rather than fixing: with budget_override: true both ceilings are unbounded (:97-104 is skipped entirely) rather than raised to a higher cap. That matches the body's "overridable only explicitly" and the docs, so it is a design choice, not a bug.
— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
The model-ranked advisor needs its own model, endpoint and spend limits; the existing advisor tier is a contributor trust tier and routes nothing. Landing a typed, validated config surface first lets the ranking decorator build on it, and the hard call ceilings (overridable only explicitly) bound cost before any model call exists. Misconfigured shadow/rank without a model degrades to off with a warning instead of blocking the hive from loading.
Closes #17154
Part of #17054 (slice 2 of the advisor phase 2 plan; later slices remain open)
— hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88 requested_by=@clubanderson