Repository navigation
🐛 fix(inference): reject requests without a configured model - #15039
hivecommons-hive[bot] merged 1 commit into
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
@thatguywob this PR is back in your court ( Failing checks:
Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path. Your relay session received this as a |
There was a problem hiding this comment.
Read this against the v5 tree rather than just the diff; it looks correct to me.
- Both production entry points call
routeWithEntitledModelbeforetranslateAnthropicToOpenAI(github_proxy_inference.go:395→429and622→645), so the new blank→DefaultModelinheritance atgithub_proxy_inference.go:248-252always runs before the new refusal atanthropic_translate.go:21-23. The third caller,forwardToInference(inference_forward.go:15), is only reached fromgateway_path_canary_test.go, so no production path loses the default. - The inherit copies the route (
rewritten := *route) the same way the entitlement rewrite below it does, so the stored snapshot stays untouched as the body says. recordAgentInferenceError(inference_last_error.go:74) andjsonEscape(github_proxy.go:434) already exist with the signatures used.
One note, not a defect: the src/docs/scale-envelope.md line-number bump is byte-identical to the one in #15044, so whichever lands second merges cleanly.
— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
There was a problem hiding this comment.
Read this against v5 (src/crux/proxy/), callers included. Looks correct to me.
What I checked:
- Both production entry points run
routeWithEntitledModelbeforetranslateAnthropicToOpenAI(github_proxy_inference.go:395→:429translator,:622→:645MITM), so the new blank→DefaultModelinheritance at the top ofrouteWithEntitledModelis applied on both paths, and it sits before theBackend != "litellm" || p.entitlements == nilearly return — so non-litellm backends and the pre-probe window inherit too, which is what the body claims. The copy-don't-mutate pattern matches the existing entitlement rewrites below it (:259,:277), and the test assertsroute.Modelis unchanged. - The third caller,
forwardToInference(inference_forward.go:15), has no non-test callers, so the missing-model guard intranslateAnthropicToOpenAIitself still covers it. - Install time already does
if model == "" { model = gw.DefaultModel }(src/cmd/hive/main.go:4400), so for most routesModelis already non-blank; the new path matters for a whitespace-only selection (which== ""misses) and for a default changed via the dashboard after install (src/pkg/dashboard/gateways.go:555). Not a defect — just noting the overlap so nobody expects this to change the common case. - No existing test calls
translateAnthropicToOpenAIwith an emptytargetModel, so the new hard error cannot regress them (CI agrees). default_modelin the error text is the real key (src/crux/config/agent_config.go:564).- Side fix: wrapping
err.Error()injsonEscapeat:441closes a latent malformed-JSON error body on the translator path; the MITM path useswriteHTTPError, which already builds its own body.
One suggestion, not a defect: the translator path returns the configuration error as HTTP 502 api_error. Since this is a local misconfiguration rather than an upstream failure, a 4xx might read better in CLI logs, but keeping 502 is consistent with the existing "translation error" handling and the dashboard surfaces the remedy either way.
— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
stage 095-pre-merge-soak → 060-reviewed: required-check-pending:Sign-off survives the squash |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of No conflicting open PRs or same-file overlaps were found. |
|
@thatguywob this PR is back in your court ( Review asks:
Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path. Your relay session received this as a |
Use the configured gateway default for a blank route model. When neither is set, return an actionable local translation error and retain it for the agent dashboard instead of forwarding an empty model to the provider. Hive-Run: hivecommons#15032 Hive-Plan: prevent-empty-inference-model Hive-Spec: issue-15032#empty-model-evidence Signed-off-by: thatguywob <thatguywob@users.noreply.github.com>
d064331 to
ef5dc5f
Compare
There was a problem hiding this comment.
Read the tree at ef5dc5f, not just the diff. Looks correct to me.
- Both live inference entry points go through
routeWithEntitledModelbefore translating (src/crux/proxy/github_proxy_inference.go:403and:631), so the new blank→DefaultModelinheritance at:249-253covers the HTTP translator and the MITM path alike. The copy-on-write matches the existing entitlement rewrite below it; the stored route is not mutated (the test asserts this atinference_missing_model_test.go:97). - The guard in
translateAnthropicToOpenAI(anthropic_translate.go:21) only fires when the model is still blank after that inheritance, so an operator withdefault_modelset sees no behaviour change. None of the 18 existingtranslateAnthropicToOpenAItest calls pass an empty target. jsonEscapeon the translator error body (github_proxy_inference.go:441) is a small correctness improvement — the new error text contains a colon and quotes that would otherwise have been embedded raw in the JSON.
Info only, not a defect: forwardToInference (inference_forward.go:14) also calls the translator but has no non-test callers, so the missing-model error there is unreachable in production.
— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
stage 095-pre-merge-soak → 060-reviewed: required-check-pending:Sign-off survives the squash |
|
Merged by hive contributor-lane auto-merge: CI green for ≥30m, reviewer verdict approve, @thatguywob holds the trusted tier in this hive's contributor registry. |
Fixes #15032
The reported LiteLLM 400 explicitly shows
model=. Hive can install a route with no model when neither the agent nor gateway supplies one, and the translator currently forwards that empty value on every request.Blank route models now inherit a configured gateway default before entitlement resolution, without mutating the stored route. If both values are blank, translation fails locally with instructions to select an agent model or set the gateway
default_model. Both inference entry points retain that error for the dashboard. The CLI's Claude model alias is never used to guess a gateway model. Operators with neither value configured still need to choose one; this change prevents invalid provider calls and exposes the remedy.Added regression coverage for both HTTP translator and MITM entry points: empty/whitespace models never reach the upstream, defaults are forwarded verbatim, explicit selections are preserved, and local errors appear in the dashboard error state. Added a changelog fragment.
Validation:
gofmt -lreturned no files. Tests were not run locally, as required byAGENTS.md; CI owns execution.— hive: backend=codex
🐝 Hive Agent:
contributor| SHA:65e68b62b