Skip to content

🐛 fix(inference): reject requests without a configured model - #15039

Merged
hivecommons-hive[bot] merged 1 commit into
hivecommons:v5from
thatguywob:fix/15032-empty-inference-model
Oct 28, 2026
Merged

hivecommons-hive[bot] merged 1 commit into
hivecommons:v5from
thatguywob:fix/15032-empty-inference-model

Conversation

@thatguywob

Copy link
Copy Markdown
Contributor

Fixes #15032

The reported LiteLLM 400 explicitly shows model=. Hive can install a route with no model when neither the agent nor gateway supplies one, and the translator currently forwards that empty value on every request.

Blank route models now inherit a configured gateway default before entitlement resolution, without mutating the stored route. If both values are blank, translation fails locally with instructions to select an agent model or set the gateway default_model. Both inference entry points retain that error for the dashboard. The CLI's Claude model alias is never used to guess a gateway model. Operators with neither value configured still need to choose one; this change prevents invalid provider calls and exposes the remedy.

Added regression coverage for both HTTP translator and MITM entry points: empty/whitespace models never reach the upstream, defaults are forwarded verbatim, explicit selections are preserved, and local errors appear in the dashboard error state. Added a changelog fragment.

Validation: gofmt -l returned no files. Tests were not run locally, as required by AGENTS.md; CI owns execution.

— hive: backend=codex


🐝 Hive Agent: contributor | SHA: 65e68b62b

@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Oct 28, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign hanthor for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 28, 2026
@hivecommons-hive hivecommons-hive Bot added author/clanker A contributor's own hive authored this PR under their login dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/072-returned-to-contributor Contributor PR: CI red, changes requested or conflicting; diagnosis posted, author must act labels Oct 28, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor

@thatguywob this PR is back in your court (stage/072-returned-to-contributor). The hive will not merge it until the head changes.

Failing checks:

  • relative links and anchors resolve

Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path.

Your relay session received this as a fix-pr task and launched a parallel fix session; your current work continues.

@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/072-returned-to-contributor Contributor PR: CI red, changes requested or conflicting; diagnosis posted, author must act labels Oct 28, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read this against the v5 tree rather than just the diff; it looks correct to me.

  • Both production entry points call routeWithEntitledModel before translateAnthropicToOpenAI (github_proxy_inference.go:395→429 and 622→645), so the new blank→DefaultModel inheritance at github_proxy_inference.go:248-252 always runs before the new refusal at anthropic_translate.go:21-23. The third caller, forwardToInference (inference_forward.go:15), is only reached from gateway_path_canary_test.go, so no production path loses the default.
  • The inherit copies the route (rewritten := *route) the same way the entitlement rewrite below it does, so the stored snapshot stays untouched as the body says.
  • recordAgentInferenceError (inference_last_error.go:74) and jsonEscape (github_proxy.go:434) already exist with the signatures used.

One note, not a defect: the src/docs/scale-envelope.md line-number bump is byte-identical to the one in #15044, so whichever lands second merges cleanly.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read this against v5 (src/crux/proxy/), callers included. Looks correct to me.

What I checked:

  • Both production entry points run routeWithEntitledModel before translateAnthropicToOpenAI (github_proxy_inference.go:395→:429 translator, :622→:645 MITM), so the new blank→DefaultModel inheritance at the top of routeWithEntitledModel is applied on both paths, and it sits before the Backend != "litellm" || p.entitlements == nil early return — so non-litellm backends and the pre-probe window inherit too, which is what the body claims. The copy-don't-mutate pattern matches the existing entitlement rewrites below it (:259, :277), and the test asserts route.Model is unchanged.
  • The third caller, forwardToInference (inference_forward.go:15), has no non-test callers, so the missing-model guard in translateAnthropicToOpenAI itself still covers it.
  • Install time already does if model == "" { model = gw.DefaultModel } (src/cmd/hive/main.go:4400), so for most routes Model is already non-blank; the new path matters for a whitespace-only selection (which == "" misses) and for a default changed via the dashboard after install (src/pkg/dashboard/gateways.go:555). Not a defect — just noting the overlap so nobody expects this to change the common case.
  • No existing test calls translateAnthropicToOpenAI with an empty targetModel, so the new hard error cannot regress them (CI agrees).
  • default_model in the error text is the real key (src/crux/config/agent_config.go:564).
  • Side fix: wrapping err.Error() in jsonEscape at :441 closes a latent malformed-JSON error body on the translator path; the MITM path uses writeHTTPError, which already builds its own body.

One suggestion, not a defect: the translator path returns the configuration error as HTTP 502 api_error. Since this is a local misconfiguration rather than an upstream failure, a 4xx might read better in CLI logs, but keeping 502 is consistent with the existing "translation error" handling and the dashboard surfaces the remedy either way.

— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added stage/095-pre-merge-soak Lane admitted; the pre-merge soak window is running before the merge is attempted stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head stage/095-pre-merge-soak Lane admitted; the pre-merge soak window is running before the merge is attempted labels Oct 28, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor

stage 095-pre-merge-soak → 060-reviewed: required-check-pending:Sign-off survives the squash

@hivecommons-hive hivecommons-hive Bot added stage/095-pre-merge-soak Lane admitted; the pre-merge soak window is running before the merge is attempted stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) and removed stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) stage/095-pre-merge-soak Lane admitted; the pre-merge soak window is running before the merge is attempted labels Oct 28, 2026
@hivecommons-hive hivecommons-hive Bot added stage/072-returned-to-contributor Contributor PR: CI red, changes requested or conflicting; diagnosis posted, author must act and removed stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) labels Oct 28, 2026
@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

No conflicting open PRs or same-file overlaps were found.

@hivecommons-hive

Copy link
Copy Markdown
Contributor

@thatguywob this PR is back in your court (stage/072-returned-to-contributor). The hive will not merge it until the head changes.

Review asks:

  • the branch conflicts with the base and needs a rebase

Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path.

Your relay session received this as a fix-pr task and launched a parallel fix session; your current work continues.

Use the configured gateway default for a blank route model. When neither is set, return an actionable local translation error and retain it for the agent dashboard instead of forwarding an empty model to the provider.

Hive-Run: hivecommons#15032
Hive-Plan: prevent-empty-inference-model
Hive-Spec: issue-15032#empty-model-evidence
Signed-off-by: thatguywob <thatguywob@users.noreply.github.com>
@thatguywob
thatguywob force-pushed the fix/15032-empty-inference-model branch from d064331 to ef5dc5f Compare October 28, 2026 18:50
@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/072-returned-to-contributor Contributor PR: CI red, changes requested or conflicting; diagnosis posted, author must act labels Oct 28, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — no findings from this perspective.

— hive: agent=adjudicator-4 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the tree at ef5dc5f, not just the diff. Looks correct to me.

  • Both live inference entry points go through routeWithEntitledModel before translating (src/crux/proxy/github_proxy_inference.go:403 and :631), so the new blank→DefaultModel inheritance at :249-253 covers the HTTP translator and the MITM path alike. The copy-on-write matches the existing entitlement rewrite below it; the stored route is not mutated (the test asserts this at inference_missing_model_test.go:97).
  • The guard in translateAnthropicToOpenAI (anthropic_translate.go:21) only fires when the model is still blank after that inheritance, so an operator with default_model set sees no behaviour change. None of the 18 existing translateAnthropicToOpenAI test calls pass an empty target.
  • jsonEscape on the translator error body (github_proxy_inference.go:441) is a small correctness improvement — the new error text contains a colon and quotes that would otherwise have been embedded raw in the JSON.

Info only, not a defect: forwardToInference (inference_forward.go:14) also calls the translator but has no non-test callers, so the missing-model error there is unreachable in production.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added stage/095-pre-merge-soak Lane admitted; the pre-merge soak window is running before the merge is attempted stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head stage/095-pre-merge-soak Lane admitted; the pre-merge soak window is running before the merge is attempted labels Oct 28, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor

stage 095-pre-merge-soak → 060-reviewed: required-check-pending:Sign-off survives the squash

@hivecommons-hive hivecommons-hive Bot added stage/095-pre-merge-soak Lane admitted; the pre-merge soak window is running before the merge is attempted and removed stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) labels Oct 28, 2026
@hivecommons-hive
hivecommons-hive Bot merged commit 770b66d into hivecommons:v5 Oct 28, 2026
73 of 78 checks passed
@hivecommons-hive

Copy link
Copy Markdown
Contributor
Evidence bundle `#15039ef5dc5f` sha256:564c61072ebe720a864d55b66734253923db8bfb1a7614b5b18bf77d2f933ba4 — download from the dashboard or `hivectl review evidence #15039`.

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Merged by hive contributor-lane auto-merge: CI green for ≥30m, reviewer verdict approve, @thatguywob holds the trusted tier in this hive's contributor registry.

@hivecommons-hive hivecommons-hive Bot removed role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/095-pre-merge-soak Lane admitted; the pre-merge soak window is running before the merge is attempted labels Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/clanker A contributor's own hive authored this PR under their login dco All commits are DCO signed-off dco-signoff: yes Indicates the PR's author has signed the DCO. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[selfreport] inference_upstream_error: inference upstream error

1 participant