Skip to content

πŸ› Fall back to the gateway default_model when the per-agent model is not entitled - #12160

Merged
hivecommons-hive[bot] merged 1 commit into
v5from
fix/9945-gateway-default-model-fallback
Oct 15, 2026
Merged

hivecommons-hive[bot] merged 1 commit into
v5from
fix/9945-gateway-default-model-fallback

Conversation

@clubanderson

Copy link
Copy Markdown
Member

Refs #9945 β€” cc @MikeSpreitzer

Root cause

After #11794 the endpoint and key were right, and Mike's new error showed the remaining dimension: the model.

team not allowed to access model. This team can only access models=['rits/zai-org/glm-5-3', 'rits/nvidia/…']. Tried to access aws/claude-sonn…

The route resolver honours a per-agent model: over the gateway default_model (correct in general). The FMA agents still carry model: claude-sonnet-4-6 from their original copilot backend (the pack template writes it for every agent). When the backend was switched to litellm, that value came along; the #4400 entitlement rewrite then mapped it to the key-info-listed aws/claude-sonnet-4-6, and LiteLLM's team scope refused it. The gateway default β€” the only thing Mike could see, and the one model the team can use β€” was never considered. The "Please run /login" prefix is just the Claude CLI's rendering of any 403; the request itself was built by Hive's translator with the route's model.

Fix

  • proxy.InferenceRoute.DefaultModel carries the backend's default (gateway default_model, or the legacy block's) beside Model, for litellm / named-gateway / watsonx routes.
  • routeWithEntitledModel: once the gateway has told us the entitled set (key-info probe or the 403 itself), a per-agent model that is neither entitled nor a spelling drift of an entitled id is replaced per request by the gateway default (when that is entitled). A warning names the agent, the configured model, the default sent and the remedy. The stored route is never mutated; an entitled explicit model still wins; no default / un-entitled default β†’ unchanged, so the gateway's own 403 still reaches the operator. Self-healing: first 403 teaches the set, second request succeeds.
  • Both Test buttons now say what they tested: the opened card's form Test (/gateways/discover) returns the same Endpoint / Default model / Using key / SHA256 / Connection OK / Agents on backend "litellm" send inference to POST … (model …) text as the saved-card Test, and flags when the endpoint typed is not saved yet. The UI renders it instead of a bare "OK β€” N models available".

Tests

  • pkg/proxy TestRouteWithEntitledModel_FallsBackToGatewayDefault β€” reproduces with the live FMA 403 body: no per-agent model β†’ default; stale claude-sonnet-4-6 β†’ default after the 403; outbound OpenAI body never carries the CLI's own model; entitled explicit model wins; drifted default resolves; un-entitled default / no default pass through.
  • cmd/hive TestBootProxyWith_LiteLLMGatewayDefaultModelRidesAlong β€” gateway default X rides on the route for agents with and without a model, named gateways and legacy-only litellm.
  • pkg/dashboard TestGatewayFormTestSuccessEchoesWhatWasTested (+ static wiring) β€” form and saved Test success diagnostics are byte-identical, include the route line, never leak the key, and warn on an unsaved endpoint.

go vet ./pkg/... ./cmd/..., go build ./..., go test ./pkg/agent/... ./pkg/dashboard/... ./pkg/config/... ./pkg/proxy/... ./pkg/inference/... ./cmd/hive/... pass (two TestApplySessionEnv_* tmux tests fail locally on macOS with "socket path too long" β€” pre-existing, unrelated).

…not entitled (Refs #9945)

An agent entry's own model: (typically left over from an earlier copilot/
claude backend, e.g. claude-sonnet-4-6) always beat the litellm gateway's
default_model, so FMA agents sent every request for aws/claude-sonnet-4-6
and 403'd with 'team not allowed to access model' while the gateway card
showed the entitled default rits/zai-org/glm-5-3.

- InferenceRoute carries DefaultModel (gateway default_model or the legacy
  block's default) beside Model for litellm, named-gateway and watsonx routes.
- routeWithEntitledModel: when the stored model is neither entitled nor a
  drift of an entitled id, but the DefaultModel is, forward the default and
  warn with the agent name and remedy. Stored route is never mutated.
- Gateway form Test (discover) now returns the same endpoint / default model
  / key SHA / route diagnostic on success as the saved-card Test, and flags
  an endpoint that is not saved yet; the UI renders it instead of a bare
  'OK β€” N models'.
- Tests: proxy fallback repro with the live FMA 403 body, boot-level route
  resolution carrying DefaultModel, dashboard form-Test success parity.

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Oct 15, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign clubanderson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 15, 2026
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#12139 ✨ dashboard settings: expose intent pattern lists and sentinel list keys same files src/pkg/dashboard/static/index.html
#12158 ✨ review: add review.shadow to record would_have_held for the review gate same files src/pkg/dashboard/static/index.html
#12159 ✨ dashboard: iPhone widget entry in the avatar menu deep-links into Scriptable same files src/pkg/dashboard/static/index.html
#12025 ✨ keep unstick steps running in conserve API budget mode same files src/cmd/hive/main.go
#12154 🌱 tokens: trajectory review β€” skip only on proven-idle agent state and not overdue, with skipped metrics same files src/cmd/hive/main.go
src/pkg/dashboard/static/index.html
#12152 πŸ› dashboard: covered-by-PR glyph from hive's claim evidence; never closing-link a reference claim same files src/pkg/dashboard/static/index.html

@hivecommons-hive

Copy link
Copy Markdown
Contributor

merged by Hive trusted-author auto-merge: author @clubanderson holds role owner and GitHub permission admin on repo; CI green at 3b90a4c.

@kubestellar-prow
kubestellar-prow Bot deleted the fix/9945-gateway-default-model-fallback branch October 15, 2026 19:04
@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the DCO. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant