Skip to content

[Bug]: vLLM provider editor hides the context-window override needed for unknown models #1409

Description

@aaaajanus2016

OpenSquilla version or commit

0.5.3

Area

Web UI

Reproduction steps

  1. Configure a keyless OpenAI-compatible vLLM endpoint and a model ID that is not present in the packaged model catalog:
    opensquilla onboard --provider vllm --model qa-model --base-url http://127.0.0.1:8000/v1 --router disabled --minimal --skip-channels --skip-search --skip-image-generation
  2. Do not set llm.context_window_tokens or a [models.vllm."qa-model"] override.
  3. Start the Gateway and open Control UI → Settings → Model Services → Edit vLLM.
  4. Inspect the visible fields in the editor.
  5. Send a short WebChat message.

Expected behavior

The editor should expose the documented per-provider/model context-window override, including the auto-detected/effective readout and local-runtime warning. A user should be able to set the real window of a self-hosted model without editing TOML or applying a global override.

Actual behavior

The visible vLLM editor contains only Base URL, HTTP proxy, API key, connection test, and direct/fallback model. There is no context-window control.

The control is mounted once in the DOM as:

input[name="setup_provider_context_window"]
visible: false
rect: 0 × 0
ancestor: <div hidden style="display: none">

With no override, the model row reports an unknown/zero context window and a short WebChat message fails before any provider request:

The request is too large for the provider context window after automatic context compaction and payload reduction.

Setting a sufficiently large global context override and restarting immediately makes the same WebChat request succeed, confirming that the endpoint itself is healthy. The global override is only a workaround; it applies to every active model.

Environment

windows 11

Diagnostic bundle and error reference

No error reference was shown. Browser console had no errors. The reproduction uses no credentials.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions