Skip to content

fix(windows): honor active model routing and credentials in CLI chat - #7214

Closed
tang-vu wants to merge 3 commits into
Osmantic:mainfrom
tang-vu:fix/native-quality-batch-20261003
Closed

tang-vu wants to merge 3 commits into
Osmantic:mainfrom
tang-vu:fix/native-quality-batch-20261003

Conversation

@tang-vu

@tang-vu tang-vu commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Closes #7298. Windows ods.ps1 chat currently bypasses an enabled switchboard and sends neither the native llama-server credential nor UTF-8 prompt bytes. Use ods/current with the saved gateway port/key when the switchboard is enabled; otherwise honor the resolved native endpoint's LLAMA_SERVER_API_KEY and saved GGUF alias. Missing required keys refuse the request. Failed requests remain on the selected route.

This current-main maintenance removes the retired Lemonade branch and replaces its obsolete wire tests. Docker llama-server retains the existing default-model behavior, configured port/API path, and no native-key forwarding.

Why this matters

The shipped Windows CLI is a real client of the configured inference route. On current main, ods.ps1 chat bypasses a selected switchboard; direct native requests omit the required credential, and Windows PowerShell 5.1 string serialization can corrupt Unicode prompts. This restores that public command across the supported current endpoints without bypassing a failed selected route.

Overlap check

Reviewed current main's endpoint resolver and native llama startup contract, plus the existing PR's original scope. This updates #7214 in place; it does not reintroduce the retired Lemonade runtime. The other maintained tang-vu model/UI PRs concern download staging and remote-peer inventory, not Windows CLI request routing. Existing endpoint/readiness contracts remain intact.

Validation

Base: 36b950130c7378c83946213569c3766d2e2b3c8b. Candidate: f6409e57d8283250dce5647694e310b6c7675a98.

The original published head 3273c97e8e8db6d60b662e03a907772242605bf3 is preserved as an ancestor through a normal main merge. Its eight historical wire tests fail five retired-Lemonade assumptions after integration onto this main. The first eight replacement regressions fail seven cases on exact unmodified main, including missing routing/auth and Unicode handling; the unchanged Docker control passes.

Agent-executed native Windows checks on the candidate source (including an independent root replay of all nine HTTP-wire cases at this exact candidate):

  • 9 PowerShell 5.1 loopback HTTP-wire cases passed: gateway alias/key, native alias/key/Unicode, missing-key refusals, HTTP failure without route bypass, default Docker behavior, configured Docker port/path, and no native key leakage to Docker.
  • Windows local endpoint resolver contracts passed.
  • Native llama-server legacy lifecycle contracts: 86 checks passed with process/task/socket fixtures.
  • Model-readiness shell contract: 16 checks passed, including the missing-model-file behavior under portable PowerShell 7. The earlier PATH-only run skipped that behavior; the final root replay used the available portable runtime.
  • Ruff E/F/W with CI E501 exclusion, whitespace, and all three added author/committer identity checks passed.

Existing Windows workflow explicitly executes test-windows-cli-chat.py; no test-only file is left unexecuted. Root also ran the available portable PSScriptAnalyzer against the CLI with the exact CI settings: zero warnings/errors. Hosted current-head checks are monitored separately.

AI Assistance

Tang Vu (tang-vu) provided project direction, scope decisions and environment setup. Codex assisted source analysis, implementation, isolated tests and review preparation. Commands above were agent-executed; personal human verification or independent maintainer release acceptance is not inferred.

Release Lane

  • Mainline change targeting main

Changed Surface

  • Installer / bootstrap / lifecycle
  • Model routing / Hermes / capabilities

Risk And Validation

Risk: High (Windows CLI model routing/authentication). Loopback fixtures exercised the public CLI without touching an installed service, customer configuration or real model. Full installed model/GPU/Portal, fleet, and independent human/maintainer release approval remain deferred. The command's existing error-reporting/exit convention is unchanged. Rollback is reverting this source scope; no stored data migration is introduced.

Operational Change Check

  • Operational validation is intentionally deferred for installed live model, GPU, Portal, hardware/fleet and independent human release lanes. Local HTTP-wire and isolated lifecycle results are recorded above.

@tang-vu tang-vu changed the title fix(windows): send CLI chat through the selected ODS model route fix(windows): honor active model routing and credentials in CLI chat Oct 16, 2026
@tang-vu tang-vu closed this Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows chat bypasses saved route, authentication and selected model

1 participant