[tang-vu]

Summary

Send Bash ods chat and its benchmark caller through the installed inference route. Enabled switchboards use ods/current through the authenticated gateway; cloud, external and host-native configurations use its existing default alias. Preserve explicit model arguments and legacy direct-local model discovery. A missing key or failed selected gateway remains a visible failure.

Closes #7852

Why this matters

The current command always contacts the local llama-server. It can answer from the wrong backend after a route switch, or fail on installations that deliberately run no local model. The fix follows the gateway renderer and existing installed-client contract. Bearer credentials reach curl through a file descriptor.

Overlap check

Searched all-state PRs for CLI chat, chat/switchboard, chat/LLAMA_SERVER_API_KEY and ods-cli/gateway, and inspected current cmd_chat, phase 07, the runtime renderer and setup chat routing. Closed #7214 changes the separate Windows PowerShell client; #3166 concerns macOS/Windows response exit codes. Neither changes the Bash route. No active equivalent Bash implementation was found.

AI Assistance

tang-vu leads this contribution, its scope decisions and quality requirements, collaborating with Codex on analysis, implementation, tests and review preparation. The validation commands below were executed by Codex.

Release Lane

• Mainline change targeting main

Changed Surface

• Installer / bootstrap / lifecycle (public Bash CLI)

• Model routing / Hermes / capabilities (CLI consumer)

Risk And Validation

High by the CLI/model-routing map. Base 5c8aeaf10b4108085bd8a8d70e87bf16ad70d4de; candidate 7da359fd6b1f223d47f2db266f545346cdf8df6f.

• Old code: eight failing route assertions (including the external/native subcases), with the legacy-local control passing.

• Candidate: python3 tests/test-cli-chat-route.py — 10 tests passed on Ubuntu 24.04 WSL2/Python 3.12, driving real Bash/curl/jq against two loopback HTTP servers. Covers gateway authentication and master-key alias, route selection, explicit model, Unicode, benchmark, missing-key refusal, failed gateway/HTTP completion, and untouched direct-local discovery.

• Fixtures pin both installation selectors, block host mutators and assert byte preservation of the installation and foreign sentinel.

• git diff --check and commit identity gate passed. Test executes through ods/tests/ci-suite.txt.

• Independent agent source review checked aliases/authentication against the actual renderer and installed-client producers; no blocking finding.

Operational Change Check

• Operational change with scoped HTTP-wire/actual-CLI validation recorded above.

• Installed model/Portal, native platform fleet and independent human release gates remain deferred.

Notes For Reviewers

No backend configuration, persisted format or other platform CLI changes. Revert restores the prior command behavior. The HTTP fixtures prove destination/authentication/failure behavior; they do not qualify a model or replace release-grade installation validation.

Scoped installed-runtime probe

The exact candidate CLI was also executed against the pre-existing isolated Ubuntu WSL2 QA runtime with ODS_MODE=local, ODS_MODEL_SWITCHBOARD=enabled and LLM_BACKEND=llama-server. ods chat sent a minimal request through the configured gateway and returned ODS_OK with exit zero. The runtime uses the actual CUDA llama.cpp server image server-cuda-b11429 (digest 883915ea20a4b350e98f7284098f4d400c21e9b366e248d397cd42d90602a84c). No lifecycle or saved configuration mutation was performed.

This pairs the candidate CLI with an existing QA backend. It is not a fresh install, whole-main deployment or fleet qualification. The HTTP regression fixtures remain the evidence for the negative/authentication cases.

Maintenance validation (2026-10-23)

Final candidate de21a7823a09f50917620c5959d10a02136fa436, merged normally with upstream main 6513ad2e0e205525f148b68fffd095cb1b347e2c. All upstream CI suite entries and this PR's test are retained. Earlier validation above is historical and does not qualify the updated candidate.

• Exact committed source exported into a private, credential-free Ubuntu 24.04 WSL2 tree. No installed services or saved installation configuration changed.

• git diff --check, commit identity checks and Gitleaks 8.30.1 commit-range scanning passed.

• Root independently reviewed final source and fixture isolation; no blocking source finding. This source review is separate from the executed checks.

• Reproduced reviewer regressions on prior CLI: missing switchboard setting used local routing; local and gateway HTTP 400 context-length bodies were hidden.

• Corrected missing/empty switchboard to the shipped enabled default. Explicit disabled/observe retain legacy/route-specific behavior. Portable curl HTTP-status capture preserves error body and failure status without requiring newer --fail-with-body.

• Actual Bash/curl/jq loopback HTTP suite: 13 tests pass, including default vs explicit mode, both 400 error bodies, auth/key alias, Unicode, explicit model, benchmark, refusal and no fallback.

• Preflight LLM-route and CLI install-directory selector suites pass. Existing test-cli-chat-error-exit.py: macOS HTTP cases pass; two Windows cases skip on Linux.

• CI inventory accounts for the new suite. Local shellcheck unavailable; hosted lint runs on this head.

Fresh installation, current-candidate model/Portal, native hardware fleet and independent human release approval remain deferred before release. Earlier installed-model probe belongs to the previous head.

CI fixture follow-up

The initial updated-head suite ran all 13 HTTP regressions successfully and found one existing-fixture mismatch: its dead-backend case omitted switchboard mode while expecting a direct llama-server diagnostic. The fixture now explicitly selects disabled switchboard for that intentional legacy-route case and pins both installation selectors. Local rerun passes the chat/benchmark assertions. Four start/diagnostic assertions still show the previously recorded isolated-container baseline failures; those four passed in the hosted run. The current head reruns hosted CI. No production behavior was weakened to satisfy this fixture.

Hosted CI final readback on de21a7823a09f50917620c5959d10a02136fa436: all 74 checks pass, including the full CI suite and Linux integration-smoke. Root reviewed the fixture follow-up separately with no blocking finding.

Integration audit repairs (2026-10-23)

The startup-diagnostics fixture now includes the real WSL bind-recovery helper, and the cloud-permission ordering fixture stubs coordinator startup. This preserves their original assertions on WSL and with the coordinator recovery change. The Datasette dependency pin is also updated to setuptools 84.1.0 for GHSA-xmj3-9gf7-h52w, which the fresh security run detected. The production Datasette image builds, pip check passes, and pip-audit reports no known vulnerabilities for its complete requirements. Local CLI and permission suites pass.

[roshangupta00750]

One regression from the --fail added to the completion curl in cmd_chat. A llama-server 4xx body is no longer shown. For example, an over-long prompt now prints LLM request failed: curl: (22) The requested URL returned error: 400 instead of the server's context length exceeded message. --fail-with-body keeps the non-zero exit and the body.

Also, when ODS_MODEL_SWITCHBOARD is missing from .env, the check treats it as legacy. The schema, compose and phases 06/07 all default it to enabled, so a .env without the key goes straight to llama-server instead of the routed path.

[tang-vu]

@roshangupta00750 Both findings are fixed in 919d926. Missing/empty switchboard settings now use the shipped enabled default; explicit disabled/observe retain their routing. HTTP failures preserve response bodies and return failure using portable curl status capture, so a 400 context-length error stays visible. New regressions failed on the previous CLI for the default setting and for local/gateway 400 diagnostics; all 13 actual Bash/curl HTTP tests now pass. Current main is merged with every CI entry retained. The updated body records exact-head validation and remaining release gates. Thank you for identifying both cases.

[roshangupta00750]

Checked on de21a7823: both are fixed. A missing or empty ODS_MODEL_SWITCHBOARD now takes the gateway route with ods/current, and _chat_curl keeps the server's body on a 4xx while still failing (status from --write-out, so it works on older curl too). tests/test-cli-chat-route.py: 13 tests OK here. Nothing further from me.