Skip to content

fix(windows): honor reasoning mode for registered model runtimes - #8169

Open
tang-vu wants to merge 2 commits into
Osmantic:mainfrom
tang-vu:fix/windows-registered-reasoning-20261025
Open

tang-vu wants to merge 2 commits into
Osmantic:mainfrom
tang-vu:fix/windows-registered-reasoning-20261025

Conversation

@tang-vu

@tang-vu tang-vu commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #8168. Native Windows models launched from a registered model store now honor LLAMA_REASONING using the selected runtime's supported flags, just as the default pinned runtime does.

The registered branch used --reasoning-format none unconditionally. On b9014 this leaves thinking enabled, so a short question can spend its entire answer budget on a visible thinking trace. Pass the mode through the existing Get-ODSNativeReasoningArgs helper against the registered executable, keeping its older-runtime fallbacks and the profile's qualified tuning arguments.

Why this matters

ods.ps1 native-llm-start is the native sign-in task's entrypoint. native-llm-restart uses the same launch preparation, including host-agent model activation and rollback. An owner who sets thinking off should receive the answer through these paths, including when using an additional model store.

Overlap check

Searched open/closed PRs for registered runtimes, reasoning, ChatTemplateFile and the changed launcher's callers. Merged #8151 normalizes mode case but leaves this registered branch format-only; merged #6685 handles macOS defaults. Open #6728 addresses selector/context parity. #8014 adds CUDA pin/routing support, while RFC #7968's desktop stack remains separate pending work. This patch changes one shared Windows launcher and three existing tests; it adds no desktop implementation or parallel runtime.

AI Assistance

tang-vu leads this Windows reliability contribution and its technical direction, scope and quality requirements, collaborating with Codex on investigation, implementation, regression tests and review. The validation commands below were executed by Codex. An independent Codex source review found no blocking findings; maintainer release approval remains separate.

Release Lane

  • Mainline change targeting main

Changed Surface

  • Installer / bootstrap / lifecycle
  • Model routing / Hermes / capabilities

Risk And Validation

Risk: High, because native model launch arguments change. Scoped real-hardware validation exercises the affected Windows entrypoints and inference; it is not a full installer/fleet run.

Windows 10 Pro x64 build 19045, non-elevated owner token, PowerShell 5.1 and portable PowerShell 7. Main base de84ff85dad7a95a12fcf2c737b7fc9496e8efcc. An isolated installation layout contains the exact tracked-main export, the real registered-store resolver, an official SHA-verified b9014 Vulkan runtime, and catalog SHA-verified Qwen3.5-2B Q4_K_M. CPU route, 4096 context, one slot, eight threads. No Docker, WSL or task registration.

Same real model request: 2+2, number only; temperature 0, seed 42, max 64 tokens Main Candidate
Native entrypoint native-llm-start native-llm-restart
Reasoning argv --reasoning-format none --reasoning off
Reply Thinking trace; answer not reached 4
Completion tokens / finish 64 / length 2 / stop

Only the changed launcher file was replaced between these real runs; model, runtime and configuration were unchanged. Artifact qualification and the launcher's model/context readiness proof both executed normally. No API request disabled thinking on behalf of the launcher.

Validation:

  • New legacy-launch and real-resolver assertions both fail on unchanged main for the expected missing reasoning behavior.
  • On each PowerShell edition: legacy suite 90 checks, Windows model-store suite 26 checks, native runtime suite 124 checks, checkpoint/reasoning compatibility suite pass. The existing compatibility suite covers b9014, b8248, format-only binaries and unsuccessful help probes.
  • Both changed PowerShell test files already execute in CI on Windows PowerShell and PowerShell 7. The existing shell contract runs through ci-suite.txt; no workflow change. Inventory: 1380 tests, 1361 executed / 19 documented exclusions.
  • Production PowerShell ScriptAnalyzer: no errors/warnings. All three PowerShell files parse. Scoped pre-commit, staged and commit-range Gitleaks, commit-identity and git diff --check gates pass. The real inference validation used 3619e42f0; final head f593974c9 retains exactly the same production bytes.
  • First hosted CI exposed one missed static expectation in test-llama-reasoning-format.sh, which still required the registered branch's format-only flag. The follow-up updates that contract to require the same selected-runtime qualification now protected behaviorally. The corrected shell contract passes locally; production bytes and the real inference receipts are unchanged.

Operational Change Check

  • This is an operational change and validation is recorded above.

The scoped equivalent proves the actual Windows start/restart path, registered artifact validation, selected-binary flags and real inference. Full clean installation, Windows 11, actual sign-in/reboot, AMD GPU execution, other live llama.cpp versions and the release fleet remain deferred to the appropriate maintainer lane. Existing profile tuning and the pinned-runtime branch retain their previous contracts. Rollback is a revert followed by a normal native restart.

Hosted CI on final head f593974c9e50f81747fb8b21ed0aaf9d791226b3: 73/73 checks successful. The first-head failure and its static-contract correction are preserved above.

Independent fleet verification - 2026-10-25

The independent Windows run now reproduces the bug and demonstrates the off-mode fix through the real native-llm-start and native-llm-restart CLI paths. It uses the verified Qwen3.5-2B Q4_K_M model and registered runtime, two CPU threads, no GPU layers, and identical prompt, seed, temperature and 64-token budget.

  • Current-main control: startup thinking is enabled despite off mode; generation enters reasoning and reaches the 64-token limit.
  • Candidate off, both start and restart: startup thinking is disabled; the answer is exactly 4, two completion tokens, normal stop.
  • Candidate on restart: startup thinking is enabled and structured reasoning correctly discusses the addition, but the 64-token budget ends before a final answer. The strict original four-case matrix therefore remains failed, not relabeled as a pass.

Cleanup verified no owned native process or qualification listener remained, fixture keys were removed, and immutable files were reverified. A separate 256-token positive on-mode control is prepared. Its first attempt was refused before runtime startup because the fixture key had already been redacted by cleanup; the repaired successor was refused by the fresh 5-GiB free-RAM gate, also before runtime startup. No admission limit was lowered. This supplemental control and final integration acceptance remain pending; the PR is unmerged.

Original matrix receipt SHA-256: e34b54e280a32a5d23255d013cfd16597a058d26bd803ef0f6c094441849acb1. These are independent fleet results; the contributor validation above is preserved.

@Lightheartdevs

Copy link
Copy Markdown
Collaborator

Thanks for making registered model runtimes honor the owner's reasoning mode through the shared compatibility helper.

Advisory status: needs human decision

The launcher change fits the existing design: it probes the selected registered executable through Get-ODSNativeReasoningArgs, preserves profile tuning, and avoids duplicating the pinned-runtime compatibility logic. I found no blocking code defect in the scoped review.

The description still records a failed strict four-case hardware matrix and a pending supplemental on-mode control. The successful off-mode start/restart results support the intended fix; exhausting a 64-token on-mode budget does not by itself demonstrate a regression. Before calling integration accepted, please complete and record the proposed 256-token positive control under the existing admission limits, or have a human maintainer explicitly resolve that acceptance item. Keep the original failed matrix visible as documented.

What I checked:

  • Exact-head diff, registered-launch and reasoning-helper context, repository validation expectations, discussion, CI wiring, and git diff --check; all 73 hosted checks pass.
  • Locally on Windows PowerShell 5.1: ods/tests/contracts/test-windows-native-llama-legacy.ps1 passes 90 checks, and ods/tests/test-windows-native-checkpoint-args.ps1 passes. I reviewed the reported hardware evidence in the description; I did not rerun real model inference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Native Windows: registered model-store runtimes ignore LLAMA_REASONING=off

2 participants