Skip to content

Place the vLLM Qwen project inside WSL2 automatically on Windows - #5834

Merged
atomantic merged 2 commits into
mainfrom
cos/task-mtjoxv5z/agent-1471a16e
Sep 2, 2026
Merged

Place the vLLM Qwen project inside WSL2 automatically on Windows#5834
atomantic merged 2 commits into
mainfrom
cos/task-mtjoxv5z/agent-1471a16e

Conversation

@atomantic

Copy link
Copy Markdown
Owner

Summary

Setting up vLLM (Qwen3.8-27B) on a Windows host used to dead-end in the readiness modal:

vLLM (Qwen3.8-27B) is already installed — starting it.
vLLM (Qwen3.8-27B) provisioning failed: on Windows this project and its ~20 GB of
weights belong on the WSL2 filesystem, not the Windows one — cloning to
C:\Users\<user>\qwen-serving would leave every weight read crossing a 9p share. Set
VLLM_QWEN_PROJECT_DIR to the distro's UNC path
(\\wsl.localhost\<distro>\home\<user>\qwen-serving) and click this again.
Installer hit an error - see logs above.

Two things are wrong there. The angle brackets are literal — the operator has to
go find those two values themselves — and every one of them is something the
machine can answer. And the line above it claims a start that never happens,
which makes the refusal read as a failed start.

PortOS now asks WSL. Before cloning, it runs the default distro's own shell
(wsl.exe -e sh -c 'echo "$WSL_DISTRO_NAME"; echo "$HOME"'), verifies the derived
\\wsl.localhost\… path is readable from Windows, and records it as
VLLM_QWEN_PROJECT_DIR in PortOS's own .env — so the once-a-minute readiness
poll, the Start button, and the next server boot all resolve the same directory
the provisioning run actually used. The Start button settles placement too, so a
project prepared by hand inside the distro (the documented path, and the only
one SGLang has) is found with no configuration at all.

The probe deliberately runs the distro's shell rather than wsl --list: wsl.exe's
own UI output is UTF-16LE, which a UTF-8 reader mangles, while the stdout of a
program it executes passes through byte for byte. The UTF-16 read is confined to
the failure path, where the distro list only decorates an error message.

It refuses only where it genuinely cannot answer, and each refusal names that
host's fix instead of a placeholder:

reason what it says
no-wsl wsl.exe did not run here — install a distro with wsl --install -d Ubuntu
no-distro WSL is present but nothing answered — names the distros it can see
internal-distro the default is docker-desktop, which is wiped on a reset — names a real one
unreadable-share the distro answered but Windows cannot read the share — wsl --shutdown

Setting VLLM_QWEN_PROJECT_DIR yourself still overrules the whole decision.

Changes

  • server/lib/wslDistro.js (new) — detectWslProjectDir(leaf); distro/home
    detection, UNC derivation, and the share-readability check. Parameterized by
    leaf name, so it is not vLLM-specific.
  • server/lib/vllmQwenProject.js — records and resolves the project
    directory. Anchored to PATHS.installRoot, not root, so a server booted from
    a CoS worktree writes where the real install reads (Boot migrations resolve data root from executing-file location — crash when run from a CoS agent worktree #1947). Resolution order:
    VLLM_QWEN_PROJECT_DIR (this run's decision) → the record → ~/qwen-serving,
    owned by one exported vllmProjectDirIsSettled() so the manager never
    re-lists it.
  • server/lib/vllmQwenProvision.jsupsertEnvLine, the replacing
    complement to the additive-by-contract mergeEnvFileContents; both now share
    one trailing-newline guard. Its replacement is a function, not a string, so a
    value carrying $& is written literally.
  • server/services/vllmQwenManager.js — settles placement before both
    provision and start; refusal prose is a reason → lead sentence table with one
    shared tail, so a fifth reason cannot ship without the "PortOS will not fall
    back to C:\" sentence.
  • server/services/localRuntimeSetup.js — each provisioning step now says
    what it is about to do, instead of announcing a start before a ~30 GB download.
  • docs/features/qwen38-rtx3090.md — §1c is now "Windows, optional".

Test plan

  • server/lib/wslDistro.test.js (new) — probe parsing, UTF-16 list decoding, UNC
    derivation, and each of the four refusal reasons, all through injected
    run/exists so no subprocess is spawned.
  • server/lib/vllmQwenProject.test.js — the record round-trips, an exported env
    var outranks it, a second recordVllmProjectDir rewrites rather than appends,
    and vllmProjectDirIsSettled agrees with the resolver. envPath is threaded
    through so a sandbox answers rather than the developer's own install.
  • server/lib/vllmQwenProvision.test.jsupsertEnvLine replaces, appends
    without splicing, and writes a $-pattern value literally.
  • server/services/vllmQwenManager.test.js — Windows detects and records; a
    settled directory spends no subprocess; a failed record still leaves the run on
    the detected directory; each refusal reaches the caller; Start settles placement
    before looking for the project.
  • Full server suite: 37 318 passing. The 20 failures are pre-existing and
    verified identical on origin/main — this machine has no Python, which those
    suites shell out to.

Follow-ups filed

Setting up vLLM (Qwen3.8-27B) on a Windows host used to dead-end: PortOS
refused to clone, printed a UNC template with `<distro>` and `<user>` left
as literal angle brackets, and told the operator to set
VLLM_QWEN_PROJECT_DIR and click the button again. Every value in that
template is something the machine can answer.

PortOS now asks WSL for them. Before cloning, it runs the default distro's
own shell (`wsl.exe -e sh -c 'echo "$WSL_DISTRO_NAME"; echo "$HOME"'` --
the distro's stdout is UTF-8, while `wsl --list` prints UTF-16LE that a
UTF-8 reader mangles), checks the derived \wsl.localhost path is readable
from Windows, and records it as VLLM_QWEN_PROJECT_DIR in PortOS's own .env
so the readiness poll, the Start button, and the next server boot all
resolve the same directory. Start does the same, so a project prepared by
hand inside the distro is found with no configuration at all.

It refuses only where it genuinely cannot answer: no WSL on the host, no
distro but a container engine's own (docker-desktop is wiped on a reset),
or a \wsl.localhost share Windows cannot read -- and each refusal names
that host's fix instead of a placeholder. Setting the env var still
overrules the whole decision.

Also fixes a misleading progress line: a provisioning click reported
"already installed - starting it" immediately before a clone and a ~30 GB
download, which made every line after it read as a failed start.

- server/lib/wslDistro.js: new; distro/home detection and UNC derivation
- server/lib/vllmQwenProject.js: record and resolve the project directory,
  anchored to the install root so a worktree boot writes where the install reads
- server/lib/vllmQwenProvision.js: upsertEnvLine, sharing one newline guard
  with mergeEnvFileContents
- server/services/vllmQwenManager.js: settle placement before provision and start
- server/services/localRuntimeSetup.js: each provisioning step says what it is
  about to do
@atomantic
atomantic merged commit 374b444 into main Sep 2, 2026
7 checks passed
@atomantic
atomantic deleted the cos/task-mtjoxv5z/agent-1471a16e branch September 2, 2026 07:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant