Repository navigation
feat(windows): local chat in the dashboard for the Windows Desktop profile - #8009
Open
IronicRayquaza wants to merge 8 commits into
Open
IronicRayquaza wants to merge 8 commits into
IronicRayquaza wants to merge 8 commits into
Conversation
Add an opt-in ODS_RUNTIME_PROFILE=windows-desktop so dashboard-api can run as a native Windows process with no Docker, no WSL and no host agent. - Refuse to start without an absolute ODS_DATA_DIR and DASHBOARD_API_KEY; the generated-key fallback no longer writes to /data (C:\data on Windows). - Never dial a host agent: AGENT_URL is empty and the shared client raises AgentUnavailable before any socket, so a Docker or WSL ODS on the same PC cannot answer. - Start no child processes: GPU probes, the curl cluster refresh, Token Spy polling and the Docker preflight are skipped. A windowless process would otherwise flash a console every five seconds. - Load only loopback services, since Compose service names resolve to nothing on a native PC and each failed lookup stalls /api/status. - Report Windows CPU and RAM through kernel32 instead of None. - Add a path-filtered windows-latest workflow that installs the hashed lock wheels-only and runs the profile tests. Refs Osmantic#7968 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mypy runs on Linux, where ctypes has no windll or WinError, so assert the Windows platform inside the two kernel32 readers. Count lock hashes in a typed dict instead of a dict mixing names and counters. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mypy 2.4 does not treat an assert on sys.platform as unreachable, so define the two readers under a module-level sys.platform == "win32" branch, with stubs that raise OSError elsewhere. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
8 tasks
Add the chat route for the Windows Desktop profile (Osmantic#7968): dashboard-api talks to the loopback llama-server that the native runtime started, with its key file. No LiteLLM, no host agent, no Portal stack. - native_llm.py reads the launcher's ODS_NATIVE_LLM_URL (loopback http only) and ODS_NATIVE_LLM_RUNTIME_DIR: the api-key file plus the runtime's own intent.json and ready.json. The key is cached by mtime and never returned or logged. - context_budget.py fits history into the model's real context before every request: measured with llama-server's /apply-template and /tokenize (the prompt it will actually evaluate), answer tokens reserved, whole turns dropped from the oldest end by binary search, a kept slice always starting with a user turn. A message that cannot fit on its own is refused before inference with its token numbers. Saved history is never shortened. - /api/local-chat/stream passes llama-server's SSE through with bounded lines and a first "ods" event reporting how many messages were left out of the model's view. /api/local-chat/status reports ready, loading, stopped or failed from /health, intent.json and ready.json. Every failure has a fixed code and sentence; exception text never reaches the browser. - The setup wizard's /api/chat uses the same native llama-server in this profile instead of a Docker hostname. - A Desktop launcher without these settings fails at startup, naming the variable. Refs Osmantic#7968 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
llama-server reuses its cached prompt only while the start of the prompt is unchanged. Re-cutting history to the full budget on every message moved the first kept turn almost every time, so a long chat on a laptop CPU re-read about 3,000 tokens (28 s with a 1B model) before each answer. The client now sends back the contextStart the previous answer reported. While the conversation still fits from that turn it is reused unchanged; when it no longer fits, history is cut to 75% of the budget so the next several messages keep the new start. Measured on an i5-10500H with Llama 3.2 1B: one 24.7 s re-read, then 0.8 s first tokens for every following message (previously about 28 s whenever the window moved). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ofile Give the Windows Desktop profile (Osmantic#7968) a chat in the dashboard itself. In that profile /api/pixel/status reports "Portal is not enabled", and Pixel's existing branch for that state now renders LocalChat instead of a link to Open WebUI, which Desktop does not have. LocalChat streams from /api/local-chat/stream and reuses Portal's PortalStreamingText (Markdown, highlighting, reveal) and the owner's Enter or Ctrl+Enter send setting. It sends back the contextStart each answer reports, so the model's prompt prefix stays cached between messages, and says when earlier messages are outside the model's view. A refused message (too long, model loading or stopped) goes back into the box with the server's sentence. Stop keeps the partial answer. Status is polled one request at a time with a timeout, and polling stops once the model is ready. History lives in this browser, bounded to the newest 50 chats within 4 MB; the open chat is never dropped, and a storage failure (private window, full quota) is shown but never blocks chatting. Chats can be saved as Markdown or JSON, switched, started and deleted. The sidebar hides Extensions, Models, GPU Monitor and Portal's own conversation list in this profile: those pages cannot work there, and a second, empty chat list beside Local chat would only confuse. The dashboard session status now reports remoteAccess: a loopback-only Desktop install has no other device to sign in from, so the sign-in gate no longer asks its owner to choose a password before the first chat. Refs Osmantic#7968 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IronicRayquaza
force-pushed
the
feat/windows-desktop-chat-ui
branch
from
October 23, 2026 07:34
04eee69 to
c9cc093
Compare
This was referenced Oct 23, 2026
The app and the model's task both start at sign-in, and the model's launcher checks its files before llama-server listens. In that window the chat status said "not answering, restart ODS" while nothing was wrong. A standard-account test caught it 17 s after sign-in. For the app's first 3 minutes, a silent model with no recorded stop or failure is now "model_starting". The UI already shows a server message for a state it does not know. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… into feat/windows-desktop-chat-ui
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Sixth step of #7968: a chat in the dashboard itself for the Windows Desktop profile, the screen an owner actually uses.
In this profile
/api/pixel/statusreports "Portal is not enabled". Pixel's existing branch for that state now rendersLocalChatinstead of a link to Open WebUI, which Desktop doesn't have.components/LocalChat.jsx:/api/local-chat/streamthrough Portal's ownPortalStreamingText(Markdown, code highlighting, reveal), so local chat looks like the rest of ODS. The owner's Enter or Ctrl+Enter send setting is respected.contextStartand the next message sends it back, so the model's prompt prefix stays cached (long-chat follow-ups take ~0.8 s instead of ~28 s on a laptop CPU; measured in feat(windows): stream local chat from the Windows Desktop llama-server #7975). When earlier messages fall outside the model's view, the chat says so and keeps them saved./api/local-chat/statusis polled one request at a time with a 5 s timeout and stops once the model is ready, so requests can't pile up.lib/localChatStore.js: history lives in this browser, bounded to the newest 50 chats within 4 MB; the open chat is never dropped. A storage failure (private window, full quota) shows a notice but never blocks chatting. Chats can be switched, started, deleted, and saved as Markdown or JSON; the file is named after the chat in any script (Vietnamese, Hindi, Japanese), and a character split across network chunks still arrives whole (tested).What a non-technical owner no longer runs into:
/api/auth/dashboard-sessionnow reportsremoteAccess, which isfalsein the Desktop profile (loopback only, so there's no other device to sign in from). The sign-in gate no longer asks that owner to "Choose your password" before the first chat. Compose is unchanged (remoteAccess: true).AI Assistance
Claude Code drafted the code, tests and this description, and ran the validation below on my Windows 11 laptop. I chose the scope and design (see #7968), and I'm accountable for the diff and for responding to review.
Release Lane
release/2.6.xmainChanged Surface
Risk And Validation
runtimeProfile === 'windows-desktop'; the API change only addsremoteAccess: trueto an existing response there.git diff --checkLive, end to end. On Windows 11 Home 26200 (i5-10500H, 16 GB RAM, CPU only), with no Docker and no WSL, I ran:
pythonw.exe;The real UI was then driven in headless Edge over the DevTools protocol (typing into the box and pressing Send):
The first render found the two issues fixed here: the "Choose your password" prompt, and the duplicate Portal chat list in the sidebar.
Operational Change Check
Notes For Reviewers
pythonw.exe(for PR6, from live runs):.cmdlauncher must use 8.3 short paths for non-ASCII folders (cmd.exemanglesJoséeven afterchcp 65001);Start-Process -Redirect*lets the child inherit the caller's pipes.🤖 Generated with Claude Code