Labels to apply (the filing account has read-only access to this repo, so GitHub silently dropped them at creation): repo-study, plan, area:devtools, model:heavy, effort:medium, planner:opus-5.
Provenance: studied from carloslfu/slotstream (MIT, © 2026 Carlos Galarza), captured 2026-09-02. Clean-room — technique only, reimplemented against PortOS modules.
Rationale
Upstream's governor treats a local model process's memory as elastic: it watches the OS's memory-pressure signal plus a slow availability poll and resizes between requests — shrink fast, grow slow, with absolute-GB dead-bands (a relative trigger can never fire when the honest adjustment is a couple of GB on a large pool), a calm-down window before growing back, and a hard rule that an explicitly user-set size is never touched because it is the user's stated intent.
PortOS's equivalent is reapIdleDaemons() / registerIdleDaemon() in server/lib/managedDaemon.js, and it knows only wall-clock idleness: a model server holding 20–40 GB resident keeps holding it until its idle window expires, however hard the rest of the machine is thrashing. The user notices this as the whole machine going sluggish while a model they finished using an hour ago sits on the memory — with nothing in the UI explaining why. The reading is already available: server/lib/memoryStats.js parses the authoritative vm_stat / meminfo numbers (deliberately, because os.freemem() lies on macOS), and server/services/cosHealthMonitor.js already reasons about host-wide pressure.
Work
- Add a pressure-aware pass to the idle reaper in
server/lib/managedDaemon.js: when memoryStats reports sustained host pressure above a threshold, release the least recently used managed model daemon early rather than waiting out its idle window. daemonLastUsedAt() already provides the ordering.
- Borrow the governor's shape, not its numbers: shrink fast / restore slow, an absolute-GB dead-band rather than a percentage, and a calm-down window so a transient spike cannot flap a daemon down and up. Release at most one daemon per tick and re-read before the next.
- Never release a daemon the user pinned. Add an explicit per-server "keep loaded" toggle next to the existing idle-release window in the Local Runtime Servers table; a pinned server is exempt exactly as an explicitly-sized pool is exempt upstream.
- Make it legible: record the reason on the daemon entry and show it in the Models → LLMs row ("released at 09:14 — host memory pressure"), so an unexpectedly stopped server is explained rather than mysterious. The existing lazy/on-demand start path already brings it back on next use.
- Keep the decision a pure function of (current state, pressure reading, recent history) so it is testable without putting the machine under real pressure — mirroring how
reapIdleDaemons is already unit-tested.
Fix: server/lib/managedDaemon.js (reapIdleDaemons, registerIdleDaemon, new pressure policy), server/lib/memoryStats.js (consumer only), per-server settings in server/services/settings.js, client/src/components/models/ModelsPanel.jsx.
Scope: medium — one new policy function plus a settings toggle and a UI reason string; the reaper, the memory reading, and the lazy-start path all already exist.
Provenance: studied from
carloslfu/slotstream(MIT, © 2026 Carlos Galarza), captured 2026-09-02. Clean-room — technique only, reimplemented against PortOS modules.Rationale
Upstream's governor treats a local model process's memory as elastic: it watches the OS's memory-pressure signal plus a slow availability poll and resizes between requests — shrink fast, grow slow, with absolute-GB dead-bands (a relative trigger can never fire when the honest adjustment is a couple of GB on a large pool), a calm-down window before growing back, and a hard rule that an explicitly user-set size is never touched because it is the user's stated intent.
PortOS's equivalent is
reapIdleDaemons()/registerIdleDaemon()inserver/lib/managedDaemon.js, and it knows only wall-clock idleness: a model server holding 20–40 GB resident keeps holding it until its idle window expires, however hard the rest of the machine is thrashing. The user notices this as the whole machine going sluggish while a model they finished using an hour ago sits on the memory — with nothing in the UI explaining why. The reading is already available:server/lib/memoryStats.jsparses the authoritativevm_stat/meminfonumbers (deliberately, becauseos.freemem()lies on macOS), andserver/services/cosHealthMonitor.jsalready reasons about host-wide pressure.Work
server/lib/managedDaemon.js: whenmemoryStatsreports sustained host pressure above a threshold, release the least recently used managed model daemon early rather than waiting out its idle window.daemonLastUsedAt()already provides the ordering.reapIdleDaemonsis already unit-tested.Fix:
server/lib/managedDaemon.js(reapIdleDaemons,registerIdleDaemon, new pressure policy),server/lib/memoryStats.js(consumer only), per-server settings inserver/services/settings.js,client/src/components/models/ModelsPanel.jsx.Scope: medium — one new policy function plus a settings toggle and a UI reason string; the reaper, the memory reading, and the lazy-start path all already exist.