GPU guides' keep-alive and G6 resource rules diverge from the canonical compose files they name

[Xore]

Summary

docs/gpu-llm-analysis-worker.md and docs/gpu-ml-worker-acceleration.md state resource-containment rules that the checked-in compose files they themselves declare canonical ("The checked-in files are canonical; the inline Compose example ... must not be copied") no longer agree with. Two distinct divergences:

1. Keep-alive: three different values presented as one knob, with the wrong one asserted as the effective unload behavior for the slot the sharing budget was written around.

• The guides' hard rule is OLLAMA_KEEP_ALIVE=10m -- unload the model when idle so the ML worker can use the GPU for retraining (LLM guide §5 :199, repeated as the worker-retention claim in its T7 :489), and acceleration guide §5's consumer table makes it a budget parameter: "On LLM requests; unloads 10 min after last use (OLLAMA_KEEP_ALIVE=10m)".
• The deployed canonical compose deliberately sets something else: analysis/ghidra/docker-compose.ghidra.yml :87 has OLLAMA_KEEP_ALIVE: '30m' under an explicit rationale comment -- "Keep the model resident between samples. A drain analyses a queue one binary at a time, and reloading several GB of weights per workflow call costs more than the inference does." That is the opposite trade-off from the 10-minute-unload-to-free-VRAM rule the docs teach.
• The reconciliation lives in code nobody's doc mentions: llm-worker/worker.py keeps its own per-request keep_alive (LLM_KEEP_ALIVE, default 10m, validated at :113/:253) and sends it inside every request body (:367, :405, :1334). Ollama honors request-level keep-alive over the server default -- so the docs' "10 minutes" is true only where llm-worker made the request (sessions/payload/report jobs), while ghidra drain and revdeck traffic on the same shared card runs under the deliberate 30m residency.

Practical consequence is mostly latent today (the ml-worker still pins torch==2.13.0+cpu, so there is no GPU retraining to collide with), but #569's lifted-overlap-rule arithmetic and acceleration-guide §5's VRAM table were reasoned around a 10-minute unload window that does not govern the heaviest consumer described in that very table (the ghidra slot at ~14.1 GiB / 32k context).

2. G6 cites a "12 GiB RAM limit" that exists nowhere.

LLM guide guardrail G6 (:545-548): "Respect the VRAM budget in §5 (MAX_LOADED_MODELS=1, NUM_PARALLEL=1, KEEP_ALIVE=10m, 12 GiB RAM limit)." A repo-wide sweep finds no mem_limit/memory: value anywhere near 12g attached to ollama or either worker -- the only nearby memory constraint in the ghidra stack is an unrelated helper service's mem_limit: 1g. MAX_LOADED_MODELS/NUM_PARALLEL do check out against the compose (:93/:102); the fourth item in a guardrail labeled "read before implementing" is a phantom from an earlier draft of the deployment.

Evidence or Why it matters

These are agent-targeted implementation guides ("Audience: a human operator or an AI coding agent implementing this feature"). An implementer auditing their own work against G6/T7 enforces a memory limit that isn't declared anywhere and rewrites a residency setting whose removal contradicts a documented rationale (ghidra reload cost). Conversely, someone reasoning about whether ml-worker retraining can start mid-drain gets the docs' 10-minute answer when the real server-side bound is 30 minutes.

What needs to change

1. Split the keep-alive documentation into its two actual layers: server default (OLLAMA_KEEP_ALIVE, currently 30m by authored decision for drain residency, quoting the compose rationale) vs worker request override (LLM_KEEP_ALIVE, 10m, sent per-request and therefore governing llm-worker traffic only). Fix all three claims: LLM guide :199, :489; acceleration guide :197 table row.
2. If Re-evaluate ml-worker/Ollama GPU-overlap-avoidance rule now that VRAM is ~20GB, not 8GB #569/AFTER RELEASE 0.1.0 #1523-era sharing-budget math should assume the heavier 30m residency for the ghidra slot during drains, say so explicitly next to the worst-case paragraph (or record why not).
3. Either restore a real memory limit named by G6 or delete the phrase and give G6 the constraints that exist.
4. Optionally note in both guides that request-level keep_alive overrides the server env var, since that distinction is what reconciles them.

Acceptance criteria

• No doc states a single effective unload-time figure for the shared ollama instance without naming which request path it applies to.
• Every constraint listed in G6 exists verbatim in a checked-in compose file (or the list shrinks to those that do).
• grep -rn "OLLAMA_KEEP_ALIVE\|LLM_KEEP_ALIVE" docs/ results each match the semantics of the corresponding compose variable.

Related

• GPU docs still publish the retired Turing-card contract #602 refuted; partial corrections left guides self-contradictory #2364 -- this family's card-identity repair; separate root cause (hardware contract vs resource parameters), same two files.
• Re-evaluate ml-worker/Ollama GPU-overlap-avoidance rule now that VRAM is ~20GB, not 8GB #569 / Re-evaluate local model selection for the real 20GB card, promote qwen3:14b #635 -- sourced the sharing-budget numbers that accelerate-guide §5's keep-alive row feeds into.
• Milestone I: shared-GPU slot scheduling, collision drills, and the 72-hour soak #84 -> AFTER RELEASE 0.1.0 #1523 -- where GPU slot scheduling/soak actually now lives.

[Xore]

A fourth value for this issue's keep-alive divergence list, found while
harvesting the #1805-c benchmark round (PR #2641, defect filed as #2644).

analysis/ghidra/benchmarks/corpus/record_baseline.py stamps
"keep_alive": "10m" into the qualification_request block of every
benchmark report — the block that gets quoted verbatim as a round's runtime
pins in docs/local-llm-model-evaluation.md. It then never sends it: the
payload it builds has model, messages, temperature, max_tokens, seed,
stream and reasoning_effort, and no keep_alive anywhere in the file.

So the effective value during every benchmark round is the container's, which
on the live host is:

OLLAMA_KEEP_ALIVE=30m

matching docker-compose.ghidra.yml's deliberate 30m, not the 10m the
benchmark reports claim and not the 10m the guides assert.

That makes the tally: guides say 10m, compose says 30m, the container runs 30m,
and the benchmark artifacts publish 10m as a pin while inheriting 30m. The
benchmark case is the one where it is actively misleading rather than merely
inconsistent, because those reports exist to make rounds reproducible.

Same file has the identical problem with context_tokens: 8192, which matters
more — Ollama serves at the loaded instance's context, so a benchmark cell
inherits whatever the shared container has resident (ollama ps currently
shows qwen3:14b at CONTEXT 8192, loaded by the worker). Detail and fix in
#2644.

Not proposing a value here — that is this issue's call. Flagging that whatever
number wins, record_baseline.py needs to either send it or stop printing it,
or the divergence just reappears inside the benchmark artifacts. Possibly
related to #2409's runtime-pin drift, which is the same approval chain from the
image side.

[Xore]

Picked up for automated fix. Branch: orchestrator/issue-2365 (base main).

[Xore]

PR #2677 merged. CI: 108 passing, 0 failing, mergeState=CLEAN. Closes #2365 confirmed.