eval: allow N background cells per session and queue same-language cells instead of rejecting them (oh-my-pi job-manager parity)

[code-yeongyu]

Summary

The eval tool admits at most ONE detached cell per language. A second cell for a busy language is rejected with detachedKernelBusyError ("The js eval kernel is busy running detached cell ..."), so the model has to wait or switch languages. oh-my-pi runs eval cells as jobs (default cap 15) and lets cells for one language queue on that language's kernel.

Expected (ideal state)

• A cell submitted while its language kernel is busy is ADMITTED as queued on that kernel's existing run queue and runs after the active cell; state is still shared per language and execution inside a kernel stays serial.
• Background capacity is a single session-wide setting maxDetachedCells (default 15). At the cap a cell keeps running in the foreground until its foreground window elapses and is then cancelled with a typed eval_background_capacity_reached error naming the live cells; a short cell is never rejected.
• stop on a queued cell dequeues it without interrupting the kernel; interrupt is cell-targeted so cancelling one cell never stops another cell's run; kernel messages (text/display/tool-call/pause) are routed to the cell that produced them.
• reset: true while the language has other live cells is refused with eval_kernel_busy_reset_refused listing them.
• New eval({ action: "list" }) reports live and recently settled cells (id, language, state, elapsed, queued-behind).
• Queued cells do not consume their run budget while waiting.

Actual

• packages/senpi-codemode/src/tool/detached-cell-manager.ts:36 keeps #detachedByLanguage: Map<EvalLanguage, ManagedCell>; detach() (L99-108) refuses when the language slot is taken; eval-tool.ts:76-82 throws the busy error before running.
• kernels/js/context-manager.ts:94-107: interrupt(reason) acts on the active run; with no active run it takes the first QUEUED run and restarts the worker.
• kernels/py/kernel.ts:59-63: interrupt settles EVERY queued run. rb/jl (kernels/shared/subprocess-kernel.ts:52-71) retire the interpreter.
• One onMessage per language is rebound on each getKernel call, so a second cell's callback would receive the first cell's output.
  (line numbers at main cd317ee647)

Scope / acceptance

• Cell-targeted queue control on every kernel (cancelQueued, interrupt(reason, cellId), per-run onStarted/onMessage, queueSnapshot).
• Capped detached set + queued state in the cell manager; tool-boundary admission; list action; reset refusal; maxDetachedCells setting + env override; prompt/footer/docs text updated; existing cap tests rewritten to the new contract (none deleted).
• Deterministic real-kernel QA script proving queued admission, list, stop, reset refusal and exactly-once notifications.

Related

• senpi feat(codemode): steering-safe foreground eval detach (S8) #1637 (steering-safe foreground eval detach, S8) - this issue is the remaining half of oh-my-pi eval parity.

[code-yeongyu]

Progress — step 1 (cell-targeted queue control) is implemented on feat/eval-multi-job-20260921, test-first.

Commit: 6138f07bf0 — feat(codemode): cell-targeted queue control for eval kernels

What it changes

• EvalKernelRunInput gains onStarted() and a run-scoped onMessage(). The JS, Python and shared subprocess queues fire onStarted exactly once when a run becomes the active run, and deliver that run's text/display/log/phase/status/tool-call/result messages to its own callback (the kernel-level getKernel sink keeps startup/crash traffic).
• EvalKernel gains cancelQueued(cellId, reason) and queueSnapshot(), and interrupt(reason, cellId) is targetable: the active cell keeps today's stop path, a queued cell is settled in place with the worker/interpreter untouched, and an unknown id returns { stateRetained: true, note: "cell not found" }.
• A kernel-wide interrupt() with no active run now settles queued runs instead of restarting the runtime. The JS worker is still retired when it is wedged in startup — that path is the only recovery for a worker that never became ready, so dropping it would have traded one defect for another.
• CellExecution aborts with its own cell id, so cancelling a queued cell can no longer stop the unrelated cell that happens to be active.

Evidence (held locally at ~/sisyphuslabs/.omo/evidence/omp-remaining-adoption-20260921/task-1/: red.txt, green.txt, mutation.txt, report.md)

• RED: production sources reverted to the base commit, new file run → 22/22 failed for the intended reasons (queue.remove is not a function, kernel.cancelQueued is not a function, no per-run routing).
• GREEN: node ../../node_modules/vitest/vitest.mjs run test/kernel-queue-cell-control.test.ts → 22 passed; whole package … run test/ → 958 passed, 14 skipped, 0 failed; bun run check → exit 0.
• Mutation: remove() made to report failure without splicing → 13 of 22 cases fail (both queues' "settles B when removed", both "refuses to remove an active or unknown cell", and every targeted-interrupt case); reverted → 22 passed again.

Per-run routing is pinned end to end: cell A's text, its result, and the timeout-pause / timeout-resume status ops raised by its host bridge call all arrive on A's own callback, queued cell B receives none of them, and the kernel-level getKernel sink receives no text/status/result at all (only lifecycle traffic such as ready). Mutation for that pair: per-run routing made to skip status in the js and py kernels → exactly the js and py routing cases fail on the timeout-pause assertion, then revert to green.

New coverage lives in packages/senpi-codemode/test/kernel-queue-cell-control.test.ts with the shared test/eval/queue-control-fixture.ts: queue-level unit cases for both queue implementations, targeted-interrupt cases against all four kernels (js/py real runtimes, rb/jl against a stand-in interpreter), a CellExecution abort case proving the queued cell never executes (sentinel file stays absent, with a control that writes it), and per-run message routing for js and py.

No PR yet — this lands with the rest of the plan's first wave.

[code-yeongyu]

Row 2 is implemented in commit dddde75d6cdd0e41d57f0a8d410cc89a45e0e3b5: capped detached-cell set, queued kernel admission, paused queue-time budgets, targeted queued stop, per-run output routing, and queued rendering/listings.

Verified: 67 acceptance tests, 132 related tests, locked static checks, and real AgentSession JS/Python QA. The <= cap mutation fails the intended third-cell assertion; restoring < returns GREEN. Worktree is clean; no push or PR was made.

Evidence: ~/sisyphuslabs/.omo/evidence/omp-remaining-adoption-20260921/task-2/ (red.txt, green.txt, mutation.txt, report.md).

[code-yeongyu]

Row 3 is implemented in commit 6826b1bd4e9d798d015abdc3fcb59ae9f42dc564: queued eval progress/results, submission-bound at-cap cancellation, configured prompt capacity, and queued footer status. Verification: 978 passing tests (14 existing skips unchanged), root check, real JavaScript-kernel QA, RED/GREEN, and mutation checks. Evidence: .omo/evidence/omp-remaining-adoption-20260921/task-3/ (red.txt, green.txt, mutation.txt, report.md, plus prompt-diff.txt). No push or PR.

[code-yeongyu]

Row 4 implemented in local commit 4ad98cfd453909417782a1c2b37537294efb5e8f: observational eval({ action: "list" }) and typed busy-kernel reset refusal, excluding the requesting cell.

Acceptance: 31 tests pass; bun run check passes. Real-session/JS-worker QA passes. Bypassing the live-cell guard fails case (3); all 42 assertion mutations fail, and GREEN is restored.

Evidence: .omo/evidence/omp-remaining-adoption-20260921/task-4/ (red.txt, green.txt, mutation.txt, report.md). No push or PR.

[code-yeongyu]

Row 5 is implemented in commit b2b843b4e97d9c1df8f77120f645a4cab6eb1370 (feat(codemode): maxDetachedCells setting, docs and multi-job QA).

• SENPI_CODEMODE_MAX_DETACHED_CELLS now controls factory and session registration through one resolver, with default 15. Guides and the internal tracker are updated.
• RED reproduced ignored environment capacity and the numeric-schema mismatch. Hard-coded-15 and schema/default mutations failed as intended; restored focused tests passed 39/39.
• node ../../node_modules/vitest/vitest.mjs run test/: 997 passed, 14 existing skipped, exit 0. Serialized bun run check: exit 0.
• bun scripts/qa/eval-multi-job.ts: 12 passing checks through a real session and JS/Python kernels, covering queued admission, list, queued cancellation, reset refusal, exactly two successful completion notices, and the never-executed B sentinel. Cleanup passed. The read-only baseline failed at B with the expected busy error.
• Live QA also exposed loss of the reset error code when session execution serializes exceptions as text. The refusal message now preserves that identifier without changing the typed exception contract.

Evidence held locally: .omo/evidence/omp-remaining-adoption-20260921/task-5/ (red.txt, green.txt, mutation.txt, qa.json, baseline-failure.txt, report.md). The lane is clean and contains one atomic row-5 commit; earlier rows are retained. No push or PR was performed. CHANGELOG.md remains reserved for the gate row, as instructed.

[code-yeongyu]

Merged via #1938 (merge commit 9fcc333d7314478b8936076df6f21800ac8c51a2; the repository disallows squash merges, so it landed as a merge commit like #1922/#1935).

What landed

eval now runs up to maxDetachedCells background cells per session (default 15, SENPI_CODEMODE_MAX_DETACHED_CELLS override) and queues same-language cells on their kernel instead of rejecting them:

• Cell-targeted queue control across all four kernels — interrupt(reason, cellId), cancelQueued(cellId), queueSnapshot(); cancelling a queued cell can no longer stop the active one.
• Queued admission with paused run budgets; kernel activation starts execution time; detachment is orthogonal to execution state.
• Foreground capacity window: at the cap a cell keeps running foreground and settles as a typed eval_background_capacity_reached cancellation at the end of its re-armed submission window — never an ungoverned wait.
• list observes live/recent cells with typed cross-language metadata; stop targets a specific cell; resetting a busy language refuses with eval_kernel_busy_reset_refused instead of stopping live work.
• maxDetachedCells is resolved at both the advertised prompt guidance and actual session registration (settings file + env override).

Evidence

• CI: all checks green (0 fail, 0 pending) — main workflow run https://github.com/code-yeongyu/senpi/actions/runs/35608269447
• Local gates (behind the shared gate lock): bun run check exit 0; senpi-codemode full suite 137 files / 1024 tests green; coding-agent scoped suite 109 files / 1305 tests green (env-scrubbed; contamination control documented — the branch touches zero coding-agent files).
• Code review: verdict COMMENT (approve with comments), 0 blockers, recommendation APPROVE; gate review: MERGE_APPROVED, no blockers.
• Per-unit RED/GREEN + mutation evidence held locally under ~/sisyphuslabs/.omo/evidence/omp-remaining-adoption-20260921/task-1..6/.
• Real-session QA: scripts/qa/eval-multi-job.ts (live kernels, file-release barriers, no timing sleeps), with the pre-adoption busy-error baseline captured as the failure control.