kb-spike: multi-message-per-Pi-process — gates the prompt-routing L4 dimension (incl. /kb load-and-cite)

[vilosource]

Context

The kb-command L4 matrix (experiments/kb-command/EXPERIMENT.md) has one row — load-and-cite — that cannot be tested with the current harness:

Operator types /kb <area-id>; a subsequent prompt asks for a fact in that area → entries injected; LLM cites the marker.

/kb is a Pi slash command, not a prompt — it triggers no LLM turn (the handler just pi.sendMessage(...)s the area's entries into the session and returns). To observe "the LLM cited it" you need a second prompt in the same Pi process, because:

• step (in scripts/spike/lib/step.sh) runs vfa run --prompt "<one prompt>" — one prompt per invocation.
• vfa run accepts only a single --prompt/--prompt-file (no message array, no --continue).
• Each step spins up a fresh Pi container with no Pi-level session continuity (it threads KB_SESSION_ID for the kb extension's file-backed SessionState — loaded areas, signals — not the Pi conversation transcript).

So step "load" --prompt "/kb foo" then step "cite" --prompt "..." → step 2's Pi never saw step 1's injected message.

What we have

3 single-step scenarios that DO work and anchor the dispatch path (load-marks-area-loaded, usage-on-empty-args, unknown-area-no-load) — they assert on vfa logs --raw <run_id> (the mykb-loaded-areas custom message in the event stream) + the persisted .sessions/<id>.json loadedAreas. Combined with the L1 test (tests/extension/kb-command.test.ts, which checks pi.sendMessage content) that's solid coverage of dispatch + injection-call + loaded-state. The gap is purely the LLM-sees-the-injected-message end-to-end leg.

Options

1. step --command "/kb foo" --then-prompt "..." — needs vfa to accept two messages in one run (Pi's print mode already loops over a messages array internally; vfa would need to surface it, e.g. --prompt repeatable or --messages-file). Cleanest. Probably an upstream vf-agents change (cf. the --append-system-prompt-file upstreaming in [vf-agents PR chore(deps): bump actions/checkout from 4 to 6 #10]).
2. vfa session continuity — step switches from vfa run to vfa session so step N+1 continues step N's Pi conversation. Bigger harness change; also changes the semantics of every other multi-step scenario (they currently rely on KB_SESSION_ID state, not conversation carry-over — that's deliberate isolation).
3. Accept the gap — keep load-and-cite scaffolded, document it (done). The L1 + the 3 single-step L4 scenarios cover everything except the LLM-reads-it leg, which is the same role:'custom' → convertToLlm path the per-turn <mykb-context> injection uses and which area-scoring/scoring-isolated.sh already exercises end-to-end.

Cross-refs

• Surfaced while fixing the /kb bug (registerCommand shape + ctx.inject) — see docs/findings-log.md row QbRwqLxJ and kb gotcha QbRwqLxJ on the mykb area.
• experiments/kb-command/EXPERIMENT.md — behavior matrix + the blocked row.
• docs/experiment-coverage.md — kb-command listed with the load-and-cite sub-gap.

[vilosource]

Reframing — this gates an L4 dimension, not one row (2026-05-12)

A real kb-pi session asked "what did we work on most recent" and the LLM ran ls /home/jasonvi/DevOps instead of consulting the injected <mykb-workspace> block or kb recent. The extension did its job (block present — L1 would confirm it); the LLM ignored it. No matrix went RED, because none of the 13 matrices / 52 scenarios tests "naturalistic user prompt → LLM tool/citation choice" — they all stimulate by directly invoking a kb feature and assert on a deterministic anchor (JSONL/state-file/marker).

So the multi-message-in-one-Pi-process capability this issue asks for isn't just needed for the single load-and-cite row — it's the gating primitive for an entire missing class of L4 scenarios (working name: prompt-routing / resume-routing):

• "what did we work on" / "where did we leave off" → LLM cites <mykb-workspace> (or runs kb recent), does not ls/find
• "tell me about area X" → LLM runs /kb X (or kb_load/kb search) before answering
• "I learned Y about Z" → LLM offers kb add fact/decision/gotcha
• "I'm done for today" → LLM offers kb work handoff + kb save

The first one is single-turn (the workspace block is already in the system prompt), but the rest — and load-and-cite — need step-1-prompt → step-2-prompt continuity in one Pi process. Same blocker, much wider blast radius.

Action items added:

• kb gotcha EFsi7MAj + docs/findings-log.md methodology row recording the coupling-validation gap.
• This issue should be re-titled/re-scoped from "needed for /kb load-and-cite L4" to "gates the prompt-routing L4 dimension"; priority bumped accordingly (it was sitting in the handoff as "pick a fix sometime").
• Any <mykb-protocol> system-prompt nudge (the proposed fix for the ls failure) must ship with at least one prompt-routing scenario proving it changes LLM behavior — which means solving this issue first. That's the right ordering.

Option 1 (vfa accepts two messages per run — --prompt repeatable or --messages-file) still looks cleanest and is the natural upstream follow-on to vf-agents PR #10.