feat(composer): send images and text files with steers and follow-ups

AGENTS.md

@@ -257,12 +257,14 @@ client-side — every client viewing the session must show the same queue.
 One sequence (`queueSeqRef`) orders every source: a get_state snapshot takes
 a number when requested and applies only if no newer snapshot or
 `queue_update` was applied (HTTP and SSE can reorder). Edit/Delete use
-`remove_queued_message` (act only on `removed: true`), Steer uses
+`remove_queued_message` (act only on `removed: true`; newer omp also returns
+the message's `images`, which Edit restores), Steer uses
 `promote_queued_message`; the chip changes when omp's next snapshot arrives.
 `handleAbort` coalesces overlapping Stops, then sends `abort_and_restore_queue`:
 omp's Esc (`clearQueue({ forInterrupt: true })`, then abort) in one step,
-returning the withdrawn user messages, which go to the session draft via
-`recoverDraftText`. It covers what a client snapshot cannot: a steer promoted
+returning the withdrawn user messages, whose texts and images go to the
+session draft via `recoverDraft`. omp labels an image-only message `[Image]`;
+that label is never restored as text. It covers what a client snapshot cannot: a steer promoted
 after the last `queue_update`, and live-steered input the run claimed but never
 recorded (omp would otherwise requeue it and drain it into a new turn right
 after the abort). Never reimplement this client-side. A failed request is
@@ -303,9 +305,9 @@ during the wait.
   running `btw_record` comes first.
 - `/btw <question>` and `/btw` are client builtins (`handleBuiltinSlashCommand`
   case `"btw"`). `ChatInput.sendSideQuestion` routes them there from both the
-  idle and the streaming submit path, *before* the attachment gate: never sent
-  as a prompt, never queued, and refused with a toast (draft and attachments
-  kept) while attachments are attached. Asking starts the wrapper
+  idle and the streaming submit path, *before* attachments are folded in:
+  never sent as a prompt, never queued, and refused with a toast (draft and
+  attachments kept) while attachments are attached. Asking starts the wrapper
   (`get_state`) and attaches SSE first when it is not open, so no early delta
   is lost; a second ask while one is starting is ignored.
 - The `btw` response, history snapshots and frames race (HTTP vs SSE): merge

CHANGELOG.md

@@ -25,13 +25,14 @@ All notable changes to **omp-web** (`@kahme247/ompweb`) are documented in this f
 - Show all of an agent's ask-tool questions in one panel, with checkboxes for multi-select, radio buttons for single-select (recommended option marked and preselected), and an **Other** free-text answer per question, submitted together. Requires an omp that supports `set_ask_dialog`; older omp keeps the one-question-at-a-time dialog.
 - Make `agent://<id>` subagent handles in chat messages clickable. Bare handles, inline code containing only a handle, and Markdown links open that subagent's result dialog; `agent://Parent/Child` opens the nested `Parent.Child` subagent when the roster knows it. A `read` of an `agent://` handle opens the same dialog from its tool row.
 - Show the full model name, including its provider (for example `anthropic/claude-sonnet-5-5`), and the reasoning effort in the subagent result dialog, for running subagents and for finished ones after a reload. Finished background subagents take them from their saved transcript, so they also appear in the composer's subagent chips.
+- Attach images and text files while the agent is running, and send them with a steer or a queued follow-up. Attaching through **+**, paste, or drag and drop now works during a run; with only attachments in the composer, **Stop** becomes **Queue**. An image-only message shows as `[Image]` in the queue. A steer or follow-up that omp refuses returns to the composer with its images, and so does a queued message taken back by **Edit** or **Stop**. Getting images back from **Edit** and **Stop** requires an omp with `abort_and_restore_queue` and image-returning `remove_queued_message` ([can1357/oh-my-pi#14179](https://github.com/can1357/oh-my-pi/pull/14179)); older omp returns only the text.
 
 ### Fixes & Improvements
 
 - Fix settings crashes, stalled session catch-up, and failed voice transcription polling in browsers without `AbortSignal.timeout`, `AbortSignal.any`, or `Promise.withResolvers`, including development-mode clients. Client requests now use `AbortController` timers and ordinary promises while retaining timeouts and cancellation (#196).
 - The **Agent environment variables** editor in Settings → System & Updates now sits below its title and description and follows the page width, instead of sitting beside them and extending past the right edge. The text box also grows with its contents.
 - Show queued steers and follow-ups on every device viewing a session. The queue panel now shows omp's own queue (omp 18.4.4 or later) instead of a copy kept by the tab that sent them, so it also stays correct across reloads.
-- **Stop** no longer loses a queued steer or lets the agent run it anyway: as with Esc in omp's terminal, every message still waiting in the queue moves back into the composer, except attached images, which are dropped. With omp's `abort_and_restore_queue` (newer omp), omp takes the queue back and aborts in one step, which also covers a steer you just promoted that this page's queue had not caught up with, and one the model already picked up through live steering (for example OpenAI models with `providers.openaiLiveSteering`). If that request fails, Stop retries it once and warns when the messages may not have come back. Older omp (18.4.4 or later) falls back to removing each message the page lists before the abort; there, a steer the page does not list or the model already took still runs after Stop.
+- **Stop** no longer loses a queued steer or lets the agent run it anyway: as with Esc in omp's terminal, every message still waiting in the queue moves back into the composer. With omp's `abort_and_restore_queue` (newer omp), omp takes the queue back and aborts in one step, which also covers a steer you just promoted that this page's queue had not caught up with, and one the model already picked up through live steering (for example OpenAI models with `providers.openaiLiveSteering`). If that request fails, Stop retries it once and warns when the messages may not have come back. Older omp (18.4.4 or later) falls back to removing each message the page lists before the abort; there, a steer the page does not list or the model already took still runs after Stop, and attached images are dropped.
 - Long voice transcriptions no longer fail with `Unexpected token '<', "<!DOCTYPE "... is not valid JSON` behind a reverse proxy. The server now keeps the recording and transcribes it as a background job (up to about 5 minutes) while the browser polls short requests, so proxy timeouts no longer cut off a slow STT endpoint. The job survives the recording browser disconnecting: any browser showing the same session picks it up, can play the recording and retry a failure, and the transcript is inserted in exactly one browser. A **Transcribe and send** (or queue) choice stays with the job, so the transcript is still sent after you switch sessions and come back; if the agent is running by then it is queued as a follow-up, and a queue choice whose run has ended is sent. Another browser that finishes the job only inserts the text. **Transcribe and send** from the review deck now sends too. Recordings stay on the server (in memory) for up to an hour or until inserted or discarded. Anyone who can reach the omp-web API can list, play and retry those recordings by session, so set `OMP_WEB_PASSWORD` when omp-web is reachable by others.
 - Sending a message to a live session no longer fails with **HTTP 404** when the session's transcript cannot be read. An empty baseline is used instead, so the prompt is still delivered. The previous behavior is what surfaced as `Failed to send message: HTTP 404` on sessions created by slash commands.
 - The Skills page lists exactly what omp resolves, through `omp skill list --json` (omp 18.3.3 or later): plugin, custom-directory and registry skills appear, and colliding names show as `namespace/name`. Skills that live outside omp-web's skill folders, such as plugin installs, show a disabled toggle because an update would undo the edit. Older omp binaries keep the built-in scan.
@@ -115,7 +116,6 @@ All notable changes to **omp-web** (`@kahme247/ompweb`) are documented in this f
 - Expand complete tool inputs inline, including multiline code and edit patches, while keeping command previews compact and output visibility unchanged.
 - Keep composer controls on one line, with equally sized Send, Stop, and Queue buttons and model names truncating before short effort labels.
 - Align the + button and primary action with matching composer insets.
-- Clearly dim Attach files while the agent is running; queued messages remain text-only.
 - Recover saved responses before reporting an empty agent reply after returning to a backgrounded page or PWA. Preserve provider errors and distinguish new runs from older answers.
 - Catch up missed conversation entries incrementally after reconnecting or returning to the page, including during active runs. Restore quiet partial responses and live tool output without duplicating history or overwriting newer updates.
 - Send prompts with image attachments in full again: commands reach OMP as one unchunked JSONL record. Protocol-v2 `rpc_chunk` framing is outbound-only, so any prompt over 1 MiB was rejected as `Unknown command: rpc_chunk` and reset the session after the prompt-ack timeout.

components/ChatInput-draft-attachments.ts

@@ -3,10 +3,7 @@ import {
   selectTextAttachments,
   type AttachedTextFileData,
 } from "@/lib/chat-attachments";
-import {
-  MAX_ATTACHED_IMAGES,
-  isBase64ImageWithinLimits,
-} from "@/lib/image-attachments";
+import { isBase64ImageWithinLimits } from "@/lib/image-attachments";
 
 export interface AttachedImage {
   data: string;   // base64, no prefix
@@ -27,10 +24,11 @@ export function draftImageToAttachedImage(image: ChatDraftImage): AttachedImage
   };
 }
 
+/** Every image comes back, even past MAX_ATTACHED_IMAGES (recovered queue
+ *  messages can exceed it); sending enforces the cap, so the user chooses. */
 export function draftImagesToAttachedImages(images: ChatDraftImage[] | undefined): AttachedImage[] {
   return (images ?? [])
     .filter(isBase64ImageWithinLimits)
-    .slice(0, MAX_ATTACHED_IMAGES)
     .map(draftImageToAttachedImage);
 }
 

components/ChatInput.navigation.test.mjs

@@ -87,6 +87,9 @@ test("attachment-only drafts stay protected across live draft-key changes and re
   assert.equal(warnsOnExit(), false);
 });
 
+// The removed message's images, as omp returns them with `removed: true`.
+const QUEUED_IMAGES = [{ data: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII=", mimeType: "image/png" }];
+
 for (const navigation of ["draft-key change", "unmount"]) {
   test(`queued Edit recovers the old draft after ${navigation} while cancellation is pending`, async () => {
     const oldKey = `edit-old-${navigation}`;
@@ -118,13 +121,14 @@ for (const navigation of ["draft-key change", "unmount"]) {
       }
       fireEvent.change(screen.getByRole("textbox"), { target: { value: "new session draft" } });
       await act(async () => {
-        release(true);
+        release(QUEUED_IMAGES);
         await cancellation;
       });
       assert.equal(screen.getByRole("textbox").value, "new session draft");
       assert.equal(getDraft(newKey).value, "new session draft");
       assert.equal(getDraft(oldKey).value, "queued question\n\ndraft updated while waiting");
       assert.deepEqual(getDraft(oldKey).files, originalFiles);
+      assert.deepEqual(getDraft(oldKey).images, QUEUED_IMAGES, "the queued message's images come back with its text");
 
       if (navigation === "unmount") {
         view.unmount();
@@ -167,13 +171,14 @@ for (const navigation of ["original composer", "same-key remount"]) {
       await act(async () => {
         fireEvent.change(screen.getByRole("textbox"), { target: { value: "typed while waiting" } });
         ref.current.insertText("and queued insertion");
-        release(true);
+        release(QUEUED_IMAGES);
         await cancellation;
       });
       const recovered = "queued question\n\ntyped while waiting and queued insertion";
       assert.equal(screen.getByRole("textbox").value, recovered);
       assert.equal(getDraft(draftKey).value, recovered);
       assert.deepEqual(getDraft(draftKey).files, originalFiles);
+      assert.deepEqual(getDraft(draftKey).images, QUEUED_IMAGES, "the queued message's images come back with its text");
 
       fireEvent.change(screen.getByRole("textbox"), { target: { value: `${recovered}\nnext user edit` } });
       assert.equal(screen.getByRole("textbox").value, `${recovered}\nnext user edit`);

components/ChatInput.queue-attachments.test.mjs

@@ -1,21 +1,139 @@
-import test from "node:test";
+import "../tests/setup-dom.mjs";
 import assert from "node:assert/strict";
+import test, { afterEach, beforeEach } from "node:test";
+import React, { act } from "react";
+import { cleanup, fireEvent, render, screen, waitFor } from "@testing-library/react/pure.js";
+import userEvent from "@testing-library/user-event";
 import { createJiti } from "jiti";
 
-// ChatInput is TSX; parse it as such. `tsconfigPaths` resolves the "@/…" alias.
 const jiti = createJiti(import.meta.url, { jsx: { runtime: "automatic" }, tsconfigPaths: true });
-const { queueAllowsAttachments } = await jiti.import("../components/ChatInput.tsx");
+const { ChatInput } = await jiti.import("./ChatInput.tsx");
+const { clearDraft, getDraft } = await jiti.import("@/lib/draft-store");
 
-test("a queued message is text-only, run or not", () => {
-  assert.equal(queueAllowsAttachments(0, 0), true);
-  assert.equal(queueAllowsAttachments(1, 0), false, "an image blocks queueing");
-  assert.equal(queueAllowsAttachments(0, 1), false, "a pasted text file blocks queueing");
-  assert.equal(queueAllowsAttachments(3, 2), false);
+const KEY = "queue-attachments";
+// 1x1 transparent PNG.
+const PNG = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNkYAAAAAYAAjCB0C8AAAAASUVORK5CYII=";
+
+// Same stand-in as ChatInput.clipboard-image.test.mjs: jsdom's FileReader
+// brand-checks against its own realm's Blob; the composer only needs
+// readAsDataURL -> `result`.
+const previousFileReader = globalThis.FileReader;
+class TestFileReader {
+  result = null;
+  onload = null;
+  onerror = null;
+  readAsDataURL(blob) {
+    blob.arrayBuffer().then(
+      (buffer) => {
+        this.result = `data:${blob.type};base64,${Buffer.from(buffer).toString("base64")}`;
+        this.onload?.();
+      },
+      (error) => this.onerror?.(error),
+    );
+  }
+}
+
+beforeEach(() => {
+  window.matchMedia = () => ({ matches: false, addEventListener() {}, removeEventListener() {} });
+  globalThis.FileReader = TestFileReader;
+  URL.createObjectURL ??= () => "blob:test";
+  URL.revokeObjectURL ??= () => {};
+});
+afterEach(() => {
+  cleanup();
+  clearDraft(KEY);
+  localStorage.clear();
+  delete window.matchMedia;
+  globalThis.FileReader = previousFileReader;
+});
+
+/** A composer mid-run with an image and a text file attached through the picker path. */
+async function renderRunningWithAttachments({ queued = true } = {}) {
+  const calls = [];
+  // Images are the last argument of every queue callback, which resolves
+  // false when omp refused the message.
+  const record = (name) => async (message, ...rest) => { calls.push({ name, message, images: rest.at(-1) }); return queued; };
+  const ref = React.createRef();
+  render(React.createElement(ChatInput, {
+    ref,
+    draftKey: KEY,
+    isStreaming: true,
+    onSend: record("onSend"),
+    onAbort: record("onAbort"),
+    onSteer: record("onSteer"),
+    onFollowUp: record("onFollowUp"),
+    onPromptWithStreamingBehavior: record("onPromptWithStreamingBehavior"),
+  }));
+  await act(async () => {
+    ref.current.addFiles([
+      new File([Buffer.from(PNG, "base64")], "dot.png", { type: "image/png" }),
+      new File(["hello"], "notes.txt", { type: "text/plain" }),
+    ]);
+  });
+  await waitFor(() => {
+    const draft = getDraft(KEY);
+    assert.equal(draft?.images.length, 1, "attaching is allowed while the agent runs");
+    assert.equal(draft?.files.length, 1);
+  });
+  return calls;
+}
+
+function assertQueuedWithAttachments(call, text) {
+  assert.ok(call.message.startsWith(text), "typed text leads the message");
+  assert.match(call.message, /Attached file: notes\.txt\n```[^\n]*\nhello\n```/, "text file is folded in");
+  assert.deepEqual(call.images?.map((image) => [image.data, image.mimeType]), [[PNG, "image/png"]]);
+}
+
+test("Enter during a run steers with the attached image and text file, then clears them", async () => {
+  const user = userEvent.setup();
+  const calls = await renderRunningWithAttachments();
+  await user.type(screen.getByRole("textbox"), "look at this");
+  await user.keyboard("{Enter}");
+  await waitFor(() => assert.equal(calls.length, 1));
+  assert.equal(calls[0].name, "onSteer", "default submit-during-run behavior steers");
+  assertQueuedWithAttachments(calls[0], "look at this");
+  await waitFor(() => assert.equal(getDraft(KEY), null));
+});
+
+test("attachments alone turn Stop into Queue, which queues them as a follow-up", async () => {
+  const calls = await renderRunningWithAttachments();
+  fireEvent.click(screen.getByRole("button", { name: /queue/i }));
+  await waitFor(() => assert.equal(calls.length, 1));
+  assert.equal(calls[0].name, "onFollowUp");
+  assertQueuedWithAttachments(calls[0], "Attached file:");
+});
+
+test("a queued web slash command is expanded and keeps the attachments", async () => {
+  const { expandWebSlashCommand } = await jiti.import("@/lib/web-slash-commands");
+  window.HTMLElement.prototype.scrollIntoView = () => {};
+  const user = userEvent.setup();
+  const calls = await renderRunningWithAttachments();
+  await user.type(screen.getByRole("textbox"), "/goal ship it");
+  await user.keyboard("{Enter}");
+  await waitFor(() => assert.equal(calls.length, 1));
+  delete window.HTMLElement.prototype.scrollIntoView;
+  assert.equal(calls[0].name, "onPromptWithStreamingBehavior");
+  assertQueuedWithAttachments(calls[0], expandWebSlashCommand("/goal ship it").prompt);
+});
+
+test("a refused follow-up goes back whole to its draft, ahead of what was typed meanwhile", async () => {
+  let refuse;
+  const calls = await renderRunningWithAttachments({ queued: new Promise((resolve) => { refuse = () => resolve(false); }) });
+  fireEvent.click(screen.getByRole("button", { name: /queue/i }));
+  await waitFor(() => assert.equal(calls.length, 1));
+  fireEvent.change(screen.getByRole("textbox"), { target: { value: "typed meanwhile" } });
+  await act(async () => { refuse(); });
+  await waitFor(() => assert.deepEqual(getDraft(KEY)?.images, [{ data: PNG, mimeType: "image/png" }]));
+  assert.equal(getDraft(KEY)?.value, `${calls[0].message}\n\ntyped meanwhile`, "the text, file contents included, comes back too");
+  assert.equal(screen.getByRole("textbox").value, `${calls[0].message}\n\ntyped meanwhile`);
 });
 
-test("the rule has no run-state input, so it cannot report the agent's state as the reason", () => {
-  // Regression guard for the shipped copy: the toast once said attachments
-  // "cannot be queued while the agent is running", which is narrower than the
-  // rule it explains (omp refuses attachments regardless of `isStreaming`).
-  assert.equal(queueAllowsAttachments.length, 2, "only the two attachment counts decide");
+test("recovered images past the attachment cap all stay in the composer", async () => {
+  const { MAX_ATTACHED_IMAGES } = await jiti.import("@/lib/image-attachments");
+  const { recoverDraft } = await jiti.import("@/lib/draft-store");
+  await renderRunningWithAttachments();
+  const recovered = Array.from({ length: MAX_ATTACHED_IMAGES }, () => ({ data: PNG, mimeType: "image/png" }));
+  await act(async () => { recoverDraft(KEY, { text: "", images: recovered }); });
+  await waitFor(() => assert.equal(getDraft(KEY)?.images.length, MAX_ATTACHED_IMAGES + 1));
+  assert.equal(document.querySelectorAll("img").length >= MAX_ATTACHED_IMAGES + 1, true, "every image has a preview to remove");
 });