[feat] add image support to cli #386
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add image-handler.ts for auto-detecting and processing image paths - Add terminal-images.ts for iTerm2/Kitty inline image rendering - Add ImageContentBlock type and ImageBlock component - Update SDK RunOptions to accept multimodal content (text + images) - Auto-detect image paths in user input (e.g. @image.png, ./path/to/image.jpg) - Display images inline in supported terminals, fallback to metadata
…rd paste, and visual cards - Add /image slash command to attach images - Add image input mode similar to bash/referral mode - Add Ctrl+V clipboard paste for images (macOS/Linux/Windows) - Add PendingImagesBanner with visual image cards and X to remove - Update keyboard actions to detect Ctrl+V for image paste - Remove pasted status messages (now shown in banner)
…tachments in user messages
- Add image-thumbnail.ts utility to extract pixel colors using Jimp - Add ImageThumbnail component rendering with Unicode half-blocks (▀) - Use OpenTUI native fg/backgroundColor styling instead of ANSI escapes - Works in terminals without iTerm2/Kitty inline image support - Falls back to emoji if image processing fails
- Store images with queued messages so they travel with the message - Clear pending images when queuing to fix banner persistence - Fix banner not showing after /image command by rendering directly - Update queue types to support QueuedMessage objects with content and images
Ensures consistent XML framing when content contains both text and images by wrapping only the first text part in <user_message> tags.
- Remove duplicate AskUserContentBlock from ContentBlock union type - Remove unused execSync import from clipboard-image.ts - Fix Ctrl+V to fall through for text paste when no image in clipboard - Fix buildUserMessageContent to wrap text in user_message tags for multipart content
- Increase timeout from 800ms to 2000ms for slower CI environments - Listen to both stdout and stderr for CLI startup detection
- Remove dead code (validateTotalImageSize, MAX_TOTAL_SIZE) - Make CLI processImageFile synchronous (no async needed) - Simplify normalizeUserProvidedPath (consolidated unicode/shell escapes) - Remove unnecessary clipboard image cleanup (OS handles temp files) - Remove verbose logging from clipboard-image.ts - Unexport internal functions (generateITerm2ImageSequence, generateKittyImageSequence) - Add logging to silent catch blocks in image-thumbnail.ts and terminal-images.ts - Extract magic numbers to named constants in image-card.tsx
- Replace inline import with proper type import in message-footer.tsx - Remove keyboard debug logging from use-chat-keyboard.ts - Convert map().filter() to reduce() in use-send-message.ts - Remove duplicate implementor helpers from constants.ts (already in implementor-helpers.ts)
Previously /image would transform the command into a prompt with the path, expecting auto-detection. Now it properly adds to pendingImages in the chat store so the image shows in the banner before sending.
- Use bilinear interpolation for sharper thumbnail downscaling - Reduce card width (18 chars) and adjust thumbnail size - Move close button outside the card border for better visibility - Remove gray background from thumbnail area
brandonkachen
force-pushed
the
image-support
branch
from
eff3582 to
094bd02
Compare
Merged bash ghost UI and other main branch features with the image support feature branch. Key changes integrated: - Bash ghost UI with dashed box display - Pending bash message handling and tool result preservation - Code search updates to include hidden dirs (.agents, .github) - Multi-select fix for empty responses - Run integration test fix - Message type updates in .agents Conflicts resolved in: - cli/src/chat.tsx - Combined image paste + bash history handlers - cli/src/commands/router.ts - Added bash execution helpers + kept image handling - cli/src/components/message-block.tsx - Added metadata + kept attachments - cli/src/components/message-with-agents.tsx - Added metadata + kept attachments - cli/src/hooks/use-chat-keyboard.ts - Added bash history + kept paste handlers - cli/src/hooks/use-send-message.ts - Added tool results + kept image processing - cli/src/state/chat-store.ts - Added bash/tool state + kept image state - cli/src/utils/keyboard-actions.ts - Added bash history + kept paste actions - packages/agent-runtime/src/util/messages.ts - Kept image wrapping with idempotency check - sdk/src/index.ts - Added buildUserMessageContent export + kept image types - sdk/src/run.ts - Fixed duplicate content property from merge
- Remove unreachable help message block from image.ts (handleImageCommand is only called with args) - Update image input mode placeholder to show Ctrl+V paste hint - Clean up unused SUPPORTED_IMAGE_EXTENSIONS import Code-reviewed: confirmed help should show in input placeholder (temporary UI) not message history
- Show "file not found" and "unsupported format" errors in the pending images banner instead of message history - Add isError flag to PendingImage type for proper error detection - Auto-remove error messages after 3 seconds - Style error-only banner with red border - Use pluralize helper for image count
- Simplify handleImageCommand to return just a string instead of complex object - Use split() instead of regex for parsing - Remove unused postUserMessage callback pattern - Add unit tests for argument parsing behavior
…ImagesBanner - Create new InputModeBanner component in its own file - Move usageBannerShowTime state management into InputModeBanner - Always render PendingImagesBanner alongside mode banners - Simplify ChatInputBar by removing conditional banner rendering
- Extract width/height from images via Jimp in processImageFile - Add calculateDisplaySize utility for aspect-ratio-preserving display - Pass dimensions through PendingImage and ImageContentBlock types - Replace hardcoded constants in image-block.tsx with dynamic sizing - Add comprehensive TDD tests for dimension extraction and sizing
…e data The action object passed to callMainPrompt had two content properties: 1. content: messageContent (conditionally added for images) 2. content: preparedContent (always added) In JS object literals, later properties overwrite earlier ones, so images were never sent to the LLM. The fix removes the duplicate property since preparedContent already handles images correctly via wrapContentForUserMessage -> buildUserMessageContent.
Move SUPPORTED_IMAGE_EXTENSIONS and image size limits (MAX_IMAGE_FILE_SIZE, MAX_IMAGE_BASE64_SIZE, MAX_TOTAL_IMAGE_SIZE) to a shared constants file.
- Extract compression logic into compressImageToFitSize() function - Replace getMimeTypeFromExtension switch with EXTENSION_TO_MIME lookup - Simplify re-exports with export * - Use early returns to flatten processImageFile validation - Remove local aliases, use imported constant names directly - Simplify extractImagePaths with addPath helper and matchAll()
- Fix race condition in addPendingImageFromFile by using functional setState - Improve validateAndAddImage to return actual error messages
…acking - Add typed status field to PendingImage (processing/ready/error) instead of string matching - Show image banner immediately on Ctrl+V before clipboard check completes - Display "X images attached, Y images processing" in banner header - Move processing indicator to note area, show generic thumbnail fallback - Add tests for image lifecycle status transitions - Simplify pending images filtering with single-pass loop
…eedback - Extract clipboard text reading logic into readClipboardText() in clipboard-image.ts - Simplify onPasteImage handler in chat.tsx using the new utility - Add user feedback message when images are still processing in router.ts
Resolved conflict in use-send-message.ts by combining: - Image support: MessageContent type, effectivePrompt for image-only messages, content param - Bash context: prepending bash history to prompts for LLM context
…sages When users send only images without text, the fallback prompt "See attached image(s)" was being passed as a separate prompt parameter but ignored by buildUserMessageContent. Now the fallback text is included directly in the content array so the model receives the instruction.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
dis p cool yo