fix: support multimodal image content in OpenAI provider#14
Open
buuzzy wants to merge 1 commit intocodeany-ai:mainfrom
Open
fix: support multimodal image content in OpenAI provider#14buuzzy wants to merge 1 commit intocodeany-ai:mainfrom
buuzzy wants to merge 1 commit intocodeany-ai:mainfrom
Conversation
The OpenAI provider's convertUserMessage() only handled 'text' and 'tool_result' content blocks, silently dropping 'image' blocks. Changes: - Convert Anthropic-style image blocks to OpenAI image_url format - Support base64 and URL image sources - Use detail:'high' for best recognition quality - Fall back to string content when no images present - Accept string | any[] prompt in query() for multimodal content
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The OpenAI provider drops
{ type: 'image' }content blocks during message conversion, so images are never sent to the model when using OpenAI-compatible endpoints.Changes:
convertUserMessage()now converts Anthropic-style image blocks ({ type: 'base64', media_type, data }) to OpenAI'simage_urlformatquery()prompt type widened tostring | any[]for multimodal content arrays