Skip to content

feat: support customized multimodal preprocess configs.#1481

Open
xanecdotex wants to merge 1 commit into
jd-opensource:mainfrom
xanecdotex:mmconfig
Open

feat: support customized multimodal preprocess configs.#1481
xanecdotex wants to merge 1 commit into
jd-opensource:mainfrom
xanecdotex:mmconfig

Conversation

@xanecdotex
Copy link
Copy Markdown
Collaborator

No description provided.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new MMMessageConfig structure to allow per-message configuration for multimodal inputs, such as pixel limits, frame counts, and FPS. These configurations are propagated from the API layer through the MMInput class to the image processors for GLM-4V and Qwen2-VL. The review feedback correctly identifies critical thread-safety issues in both Glm4VImageProcessor and Qwen2VLImageProcessor, where request-specific configurations are being stored in shared member variables, leading to potential race conditions in concurrent environments. Additionally, the logic for handling multiple mm_config instances within a single request needs to be clarified to avoid unintentional overwriting.

Comment thread xllm/processors/glm4v_image_processor.cpp Outdated
Comment thread xllm/processors/qwen2_vl_image_processor.cpp Outdated
Comment thread xllm/core/framework/request/mm_input.cpp Outdated
@xanecdotex xanecdotex marked this pull request as draft May 19, 2026 08:07
@xanecdotex xanecdotex marked this pull request as ready for review May 20, 2026 08:33
@xanecdotex xanecdotex requested a review from wly-115 May 20, 2026 10:05
@xanecdotex xanecdotex marked this pull request as draft May 20, 2026 10:15
@xanecdotex xanecdotex marked this pull request as ready for review May 22, 2026 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant