Skip to content

feat: Support Flux2 in other common components#1463

Open
wang-shuibin wants to merge 4 commits into
jd-opensource:mainfrom
wang-shuibin:flux2_others
Open

feat: Support Flux2 in other common components#1463
wang-shuibin wants to merge 4 commits into
jd-opensource:mainfrom
wang-shuibin:flux2_others

Conversation

@wang-shuibin
Copy link
Copy Markdown

This PR implements adaptation for Flux2 in common components. Meanwhile, accuracy verification for the Flux2 model has been finished Flux2 model.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements a Mistral prompt-to-message conversion flow and extends the Tokenizer interface to support a max_sequence_length parameter with padding logic. Feedback highlights several critical issues: a potential out-of-bounds crash and debug code in EmbedWorkerImpl, a violation of the repository style guide regarding global flag registration in help_formatter.h, and multiple safety concerns in LLMMaster including missing error callbacks, exception-unsafe JSON parsing, and the failure to clear stale prompt tokens after a prompt update.

Comment thread xllm/core/runtime/embed_worker_impl.cpp Outdated
Comment thread xllm/core/common/global_flags.cpp Outdated
Comment thread xllm/core/distributed_runtime/llm_master.cpp
Comment thread xllm/core/distributed_runtime/llm_master.cpp Outdated
Comment thread xllm/core/distributed_runtime/llm_master.cpp Outdated
Copy link
Copy Markdown
Author

@wang-shuibin wang-shuibin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR adapts common components for Flux2 and completes accuracy verification for the Flux2 model. Additionally, it fixes a bug in xllm/core/layers/common/add_matmul.cpp.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant