[Model][SM70] Accept DFlash embed replica across split load_weights c… - #651
ProprietaryLegal wants to merge 1 commit into
Conversation
…alls AutoWeightsLoader groups consecutive tensors by top-level prefix, so a checkpoint whose shard order interleaves model.language_model.* with lm_head/model.visual.* (the 33-shard nvidia/GLM-5.3-Flash-NVFP4 layout) calls Glm5NextModel.load_weights more than once on the final pipeline stage. The DFlash target-embedding replica check assumed a single call and raised on the group without embed_tokens.weight. Record the replica as loaded when any call delivers it; spec_decode/dflash/utils.py still refuses to borrow an unloaded replica. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015W5aeeDsFXgtoDWJZjtDpZ Signed-off-by: Nick <mermiges@gmail.com>
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in PRs do not trigger a full CI run by default. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. Agent GuidelinesIMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban. 🚀 |
…alls
AutoWeightsLoader groups consecutive tensors by top-level prefix, so a checkpoint whose shard order interleaves model.language_model.* with lm_head/model.visual.* (the 33-shard nvidia/GLM-5.3-Flash-NVFP4 layout) calls Glm5NextModel.load_weights more than once on the final pipeline stage. The DFlash target-embedding replica check assumed a single call and raised on the group without embed_tokens.weight. Record the replica as loaded when any call delivers it; spec_decode/dflash/utils.py still refuses to borrow an unloaded replica.
Claude-Session: https://claude.ai/code/session_015W5aeeDsFXgtoDWJZjtDpZ
Purpose
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.