Skip to content

[model] fix: Map DeepSeek V4 hash layer config - #3

Draft
bobboli wants to merge 1 commit into
backport/dsv4-bridgefrom
fix/dsv4-integration
Draft

[model] fix: Map DeepSeek V4 hash layer config#3
bobboli wants to merge 1 commit into
backport/dsv4-bridgefrom
fix/dsv4-integration

Conversation

@bobboli

@bobboli bobboli commented Aug 19, 2026

Copy link
Copy Markdown
Owner

Summary

  • map the DeepSeek V4 hash-layer count to the Megatron-Core dsv4_n_hash_layers field
  • read the same field when exporting the Hugging Face config
  • add import and export regression coverage

The previous code assigned moe_n_hash_layers dynamically. The Megatron-LM runtime defines the hash-layer count as TransformerConfig.dsv4_n_hash_layers and reads it in Router._init_routing_mode() to select hash routing. It does not read moe_n_hash_layers.

As a result, the trainer constructed learned routers for the first three layers while SGLang used deterministic hash routing. That caused both the R3 replay failure and the large rollout/trainer mismatch.

Verification

  • uv run python -m pytest tests/unit_tests/models/deepseek/test_deepseek_v4_bridge.py -q: 16 passed
  • targeted pre-commit hooks: passed

@bobboli
bobboli force-pushed the fix/dsv4-integration branch from c0524b4 to d8c14fc Compare August 19, 2026 08:55
@bobboli bobboli changed the title [model, conversion] fix: Complete DeepSeek V4 direct-HF integration [model, conversion] fix: Support DeepSeek V4 weight export and hash routing Aug 19, 2026
@bobboli
bobboli force-pushed the fix/dsv4-integration branch from d8c14fc to 3159640 Compare August 19, 2026 09:18
@bobboli bobboli changed the title [model, conversion] fix: Support DeepSeek V4 weight export and hash routing [model] fix: Map DeepSeek V4 hash layer config Aug 19, 2026
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant