All pull requests

Pull requests

• Qwen3-30B-A3B: flex_flash and varlen attention configs (+36% / +42%)

  #74·djmatusz-lambda opened Oct 7, 2026Collaborator·

  ·

• mHc Grad Chain implemented from TE

  #70·djmatusz-lambda opened Sep 25, 2026Collaborator·33/33·

  ·

• Adding BF16 GeMM/TF32 Accumulation Logic to compressor.py

  #69·djmatusz-lambda opened Sep 25, 2026Collaborator·32/33·

  ·

• BF16 GeMM with FP32 Accumulation; inert

  #68·djmatusz-lambda opened Sep 25, 2026Collaborator·31/33·

  ·

• TMA-store row copy (MINIMAL_ASYNC_EP_COPY_TMA=1)

  #67·djmatusz-lambda opened Sep 25, 2026Collaborator·30/33·

  ·

• FSDP direct reduce-scatter Implemented from Megatron

  #66·djmatusz-lambda opened Sep 25, 2026Collaborator·29/33·

  ·

• Flipping Determinism Off for cuDNN DSA

  #65·djmatusz-lambda opened Sep 25, 2026Collaborator·28/33·

  ·

• Fun Profiling

  #64·djmatusz-lambda opened Sep 25, 2026Collaborator·27/33·

  ·

• FSDP direct gather

  #63·djmatusz-lambda opened Sep 25, 2026Collaborator·26/33·

  ·

• Packed Experts

  #62·djmatusz-lambda opened Sep 25, 2026Collaborator·25/33·

  ·

• EP Sweep Attempt

  #61·djmatusz-lambda opened Sep 25, 2026Collaborator·24/33·

  ·

• Replacing Fixed Buffers in MinimalAsyncEP

  #60·djmatusz-lambda opened Sep 25, 2026Collaborator·23/33·

  ·

• Dual Microbatch Implementation tried from Megatron-LM

  #59·djmatusz-lambda opened Sep 25, 2026Collaborator·22/33·

  ·

• Env knobs for the row-copy launch geometry with MinimalAsyncEP

  #58·djmatusz-lambda opened Sep 25, 2026Collaborator·21/33·

  ·

• Expert Load Balancing (massive win)

  #57·djmatusz-lambda opened Sep 25, 2026Collaborator·20/33·

  ·

• FSDP_PREFETCH_DEPTH knob for how deep you want to prefetch

  #56·djmatusz-lambda opened Sep 25, 2026Collaborator·19/33·

  ·

• Split/31 bounded swiglu

  #55·djmatusz-lambda opened Sep 25, 2026Collaborator·18/33·

  ·

• HybridEP Attempt

  #54·djmatusz-lambda opened Sep 25, 2026Collaborator·17/33·

  ·

• Data streamed from local storage

  #53·djmatusz-lambda opened Sep 25, 2026Collaborator·16/33·

  ·

• Persistent backward workspace in DSA

  #52·djmatusz-lambda opened Sep 25, 2026Collaborator·15/33·

  ·

• TE_REDUCE_AMAX=0 skips TE's per-module amax all-reduce

  #51·djmatusz-lambda opened Sep 25, 2026Collaborator·14/33·

  ·

• Selective AC Attempts

  #50·djmatusz-lambda opened Sep 25, 2026Collaborator·13/33·

  ·

• Offload Full AC inputs to Pinned Memory

  #49·djmatusz-lambda opened Sep 25, 2026Collaborator·12/33·

  ·

• AdamW Optimizer state offload, taken from Megatron-LM #6244 (Mem Optimization)

  #48·djmatusz-lambda opened Sep 25, 2026Collaborator·11/33·

  ·

• Fused mHC taken from TE

  #47·djmatusz-lambda opened Sep 25, 2026Collaborator·10/33·

  ·