Search results

AdamW performing Adam without weight decay?

Status: Open.

#71 In HomebrewML/HeavyBall;

· tom-jod opened on Jul 1, 2025

MARS gamma parameter

Status: Open.

#62 In HomebrewML/HeavyBall;

· tom-jod opened on May 19, 2025

Significant degradation between versions

Status: Open.

#58 In HomebrewML/HeavyBall;

· keveman opened on Apr 22, 2025

[Feature request] SCION optimizer

Status: Open.

#54 In HomebrewML/HeavyBall;

· sozforex opened on Mar 22, 2025

[Feature] Sophia Optimizer

Status: Open.

#49 In HomebrewML/HeavyBall;

· namgyu-youn opened on Mar 4, 2025

[Feature Request] AdaLomo optimizer

Status: Open.

#35 In HomebrewML/HeavyBall;

· akbar2habibullah opened on Jan 29, 2025

CUDA Illegal memory access when starting training

Status: Open.

#18 In HomebrewML/HeavyBall;

· marthinwurer opened on Nov 30, 2024

[Feature] Make HeavyBall compatible with FSDP2 (DTensor)

Status: Open.

#15 In HomebrewML/HeavyBall;

· casper-hansen opened on Nov 22, 2024

fix for copy_stochastic with Dtensor

Status: Open.

#14 In HomebrewML/HeavyBall;

· ethansmith2000 opened on Nov 22, 2024

[Question] Distributed Training

Status: Open.

#1 In HomebrewML/HeavyBall;

· zaptrem opened on Nov 5, 2024