Skip to content

perf(training): batched Muon, distributed across DDP ranks - #129

Merged
briney merged 3 commits into
mainfrom
perf/distributed-muon
Nov 16, 2026
Merged

briney merged 3 commits into
mainfrom
perf/distributed-muon

Conversation

@briney

@briney briney commented Nov 15, 2026

Copy link
Copy Markdown
Owner

briney and others added 3 commits November 15, 2026 12:27
Refs #53

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Same-shape matrices are orthogonalized in one stacked Newton-Schulz call, each
rank computes only the matrices it owns, and a sum all-reduce per shape gives
every rank the full update. State is unchanged from torch.optim.Muon; HSDP
keeps stock Muon (#128).

Refs #53

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Refs #53

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@briney

briney commented Nov 16, 2026

Copy link
Copy Markdown
Owner Author

@briney
briney merged commit a445037 into main Nov 16, 2026
3 checks passed
@briney
briney deleted the perf/distributed-muon branch November 16, 2026 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant