Skip to content

Compile the coverage loss with the model when compile is set - #10

Merged
davek44 merged 1 commit into
mainfrom
compile-loss
Oct 16, 2026
Merged

davek44 merged 1 commit into
mainfrom
compile-loss

Conversation

@davek44

@davek44 davek44 commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator

With compile: true, only the model was compiled; the elementwise poisson_mn loss over B × T × L ran eagerly. This PR compiles the loss too (dynamic=False, one specialization per dataset/batch shape). MLM is excluded.

Evidence (RTX 4090, L 16,384, no graph breaks, 6 graphs — one per shape and grad mode):

case rel loss rel grad eager ms compiled ms
human (B 4, T 768) 1.1e-7 2.2e-5 10.19 1.79
mouse (B 4, T 384) 0 2.4e-5 5.10 0.90
human, track weights 1.1e-7 3.0e-5 10.19 1.79
human, final batch (B 2) 1.1e-7 2.2e-5 5.09 0.90

5.7× faster than eager, with ~1/5 the transient memory. Each new (dataset, batch shape) pays one compile, like the model itself.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@davek44
davek44 merged commit 3d8267a into main Oct 16, 2026
3 checks passed
@davek44
davek44 deleted the compile-loss branch October 16, 2026 05:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant