Skip to content
#

llm-pretraining

Here are 14 public repositories matching this topic...

Unofficial reproduction of Nemotron-CLIMB (CLustering-based Iterative Data Mixture Bootstrapping, NeurIPS 2025, arXiv:2504.13161) on Ascend NPUs — 5 rounds, ≈3,900 NPU-hours; learned mixtures beat every fixed-ratio baseline (+0.014–0.043 STEM); the selection mechanism is the weak link — fixed and validated live.

  • Updated Oct 29, 2026
  • Python

Benchmark decontamination for LLM pretraining data: fast, auditable n-gram detection and removal of evaluation-benchmark text in training corpora (Rust).

  • Updated Nov 9, 2026
  • Python

Add this topic to your repo

To associate your repository with the llm-pretraining topic, visit your repo's landing page and select "manage topics."

Learn more