A 48.6M-parameter Co4 language model trained on 1.9B tokens, with hybrid Muon plus AdamW optimization and a retention-aware post-training study.
nlp machine-learning deep-learning reproducible-research language-modeling pytorch transformer muon language-model from-scratch post-training gqa pretraining cpu-training large-language-model llm-evaluation small-language-model causal-language-model co4 efficent-llm
-
Updated
Oct 1, 2026 - Python