Mixture of datasets#138

Open

m1kush wants to merge 42 commits into

mainfrom

mixture-of-datasets

Commits on Jan 1, 2026

add skeleton of mixture of datasets with new configuration files and refactor dataset classes
m1kush
committed
refactor dataset handling to support mixture of datasets with weighted sampling
m1kush
committed
add skeleton of mixture of datasets with new configuration files and refactor dataset classes
m1kush
committed
refactor dataset initialization to always use MixtureOfDatasets for improved consistency
m1kush
committed
add skeleton of mixture of datasets with new configuration files and refactor dataset classes
m1kush
committed
add tests for tokenize_fn to verify behavior across different models
m1kush
committed
update dataloader configurations to use get_mixture_of_datasets_dataloader and standardize tokenize_fn across models
m1kush
committed
refactor dataset configurations to use path-weight pairs and update dataloader settings
m1kush
committed
update dataset weights in fineweb.yaml and clean up smollm_corpus.yaml configurations
m1kush
committed

Commits on Mar 18, 2026

Commits on Mar 25, 2026

update smollm config files
m1kush
committed

Commits on Mar 28, 2026

Commits on Apr 10, 2026

update dataset references and checkpoint paths in configuration files
m1kush
committed