Subject-Based Supervised Fine-tuning

This study conducted to search if decreasing model number by domain-grouped training would reduce the dilution of learned parameters at merging and improve the model performance. In order to perform experiment, we have split dataset into four parts based on subject: Finance, Economics, Politics and Government. After that we grouped subjects based on their similarity as Finance-Economics and Politics-Government. Moreover mixed grouping is also applied, non-related subjects are united. After all this grouping Meta's Llama-3.2-1B-Instruct model is fine-tuned. This way we obtained eight models, each fine-tined in different dataset partition with same sample size:

• fine_tuned_Llama1B_Finance
• fine_tuned_Llama1B_Economics
• fine_tuned_Llama1B_Politics
• fine_tuned_Llama1B_Government
• fine_tuned_Llama1B_Money
• fine_tuned_Llama1B_Social
• fine_tuned_Llama1B_Mixed_Politics_Finance
• fine_tuned_Llama1B_Mixed_Government_Economics

After fine-tuning, models are merged as grouped below:

• fine_tuned_Llama1B_Merged_Cat: fine_tuned_Llama1B_Finance, fine_tuned_Llama1B_Economics, fine_tuned_Llama1B_Government, fine_tuned_Llama1B_Politics
• fine_tuned_Llama1B_Merged_Subject: fine_tuned_Llama1B_Money, fine_tuned_Llama1B_Social
• fine_tuned_Llama1B_Merged_Mixed: fine_tuned_Llama1B_Mixed_Politics_Finance, fine_tuned_Llama1B_Mixed_Government_Economics

At the end, all eleven model are tested. Each fine-tuned model is evaluated in their fine-tuned category-ies. On the other hand merged models are tested in all four category.

Important

Because of the corrupted shards in .zst file, dataset must be cleaned first. Before everything, run "python load_and_clean_dataset.py". This way dataset is loaded in chunks first and united afterwards. Chunk sample size can be changed from the file if your RAM has more storage. You can delete chunk folders after dataset is fully loaded and saved.

After each model is trained through "python train.py", merge lora parameters with base model executing "python merge_lora.py". Following lora merge, merge final models as specified in yaml files reside in "marge_yaml" folder by "sh merge.sh". After that, folders inside of "output/temporary" can be removed and models can be test from now on.

Steps

1. python load_and_clean_dataset.py
2. python train.py
3. python merge_lora.py
4. sh merge.sh
5. python test.py
6. python test_inference.py