Hello, and thank you for the great work.
I am fine-tuning the vanilla Qwen2.5-3B (bf16) model using the baseline training script, but training does not seem to progress.
Even after 500+ steps, both loss and reward remain at 0, which differs from the behavior reported in the paper.
I suspect this may be related to the format reward, but I am not sure how to address it.
Is there a known issue or recommended setup for the format reward?
Should the model first be SFT-trained to learn the output formatting, instead of starting GRPO training directly from a cold start?
Any advice would be greatly appreciated.
Thank you for your help.
Hello, and thank you for the great work.
I am fine-tuning the vanilla Qwen2.5-3B (bf16) model using the baseline training script, but training does not seem to progress.
Even after 500+ steps, both loss and reward remain at 0, which differs from the behavior reported in the paper.
I suspect this may be related to the format reward, but I am not sure how to address it.
Is there a known issue or recommended setup for the format reward?
Should the model first be SFT-trained to learn the output formatting, instead of starting GRPO training directly from a cold start?
Any advice would be greatly appreciated.
Thank you for your help.