About the RL stage

[LiuRicky]

Hi authors,

Thanks for your interesting work.

I have carefully read the issue #19 . Howevere, since the model archtecture is changed (not standard QwenVL, but with a spatial encoder), I wonder which rollout engine is used. Did you use the vllm or sglang? Could you please share your RL code for reference?

Looking forward to your reply!

[diankun-wu]

Hi, Thanks for your interest in our work.

Our RL framework is based on R1-V, whose underlying implementation uses TRL’s GRPO_Trainer, specifically: src/r1-v/src/open_r1/trainer/grpo_trainer.py.

During RL training, we only use the Transformers library as the rollout backend. We have not adapted vLLM or SGLang for rollout at this stage.

Please feel free to discuss further if you have any questions.

Best regards.