Roadmap for LightRFT v0.1.1

[puyuan1996]

🗺️ Roadmap for LightRFT v0.1.1

Expected Release: January 2026

✨ New Features

• Multimodal Support

  • Add video support for reinforcement finetuning (feat(wzn): add video support for reinforcement finetuning #4)

• Training & Evaluation

  • Implement and optimize evaluation for SRM (Step-wise Reward Model) and GRM (Generative Reward Model) trainers (feat(wzn): implement and optimize evaluation for SRM and GRM trainers #12)
  • Add high entropy token selection mechanism (feature(sunjx): add high entropy token selection #6)

• Analysis

  • Add analysis metrics to saved trajectories ( feature(sunjx): add some analysis metrics to saved trajectories #5)

⚙️ Compatibility & Dependencies

• Library Updates
  • [WIP] Adapt LightRFT to latest versions of sglang, vllm, and deepspeed (polish(pu): adapt lightrft to latest versions of sglang #24)
• Framework Compatibility
  • Rename dtype to torch_dtype for better transformers compatibility (fix(wzn): rename dtype to torch_dtype in for transformers compatibility #7)

🐛 Bug Fixes & Maintenance

• Fixes
  • Fix bug in GRM dataset message formatting and evaluation logic (fix(wzn): fix bug in GRM dataset message formatting and evaluation logic #8)
  • Remove redundant tuple nesting in prepare_reward_model return when using FSDP (fix(wzn): remove redundant tuple nesting in prepare_reward_model return when using FSDP #15)
• Code Style
  • Fix make fcheck in lightrft/datasets for linting errors (style(wzn): fix make fcheck in lightrft/datasets for linting errors #10)

📚 Documentation

• Deployment
  • Setup documentation deploy actions (doc(nyz): setup doc deploy actions #18)
• Content Updates
  • Update Python typing lint (style(nyz): polish typing lint #26)
  • Polish API comment doc details (polish(pu): polish api doc #21)
  • Update GRM on T2I benchmark results and analysis in best practices (docs(wzn): update GRM on T2I benchmark results and analysis in best practice #9)
  • Update general documentation and README for v0.1.1 (doc(nyz): update doc and README for v0.1.1 #2)