metadata
license: apache-2.0
Training Multimodal Reward Model Through Stable Reinforcement Learning
🔥 We are proud to open-source R1-Reward, a comprehensive project for improve reward modeling through reinforcement learning. This release includes:
- R1-Reward Model: A state-of-the-art (SOTA) multimodal reward model demonstrating substantial gains (Voting@15):
- 13.5% improvement on VL Reward-Bench.
- 3.5% improvement on MM-RLHF Reward-Bench.
- 14.6% improvement on Multimodal Reward Bench.
- StableReinforce Algorithm: A novel reinforcement learning method that enhances the Reinforce++ approach by improving training loss stability, advantage estimation, and reward function design.
- Open-Source Resources: We provide the R1-Reward model, the R1-Reward RL training dataset, and inference code for IXC-Reward，MM-RLHF Reward and R1-Reward on the three benchmarks in Figure 1.
Citation
If you find it useful for your research and applications, please cite related papers/blogs using this BibTeX:
@article{zhang2025r1,
title={R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learning},
author={Zhang, Yi-Fan and Lu, Xingyu and Hu, Xiao and Fu, Chaoyou and Wen, Bin and Zhang, Tianke and Liu, Changyi and Jiang, Kaiyu and Chen, Kaibing and Tang, Kaiyu and others},
journal={arXiv preprint arXiv:2505.02835},
year={2025}
}
Related Projects
- MM-RLHF: The Next Step Forward in Multimodal LLM Alignment
- MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?
- MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMs
- Beyond LLaVA-HD: Diving into High-Resolution Large Multimodal Models
- VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction