LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL
Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang, Xingzhong Xu, Xin Geng, Xu Yang
2025-03-12
Summary
This paper talks about LMM-R1, a method that boosts small AI models’ ability to solve problems using both images and text by first training them on text-only logic puzzles and then transferring those skills to handle pictures and words together.
What's the problem?
Small AI models (3B parameters) struggle to combine image understanding with logical thinking, and existing methods need lots of expensive training data mixing both text and visuals.
What's the solution?
LMM-R1 uses a two-step training: first, it sharpens the AI’s reasoning skills using text-only questions and rule-based rewards, then it applies those skills to image-text tasks without needing tons of new data.
Why it matters?
This helps smaller AI models work better in apps like visual assistants or medical imaging by improving their reasoning without requiring huge datasets or supercomputers.
Abstract
Enhancing reasoning in Large Multimodal Models (LMMs) faces unique challenges from the complex interplay between visual perception and logical reasoning, particularly in compact 3B-parameter architectures where architectural constraints limit reasoning capacity and modality alignment. While rule-based reinforcement learning (RL) excels in text-only domains, its multimodal extension confronts two critical barriers: (1) data limitations due to ambiguous answers and scarce complex reasoning examples, and (2) degraded foundational reasoning induced by multimodal pretraining. To address these challenges, we propose \method, a two-stage framework adapting rule-based RL for multimodal reasoning through Foundational Reasoning Enhancement (FRE) followed by Multimodal Generalization Training (MGT). The FRE stage first strengthens reasoning abilities using text-only data with rule-based RL, then the MGT stage generalizes these reasoning capabilities to multimodal domains. Experiments on Qwen2.5-VL-Instruct-3B demonstrate that \method achieves 4.83\% and 4.5\% average improvements over baselines in multimodal and text-only benchmarks, respectively, with a 3.63\% gain in complex Football Game tasks. These results validate that text-based reasoning enhancement enables effective multimodal generalization, offering a data-efficient paradigm that bypasses costly high-quality multimodal training data.