Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image
Yushi Hu, Reyhane Askari-Hemmat, Melissa Hall, Emily Dinan, Luke Zettlemoyer, Marjan Ghazvininejad
2025-12-19
Summary
This paper introduces a new way to test how well computer programs understand and judge both images and text together, which is important for building better AI systems that can handle both types of information.
What's the problem?
Currently, we have good ways to evaluate AI models that *only* work with text, or *only* work with images. However, it's much harder to evaluate models that can understand and generate content using both images and text at the same time, like creating images from text descriptions or having a conversation that includes images. There wasn't a good, comprehensive benchmark to measure how well these 'multimodal' AI models are doing, specifically when it comes to ranking different outputs based on quality.
What's the solution?
The researchers created a benchmark called Multimodal RewardBench 2 (MMRB2). This benchmark includes four different tasks – turning text into images, editing existing images, generating content that mixes images and text, and reasoning about images and text together. They asked human experts to compare pairs of AI-generated responses for each task and choose which one was better. This created a large dataset of 4,000 ranked examples. They then used this dataset to test how well different AI models, including the latest ones like Gemini 3 Pro and GPT-5, could predict which response humans would prefer.
Why it matters?
This work is important because it provides a standardized way to measure the progress of AI models that can handle both images and text. By identifying where current models struggle, it helps researchers focus on improving these models and building AI systems that are better at understanding and interacting with the world in a more human-like way. The benchmark also shows a strong link between how well a model performs on MMRB2 and how well it performs on actual tasks, making it a useful tool for developing more capable AI.
Abstract
Reward models (RMs) are essential for training large language models (LLMs), but remain underexplored for omni models that handle interleaved image and text sequences. We introduce Multimodal RewardBench 2 (MMRB2), the first comprehensive benchmark for reward models on multimodal understanding and (interleaved) generation. MMRB2 spans four tasks: text-to-image, image editing, interleaved generation, and multimodal reasoning ("thinking-with-images"), providing 1,000 expert-annotated preference pairs per task from 23 models and agents across 21 source tasks. MMRB2 is designed with: (1) practical but challenging prompts; (2) responses from state-of-the-art models and agents; and (3) preference pairs with strong human-expert consensus, curated via an ensemble filtering strategy. Using MMRB2, we study existing judges for each subtask, including multimodal LLM-as-a-judge and models trained with human preferences. The latest Gemini 3 Pro attains 75-80% accuracy. GPT-5 and Gemini 2.5 Pro reach 66-75% accuracy, compared to >90% for humans, yet surpass the widely used GPT-4o (59%). The best performing open-source model Qwen3-VL-32B achieves similar accuracies as Gemini 2.5 Flash (64%). We also show that MMRB2 performance strongly correlates with downstream task success using Best-of-N sampling and conduct an in-depth analysis that shows key areas to improve the reward models going forward.