T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generation
Kaiyue Sun, Rongyao Fang, Chengqi Duan, Xian Liu, Xihui Liu
2025-08-26
Summary
This paper introduces a new way to test how well artificial intelligence models can understand and follow instructions when creating images from text.
What's the problem?
Current methods for evaluating text-to-image AI aren't very good at checking if the AI actually *understands* the text, or if it's just making a pretty picture. The AI might get the words right, but miss the deeper meaning or logical connections within the request. This means we don't really know how well these models can reason.
What's the solution?
The researchers created a benchmark called T2I-ReasonBench, which tests four different types of reasoning skills: understanding sayings, designing images based on detailed descriptions, using knowledge about objects, and applying scientific knowledge. They then used this benchmark to test several different text-to-image models, looking at both how accurately the images matched the text's meaning and how good the images looked overall. They used a two-step process to evaluate both reasoning and image quality.
Why it matters?
This work is important because it gives us a better way to measure the intelligence of text-to-image AI. By understanding where these models struggle with reasoning, we can improve them and build AI that can truly understand and respond to our requests, not just generate visually appealing images.
Abstract
We propose T2I-ReasonBench, a benchmark evaluating reasoning capabilities of text-to-image (T2I) models. It consists of four dimensions: Idiom Interpretation, Textual Image Design, Entity-Reasoning and Scientific-Reasoning. We propose a two-stage evaluation protocol to assess the reasoning accuracy and image quality. We benchmark various T2I generation models, and provide comprehensive analysis on their performances.