Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey
Yaoting Wang, Shengqiong Wu, Yuecheng Zhang, William Wang, Ziwei Liu, Jiebo Luo, Hao Fei
2025-03-18
Summary
This paper provides a comprehensive overview of Multimodal Chain-of-Thought (MCoT) reasoning, a method that extends the step-by-step reasoning process to AI systems that process multiple types of data like images, videos, and text.
What's the problem?
While MCoT reasoning is gaining popularity, there's a lack of a clear and up-to-date review of the field, making it difficult to understand the concepts, methodologies, and challenges involved.
What's the solution?
The paper presents a systematic survey of MCoT reasoning, explaining the foundational concepts, providing a comprehensive classification of current methodologies, and analyzing these methodologies across various applications. It also identifies existing challenges and suggests future research directions.
Why it matters?
This work matters because it provides a valuable resource for researchers and practitioners interested in MCoT reasoning, helping them understand the current state of the field and identify promising areas for future research, ultimately fostering innovation in multimodal AI.
Abstract
By extending the advantage of chain-of-thought (CoT) reasoning in human-like step-by-step processes to multimodal contexts, multimodal CoT (MCoT) reasoning has recently garnered significant research attention, especially in the integration with multimodal large language models (MLLMs). Existing MCoT studies design various methodologies and innovative reasoning paradigms to address the unique challenges of image, video, speech, audio, 3D, and structured data across different modalities, achieving extensive success in applications such as robotics, healthcare, autonomous driving, and multimodal generation. However, MCoT still presents distinct challenges and opportunities that require further focus to ensure consistent thriving in this field, where, unfortunately, an up-to-date review of this domain is lacking. To bridge this gap, we present the first systematic survey of MCoT reasoning, elucidating the relevant foundational concepts and definitions. We offer a comprehensive taxonomy and an in-depth analysis of current methodologies from diverse perspectives across various application scenarios. Furthermore, we provide insights into existing challenges and future research directions, aiming to foster innovation toward multimodal AGI.