Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment
Ziang Yan, Zhilin Li, Yinan He, Chenting Wang, Kunchang Li, Xinhao Li, Xiangyu Zeng, Zilei Wang, Yali Wang, Yu Qiao, Limin Wang, Yi Wang
2024-12-30
Summary
This paper talks about Task Preference Optimization (TPO), a new method designed to improve how multimodal large language models (MLLMs) understand and perform visual tasks by aligning them with specific preferences.
What's the problem?
Current multimodal large language models struggle to accurately understand fine details in images and videos, even though they can handle a wide range of visual applications. Many existing approaches either focus on specific tasks or use tools that don’t fully optimize the model's overall performance. This can lead to limitations in how well these models can interpret and generate content based on visual information.
What's the solution?
To address these issues, the authors propose TPO, which uses learnable task tokens to connect different visual tasks with the MLLM. By training the model with rich visual labels and allowing it to learn from multiple tasks simultaneously, TPO enhances the model's ability to understand and generate responses related to various visual inputs. The authors found that this method led to a 14.6% improvement in performance across different benchmarks compared to previous models, making it more effective at handling complex visual tasks.
Why it matters?
This research is important because it pushes the boundaries of how AI can understand and interact with visual data. By improving the capabilities of multimodal models, TPO can lead to better applications in fields like robotics, healthcare, and entertainment, where precise visual understanding is crucial for effective decision-making and user interaction.
Abstract
Current multimodal large language models (MLLMs) struggle with fine-grained or precise understanding of visuals though they give comprehensive perception and reasoning in a spectrum of vision applications. Recent studies either develop tool-using or unify specific visual tasks into the autoregressive framework, often at the expense of overall multimodal performance. To address this issue and enhance MLLMs with visual tasks in a scalable fashion, we propose Task Preference Optimization (TPO), a novel method that utilizes differentiable task preferences derived from typical fine-grained visual tasks. TPO introduces learnable task tokens that establish connections between multiple task-specific heads and the MLLM. By leveraging rich visual labels during training, TPO significantly enhances the MLLM's multimodal capabilities and task-specific performance. Through multi-task co-training within TPO, we observe synergistic benefits that elevate individual task performance beyond what is achievable through single-task training methodologies. Our instantiation of this approach with VideoChat and LLaVA demonstrates an overall 14.6% improvement in multimodal performance compared to baseline models. Additionally, MLLM-TPO demonstrates robust zero-shot capabilities across various tasks, performing comparably to state-of-the-art supervised models. The code will be released at https://github.com/OpenGVLab/TPO