MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation
Chenxi Wang, Xiang Chen, Ningyu Zhang, Bozhong Tian, Haoming Xu, Shumin Deng, Huajun Chen
2024-10-16
Summary
This paper discusses a new method called Dynamic Correction Decoding (DeCo) designed to reduce 'hallucinations' in Multimodal Large Language Models (MLLMs), which are instances where the models generate incorrect or nonsensical outputs.
What's the problem?
MLLMs often produce outputs that don't match the visual information they process, leading to hallucinations. This happens because the model's strong language knowledge can overshadow the visual details it recognizes, resulting in errors. Understanding why these hallucinations occur is challenging, and existing methods to fix them are not always effective.
What's the solution?
The authors propose DeCo, a method that improves how MLLMs generate outputs by selecting the most relevant information from earlier processing layers and adjusting the final output based on this information. This approach allows the model to better integrate visual and textual data, leading to more accurate results. DeCo can be used with various existing decoding strategies and works with different MLLMs without needing additional training.
Why it matters?
This research is important because it addresses a significant issue in AI models that combine text and images. By reducing hallucinations, DeCo can enhance the reliability of MLLMs in real-world applications, such as medical diagnostics or content generation, where accuracy is crucial. This advancement can lead to more trustworthy AI systems that better understand and represent complex information.
Abstract
Multimodal Large Language Models (MLLMs) frequently exhibit hallucination phenomena, but the underlying reasons remain poorly understood. In this paper, we present an empirical analysis and find that, although MLLMs incorrectly generate the objects in the final output, they are actually able to recognize visual objects in the preceding layers. We speculate that this may be due to the strong knowledge priors of the language model suppressing the visual information, leading to hallucinations. Motivated by this, we propose a novel dynamic correction decoding method for MLLMs (DeCo), which adaptively selects the appropriate preceding layers and proportionally integrates knowledge into the final layer to adjust the output logits. Note that DeCo is model agnostic and can be seamlessly incorporated with various classic decoding strategies and applied to different MLLMs. We evaluate DeCo on widely-used benchmarks, demonstrating that it can reduce hallucination rates by a large margin compared to baselines, highlighting its potential to mitigate hallucinations. Code is available at https://github.com/zjunlp/DeCo.