ViDiC: Video Difference Captioning
Jiangtao Wu, Shihao Li, Zhaozhou Bian, Yuanxing Zhang, Jialu Chen, Runzhe Wen, An Ping, Yiwen He, Jiakai Wang, Jiaheng Liu
2025-12-04
Summary
This paper focuses on how well computers can understand and describe the differences between two videos, going beyond just identifying *what* changed to understanding *how* things changed over time.
What's the problem?
Current computer systems are good at describing what's different between two still images, but they struggle with videos because they don't fully grasp motion, how events unfold, or if edits to a video are consistent. They can't really 'compare' videos in a way that humans do, noticing subtle changes in style, movement, or even camera angles.
What's the solution?
The researchers created a new task called Video Difference Captioning (ViDiC) and a dataset called ViDiC-1K. This dataset contains 1,000 pairs of videos, each with detailed notes describing what's similar and different about them, covering things like the subject, style, motion, and location. They also developed a way to automatically check how well computer models are doing at identifying both similarities *and* differences, using another language model as a judge.
Why it matters?
This work is important because it provides a challenging test for computer systems to improve their understanding of videos. By focusing on comparative reasoning – understanding what’s changed and how – it pushes the field closer to creating AI that can truly 'watch' and understand videos like humans do, which is crucial for things like video editing, content analysis, and more advanced AI applications.
Abstract
Understanding visual differences between dynamic scenes requires the comparative perception of compositional, spatial, and temporal changes--a capability that remains underexplored in existing vision-language systems. While prior work on Image Difference Captioning (IDC) has enabled models to describe semantic changes between static images, these approaches fail to capture motion continuity, event evolution, or editing consistency over time. We introduce the ViDiC (Video Difference Captioning) task and its corresponding ViDiC-1K dataset, designed to evaluate the ability of Multimodal Large Language Models (MLLMs) to provide fine-grained descriptions of similarities and differences between video pairs. ViDiC-1K comprises 1,000 curated video pairs annotated with over 4,000 comparative checklist items, covering seven categories: subject, style, background, cinematography, motion, location, and playback techniques. To ensure reliable evaluation, we propose a dual-checklist framework that measures the accuracy of similarity and difference separately, based on the LLM-as-a-Judge protocol. Experiments on nineteen representative multimodal models reveal a significant performance gap in their comparative description and difference perception abilities. We hope ViDiC-1K can be a challenging benchmark that lays a solid foundation for advancing video understanding, edit awareness, and comparative reasoning in multimodal intelligence.