Scene-Centric Unsupervised Panoptic Segmentation
Oliver Hahn, Christoph Reich, Nikita Araslanov, Daniel Cremers, Christian Rupprecht, Stefan Roth
2025-04-04
Summary
This paper talks about a new AI method (CUPS) that automatically labels images by separating objects and backgrounds without needing human-labeled examples, using depth and motion clues from videos or stereo images.
What's the problem?
Existing AI tools for detailed image labeling require tons of manually tagged data, which is slow and expensive to create, especially for complex scenes with multiple overlapping objects.
What's the solution?
CUPS uses depth and motion info from videos or stereo cameras to guess labels for objects and backgrounds, then trains itself using these guesses to get better at identifying things like cars or roads in new images.
Why it matters?
This helps create AI systems for self-driving cars or robots that understand their surroundings without needing costly human labeling, making the tech faster and cheaper to develop.
Abstract
Unsupervised panoptic segmentation aims to partition an image into semantically meaningful regions and distinct object instances without training on manually annotated data. In contrast to prior work on unsupervised panoptic scene understanding, we eliminate the need for object-centric training data, enabling the unsupervised understanding of complex scenes. To that end, we present the first unsupervised panoptic method that directly trains on scene-centric imagery. In particular, we propose an approach to obtain high-resolution panoptic pseudo labels on complex scene-centric data, combining visual representations, depth, and motion cues. Utilizing both pseudo-label training and a panoptic self-training strategy yields a novel approach that accurately predicts panoptic segmentation of complex scenes without requiring any human annotations. Our approach significantly improves panoptic quality, e.g., surpassing the recent state of the art in unsupervised panoptic segmentation on Cityscapes by 9.4% points in PQ.