OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting
Yongsheng Yu, Ziyun Zeng, Haitian Zheng, Jiebo Luo
2025-03-14
Summary
This paper talks about OmniPaint, an AI tool that edits photos by removing or adding objects realistically, like erasing a person from a picture or inserting a new item while keeping the scene natural-looking.
What's the problem?
Current AI editors struggle to remove objects without leaving weird gaps or add new ones that look fake, especially when dealing with complex lighting and textures.
What's the solution?
OmniPaint uses two specialized AI models: one focuses on cleaning up backgrounds after removal, and the other handles blending new objects naturally, trained with a mix of real examples and AI-generated data to improve accuracy.
Why it matters?
This makes photo editing tools more powerful for tasks like fixing old photos, creating ads, or designing visuals without needing professional skills or expensive software.
Abstract
Diffusion-based generative models have revolutionized object-oriented image editing, yet their deployment in realistic object removal and insertion remains hampered by challenges such as the intricate interplay of physical effects and insufficient paired training data. In this work, we introduce OmniPaint, a unified framework that re-conceptualizes object removal and insertion as interdependent processes rather than isolated tasks. Leveraging a pre-trained diffusion prior along with a progressive training pipeline comprising initial paired sample optimization and subsequent large-scale unpaired refinement via CycleFlow, OmniPaint achieves precise foreground elimination and seamless object insertion while faithfully preserving scene geometry and intrinsic properties. Furthermore, our novel CFD metric offers a robust, reference-free evaluation of context consistency and object hallucination, establishing a new benchmark for high-fidelity image editing. Project page: https://yeates.github.io/OmniPaint-Page/