TurboEdit: Instant text-based image editing
Zongze Wu, Nicholas Kolkin, Jonathan Brandt, Richard Zhang, Eli Shechtman
2024-08-19
Summary
This paper presents TurboEdit, a new method for quickly and accurately editing images using text descriptions.
What's the problem?
Editing images precisely can be difficult and time-consuming, especially when trying to change specific details without affecting the entire image. Traditional methods often require complex processes that are not efficient.
What's the solution?
TurboEdit introduces an encoder-based technique that allows for iterative image editing. It uses a method called iterative inversion, which helps correct images step by step. By conditioning the editing process on detailed text prompts, users can easily change one specific attribute of an image while keeping everything else the same. This method is fast and requires only a few evaluations to achieve high-quality results, making it much quicker than previous techniques.
Why it matters?
This research is important because it simplifies the process of image editing, allowing users to make realistic changes in real-time. This can be useful in various fields such as graphic design, marketing, and social media, where quick and effective image manipulation is often needed.
Abstract
We address the challenges of precise image inversion and disentangled image editing in the context of few-step diffusion models. We introduce an encoder based iterative inversion technique. The inversion network is conditioned on the input image and the reconstructed image from the previous step, allowing for correction of the next reconstruction towards the input image. We demonstrate that disentangled controls can be easily achieved in the few-step diffusion model by conditioning on an (automatically generated) detailed text prompt. To manipulate the inverted image, we freeze the noise maps and modify one attribute in the text prompt (either manually or via instruction based editing driven by an LLM), resulting in the generation of a new image similar to the input image with only one attribute changed. It can further control the editing strength and accept instructive text prompt. Our approach facilitates realistic text-guided image edits in real-time, requiring only 8 number of functional evaluations (NFEs) in inversion (one-time cost) and 4 NFEs per edit. Our method is not only fast, but also significantly outperforms state-of-the-art multi-step diffusion editing techniques.