FlowOpt: Fast Optimization Through Whole Flow Processes for Training-Free Editing
Or Ronai, Vladimir Kulikov, Tomer Michaeli
2025-10-28
Summary
This paper introduces a new method called FlowOpt for controlling how images are created by advanced AI models like diffusion and flow-matching models. These models are great at generating images, but it's usually hard to precisely tell them *what* to create during the generation process.
What's the problem?
Current methods for controlling these image-generating models focus on tweaking the image step-by-step as it's being made. This is inefficient and doesn't allow for optimizing the final image as a whole. Directly controlling the final image with traditional optimization techniques is too slow because of how these models work – they build the image gradually, and you can't easily calculate how changes at the end affect the beginning.
What's the solution?
FlowOpt treats the entire image generation process as a 'black box'. Instead of looking at each step individually, it adjusts the starting point of the process to steer the final image towards the desired result. It doesn't need to understand the inner workings of the AI model, making it faster and more flexible. The researchers also figured out a way to determine how much to adjust the starting point at each step to ensure the process converges to a good solution, and allows for stopping the process early if the result looks good enough.
Why it matters?
FlowOpt is important because it makes it much easier to edit and personalize images generated by these powerful AI models. It achieves better results than existing methods while using a similar amount of computing power, opening the door for more precise and efficient image manipulation for tasks like editing, restoring, and creating images based on text descriptions.
Abstract
The remarkable success of diffusion and flow-matching models has ignited a surge of works on adapting them at test time for controlled generation tasks. Examples range from image editing to restoration, compression and personalization. However, due to the iterative nature of the sampling process in those models, it is computationally impractical to use gradient-based optimization to directly control the image generated at the end of the process. As a result, existing methods typically resort to manipulating each timestep separately. Here we introduce FlowOpt - a zero-order (gradient-free) optimization framework that treats the entire flow process as a black box, enabling optimization through the whole sampling path without backpropagation through the model. Our method is both highly efficient and allows users to monitor the intermediate optimization results and perform early stopping if desired. We prove a sufficient condition on FlowOpt's step-size, under which convergence to the global optimum is guaranteed. We further show how to empirically estimate this upper bound so as to choose an appropriate step-size. We demonstrate how FlowOpt can be used for image editing, showcasing two options: (i) inversion (determining the initial noise that generates a given image), and (ii) directly steering the edited image to be similar to the source image while conforming to a target text prompt. In both cases, FlowOpt achieves state-of-the-art results while using roughly the same number of neural function evaluations (NFEs) as existing methods. Code and examples are available on the project's webpage.