POLARIS: Projection-Orthogonal Least Squares for Robust and Adaptive Inversion in Diffusion Models
Wenshuo Chen, Haosen Li, Shaofeng Liang, Lei Wang, Haozhe Jia, Kaishen Yuan, Jieming Wu, Bowen Tian, Yutao Yue
2025-12-02
Summary
This paper focuses on improving how diffusion models, which are really good at tasks like editing and fixing images, work when trying to recreate an original image from a noisy version. It identifies a hidden flaw in the process and offers a simple fix.
What's the problem?
Diffusion models work by gradually removing noise from an image. A key step involves estimating the noise at each stage, but the paper reveals that this estimation isn't perfect. The small errors in estimating the noise build up over time, leading to a degraded or inaccurate reconstruction of the original image. This 'noise error' is a significant problem that hasn't been fully addressed before.
What's the solution?
The researchers developed a new method called POLARIS. Instead of trying to correct for the errors *after* they accumulate, POLARIS focuses on minimizing the errors at *each individual step* of the noise removal process. It does this by treating a specific setting, called the guidance scale, as something that changes at each step, and uses a mathematical formula to make sure the noise is removed as accurately as possible. Surprisingly, this improvement only requires changing one line of code in existing diffusion models.
Why it matters?
This work is important because it makes diffusion models more reliable and accurate. By reducing the noise approximation errors, POLARIS leads to better quality reconstructions and improves the performance of tasks that rely on these models, like image editing and restoration, without adding significant computational cost.
Abstract
The Inversion-Denoising Paradigm, which is based on diffusion models, excels in diverse image editing and restoration tasks. We revisit its mechanism and reveal a critical, overlooked factor in reconstruction degradation: the approximate noise error. This error stems from approximating the noise at step t with the prediction at step t-1, resulting in severe error accumulation throughout the inversion process. We introduce Projection-Orthogonal Least Squares for Robust and Adaptive Inversion (POLARIS), which reformulates inversion from an error-compensation problem into an error-origin problem. Rather than optimizing embeddings or latent codes to offset accumulated drift, POLARIS treats the guidance scale ω as a step-wise variable and derives a mathematically grounded formula to minimize inversion error at each step. Remarkably, POLARIS improves inversion latent quality with just one line of code. With negligible performance overhead, it substantially mitigates noise approximation errors and consistently improves the accuracy of downstream tasks.