TAG:Tangential Amplifying Guidance for Hallucination-Resistant Diffusion Sampling
Hyunmin Cho, Donghoon Ahn, Susung Hong, Jee Eun Kim, Seungryong Kim, Kyong Hwan Jin
2025-10-13
Summary
This paper introduces a new technique called Tangential Amplifying Guidance, or TAG, to improve the quality of images created by diffusion models.
What's the problem?
Diffusion models are really good at making images, but sometimes they create images that don't quite make sense or include things that aren't supposed to be there – these are called semantic inconsistencies or hallucinations. Existing methods to fix this often require extra work, like adding more calculations or changing how the model itself is built, which slows things down.
What's the solution?
TAG works by subtly adjusting the image creation process *during* generation, without changing the core diffusion model. It looks at a partially created image and figures out how to nudge the process in a direction that makes the final image more realistic and consistent. It does this by focusing on the 'tangential' parts of the image creation signal, essentially amplifying the details that will lead to a better result, using a mathematical idea called a Taylor expansion to ensure it's moving towards more likely outcomes.
Why it matters?
TAG is important because it's a simple and efficient way to improve the quality of images made by diffusion models. It doesn't require a lot of extra computing power or changes to the model itself, making it easy to use with different types of diffusion models and offering a new way to think about guiding these powerful image generators.
Abstract
Recent diffusion models achieve the state-of-the-art performance in image generation, but often suffer from semantic inconsistencies or hallucinations. While various inference-time guidance methods can enhance generation, they often operate indirectly by relying on external signals or architectural modifications, which introduces additional computational overhead. In this paper, we propose Tangential Amplifying Guidance (TAG), a more efficient and direct guidance method that operates solely on trajectory signals without modifying the underlying diffusion model. TAG leverages an intermediate sample as a projection basis and amplifies the tangential components of the estimated scores with respect to this basis to correct the sampling trajectory. We formalize this guidance process by leveraging a first-order Taylor expansion, which demonstrates that amplifying the tangential component steers the state toward higher-probability regions, thereby reducing inconsistencies and enhancing sample quality. TAG is a plug-and-play, architecture-agnostic module that improves diffusion sampling fidelity with minimal computational addition, offering a new perspective on diffusion guidance.