Glance: Accelerating Diffusion Models with 1 Sample
Zhuobai Dong, Rui Zhao, Songjie Wu, Junchao Yi, Linjie Li, Zhengyuan Yang, Lijuan Wang, Alex Jinpeng Wang
2025-12-03
Summary
This paper focuses on making diffusion models, which are really good at creating images, faster and more practical to use.
What's the problem?
Diffusion models are powerful but slow because they require many steps to generate a single image, making them computationally expensive and time-consuming. Previous attempts to speed them up involved retraining the entire model, which is costly and can sometimes make the model worse at creating diverse images.
What's the solution?
The researchers came up with a way to speed up diffusion models without completely retraining them. They realized that some stages of the image creation process need more refinement than others. So, they created small 'helper' modules, called LoRA adapters, that focus on speeding up specific phases – one for the early, important stages and another for the later, less critical stages. Surprisingly, these helpers only needed to be trained with a tiny amount of data, just one image, and a short amount of time to work well.
Why it matters?
This work is important because it makes high-quality image generation more accessible. By significantly reducing the time it takes to create an image without sacrificing quality, it opens the door for wider use of diffusion models in applications where speed and efficiency are crucial, like real-time image editing or content creation.
Abstract
Diffusion models have achieved remarkable success in image generation, yet their deployment remains constrained by the heavy computational cost and the need for numerous inference steps. Previous efforts on fewer-step distillation attempt to skip redundant steps by training compact student models, yet they often suffer from heavy retraining costs and degraded generalization. In this work, we take a different perspective: we accelerate smartly, not evenly, applying smaller speedups to early semantic stages and larger ones to later redundant phases. We instantiate this phase-aware strategy with two experts that specialize in slow and fast denoising phases. Surprisingly, instead of investing massive effort in retraining student models, we find that simply equipping the base model with lightweight LoRA adapters achieves both efficient acceleration and strong generalization. We refer to these two adapters as Slow-LoRA and Fast-LoRA. Through extensive experiments, our method achieves up to 5 acceleration over the base model while maintaining comparable visual quality across diverse benchmarks. Remarkably, the LoRA experts are trained with only 1 samples on a single V100 within one hour, yet the resulting models generalize strongly on unseen prompts.