Image-Free Timestep Distillation via Continuous-Time Consistency with Trajectory-Sampled Pairs
Bao Tang, Shuai Zhang, Yueting Zhu, Jijun Xiang, Xin Yang, Li Yu, Wenyu Liu, Xinggang Wang
2025-11-27
Summary
This paper introduces a new method called Trajectory-Backward Consistency Model, or TBCM, to make diffusion models faster and more efficient at creating images.
What's the problem?
Current methods for speeding up diffusion models, which involve 'distilling' knowledge from a larger, slower model to a smaller, faster one, still require a lot of extra training data and computing power. This makes it hard to use these techniques when resources are limited or when you want to apply them to new types of images where you don't have a huge dataset already.
What's the solution?
TBCM solves this problem by cleverly extracting information directly from the way the original, high-quality model *generates* images. Instead of needing a separate dataset to train the faster model, it learns by observing the steps the original model takes. This 'self-contained' approach simplifies the process and reduces the need for resources. It's like learning to paint by watching a master artist, rather than needing a whole set of practice canvases.
Why it matters?
This research is important because it makes advanced image generation technology more accessible. By reducing the need for large datasets and powerful computers, TBCM opens the door to using these models in more places and for more applications. The paper also helps us understand why some of these speed-up methods work better than others, which can guide future research in this area, and it achieves impressive results in image quality and speed.
Abstract
Timestep distillation is an effective approach for improving the generation efficiency of diffusion models. The Consistency Model (CM), as a trajectory-based framework, demonstrates significant potential due to its strong theoretical foundation and high-quality few-step generation. Nevertheless, current continuous-time consistency distillation methods still rely heavily on training data and computational resources, hindering their deployment in resource-constrained scenarios and limiting their scalability to diverse domains. To address this issue, we propose Trajectory-Backward Consistency Model (TBCM), which eliminates the dependence on external training data by extracting latent representations directly from the teacher model's generation trajectory. Unlike conventional methods that require VAE encoding and large-scale datasets, our self-contained distillation paradigm significantly improves both efficiency and simplicity. Moreover, the trajectory-extracted samples naturally bridge the distribution gap between training and inference, thereby enabling more effective knowledge transfer. Empirically, TBCM achieves 6.52 FID and 28.08 CLIP scores on MJHQ-30k under one-step generation, while reducing training time by approximately 40% compared to Sana-Sprint and saving a substantial amount of GPU memory, demonstrating superior efficiency without sacrificing quality. We further reveal the diffusion-generation space discrepancy in continuous-time consistency distillation and analyze how sampling strategies affect distillation performance, offering insights for future distillation research. GitHub Link: https://github.com/hustvl/TBCM.