Consistency Flow Matching: Defining Straight Flows with Velocity Consistency
Ling Yang, Zixiang Zhang, Zhilong Zhang, Xingchao Liu, Minkai Xu, Wentao Zhang, Chenlin Meng, Stefano Ermon, Bin Cui
2024-07-03
Summary
This paper talks about a new method called Consistency Flow Matching (Consistency-FM) that improves how we generate samples from noise by creating straight paths in data transformations, making the process faster and more efficient.
What's the problem?
The main problem is that existing methods for transforming noise into useful data often require a lot of calculations and can be slow. These methods struggle to create high-quality samples efficiently, especially when dealing with complex data distributions.
What's the solution?
To solve this, the authors developed Consistency-FM, which focuses on creating straight paths for data transformation by ensuring that the speed of the transformation remains consistent. This method allows for better control over how data flows from noise to a final sample. They also introduced a multi-segment training approach that helps the model learn more effectively and balance speed with sample quality. Their experiments showed that Consistency-FM is significantly faster, converging 4.4 times quicker than previous models while producing better results.
Why it matters?
This research is important because it enhances the efficiency of generating high-quality data from noise, which is crucial in fields like artificial intelligence and machine learning. By improving both speed and quality, Consistency-FM can lead to better performance in applications such as image generation and other creative tasks.
Abstract
Flow matching (FM) is a general framework for defining probability paths via Ordinary Differential Equations (ODEs) to transform between noise and data samples. Recent approaches attempt to straighten these flow trajectories to generate high-quality samples with fewer function evaluations, typically through iterative rectification methods or optimal transport solutions. In this paper, we introduce Consistency Flow Matching (Consistency-FM), a novel FM method that explicitly enforces self-consistency in the velocity field. Consistency-FM directly defines straight flows starting from different times to the same endpoint, imposing constraints on their velocity values. Additionally, we propose a multi-segment training approach for Consistency-FM to enhance expressiveness, achieving a better trade-off between sampling quality and speed. Preliminary experiments demonstrate that our Consistency-FM significantly improves training efficiency by converging 4.4x faster than consistency models and 1.7x faster than rectified flow models while achieving better generation quality. Our code is available at: https://github.com/YangLing0818/consistency_flow_matching