MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splatting
Ruijie Zhu, Yanzhe Liang, Hanzhi Chang, Jiacheng Deng, Jiahao Lu, Wenfei Yang, Tianzhu Zhang, Yongdong Zhang
2024-10-13
Summary
This paper introduces MotionGS, a new method for improving the reconstruction of dynamic 3D scenes by using explicit motion guidance in the process of deformable 3D Gaussian splatting.
What's the problem?
Reconstructing dynamic scenes in 3D is a challenging task. Traditional methods often struggle because they don't adequately consider how objects move, leading to difficulties in optimizing the model and poorer performance. This lack of motion guidance can result in less accurate and realistic 3D models.
What's the solution?
To address these issues, the authors developed MotionGS, which incorporates explicit motion information to guide the deformation of 3D Gaussians. They introduced an optical flow decoupling module that separates camera movement from object movement, allowing for more precise control over how the 3D models adapt to motion. Additionally, they proposed a camera pose refinement module to improve the accuracy of camera positions during reconstruction. Their experiments showed that MotionGS significantly outperformed existing methods in creating detailed and accurate 3D representations of dynamic scenes.
Why it matters?
This research is important because it enhances the ability to create realistic 3D models of moving objects and environments, which is crucial for applications in virtual reality, gaming, and film production. By effectively incorporating motion guidance, MotionGS represents a significant step forward in the field of computer vision and dynamic scene reconstruction.
Abstract
Dynamic scene reconstruction is a long-term challenge in the field of 3D vision. Recently, the emergence of 3D Gaussian Splatting has provided new insights into this problem. Although subsequent efforts rapidly extend static 3D Gaussian to dynamic scenes, they often lack explicit constraints on object motion, leading to optimization difficulties and performance degradation. To address the above issues, we propose a novel deformable 3D Gaussian splatting framework called MotionGS, which explores explicit motion priors to guide the deformation of 3D Gaussians. Specifically, we first introduce an optical flow decoupling module that decouples optical flow into camera flow and motion flow, corresponding to camera movement and object motion respectively. Then the motion flow can effectively constrain the deformation of 3D Gaussians, thus simulating the motion of dynamic objects. Additionally, a camera pose refinement module is proposed to alternately optimize 3D Gaussians and camera poses, mitigating the impact of inaccurate camera poses. Extensive experiments in the monocular dynamic scenes validate that MotionGS surpasses state-of-the-art methods and exhibits significant superiority in both qualitative and quantitative results. Project page: https://ruijiezhu94.github.io/MotionGS_page