SplineGS: Robust Motion-Adaptive Spline for Real-Time Dynamic 3D Gaussians from Monocular Video
Jongmin Park, Minh-Quan Viet Bui, Juan Luis Gonzalez Bello, Jaeho Moon, Jihyong Oh, Munchurl Kim
2024-12-17
Summary
This paper introduces SplineGS, a new method for creating high-quality 3D models from single videos, allowing for better and faster rendering of dynamic scenes without needing complex setups.
What's the problem?
Creating 3D models from videos is difficult because you usually need multiple camera angles to get a clear understanding of the scene. When using just one camera (monocular video), it's hard to capture all the details, especially when things are moving. Existing methods often require long processing times and can struggle with accuracy, especially in dynamic environments.
What's the solution?
SplineGS solves these problems by using a unique technique called Motion-Adaptive Spline (MAS) that represents moving objects as smooth curves. This method allows the system to efficiently create and render 3D models from single videos by simplifying the data it processes. It also includes a new way to optimize how the camera settings and 3D attributes are estimated, making it more robust in real-world situations. This means SplineGS can produce high-quality visuals quickly, even in complex scenes.
Why it matters?
This research is important because it makes it easier and faster to create realistic 3D models from everyday videos. This technology could be used in various fields like gaming, virtual reality, and film production, where high-quality visuals are essential. By improving how we can generate 3D content from simple video inputs, SplineGS opens up new possibilities for creativity and innovation in digital media.
Abstract
Synthesizing novel views from in-the-wild monocular videos is challenging due to scene dynamics and the lack of multi-view cues. To address this, we propose SplineGS, a COLMAP-free dynamic 3D Gaussian Splatting (3DGS) framework for high-quality reconstruction and fast rendering from monocular videos. At its core is a novel Motion-Adaptive Spline (MAS) method, which represents continuous dynamic 3D Gaussian trajectories using cubic Hermite splines with a small number of control points. For MAS, we introduce a Motion-Adaptive Control points Pruning (MACP) method to model the deformation of each dynamic 3D Gaussian across varying motions, progressively pruning control points while maintaining dynamic modeling integrity. Additionally, we present a joint optimization strategy for camera parameter estimation and 3D Gaussian attributes, leveraging photometric and geometric consistency. This eliminates the need for Structure-from-Motion preprocessing and enhances SplineGS's robustness in real-world conditions. Experiments show that SplineGS significantly outperforms state-of-the-art methods in novel view synthesis quality for dynamic scenes from monocular videos, achieving thousands times faster rendering speed.