Building Interactable Replicas of Complex Articulated Objects via Gaussian Splatting
Yu Liu, Baoxiong Jia, Ruijie Lu, Junfeng Ni, Song-Chun Zhu, Siyuan Huang
2025-02-28
Summary
This paper talks about ArtGS, a new method for creating 3D models of complex objects that can move and bend, like robots or animals. It uses a technique called Gaussian splatting to make more accurate and flexible models than previous methods.
What's the problem?
Current ways of making 3D models of objects with moving parts often struggle to accurately capture how all the parts fit together and move, especially for complicated objects with many parts. This makes it hard to create realistic and interactive 3D models of things like robots or animals.
What's the solution?
The researchers created ArtGS, which uses tiny 3D blobs called Gaussians to represent objects. They came up with clever ways to align these blobs across different poses of the object and to model how parts move together, kind of like how skin moves over bones. They also developed a step-by-step process to build up the model from rough to detailed.
Why it matters?
This matters because it could lead to much better 3D models of complex objects that move, which could be used in video games, virtual reality, or even to help design and test new robots or products. It makes it easier to create models that look realistic and move naturally, which could improve many areas of computer graphics and simulation.
Abstract
Building articulated objects is a key challenge in computer vision. Existing methods often fail to effectively integrate information across different object states, limiting the accuracy of part-mesh reconstruction and part dynamics modeling, particularly for complex multi-part articulated objects. We introduce ArtGS, a novel approach that leverages <PRE_TAG>3D Gaussians</POST_TAG> as a flexible and efficient representation to address these issues. Our method incorporates canonical Gaussians with <PRE_TAG>coarse-to-fine initialization</POST_TAG> and updates for aligning articulated part information across different object states, and employs a <PRE_TAG>skinning-inspired part dynamics modeling</POST_TAG> module to improve both part-mesh reconstruction and <PRE_TAG>articulation learning</POST_TAG>. Extensive experiments on both synthetic and real-world datasets, including a new benchmark for complex multi-part objects, demonstrate that ArtGS achieves state-of-the-art performance in <PRE_TAG>joint parameter estimation</POST_TAG> and part mesh reconstruction. Our approach significantly improves reconstruction quality and efficiency, especially for multi-part articulated objects. Additionally, we provide comprehensive analyses of our design choices, validating the effectiveness of each component to highlight potential areas for future improvement.