WalkTheDog: Cross-Morphology Motion Alignment via Phase Manifolds
Peizhuo Li, Sebastian Starke, Yuting Ye, Olga Sorkine-Hornung
2024-07-30
Summary
This paper introduces WalkTheDog, a new method for aligning the movements of different characters, like a human and a dog, using a special system called a phase manifold. This method helps in transferring motion from one character to another while keeping the style and timing intact.
What's the problem?
When animating characters with different body shapes (morphologies), it's challenging to transfer motion data accurately. Traditional methods often struggle because they don't effectively capture the unique movements of each character, leading to awkward animations. This makes it hard for animators to reuse motion data across different characters.
What's the solution?
To solve this problem, the authors developed WalkTheDog, which uses a phase manifold to represent and align the motions of different characters. The method learns a shared phase manifold that groups similar movements together, allowing for smooth transitions between characters. By using a technique called vector quantized periodic autoencoder, they can capture the essential timing and style of movements without needing any additional supervision. This means that even if the characters have different body shapes, their motions can still be aligned and transferred effectively.
Why it matters?
This research is important because it simplifies the process of animating characters with different shapes by allowing animators to easily transfer motion data between them. This can save time and resources in animation production, making it easier to create diverse animated content. Additionally, it opens up new possibilities for using motion capture data in innovative ways, enhancing the overall quality of animations.
Abstract
We present a new approach for understanding the periodicity structure and semantics of motion datasets, independently of the morphology and skeletal structure of characters. Unlike existing methods using an overly sparse high-dimensional latent, we propose a phase manifold consisting of multiple closed curves, each corresponding to a latent amplitude. With our proposed vector quantized periodic autoencoder, we learn a shared phase manifold for multiple characters, such as a human and a dog, without any supervision. This is achieved by exploiting the discrete structure and a shallow network as bottlenecks, such that semantically similar motions are clustered into the same curve of the manifold, and the motions within the same component are aligned temporally by the phase variable. In combination with an improved motion matching framework, we demonstrate the manifold's capability of timing and semantics alignment in several applications, including motion retrieval, transfer and stylization. Code and pre-trained models for this paper are available at https://peizhuoli.github.io/walkthedog.