AnchorCrafter: Animate CyberAnchors Saling Your Products via Human-Object Interacting Video Generation
Ziyi Xu, Ziyao Huang, Juan Cao, Yong Zhang, Xiaodong Cun, Qing Shuai, Yuchen Wang, Linchao Bao, Jintao Li, Fan Tang
2024-11-27
Summary
This paper presents AnchorCrafter, a new system that automatically creates engaging product promotion videos by showing how people interact with products in a realistic way.
What's the problem?
Creating effective promotional videos for products can be difficult because existing methods struggle to accurately show how humans and objects interact. This lack of understanding can lead to videos that are not visually appealing or informative, which is a problem for online shopping and advertising.
What's the solution?
The authors developed AnchorCrafter, a system that uses advanced techniques to generate high-quality videos featuring a person and a product. It includes two main innovations: one improves how the system recognizes and depicts the appearance of objects from different angles, while the other allows for complex interactions between the person and the product. Additionally, they introduced a training method that helps the system learn more details about the objects being shown.
Why it matters?
This research is important because it enhances how products are marketed online. By creating more realistic and engaging promotional videos, businesses can better attract customers and improve their sales. This technology could transform online commerce by making product presentations more dynamic and appealing.
Abstract
The automatic generation of anchor-style product promotion videos presents promising opportunities in online commerce, advertising, and consumer engagement. However, this remains a challenging task despite significant advancements in pose-guided human video generation. In addressing this challenge, we identify the integration of human-object interactions (HOI) into pose-guided human video generation as a core issue. To this end, we introduce AnchorCrafter, a novel diffusion-based system designed to generate 2D videos featuring a target human and a customized object, achieving high visual fidelity and controllable interactions. Specifically, we propose two key innovations: the HOI-appearance perception, which enhances object appearance recognition from arbitrary multi-view perspectives and disentangles object and human appearance, and the HOI-motion injection, which enables complex human-object interactions by overcoming challenges in object trajectory conditioning and inter-occlusion management. Additionally, we introduce the HOI-region reweighting loss, a training objective that enhances the learning of object details. Extensive experiments demonstrate that our proposed system outperforms existing methods in preserving object appearance and shape awareness, while simultaneously maintaining consistency in human appearance and motion. Project page: https://cangcz.github.io/Anchor-Crafter/