Summary

UniAnimate-DiT is an open-source framework for human image animation from Alibaba, leveraging large-scale video diffusion transformers to generate visually striking and temporally consistent animations. Built on the Wan2.1-14B-I2V video diffusion model, it uses Low-Rank Adaptation (LoRA) for efficient fine-tuning, significantly reducing memory overhead while maintaining high generative performance. The system animates a single reference image based on a sequence of target poses, enabling the creation of lifelike motion that closely follows user-defined dynamics.

Key Features

Human image animation built on the Wan2.1-14B-I2V video diffusion transformer
Animates a single reference image from a sequence of target poses
Efficient LoRA-based fine-tuning with reduced memory overhead
Lightweight pose encoder with stacked 3D convolutional layers for accurate motion extraction
Feature-level integration of appearance and pose for enhanced alignment
Trained on 480p video and generalizes to 720p inference
Long video generation with an overlapped sliding window strategy
Open-source training and inference code

About UniAnimate-DiT


A key innovation is its ability to integrate both appearance and pose information through a simple yet effective concatenation and summing operation at the feature level, which enhances alignment between the reference image and the generated animation. Its modular architecture incorporates a lightweight pose encoder with stacked 3D convolutional layers, ensuring accurate extraction and representation of motion features from driving poses, and it incorporates the pose information of the reference image for enhanced pose alignment.


UniAnimate-DiT is trained on approximately 10,000 human dance videos at 480p resolution, enabling it to generalize across a wide range of actions, lighting conditions and backgrounds, and during inference it can seamlessly upscale its outputs to 720p. It supports long video generation through an overlapped sliding window strategy, making it suitable for character-driven storytelling, educational content and virtual avatars, with both training and inference code released openly.

Get more likes & reach the top of search results by adding this button on your site!

Embed button preview - Light theme
Embed button preview - Dark theme
TurboType Banner

Subscribe to the AI Search Newsletter

Get top updates in AI to your inbox every weekend. It's free!