✦
Summary
VACE is an all-in-one video creation and editing framework from Alibaba's Tongyi Lab that unifies multiple video synthesis and editing tasks into a single model. It supports text-to-video generation, reference-to-video generation, video-to-video editing and masked video-to-video editing, integrating diverse inputs such as text, images, videos and masks into a unified Video Condition Unit (VCU). The model is built on a Diffusion Transformer architecture enhanced with a Context Adapter structure, allowing it to adjust generation strategies based on task requirements and produce high-quality, temporally consistent video.
Key Features
Unified Video Condition Unit (VCU) integrating text, image, video and mask inputs
Text-to-video, reference-to-video and image-to-video generation in a single model
Video-to-video editing with pose, depth, structure and color controls
Masked video-to-video editing for inpainting, subject replacement and canvas extension
Concept Decoupling for independent editing of characters, backgrounds and actions
Free composition of tasks, including Move-Anything, Swap-Anything, Expand-Anything and Animate-Anything
Context Adapter structure that dynamically adjusts the generation strategy per task
Open-source models in 1.3B and 14B sizes built on Wan2.1 and LTX-Video
About VACE
One of VACE's key innovations is its Concept Decoupling strategy, which separates video elements like characters, backgrounds and actions so they can be modified independently without disrupting overall scene coherence. This enables targeted edits such as swapping subjects, changing motion trajectories or extending video frames with intelligent content filling, and its modular design supports compositional task combinations for complex scenarios such as long video re-rendering or multi-condition editing.
VACE is suited to social media content creation, film post-production, advertising, education and interactive media, offering rapid generation of short videos from text or reference images as well as fine-grained local edits using spatiotemporal masks. It was accepted at ICCV 2025 and is open-sourced with inference code, preprocessing tools and Gradio demos, with 1.3B and 14B model releases available on Hugging Face, GitHub and ModelScope.
