Key Features
Uses masked visual actions for unified robot world modeling.
Supports forward modeling from action representations to video.
Supports inverse modeling for recovering action-related structure.
Uses rendered robot control videos as visual conditioning signals.
Works with reference images and text prompts in generation workflows.
Evaluates unseen grippers and unseen embodiments.
Links to public code for finetuning and inference.
Includes direct MP4 videos showing robot control comparisons.
About Masked Visual Actions
The method uses rendered control videos, reference images, and prompts to generate corresponding real-world robot video behavior. Its project examples include DROID training-domain cases, unseen grippers, unseen embodiments, and comparisons with baseline control representations.
Masked Visual Actions is useful for robotics researchers, world-model developers, and teams building action-conditioned video prediction systems. It provides a practical bridge between robot control signals and generative video models by making action information visible and transferable.