WildSmoke: Ready-to-Use Dynamic 3D Smoke Assets from a Single Video in the Wild
Yuqiu Liu, Jialin Song, Manolis Savva, Wuyang Chen
2025-09-18
Summary
This paper presents a new method for creating realistic 3D smoke effects from everyday videos, and then letting users interact with and change those smoke simulations.
What's the problem?
Currently, creating 3D smoke realistically requires videos shot in very controlled settings, like a lab. Real-world videos, filmed outside of a lab, are much harder to work with because of things like messy backgrounds, unclear camera movements, and the difficulty of figuring out what the smoke actually looks like from different angles. The researchers identified three main hurdles: separating the smoke from the background, figuring out where the smoke particles start and how the camera is moving, and creating a consistent view of the smoke from multiple perspectives.
What's the solution?
The researchers developed a system that tackles these problems step-by-step. First, it isolates the smoke from the background. Then, it estimates where the smoke is located in 3D space and how the camera moved during the video. Finally, it uses this information to create a 3D model of the smoke that looks realistic from any viewpoint. They improved upon existing methods and their system produces higher quality smoke reconstructions in real-world videos.
Why it matters?
This work is important because it makes it much easier to create realistic smoke effects for movies, games, or other visual applications using footage anyone can record. It also allows artists to not just *create* the smoke, but to actually manipulate and edit it interactively, opening up new possibilities for visual effects and design.
Abstract
We propose a pipeline to extract and reconstruct dynamic 3D smoke assets from a single in-the-wild video, and further integrate interactive simulation for smoke design and editing. Recent developments in 3D vision have significantly improved reconstructing and rendering fluid dynamics, supporting realistic and temporally consistent view synthesis. However, current fluid reconstructions rely heavily on carefully controlled clean lab environments, whereas real-world videos captured in the wild are largely underexplored. We pinpoint three key challenges of reconstructing smoke in real-world videos and design targeted techniques, including smoke extraction with background removal, initialization of smoke particles and camera poses, and inferring multi-view videos. Our method not only outperforms previous reconstruction and generation methods with high-quality smoke reconstructions (+2.22 average PSNR on wild videos), but also enables diverse and realistic editing of fluid dynamics by simulating our smoke assets. We provide our models, data, and 4D smoke assets at [https://autumnyq.github.io/WildSmoke](https://autumnyq.github.io/WildSmoke).