Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass
Jianing Yang, Alexander Sax, Kevin J. Liang, Mikael Henaff, Hao Tang, Ang Cao, Joyce Chai, Franziska Meier, Matt Feiszli
2025-01-23
Summary
This paper talks about Fast3R, a new method for creating 3D models from lots of 2D images quickly and accurately. It's like having a super-fast computer program that can look at hundreds of photos of an object from different angles and turn them into a detailed 3D model in one go.
What's the problem?
Creating 3D models from multiple 2D images is really important for things like virtual reality and robotics, but it's tough to do quickly and accurately. Current methods, like one called DUSt3R, look at images two at a time and then try to piece everything together, which takes a long time and can lead to mistakes, especially when dealing with lots of images. It's like trying to build a puzzle by only looking at two pieces at a time – it's slow and you might put things in the wrong place.
What's the solution?
The researchers created Fast3R, which uses a clever AI technique called a Transformer to look at all the images at once, instead of just two at a time. This is like being able to see all the puzzle pieces laid out together. Fast3R can process over 1000 images in one go, which is much faster than older methods. They tested Fast3R on different tasks, like figuring out where cameras were positioned when taking photos and creating 3D models, and found that it worked better and faster than other methods.
Why it matters?
This matters because it could make creating 3D models much faster and more accurate, which is super important for lots of cool technology. Imagine being able to quickly make detailed 3D models for video games, or helping robots understand their surroundings better. It could also be really useful for things like preserving historical sites digitally or creating virtual reality experiences. By making the process faster and more accurate, Fast3R could help push forward all sorts of technologies that rely on 3D models, making them better and more realistic.
Abstract
Multi-view 3D reconstruction remains a core challenge in computer vision, particularly in applications requiring accurate and scalable representations across diverse perspectives. Current leading methods such as DUSt3R employ a fundamentally pairwise approach, processing images in pairs and necessitating costly global alignment procedures to reconstruct from multiple views. In this work, we propose Fast 3D Reconstruction (Fast3R), a novel multi-view generalization to DUSt3R that achieves efficient and scalable 3D reconstruction by processing many views in parallel. Fast3R's Transformer-based architecture forwards N images in a single forward pass, bypassing the need for iterative alignment. Through extensive experiments on camera pose estimation and 3D reconstruction, Fast3R demonstrates state-of-the-art performance, with significant improvements in inference speed and reduced error accumulation. These results establish Fast3R as a robust alternative for multi-view applications, offering enhanced scalability without compromising reconstruction accuracy.