RadarGen: Automotive Radar Point Cloud Generation from Cameras
Tomer Borreda, Fangqiang Ding, Sanja Fidler, Shengyu Huang, Or Litany
2025-12-22
Summary
This paper introduces RadarGen, a new computer program that creates realistic radar data – the kind of information cars use to 'see' around them – from regular camera images.
What's the problem?
Currently, it's hard to get enough real-world radar data to train self-driving car systems effectively. Real radar data is expensive and difficult to collect in all the situations a car might encounter. Simulating radar data is also tricky because it needs to accurately reflect how radar actually works, including things like how objects bounce radar signals back (radar cross section) and how fast they're moving (Doppler effect). Existing simulations often don't quite match up with what real radar sensors pick up, which makes it harder to build reliable self-driving features.
What's the solution?
RadarGen solves this by using a type of artificial intelligence called a diffusion model. It takes images from multiple cameras and translates them into a 'bird's-eye view' map representing what the radar would 'see'. This map includes information about the location of objects, how strongly they reflect radar, and their speed. The program is 'guided' by information from other AI models that understand depth, what objects *are* (like cars or pedestrians), and how they're moving, ensuring the generated radar data is physically realistic. Finally, it converts this map back into a standard radar point cloud format.
Why it matters?
This work is important because it offers a way to create large amounts of realistic radar data without needing to physically collect it. This can significantly speed up the development and testing of self-driving car technology. Because it starts with camera images, it can easily work with existing datasets and simulations, making it a scalable solution for improving the accuracy and safety of autonomous vehicles and bridging the gap between simulated and real-world perception.
Abstract
We present RadarGen, a diffusion model for synthesizing realistic automotive radar point clouds from multi-view camera imagery. RadarGen adapts efficient image-latent diffusion to the radar domain by representing radar measurements in bird's-eye-view form that encodes spatial structure together with radar cross section (RCS) and Doppler attributes. A lightweight recovery step reconstructs point clouds from the generated maps. To better align generation with the visual scene, RadarGen incorporates BEV-aligned depth, semantic, and motion cues extracted from pretrained foundation models, which guide the stochastic generation process toward physically plausible radar patterns. Conditioning on images makes the approach broadly compatible, in principle, with existing visual datasets and simulation frameworks, offering a scalable direction for multimodal generative simulation. Evaluations on large-scale driving data show that RadarGen captures characteristic radar measurement distributions and reduces the gap to perception models trained on real data, marking a step toward unified generative simulation across sensing modalities.