PhysicsGen: Can Generative Models Learn from Images to Predict Complex Physical Relations?
Martin Spitznagel, Jan Vaillant, Janis Keuper
2025-03-13
Summary
This paper talks about PhysicsGen, a test to see if AI image-generating tools can learn real-world physics from pictures, like predicting how sound travels through a city or how light bends through a lens.
What's the problem?
AI models that create images are great at making things look real but struggle to predict actual physics correctly, especially for complicated stuff like moving objects or sound waves.
What's the solution?
PhysicsGen uses 300,000 image pairs showing physics problems (like sound or light behavior) to train and test AI models, finding they’re fast but often wrong about the physics.
Why it matters?
This helps scientists build better AI for simulations in areas like material design or medicine, where speed matters but accuracy is critical to avoid dangerous mistakes.
Abstract
The image-to-image translation abilities of generative learning models have recently made significant progress in the estimation of complex (steered) mappings between image distributions. While appearance based tasks like image in-painting or style transfer have been studied at length, we propose to investigate the potential of generative models in the context of physical simulations. Providing a dataset of 300k image-pairs and baseline evaluations for three different physical simulation tasks, we propose a benchmark to investigate the following research questions: i) are generative models able to learn complex physical relations from input-output image pairs? ii) what speedups can be achieved by replacing differential equation based simulations? While baseline evaluations of different current models show the potential for high speedups (ii), these results also show strong limitations toward the physical correctness (i). This underlines the need for new methods to enforce physical correctness. Data, baseline models and evaluation code http://www.physics-gen.org.