Training a Foundation Model for Materials on a Budget
Teddy Koker, Tess Smidt
2025-08-28
Summary
This paper introduces Nequix, a new computer model designed to predict the properties of materials. It's a type of 'foundation model,' meaning it's built to be generally useful for a wide range of materials science problems.
What's the problem?
Developing these powerful foundation models for materials is really expensive in terms of computing time and resources. Training them often requires a huge amount of processing power, making it difficult for many research labs to participate in this cutting-edge research. Essentially, the cost is a barrier to entry.
What's the solution?
The researchers created Nequix to be a more efficient model. They simplified the design compared to existing models and used clever training techniques, like a specific type of normalization and a faster optimization algorithm called Muon. They built it using a programming language called JAX, resulting in a model with fewer adjustable parameters and a significantly reduced training time – only 500 hours on powerful GPUs.
Why it matters?
Nequix achieves performance comparable to much larger and more expensive models, ranking highly on standard materials science benchmarks. Importantly, it does this with a fraction of the training cost and runs much faster when making predictions. This means more researchers can access and use these advanced modeling techniques, accelerating materials discovery and design.
Abstract
Foundation models for materials modeling are advancing quickly, but their training remains expensive, often placing state-of-the-art methods out of reach for many research groups. We introduce Nequix, a compact E(3)-equivariant potential that pairs a simplified NequIP design with modern training practices, including equivariant root-mean-square layer normalization and the Muon optimizer, to retain accuracy while substantially reducing compute requirements. Built in JAX, Nequix has 700K parameters and was trained in 500 A100-GPU hours. On the Matbench-Discovery and MDR Phonon benchmarks, Nequix ranks third overall while requiring less than one quarter of the training cost of most other methods, and it delivers an order-of-magnitude faster inference speed than the current top-ranked model. We release model weights and fully reproducible codebase at https://github.com/atomicarchitects/nequix