COREA: Coarse-to-Fine 3D Representation Alignment Between Relightable 3D Gaussians and SDF via Bidirectional 3D-to-3D Supervision
Jaeyoon Lee, Hojoon Jung, Sungtae Hwang, Jihyong Oh, Jongwon Choi
2025-12-10
Summary
This paper introduces COREA, a new system for creating realistic 3D models from images, focusing on both the shape of the object and how light interacts with its surface.
What's the problem?
Current methods for creating 3D models from images often struggle with accurately capturing fine details and realistically simulating how light bounces off the object. They typically learn the 3D shape indirectly from 2D pictures, which can lead to blurry surfaces and inaccurate lighting effects. Essentially, they don't directly 'understand' the 3D geometry in a way that supports high-quality rendering.
What's the solution?
COREA tackles this by learning both the 3D shape and lighting information *together* in 3D space. It uses two main tools: 3D Gaussians (which are like tiny, blurry 3D points that build up the shape) and a Signed Distance Field (which defines the surface of the object mathematically). The system first roughly aligns these two tools using depth information, then refines the details using depth gradients and surface normals. A clever 'density control' keeps the model from becoming too complex and memory-intensive.
Why it matters?
This research is important because it allows for the creation of more realistic and detailed 3D models from images. This has implications for a lot of fields, including virtual reality, augmented reality, special effects in movies, and even robotics where robots need to 'see' and understand the 3D world around them. By combining shape and lighting in a single system, COREA produces better results than existing methods.
Abstract
We present COREA, the first unified framework that jointly learns relightable 3D Gaussians and a Signed Distance Field (SDF) for accurate geometry reconstruction and faithful relighting. While recent 3D Gaussian Splatting (3DGS) methods have extended toward mesh reconstruction and physically-based rendering (PBR), their geometry is still learned from 2D renderings, leading to coarse surfaces and unreliable BRDF-lighting decomposition. To address these limitations, COREA introduces a coarse-to-fine bidirectional 3D-to-3D alignment strategy that allows geometric signals to be learned directly in 3D space. Within this strategy, depth provides coarse alignment between the two representations, while depth gradients and normals refine fine-scale structure, and the resulting geometry supports stable BRDF-lighting decomposition. A density-control mechanism further stabilizes Gaussian growth, balancing geometric fidelity with memory efficiency. Experiments on standard benchmarks demonstrate that COREA achieves superior performance in novel-view synthesis, mesh reconstruction, and PBR within a unified framework.