Frequency-Adaptive Sharpness Regularization for Improving 3D Gaussian Splatting Generalization
Youngsik Yun, Dongjun Gu, Youngjung Uh
2025-11-27
Summary
This paper focuses on improving 3D Gaussian Splatting, a technique for creating 3D scenes from images, specifically when you only have a few images to work with.
What's the problem?
3D Gaussian Splatting is really good at recreating scenes from the images it's trained on, but it struggles to create new views of the scene that weren't in the original images. This happens because it essentially memorizes the training images instead of learning a general understanding of the 3D structure, which is called overfitting. When you try to show it a new angle, it doesn't know what to do and the results look bad.
What's the solution?
The researchers came up with a new way to train 3D Gaussian Splatting called Frequency-Adaptive Sharpness Regularization, or FASR. It's like adding a little 'hint' during training that encourages the system to find a solution that isn't too specific to the original images. Existing methods like Sharpness-Aware Minimization tried this, but they ended up smoothing out important details. FASR is smarter; it adjusts how much 'hint' it gives based on the complexity of the image, preserving fine details while still improving how well it generalizes to new viewpoints.
Why it matters?
This work is important because it makes 3D scene reconstruction more practical. If you only have a few pictures of something, you can now create a much more realistic and complete 3D model that looks good from any angle. This has applications in things like virtual reality, robotics, and creating 3D content more easily.
Abstract
Despite 3D Gaussian Splatting (3DGS) excelling in most configurations, it lacks generalization across novel viewpoints in a few-shot scenario because it overfits to the sparse observations. We revisit 3DGS optimization from a machine learning perspective, framing novel view synthesis as a generalization problem to unseen viewpoints-an underexplored direction. We propose Frequency-Adaptive Sharpness Regularization (FASR), which reformulates the 3DGS training objective, thereby guiding 3DGS to converge toward a better generalization solution. Although Sharpness-Aware Minimization (SAM) similarly reduces the sharpness of the loss landscape to improve generalization of classification models, directly employing it to 3DGS is suboptimal due to the discrepancy between the tasks. Specifically, it hinders reconstructing high-frequency details due to excessive regularization, while reducing its strength leads to under-penalizing sharpness. To address this, we reflect the local frequency of images to set the regularization weight and the neighborhood radius when estimating the local sharpness. It prevents floater artifacts in novel viewpoints and reconstructs fine details that SAM tends to oversmooth. Across datasets with various configurations, our method consistently improves a wide range of baselines. Code will be available at https://bbangsik13.github.io/FASR.