NaTex: Seamless Texture Generation as Latent Color Diffusion
Zeqiang Lai, Yunfei Zhao, Zibo Zhao, Xin Yang, Xin Huang, Jingwei Huang, Xiangyu Yue, Chunchao Guo
2025-11-21
Summary
This paper introduces NaTex, a new system for creating textures on 3D models directly in 3D space, rather than relying on older methods that first create 2D images and then 'wrap' them onto the model.
What's the problem?
Existing methods for generating textures often struggle with several issues. When they create textures from multiple 2D views, they have trouble with areas hidden from some views, requiring 'inpainting' or guessing. It's also hard to get the texture to line up perfectly with the edges of the 3D model, and ensuring the texture looks consistent in color and detail from all angles is a challenge. Basically, these methods are indirect and introduce errors along the way.
What's the solution?
NaTex tackles these problems by thinking of texture as a dense cloud of colored points in 3D. They developed a system called 'latent color diffusion' which uses a special type of neural network, a diffusion transformer, to reconstruct and generate textures. This network is trained specifically on 3D data. Crucially, NaTex directly uses the 3D shape of the model to guide the texture creation, ensuring precise alignment and consistency. They also designed the system so that the parts responsible for understanding the 3D shape and the color work closely together, providing detailed guidance for the texture.
Why it matters?
NaTex is a significant improvement because it creates textures that are more coherent, meaning they look more natural and consistent across the entire model, and more accurately aligned with the model's shape. It also works well with different applications like creating new materials, improving existing textures, and even automatically adding textures to different parts of a 3D model based on what those parts are.
Abstract
We present NaTex, a native texture generation framework that predicts texture color directly in 3D space. In contrast to previous approaches that rely on baking 2D multi-view images synthesized by geometry-conditioned Multi-View Diffusion models (MVDs), NaTex avoids several inherent limitations of the MVD pipeline. These include difficulties in handling occluded regions that require inpainting, achieving precise mesh-texture alignment along boundaries, and maintaining cross-view consistency and coherence in both content and color intensity. NaTex features a novel paradigm that addresses the aforementioned issues by viewing texture as a dense color point cloud. Driven by this idea, we propose latent color diffusion, which comprises a geometry-awared color point cloud VAE and a multi-control diffusion transformer (DiT), entirely trained from scratch using 3D data, for texture reconstruction and generation. To enable precise alignment, we introduce native geometry control that conditions the DiT on direct 3D spatial information via positional embeddings and geometry latents. We co-design the VAE-DiT architecture, where the geometry latents are extracted via a dedicated geometry branch tightly coupled with the color VAE, providing fine-grained surface guidance that maintains strong correspondence with the texture. With these designs, NaTex demonstrates strong performance, significantly outperforming previous methods in texture coherence and alignment. Moreover, NaTex also exhibits strong generalization capabilities, either training-free or with simple tuning, for various downstream applications, e.g., material generation, texture refinement, and part segmentation and texturing.