FlexiTex: Enhancing Texture Generation with Visual Guidance
DaDong Jiang, Xianghui Yang, Zibo Zhao, Sheng Zhang, Jiaao Yu, Zeqiang Lai, Shaoxiong Yang, Chunchao Guo, Xiaobo Zhou, Zhihui Ke
2024-09-20
Summary
This paper introduces FlexiTex, a new method for generating high-quality textures for images and 3D models by using visual guidance. It aims to improve the clarity and consistency of textures compared to previous methods that relied solely on abstract text descriptions.
What's the problem?
The main problem with existing texture generation methods is that they often produce blurry or inconsistent patterns because they depend on vague text prompts. These prompts do not provide enough detailed information about the textures or shapes needed, leading to poor results in texture quality.
What's the solution?
FlexiTex addresses this issue by incorporating a Visual Guidance Enhancement module, which uses specific visual information to clarify the text prompts and maintain fine details in the textures. Additionally, it features a Direction-Aware Adaptation module that creates direction prompts based on different camera angles, ensuring that the generated textures remain consistent and semantically correct across various views.
Why it matters?
This advancement is significant because it allows for more realistic and visually appealing textures in various applications, such as video games, movies, and virtual reality. By improving texture generation, FlexiTex can enhance the overall quality of digital content, making it more engaging and immersive for users.
Abstract
Recent texture generation methods achieve impressive results due to the powerful generative prior they leverage from large-scale text-to-image diffusion models. However, abstract textual prompts are limited in providing global textural or shape information, which results in the texture generation methods producing blurry or inconsistent patterns. To tackle this, we present FlexiTex, embedding rich information via visual guidance to generate a high-quality texture. The core of FlexiTex is the Visual Guidance Enhancement module, which incorporates more specific information from visual guidance to reduce ambiguity in the text prompt and preserve high-frequency details. To further enhance the visual guidance, we introduce a Direction-Aware Adaptation module that automatically designs direction prompts based on different camera poses, avoiding the Janus problem and maintaining semantically global consistency. Benefiting from the visual guidance, FlexiTex produces quantitatively and qualitatively sound results, demonstrating its potential to advance texture generation for real-world applications.