Illustrious: an Open Advanced Illustration Model
Sang Hyun Park, Jun Young Koh, Junha Lee, Joy Song, Dongha Kim, Hoyeon Moon, Hyunju Lee, Min Song
2024-10-02
Summary
This paper discusses Illustrious, an advanced text-to-image model specifically designed for creating high-quality anime images from text descriptions.
What's the problem?
Creating detailed and visually appealing anime images from text is challenging. Existing models often struggle with low resolution, limited color range, and poor representation of character details, which can lead to unsatisfactory results.
What's the solution?
To overcome these challenges, the authors focused on three key improvements: optimizing batch size and dropout control for faster learning, increasing the training resolution to enhance image detail (allowing generation of images over 20 megapixels), and using refined multi-level captions to provide better guidance for image generation. Through extensive testing and analysis, Illustrious demonstrated superior performance compared to other models in generating anime-style illustrations.
Why it matters?
This research is significant because it pushes the boundaries of what AI can achieve in generating high-quality anime images. By improving the capabilities of text-to-image models, Illustrious can be used in various applications such as animation, gaming, and digital art, making it easier for creators to bring their ideas to life.
Abstract
In this work, we share the insights for achieving state-of-the-art quality in our text-to-image anime image generative model, called Illustrious. To achieve high resolution, dynamic color range images, and high restoration ability, we focus on three critical approaches for model improvement. First, we delve into the significance of the batch size and dropout control, which enables faster learning of controllable token based concept activations. Second, we increase the training resolution of images, affecting the accurate depiction of character anatomy in much higher resolution, extending its generation capability over 20MP with proper methods. Finally, we propose the refined multi-level captions, covering all tags and various natural language captions as a critical factor for model development. Through extensive analysis and experiments, Illustrious demonstrates state-of-the-art performance in terms of animation style, outperforming widely-used models in illustration domains, propelling easier customization and personalization with nature of open source. We plan to publicly release updated Illustrious model series sequentially as well as sustainable plans for improvements.