DreamCache: Finetuning-Free Lightweight Personalized Image Generation via Feature Caching
Emanuele Aiello, Umberto Michieli, Diego Valsesia, Mete Ozay, Enrico Magli
2024-11-28
Summary
This paper presents DreamCache, a new method for generating personalized images quickly and efficiently without needing extensive training or complex setups.
What's the problem?
Creating personalized images often requires a lot of manual work and complex training processes. Existing methods can be slow, expensive, and inflexible, making it hard to generate images that accurately reflect individual preferences without a lot of effort.
What's the solution?
The authors introduce DreamCache, which uses a feature caching system to store important details from reference images. This allows the model to generate personalized images without needing to retrain or rely on detailed text descriptions. By using lightweight adapters that modify the image features based on the cached data, DreamCache can produce high-quality images much faster and with less computational cost than previous methods.
Why it matters?
This research is significant because it makes personalized image generation more accessible and efficient. By simplifying the process, DreamCache enables artists and creators to generate customized images quickly, enhancing creativity and productivity in fields like art, advertising, and design.
Abstract
Personalized image generation requires text-to-image generative models that capture the core features of a reference subject to allow for controlled generation across different contexts. Existing methods face challenges due to complex training requirements, high inference costs, limited flexibility, or a combination of these issues. In this paper, we introduce DreamCache, a scalable approach for efficient and high-quality personalized image generation. By caching a small number of reference image features from a subset of layers and a single timestep of the pretrained diffusion denoiser, DreamCache enables dynamic modulation of the generated image features through lightweight, trained conditioning adapters. DreamCache achieves state-of-the-art image and text alignment, utilizing an order of magnitude fewer extra parameters, and is both more computationally effective and versatile than existing models.