BLIP3o is a unified multimodal model that combines the reasoning and instruction following strength of autoregressive models with the generative power of diffusion models. Unlike prior works that diffuse VAE features or raw pixels, BLIP3o diffuses semantically rich CLIP image features, enabling a powerful and efficient architecture for both image understanding and generation. This model is fully open-source, including training data, training recipe, model weights, and code.
BLIP3o has achieved state-of-the-art performance across a wide range of image understanding and generation benchmarks. The model is trained on a large dataset of 20 million images with detailed captions and 4 million images with short captions. The dataset is compressed into tar archives, making it easy to download and use. The model is also supported by a demo that allows users to try out the model in their browser.
BLIP3o supports a variety of tasks, including text-to-text, image-to-text, text-to-image, image-to-image, and multitask training. The model also supports different image generation methods, such as CLIP + MSE, CLIP + Flow Matching, VAE + Flow Matching, Transfusion, and LMFusion. Additionally, the model supports different autoregressive backbones, including Qwen-2.5-VL and LLaMA 3. The model is designed to be flexible and adaptable to different use cases and applications.
Get more likes & reach the top of search results by adding this button on your site!
Get top updates in AI to your inbox every weekend. It's free!