Summary

CLAP2 is LAION's contrastive language-audio pretraining model family, a dual encoder that turns audio and text into single vectors so cosine similarity gives audio-text matching for search, zero-shot classification or features. Five sizes ship, from CLAP2-XXS to CLAP2-L at 27M to 340M parameters, and any size can be swapped in by changing one Hugging Face hub ID. The weights are open and free, usable in about eight lines of Python through open_clip, and the family is a released model even though the project page also presents its research paper.

Key Features

Audio-text dual encoder with cosine-similarity matching for search and classification
Five model sizes from XXS to Large, 27M to 340M parameters, swappable by one hub ID
Encodes audio clips of any length up to 30 seconds into a single vector
Zero-shot audio classification and audio search
Works in a few lines of Python via open_clip with create_model_from_pretrained
Best MAEB-30 score of any model trained from scratch, 53.6% for CLAP2-L
Trained on LAION-Audio-XL: 1.48B samples collected into 365M clips with short and long captions
Precomputed audio and caption embeddings dataset available for 365M clips

About CLAP2


Each model encodes audio clips of any length up to 30 seconds into a single vector, and CLAP2-L scores 53.6% on MAEB-30, the best of any model trained from scratch. Training used LAION-Audio-XL, built from 1.48 billion collected samples into 365 million clips with short and long captions, and precomputed audio and caption embeddings for those 365 million clips are published as a dataset. The code path is the OpenCLIP library: pip install open_clip_torch version 4.2.1 or newer, then create_model_from_pretrained with the hub ID.


CLAP2 is aimed at developers building audio search, zero-shot audio classification and audio features into applications, and at researchers working with audio-text models. The model repos on Hugging Face are laion/clap2-xxs, clap2-xs, clap2-small, clap2-medium and clap2-large-audioxl-s1B, and the embeddings dataset is laion/laion-audio-xl-embeddings.

Get more likes & reach the top of search results by adding this button on your site!

Embed button preview - Light theme
Embed button preview - Dark theme
TurboType Banner

Subscribe to the AI Search Newsletter

Get top updates in AI to your inbox every weekend. It's free!