Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Tasks
Yauhen Babakhin, Radek Osmulski, Ronay Ak, Gabriel Moreira, Mengyao Xu, Benedikt Schifferer, Bo Liu, Even Oldridge
2025-11-11
Summary
This paper introduces a new, high-performing text embedding model called llama-embed-nemotron-8b, which is completely open-source, meaning its code and data are publicly available.
What's the problem?
Existing text embedding models often achieve good results, but the details of how they were trained – the data they used and the specific techniques – are usually kept secret, making it hard for others to understand, improve, or even trust them. This limits progress in the field and makes it difficult to adapt these models to specific needs.
What's the solution?
The researchers created llama-embed-nemotron-8b, a model that performs exceptionally well on a variety of text embedding tasks, especially in multiple languages. They achieved this by carefully combining publicly available data with new, artificially created data generated by other AI models. They also thoroughly tested different training methods and model configurations, and are sharing the results of these tests so others can learn from their work. The model can also be customized with instructions to improve performance for specific tasks.
Why it matters?
This work is important because it provides a powerful, transparent, and adaptable text embedding model that anyone can use and build upon. By making everything open-source, the researchers are fostering collaboration and accelerating innovation in the field of natural language processing, and offering a solution that works well even with languages that don't have a lot of existing data.
Abstract
We introduce llama-embed-nemotron-8b, an open-weights text embedding model that achieves state-of-the-art performance on the Multilingual Massive Text Embedding Benchmark (MMTEB) leaderboard as of October 21, 2025. While recent models show strong performance, their training data or methodologies are often not fully disclosed. We aim to address this by developing a fully open-source model, publicly releasing its weights and detailed ablation studies, and planning to share the curated training datasets. Our model demonstrates superior performance across all major embedding tasks -- including retrieval, classification and semantic textual similarity (STS) -- and excels in challenging multilingual scenarios, such as low-resource languages and cross-lingual setups. This state-of-the-art performance is driven by a novel data mix of 16.1 million query-document pairs, split between 7.7 million samples from public datasets and 8.4 million synthetically generated examples from various open-weight LLMs. One of our key contributions is a detailed ablation study analyzing core design choices, including a comparison of contrastive loss implementations, an evaluation of synthetic data generation (SDG) strategies, and the impact of model merging. The llama-embed-nemotron-8b is an instruction-aware model, supporting user-defined instructions to enhance performance for specific use-cases. This combination of top-tier performance, broad applicability, and user-driven flexibility enables it to serve as a universal text embedding solution.