LLaMAX2: Your Translation-Enhanced Model also Performs Well in Reasoning
Changjiang Gao, Zixian Huang, Jingyang Gong, Shujian Huang, Lei Li, Fei Yuan
2025-10-14
Summary
This paper focuses on improving translation quality in large language models without sacrificing their ability to reason and solve problems. It introduces a new method for building better multilingual models, called Qwen3-XPlus.
What's the problem?
Typically, when large language models are specifically trained to be good at translation, their general reasoning skills tend to get worse. It's hard to make a model excellent at both understanding language *and* thinking logically. This is especially true for languages that don't have a lot of readily available translation data.
What's the solution?
The researchers developed a new 'recipe' for building these models. They started with a model already good at following instructions and then fine-tuned it specifically for translation, but only adjusted certain parts of the model using parallel data – meaning data with the same text in multiple languages. This careful, targeted training process created the Qwen3-XPlus models, which show significant improvements in translation, particularly for less common languages like Swahili.
Why it matters?
This work is important because it shows a way to create multilingual models that are both accurate translators *and* capable reasoners. It’s a more efficient approach than previous methods, requiring less complex training and making it easier to build models for a wider variety of languages, even those with limited resources. The models and code are also publicly available, allowing others to build upon this research.
Abstract
General Large Language Models (LLMs) excel in reasoning, but those enhanced for translation struggle with reasoning tasks. To address this, we propose a novel translationenhanced recipe that begins with instruct models and applies layer-selective tuning only on parallel data. Following this pipeline, we introduce the Qwen3-XPlus models, which demonstrate significant improvements in translation performance across both high- and lowresource languages, achieving 15+ spBLEU and 40+ xComet in low-resource languages, like Swahili. Interestingly, training only with small parallel datasets, Qwen3-XPlus achieves an average improvement of 1+ points on 7 multilingual tasks while maintaining proficiency comparable to the Qwen3 instruct model in 15 popular reasoning datasets. This work offers a promising approach to multilingual enhancement, significantly reducing complexity and enhancing accessibility for a wider range of languages. The code and model are publicly available.