Language Surgery in Multilingual Large Language Models

Summary

This paper talks about how large language models (LLMs) naturally align the way they represent different languages inside their middle layers. It introduces a new method called Inference-Time Language Control (ITLC), which allows the model to control and switch languages precisely during use without losing the meaning of what it says.

What's the problem?

The problem is that while LLMs can understand and generate many languages, they sometimes get confused between languages or mix them up, leading to inconsistent or incorrect outputs when switching languages. This makes it hard for the models to perform well in tasks that require precise control over which language is being used.

What's the solution?

The solution is ITLC, which uses a technique called latent injection to manipulate the language-specific information inside the model at inference time. This method takes advantage of the natural alignment in the model’s internal language representations to separate language-specific elements cleanly and control them independently. This way, the model can generate text in the target language more accurately without mixing languages or changing the meaning.

Why it matters?

This matters because it improves multilingual AI systems, allowing them to switch languages reliably and produce consistent, correct outputs. This advance helps in building better translation tools, cross-lingual communication systems, and any applications where accurate control over multiple languages is important, making AI more useful and effective in diverse global settings.