Breaking Boundaries: Investigating the Effects of Model Editing on Cross-linguistic Performance
Somnath Banerjee, Avik Halder, Rajarshi Mandal, Sayan Layek, Ian Soboroff, Rima Hazra, Animesh Mukherjee
2024-06-20
Summary
This paper explores how to improve the performance of language models for different languages, aiming for fairness in AI by using techniques to edit knowledge in models like BERT and GPT.
What's the problem?
While pretrained language models (PLMs) have greatly advanced natural language processing (NLP), they primarily benefit English and create imbalances for other languages. This means that speakers of less-represented languages may not get the same quality of AI responses, leading to unfair treatment and missed opportunities in communication and technology.
What's the solution?
The researchers tested various models, including Mistral and TowerInstruct, to see how well they performed across multiple languages. They used editing techniques called 'each language for itself' (ELFI) and 'each language for others' (ELFO) to evaluate how these models could be improved. Their findings showed that with the right adjustments, language models can better handle different languages, helping to bridge the gap between high-resource and low-resource languages.
Why it matters?
This research is crucial because it highlights the need for linguistic equity in AI technologies. By ensuring that AI can perform well across various languages, we can create more inclusive tools that serve everyone, regardless of their language background. This can lead to better communication, access to information, and opportunities for people around the world.
Abstract
The integration of pretrained language models (PLMs) like BERT and GPT has revolutionized NLP, particularly for English, but it has also created linguistic imbalances. This paper strategically identifies the need for linguistic equity by examining several knowledge editing techniques in multilingual contexts. We evaluate the performance of models such as Mistral, TowerInstruct, OpenHathi, Tamil-Llama, and Kan-Llama across languages including English, German, French, Italian, Spanish, Hindi, Tamil, and Kannada. Our research identifies significant discrepancies in normal and merged models concerning cross-lingual consistency. We employ strategies like 'each language for itself' (ELFI) and 'each language for others' (ELFO) to stress-test these models. Our findings demonstrate the potential for LLMs to overcome linguistic barriers, laying the groundwork for future research in achieving linguistic inclusivity in AI technologies.