Afri-MCQA: Multimodal Cultural Question Answering for African Languages
Atnafu Lambebo Tonja, Srija Anand, Emilio Villa-Cueva, Israel Abebe Azime, Jesujoba Oluwadara Alabi, Muhidin A. Mohamed, Debela Desalegn Yadeta, Negasi Haile Abadi, Abigail Oppong, Nnaemeka Casmir Obiefuna, Idris Abdulmumin, Naome A Etori, Eric Peter Wairagala, Kanda Patrick Tshinu, Imanigirimbabazi Emmanuel, Gabofetswe Malema, Alham Fikri Aji, David Ifeoluwa Adelani, Thamar Solorio
2026-01-12
Summary
This research introduces a new benchmark called Afri-MCQA, which is a collection of questions and answers in 15 different African languages, along with their English translations, and both text and speech versions. It's designed to test how well AI models understand both the language and the culture behind the questions.
What's the problem?
Current AI models, especially large language models, are really good at processing English, but they struggle with other languages, particularly those from Africa. This isn't just a language issue; it's also about understanding the cultural context needed to answer questions correctly. There's a lack of resources to properly test and improve AI performance in African languages, leading to systems that don't work well for African users.
What's the solution?
The researchers created Afri-MCQA, a dataset of over 7,500 question-answer pairs created *by* native speakers. They then tested existing AI models on this dataset, using both text and speech. They also included tests to separate whether the AI was failing because of language difficulties or because it lacked cultural knowledge. This allows them to pinpoint exactly where the models are falling short.
Why it matters?
This work is important because it highlights the significant gap in AI development for African languages and cultures. By providing a benchmark like Afri-MCQA, researchers can now focus on building AI systems that are more inclusive and useful for people across Africa, and it points to the need for AI development that prioritizes speech and cultural understanding.
Abstract
Africa is home to over one-third of the world's languages, yet remains underrepresented in AI research. We introduce Afri-MCQA, the first Multilingual Cultural Question-Answering benchmark covering 7.5k Q&A pairs across 15 African languages from 12 countries. The benchmark offers parallel English-African language Q&A pairs across text and speech modalities and was entirely created by native speakers. Benchmarking large language models (LLMs) on Afri-MCQA shows that open-weight models perform poorly across evaluated cultures, with near-zero accuracy on open-ended VQA when queried in native language or speech. To evaluate linguistic competence, we include control experiments meant to assess this specific aspect separate from cultural knowledge, and we observe significant performance gaps between native languages and English for both text and speech. These findings underscore the need for speech-first approaches, culturally grounded pretraining, and cross-lingual cultural transfer. To support more inclusive multimodal AI development in African languages, we release our Afri-MCQA under academic license or CC BY-NC 4.0 on HuggingFace (https://huggingface.co/datasets/Atnafu/Afri-MCQA)