ATHAR: A High-Quality and Diverse Dataset for Classical Arabic to English Translation
Mohammed Khalil, Mohammed Sabry
2024-07-30
Summary
This paper introduces ATHAR, a new dataset designed for translating Classical Arabic texts into English. It includes 66,000 high-quality translation samples covering various subjects like science, culture, and philosophy.
What's the problem?
There is a lack of translation datasets for Classical Arabic, which limits the development of effective translation systems. Most existing datasets are small or focus on narrow topics, making it difficult to create high-quality translations that capture the richness of Classical Arabic literature.
What's the solution?
To address this issue, the authors created the ATHAR dataset, which contains a large number of diverse translation samples. This dataset allows language models to learn from a wide range of topics and improves their ability to translate Classical Arabic into English accurately. The authors also evaluated current language models to show how they can benefit from using this new dataset for better performance.
Why it matters?
This research is important because it helps preserve and share the knowledge contained in Classical Arabic literature with a broader audience. By providing a comprehensive dataset, ATHAR can enhance the capabilities of translation systems, making it easier for people to access and understand important cultural and scientific texts from the Arab world.
Abstract
Classical Arabic represents a significant era, encompassing the golden age of Arab culture, philosophy, and scientific literature. With a broad consensus on the importance of translating these literatures to enrich knowledge dissemination across communities, the advent of large language models (LLMs) and translation systems offers promising tools to facilitate this goal. However, we have identified a scarcity of translation datasets in Classical Arabic, which are often limited in scope and topics, hindering the development of high-quality translation systems. In response, we present the ATHAR dataset, comprising 66,000 high-quality Classical Arabic to English translation samples that cover a wide array of subjects including science, culture, and philosophy. Furthermore, we assess the performance of current state-of-the-art LLMs under various settings, concluding that there is a need for such datasets in current systems. Our findings highlight how models can benefit from fine-tuning or incorporating this dataset into their pretraining pipelines. The dataset is publicly available on the HuggingFace Data Hub at https://huggingface.co/datasets/mohamed-khalil/ATHAR.