HiKE: Hierarchical Evaluation Framework for Korean-English Code-Switching Speech Recognition
Gio Paik, Yongbeom Kim, Soungmin Lee, Sangmin Ahn, Chanwoo Kim
2025-10-07
Summary
This paper introduces a new tool, called HiKE, designed to test how well computers can understand speech when people switch between Korean and English in the same sentence.
What's the problem?
Current speech recognition technology is really good at understanding single languages, but it struggles when people naturally mix languages – a phenomenon called code-switching – which happens all the time in real conversations. There wasn't a good way to specifically test and improve these systems' ability to handle this kind of mixed speech, especially for Korean and English.
What's the solution?
The researchers created HiKE, a large collection of Korean-English speech where people actually switch between the languages. This collection isn't just audio; it also includes detailed labels showing *where* the language switches happen – at the word, phrase, or even sentence level. They then tested existing speech recognition programs on HiKE and showed that while they initially weren't very good at understanding code-switching, they could be improved by training them specifically on this new dataset.
Why it matters?
This work is important because it provides a standard way to measure and improve speech recognition for code-switching. This will lead to better voice assistants and other speech-based technologies for people who regularly use multiple languages, making these technologies more accessible and useful in everyday life.
Abstract
Despite advances in multilingual automatic speech recognition (ASR), code-switching (CS), the mixing of languages within an utterance common in daily speech, remains a severely underexplored challenge. In this paper, we introduce HiKE: the Hierarchical Korean-English code-switching benchmark, the first globally accessible evaluation framework for Korean-English CS, aiming to provide a means for the precise evaluation of multilingual ASR models and to foster research in the field. The proposed framework not only consists of high-quality, natural CS data across various topics, but also provides meticulous loanword labels and a hierarchical CS-level labeling scheme (word, phrase, and sentence) that together enable a systematic evaluation of a model's ability to handle each distinct level of code-switching. Through evaluations of diverse multilingual ASR models and fine-tuning experiments, this paper demonstrates that while most multilingual ASR models initially struggle with CS-ASR, this capability can be enabled through fine-tuning with CS data. HiKE will be available at https://github.com/ThetaOne-AI/HiKE.