Word Form Matters: LLMs' Semantic Reconstruction under Typoglycemia
Chenxi Wang, Tianle Gu, Zhongyu Wei, Lang Gao, Zirui Song, Xiuying Chen
2025-03-04
Summary
This paper talks about how AI language models (LLMs) understand scrambled words, a skill humans have called Typoglycemia. The researchers wanted to figure out if LLMs use the same methods as humans to make sense of jumbled words.
What's the problem?
While we know that humans can read scrambled words by looking at the overall shape of the word and using context clues, we don't fully understand how AI models do this. It's important to know if AI uses similar strategies to humans or if they have their own unique way of figuring out scrambled words.
What's the solution?
The researchers created a new way to measure how well AI models understand scrambled words, called SemRecScore. They then used this tool to test how different factors, like word shape and context, affect the AI's ability to understand jumbled words. They also looked at how the AI pays attention to different parts of scrambled words.
Why it matters?
This research matters because it helps us understand the differences between how humans and AI process language. By showing that AI models focus mainly on word shape, while humans are more flexible in using both word shape and context, we can see ways to improve AI language models. Making AI more human-like in how it understands scrambled words could lead to better language processing tools that can handle messy or imperfect text more effectively.
Abstract
Human readers can efficiently comprehend scrambled words, a phenomenon known as Typoglycemia, primarily by relying on word form; if word form alone is insufficient, they further utilize contextual cues for interpretation. While advanced large language models (LLMs) exhibit similar abilities, the underlying mechanisms remain unclear. To investigate this, we conduct controlled experiments to analyze the roles of word form and contextual information in semantic reconstruction and examine LLM attention patterns. Specifically, we first propose SemRecScore, a reliable metric to quantify the degree of semantic reconstruction, and validate its effectiveness. Using this metric, we study how word form and contextual information influence LLMs' semantic reconstruction ability, identifying word form as the core factor in this process. Furthermore, we analyze how LLMs utilize word form and find that they rely on specialized attention heads to extract and process word form information, with this mechanism remaining stable across varying levels of word scrambling. This distinction between LLMs' fixed attention patterns primarily focused on word form and human readers' adaptive strategy in balancing word form and contextual information provides insights into enhancing LLM performance by incorporating human-like, context-aware mechanisms.