Empirical Study of Mutual Reinforcement Effect and Application in Few-shot Text Classification Tasks via Prompt
Chengguang Gan, Tatsunori Mori
2024-10-16
Summary
This paper explores the Mutual Reinforcement Effect (MRE) in text classification, showing how word-level and text-level classifications can enhance each other’s performance, particularly in few-shot learning scenarios.
What's the problem?
Current methods for text classification often focus on either word-level or text-level analysis separately, which limits their effectiveness. There hasn't been enough research to show how these two levels can work together to improve overall classification performance, especially when there is little training data available (few-shot learning).
What's the solution?
The authors conducted experiments on 21 different datasets to investigate the MRE and found that when word-level information is used alongside text-level classification, both levels perform better. They applied this concept in a new way by using word-level details as prompts to help the model make more accurate predictions about the entire text. Their results showed significant improvements in classification accuracy across many datasets.
Why it matters?
This research is important because it provides new insights into how combining different levels of analysis can enhance machine learning models. By demonstrating the benefits of the Mutual Reinforcement Effect, this work can lead to better text classification systems that require less training data while still achieving high accuracy, which is valuable in many applications like sentiment analysis, spam detection, and more.
Abstract
The Mutual Reinforcement Effect (MRE) investigates the synergistic relationship between word-level and text-level classifications in text classification tasks. It posits that the performance of both classification levels can be mutually enhanced. However, this mechanism has not been adequately demonstrated or explained in prior research. To address this gap, we employ empirical experiment to observe and substantiate the MRE theory. Our experiments on 21 MRE mix datasets revealed the presence of MRE in the model and its impact. Specifically, we conducted compare experiments use fine-tune. The results of findings from comparison experiments corroborates the existence of MRE. Furthermore, we extended the application of MRE to prompt learning, utilizing word-level information as a verbalizer to bolster the model's prediction of text-level classification labels. In our final experiment, the F1-score significantly surpassed the baseline in 18 out of 21 MRE Mix datasets, further validating the notion that word-level information enhances the language model's comprehension of the text as a whole.