TinyEmo: Scaling down Emotional Reasoning via Metric Projection
Cristian Gutierrez
2024-10-10
Summary
This paper introduces TinyEmo, a set of small language models designed to understand and classify emotions efficiently.
What's the problem?
Understanding and classifying emotions in text is important but can be challenging for large language models (LLMs), which often require a lot of computational resources. Many existing models are large and complex, making them less efficient and harder to use in practical applications.
What's the solution?
To solve this problem, the authors developed TinyEmo, which includes several key features: a synthetic dataset for training, a Metric Projector that helps classify emotions more efficiently, a multi-modal model that can handle both text and other data types, and a framework for detecting bias. TinyEmo is smaller than many existing models but still performs well in classifying emotions, with the smallest version having only 700 million parameters, while larger models often exceed 7 billion parameters. This efficiency allows TinyEmo to work with diverse emotional datasets effectively.
Why it matters?
This research is important because it demonstrates that smaller models can achieve high performance in emotional reasoning tasks without the heavy resource requirements of larger models. By making emotion classification more accessible and efficient, TinyEmo can help improve applications in areas like mental health support, customer service, and content moderation.
Abstract
This paper introduces TinyEmo, a family of small multi-modal language models for emotional reasoning and classification. Our approach features: (1) a synthetic emotional instruct dataset for both pre-training and fine-tuning stages, (2) a Metric Projector that delegates classification from the language model allowing for more efficient training and inference, (3) a multi-modal large language model (MM-LLM) for emotional reasoning, and (4) a semi-automated framework for bias detection. TinyEmo is able to perform emotion classification and emotional reasoning, all while using substantially fewer parameters than comparable models. This efficiency allows us to freely incorporate more diverse emotional datasets, enabling strong performance on classification tasks, with our smallest model (700M parameters) outperforming larger state-of-the-art models based on general-purpose MM-LLMs with over 7B parameters. Additionally, the Metric Projector allows for interpretability and indirect bias detection in large models without additional training, offering an approach to understand and improve AI systems. We release code, models, and dataset at https://github.com/ggcr/TinyEmo