Show Less, Instruct More: Enriching Prompts with Definitions and Guidelines for Zero-Shot NER
Andrew Zamai, Andrea Zugarini, Leonardo Rigutini, Marco Ernandes, Marco Maggini
2024-07-02
Summary
This paper talks about SLIMER, a new method for Named Entity Recognition (NER) that improves how AI models identify and classify named entities, like people and organizations, in text. It focuses on using enriched prompts with definitions and guidelines to help the model recognize entities it has never seen before.
What's the problem?
Traditional NER systems often require a lot of labeled examples to perform well, and they struggle to identify new types of entities that haven't been included in their training data. Many existing models are trained on large sets of entity types but still have difficulty generalizing to new or different contexts, especially when they encounter out-of-domain data (data that comes from a different context than what they were trained on). This limitation means that these models can miss important information in texts.
What's the solution?
To address this issue, the authors developed SLIMER, which uses a different approach by providing the model with fewer examples and enhancing its prompts with definitions and guidelines. This helps the model understand what each entity type is and how to identify it, even if it hasn't seen that specific type before. The experiments showed that this method leads to better performance and faster learning when it comes to labeling unseen named entities. SLIMER is also able to compete with other advanced models while being trained on a smaller set of entity types.
Why it matters?
This research is important because it makes NER technology more flexible and capable of handling a wider variety of texts without needing extensive training on every possible entity type. By improving how models learn from definitions and guidelines, SLIMER can enhance applications in fields like information extraction, search engines, and any area where understanding text is crucial. This advancement could lead to more accurate AI systems capable of processing diverse languages and contexts.
Abstract
Recently, several specialized instruction-tuned Large Language Models (LLMs) for Named Entity Recognition (NER) have emerged. Compared to traditional NER approaches, these models have strong generalization capabilities. Existing LLMs mainly focus on zero-shot NER in out-of-domain distributions, being fine-tuned on an extensive number of entity classes that often highly or completely overlap with test sets. In this work instead, we propose SLIMER, an approach designed to tackle never-seen-before named entity tags by instructing the model on fewer examples, and by leveraging a prompt enriched with definition and guidelines. Experiments demonstrate that definition and guidelines yield better performance, faster and more robust learning, particularly when labelling unseen Named Entities. Furthermore, SLIMER performs comparably to state-of-the-art approaches in out-of-domain zero-shot NER, while being trained on a reduced tag set.