NER Retriever: Zero-Shot Named Entity Retrieval with Type-Aware Embeddings
Or Shachar, Uri Katz, Yoav Goldberg, Oren Glickman
2025-09-05
Summary
This paper introduces a new system called NER Retriever that helps find documents mentioning specific types of things, even if you don't know those types beforehand. It's a way to do 'Named Entity Retrieval' without needing a pre-defined list of categories.
What's the problem?
Normally, when you want a computer to find things like 'companies' or 'locations' in text, you need to tell it exactly what those things *are* – a fixed set of categories. This is difficult because you often want to search for things you haven't specifically planned for, described in your own words. Existing methods either rely on these fixed categories or require a lot of training to understand new ones, and they aren't very good at understanding what you *mean* when you describe the type of thing you're looking for.
What's the solution?
The researchers used a large language model, the kind powering many AI tools, and found that the internal 'thinking' of the model – specifically, data from the middle layers – contains a lot of information about different types of things. They then created a small extra piece that 'fine-tunes' this internal information, making it even better at understanding the descriptions you give for the types of things you want to find. This allows the system to translate your description into a way to search for relevant documents efficiently.
Why it matters?
This work is important because it provides a way to search for information about things without needing to pre-define what those things are. It's more flexible and scalable than previous methods, and it shows that we can get useful information from the inner workings of large language models, not just their final outputs. This could be useful in many areas, like quickly finding information about emerging trends or specific events.
Abstract
We present NER Retriever, a zero-shot retrieval framework for ad-hoc Named Entity Retrieval, a variant of Named Entity Recognition (NER), where the types of interest are not provided in advance, and a user-defined type description is used to retrieve documents mentioning entities of that type. Instead of relying on fixed schemas or fine-tuned models, our method builds on internal representations of large language models (LLMs) to embed both entity mentions and user-provided open-ended type descriptions into a shared semantic space. We show that internal representations, specifically the value vectors from mid-layer transformer blocks, encode fine-grained type information more effectively than commonly used top-layer embeddings. To refine these representations, we train a lightweight contrastive projection network that aligns type-compatible entities while separating unrelated types. The resulting entity embeddings are compact, type-aware, and well-suited for nearest-neighbor search. Evaluated on three benchmarks, NER Retriever significantly outperforms both lexical and dense sentence-level retrieval baselines. Our findings provide empirical support for representation selection within LLMs and demonstrate a practical solution for scalable, schema-free entity retrieval. The NER Retriever Codebase is publicly available at https://github.com/ShacharOr100/ner_retriever