Knowledge Navigator: LLM-guided Browsing Framework for Exploratory Search in Scientific Literature
Uri Katz, Mosh Levy, Yoav Goldberg
2024-08-29
Summary
This paper presents Knowledge Navigator, a system designed to improve how researchers explore and search through scientific literature by organizing documents into a structured format.
What's the problem?
With the rapid increase in scientific papers and articles, it has become difficult for researchers to find relevant information quickly. Traditional search methods often return a long list of documents that are hard to navigate, making it challenging to discover useful insights or specific topics within a vast amount of data.
What's the solution?
Knowledge Navigator addresses this problem by organizing search results into a two-level hierarchy of topics and subtopics, making it easier for users to see the main themes in a field and drill down into specific areas of interest. The system combines the capabilities of large language models (LLMs) with clustering techniques to enhance the browsing experience. This allows users to refine their searches and find additional relevant documents efficiently.
Why it matters?
This research is significant because it provides a more effective way for scientists and researchers to explore complex information. By improving how scientific literature is organized and accessed, Knowledge Navigator can help accelerate discoveries and advancements in various fields by making it easier to find and connect relevant research.
Abstract
The exponential growth of scientific literature necessitates advanced tools for effective knowledge exploration. We present Knowledge Navigator, a system designed to enhance exploratory search abilities by organizing and structuring the retrieved documents from broad topical queries into a navigable, two-level hierarchy of named and descriptive scientific topics and subtopics. This structured organization provides an overall view of the research themes in a domain, while also enabling iterative search and deeper knowledge discovery within specific subtopics by allowing users to refine their focus and retrieve additional relevant documents. Knowledge Navigator combines LLM capabilities with cluster-based methods to enable an effective browsing method. We demonstrate our approach's effectiveness through automatic and manual evaluations on two novel benchmarks, CLUSTREC-COVID and SCITOC. Our code, prompts, and benchmarks are made publicly available.