LLM-guided Hierarchical Retrieval
Nilesh Gupta, Wei-Cheng Chang, Ngot Bui, Cho-Jui Hsieh, Inderjit S. Dhillon
2025-10-17
Summary
This paper introduces LATTICE, a new way for information retrieval systems to find relevant information in large collections of documents. It aims to improve upon existing methods that struggle with complex questions needing deeper understanding, and keeping information up-to-date.
What's the problem?
Current search systems, even those using powerful Large Language Models (LLMs), have trouble with complicated questions. Traditional methods rely on finding keywords or similar meanings, which isn't enough for reasoning-based queries. Updating these systems with new information is also difficult, and trying to feed an LLM the entire document collection at once is too computationally expensive. Essentially, existing systems either don't understand the question well enough, can't easily learn new things, or can't handle the sheer volume of information.
What's the solution?
LATTICE solves this by organizing the entire collection of documents into a tree-like structure, kind of like a detailed outline. This happens in two steps: first, the system automatically creates this structure using summaries of the documents. Second, when a question is asked, an LLM navigates this tree to find the most relevant information. A key challenge is that the LLM's judgments aren't always perfect, so the system uses a special algorithm to estimate how relevant each branch of the tree is, combining these estimates to find the best path to the answer.
Why it matters?
This research is important because it significantly improves the ability of search systems to answer complex questions that require reasoning. It achieves better results than previous methods, especially when no additional training is involved, and performs comparably to systems that *have* been specifically trained. This means LATTICE can be used effectively right away, and it offers a scalable way to handle very large amounts of information, making more powerful and accurate search possible.
Abstract
Modern IR systems are increasingly tasked with answering complex, multi-faceted queries that require deep reasoning rather than simple keyword or semantic matching. While LLM-based IR has shown great promise, the prevailing retrieve-then-rerank paradigm inherits the limitations of embedding-based retrieval; parametric generative approaches are difficult to update with new information; and long-context methods that place the entire corpus in context are computationally infeasible for large document collections. To address these challenges, we introduce LATTICE, a hierarchical retrieval framework that enables an LLM to reason over and navigate large corpora with logarithmic search complexity by imposing a semantic tree structure on the corpus. Our approach consists of two stages: (1) an offline phase that organizes the corpus into a semantic hierarchy via either a bottom-up agglomerative strategy or a top-down divisive strategy using multi-level summaries and (2) an online traversal phase where a search LLM navigates this tree. A central challenge in such LLM-guided search is that the model's relevance judgments are noisy, context-dependent, and unaware of the hierarchy, making cross-branch and cross-level comparisons difficult. To overcome this, we propose a traversal algorithm that estimates calibrated latent relevance scores from local LLM outputs and aggregates them into a global path relevance metric. Our training-free framework achieves state-of-the-art zero-shot performance on the reasoning-intensive BRIGHT benchmark, demonstrating up to 9% improvement in Recall@100 and 5% in nDCG@10 over the next best zero-shot baseline. Furthermore, compared to the fine-tuned SOTA method DIVER-v2, LATTICE attains comparable results on BRIGHT subsets that use a static corpus for evaluation.