Agentic-R: Learning to Retrieve for Agentic Search
Wenhan Liu, Xinyu Ma, Yutao Zhu, Yuchen Li, Daiting Shi, Dawei Yin, Zhicheng Dou
2026-01-21
Summary
This paper focuses on improving how search engines work when used by 'agents' – programs that think through problems step-by-step and search for information as they go. It's about making the search part of these agents smarter.
What's the problem?
Current search systems used with these agents mostly find information that's *similar* to what the agent is asking about. However, similarity doesn't always mean helpfulness. Just because a passage is related to the question doesn't mean it will actually lead to the right answer. Traditional search training focuses on whether a passage is relevant to the initial question, but doesn't consider if that passage ultimately helps the agent find the *correct* answer after multiple steps of reasoning.
What's the solution?
The researchers developed a new way to train search systems specifically for these agent-based tasks. Instead of just checking if a passage is relevant to the question, their training also looks at whether the passage helps the agent arrive at the correct final answer. They also created a system where the search engine and the agent learn *together* – as the agent gets better at asking questions, the search engine gets better at finding helpful information, and vice versa. This happens through repeated training cycles.
Why it matters?
This work is important because it improves the performance of AI agents that need to find information to solve complex problems. By making the search component more effective, these agents can become more reliable and accurate, leading to better results in areas like question answering and problem solving. It moves beyond simply finding related text to finding text that actively contributes to a correct solution.
Abstract
Agentic search has recently emerged as a powerful paradigm, where an agent interleaves multi-step reasoning with on-demand retrieval to solve complex questions. Despite its success, how to design a retriever for agentic search remains largely underexplored. Existing search agents typically rely on similarity-based retrievers, while similar passages are not always useful for final answer generation. In this paper, we propose a novel retriever training framework tailored for agentic search. Unlike retrievers designed for single-turn retrieval-augmented generation (RAG) that only rely on local passage utility, we propose to use both local query-passage relevance and global answer correctness to measure passage utility in a multi-turn agentic search. We further introduce an iterative training strategy, where the search agent and the retriever are optimized bidirectionally and iteratively. Different from RAG retrievers that are only trained once with fixed questions, our retriever is continuously improved using evolving and higher-quality queries from the agent. Extensive experiments on seven single-hop and multi-hop QA benchmarks demonstrate that our retriever, termed , consistently outperforms strong baselines across different search agents. Our codes are available at: https://github.com/8421BCD/Agentic-R.