Search and Refine During Think: Autonomous Retrieval-Augmented Reasoning of LLMs
Yaorui Shi, Sihang Li, Chang Wu, Zhiyuan Liu, Junfeng Fang, Hengxing Cai, An Zhang, Xiang Wang
2025-05-28
Summary
This paper talks about AutoRefine, a new method that helps large language models get better at answering tough questions by teaching them to search for information and improve their answers step by step.
What's the problem?
The problem is that even though language models can look up information to help answer questions, they often don't do a good job of refining or updating their answers as they learn more, which can lead to mistakes or incomplete responses.
What's the solution?
To solve this, the researchers created AutoRefine, which uses reinforcement learning to train the AI to search for new information and keep improving its answers through several rounds. This way, the model can build on what it finds and give more accurate and well-thought-out responses.
Why it matters?
This matters because it makes AI much more reliable for answering complicated questions, helping students, researchers, and anyone who needs detailed and up-to-date information.
Abstract
AutoRefine, a reinforcement learning framework for large language models, enhances retrieval-augmented reasoning by iteratively refining knowledge and optimizing searches, leading to improved performance in complex question-answering tasks.