Adaptive Multi-Agent Response Refinement in Conversational Systems
Soyeong Jeong, Aparna Elangovan, Emine Yilmaz, Oleg Rokhlenko
2025-11-12
Summary
This paper explores a new way to make large language models, the ones powering chatbots, give better and more reliable answers in conversations.
What's the problem?
Large language models are really good at *sounding* human, but they sometimes get facts wrong, forget what they know about you, or just don't make sense. It's not realistic to expect users to constantly check every response and ask for corrections, so there's a need to automatically improve the answers before they're shown.
What's the solution?
The researchers created a system where multiple 'agents,' each with a specific job, work together to refine the chatbot's responses. One agent checks for factual accuracy, another focuses on making the response personal to the user, and a third ensures the response flows logically. Importantly, the system doesn't just go through these agents in a set order; it intelligently decides which agents are most needed for each specific question and coordinates their feedback to create a better final answer.
Why it matters?
This research is important because it offers a way to build more trustworthy and helpful conversational AI. By focusing on factuality, personalization, and coherence, and by using a collaborative approach, the system significantly improves the quality of responses, especially when the conversation requires specific knowledge or understanding of the user's preferences.
Abstract
Large Language Models (LLMs) have demonstrated remarkable success in conversational systems by generating human-like responses. However, they can fall short, especially when required to account for personalization or specific knowledge. In real-life settings, it is impractical to rely on users to detect these errors and request a new response. One way to address this problem is to refine the response before returning it to the user. While existing approaches focus on refining responses within a single LLM, this method struggles to consider diverse aspects needed for effective conversations. In this work, we propose refining responses through a multi-agent framework, where each agent is assigned a specific role for each aspect. We focus on three key aspects crucial to conversational quality: factuality, personalization, and coherence. Each agent is responsible for reviewing and refining one of these aspects, and their feedback is then merged to improve the overall response. To enhance collaboration among them, we introduce a dynamic communication strategy. Instead of following a fixed sequence of agents, our approach adaptively selects and coordinates the most relevant agents based on the specific requirements of each query. We validate our framework on challenging conversational datasets, demonstrating that ours significantly outperforms relevant baselines, particularly in tasks involving knowledge or user's persona, or both.