Learning User Preferences Through Interaction for Long-Term Collaboration
Shuhaib Mehri, Priyanka Kargupta, Tal August, Dilek Hakkani-Tür
2026-01-09
Summary
This paper focuses on building better conversational AI agents – like chatbots – that can learn what individual users like and dislike over time to create more helpful and enjoyable interactions.
What's the problem?
Currently, most chatbots treat each conversation as brand new, forgetting what they’ve learned about a user in previous interactions. This means they can’t really adapt to your specific preferences, leading to frustrating or inefficient conversations over the long run. The challenge is to create agents that remember and utilize past interactions to improve future collaborations.
What's the solution?
The researchers created a testing environment called MultiSessionCollab to specifically measure how well agents learn user preferences across multiple conversations. They then built agents with a 'memory' component that stores and updates information about each user as they interact. Importantly, they figured out a way to train these agents using simulated user behavior, helping them learn to reflect on conversations and improve their memory more effectively. They tested these memory-equipped agents and found they performed better in terms of completing tasks, requiring fewer back-and-forths, and reducing the effort needed from the user.
Why it matters?
This work is important because it moves us closer to AI assistants that feel truly personalized and helpful. By enabling agents to remember and adapt to individual users, we can build more effective and satisfying long-term relationships with AI, making them better collaborators in everyday tasks. The human user study confirms that this memory feature actually improves the user experience in real-world scenarios.
Abstract
As conversational agents accumulate experience collaborating with users, adapting to user preferences is essential for fostering long-term relationships and improving collaboration quality over time. We introduce MultiSessionCollab, a benchmark that evaluates how well agents can learn user preferences and leverage them to improve collaboration quality throughout multiple sessions. To develop agents that succeed in this setting, we present long-term collaborative agents equipped with a memory that persists and refines user preference as interaction experience accumulates. Moreover, we demonstrate that learning signals can be derived from user simulator behavior in MultiSessionCollab to train agents to generate more comprehensive reflections and update their memory more effectively. Extensive experiments show that equipping agents with memory improves long-term collaboration, yielding higher task success rates, more efficient interactions, and reduced user effort. Finally, we conduct a human user study that demonstrates that memory helps improve user experience in real-world settings.