OnGoal: Tracking and Visualizing Conversational Goals in Multi-Turn Dialogue with Large Language Models
Adam Coscia, Shunan Guo, Eunyee Koh, Alex Endert
2025-08-29
Summary
This paper introduces OnGoal, a new way to interact with large language models (LLMs) like ChatGPT, designed to help users keep track of their progress when having long conversations.
What's the problem?
When you have a lengthy back-and-forth with an LLM to achieve a specific goal, like writing a story or planning a trip, it can be hard to remember what you’ve already discussed and whether the conversation is actually moving you closer to your desired outcome. It's easy to get lost or feel like you're going in circles, and it's difficult to judge if the LLM is truly understanding and helping you reach your goal.
What's the solution?
OnGoal tackles this by building a chat interface that gives users constant updates on how well the conversation aligns with their initial goal. It doesn't just tell you if things are on track, but *why*, providing examples to explain its reasoning. It also shows a history of your progress, letting you see how the conversation has evolved over time. The researchers tested OnGoal with 20 people who were asked to complete a writing task, comparing it to a standard chat interface without these features.
Why it matters?
The study found that people using OnGoal were able to achieve their goals more quickly and with less effort. They were also more willing to experiment with different ways of prompting the LLM when things weren't going as planned. This suggests that providing clear feedback and visualizing progress can make interacting with LLMs more engaging, less frustrating, and ultimately more productive, and can inform the design of future LLM interfaces to be more user-friendly and effective.
Abstract
As multi-turn dialogues with large language models (LLMs) grow longer and more complex, how can users better evaluate and review progress on their conversational goals? We present OnGoal, an LLM chat interface that helps users better manage goal progress. OnGoal provides real-time feedback on goal alignment through LLM-assisted evaluation, explanations for evaluation results with examples, and overviews of goal progression over time, enabling users to navigate complex dialogues more effectively. Through a study with 20 participants on a writing task, we evaluate OnGoal against a baseline chat interface without goal tracking. Using OnGoal, participants spent less time and effort to achieve their goals while exploring new prompting strategies to overcome miscommunication, suggesting tracking and visualizing goals can enhance engagement and resilience in LLM dialogues. Our findings inspired design implications for future LLM chat interfaces that improve goal communication, reduce cognitive load, enhance interactivity, and enable feedback to improve LLM performance.