MARS: A Multi-Agent Framework Incorporating Socratic Guidance for Automated Prompt Optimization
Jian Zhang, Zhangqi Wang, Haiping Zhu, Jun Liu, Qika Lin, Erik Cambria
2025-03-24
Summary
This paper is about making AI better at creating prompts (instructions) for other AI models, so they give better answers.
What's the problem?
It's hard to design good prompts for AI models, and current methods aren't very flexible or efficient.
What's the solution?
The researchers created a system called MARS that uses multiple AI agents working together, with one agent acting like a teacher asking questions to guide the others. This helps them find better prompts more effectively.
Why it matters?
This work matters because it can improve the performance of AI models by automatically creating better prompts, leading to more accurate and helpful responses.
Abstract
The basic question-answering format of large language models involves inputting a prompt and receiving a response, and the quality of the prompt directly impacts the effectiveness of the response. Automated Prompt Optimization (APO) aims to break free from the cognitive biases of manually designed prompts and explores a broader design space for prompts. However, existing APO methods suffer from limited flexibility of fixed templates and inefficient search in prompt spaces as key issues. To this end, we propose a Multi-Agent framework Incorporating Socratic guidance (MARS), which utilizes multi-agent fusion technology for automatic planning, with gradual continuous optimization and evaluation. Specifically, MARS comprises seven agents, each with distinct functionalities, which autonomously use the Planner to devise an optimization path that ensures flexibility. Additionally, it employs a Teacher-Critic-Student Socratic dialogue pattern to iteratively optimize the prompts while conducting effective search. We conduct extensive experiments on various datasets to validate the effectiveness of our method, and perform additional analytical experiments to assess the model's advancement as well as the interpretability.