This Is Your Doge, If It Please You: Exploring Deception and Robustness in Mixture of LLMs
Lorenz Wolf, Sangwoong Yoon, Ilija Bogunovic
2025-03-11
Summary
This paper talks about how teams of AI language models (working together) can be tricked by one bad AI into giving wrong answers, and how to fix it using ideas from old voting systems.
What's the problem?
When multiple AI models team up to solve problems, just one sneaky AI that lies can ruin the whole group’s answers, making them much worse than a single honest AI.
What's the solution?
The researchers borrowed ideas from an old Venetian voting system (where leaders were picked randomly to stop cheating) to create defenses that block lying AIs from influencing the group’s decisions.
Why it matters?
This helps keep AI teams reliable and safe, so they can solve problems together without being fooled by bad actors, which is crucial for things like medical advice or legal help.
Abstract
Mixture of large language model (LLMs) Agents (MoA) architectures achieve state-of-the-art performance on prominent benchmarks like AlpacaEval 2.0 by leveraging the collaboration of multiple LLMs at inference time. Despite these successes, an evaluation of the safety and reliability of MoA is missing. We present the first comprehensive study of MoA's robustness against deceptive LLM agents that deliberately provide misleading responses. We examine factors like the propagation of deceptive information, model size, and information availability, and uncover critical vulnerabilities. On AlpacaEval 2.0, the popular LLaMA 3.1-70B model achieves a length-controlled Win Rate (LC WR) of 49.2% when coupled with 3-layer MoA (6 LLM agents). However, we demonstrate that introducing only a single carefully-instructed deceptive agent into the MoA can reduce performance to 37.9%, effectively nullifying all MoA gains. On QuALITY, a multiple-choice comprehension task, the impact is also severe, with accuracy plummeting by a staggering 48.5%. Inspired in part by the historical Doge of Venice voting process, designed to minimize influence and deception, we propose a range of unsupervised defense mechanisms that recover most of the lost performance.