Interact, Instruct to Improve: A LLM-Driven Parallel Actor-Reasoner Framework for Enhancing Autonomous Vehicle Interactions
Shiyu Fang, Jiaqi Liu, Chengkai Xu, Chen Lv, Peng Hang, Jian Sun
2025-03-06
Summary
This paper talks about a new system called the Actor-Reasoner framework that helps self-driving cars communicate better with human drivers on the road
What's the problem?
Self-driving cars are becoming more common, but they still struggle to interact effectively with human-driven vehicles. They need to make quick decisions, but the advanced AI systems that could help them communicate are often too slow to use in real-time driving situations
What's the solution?
The researchers created a two-part system. The 'Reasoner' part uses a powerful AI to think about different driving scenarios and create a database of interactions. The 'Actor' part can quickly access this database to make decisions in real-time. They also added special modules to help the system deal with different types of human drivers and retrieve the right information quickly
Why it matters?
This matters because it could make self-driving cars safer and more efficient on the road. By improving how they interact with human drivers, it could help reduce accidents and traffic problems. This research brings us closer to a future where self-driving cars can smoothly share the road with human drivers, making transportation safer and more convenient for everyone
Abstract
Autonomous Vehicles (AVs) have entered the commercialization stage, but their limited ability to interact and express intentions still poses challenges in interactions with Human-driven Vehicles (HVs). Recent advances in large language models (LLMs) enable bidirectional human-machine communication, but the conflict between slow inference speed and the need for real-time decision-making challenges practical deployment. To address these issues, this paper introduces a parallel Actor-Reasoner framework designed to enable explicit bidirectional AV-HV interactions across multiple scenarios. First, by facilitating interactions between the LLM-driven Reasoner and heterogeneous simulated HVs during training, an interaction memory database, referred to as the Actor, is established. Then, by introducing the memory partition module and the two-layer memory retrieval module, the Actor's ability to handle heterogeneous HVs is significantly enhanced. Ablation studies and comparisons with other decision-making methods demonstrate that the proposed Actor-Reasoner framework significantly improves safety and efficiency. Finally, with the combination of the external Human-Machine Interface (eHMI) information derived from Reasoner's reasoning and the feasible action solutions retrieved from the Actor, the effectiveness of the proposed Actor-Reasoner is confirmed in multi-scenario field interactions. Our code is available at https://github.com/FanGShiYuu/Actor-Reasoner.