SRMT: Shared Memory for Multi-agent Lifelong Pathfinding
Alsu Sagirova, Yuri Kuratov, Mikhail Burtsev
2025-01-24
Summary
This paper talks about a new way to help multiple AI agents work together better, called Shared Recurrent Memory Transformer (SRMT). It's like giving a group of robots a shared notebook where they can write down and read important information to coordinate their actions.
What's the problem?
In multi-agent reinforcement learning (MARL), it's hard for AI agents to work together effectively because they need to predict what other agents will do. This is like trying to play a team sport where you can't talk to your teammates - you have to guess what they'll do next, which can lead to confusion and mistakes.
What's the solution?
The researchers created SRMT, which allows AI agents to share their 'memories' or important information with each other. Instead of trying to guess what others will do, agents can access a shared pool of information. They tested SRMT on navigation tasks where multiple agents had to find their way through mazes and narrow corridors. SRMT helped the agents coordinate better, especially in tricky situations like when they all needed to pass through a narrow space.
Why it matters?
This matters because it could make AI systems that involve multiple agents much more effective. Imagine a group of delivery robots that can silently coordinate to avoid traffic jams, or a team of virtual assistants that can work together seamlessly to solve complex problems. By improving how AI agents share information and coordinate, SRMT could lead to smarter, more efficient AI systems in various fields like robotics, logistics, and even virtual environments like video games.
Abstract
Multi-agent reinforcement learning (MARL) demonstrates significant progress in solving cooperative and competitive multi-agent problems in various environments. One of the principal challenges in MARL is the need for explicit prediction of the agents' behavior to achieve cooperation. To resolve this issue, we propose the Shared Recurrent Memory Transformer (SRMT) which extends memory transformers to multi-agent settings by pooling and globally broadcasting individual working memories, enabling agents to exchange information implicitly and coordinate their actions. We evaluate SRMT on the Partially Observable Multi-Agent Pathfinding problem in a toy Bottleneck navigation task that requires agents to pass through a narrow corridor and on a POGEMA benchmark set of tasks. In the Bottleneck task, SRMT consistently outperforms a variety of reinforcement learning baselines, especially under sparse rewards, and generalizes effectively to longer corridors than those seen during training. On POGEMA maps, including Mazes, Random, and MovingAI, SRMT is competitive with recent MARL, hybrid, and planning-based algorithms. These results suggest that incorporating shared recurrent memory into the transformer-based architectures can enhance coordination in decentralized multi-agent systems. The source code for training and evaluation is available on GitHub: https://github.com/Aloriosa/srmt.