PANORAMA: The Rise of Omnidirectional Vision in the Embodied AI Era
Xu Zheng, Chenfei Liao, Ziqiao Weng, Kaiyu Lei, Zihao Dongfang, Haocong He, Yuanhuiyi Lyu, Lutao Jiang, Lu Qi, Li Chen, Danda Pani Paudel, Kailun Yang, Linfeng Zhang, Luc Van Gool, Xuming Hu
2025-09-18
Summary
This paper discusses the growing importance of omnidirectional vision – basically, cameras that can see in all directions at once – and how it's becoming more popular with the rise of 'embodied AI', which is AI that interacts with the real world.
What's the problem?
Traditionally, most computer vision research has focused on standard cameras that have a limited field of view, like our own eyes. While those work well, they don't give a complete picture of the surroundings. Omnidirectional vision *should* be better because it provides a full 360-degree view, but research in this area hasn't kept pace with traditional camera technology, leaving a gap in our ability to build AI systems that truly understand their environment.
What's the solution?
The paper reviews recent progress in several areas of omnidirectional vision, including creating realistic images, interpreting what those images mean, and understanding the overall scene. They also propose a system called PANORAMA, which outlines how all the different parts of an omnidirectional vision system should work together for AI applications. It's a blueprint for building better AI that can 'see' everything around it.
Why it matters?
This work is important because as AI systems become more common in things like robots and automated inspections, they need to be able to reliably understand their surroundings. Omnidirectional vision offers a more complete and robust way to do that, and this paper helps chart a course for future research and development in this field, ultimately leading to more capable and safer AI systems.
Abstract
Omnidirectional vision, using 360-degree vision to understand the environment, has become increasingly critical across domains like robotics, industrial inspection, and environmental monitoring. Compared to traditional pinhole vision, omnidirectional vision provides holistic environmental awareness, significantly enhancing the completeness of scene perception and the reliability of decision-making. However, foundational research in this area has historically lagged behind traditional pinhole vision. This talk presents an emerging trend in the embodied AI era: the rapid development of omnidirectional vision, driven by growing industrial demand and academic interest. We highlight recent breakthroughs in omnidirectional generation, omnidirectional perception, omnidirectional understanding, and related datasets. Drawing on insights from both academia and industry, we propose an ideal panoramic system architecture in the embodied AI era, PANORAMA, which consists of four key subsystems. Moreover, we offer in-depth opinions related to emerging trends and cross-community impacts at the intersection of panoramic vision and embodied AI, along with the future roadmap and open challenges. This overview synthesizes state-of-the-art advancements and outlines challenges and opportunities for future research in building robust, general-purpose omnidirectional AI systems in the embodied AI era.