HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluation
Shaina Raza, Aravind Narayanan, Vahid Reza Khazaie, Ashmal Vayani, Mukund S. Chettiar, Amandeep Singh, Mubarak Shah, Deval Pandya
2025-05-22
Summary
This paper talks about HumaniBench, a new way to test how well advanced AI models that work with both pictures and text follow important human values like fairness, ethics, and empathy.
What's the problem?
As AI models get better at understanding and responding to images and questions, it's tough to make sure they treat everyone fairly, act ethically, and show understanding, especially since most tests don't focus on these human-centered qualities.
What's the solution?
The researchers created HumaniBench, which uses 32,000 real-world examples of images and related questions to check how well these AI models follow seven key principles that matter to people, such as being inclusive and ethical.
Why it matters?
This matters because it helps make sure AI systems are not just smart but also trustworthy and respectful, which is really important as they become a bigger part of our daily lives.
Abstract
HumaniBench evaluates state-of-the-art LMMs on seven human-centered AI principles using 32K real-world image-question pairs to ensure fairness, ethics, empathy, and inclusivity.