Seeing Faces in Things: A Model and Dataset for Pareidolia
Mark Hamilton, Simon Stent, Vasha DuTell, Anne Harrington, Jennifer Corbett, Ruth Rosenholtz, William T. Freeman
2024-09-25
Summary
This paper discusses a study on pareidolia, which is when people see faces in random objects, like coffee stains or clouds. The researchers created a dataset called 'Faces in Things' to explore how well computers can detect these pareidolic faces compared to humans.
What's the problem?
Humans are naturally good at recognizing faces, even in unusual places, which can be helpful for survival. However, this ability can also lead to false detections, where we mistakenly see faces where there are none. Current computer vision systems struggle to match human performance in recognizing these pareidolic faces, which indicates a gap between human perception and machine detection.
What's the solution?
To investigate this issue, the researchers collected a dataset of 5,000 images containing pareidolic faces that were annotated by humans. They tested a state-of-the-art face detection model using this dataset and found that it did not perform as well as humans in recognizing these faces. They also developed a statistical model to predict the conditions under which pareidolia is most likely to occur and confirmed their predictions through experiments with human subjects and face detection systems.
Why it matters?
This research is important because it helps us understand how both humans and machines perceive faces in unusual contexts. By identifying the limitations of current face detection technology, the study can guide future improvements in computer vision systems. This has implications for various applications, such as improving AI's ability to interpret images in real-world scenarios and enhancing technologies used in security and surveillance.
Abstract
The human visual system is well-tuned to detect faces of all shapes and sizes. While this brings obvious survival advantages, such as a better chance of spotting unknown predators in the bush, it also leads to spurious face detections. ``Face pareidolia'' describes the perception of face-like structure among otherwise random stimuli: seeing faces in coffee stains or clouds in the sky. In this paper, we study face pareidolia from a computer vision perspective. We present an image dataset of ``Faces in Things'', consisting of five thousand web images with human-annotated pareidolic faces. Using this dataset, we examine the extent to which a state-of-the-art human face detector exhibits pareidolia, and find a significant behavioral gap between humans and machines. We find that the evolutionary need for humans to detect animal faces, as well as human faces, may explain some of this gap. Finally, we propose a simple statistical model of pareidolia in images. Through studies on human subjects and our pareidolic face detectors we confirm a key prediction of our model regarding what image conditions are most likely to induce pareidolia. Dataset and Website: https://aka.ms/faces-in-things