Perplexity Trap: PLM-Based Retrievers Overrate Low Perplexity Documents
Haoyu Wang, Sunhao Dai, Haiyuan Zhao, Liang Pang, Xiao Zhang, Gang Wang, Zhenhua Dong, Jun Xu, Ji-Rong Wen
2025-03-12
Summary
This paper talks about how AI search tools unfairly favor AI-generated text over human-written content because it’s easier for the model to process, even when both are equally good.
What's the problem?
AI search engines give higher rankings to AI-made documents just because they’re simpler to read for the AI, which pushes human content lower even when it’s just as useful.
What's the solution?
Researchers created CDC, a tool that measures how much the AI’s rankings are skewed by text simplicity and then adjusts the scores to prioritize content quality over ease of processing.
Why it matters?
This helps keep search results fair as AI-generated content grows, ensuring useful human-made info doesn’t get buried and maintaining trust in online information.
Abstract
Previous studies have found that PLM-based retrieval models exhibit a preference for LLM-generated content, assigning higher relevance scores to these documents even when their semantic quality is comparable to human-written ones. This phenomenon, known as source bias, threatens the sustainable development of the information access ecosystem. However, the underlying causes of source bias remain unexplored. In this paper, we explain the process of information retrieval with a causal graph and discover that PLM-based retrievers learn perplexity features for relevance estimation, causing source bias by ranking the documents with low perplexity higher. Theoretical analysis further reveals that the phenomenon stems from the positive correlation between the gradients of the loss functions in language modeling task and retrieval task. Based on the analysis, a causal-inspired inference-time debiasing method is proposed, called Causal Diagnosis and Correction (CDC). CDC first diagnoses the bias effect of the perplexity and then separates the bias effect from the overall estimated relevance score. Experimental results across three domains demonstrate the superior debiasing effectiveness of CDC, emphasizing the validity of our proposed explanatory framework. Source codes are available at https://github.com/WhyDwelledOnAi/Perplexity-Trap.