Optimizing Decomposition for Optimal Claim Verification
Yining Lu, Noah Ziems, Hy Dang, Meng Jiang
2025-03-20
Summary
This paper is about improving how AI checks if long pieces of writing are true by breaking them down into smaller parts.
What's the problem?
The way AI breaks down the writing and the way it checks the facts don't always work well together, leading to mistakes.
What's the solution?
The researchers created a system where the AI learns the best way to break down the writing so it can check the facts more accurately.
Why it matters?
This work matters because it can help AI better identify false information in news articles, research papers, and other long texts.
Abstract
Current research on the Decompose-Then-Verify paradigm for evaluating the factuality of long-form text typically treats decomposition and verification in isolation, overlooking their interactions and potential misalignment. We find that existing decomposition policies, typically hand-crafted demonstrations, do not align well with downstream verifiers in terms of atomicity -- a novel metric quantifying information density -- leading to suboptimal verification results. We formulate finding the optimal decomposition policy for optimal verification as a bilevel optimization problem. To approximate a solution for this strongly NP-hard problem, we propose dynamic decomposition, a reinforcement learning framework that leverages verifier feedback to learn a policy for dynamically decomposing claims to verifier-preferred atomicity. Experimental results show that dynamic decomposition outperforms existing decomposition policies, improving verification confidence by 0.07 and accuracy by 0.12 (on a 0-1 scale) on average across varying verifiers, datasets, and atomcities of input claims.