FactAlign: Long-form Factuality Alignment of Large Language Models
Chao-Wei Huang, Yun-Nung Chen
2024-10-03
Summary
This paper discusses FactAlign, a new framework designed to improve the accuracy of long responses generated by large language models (LLMs) by ensuring they provide factual information.
What's the problem?
Large language models can generate text that sounds good but often includes incorrect or misleading information, especially in longer responses. This issue, known as 'hallucination,' makes it hard to trust the information these models provide, which is particularly problematic for users seeking reliable answers.
What's the solution?
To solve this problem, the authors introduced FactAlign, which uses a method called fKTO to align the model's output with factual information at a sentence level. This means that instead of just checking if the overall response is correct, the model evaluates each sentence to ensure it is based on true facts. The researchers tested FactAlign on various tasks and found that it significantly improved the factual accuracy of the responses while still being helpful and informative.
Why it matters?
This research is important because it enhances the reliability of AI-generated content, making it safer and more trustworthy for users. As AI becomes more integrated into everyday applications like education and customer service, having accurate and factual information is crucial for maintaining user trust and ensuring effective communication.
Abstract
Large language models have demonstrated significant potential as the next-generation information access engines. However, their reliability is hindered by issues of hallucination and generating non-factual content. This is particularly problematic in long-form responses, where assessing and ensuring factual accuracy is complex. In this paper, we address this gap by proposing FactAlign, a novel alignment framework designed to enhance the factuality of LLMs' long-form responses while maintaining their helpfulness. We introduce fKTO, a fine-grained, sentence-level alignment algorithm that extends the Kahneman-Tversky Optimization (KTO) alignment method. Leveraging recent advances in automatic factuality evaluation, FactAlign utilizes fine-grained factuality assessments to guide the alignment process. Our experiments on open-domain prompts and information-seeking questions demonstrate that FactAlign significantly improves the factual accuracy of LLM responses while also improving their helpfulness. Further analyses identify that FactAlign is capable of training LLMs to provide more information without losing factual precision, thus improving the factual F1 score. Our source code, datasets, and trained models are publicly available at https://github.com/MiuLab/FactAlign