BERT-VBD: Vietnamese Multi-Document Summarization Framework
Tuan-Cuong Vuong, Trang Mai Xuan, Thien Van Luong
2024-09-19
Summary
This paper discusses BERT-VBD, a new framework for summarizing multiple documents in Vietnamese by combining two different summarization techniques: extractive and abstractive methods.
What's the problem?
Summarizing information from multiple documents can be challenging because existing methods often rely on either extractive summarization (which pulls out key sentences from the text) or abstractive summarization (which rewrites the content in a more concise way). Each method has its own limitations, and there hasn't been much research on combining these techniques, especially for the Vietnamese language.
What's the solution?
The authors developed a two-part framework that first uses an extractive approach to identify important sentences from each document using a modified BERT model. Then, it applies an abstractive summarization model called VBD-LLaMA2-7B-50b to create a final summary. This combination allows the framework to leverage the strengths of both methods, resulting in better summaries. The framework achieved impressive results, scoring 39.6% on ROUGE-2 metrics when tested on the VN-MDS dataset, outperforming other existing methods.
Why it matters?
This research is important because it enhances the ability to summarize multiple documents in Vietnamese, which can help improve access to information and support better understanding of large amounts of text. By developing a framework that effectively combines different summarization techniques, it contributes to advancements in natural language processing and makes it easier for speakers of Vietnamese to engage with written content.
Abstract
In tackling the challenge of Multi-Document Summarization (MDS), numerous methods have been proposed, spanning both extractive and abstractive summarization techniques. However, each approach has its own limitations, making it less effective to rely solely on either one. An emerging and promising strategy involves a synergistic fusion of extractive and abstractive summarization methods. Despite the plethora of studies in this domain, research on the combined methodology remains scarce, particularly in the context of Vietnamese language processing. This paper presents a novel Vietnamese MDS framework leveraging a two-component pipeline architecture that integrates extractive and abstractive techniques. The first component employs an extractive approach to identify key sentences within each document. This is achieved by a modification of the pre-trained BERT network, which derives semantically meaningful phrase embeddings using siamese and triplet network structures. The second component utilizes the VBD-LLaMA2-7B-50b model for abstractive summarization, ultimately generating the final summary document. Our proposed framework demonstrates a positive performance, attaining ROUGE-2 scores of 39.6% on the VN-MDS dataset and outperforming the state-of-the-art baselines.