Project SHADOW: Symbolic Higher-order Associative Deductive reasoning On Wikidata using LM probing
Hanna Abi Akl
2024-08-28
Summary
This paper introduces Project SHADOW, a specialized language model designed to improve the process of constructing knowledge bases using associative reasoning on data from Wikidata.
What's the problem?
Creating knowledge bases from large amounts of data can be challenging. Traditional methods often struggle with reasoning and making connections between different pieces of information, which can lead to incomplete or inaccurate knowledge bases. Existing models may not perform well when it comes to understanding complex relationships in data.
What's the solution?
Project SHADOW addresses these issues by using a fine-tuned language model that employs associative deductive reasoning. This model is trained specifically for tasks related to knowledge base construction, allowing it to better understand and predict relationships in data. In tests, SHADOW outperformed previous models by 20%, achieving a high accuracy score (F1 score of 68.72%) in completing knowledge triples, which are essential for building structured data.
Why it matters?
This research is important because it enhances the ability to create accurate and comprehensive knowledge bases, which are crucial for applications like search engines, AI assistants, and data analysis tools. By improving how machines understand and organize information, Project SHADOW can help make data more accessible and useful for various fields.
Abstract
We introduce SHADOW, a fine-tuned language model trained on an intermediate task using associative deductive reasoning, and measure its performance on a knowledge base construction task using Wikidata triple completion. We evaluate SHADOW on the LM-KBC 2024 challenge and show that it outperforms the baseline solution by 20% with a F1 score of 68.72%.