TabDSR: Decompose, Sanitize, and Reason for Complex Numerical Reasoning in Tabular Data

Summary

This paper introduces a new system, called \method, designed to help large language models (LLMs) get better at answering complex questions based on information found in tables, like spreadsheets or databases.

What's the problem?

LLMs often struggle with tasks that require reasoning about data in tables. This is because questions can be complicated, the data itself might have errors or irrelevant information, and LLMs aren't naturally good at doing math or precise numerical calculations. Basically, they have trouble turning table data into accurate answers for tricky questions.

What's the solution?

The researchers created \method, which works in three main steps. First, it breaks down a complicated question into smaller, more manageable parts. Second, it cleans up the table by removing errors and focusing on the most relevant data. Finally, it uses a 'program-of-thoughts' approach, meaning it generates code that can be executed to perform calculations and find the final answer from the cleaned-up table. They also created a new dataset, CalTab151, to fairly test these kinds of systems and prevent cheating.

Why it matters?

This work is important because it significantly improves the ability of LLMs to handle real-world data analysis tasks. By making LLMs better at understanding and reasoning with tabular data, we can unlock their potential for a wider range of applications, like financial analysis, scientific research, and business intelligence. The system outperforms existing methods and works well with many different LLMs.