General Reasoning Requires Learning to Reason from the Get-go
Seungwook Han, Jyothish Pari, Samuel J. Gershman, Pulkit Agrawal
2025-03-04
Summary
This paper talks about how to improve AI systems so they can reason better and handle new problems more effectively, which is a key step toward creating truly intelligent machines.
What's the problem?
While large AI models are good at tasks like math, programming, and commonsense reasoning, they often fail when faced with new or unfamiliar situations. This happens because their reasoning abilities are too closely tied to the specific data they were trained on, making it hard for them to adapt to new challenges.
What's the solution?
The researchers suggest separating reasoning skills from knowledge in AI models. They propose three main ideas: training AI to reason from scratch using reinforcement learning instead of just predicting the next word, giving the AI practice with simpler tasks first to build reasoning skills that can be applied to harder tasks, and designing the AI to focus on smaller chunks of information to avoid relying on patterns that don’t generalize well. They also suggest combining this reasoning system with a large external memory that stores knowledge the AI can retrieve when needed.
Why it matters?
This matters because it could help create AI systems that are not only useful but also capable of solving new and complex problems in a way similar to human intelligence. By improving how AI learns to reason, this research could lead to smarter systems that can adapt to a wider range of real-world challenges.
Abstract
Large Language Models (LLMs) have demonstrated impressive real-world utility, exemplifying artificial useful intelligence (AUI). However, their ability to reason adaptively and robustly -- the hallmarks of artificial general intelligence (AGI) -- remains fragile. While LLMs seemingly succeed in commonsense reasoning, programming, and mathematics, they struggle to generalize algorithmic understanding across novel contexts. Our experiments with algorithmic tasks in esoteric <PRE_TAG>programming languages</POST_TAG> reveal that LLM's reasoning overfits to the training data and is limited in its transferability. We hypothesize that the core issue underlying such limited transferability is the coupling of reasoning and knowledge in LLMs. To transition from AUI to AGI, we propose disentangling knowledge and reasoning through three key directions: (1) pretaining to reason using RL from scratch as an alternative to the widely used next-token prediction pretraining, (2) using a curriculum of synthetic tasks to ease the learning of a reasoning prior for RL that can then be transferred to natural language tasks, and (3) learning more generalizable reasoning functions using a small context window to reduce exploiting spurious correlations between tokens. Such a reasoning system coupled with a trained retrieval system and a large external memory bank as a knowledge store can overcome several limitations of existing architectures at learning to reason in novel scenarios.