Reasoning Model is Stubborn: Diagnosing Instruction Overriding in Reasoning Models

Summary

This paper talks about how large language models can be stubborn and sometimes ignore new instructions, sticking to the way they usually solve problems even when told to do something different.

What's the problem?

The problem is that these AI models often don't follow specific instructions very well, especially when it comes to reasoning tasks. Instead, they tend to fall back on their usual methods, which can make them less flexible and less useful in situations where following directions exactly is important.

What's the solution?

The researchers created a special set of tests to see when and how these models ignore instructions. By analyzing the results, they were able to spot patterns and understand why the models act this way, which helps diagnose the issue of reasoning rigidity.

Why it matters?

This is important because if we want AI to be more helpful and reliable, especially in situations where it needs to follow specific steps or rules, we need to understand and fix this stubbornness so the models can adapt better to new instructions.