Therefore I am. I Think

Summary

This research investigates whether large language models actually *think* through a problem before deciding on an answer, or if they make a decision first and then create a justification for it afterward.

What's the problem?

We often assume that when a language model gives a detailed, step-by-step explanation (called 'chain-of-thought' reasoning), it's genuinely working through the problem logically. However, it's unclear if the model first arrives at a conclusion and *then* generates the reasoning to support it, or if the reasoning process truly leads to the decision. The core question is: does the decision come before or after the thinking?

What's the solution?

The researchers found strong evidence that the decision is often made very early in the process, even before the model starts generating the text of its reasoning. They used a technique to 'read' the model's internal activity (activations) and accurately predict what decision the model would make, sometimes even before a single word of explanation was produced. They also directly manipulated these internal signals, and found that changing the 'decision direction' caused the model to change its answer and then adjust its reasoning to fit the new answer, rather than sticking to its original thought process.

Why it matters?

This is important because it challenges our understanding of how these powerful language models work. If models are primarily justifying pre-existing decisions rather than truly reasoning, it raises concerns about their reliability and potential for bias. It suggests that the impressive 'thinking' we see might be more of a sophisticated storytelling ability than genuine problem-solving.