The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1
Kaiwen Zhou, Chengzhi Liu, Xuandong Zhao, Shreedhar Jangam, Jayanth Srinivasa, Gaowen Liu, Dawn Song, Xin Eric Wang
2025-02-19
Summary
This paper talks about the safety risks associated with advanced AI models called large reasoning models, particularly focusing on OpenAI-o3 and DeepSeek-R1. These models are really good at solving complex problems, but they might also be dangerous if misused.
What's the problem?
While these new AI models are much better at reasoning and problem-solving than older ones, they also come with serious safety concerns. The researchers worry that because some of these models are open-source (meaning anyone can access and use them), they could be used for harmful purposes. It's like having a super-smart robot that can solve difficult puzzles, but also potentially figure out how to break into a bank vault if someone asks it to.
What's the solution?
The researchers did a thorough safety check on these AI models. They used special tests to see how well the models follow safety rules and how easily they can be tricked into doing bad things. They looked at everything from how the models think through problems to their final answers. This helped them find four main issues, including that some models are much less safe than others and that the smarter the AI gets, the more dangerous it could be if it's asked to do something harmful.
Why it matters?
This matters because as AI gets smarter, we need to make sure it's also safe. The study shows that even though these new AI models are really impressive, they could be dangerous if they fall into the wrong hands. It's like creating a powerful new technology - we need to understand both its benefits and its risks. The researchers' work helps us see where we need to improve the safety of these AI models, which is crucial as they become more common in our everyday lives.
Abstract
The rapid development of large reasoning models, such as OpenAI-o3 and DeepSeek-R1, has led to significant improvements in complex reasoning over non-reasoning large language models~(LLMs). However, their enhanced capabilities, combined with the open-source access of models like DeepSeek-R1, raise serious safety concerns, particularly regarding their potential for misuse. In this work, we present a comprehensive safety assessment of these reasoning models, leveraging established safety benchmarks to evaluate their compliance with safety regulations. Furthermore, we investigate their susceptibility to adversarial attacks, such as jailbreaking and prompt injection, to assess their robustness in real-world applications. Through our multi-faceted analysis, we uncover four key findings: (1) There is a significant safety gap between the open-source R1 models and the o3-mini model, on both safety benchmark and attack, suggesting more safety effort on R1 is needed. (2) The distilled reasoning model shows poorer safety performance compared to its safety-aligned base models. (3) The stronger the model's reasoning ability, the greater the potential harm it may cause when answering unsafe questions. (4) The thinking process in R1 models pose greater safety concerns than their final answers. Our study provides insights into the security implications of reasoning models and highlights the need for further advancements in R1 models' safety to close the gap.