Clinical knowledge in LLMs does not translate to human interactions

Summary

This paper talks about how large language models (LLMs), which are really good at passing medical exams, still struggle when it comes to actually giving helpful medical advice to real people.

What's the problem?

The problem is that even though these AI models know a lot of medical facts and can answer test questions, they often can't communicate clearly or safely with regular people who are looking for medical help. This means their advice might be confusing, incomplete, or even wrong when someone needs real guidance.

What's the solution?

The researchers tested these language models by having them interact with people who asked for medical advice, and they found that the models didn't do as well as expected. The study highlights that just knowing the facts isn't enough—the models also need to understand how to talk to people in a way that is accurate, clear, and sensitive to their needs.

Why it matters?

This matters because it shows that AI isn't ready to replace doctors or nurses when it comes to helping patients directly, and that more work is needed to make sure these systems can interact safely and effectively with people in real-life situations.