Exploring the Inquiry-Diagnosis Relationship with Advanced Patient Simulators
Zhaocheng Liu, Quan Tu, Wen Ye, Yu Xiao, Zhishou Zhang, Hengfu Cui, Yalun Zhu, Qiang Ju, Shizheng Li, Jian Xie
2025-01-17
Summary
This paper talks about improving online medical consultations by creating a realistic patient simulator and studying how doctors ask questions and make diagnoses. The researchers wanted to understand how the quality of questions asked affects the accuracy of diagnoses in virtual healthcare settings.
What's the problem?
In online medical consultations, doctors can't physically examine patients and have to rely entirely on asking questions to gather information. This makes diagnosing illnesses even harder than usual. While new AI language models show promise in helping with online consultations, most research has focused on improving diagnoses rather than looking at how doctors ask questions. This means we don't fully understand how the quality of questions affects the accuracy of diagnoses in virtual healthcare.
What's the solution?
The researchers created a super realistic 'virtual patient' by studying real conversations between doctors and patients. They used this virtual patient to simulate lots of medical consultations. They then looked at how different ways of asking questions affected the accuracy of diagnoses. They also categorized the types of questions doctors ask into four main groups to understand why some approaches work better than others.
Why it matters?
This research matters because as virtual healthcare becomes more common, we need to make sure doctors can diagnose patients accurately without seeing them in person. By understanding how asking good questions leads to better diagnoses, we can train doctors and AI systems to provide better online healthcare. This could make virtual doctor visits more effective, potentially improving healthcare access for people who can't easily visit a doctor in person.
Abstract
Online medical consultation (OMC) restricts doctors to gathering patient information solely through inquiries, making the already complex sequential decision-making process of diagnosis even more challenging. Recently, the rapid advancement of large language models has demonstrated a significant potential to transform OMC. However, most studies have primarily focused on improving diagnostic accuracy under conditions of relatively sufficient information, while paying limited attention to the "inquiry" phase of the consultation process. This lack of focus has left the relationship between "inquiry" and "diagnosis" insufficiently explored. In this paper, we first extract real patient interaction strategies from authentic doctor-patient conversations and use these strategies to guide the training of a patient simulator that closely mirrors real-world behavior. By inputting medical records into our patient simulator to simulate patient responses, we conduct extensive experiments to explore the relationship between "inquiry" and "diagnosis" in the consultation process. Experimental results demonstrate that inquiry and diagnosis adhere to the Liebig's law: poor inquiry quality limits the effectiveness of diagnosis, regardless of diagnostic capability, and vice versa. Furthermore, the experiments reveal significant differences in the inquiry performance of various models. To investigate this phenomenon, we categorize the inquiry process into four types: (1) chief complaint inquiry; (2) specification of known symptoms; (3) inquiry about accompanying symptoms; and (4) gathering family or medical history. We analyze the distribution of inquiries across the four types for different models to explore the reasons behind their significant performance differences. We plan to open-source the weights and related code of our patient simulator at https://github.com/LIO-H-ZEN/PatientSimulator.