ChatGPT is capable of providing more precise answers to patients’ questions than doctors

A new study by researchers at the University of California has found that an artificial intelligence chatbot named ChatGPT can provide more empathetic and higher-quality advice to patients’ questions than doctors. The study, published in JAMA Internal Medicine, compared written responses from doctors and ChatGPT to real-world health questions from the social media platform Reddit’s r/AskDocs community. A panel of licensed healthcare professionals preferred ChatGPT’s responses 79% of the time and rated them higher in quality and more empathetic.

While the study suggests that AI assistants like ChatGPT can be integrated into health systems to improve doctors’ responses to patient questions, the researchers emphasized that AI assistants are not intended to replace doctors. Instead, they believe that doctors working together with technologies like ChatGPT may revolutionize medicine. Dr John W Ayers from the Qualcomm Institute within the University of California San Diego, who led the study, said that “AI-augmented care is the future of medicine.”

The team turned to Reddit to obtain a large and diverse sample of healthcare questions and doctors’ answers that do not contain identifiable personal information. The researchers randomly sampled 195 exchanges from AskDocs where a verified doctor responded to a public question. They provided the original question to ChatGPT and asked it to author a response. A panel of three licensed healthcare professionals assessed each question and the corresponding responses and were blinded to whether the response originated from a doctor or ChatGPT. The study shows promise for AI assistants in healthcare, but the researchers emphasized the need for integrating AI assistants into healthcare messaging to be done in the context of a randomized controlled trial.