Using gen AI to translate medical jargon in discharge notes

March 13, 2024 - Using too much medical jargon in discharge notes may not be conducive to patient engagement, but researchers from NYU Langone are exploring how generative AI can fix that problem.

In a new study published in JAMA Network Open, the researchers showed that generative AI can effectively lower the reading level of discharge notes from an 11th-grade reading level to a sixth-grade level. Most experts agree that patient engagement and education materials should be written at a sixth-grade reading level in consideration of varying health literacy levels.

Discharge notes are essential patient education tools. Around half of patients forget their treatment plans after they leave the hospital or clinic, so these documents are a key reference point for those working to manage their recovery or a chronic illness at home.

But discharge notes aren’t helpful when they are filled with too much medical jargon, most of which is not understandable to the layperson, the NYU Langone researchers said. Previous studies have shown that too much medical jargon in clinical notes can be confusing to patients, and it can even make it hard for patients to understand their own health status.

Generative AI could be a solution, the researchers found.

Using GPT-4, which is the latest tool from OpenAI, the researchers transformed discharge summaries for 50 patients discharged from the General Internal Medicine service at NYU into a patient-friendly format. Overall, the technology was successful.

After running the clinical notes through GPT-4, the researchers said notes dropped from an 11th-grade reading level to a sixth-grade reading level. Providing clinical guidance and patient education at a sixth-grade reading level is considered the gold standard in patient engagement, the researchers pointed out.

The augmented discharge summaries also scored higher on the Patient Education Materials Assessment (PEMAT) tool, getting an understandability score of 81 percent, up from 13 percent on the original summaries.

Notably, improving the understandability of the clinical notes did not compromise accuracy, the data showed. Two independent reviewing physicians gave 54 percent of AI-generated notes the highest possible accuracy rating. Another 56 percent were deemed entirely complete.

“Having more than half of the AI reports generated being accurate and complete is an amazing start,” Jonah Zaretsky, MD, associate chief of medicine at NYU Langone Hospital—Brooklyn, the study’s first author, said in a statement. “Even at the current level of performance, which we expect to improve shortly, the scores achieved by the AI tool suggest that it can be taught to recognize subtleties.”

Of course, clinician oversight of the tool will be required, said Jonah Feldman, MD, the medical director of clinical transformation and informatics at NYU Langone’s Medical Center Information Technology (MCIT) Department of Informatics.

“GPT-4 worked well alone, with some gaps in accuracy and completeness, but did more than well enough to be highly effective when combined with physician oversight, the way it would be used in the real world,” Feldman, also the study’s senior author, said in the press release. “One focus of the study was on how much work physicians must do to oversee the tool, and the answer is very little. Such tools could reduce patient anxiety even as they save each provider hours each week in medical paperwork, a major source of burnout.”

In addition to reviewing any changes that AI may make to a clinical note, clinicians should be in charge of prompting the AI in the first place. Prompting is a nuanced art, the researchers suggested. Depending on how someone prompts an AI chatbot, the end result could vary.

Because clinicians are intimately familiar with their patients’ cases, they are best equipped to prompt AI to make discharge summaries more understandable.

These findings come as healthcare experts continue to explore the potential applications of generative AI in healthcare. In the patient engagement space, AI-powered chatbots are proving effective at answering some basic questions and taking some inbox burden off of providers.

This latest insight shows that the technology can also help providers better tailor their patient education efforts.