A recording device is being used to improve communication between medical residents and preceptors at Mount Sinai.

Precision medicine, which uses an individual patient’s genetic information to tailor treatment, has shown evidence of improving outcomes, particularly for those who don’t respond well to standard treatment. Mount Sinai educators are now exploring whether the same concept of precision can be applied to the way medical training is carried out, leading to improvements in learning efficiency.

Just as no two patients are the same, teachers and learners all differ in the way they teach, mentor, learn, and process information. A team of medical educators at the Icahn School of Medicine at Mount Sinai is developing a framework for “precision education,” leveraging artificial intelligence (AI) to establish evidence-based metrics for how mentors and trainees operate in a learning setting.

The project leverages new technologies, including AI analysis, to analyze the human-defined metrics of how patients, residents, and their attendings/preceptors communicate. A goal is to study communication patterns and explore whether certain approaches are associated with differences in patient engagement or care outcomes. Residents and their preceptors would then use these insights to try to adjust their communication styles to better match those of the people they are interacting with.

“Precision education is a relatively new concept—it started to be discussed in the post-COVID era—and in a sense, builds on the success of precision medicine,” says Deborah Edelman, MD, Associate Program Director of the Internal Medicine Residency, Mount Sinai Morningside-West.

Deborah Edelman, MD, Associate Program Director of the Internal Medicine Residency, Mount Sinai Morningside-West

“When people start residency, they don’t all start with the same levels of information, and they don’t all take the same journey to get to the end,” says Dr. Edelman. Almost all teaching programs—in medicine and other fields—aim to account for the middle of the bell curve. “Wouldn’t it be great if we can target all parts of the curve and help everyone maximize education?”

The framework that Dr. Edelman’s team is building is backed by a $1.1 million grant from the American Medical Association (AMA), which has tasked 11 institutions to develop precision education systems over four years. The grant program stems from AMA’s ChangeMedEd initiative, aimed at innovating medical education across the United States.

Linking communication styles to outcomes

Assessing the effectiveness of residency training starts with examining how feedback occurs. Currently, this is conducted with feedback forms.

“It works fine, but there are definitely areas for improvement,” says Dr. Edelman. “It can be biased—affected by whether a trainee likes the faculty member or not. The feedback is very subjective and variable, and if multiple suggestions conflict, it makes it hard for the faculty member to know what to change or improve.”

The team from the Icahn School of Medicine, as part of its AMA grant proposal, has designed a system that uses a recording device, such as a cell phone secured with HIPAA protections, to capture how residents interact with patients, and how residents discuss that patient interaction with their attending mentors.

That information is deidentified and parsed out by a large language model into structured variables relevant to the mentor, trainee, and patient. These could include:

  • Linguistic: talk time balance between parties, interruptions, word complexity, or sentence length
  • Patient outcomes: medication adherence, preventive care uptake, or visit adherence and continuity
  • Faculty assessment: Accreditation Council for Graduate Medical Education surveys, or narrative feedback
  • Demographic: race, gender, language preference, or socioeconomic status

“The idea is to gain actionable insights into how people communicate, and see how different combinations of variables are linked to patient outcomes,” says Dr. Edelman, who is also Associate Professor of Medicine (General Internal Medicine) at the Icahn School of Medicine. “When we have a collection of metrics, we can start to form phenotypes of how a person communicates, and from that point, it’s easier to see what works and what has room to improve.”

An overview of the Mount Sinai proposal for its precision education system, which uses ambient listening to improve communication skills (click here to view a larger image).

It’s important to highlight that the framework isn’t meant to cast judgment on any one communication style over others, notes Dr. Edelman. The framework is meant to demystify the process of communication by linking it to results, and to acknowledge the individual nature of each teacher, learner, or patient.

Scaling from Mount Sinai to nationwide

A pilot is underway to test the ambient listening system with Mount Sinai residents in OB/GYN and internal medicine programs. The pilot is born from a collaboration between the Departments of Artificial Intelligence and Human Health, Graduate Medical Education (GME), Digital and Technology Partners, and the various clinical departments at Mount Sinai to ensure patient information is handled safely and ethically.

“Our programs care for some of New York City’s most underserved populations, and we are committed to developing tools to advance health equity,” says Dr. Edelman.

Andrea Schecter, MD, Medical Director, Ambulatory Practices, Raquel and Jaime Gilinski Department of Obstetrics, Gynecology and Reproductive Science, Mount Sinai Health System (right), with Madeleine Reznik, MD, PGY-2 Internal Medicine resident, Mount Sinai Morningside and Mount Sinai West (left), trialing a precepting session with a recording device as part of the proposed precision education framework. As residents engage in their one-on-one sessions with their preceptors, their communication styles are recorded and analyzed by large language models.

Dr. Edelman (right) is in the process of assessing ideal recording setups for different interaction types, such as using an external microphone in a trial precepting session with Alexandria Still, MD, PGY-4 Internal Medicine and Chief Resident, Mount Sinai Morningside and Mount Sinai West (left), to improve recording quality.

A road map for the four years has been drawn out, with system development and testing in the first year. This will be followed by a staggered rollout and data collection in the second year, system refinement and data analysis in the third year, and scaling for dissemination to GME programs across the country in the final year.

“Precision education could be a big step for medical education, just as precision medicine has been for patients,” says Dr. Edelman. “Everybody wants to feel seen and heard. And when we have a system that is set up to listen to them and ties their communication to evidence-based metrics, nobody has their time wasted.”