Akhil Vaid, MD, left, and Girish Nadkarni, MD, MPH, right, are working to make artificial intelligence models more feasible for reading electrocardiograms, using a novel transformer neural network approach.
However, most AI models for ECG analysis use a particular deep learning method called convolutional neural networks (CNNs). CNNs require large training datasets to make diagnoses, which spell limitations when it comes to rare heart diseases that do not have a wealth of data.
Researchers at the Icahn School of Medicine at Mount Sinai have developed an AI model, called HeartBEiT, for ECG analysis, which works by interpreting ECGs as language.
The model uses a transformer-based neural network, a class of network that is unlike conventional networks but does serve as a basis for popular generative language models, such as ChatGPT.
Here’s how HeartBEiT works as an artificial intelligence deep-learning model, and how it compares to CNNs.
In this Q&A, Dr. Vaid discusses the impact of this new AI model on reading ECGs.
What was the motivation for your study?
Deep learning as applied to ECGs has had much success, but most deep learning studies for ECGs use convolutional neural networks, which have limitations.
Recently, the transformer class of models has assumed a position of importance. These models function by establishing relationships between parts of the data they see. Generative transformer models such as the popular ChatGPT utilize this understanding to generate plain-language text.
By using another generative image model, HeartBEiT creates representations of the ECG that may be considered “words,” and the whole ECG may be considered a single “document.” HeartBEiT understands the relationship between these words within the context of the document, and uses this understanding to perform diagnostic tasks better.
Our model forms a universal starting point for any ECG-based study. When comparing our model to popular CNN architectures on diagnostic tasks, HeartBEiT ended up with equivalent performance and better explanations for the model’s thinking and choices using as little as a tenth of the data required by other approaches.
Additionally, HeartBEiT generates very specific explanations of which parts of an ECG were most responsible for pushing a model towards making a diagnosis.
What are the limitations of the study?
Pre-training the model takes a fair amount of time. However, fine-tuning it for a specific diagnosis is a very quick process that can be accomplished in a few minutes.
Deployment of this model and its derivatives into clinical practice can greatly enhance the manner in which clinicians interact with ECGs. We are no longer limited to models for commonly seen conditions, since the paradigm can be extended to nearly any pathology.
What is your plan for following up on this study?
We intend to scale up the model so that it can capture even more detail. We also intend to validate this approach externally, in places outside Mount Sinai.
Learn more about how Mount Sinai researchers and clinicians are leveraging machine learning to improve patient lives