By Deborah Borfitz
April 7, 2022 | Scientists at the Mayo Clinic are applying artificial intelligence (AI) to pick up auditory signals of heart health undetectable by the human ear, notably subtle acoustic changes that might not be intuitive or formally taught in medical school or during residency and fellowship training. Their vocal biomarker is based on the supposition that cardiovascular disease is likely expressed throughout the body at large, including the phonatory system, according to cardiology fellow Jaskanwal Deep Singh Sara, M.D.
Sara has been working on the voice signal analysis technology, together with cardiologist Amir Lerman, M.D., since he came to the Mayo Clinic eight years ago. But enthusiasm around the approach in the broader medical community ramped up considerably during the pandemic along with increased adoption of remote healthcare and telemedicine, he says.
In a study Sara presented at the recent American College of Cardiology’s 71st Annual Scientific Session, the AI-based voice analysis system was used for the first time to predict coronary artery disease (CAD) outcomes in 108 patients referred for a coronary angiogram who were prospectively tracked after an initial screening. Each patient recorded three, 30-second voice samples (one read from a prepared text and the others where they spoke freely about a positive and negative experience) that were analyzed by a smartphone app developed by Israel startup Vocalis Health.
Participants were tracked for an average of two years and those with a high voice biomarker score were found to be significantly more likely to suffer major problems associated with CAD, including visiting the hospital for chest pain or suffering acute coronary syndrome relative to those who had a low score, Sara reports. The high scorers were also more likely to have a positive stress test or be diagnosed with CAD during a subsequent angiogram.
The AI algorithm will never replace a clinical examination or conventional wisdom, he says, but it might one day be used as an adjunct to existing strategies. It might also be used for screening people for heart disease, who can then be referred for more comprehensive follow-up evaluation.
A more exciting possibility, from Sara’s standpoint, is that individuals with known heart disease could take serial recordings of their voice that over time might reveal signs of trouble—patients with heart failure experiencing decompensation, for instance, or those with CAD getting increased ischemia. Based on the longitudinal mapping, physicians would know when to up-titrate patients’ therapy or if they need to come in for a visit to sort out a problem in person.
Systemic Issues
When doing physical exams and taking patient histories, physicians have always instinctively listened for diagnostic “clues” based on the way individuals speak in terms of intonation, volume, and pitch, as well as the sort of language they use and their speech tempo, says Sara. In fact, the inspiration to try to quantify those signals for assessing heart health came from research looking at biomarkers for neurological and psychiatric conditions where vocal effects at first blush seem more plausible.
In the cardiovascular arena, word choice and speech cadence are likely to matter less than in diseases such as Alzheimer’s and Parkinson’s, he continues. What’s probably more relevant is the sound quality, including the pitch, frequency, and amplitude of the voice.
Sara’s first study, conducted with collaborators from Mayo Clinic and Vocalis Health (then called Beyond Verbal Communications) and published in Mayo Clinic Proceedings (DOI:10.1016/j.mayocp.2017.12.025), found that six individual voice features corresponded with baseline CAD. To make vocal signal detection less cumbersome, he says, the research team then integrated more than 200 relevant and extractable voice features into a single biomarker and trained the algorithm to analyze more than 80 of them on a population of more than 10,000 patients in Israel with a variety of chronic diseases (mainly heart failure).
In the next study, researchers used the pre-identified and pre-trained voice biomarker to learn it was also associated with pulmonary hypertension severity, continues Sara. This was followed by the latest study where one-third of patients were categorized as having a high score on a scale of -1 to 1.
Separately, the same voice biomarker was used by an outside group who found an additional association with heart failure hospitalizations and mortality, he adds. Collectively, study findings all point to the conclusion that the algorithm may be picking up something more systemic rather than what’s unique to individual disease states. “We really don’t know yet.”
Systemic problems such as dysregulated inflammation, an abnormal autonomic nervous system, and atherosclerosis are common across heart failure, pulmonary hypertension, and CAD, notes Sara. “It might be what we are picking up on is disease of the heart occurring in parallel with disease elsewhere in the body… that is directly affecting the voice box.”
Mayo Voice Protocol?
Before the voice analysis algorithm could potentially be useful in the clinic, it needs to be studied in larger, more diverse populations and across more disease types to see if it has an impact on clinical outcomes when used to make treatment decisions, Sara says. “We also need to learn a bit more about [the underlying mechanism] we’re picking up on, so we have a greater intuitive understanding and confidence in what we’re doing when relying on it [as a decision support tool].”
Countering expected resistance to the idea that the voice can be a barometer of cardiovascular health is “where a lot of our work is ahead of us,” Sara says. “We have to start to understand what this technology is deciphering with someone with coronary artery disease and for someone with pulmonary hypertension. Then I think we will get more buy-in and a chance to use the technology clinically.”
What the Mayo team doesn’t want is to see their findings overstated and turned into a self-diagnosis tool. That, says Sara, “could lead to a lot of problems with false-positives and false-negatives, inappropriate referrals, undue patient anxiety, [and] unnecessary testing.”
The intent here is to have the voice biomarker used in the right setting in appropriately selected patients, under the supervision of providers who understand the limitations and the advantages of the technology, he adds. “I don’t think this could be a standalone tool and I don’t think we should aim for it to be used as an independent commercial technology because, in its current form, it would cause more problems than benefits.”
As Sara imagines it, the voice technology could perhaps be incorporated into a treatment algorithm as a remote healthcare tool associated with, say, a “Mayo voice protocol” that could be the subject of a pragmatic trial to learn if it improves patient outcomes, reduces cost, and enhances satisfaction and quality of life. If so, it could then be tried in larger populations to see if the findings hold.
The Mayo Clinic has embraced AI in a big way over the past few years, well after Sara and his colleagues began working on the voice analysis technology. It was a “fortunate… convergence” that the two domains coincidentally came together alongside COVID-19, helping to propel interest and progress, he says.