#ACTRIMS2022 – Algorithm Predicts Relapse Risk Using EHR Data
Using a two-step machine learning strategy, researchers have developed an algorithm to predict the risk of multiple sclerosis (MS) relapse based on data gleaned from electronic health records.
“The two-step machine learning model predicts a patient’s future one-year MS relapse risk with clinically actionable accuracy, comparable to other clinical prediction tools,” said Zongqi Xia, MD, PhD, with the University of Pittsburgh’s Department of Neurology.
Xia discussed the findings at the Americas Committee for Treatment and Research in Multiple Sclerosis (ACTRIMS) Forum 2022 in the presentation, “Electronic Health Records as Prognostic Marker of Outcome and Platform for Clinical Discovery in MS.”
Electronic health records, or EHRs, are computer-based systems that healthcare providers use to keep track of medical data. If you’ve ever had a doctor making notes into a computer during a visit, those notes are probably kept in an EHR. By definition, EHRs contain a wealth of medical information for people in the real world — information that can also be useful for researchers.
MS, for most patients, is marked by bouts of sudden symptom worsening, called relapses. These relapses can have a substantial impact on their lives, leading researchers to look for better ways to predict the risk of relapse and adjust care as needed.
“We set out to develop a clinical tool for predicting future MS relapse leveraging readily available EHR data,” Xia said.
The researchers used a two-step machine learning algorithm to predict relapse risk based on EHR data gathered during the CLIMB study, which is investigating long-term disease course in patients receiving currently available treatments.
Machine learning works by feeding a set of data into a computer — in this case EHR data — along with some mathematical rules, which the computer uses to generate algorithms that can interpret it.
Researchers first designed a machine learning algorithm to predict the number of relapses a patient had the previous year, based on EHR data for the same time frame. From this, the computer predicted future relapse risk based on age, sex, disease duration, and the number of prior relapses as estimated in the first step.
This two-step process was needed because “recent prior relapse status informs future relapse [risk], but ascertaining past relapse history through chart review is labor-intensive and impractical,” Xia said.
The final algorithm’s ability to predict relapse risk at one year was then assessed by calculating the area under the receiver operating characteristic curve (AUC) — a statistical measure of how well a test can divide between two groups (i.e., relapse or not). AUC values can range from 0.5 to 1, with higher values indicating a better ability to differentiate the groups.
The AUC for the researchers’ algorithm was 0.707. Notably, the algorithm performed similarly to an algorithm that used actual prior relapse data.
Closer analyses of the data suggested that the two-step algorithm might be best suited as a high-specificity prognostic algorithm, Xia said. In other words, one used to identify patients who are most likely to not have relapses.
The researchers also conducted similar analyses on EHR data from CLIMB that sought to compare different disease modifying therapies (DMTs) on relapse outcomes, namely differences between one and two-year relapse rates, and the relative risk of non-relapse at two years.
“The clinical motivation was that, as DMT options grow, there is a growing need for comparative effectiveness analyses to guide clinical decisions,” Xia said. “However, there is no randomized clinical trial and limited real world evidence for head-to-head comparisons between some of the commonly prescribed DMTs.”
A comparison of Tecfidera (dimethyl fumarate) and Gilenya (fingolimod) showed no significant differences in relapse outcomes between them.
Comparing Tysabri (natalizumab) with rituximab showed that relapse rates at one and two years were lower with rituximab, and the time to first relapse was longer with rituximab.
Rituximab is an anti-CD20 therapy not approved for MS, but is often used off-label in the indication. It works via the same general mechanism of action as other anti-CD20 therapies for MS, like Ocrevus (ocrelizumab) and Kesimpta (ofatumumab).
“We showed real-world evidence of the equivalent effectiveness between [Tecfidera] and [Gilenya], and superior effectiveness of rituximab relative to [Tysabri] for relapse reduction,” Xia said.
Xia noted that these analyses only yielded consistent results when machine learning was used to analyze all parts of the EHR collectively, not when only expert-defined EHR factors were used.
“Adjustment for the high-dimensional EHR features resulted in consistent relative efficacy in favor of rituximab over [Tysabri] for all three relapse outcomes, whereas adjustment for the expert-defined confounders yielded inconsistent results,” Xia said.
The results “support the feasibility of conducting similar analysis using EHR data for treatment assignment,” Xia said. The scientists are also planning to use similar strategies to predict disability progression and treatment side effects.
Editor’s note: The Multiple Sclerosis News Today team is providing in-depth coverage of the ACTRIMS Forum 2022 Feb. 24–26. Go here to see the latest stories from the conference.
About the Author
