Study examines use of autonomous AI for diabetic eye disease

(Image Credit: AdobeStock)

Researchers from Johns Hopkins Medicine and the University of Wisconsin-Madison have authored a study on the application of autonomous artificial intelligence and testing for diabetic eye disease (DED).

The study, titled Autonomous Artificial Intelligence for Diabetic Eye Disease Increases Access and Health Equity in Underserved Populations, was published in Nature.¹

Michael D. Abramoff, MD, PhD, founder and executive chairman of Digital Diagnostics and creator of the LumineticsCore AI system for diagnosing diabetic retinopathy, served as a co-author.

The study drew on data from 2019 through 2021, compared adherence to annual DED testing among 2 groups of Johns Hopkins Medicine patients. One group underwent testing at sites that adopted Digital Diagnostics’ autonomous AI diagnostic tool, LumineticsCore, while the other group’s testing sites continued without utilizing AI testing.

At sites with autonomous AI testing, researchers observed the following changes:

• Clinics equipped with LumineticsCore experienced a 7.6 percentage point greater increase in DED testing than non-AI sites from 2019 to 2021.
• The adherence rate for DED testing among Black/African Americans increased by 12.2 percentage points in clinics using LumineticsCore and decreased by 0.6 percentage points at non-AI sites.
• With LumineticsCore, the adherence rate gap for diabetic eye disease testing between Asian Americans and Black/African Americans shrank from 15.6% in 2019 to 3.5% in 2021.

“One reason that made me create autonomous AI was its potential to address health disparities and

improve access equity in care,” Abramoff said. “These findings are encouraging. They suggest that AI

could play a vital role in narrowing health disparities and improving care access for those who historically have been the least likely to receive it. Further research will help to confirm our findings and address other questions about autonomous AI in healthcare.”

According to the study, in 2020, Johns Hopkins Medicine (JHM) began deploying autonomous AI for DED testing.

“In this study, we aimed to determine whether autonomous AI implementation was associated with increased adherence to annual dry eye disease testing, and how this differed across patient populations,” the researchers wrote.

This was a retrospective study approved by the Institutional Review Board of the Johns Hopkins School of Medicine. The study also included patients with diabetes mellitus who were managed at primary care sites of Johns Hopkins Medicine in the calendar years 2019 (pre-AI deployment) and 2021 (post-AI deployment). Subject demographic information that was retrieved from the electronic health records system included gender, age, race, ethnicity, preferred language, insurance status, ZIP code of residence, national area deprivation index (ADI), and inflation-adjusted median household income in the past 12 months.

In the study, a total of 17,674 patients with diabetes were managed at Johns Hopkins Medicine in 2019. Most patients were female (53%) and under 65 years old (69%). The 2 most highly represented racial groups were White (45.2%) and Black or African American (40.6%), and the two most common insurance coverages were commercial/other (48.7%) and Medicare (30.0%). Nearly all patients resided in an urban setting.¹

A total of 17,590 patients with diabetes were managed at Johns Hopkins Medicine in 2021. Again, most patients were female (51.1%) and under 65 years old (71.1%). Most patients were White (47.9%) and Black or African American (37.1%), and most patients had either commercial/other insurance (53.1%) or Medicare (28.1%).

Overall, the patient demographics between AI-switched sites and non-AI sites were similar. Patients at non-AI sites had a higher inflation-adjusted mean income ($90,200 in 2019 and $98,000 in 2021) than patients at AI-switched sites ($63,400 in 2019 and $63,400 in 2021). Additionally, more patients at non-AI sites were covered under military insurance compared to AI sites (13.0% vs. 6.2%).¹

Diabetic eye disease affects a third of people with diabetes mellitus (DM) and is a leading cause of blindness and visual impairment in working-aged adults in the developed world.²

“In this study, we examined the change in annual dry eye disease testing adherence rate before and after implementation of autonomous AI technology at primary care clinics at Johns Hopkins Medicine,” the researchers wrote.

From 2019 to 2021, at the end of the COVID pandemic, which caused widespread adherence issues, we observed a substantial increase in adherence rate among AI-switched sites, while the adherence rate among non-AI sites remained unchanged.

The researchers noted that this improvement in AI-switched sites over non-AI sites, 7.6 percentage points, remained statistically significant after adjustment by propensity score weighting methods. Among the AI-switched sites, the patient populations that experienced substantial improvement in adherence rate included Black or African American patients, patients with Medicaid insurance coverage, and patients with high ADI scores. Therefore, our data suggests that deployment of autonomous AI improved access to retinal evaluation and health equity in these historically disadvantaged patient groups. Our additional observations are as follows.

First, the overall adherence rate across all sites was 42.2% in 2019, which was lower than the nationwide average of 58.3%, but higher than the 34% seen in other low-income metropolitan populations in the United States. The overall adherence rate in 2021 increased slightly to 44.8%. However, the adherence rate among AI-switched sites substantially increased to 54.5%, much closer to the nationwide average.¹

“By extrapolation of our data, large scale deployment of this technology across the entire health system could substantially increase overall adherence rate, which in turn could improve HEDIS metrics, Centers for Medicare and Medicaid Services (CMS) Merit-based Incentive Payment System (MIPS) rating, and payer reimbursement,” the researchers added.

Second, among the AI-switched sites, there were significant outsize increases in adherence rates in the Black or African American ( + 12%) and Native Hawaiian or Other Pacific Islander ( + 19%) patient populations from 2019 to 2021. In contrast, over the same time period, the adherence rates for these two patient groups actually decreased by 0.6% and 1%, respectively, among non-AI sites.

“These data suggest that the deployment of autonomous AI improved access when it comes to dry eye disease management, particularly for historically disadvantaged populations,” the researchers noted in the study. “Prior studies have found that the African American population uses eye care services at much lower rates than White patients, even though Black patients with type 2 diabetes are significantly more likely to develop retinopathy than White patients.”

Moreover, the researchers’ data also suggests that autonomous AI was associated with improved health equity and smaller care gaps between patient groups. Among AI-switched sites and before AI deployment in 2019, a large adherence rate gap existed between Asian Americans and Black/African Americans (61.1% vs. 45.5%). This 15.6% gap shrank to 3.5% by 2021.

The researchers also found that among AI-switched sites and before AI deployment in 2019, a large adherence gap existed between patients with military insurance and patients with Medicaid insurance (63.1% vs. 30.2%). This 32.9% gap shrank to 21.1% by 2021. Lastly, among AI-switched sites and before AI deployment in 2019, an adherence rate gap of 8.4% existed between the most socioeconomically advantaged (ADI 1st quartile) and the most socioeconomically disadvantaged (ADI 4th quartile). By 2021, the adherence rate gap between patients from all 4 quartiles had closed.¹

The researchers also witnessed treatment heterogeneity with improvement in adherence rate even after autonomous AI deployment. Though autonomous AI improved access to retinal evaluation for the most disadvantaged patient groups and reduced care gaps, such improvement was not universal.

“Our study is limited by its retrospective nature and the fact that nearly all patients included in our study live in a metropolitan area, so our observations may not generalize to patient populations living in micropolitan, small town, and rural residences,” the researchers noted in the study. “However, our data demonstrated that deployment of autonomous AI for dry eye disease testing in the primary care setting is highly associated with improvement in adherence rate, patient access and health equity.“

The researchers did employ propensity score weighting methods to address the inherent limitations of an observational study and are reassured that our analyses showed a significant impact of autonomous AI on dry eye disease testing adherence rates.

References:

1. Huang, J.J., Channa, R., Wolf, R.M. et al. Autonomous artificial intelligence for diabetic eye disease increases access and health equity in underserved populations. npj Digit. Med. 7, 196 (2024). https://doi.org/10.1038/s41746-024-01197-3

2. Diabetic retinopathy. Cheung, Ning et al.The Lancet, Volume 376, Issue 9735, 124 – 136 Accessed August 28, 2024.