Issue link: https://www.e-digitaleditions.com/i/1501973
16 | Summer 2023 "In some of my work and the work that my team has done, we looked at the narrative data in the electronic health records—the written notes from the patient interactions. What we found is that people may be using biased or stigmatizing language during that interaction that can get propagated very quickly in machine learning—in the algorithm—if you don't know how to look for this language and then correct for it." Karnik notes that there have been some significant failures in this area, largely by companies that were developing predictive algorithms. For instance, to help hospitals plan for patient care, one algorithm looked at predictors of patients admitted into intensive care units. The algorithm was widely used by hospitals to anticipate which patients were likely to need intensive medical services. "Unfortunately, it turned out that the algorithm was basically racist in terms of how it underpredicted risk for Black patients," says Karnik. "In doing so, this algorithm led to denial of care for Black patients who likely needed secondary support services. "The computer had miscalculated because of an issue known as 'overfitting of the model.' It took a piece of what it had learned and assumed that this piece represented reality. Despite the fact that the model didn't explicitly look at race, it inadvertently did so by looking at the cost of care. This is because less money is spent on Black patients versus white patients with the same level of medical need. By assuming that cost was a good proxy for medical need, the algorithm made a mistake that biased the outcomes against Black patients who actually have a greater need because they generally receive poorer care and services, and often present later in the disease course. These differences are rooted in structural disparities and healthcare inequity. "The people who developed that algorithm didn't do the type of bias assessments that would have caught this racial bias ahead of time," Karnik explains. The algorithm got deployed widely, and only after some time had passed did people realize that it was making poor predictions and potentially putting Black patients at greater risk. One of UIC's core missions is to promote health equity and address racial, ethnic, gender and other disparities. "We are uniquely positioned to develop methods that rigorously look for these types of biases and advance techniques in machine learning that might help prevent these types of mistakes from being disseminated in the first place," Karnik says. "Our extremely diverse patient population is our strength," adds University of Illinois College of Medicine Executive Dean Mark I. Rosenblatt, MD, Advancing Medicine in the Age of AI Mark I. Rosenblatt, MD, PhD, MBA, MHA Executive Dean Bellur S. Prabhakar, MSc, PhD Interim Senior Associate Dean for Research Niranjan S. Karnik, MA, MD, PhD Visiting Professor of Psychiatry and Interim Director of CHEMA Jerry A. Krishnan, MD, PhD Professor of Medicine and Public Health R. V. Paul Chan, MD, MSc, MBA Head of the Department of Ophthalmology Executive Dean Mark Rosenblatt and four colleagues collectively developed the concept for the Center for Health Equity using Machine Learning and Artificial Intelligence (CHEMA). Their pioneering research at the intersection of medicine and technology is paving the way for more equitable and effective healthcare in the communities served by UI Health and beyond. Bellur S. Prabhakar, MSc, PhD Interim Senior Associate Dean for Research " " CHEMA is an exciting next step for the College of Medicine, and we believe that the impact of this initiative will extend across research, clinical practice and teaching domains.