A research team from Massachusetts Institute of technology and Harvard University published an article in the medical journal Lancet digital health, saying that AI programs can distinguish the race of patients from X-ray and CT scanning results, with an accuracy of 90% This is not good news, because even scientists have not figured out how these AI programs can distinguish races.
Original title: will AI also be racist? Lancet article: AI diagnostic system may output racially biased results
"When my graduate students showed me some of the results in this paper, I thought it must have gone wrong." Marzyeh ghassemi, an assistant professor at the Massachusetts Institute of technology and one of the authors who analyzed the corresponding topics, told the media, "when they told me, I really thought my students were crazy."
Marzyeh Ghassemi
As mentioned in this article, the AI diagnostic system seems to diagnose and treat patients according to their race, rather than their personal physical condition.
This practice will damage the patient's health.
The researchers mentioned a case in which the AI program was more likely to miss physical lesions in black and female patients when examining chest X-rays.
The purpose of this study is to confirm the extent to which AI systems detect human race from medical images and how they detect race information.
To this end, the research team used medical images of different parts of the human body to train the AI system. The medical images provided to the AI system do not contain obvious racial markers such as hair texture, skin color, BMI or bone mineral density.
Through the test, the researchers found that the accuracy of AI system in identifying human race was as high as 90%. No matter which part of the body is in the medical image, the AI system can recognize the racial information.
What is more surprising is that the AI system can accurately identify the race even in the severely missing or damaged medical images.
What researchers are more concerned about is not that the AI system can detect human race itself, but that the clinical performance of the AI system will be affected by these racial biases. Doctors may ignore the errors in the diagnosis results of AI system.
"Ai's ability to predict racial identity is not important in itself, but it is likely that this ability exists in many medical image analysis models, which will worsen the existing racial differences in clinical practice," the authors said
At present, humans cannot confirm which features of the medical image the AI system detects the patient's race. In addition, AI can identify the patient's race from the medical image of any part of the body and the severely damaged medical image, which means that it will be very difficult to create an AI system without racial bias using medical imaging technology.
Ghassemi told the media that she guessed that maybe the medical image recorded the melanin level in the patient's skin in an unknown way, which was recognized by the AI system**
According to the research results, there may also be some inborn differences between races.
Alan Goodman, a professor of biological anthropology at Hampshire College and one of the authors of racism is not race, told the media that he did not agree with this statement.
Alan Goodman
In previous studies, scientists have found it difficult to find consistent ethnic differences in the human genome, but they can often find consistent genetic differences according to the evolution of human ancestors. Therefore, the genetic differences between people are more likely to come from the different characteristics of the evolution of human individual ancestors, rather than race.
Ghassemi said that more research is needed to draw a clear conclusion on this issue.
"We need to suspend the implementation of the AI system," said Leo Anthony celi, a scientist and doctor at the Massachusetts Institute of technology. "Before we confirm that the AI system has not made racist or sexist decisions, we cannot rush to bring it into hospitals and clinics."