According to new research, deep learning models based on artificial intelligence can identify someone's race merely by looking at their X-rays, which would be impossible for a human doctor looking at the same photos.
The findings raise several serious concerns concerning AI's role in medical diagnosis, assessment, and treatment: Could computer algorithms mistakenly apply racial bias when analysing photographs like these?
An international team of health researchers from the United States, Canada, and Taiwan tested their AI on X-ray images that the computer program had never seen before after training it with hundreds of thousands of existing X-ray images annotated with specifics of the patient's race.
Even when the scans were taken from people of the same age and sex, the AI was able to predict the patient's claimed racial identification on these photos with amazing accuracy. With some groups of photos, the system achieved 90% accuracy.
"We aimed to conduct a comprehensive evaluation of the ability of AI to recognize a patient's racial identity from medical images," the researchers wrote in their published report.
"We show that standard AI deep learning models can be trained to predict race from medical images with high performance across multiple imaging modalities, which was sustained under external validation conditions."
Artificial intelligence scans of X-ray pictures were more likely to miss indicators of sickness among Black persons, according to earlier research. Scientists must first figure out why this is happening.
Artificial intelligence (AI) is designed to replicate human thinking in order to discover patterns in data fast. However, this means it is susceptible to the same biases unintentionally. Worse, their intricacy makes it difficult to divorce our prejudices from them.
Scientists are now unsure why the AI system is so good at identifying race from photographs that don't appear to contain such information. Even with minimal information, such as omitting hints about bone density or focusing on a tiny portion of the body, the models were very good at predicting the race represented in the file.
It's likely that the system is detecting melanin, the pigment that gives skin its color, in ways that science has yet to discover.
"Our finding that AI can accurately predict self-reported race, even from corrupted, cropped, and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging," write the researchers.
The study adds to a growing body of data that AI systems can often replicate human biases and prejudices, whether they be racist, sexist, or otherwise. Skewed training data can lead to skewed findings, rendering them useless.
This must be balanced against artificial intelligence's strong ability to process far more data much faster than humans can, in anything from disease diagnosis to climate change predictions.
There are many unsolved questions from the study, but for the time being, it's crucial to be conscious of the possibility of racial bias in artificial intelligence systems - especially if we're going to give them more responsibility in the future.
"We need to take a pause," the Massachusetts Institute of Technology's research scientist and physician Leo Anthony Celi told the Boston Globe.
"We cannot rush bringing the algorithms to hospitals and clinics until we're sure they're not making racist decisions or sexist decisions."
