Saturday, March 6, 2021
  • Setup menu at Appearance » Menus and assign menu to Top Bar Navigation
Advertisement
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
No Result
View All Result
Home Machine Learning

AI systems trained on data skewed by sex are worse at diagnosing disease

May 26, 2020
in Machine Learning
AI systems trained on data skewed by sex are worse at diagnosing disease
586
SHARES
3.3k
VIEWS
Share on FacebookShare on Twitter

The artificial intelligence model showed great promise in predicting which patients treated in U.S. Veterans Affairs hospitals would experience a sudden decline in kidney function. But it also came with a crucial caveat: Women represented only about 6% of the patients whose data were used to train the algorithm, and it performed worse when tested on women.

The shortcomings of that high-profile algorithm, built by the Google sister company DeepMind, highlight a problem that machine learning researchers working in medicine are increasingly worried about. And it’s an issue that may be more pervasive — and more insidious — than experts previously realized, new research suggests.

You might also like

Will AI Replace Lawyers & Other Myths: Legal AI Mythbusters | Onit

Reducing Blind Spots in Cybersecurity: 3 Ways Machine Learning Can Help

Explainable Machine Learning, Model Transparency, and the Right to Explanation « Machine Learning Times

The study, led by researchers in Argentina and published Monday in the journal PNAS, found that when female patients were excluded from or significantly underrepresented in the training data used to develop a machine learning model, the algorithm performed worse in diagnosing them when tested across across a wide range of medical conditions affecting the chest area. The same pattern was seen when men were left out or underrepresented.

advertisement

“It’s such a valuable cautionary tale about how bias gets into algorithms,” said Ziad Obermeyer, a University of California, Berkeley, physician who studies machine learning and its clinical and health policy applications. “The combination of their results, with the fact that the datasets that these algorithms are trained on often don’t pay attention to these measures of diversity, feels really important,” added Obermeyer, who was not involved in the study.

The researchers in Argentina focused on one of the most popular applications of AI in medicine: analyzing images to try to make a diagnosis. The systems they examined were tasked with analyzing an X-ray image of the chest region to detect the presence or absence of 14 medical conditions including hernias, pneumonia, and an enlargement of the heart.

advertisement

The researchers evaluated three open-source machine learning algorithms — known as DenseNet-121, ResNet, and Inception-v3 — that are widely used experimentally in the research community, but are not yet deployed commercially for use in the clinic. The researchers set out to train the models on data from two open-source datasets — maintained by the National Institutes of Health and Stanford University — containing chest X-ray images from tens of thousands of patients.

Those two datasets are reasonably balanced by sex — 56.5% of the images in the NIH dataset are from male patients, compared to roughly 60% in the Stanford dataset — and so in a normal research setting, AI researchers using these datasets wouldn’t have to worry much about sex skew in their training data.

But for the purposes of their experiment, the researchers purposefully introduced skew by looking at just a subset of those images broken down starkly along the lines of biological sex. They set up five different training datasets featuring varying breakdowns: 100% images from female patients, 100% images from male patients, 75% images from female patients and 25% images from male patients, 75% images from male patients and 25% images from female patients, and a 50/50 split. (The researchers accounted for sample size by using the same number of images in each dataset.)


The researchers trained the algorithms on each dataset, and then tested them on images from either male or female patients. The trend was clear: Across medical conditions, the algorithms performed worse when tested in patients whose sex was underrepresented in the training data. And being overrepresented also didn’t seem to put either sex at an advantage: When the model was trained exclusively or mostly on women, it didn’t perform better when tested on women compared to when the training data were evenly split by sex.

Enzo Ferrante, the senior author of the study, said the research was inspired in part by an uproar sparked in 2015 when a Google image recognition algorithm — inadequately trained on images of people with dark skin — mistakenly characterized photos of Black people as gorillas.

The findings published in the PNAS paper should reiterate to AI researchers how important it is to use diverse training data that “includes all the characteristics of the people in which you will test the model,” said Ferrante, who studies medical image computing at a top university and a leading research institute in Argentina.

The research did not interrogate why, exactly, a model trained on men fared worse when tested on women. Some of it is likely physiological — men and women’s chests are anatomically different, after all — but there may also be other factors at play.

For example, women may get diagnosed or get X-rays taken earlier or later in the progression of their disease compared to men, which could affect how those images appear in aggregate in the training data, said Irene Chen, a Ph.D. student at MIT’s Computer Science and Artificial Intelligence Lab who studies equitable machine learning for clinical decision making and was not involved in the PNAS paper.

Those potential differences in the way men and women are diagnosed represent “a much more troubling mechanism, because it means that those biases are built into the outcomes that are coded in the dataset,” Obermeyer said.

The challenges of equitable sex representation in training data loom larger in certain cases. For some diseases and populations, even the most conscientious AI researchers have no choice but to work with a dataset that’s extremely skewed in terms of sex.

Take autism, which is diagnosed at a significantly higher rate in boys than in girls, in part due to the fact that the condition manifests differently between sexes. And researchers studying military populations — such as the DeepMind team, which unveiled its algorithm to predict a kidney condition last summer in the journal Nature — must also work with data that skews heavily male.

On the flip side, researchers developing algorithms for use in breast cancer must sift through data almost entirely from female patients — and their models may not work as well when used in men with breast cancer. Moreover, it may prove difficult to build algorithms for medical conditions affecting the intersex community, as with other rare conditions, because there’s just not enough patients to supply the training data necessary for models to be accurate.

Companies have been built and academic careers devoted to finding technical ways to get around such challenges. A key first step, the researchers consulted by STAT said, is awareness of the limitations imposed by training datasets that are not representative. “We should communicate them, we should quantify them, and then we should work with clinicians and anyone who wants to use these algorithms to figure out a way to accommodate any of the limitations of artificial intelligence,” Chen said.

The PNAS paper is the latest research to explore the impact of bias in algorithms meant for clinical use. In a paper posted on a preprint server earlier this year, a team led by researchers at the University of Toronto also examined chest X-ray data used to train diagnostic algorithms; they found that factors like the sex, age, race, and the insurance type of patients in that training data were associated with how well the models performed.

In a widely circulated paper published last fall, a team led by researchers at the University of Chicago found racial bias in a UnitedHealth Group algorithm widely used by U.S. health systems to identify patients who might need extra support. The algorithm, sold by UnitedHealth’s Optum unit, relied on insurance claims data and cost projections that ended up classifying white patients as being sicker than their Black peers — even if the Black patients were just as ill.

This is part of a yearlong series of articles exploring the use of artificial intelligence in health care that is partly funded by a grant from the Commonwealth Fund.


Credit: Google News

Previous Post

How Important is Self Learning in Data Science?

Next Post

Khloe Kardashian’s Trashy Behavior Is Co-Signed By Tristan Thompson

Related Posts

Will AI Replace Lawyers & Other Myths: Legal AI Mythbusters | Onit
Machine Learning

Will AI Replace Lawyers & Other Myths: Legal AI Mythbusters | Onit

March 6, 2021
Machine learning the news for better macroeconomic forecasting
Machine Learning

Reducing Blind Spots in Cybersecurity: 3 Ways Machine Learning Can Help

March 6, 2021
The ML Times Is Growing – A Letter from the New Editor in Chief – Machine Learning Times
Machine Learning

Explainable Machine Learning, Model Transparency, and the Right to Explanation « Machine Learning Times

March 5, 2021
How to Boost Machine Learning in Healthcare Market Compound Annual Growth Rate (CAGR)? – KSU
Machine Learning

How to Boost Machine Learning in Healthcare Market Compound Annual Growth Rate (CAGR)? – KSU

March 5, 2021
Comprehensive Report on Machine Learning Market 2021 | Size, Growth, Demand, Opportunities & Forecast To 2027
Machine Learning

Comprehensive Report on Machine Learning Market 2021 | Size, Growth, Demand, Opportunities & Forecast To 2027

March 5, 2021
Next Post
Khloe Kardashian’s Trashy Behavior Is Co-Signed By Tristan Thompson

Khloe Kardashian’s Trashy Behavior Is Co-Signed By Tristan Thompson

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recommended

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

January 6, 2019
Microsoft, Google Use Artificial Intelligence to Fight Hackers

Microsoft, Google Use Artificial Intelligence to Fight Hackers

January 6, 2019

Categories

  • Artificial Intelligence
  • Big Data
  • Blockchain
  • Crypto News
  • Data Science
  • Digital Marketing
  • Internet Privacy
  • Internet Security
  • Learn to Code
  • Machine Learning
  • Marketing Technology
  • Neural Networks
  • Technology Companies

Don't miss it

These two unusual versions of ransomware tell us a lot about how attacks are evolving
Internet Security

These two unusual versions of ransomware tell us a lot about how attacks are evolving

March 6, 2021
Researchers Find 3 New Malware Strains Used by SolarWinds Hackers
Internet Privacy

Researchers Find 3 New Malware Strains Used by SolarWinds Hackers

March 6, 2021
Analysis: The increasing scope of UK cryptocurrency regulation
Blockchain

Analysis: The increasing scope of UK cryptocurrency regulation

March 6, 2021
Will AI Replace Lawyers & Other Myths: Legal AI Mythbusters | Onit
Machine Learning

Will AI Replace Lawyers & Other Myths: Legal AI Mythbusters | Onit

March 6, 2021
Labeling Service Case Study — Video Annotation — License Plate Recognition | by ByteBridge | Feb, 2021
Neural Networks

Labeling Service Case Study — Video Annotation — License Plate Recognition | by ByteBridge | Feb, 2021

March 6, 2021
Microsoft: We’ve found three more pieces of malware used by the SolarWinds attackers
Internet Security

Microsoft: We’ve found three more pieces of malware used by the SolarWinds attackers

March 6, 2021
NikolaNews

NikolaNews.com is an online News Portal which aims to share news about blockchain, AI, Big Data, and Data Privacy and more!

What’s New Here?

  • These two unusual versions of ransomware tell us a lot about how attacks are evolving March 6, 2021
  • Researchers Find 3 New Malware Strains Used by SolarWinds Hackers March 6, 2021
  • Analysis: The increasing scope of UK cryptocurrency regulation March 6, 2021
  • Will AI Replace Lawyers & Other Myths: Legal AI Mythbusters | Onit March 6, 2021

Subscribe to get more!

© 2019 NikolaNews.com - Global Tech Updates

No Result
View All Result
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News

© 2019 NikolaNews.com - Global Tech Updates