ACR and SIIM Recognize Winners of Pneumothorax Detection Machine Learning Challenge

Newswise — The American College of Radiology (ACR) and the Society for Imaging Informatics in Medicine (SIIM) announced the official results of their first machine learning challenge today during the SIIM-ACR Pneumothorax Challenge ceremony at SIIM’s 4th annual Conference on Machine Intelligence in Medical Imaging (C-MIMI).

The SIIM-ACR Pneumothorax Detection and Localization Challenge required teams to develop high quality pneumothorax detection algorithms to prioritize patients for expedited review and treatment. A total of 1,475 teams took part in the challenge, and 352 submitted results during the evaluation phase of the competition.

The challenge made use of a publicly available chest radiograph dataset from the National Institutes of Health (NIH). The augmented annotations were created by radiologists from SIIM and the Society of Thoracic Radiology (STR), under the leadership of Carol Wu, MD, using a commercial web-based tool from MD.ai. The augmented annotations also follow the ACR Data Science Institute’s structured artificial intelligence (AI) use case for pneumothorax detection.

“SIIM is very pleased to have cooperated with the ACR, Google, Kaggle and the Society of Thoracic Radiology in hosting this challenge,” said Steve Langer, PhD, CIIP, FSIIM, Informatics Physicist and Radiology Imaging Architect at Mayo Clinic, and a Co-Chair of the SIIM Machine Learning Committee. “In addition to the medical and data science aspects, SIIM introduced the use of FHIR and DICOMweb in a medical imaging data challenge for the first time in Kaggle’s history, as those API’s are key in moving AI tools into clinical production.”

“Kaggle challenges like this one now incorporate some useful parameters that are more likely to result in the winners producing AI tools with potential for clinical production,” said Bibb Allen Jr., MD, FACR, ACR Data Science Institute Chief Medical Officer. “Congratulations to the winners. They have developed new healthcare solutions that may one day improve patient care.”

The challenge was run on a Kaggle, Inc. platform (owned by Google LLC), which provides access to datasets, a discussion forum for participants, the repository of submitted results and a leaderboard that runs throughout the challenge. 

The Top 10 Winning teams are: 

1. [dsmlkz] sneddy
2. X5
3. bestfitting
4. [ods.ai] amirassov
5. earhian
6. xknife
7. See & Eduardo
8. Ian Pan & Felipe Kitamura
9. [ods.ai] Scizzo
10. [ods.ai] Yury & Konstantin

Complete results and detailed challenge information is available on the SIIM website and the Kaggle website.

About the American College of Radiology The American College of Radiology is a membership organization of 38,000 radiologists, radiation oncologists and medical physicists dedicated to serving patients and society by empowering radiology professionals to advance the practice, science and professions of radiological care.

About the Society for Imaging Informatics in Medicine The Society for Imaging Informatics in Medicine (SIIM) is the leading healthcare professional organization for those interested in the current and future use of informatics in medical imaging. The society’s mission is to advance medical imaging informatics across the enterprise through education, research, and innovation in a multi-disciplinary community.

About the Society of Thoracic Radiology The Society of Thoracic Radiology (STR), founded in 1982, is a premier professional organization dedicated to promoting cardiothoracic imaging for the excellence in patient care through research and education.

Contact:

For ACR: Shawn Farley or Meghan Swope, pr@acr.org

For SIIM: Anna Zawacki, azawacki@siim.org

For STR: Carol C. Wu, MD, ccwu1@mdanderson.org