De-Sheng Chen,* Tong-Fu Wang,* Jia-Wang Zhu, Bo Zhu, Zeng-Liang Wang, Jian-Gang Cao, Cai-Hong Feng, Jun-Wei Zhao
Department of Sports Medicine and Arthroscopy, Tianjin Hospital of Tianjin University, Tianjin, People’s Republic of China
Correspondence: Jia-Wang Zhu
Department of Sports Medicine and Arthroscopy, Tianjin Hospital of Tianjin University, Tianjin, People’s Republic of China
Email [email protected]
Purpose: We aim to present an unsupervised machine learning application in anterior cruciate ligament (ACL) rupture and evaluate whether supervised machine learning-derived radiomics features enable prediction of ACL rupture accurately.
Patients and Methods: Sixty-eight patients were reviewed. Their demographic features were recorded, radiomics features were extracted, and the input dataset was defined as a collection of demographic features and radiomics features. The input dataset was automatically classified by the unsupervised machine learning algorithm. Then, we used a supervised machine learning algorithm to construct a radiomics model. The t-test and least absolute shrinkage and selection operator (LASSO) method were used for feature selection, random forest and support vector machine (SVM) were used as machine learning classifiers. For each model, the sensitivity, specificity, accuracy, and the area under the curve (AUC) of receiver operating characteristic (ROC) curves were calculated to evaluate model performance.
Results: In total, 5 demographic features were recorded and 106 radiomics features were extracted. By applying the unsupervised machine learning algorithm, patients were divided into 5 groups. Group 5 had the highest incidence of ACL rupture and left knee involvement. There were significant differences in left knee involvement among the groups. Forty-three radiomics features were extracted using t-test and 7 radiomics features were extracted using LASSO method. We found that the combination of LASSO selection method and random forest classifier has the highest sensitivity, specificity, accuracy, and AUC. The 7 radiomics features extracted by LASSO method were potential predictors for ACL rupture.
Conclusion: We validated the clinical application of unsupervised machine learning involving ACL rupture. Moreover, we found 7 radiomics features which were potential predictors for ACL rupture. The study indicated that radiomics could be a valuable method in the prediction of ACL rupture.
Keywords: unsupervised machine learning, supervised machine learning, radiomics, anterior cruciate ligament rupture
Anterior cruciate ligament (ACL) rupture is a common and devastating knee injury.1,2 Clinically, MRI has been widely used for the assessment of ACL rupture.3,4 The typical MRI findings of ACL rupture include abnormal orientation and abnormal signal intensity of ACL.5–7 In a study conducted by Zhao et al., 78 participants were investigated; the diagnostic sensitivity, specificity, and accuracy of MRI for ACL injury were 95.45%, 91.67%, and 94.87%.5
Although MRI has high sensitivity and specificity, the diagnosis rate at the early stage of ACL rupture remains poor.8–10 The potential causes include less experienced radiologists and insufficient awareness of physicians, especially for emergency physicians.11,12 Delayed diagnosis of ACL rupture usually results in higher risk of secondary knee injury and higher economic burden.8,13–15 In a study conducted by Church and Keating, 183 participants were investigated; a higher incidence of meniscal tears was observed in patients undergoing ACL reconstruction more than 12 months from injury.16 Therefore, early diagnosis of ACL rupture is crucial.
Recently, remarkable achievements have been made in knee MRI analysis using machine learning techniques.17,18 Bien et al. applied a deep learning model to analyze a dataset including 1370 knee MRI exams; they found that their deep learning model can generate accurate clinical pathology classifications and improve clinical experts’ performance.17 Usually, machine learning methods include an unsupervised algorithm, a supervised algorithm, and a semi-supervised algorithm. For a supervised machine learning algorithm, the algorithm learns on a labeled dataset and predicts the unlabeled dataset. In contrast, an unsupervised machine learning algorithm does not require labels and tries to work on its own to discover information.19,20
Radiomics focuses on extracting huge amounts of information from medical images.21,22 The hypothesis is that massive “hidden” information that is hardly perceived by radiologists and physicians could be quantitatively mined using a radiomics approach from medical images.23 Indeed, numerous studies have confirmed that radiomics can be used in the diagnosis and prognosis of diseases, with the advantages of non-invasiveness, low cost, and high efficiency.24–26 The key challenge of radiomics is the analysis of this “hidden” information. Coincidentally, many machine learning algorithms are essentially dimensionality reduction for big data. Several studies have indicated that the novel combination of machine learning and radiomics contributes to maximize physician performance and reduce errors, paving the path for modern precision and personalized medicine.18,27–29
In this study, we aim to present unsupervised machine learning application in anterior cruciate ligament (ACL) rupture and evaluate whether supervised machine learning-derived radiomics features enable accurate prediction of ACL rupture.
Patients and Methods
Sixty-eight patients undergoing knee surgery between October 2020 and December 2020 were reviewed. The inclusion criteria were: (1) first knee surgery; (2) surgery for knee disease; (3) the disease was definitively confirmed by arthroscopy. The exclusion criteria were: (1) simultaneous bilateral knee surgery; (2) revision surgery; (3) follow-up data missing. The patients’ demographic features were recorded. The study was approved by the institutional review board, and written informed consent was obtained.
MR Imaging Segmentation and Radiomics Feature Extraction
T1-weighted imaging (T1WI) of the knee was obtained in all patients. A musculoskeletal radiologist with 10 years of experience delineated regions of interest (ROIs) on T1WI using 3D Slicer software 4.11 (http://www.slicer.org/). The superior border of ROIs was at the level of the upper edge of the patella. The inferior border of ROIs was at the level of tibial insertion of the patella tendon (Figure 1). Then, all ROIs were manually checked by a musculoskeletal radiologist with 15 years of experience who were blind to the medical records of the patients. All disagreements were resolved through consensus.
Figure 1 Image acquisition and segmentation.
The radiomics features were extracted using an open-source Pyradiomics package (https://github.com/Radiomics/pyradiomics). All radiomics features were normalized to a value of 0 to 1.
First, we used an unsupervised machine learning algorithm which has been validated and extensively tested in other diseases to convert the input dataset into a compact representation space.30–32 The input dataset was defined as a collection of demographic features and radiomics features. In the representation space, the subjects were positioned according to their similarity, while blinded to the patient’s outcome status. Once positioned in the compact representation space, subjects were clustered to identify phenotypically distinct categories of patients undergoing knee surgery. Clustering was performed using the K-means clustering algorithm and the best value of k was determined automatically.
Second, we used a supervised machine learning algorithm to construct a radiomics model based on extracted radiomics features. The t-test and least absolute shrinkage and selection operator (LASSO) method were adopted to select potential radiomics features contributing to the prediction of ACL rupture. Briefly, the LASSO technique constructs a model by constructing a penalty function, the LASSO model penalty function enables an efficient shrinking of coefficients to zero. The complexity of the LASSO model is controlled by the penalty parameter lambda, the maximum lambda value was selected with minimum mean-squared error (MSE) values. Then, random forest and support vector machine (SVM) were used as machine learning classifiers. For each model, the sensitivity, specificity, accuracy, and the area under the curve (AUC) of receiver operating characteristic (ROC) curves were calculated to evaluate model performance.
The machine learning algorithms and the statistical analysis were implemented in Python (Python Software Foundation). Multiple comparisons among groups were analyzed by chi-square test for categorical variables, and ANOVA for continuous variables. A two-sided P value < 0.05 was considered significant.
Characteristics of the Participants
There were 30 males and 38 females in the study. Five demographic features were recorded including gender, age, involved knee, trauma-history, and diagnosis. The demographic characteristics of the patients are listed in Table 1.
Table 1 The Demographic Characteristics of the Patients
In total, 106 radiomics features were extracted, including first-order features (n = 18), shape and size features (n = 13), and textural features (n = 75). The textural features included the gray-level co-occurrence matrix (GLCM), gray-level dependence matrix (GLDM), gray-level run-length matrix (GLRLM), gray-level size zone matrix (GLSZM), and neighbouring gray tone difference matrix (NGTDM).
In total, the input dataset included 111 features (5 clinical features and 106 radiomics features).
Results of the Unsupervised Machine Learning Algorithm
By applying the unsupervised machine learning algorithm, patients were divided into 5 groups. The scatter plot of principal component is shown in Figure 2. Group 5 has the highest incidence of ACL rupture and left knee involved. There were significant differences in left knee involvement among the groups. The demographic characteristics among groups are listed in Table 2.
Table 2 The Demographic Characteristics Among Groups
Figure 2 The patients were divided into 5 groups, group 1 is represented as blue dots, group 2 is represented as green dots, group 3 is represented as black dots, group 4 is represented as red dots, and group 5 is represented as yellow dots.
Results of the Supervised Machine Learning Algorithm
Based on the t-test, 43 radiomics features were extracted and considered as potential predictors of ACL rupture, meanwhile, the sensitivity, specificity, accuracy, and AUC corresponding to random forest and SVM classifiers are shown in Table 3.
Table 3 The Sensitivity, Specificity, Accuracy, and AUC of the Four Prediction Models
Based on the LASSO method, 7 features were extracted and considered as potential predictors of ACL rupture. The MSE values and the corresponding lambda values are shown in Figure 3. The values of the coefficients and the corresponding lambda values are shown in Figure 4. Meanwhile, the sensitivity, specificity, accuracy, and AUC corresponding to random forest and SVM classifiers are listed in Table 3.
Figure 3 Different mean-squared error (MSE) values within the range of lambda, maximum lambda was selected with minimum MSE value.
Figure 4 The values of the coefficients and the corresponding lambda values, each curve represents each feature in the model.
We found that the combination of LASSO selection method and random forest classifier has the highest sensitivity, specificity, accuracy, and AUC. The AUC of the four prediction models are shown in Figure 5. The 7 radiomics features extracted by LASSO selection method are listed in Table 4, including 1 shape feature, 1 GLCM feature, 1 GLDM feature, 2 GLSZM features, and 2 NGTDM features. The radiomics features’ weights are shown in Figure 6.
Table 4 The Radiomics Features Extracted by LASSO Selection Method
Figure 5 The AUC of the four prediction models. (A) t-test and random forest, (B) t-test and SVM, (C) LASSO and random forest, (D) LASSO and SVM.
Figure 6 The radiomics features weight.
Anterior cruciate ligament (ACL) rupture is a common and devastating knee injury.1,2 In this study, we validated the clinical application of unsupervised machine learning involving ACL rupture. Moreover, we found 7 radiomics features which were potential predictors of ACL rupture, by applying supervised machine learning. The prediction model (the combination of LASSO selection method and random forest classifier) has high sensitivity, specificity, accuracy, and AUC.
By applying the unsupervised machine learning algorithm, we found that the highest incidence rate of left knee involvement followed by the highest incidence of ACL rupture, which indicated that left knee could be a risk factor for ACL rupture. In a study conducted by Westin et al., 339 alpine ski students were investigated, the majority of ACL injuries significantly occurred in the left knee.33 Our study showed that the unsupervised machine learning algorithm could be applied effectively in the research of ACL injury.
By applying supervised machine learning-derived radiomics, we found 7 radiomics features were potential predictors for ACL rupture, which indicated that radiomics could be a valuable method in the prediction of ACL rupture. So far, many deep learning algorithms have been proposed for identifying ACL rupture,17,18,34 as far as we know, this is the first study to investigate machine learning-derived radiomics features for ACL rupture. Although the detection of ACL rupture is not a diagnostic challenge for specialized musculoskeletal radiologists and sports physicians, our study may be helpful for non-trained radiologists and non-sport physicians; especially, our study may provide a “second-reader opinion” in a rural area without access to subspecialty radiology or subspecialty radiology interpretation is not readily available.
The limitations of our study include the relatively small numbers of patients and it being a retrospective single-center study. Due to the relatively small numbers of patients, we may not be able to screen out the most valuable and stable radiomics features, and the developed model may not be the most effective. A further larger, multicenter study was needed to confirm our findings.
We validated the clinical application of unsupervised machine learning involving ACL rupture. Moreover, we found 7 radiomics features which were potential predictors for ACL rupture. It indicated that radiomics could be a valuable method in the prediction of ACL rupture.
All authors made substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; took part in drafting the article or revising it critically for important intellectual content; agreed to submit to the current journal; gave final approval of the version to be published; and agree to be accountable for all aspects of the work.
The authors report no conflicts of interest in this work.
1. Zhang P, Han F, Li Y, et al. Local delivery of controlled-release simvastatin to improve the biocompatibility of polyethylene terephthalate artificial ligaments for reconstruction of the anterior cruciate ligament. Int J Nanomedicine. 2016;11:465–478. doi:10.2147/IJN.S95032
2. Gianotti Simon M, Marshall Stephen W, Hume Patria A, et al. Incidence of anterior cruciate ligament injury and other knee ligament injuries: a national population-based study. J Sci Med Sport. 2009;12(6):622–627. doi:10.1016/j.jsams.2008.07.005
3. Niki Y, Yasuoka T, Kobayashi S, et al. Feasibility of T1rho and T2 map magnetic resonance imaging for evaluating graft maturation after anatomic double-bundle anterior cruciate ligament reconstruction. J Orthop Surg Res. 2019;14(1):140. doi:10.1186/s13018-019-1193-y
4. Ouyang X, Wang YH, Wang J, et al. MRI measurement on intercondylar notch after anterior cruciate ligament rupture and its correlation. Exp Ther Med. 2016;11(4):1275–1278. doi:10.3892/etm.2016.3078
5. Zhao M, Zhou Y, Chang J, et al. The accuracy of MRI in the diagnosis of anterior cruciate ligament injury. Ann Transl Med. 2020;8(24):1657. doi:10.21037/atm-20-7391
6. Kam CK, Chee Daniel WY, Peh Wilfred CG. Magnetic resonance imaging of cruciate ligament injuries of the knee. Can Assoc Radiol J. 2010;61(2):80–89. doi:10.1016/j.carj.2009.11.003
7. Helito CP, Helito PVP, Costa HP, et al. Assessment of the anterolateral ligament of the knee by magnetic resonance imaging in acute injuries of the anterior cruciate ligament. Arthroscopy. 2017;33(1):140–146. doi:10.1016/j.arthro.2016.05.009
8. Arastu MH, Grange S, Twyman R. Prevalence and consequences of delayed diagnosis of anterior cruciate ligament ruptures. Knee Surg Sports Traumatol Arthrosc. 2015;23(4):1201–1205. doi:10.1007/s00167-014-2947-z
9. Hartnett N, Tregonning RJ. Delay in diagnosis of anterior cruciate ligament injury in sport. N Z Med J. 2001;114(1124):11–13.
10. Noyes FR, Paulos L, Mooar LA, et al. Knee sprains and acute knee hemarthrosis: misdiagnosis of anterior cruciate ligament tears. Phys Ther. 1980;60(12):1596–1601. doi:10.1093/ptj/60.12.1596
11. Guillodo Y, Rannou N, Dubrana F, et al. Diagnosis of anterior cruciate ligament rupture in an emergency department. J Trauma. 2008;65(5):1078–1082. doi:10.1097/TA.0b013e3181469b7d
12. Krampla W, Roesel M, Svoboda K, et al. MRI of the knee: how do field strength and radiologist’s experience influence diagnostic accuracy and interobserver correlation in assessing chondral and meniscal lesions and the integrity of the anterior cruciate ligament? Eur Radiol. 2009;19(6):1519–1528. doi:10.1007/s00330-009-1298-5
13. Chhadia Ankur M, Inacio Maria CS, Maletis Gregory B, et al. Are meniscus and cartilage injuries related to time to anterior cruciate ligament reconstruction? Am J Sports Med. 2011;39(9):1894–1899. doi:10.1177/0363546511410380
14. Michalitsis S, Vlychou M, Malizos Konstantinos N, et al. Meniscal and articular cartilage lesions in the anterior cruciate ligament-deficient knee: correlation between time from injury and knee scores. Knee Surg Sports Traumatol Arthrosc. 2015;23(1):232–239. doi:10.1007/s00167-013-2497-9
15. Sri-Ram K, Salmon LJ, Pinczewski LA, et al. The incidence of secondary pathology after anterior cruciate ligament rupture in 5086 patients requiring ligament reconstruction. Bone Joint J. 2013;95–B(1):59–64. doi:10.1302/0301-620X.95B1.29636
16. Church S, Keating JF. Reconstruction of the anterior cruciate ligament: timing of surgery and the incidence of meniscal tears and degenerative change. J Bone Joint Surg Br. 2005;87(12):1639–1642. doi:10.1302/0301-620X.87B12.16916
17. Bien N, Rajpurkar P, Ball Robyn L, et al. Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of MRNet. PLoS Med. 2018;15(11):e1002699. doi:10.1371/journal.pmed.1002699
18. Germann C, Marbach G, Civardi F, et al. Deep convolutional neural network-based diagnosis of anterior cruciate ligament tears: performance comparison of homogenous versus heterogeneous knee MRI cohorts with different pulse sequence protocols and 1.5-T and 3-T magnetic field strengths. Invest Radiol. 2020;55(8):499–506. doi:10.1097/RLI.0000000000000664
19. Libbrecht MW, Noble WS. Machine learning applications in genetics and genomics. Nat Rev Genet. 2015;16(6):321–332. doi:10.1038/nrg3920
20. Reig B, Heacock L, Geras KJ, et al. Machine learning in breast MRI. J Magn Reson Imaging. 2020;52(4).
21. Parekh V, Jacobs Michael A. Radiomics: a new application from established techniques. Expert Rev Precis Med Drug Dev. 2016;1(2):207–226. doi:10.1080/23808993.2016.1164013
22. Zhang B, He X, Ouyang F, et al. Radiomic machine-learning classifiers for prognostic biomarkers of advanced nasopharyngeal carcinoma. Cancer Lett. 2017;403:21–27. doi:10.1016/j.canlet.2017.06.004
23. Shi Y, Wahle E, Du Q, et al. Associations between statin/omega3 usage and MRI-based radiomics signatures in prostate cancer. Diagnostics (Basel, Switzerland). 2021;11(1). doi:10.3390/diagnostics11010085
24. Xun X, Zhang H-L, Liu Q-P, et al. Radiomic analysis of contrast-enhanced CT predicts microvascular invasion and outcome in hepatocellular carcinoma. J Hepatol. 2019;70(6):1133–1144. doi:10.1016/j.jhep.2019.02.023
25. Han L, Zhu Y, Liu Z, et al. Radiomic nomogram for prediction of axillary lymph node metastasis in breast cancer. Eur Radiol. 2019;29(7):3820–3829. doi:10.1007/s00330-018-5981-2
26. Coroller Thibaud P, Grossmann P, Hou Y, et al. CT-based radiomic signature predicts distant metastasis in lung adenocarcinoma. Radiother Oncol. 2015;114(3):345–350. doi:10.1016/j.radonc.2015.02.015
27. Giger Maryellen L. Machine learning in medical imaging. J Am Coll Radiol. 2018;15(3PtB):512–520. doi:10.1016/j.jacr.2017.12.028
28. Choi YS, Ahn SS, Chang JH, et al. Machine learning and radiomic phenotyping of lower grade gliomas: improving survival prediction. Eur Radiol. 2020;30(7):3834–3842. doi:10.1007/s00330-020-06737-5
29. Giraud P, Giraud P, Gasnier A, et al. Radiomics and machine learning for radiotherapy in head and neck cancers. Front Oncol. 2019;9:174. doi:10.3389/fonc.2019.00174
30. Cikes M, Sanchez-Martinez S, Claggett B, et al. Machine learning-based phenogrouping in heart failure to identify responders to cardiac resynchronization therapy. Eur J Heart Fail. 2019;21(1):74–85. doi:10.1002/ejhf.1333
31. Li H, Xu C, Xin B, et al. F-FDG PET/CT radiomic analysis with machine learning for identifying bone marrow involvement in the patients with suspected relapsed acute leukemia. Theranostics. 2019;9(16):4730–4739. doi:10.7150/thno.33841
32. Li H, Galperin-Aizenberg M, Pryma D, et al. Unsupervised machine learning of radiomic features for predicting treatment response and overall survival of early stage non-small cell lung cancer patients treated with stereotactic body radiation therapy. Radiother Oncol. 2018;129(2):218–226. doi:10.1016/j.radonc.2018.06.025
33. Westin M, Harringe Marita L, Engström B, et al. Risk factors for anterior cruciate ligament injury in competitive adolescent alpine skiers. Orthop J Sports Med. 2018;6(4):2325967118766830. doi:10.1177/2325967118766830
34. Chang Peter D, Wong Tony T, Rasiej Michael J. Deep learning for detection of complete anterior cruciate ligament tear. J Digit Imaging. 2019;32(6):980–986. doi:10.1007/s10278-019-00193-4
Credit: Google News