%0 Journal Article
%@ 1438-8871
%I JMIR Publications
%V 23
%N 1
%P e25535
%T Accurately Differentiating Between Patients With COVID-19, Patients With Other Viral Infections, and Healthy Individuals: Multimodal Late Fusion Learning Approach
%A Xu,Ming
%A Ouyang,Liu
%A Han,Lei
%A Sun,Kai
%A Yu,Tingting
%A Li,Qian
%A Tian,Hua
%A Safarnejad,Lida
%A Zhang,Hengdong
%A Gao,Yue
%A Bao,Forrest Sheng
%A Chen,Yuanfang
%A Robinson,Patrick
%A Ge,Yaorong
%A Zhu,Baoli
%A Liu,Jie
%A Chen,Shi
%+ Department of Radiology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, 1277 Jiefang Avenue, Wuhan, China, 86 13469981699, liu_jie0823@163.com
%K COVID-19
%K machine learning
%K deep learning
%K multimodal
%K feature fusion
%K biomedical imaging
%K diagnosis support
%K diagnosis
%K imaging
%K differentiation
%K testing
%K diagnostic
%D 2021
%7 6.1.2021
%9 Original Paper
%J J Med Internet Res
%G English
%X Background: Effectively identifying patients with COVID-19 using nonpolymerase chain reaction biomedical data is critical for achieving optimal clinical outcomes. Currently, there is a lack of comprehensive understanding in various biomedical features and appropriate analytical approaches for enabling the early detection and effective diagnosis of patients with COVID-19. Objective: We aimed to combine low-dimensional clinical and lab testing data, as well as high-dimensional computed tomography (CT) imaging data, to accurately differentiate between healthy individuals, patients with COVID-19, and patients with non-COVID viral pneumonia, especially at the early stage of infection. Methods: In this study, we recruited 214 patients with nonsevere COVID-19, 148 patients with severe COVID-19, 198 noninfected healthy participants, and 129 patients with non-COVID viral pneumonia. The participants’ clinical information (ie, 23 features), lab testing results (ie, 10 features), and CT scans upon admission were acquired and used as 3 input feature modalities. To enable the late fusion of multimodal features, we constructed a deep learning model to extract a 10-feature high-level representation of CT scans. We then developed 3 machine learning models (ie, k-nearest neighbor, random forest, and support vector machine models) based on the combined 43 features from all 3 modalities to differentiate between the following 4 classes: nonsevere, severe, healthy, and viral pneumonia. Results: Multimodal features provided substantial performance gain from the use of any single feature modality. All 3 machine learning models had high overall prediction accuracy (95.4%-97.7%) and high class-specific prediction accuracy (90.6%-99.9%). Conclusions: Compared to the existing binary classification benchmarks that are often focused on single-feature modality, this study’s hybrid deep learning-machine learning framework provided a novel and effective breakthrough for clinical applications. Our findings, which come from a relatively large sample size, and analytical workflow will supplement and assist with clinical decision support for current COVID-19 diagnostic methods and other clinical applications with high-dimensional multimodal biomedical features. 
%M 33404516
%R 10.2196/25535
%U http://www.jmir.org/2021/1/e25535/
%U https://doi.org/10.2196/25535
%U http://www.ncbi.nlm.nih.gov/pubmed/33404516