AI or Artificial Intelligence Algorithm Developed By U.S. NIH Can Diagnose COVID-19 Better Than Most Physicians

AI-Artificial Intelligence-COVID-19: Researchers from the U.S. NIH, University of Central Florida, State University of New York, NVIDIA Corporation and University of Milano have developed an  algorithm can accurately identify COVID-19 cases, as well as distinguish them from influenza.


 
The study shows that artificial intelligence can be nearly as accurate if not even better than a physician in diagnosing COVID-19 in the lungs and also shows how the new AI platform can also overcome some of the challenges of current testing.
 
Preliminary studies indicate chest CT has a high sensitivity for detection of COVID-19 lung pathology and several groups have demonstrated the potential for AI-based diagnosis, reporting as high as 95% detection accuracies.
 
Implementation of these AI efforts at new institutions are hampered by the tendency for AI to overfit to training populations, including technical bias from institutional-specific scanners to clinical population bias due to regional variation in the use and timing of CT. Therefore, this study was specifically designed to maximize the potential for generalizability. The hypothesis was that an algorithm trained from a highly diverse multinational dataset will maintain sufficient performance accuracy when applied to new centers, compared with algorithms trained and testing in only one center.
 
To achieve this, COVID-19 CT scans were obtained from four hospitals across China, Italy, and Japan, where there was a wide variety in clinical timing and practice for CT acquisition. Such CT indications included screening-based settings (i.e., fever clinics), where patients underwent CT the same day as initial positive PCR (China), but also included advanced disease, such as inpatient hospitalization settings at physician’s discretion (Italy). Furthermore, the inclusion of patients undergoing routine clinical CT scans for a variety of indications including acute care, trauma, oncology, and various inpatient settings was designed to expose the algorithm to diverse clinical presentations.
 
 Here the research team achieved 0.949 AUC in a testing population of 1337 patients resulting in 90.8% accuracy for classification of COVID-19 on chest CT.
 
The study findings are published in the journal:  Nature Communications. https://www.nature.com/articles/s41467-020-17971-2
 
The research team demonstrated that an AI algorithm could be trained to classify COVID-19 pneumonia in computed tomography (CT) scans with up to 90 percent accuracy, as well as correctly identify positive cases 84 percent of the time and negative cases 93 percent of the time.
 
In reality CT scans offer a deeper insight into COVID-19 diagnosis and progression as compared to the often-used reverse transcription-polymerase chain reaction, or RT-PCR, tests. These tests have high false negative rates, delays in processing and other challenges.
 
Interestingly another benefit to CT scans is that they can detect COVID-19 in people without symptoms, in those who have early symptoms, during the height of the disease and after symptoms resolve.
 
CT scans are however not always recommended as a diagnostic tool for COVID-19 because the dise ase often looks similar to influenza-associated pneumonias on the scans.
 
The unique University of Central Florida (UCF) co-developed algorithm can overcome this problem by accurately identifying COVID-19 cases, as well as distinguishing them from influenza, thus serving as a great potential aid for physicians, says Dr Ulas Bagci, an Assistant Professor in UCF's Department of Computer Science.
 
Dr Bagci co-author of the study and who helped lead the research commented, “We demonstrated that a deep learning-based AI approach can serve as a standardized and objective tool to assist healthcare systems as well as patients. It can be used as a complementary test tool in very specific limited populations, and it can be used rapidly and at large scale in the unfortunate event of a recurrent outbreak."
 
Dr Bagci is a renowned expert in developing AI to assist physicians, including using it to detect pancreatic and lung cancers in CT scans.
 
Dr Bagci also has two large, National Institutes of Health grants exploring these topics, including $2.5 million for using deep learning to examine pancreatic cystic tumors and more than $2 million to study the use of artificial intelligence for lung cancer screening and diagnosis.
 
In order to perform the study, the researchers trained a computer algorithm to recognize COVID-19 in lung CT scans of 1,280 multinational patients from China, Japan and Italy.
 
Subsequently they tested the algorithm on CT scans of 1,337 patients with lung diseases ranging from COVID-19 to cancer and non-COVID pneumonia.
 
Upon comparing the computer's diagnoses with ones confirmed by physicians, they found that the algorithm was extremely proficient in accurately diagnosing COVID-19 pneumonia in the lungs and distinguishing it from other diseases, especially when examining CT scans in the early stages of disease progression.
 
Dr Bagci added, "We showed that robust AI models can achieve up to 90 percent accuracy in independent test populations, maintain high specificity in non-COVID-19 related pneumonias, and demonstrate sufficient generalizability to unseen patient populations and centers."
 
There are several limitations to the study. Model training was limited to patients with positive RT-PCR testing and COVID-19 related pneumonia on chest CT in order to differentiate between COVID-19 related disease and other pathologies. However, CT is often negative despite positive RT-PCR test. Given that viral infectiousness can often predate symptoms, CT plus RT-PCR is likely a more accurate and sensitive strategy than either alone, although this is somewhat speculative. Delayed RT-PCR or limitations in access or availability could also make CT testing more attractive for specific subsets of patients or in a resource constrained environment, such as persons under investigation for exposure history or contact tracing, triage for resource utilization, prognosis, or to assist with isolation compliance, although this is speculative. Finally, the AI algorithm aims to classify chest CT scans as positive vs. negative for COVID-19 pneumonia and in positively classified CT scans, it delivers a saliency map for visualization of AI-associated predictions. While useful for general visualization of AI output, this does not delineate COVID-19 burden, which may be more accurately depicted by segmentation algorithms.
 
The University of Central Florida researcher is a longtime collaborator with study co-authors Baris Turkbey and Bradford J. Wood. Turkbey is an associate research physician at the NIH's National Cancer Institute Molecular Imaging Branch, and Wood is the director of NIH's Center for Interventional Oncology and chief of interventional radiology with NIH's Clinical Center.
 
The study team concluded, “An AI system derived from heterogeneous multinational training data delivers acceptable performance metrics for the classification of chest CT for COVID-19 infection. While CT imaging may not necessarily be actively used in the diagnosis and screening for COVID-19, this deep learning-based AI approach may serve as a standardized and objective tool to assist the assessment of imaging findings of COVID-19 and may potentially be useful as a research tool, clinical trial response metric, or perhaps as a complementary test tool in very specific limited populations or for recurrent outbreaks settings.”
 
For more on AI-Artificial Intelligence-COVID-19, keep on logging to Thailand Medical News.