Value of a deep learning-based algorithm for detecting Lung-RADS category 4 nodules on chest radiographs in a health checkup population: estimation of the sample size for a randomized controlled trial

Ju Gang Nam, Hyun Jin Kim, Eun Hee Lee, Wonju Hong, Jongsoo Park, Eui Jin Hwang, Chang Min Park, Jin Mo Goo

Research output: Contribution to journalArticlepeer-review

Abstract

Objective: To explore the value of a deep learning-based algorithm in detecting Lung CT Screening Reporting and Data System category 4 nodules on chest radiographs from an asymptomatic health checkup population. Methods: Data from an annual retrospective cohort of individuals who underwent chest radiographs for health checkup purposes and chest CT scanning within 3 months were collected. Among 3073 individuals, 118 with category 4 nodules on CT were selected. A reader performance test was performed using those 118 radiographs and randomly selected 51 individuals without any nodules. Four radiologists independently evaluated the radiographs without and with the results of the algorithm; and sensitivities/specificities were compared. The sample size needed to confirm the difference in detection rates was calculated, i.e., the number of true-positive radiographs divided by the total number of radiographs. Results: The sensitivity of the radiologists substantially increased aided by the algorithm (38.8% [183/472] to 45.1% [213/472]; p <.001) without significant change in specificity (94.1% [192/204] vs. 92.2% [188/204]; p =.22). Pooled radiologists detected more nodules with the algorithm (32.0% [156/488] vs. 38.9% [190/488]; p <.001), without alteration of false-positive rates (0.09 [62/676], both). Pooled detection rates for the annual cohort were 1.49% (183/12,292) and 1.73% (213/12,292) without and with the algorithm, respectively. A sample size of 41,776 in each arm would be required to demonstrate significant detection rate difference with < 5% type I error and > 80% power. Conclusion: Although readers substantially increased sensitivity in detecting nodules on chest radiographs from a health checkup population aided by the algorithm, detection rate difference was only 0.24%, requiring a sample size >80,000 for a randomized controlled trial. Key Points: • Aided by a deep learning algorithm, pooled radiologists improved their sensitivity in detecting Lung-RADS category 4 nodules on chest radiographs from a health checkup population (38.8% [183/472] to 45.1% [213/472]; p <.001), without increasing false-positive rate. • The prevalence of the Lung-RADS category 4 nodules was 3.8% (118/3073) on the population, resulting in only 0.24% increase of the detection rate for the radiologists with assistance of the algorithm. • To confirm the significant detection rate increase by a randomized controlled trial, a sample size of 84,000 would be required.

Original languageEnglish
Pages (from-to)213-222
Number of pages10
JournalEuropean Radiology
Volume32
Issue number1
DOIs
StatePublished - Jan 2022

Keywords

  • Deep learning
  • Image interpretation, computer-assisted
  • Lung neoplasms
  • Radiography

Fingerprint

Dive into the research topics of 'Value of a deep learning-based algorithm for detecting Lung-RADS category 4 nodules on chest radiographs in a health checkup population: estimation of the sample size for a randomized controlled trial'. Together they form a unique fingerprint.

Cite this