Generating Medical Reports With a Novel Deep Learning Architecture

Abstract

ABSTRACT The writing of medical reports by doctors in hospitals is a critical and sensitive process that is time‐consuming, prone to human error, and requires medical experts on site. Existing work on autonomous medical report generation using medical images as input has not achieved sufficiently high success. The goal of this paper is to present a new, fast, and high‐performance method. For the autonomous generation of paragraph‐level medical reports. A deep learning‐based hybrid encoder–decoder architecture called G‐CNX is developed to generate meaningful reports. ConvNeXtBase is used on the encoder side, and GRU‐based RNN is used on the decoder side. Images and reports from the Indiana University Chest X‐ray and ROCOv2 data sets were used in the training, validation, and testing processes of the study. The results of the experiments showed that the autonomously generated medical reports had the highest performance compared to other studies in the literature. In the Indiana University Chest X‐ray data set, success rates of 0.6544, 0.5035, 0.3682, 0.2766, 0.2766, and 0.4277 were obtained in Bleu‐1, Bleu‐2, Bleu‐3, Bleu‐4, and Rouge evaluation metrics, respectively. In the ROCOv2 data set, success scores of 0.5593 and 0.3990 were obtained in Bleu‐1 and Rouge evaluation metrics, respectively. In addition to numerical quantifiable analysis, the results of the study were also analyzed observationally and based on density plots. Statistical significance tests were also conducted to prove the reliability of the results. The results show that the test results obtained in the study have semantic properties similar to those of reports written by real doctors and that the autonomous reports produced are consistent and reliable. The proposed method can improve the efficiency of medical reporting, reduce the workload of specialized doctors, and improve the quality of diagnosis and treatment processes.

Topics & Concepts