Automatic Generation of Chest X-Ray Medical Imaging Reports using LSTM-CNN
Vivek Tiwari, Krutika Bapat, Kushashwa R. Shrimali, Saurabh Singh, Basant Tiwari, Swati Jain, Hemant Kumar Sharma
Abstract
Generating medical reports manually is a difficult task, especially in rural areas and in urgent medical cases, where there is an emergency. It can also be error-prone for inexperienced physicians to generate a medical report. There are various deep learning methodologies such as Image captioning, image classification that has been implemented earlier to solve this problem. Generating a medical report automatically is a difficult task, considering the less amount of open-source data available and the paired data which contains medical Images and the report is also limited. One of the challenging tasks is data bias in medical Imaging. A generative encoder-decoder model is suggested to solve this problem in an efficient way. There are various other challenges. First, the medical report itself contains various heterogeneous information such as paragraphs, tags, keywords. Secondly, it is also difficult to identify the abnormal regions in medical images. To solve this problem, a multi-task framework is built, which can perform tag generation and paragraph generation. LSTM (Long Short Term Memory) is built to generate long heterogeneous paragraphs in the medical report. The model working is demonstrated on Chest X-Ray dataset and also on pathology dataset.