Modelling radiological features fusion and explainable AI in pneumonia detection: A graph- based deep learning and transformer approach
Pratham Kaushik, Eshika Jain, Vinay Kukreja, Shanmugasundaram Hariharan, Murugaperumal Krishnamoorthy, Vandana Ahuja, Abhishek Bhattacherjee, Rajesh Kumar Kaushal, Shih‐Yu Chen
Abstract
Objectives This study aims to develop a deep learning-based methodology for pneumonia detection using lung CT scans. The research focuses on integrating U-Net for lung segmentation, graph-based feature representations, and transformer-based models to improve interpretability and diagnostic accuracy. Methodology We utilized 500 annotated lung CT scans, split into 70% training (350 images), 15% validation (75 images), and 15% testing (75 images). U-Net was employed for lung segmentation, with segmentation performance evaluated using Dice Similarity Coefficient (DSC) and Intersection over Union (IoU). The classification model incorporated feature fusion, combining handcrafted radiological features with transformer-based feature extraction. The model’s noise sensitivity and interpretability were assessed using Grad-CAM and attention maps. Results The segmentation model achieved a DSC of 0.943 on the training set, 0.928 on the validation set, and 0.910 on the test set. The IoU values were 0.890, 0.872, and 0.850, respectively. The model’s inference time was 25 milliseconds per image. The classification model achieved an overall accuracy of 94%, with pneumonia cases showing precision, recall, and F1-scores of 0.95, 0.94, and 0.94, respectively. For normal cases, precision, recall, and F1-scores were 0.92, 0.94, and 0.93. The model’s performance improved over 20 epochs, with training accuracy reaching 0.94 and validation accuracy reaching 0.88. The feature fusion method achieved an attention score of 0.88, with node intensity scores of 0.85. Noise sensitivity analysis revealed a performance drop beyond a noise level of 0.3. Conclusions This study demonstrates the potential of AI-driven solutions for pneumonia detection, achieving high accuracy, robustness, and interpretability. While the model performs well, further optimization is needed to address noise sensitivity and enhance model generalization for real-world applications.