Histopathological classification of colorectal cancer based on domain-specific transfer learning and multi-model feature fusion
Ke Qi, Yan Chai Hum, Wun‐She Yap, Tian Swee Tan, Humaira Nisar, Hamam Mokayed, Ang Li, Rong Gao, YuJian Gan
Abstract
Colorectal cancer (CRC) poses a significant global health burden, where early and accurate diagnosis is vital to improving patient outcomes. However, the structural complexity of CRC histopathological images renders manual analysis time-consuming and error-prone. This study aims to develop an automated deep learning framework that enhances classification accuracy and efficiency in CRC diagnosis. The proposed model integrates domain-specific transfer learning and multi-model feature fusion to address challenges such as multi-scale structures, noisy labels, class imbalance, and fine-grained subtype classification. The model first applies domain-specific transfer learning to extract highly relevant features from histopathological images. A multi-head self-attention mechanism then fuses features from multiple pre-trained models, followed by a multilayer perceptron (MLP) classifier for final prediction. The framework was evaluated on three publicly available CRC datasets: EBHI, Chaoyang, and COAD. The model achieved a classification accuracy of 99.68% on the EBHI dataset (200 × subset), 86.72% on the Chaoyang dataset, and 99.44% on the COAD dataset. These results demonstrate strong generalization across diverse and complex histopathological image conditions. This study highlights the effectiveness of combining domain-specific transfer learning with multi-model feature fusion and attention mechanisms for CRC classification. The proposed model offers a reliable and efficient tool to support pathologists in diagnostic workflows, with the potential to reduce manual workload and improve diagnostic consistency.