C<sup>2</sup>MA-Net: Cross-Modal Cross-Attention Network for Acute Ischemic Stroke Lesion Segmentation Based on CT Perfusion Scans
Tianyu Shi, Huiyan Jiang, Bin Zheng
Abstract
<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Objective:</i> Based on the hypothesis that adding a cross-modal and cross-attention (C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> MA) mechanism into a deep learning network improves accuracy and efficacy of medical image segmentation, we propose to test a novel network to segment acute ischemic stroke (AIS) lesions from four CT perfusion (CTP) maps. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Methods:</i> The proposed network uses a C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> MA module directly to establish a spatial-wise relationship by using the multigroup non-local attention operation between two modal features and performs dynamic group-wise recalibration through group attention block. This C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> MA-Net has a multipath encoder-decoder architecture, in which each modality is processed in different streams on the encoding path, and the pair related parameter modalities are used to bridge attention across multimodal information through the C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> MA module. A public dataset involving 94 training and 62 test cases are used to build and evaluate the C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> MA-Net. AIS segmentation results on testing cases are analyzed and compared with other state-of-the-art models reported in the literature. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Results:</i> By calculating several average evaluation scores, C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> MA-network improves Recall and F2 scores by 6% and 1%, respectively. In the ablation experiment, the F1 score of C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> MA-Net is at least 7.8% higher than that of single-input single-modal self-attention networks. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Conclusion:</i> This study demonstrates advantages of applying C <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> MA-network to segment AIS lesions, which yields promising segmentation accuracy, and achieves semantic decoupling by processing different parameter modalities separately. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Significance:</i> Proving the potential of cross-modal interactions in attention to assist identifying new imaging biomarkers for more accurately predicting AIS prognosis in future studies.