Deep multimodal state-space fusion of endoscopic-radiomic and clinical data for survival prediction in colorectal cancer
Ning Wang, Jiajing Lin, W. J. Li, Yahui Lyu, Yiqing Jiang, Zhizhan Ni, Qi Huang, Hong Chen, Qiang Yan, Chenshen Huang
Abstract
Integrating complementary surface and cross sectional cues is central to preoperative assessment of colorectal cancer, but technically challenging because endoscopic images and pelvic CT encode anatomy at different scales. Here we present HydraMamba, a multimodal selective state space framework that fuses endoscopy and CT for joint lesion segmentation, lesion detection, and survival prediction. The model couples a shared state space backbone with two lightweight modules. Across the endoscopic dataset and the CT dataset, HydraMamba achieved state-of-the-art lesion analysis (endoscopy: Dice 0.856, F1 0.918; CT: Dice 0.812, F1 0.888) and delivered calibrated survival modeling on the CT dataset (Harrell's C index 0.832, Uno's C@1y 0.853, integrated Brier score 0.161, calibration slope ≈1.01). By unifying endoscopic and CT information in a single coherent architecture, HydraMamba provides an accurate and well-calibrated foundation for lesion analysis and prognostication in colorectal cancer.