A knowledge-driven deep learning framework for organoid morphological segmentation and characterization
Yiming Qin, Jiajia Li, Yin Heng, Zheyuan Wang, Dezhi Wu, Mahi Rahman, Pengwei Hu, Tobias Plötz, Alexander V. Hopp, Nicholas A. Kurniawan, Mathias Winkel, Philipp H. P. Harbach, Chunling Tang, Feng Tan
Abstract
BACKGROUND: Organoids have great potential to revolutionize various aspects of biomedical research and healthcare. Researchers typically use the fluorescence-based approach to analyse their dynamics, which requires specialized equipment and may interfere with their growth. Therefore, it is an open challenge to develop a general framework to analyse organoid dynamics under non-invasive and low-resource settings. RESULTS: In this paper, we present a knowledge-driven deep learning system named TransOrga-plus to automatically analyse organoid dynamics in a non-invasive manner. Given a bright-field microscopic image, TransOrga-plus detects organoids through a multi-modal transformer-based segmentation module. To provide customized and robust organoid analysis, a biological knowledge-driven branch is embedded into the segmentation module which integrates biological knowledge, e.g. the morphological characteristics of organoids, into the analysis process. Then, based on the detection results, a lightweight multi-object tracking module based on the decoupling of visual and identity features is introduced to track organoids over time. Finally, TransOrga-plus outputs the dynamics analysis to assist biologists for further research. To train and validate our framework, we curate a large-scale organoid dataset encompassing diverse tissue types and various microscopic imaging settings. Extensive experimental results demonstrate that our method outperforms all baselines in organoid analysis. The results show that TransOrga-plus provides comparable analytical results to biologists and significantly accelerates organoid work process. CONCLUSIONS: In conclusion, TransOrga-plus integrates the biological expertise with cutting-edge deep learning-based model and enables the non-invasive analysis of various organoids from complex, low-resource, and time-lapse situations.