Deep learning quantifies pathologists’ visual patterns for whole slide image diagnosis
Tianhang Nan, Song Zheng, Siyuan Qiao, Quan Hao, Xin Gao, Jun Niu, Bin Zheng, Chunfang Guo, Yue Zhang, Xiaoqin Wang, Liping Zhao, Ze Wu, Yaoxing Guo, Xingyu Li, Mingchen Zou, Mingchen Zou, Yang Zhao, Wei Qian, Hong‐Duo Chen, Ruiqun Qi, Xing‐Hua Gao, Xiaoyu Cui, Xing‐Hua Gao, Xiaoyu Cui
Abstract
Based on the expertise of pathologists, the pixelwise manual annotation has provided substantial support for training deep learning models of whole slide images (WSI)-assisted diagnostic. However, the collection of pixelwise annotation demands massive annotation time from pathologists, leading to a high burden of medical manpower resources, hindering to construct larger datasets and more precise diagnostic models. To obtain pathologists’ expertise with minimal pathologist workloads then achieve precise diagnostics, we collect the image review patterns of pathologists by eye-tracking devices. Simultaneously, we design a deep learning system: Pathology Expertise Acquisition Network (PEAN), based on the collected visual patterns, which can decode pathologists’ expertise and then diagnose WSIs. Eye-trackers reduce the time required for annotating WSIs to 4%, of the manual annotation. We evaluate PEAN on 5881 WSIs and 5 categories of skin lesions, achieving a high area under the curve of 0.992 and an accuracy of 96.3% on diagnostic prediction. This study fills the gap in existing models’ inability to learn from the diagnostic processes of pathologists. Its efficient data annotation and precise diagnostics provide assistance in both large-scale data collection and clinical care. This study uses deep learning and gaze-tracking to track pathologists' work and learn how they review tissue images. This “learned expertise” was applied to guide artificial intelligence models, such as weakly supervised learning and reinforcement learning, to achieve accurate diagnosis of Whole Slide Images.