A Deep‐Learning Framework for the Automated Recognition of Molecules in Scanning‐Probe‐Microscopy Images
Zhiwen Zhu, Jiayi Lu, Fengru Zheng, Cheng Chen, Yang Lv, Hao Jiang, Yuyi Yan, Akimitsu Narita, Kläus Müllen, Xiaoye Wang, Qiang Sun
Abstract
Computer vision as a subcategory of deep learning tackles complex vision tasks by dealing with data of images. Molecular images with exceptionally high resolution have been achieved thanks to the development of techniques like scanning probe microscopy (SPM). However, extracting useful information from SPM image data requires careful analysis which heavily relies on human supervision. In this work, we develop a deep learning framework using an advanced computer vision algorithm, Mask R-CNN, to address the challenge of molecule detection, classification and instance segmentation in binary molecular nanostructures. We employ the framework to determine two triangular-shaped molecules of similar STM appearance. Our framework could accurately differentiate two molecules and label their positions. We foresee that the application of computer vision in SPM images will become an indispensable part in the field, accelerating data mining and the discovery of new materials.