Deep learning-based activity recognition and fine motor identification using 2D skeletons of cynomolgus monkeys
Chuxi Li, 复旦大学信息科学与工程学院微纳中心, 上海 200433, 中国, Zifan Xiao, Yerong Li, Zhi‐Nan Chen, Xun Ji, Yiqun Liu, Shufei Feng, Zhen Zhang, Kaiming Zhang, Jianfeng Feng, Trevor W. Robbins, Shisheng Xiong, Yongchang Chen, Xiao Xiao, 复旦大学类脑智能研究院认知神经科学中心; 复旦大学华山医院麻醉科; 复旦大学脑科学前沿科学中心; 计算神经科学与类脑智能教育部重点实验室, 上海 200433, 中国, 南京大学匡亚明学院, 江苏 南京 210023, 中国, 复旦大学计算机科学技术学院; 上海市智能信息处理重点实验室, 上海 200433, 中国, 昆明理工大学灵长类转化医学研究院; 省部共建非人灵长类生物医学国家重点实验室, 云南 昆明 650500, 中国, 新视国际责任有限公司, 加利福尼亚 92656, 美国, 剑桥大学行为与临床神经科学研究所, 剑桥 CB2 1TN, 英国
Abstract
Video-based action recognition is becoming a vital tool in clinical research and neuroscientific study for disorder detection and prediction. However, action recognition currently used in non-human primate (NHP) research relies heavily on intense manual labor and lacks standardized assessment. In this work, we established two standard benchmark datasets of NHPs in the laboratory: MonkeyinLab (MiL), which includes 13 categories of actions and postures, and MiL2D, which includes sequences of two-dimensional (2D) skeleton features. Furthermore, based on recent methodological advances in deep learning and skeleton visualization, we introduced the MonkeyMonitorKit (MonKit) toolbox for automatic action recognition, posture estimation, and identification of fine motor activity in monkeys. Using the datasets and MonKit, we evaluated the daily behaviors of wild-type cynomolgus monkeys within their home cages and experimental environments and compared these observations with the behaviors exhibited by cynomolgus monkeys possessing mutations in the <i>MECP2</i> gene as a disease model of Rett syndrome (RTT). MonKit was used to assess motor function, stereotyped behaviors, and depressive phenotypes, with the outcomes compared with human manual detection. MonKit established consistent criteria for identifying behavior in NHPs with high accuracy and efficiency, thus providing a novel and comprehensive tool for assessing phenotypic behavior in monkeys.