SAIL: A Deep-Learning-Based System for Automatic Gait Assessment From TUG Videos
Yanhong Wang, Qiaosha Zou, Yanmin Tang, Qing Wang, Jing Ding, Xin Wang, C.‐J. Richard Shi
Abstract
Gait disorders are common in the elderly people, seriously hinder patients’ mobility and sometimes indicate underlying severe neurological diseases. Timely and automatic diagnosis of gait disorders is greatly desired. Existing methods with wearable devices put burdens on patients. We establish a video-based algorithm named <i>SAIL</i> to perform contactless gait assessment automatically. The SAIL contains three parts, namely, <i>skeleton detector</i>, <i>parameter extractor</i>, and <i>gait classifier</i>. Using a pose estimation algorithm, the skeleton detector converts RGB videos to a human skeleton sequence. Then, the parameter extractor extracts gait parameters from skeletons with a signal detection technique. Finally, a trained Support vector machine is used as a gait classifier to detect abnormal gait. The SAIL achieves 86.2% sensitivity and 98.5% specificity for abnormal gait detection on our <i>SAIL-TUG</i> dataset, outperforming general clinic doctors with 76.4% and 97.4%, respectively. Nine gait parameters and the binary gait classification result are included in the final gait report. We implement an automatic gait assessment system based on SAIL and deployed the user-interface software in more than 60 hospitals for practical applications. More than 30 000 gait reports have been automatically generated. Moreover, we establish a publicly available dataset named <i>SAIL-TUG</i> including 404 annotated Timed “Up & Go” videos.