Computer Vision for Gait Assessment in Cerebral Palsy: Metric Learning and Confidence Estimation
Peijun Zhao, Moisés Alencastre-Miranda, Zhan Shen, Ciaran O’Neill, David Whiteman, Javier Gervas‐Arruga, Hermano Igo Krebs
Abstract
Assessing the motor impairments of individuals with neurological disorders holds significant importance in clinical practice. Currently, these clinical assessments are time-intensive and depend on qualitative scales administered by trained healthcare professionals at the clinic. These evaluations provide only coarse snapshots of a person’s abilities, failing to track quantitatively the detail and minutiae of recovery over time. To overcome these limitations, we introduce a novel machine learning approach that can be administered anywhere including home. It leverages a spatial-temporal graph convolutional network (STGCN) to extract motion characteristics from pose data obtained from monocular video captured by portable devices like smartphones and tablets. We propose an end-to-end model, achieving an accuracy rate of approximately <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${76}.{6}\%$ </tex-math></inline-formula> in assessing children with Cerebral Palsy (CP) using the Gross Motor Function Classification System (GMFCS). This represents a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${5}\%$ </tex-math></inline-formula> improvement in accuracy compared to the current state-of-the-art techniques and demonstrates strong agreement with professional assessments, as indicated by the weighted Cohen’s Kappa (<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\kappa _{\textit {lw}} = {0}.{733}$ </tex-math></inline-formula>). In addition, we introduce the use of metric learning through triplet loss and self-supervised training to better handle situations with a limited number of training samples and enable confidence estimation. Setting a confidence threshold at <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${0}.{95}$ </tex-math></inline-formula>, we attain an impressive estimation accuracy of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${88}\%$ </tex-math></inline-formula>. Notably, our method can be efficiently implemented on a wide range of mobile devices, providing real-time or near real-time results.