Smile Action Unit detection from distal wearable Electromyography and Computer Vision
Monica Perusquía-Hernández, Felix Dollack, Chun Kwang Tan, Shushi Namba, Saho Ayabe‐Kanamura, Kenji Suzuki
Abstract
Distal facial Electromyography (EMG) can be used to detect smiles and frowns with reasonable accuracy. It capitalises on volume conduction to detect relevant muscle activity, even when the electrodes are not placed directly on the source muscle. The main advantage of this method is to prevent occlusion and obstruction of the facial expression production, whilst allowing EMG measurements. However, measuring EMG distally entails that the exact source of the facial movement is unknown. Therefore, we investigated whether we could identify specific Facial Action Units (AUs) from distal facial EMG after an initial calibration phase with Computer Vision (CV). We compared Support Vector Machines (SVM) and Random Forest (RF) with several types of feature engineering and early fusion of the two modalities. The detection performance for AU6 (Orbicularis Oculi) and AU12 (Zygomaticus Major) was estimated by calculating the agreement with Facial Action Coding System (FACS) certified coders. The best results were achieved using Random Forest. Using a fusion of CV and EMG features resulted in F1 scores of 0.83 for AU6; and the fusion of engineered EMG plus CV returned an F1 score of 0.81 for AU12. Both these results are well above the CV baseline that shows F1 scores of 0.56 and 0.62 for AU6 and AU12 respectively. This demonstrates the potential of distal EMG to detect individual facial movements. It also enables researchers to compare the results measured with this wearable device to psychological research on facial expressions using FACS. Using a wearable enables measurements with higher ecological validity. Finally, we observed that EMG activity starts before the onset of visually perceived movement. Because of this, the agreement between EMG-based methods and FACS coders might be underestimating the ground truth.