Human-Human Interaction Recognition Using Mask R-CNN and Multi-Class SVM

Abstract

Computer vision systems face significant challenges in human-human interaction recognition in dynamic environments, particularly with cluttered backgrounds. In this paper, we propose a novel approach that leverages OpenPose for accurate keypoint detection and Mask R-CNN, pre-trained on the COCO dataset, for precise human segmentation-chosen for their robustness in handling occlusions and complex interactions. To optimize the extracted features, we apply Linear Discriminant Analysis (LDA) for dimensionality reduction and class separation, ensuring efficient classification using a multi-class support vector machine (SVM). By utilizing these pre-trained models, our system achieves high accuracy with a reduced feature set, demonstrating that excellent performance can be achieved without extensive retraining. We classify six distinct interaction types-including handshaking, pushing, kicking, and punching-while addressing challenges such as fast motion and interaction complexity within the dataset. The system achieves competitive accuracy on the UT Interaction dataset (89.1% on Set 1, 86.6% on Set 2) and demonstrates computational efficiency, making it scalable for real-world applications like public surveillance.

Topics & Concepts