Towards LLM-Powered Ambient Sensor Based Multi-Person Human Activity Recognition
Xi Chen, Julien Cumin, Fano Ramparany, Dominique Vaufreydaz
Abstract
Human Activity Recognition (HAR) is one of the central problems in fields such as healthcare, elderly care, and security at home. However, traditional ambient-sensor-based HAR approaches face challenges including data scarcity, difficulties in model generalization, and the complexity of recognizing activities in multi-person scenarios. This paper proposes a large-language-model-based framework called LAHAR which addresses HAR in multi-person scenarios. By endowing LLMs with inter-sensor relevance estimation and sensor-subject relevance estimation abilities, LAHAR can assign sensor events to the corresponding subjects. By providing action-level descriptions of sensor events and subsequently performing activity-level reasoning based on these descriptions, LAHAR is ultimately able to process data spanning several tens of hours with second-level resolution and results in an activity timeline for each subject. We validated LAHAR on the ARAS dataset. The results demonstrate that LAHAR achieves comparable accuracy to the state-of-the-art method at higher resolutions and maintains robustness in multiperson scenarios.