ContextLLM: Meaningful Context Reasoning from Multi-Sensor and Multi-Device Data Using LLMs
Kevin Post, Reo Kuchida, Mayowa Olapade, Zhigang Yin, Petteri Nurmi, Huber Flores
Abstract
Conventional context awareness and activity recognition models produce abstract outputs that offer limited insights into user behavior and situational context. These can be significantly enhanced by leveraging multi-sensor and multi-device data streams. However, the aggregation and modelling of context sensor data presents complex challenges that require advanced inference capabilities. We introduce ContextLLM, a context-driven solution powered by Large Language Models (LLMs), designed to transform sparse, abstract insights from various sensors and devices into a detailed, descriptive context. Through rigorous experiments using a well-established benchmark dataset for activity recognition, we demonstrate that ContextLLM can significantly enhance context understanding. However, our analysis also highlights how the quality and complexity of sensor data representations impact the LLM's ability to accurately deduce context. Building on these findings, we develop a research agenda that outlines key challenges, and conclude with a discussion on the limitations and practical considerations of LLM-based reasoning in context-aware applications.