Adaptive On-Device Model Update for Responsive Video Analytics in Adverse Environments

Abstract

While advanced lightweight models excel at real-time inference on resource-constrained end cameras in general scenarios, they often face limitations in adverse environments because of poor generalization ability. To achieve accurate inference in adverse environments, it becomes imperative to design adaptive model update strategies that can efficiently respond to the occurrence of adverse environments. In this paper, we propose a video analytics system that can continuously and responsively update the on-device lightweight model to handle various adverse environments. Our system consists of three modules, namely, a key frame extractor, a trigger controller, and a retraining manager. The key frame extractor identifies the most informative frames with minimal redundancy for bandwidth-efficient transmission. Those key frames are then used for potential model retraining and updating. Once the trigger controller detects a notable accuracy drop above an adaptive threshold within those selected key frames, it initiates the retraining process and evaluates the current urgency level. Then the retraining manager responds by generating the optimal retraining configuration that strikes a balance between inference accuracy and retraining latency. The retrained model is subsequently enforced to the end camera for responsive update. The designed system is prototyped on typical end devices and an edge server. Extensive experimental results under real-world datasets demonstrate that, the designed system is robust to handle various adverse environments, significantly improving the overall detection accuracy (up to 29%) and reducing more than 50% of the retraining time.

Topics & Concepts