RS-rPPG: Robust Self-Supervised Learning for rPPG

Abstract

Remote photoplethysmography (rPPG) measures cardiac signals remotely from facial videos, leading to promising applications in telemedicine, face anti-spoofing, emotion analysis, etc. However, recent supervised approaches are limited by data scarcity and current self-supervised rPPG methods struggle to learn physiological features from data recorded in challenging scenarios, which contain overwhelming environmental noise caused by head movements, illumination variations, and recording device changes. We propose a novel contrastive framework that leverages a large set of priors, that enable learning robust and transferable features even from challenging datasets. Ours is the first method to focus on self-supervised learning on challenging data and the first method to use such a large set of priors. The priors include a novel traditional augmentation method, leveraging spatial-temporal maps and self-attention based transformer for SSL. We show that it outperforms current self-supervised methods on four public datasets, especially on the more challenging data where it reaches close to supervised performance. Our code is available at: https://github.com/marukosan93/RS-rPPG

Topics & Concepts