CCSpO<sub>2</sub>Net: Camera-Based Contactless Oxygen Saturation Measurement Foundation Model in Clinical Settings
Xiantao Sun, T.C. Wen, Weihai Chen, Bin Huang
Abstract
Oxygen saturation (SpO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> ) is a vital parameter in clinical practice. Clinical research suggests that SpO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> may significantly decrease in patients with lung infections before evident symptoms, such as COVID-19, appear. Therefore, monitoring the cardiopulmonary status of individuals with lung disease is indispensable. Although contactless SpO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> measurements perform well in laboratory settings, their clinical application still faces several challenges. To bridge this gap, a camera-based contactless SpO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> measurement foundation model called CCSpO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> Net is proposed, which is appropriate for both clinical and laboratory settings. This approach is based on a spatial feature extractor (SFE) and a global temporal feature extractor and estimator (GTFEE). The SFE employs two residual networks to enhance interchannel correlations and capture spatial information. The GTFEE module consists of a multilayer perceptron mixer (MLP-Mixer) network and an SpO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> estimator. The MLP-Mixer operation facilitates global temporal information exchange across positions by rearranging and swapping the features along the channel dimension. Additionally, the Segment Anything Model (SAM) was first utilized to accurately extract skin regions at the pixel level to measure SpO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> . Finally, we are the first to explore the topic of contactless SpO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> estimation based on facial videos in a clinical environment; remarkable results were achieved, with a mean absolute error (MAE) of 1.79 and a root mean square error (RMSE) of 2.85 on the intensive care unit (ICU) dataset. Moreover, the extensive experimental results further validate that CCSpO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> Net outperforms the state-of-the-art approaches significantly, achieving MAEs of 0.54 (improved by 16.7%) and 2.48 (improved by 101.6%) on the PURE and smartphone camera oximetry (SCO) datasets, respectively.