Remote Physiological Measurement in an Open World

Many life-sustaining vital signs can be measured optically using a camera. Of particular interest is remote photoplethysmography (rPPG), a technique for non-contact blood volume pulse estimation from video. This presents a unique opportunity for monitoring health at a global scale using accessible consumer devices such as mobile phones. Yet, estimating these signals outside of controlled laboratory settings is still an unsolved and challenging problem. This dissertation proposes new data-driven methods to accurately estimate vital signs from video. We collected multiple video datasets of subjects moving and talking with simultaneous physiological ground truth to train and validate rPPG.

After finding the immense promise of deep learning for remote vitals estimation, we reveal neural network's susceptibility to adversarial attacks and ``hallucination'' when anomalies occur. Next, we explore the unique property of camera-based systems to perform spatial measurements. By examining the pulse wave’s time lags between different peripheral sites on the body (e.g. hands, face, arms, and legs), we show that rPPG can be used to estimate the pulse transit time. Given the challenges of collecting diverse video datasets with ground truth, we implemented a new non-contrastive unsupervised method for training artificial neural networks to learn rPPG from video without labels. We find that this framework is effective with very little video, enabling personalized and adaptive models for camera-based physiological measurement. Lastly, we show that the proposed framework is general enough for any type of band-limited periodic signal by applying it to remote respiration estimation.