Anti-motion imaging photoplethysmography via self-adaptive multi-ROI tracking and selection

Objective. The imaging photoplethysmography (IPPG) technique allows people to measure heart rate (HR) from face videos. However, motion artifacts caused by rigid head movements and nonrigid facial muscular movements are one of the key challenges. Approach. This paper proposes a self-adaptive region of interest (ROI) pre-tracking and signal selection method to resist motion artifacts. Based on robust facial landmark detection, we split the whole facial skin (including the forehead, cheeks, and chin) symmetrically into small circular regions. And two symmetric sub-regions constitute a complete ROI. These ROIs are tracked and the motion state is simultaneously assessed to automatically determine the visibility of these ROIs. The obscured or invisible sub-regions will be discarded while the corresponding symmetric sub-regions will be retained as available ROIs to ensure the continuity of the IPPG signal. In addition, based on the frequency spectrum features of IPPG signals extracted from different ROIs, a self-adaptive selection module is constructed to select the optimum IPPG signal for HR calculation. All these operations are updated per frame dynamically for the real-time monitor. Results. Experimental results on the four public databases show that the IPPG signal derived by our proposed method exhibits higher quality for more accurate HR estimation. Compared with the previous method, metrics of the evaluated HR value on our approach demonstrates superior or comparable performance on PURE, VIPL-HR, UBFC-RPPG and MAHNOB-HCI datasets. For instance, the RMSEs on PURE, VIPL-HR, and UBFC-RPPG datasets decrease from 4.29, 7.62, and 3.80 to 4.15, 3.87, and 3.35, respectively. Significance. Our proposed method can help enhance the robustness of IPPG in real applications, especially given motion disturbances.