PulseGAN: Learning to generate realistic pulse waveforms in remote photoplethysmography
About
Remote photoplethysmography (rPPG) is a non-contact technique for measuring cardiac signals from facial videos. High-quality rPPG pulse signals are urgently demanded in many fields, such as health monitoring and emotion recognition. However, most of the existing rPPG methods can only be used to get average heart rate (HR) values due to the limitation of inaccurate pulse signals. In this paper, a new framework based on generative adversarial network, called PulseGAN, is introduced to generate realistic rPPG pulse signals through denoising the chrominance signals. Considering that the cardiac signal is quasi-periodic and has apparent time-frequency characteristics, the error losses defined in time and spectrum domains are both employed with the adversarial loss to enforce the model generating accurate pulse waveforms as its reference. The proposed framework is tested on the public UBFC-RPPG database in both within-database and cross-database configurations. The results show that the PulseGAN framework can effectively improve the waveform quality, thereby enhancing the accuracy of HR, the heart rate variability (HRV) and the interbeat interval (IBI). The proposed method achieves the best performance compared to the denoising autoencoder (DAE) and CHROM, with the mean absolute error of AVNN (the average of all normal-to-normal intervals) improving 20.85% and 41.19%, and the mean absolute error of SDNN (the standard deviation of all NN intervals) improving 20.28% and 37.53%, respectively, in the cross-database test. This framework can be easily extended to other existing deep learning based rPPG methods, which is expected to expand the application scope of rPPG techniques.