MERL-Rice Near-INfrared Pulse (MR. NIRP) Indoor and Car dataset download [link].

There are several challenges for camera-based vital signs measurements unique to the driver monitoring context which current remote photoplethysmography (rPPG) algorithms cannot account for. There are drastic illumination changes on the driver’s face and the amount of motion during driving is significant.

We have built an active narrowband near infrared (NIR) illumination system with a matching narrow filter on the camera to significantly reduce the outside light variations reaching the driver’s face. Specifically, we have found that using 940 nm wavelength reduces sunlight effects the best. However the SNR of rPPG signals is much lower in NIR than in the visible wavelengths making our system more prone to noise.

To account for the low SNR of rPPG signals recorded with NIR cameras, we developed an optimization-based rPPG signal tracking and denoising algorithm (SparsePPG) based on Robust Principal Components Analysis and sparse frequency spectrum estimation. We have collected data in the lab and in the car with both NIR and broadband RGB cameras and we have shown that our NIR system performs better during driving than RGB and achieves comparable accuracy in the lab to the benchmark RGB camera.

I have interned and collaborated with Tim Marks and Hassan Mansour at Mitsubishi Electric Research Lab on this project. The initial results from this work were published in CVPR-CVPM 2018 [pdf] and a full version was published in IEEE Transactions on Intelligent Transportation Systems in 2020 [pdf] [video].

We received the best graduate poster and demo award for this work at 2019 ECE Corporate Affiliates Day at Rice [poster].