AI-powered heart rate monitoring gets even more accurate

Heart rate is one of the most basic health metrics, used for cardiovascular assessment, stress monitoring and disease detection. Traditional methods like ECG and pulse oximeters need direct skin contact and dedicated medical devices. But video photoplethysmography (vPPG) offers a new way— contactless, camera-based heart rate monitoring using only a smartphone or laptop.

A new study has validated Shen’s vPPG, demonstrating accuracy comparable to medical device measurements and proving it’s a reliable and accessible tool for real-time heart rate measurement.

Key insights

  • Shen’s vPPG shows accuracy comparable to ECG. Video-based heart rate measurements have an average error as low as 0.1 bpm, with a correlation coefficient above 0.99, matching ECG-based readings.
  • It works for shorter measurements as well. The technology maintained high precision for 60-second average heart rate as well as 10-second or 4-second periods with less than 1 bpm error in 94.5% to 100% of cases.
  • AI health monitoring offers wider accessibility. By eliminating the need for specialized hardware, Shen AI makes accurate, real-time health tracking possible for everyone, with just a smartphone or laptop.

Research introduction

The following is an excerpt of the “Facial video photoplethysmography for measuring average and instantaneous heart rate: a pilot validation study” paper.

Heart rate (HR), or pulse rate, is one of the vital signs used in medical examinations for the basic assessment of the condition of the cardiovascular system. Resting HR can help detect various diseases and predict cardiovascular and all-cause mortality. In particular, a UK Biobank study on over 500,000 individuals followed for up to 12 years showed that a 10-bpm increase in resting HR was associated with a 22% and 19% greater risk for all-cause mortality in men and women, respectively. A meta-analysis including over 1.2m individuals followed for up to 40 years showed that higher resting HR was associated with increased cardiovascular and all-cause mortality, independent of traditional cardiovascular risk factors. Moreover, an increase in resting HR over time has also been shown to be associated with higher all-cause mortality, with the risk of death increasing by 33% for every 10-bpm increase over six years. Therefore, regular HR assessment is an important part of prevention. HR also reflects the body’s response to stress, emotions, exercise, or other stimuli, and hence it can be used for self-monitoring in both health and wellness context. It can also be used for monitoring of the cardiac rehabilitation process as well as to monitor fatigue and recovery process in athletes to prevent overtraining and optimize training effectiveness. Moreover, it may be used to monitor drivers’ physiological state or to monitor human-computer interactions.

Photoplethysmography (PPG), i.e. an optical technique of detecting blood pulsations in the skin vasculature, is commonly used for measurements or continuous monitoring of HR and other vital signs by means of special probes/clips attached typically to a finger or ear lobe. PPG sensors are also increasingly being integrated into a variety of wearable devices, such as smartwatches or wristbands. The two main limitations of classic PPG-based measurements are: 1) the need for a special device, such as a pulse oximeter or a wearable equipped with PPG technology, which naturally limits the availability of such measurements, and 2) the need for skin contact, which is usually not as important in the case of self-measurements using one’s own device, but may be more important in healthcare facilities or in the case of measurements taken by another person or using someone else’s device, especially in the event of an epidemic.

The answer to the above limitations may be video-based PPG (vPPG), i.e. a remote PPG technique, also known as remote or imaging photoplethysmography, that uses digital video images of the skin to detect tiny changes in skin colour caused by blood pulsations in superficial blood vessels and the resulting changes in the blood absorption of light incident on the skin (mainly by haemoglobin). Such vPPG measurements could be performed using special light sources (e.g. LEDs with specific wavelengths) and a computer software to analyse video images from professional digital cameras.

However, such measurements can also be performed using ambient, white light (natural or artificial) as the source of light illuminating the skin, a consumer-grade camera integrated into a smartphone as the image sensor, and the smartphone processing power to analyse video images using a mobile app, thus making this technology accessible to most smartphone users without the need for any other device and without requiring skin contact. The possibility of using vPPG for contactless monitoring of HR and other vital signs has attracted a lot of attention in recent years, including the possibility of remote measurement of vital signs in telemedicine applications, for patient triage
purposes, or for monitoring drivers.

In this study, we investigated vPPG technology developed by MX Labs (Tallinn, Estonia) called Shen.AI Vitals. This technology uses face detection and tracking algorithms to obtain vPPG signals from several regions of facial skin during a 1-min video recording and then employs various signal processing algorithms to analyse and combine information from these signals (in the red, green, and blue channels) to estimate HR as well as other physiological parameters. In particular, two types of HR values are provided -– after the measurement, HR averaged over the entire 1 minute is provided, whereas during the measurement (every 1 second), average values from shorter periods are provided, i.e. the average HR from the previous 10 seconds (default) or the previous 4 seconds (optional).

The aim of our study was to assess the accuracy and precision of HR measurements performed using the tested vPPG technology and a smartphone camera by comparing them with reference values obtained from a simultaneously recorded electrocardiogram (ECG).

Research details

Title: Facial Video Photoplethysmography for Measuring Average and Instantaneous Heart Rate: A Pilot Validation Study
Authors: Leszek Pstras, Tymoteusz Okupnik, Beata Ponikowska, Bartlomiej Paleczny
Institutions: Nalecz Institute of Biocybernetics and Biomedical Engineering, Polish Academy of Sciences & Wroclaw Medical University
Published February 16, 2025

Read the full study here.