The technology behind Shen AI

‍

Behind every Shen AI measurement are two complementary sensing technologies: remote photoplethysmography (rPPG) and remote ballistocardiography (rBCG). Together, they capture subtle physiological signals that can be analyzed to estimate metrics such as heart rate, blood pressure, breathing rate, heart rate variability (HRV), and more.

In the following video, Shen AI CTO Przemek Jaworski explains how rPPG and rBCG work together to enable camera-based vital sign monitoring.

‍

‍

Remote photoplethysmography (rPPG)

rPPG measures tiny changes in skin color caused by blood flowing through the blood vessels with each heartbeat.

These changes are invisible to the human eye but can be detected by a camera and processed using computer vision algorithms. From this signal, Shen AI extracts information that supports measurements such as heart rate, breathing rate, blood pressure estimation, heart rate variability (HRV), and other physiological markers.

‍

Remote ballistocardiography (rBCG)

While rPPG looks at changes in skin color, rBCG analyzes extremely small head movements caused by the mechanical action of the heart pumping blood through the body.

These movements are also invisible during normal use, but they provide additional information about cardiovascular activity.

Because rBCG is based on motion rather than color, it complements the rPPG signal and helps strengthen the overall measurement.

‍

Why Shen AI uses both

No single physiological signal is perfect in every situation. Lighting conditions, skin characteristics, camera quality, and natural user movement can all affect the quality of a measurement.

By combining information from both rPPG and rBCG, Shen AI uses a multimodal approach that improves signal robustness and provides additional information for the SDK's machine learning models. Rather than relying on a single source of data, the SDK analyzes multiple physiological signals captured during the same measurement.

This multimodal sensing approach was introduced in Shen AI SDK 2.9 and is now part of the standard measurement pipeline.

‍