← All topics
05
Technology
6 min

Technology

How Shen AI measures physiological signals

Shen AI uses a standard camera to capture subtle changes in facial appearance and movement. Its technology combines multiple signals from a single video stream to extract physiological information without dedicated measurement hardware.

The approach brings together signal processing, computer vision and AI to turn camera data into physiological measurements, while continuously accounting for factors that can affect signal quality.

From camera to physiological signals

A Shen AI measurement starts with a live video stream from the user's camera.

The SDK identifies and tracks the relevant facial region, extracts physiological signals from the video, reduces noise and analyzes the resulting information. The processed signals are then interpreted by Shen AI's AI models to produce the selected measurement results.

This process takes place locally on the user's device or within the web browser, depending on the implementation.

Two complementary sensing modalities

A core part of Shen AI's technology is the combination of remote photoplethysmography (rPPG) and remote ballistocardiography (rBCG) from a single video stream.

rPPG

Remote photoplethysmography detects subtle changes in skin color associated with changes in blood volume caused by the cardiovascular cycle.

These changes are not visible to the human eye, but they can be captured by a camera and extracted as a physiological signal.

rBCG

Remote ballistocardiography captures tiny movements of the head and body caused by the mechanical activity of the heartbeat.

This provides a complementary source of physiological information to the optical signal captured through rPPG.

‍

The Multisensing Engine

Rather than relying on one signal alone, Shen AI combines information from both modalities.

This dual-modality approach is designed to improve the robustness of physiological measurement across different devices and real-world conditions. It can help the system maintain useful signal information when one source is affected by factors such as lighting or minor movement.

At the center of this approach is the Multisensing Engine — Shen AI's technology for combining and interpreting multiple physiological signals captured through the camera.

The engine brings together the different stages of the measurement pipeline, from signal extraction and processing to AI-based inference. It is designed to distinguish relevant physiological information from noise and other variations present in real-world camera data.

This allows Shen AI to use complementary information from the same video stream rather than depending on a single sensing method.

Designed for real-world conditions

Shen AI is designed to work with everyday cameras rather than dedicated physiological sensors.

The technology is hardware-agnostic and can be integrated into products using cameras available on common consumer devices. At the same time, the quality of the camera input and the conditions in which a measurement takes place still matter.

Lighting, face position, movement, camera characteristics and other environmental factors can affect the captured signal. Shen AI therefore continuously evaluates signal quality during a measurement and can provide guidance when conditions need to be improved.

Learn more about signal quality & measurement confidence →

From raw video to usable information

The measurement process can be viewed as a series of connected stages:

Camera input → Face detection & tracking → Signal extraction → Signal processing → Multisensing Engine → AI inference → Measurement results

Each stage contributes to turning a raw video stream into meaningful physiological information.

The modular architecture also allows Shen AI's measurement technology to be integrated into different products and user experiences through the SDK, without requiring customers to build the underlying signal-processing and inference technology themselves.

Built for changing conditions

Camera-based measurement does not happen in a controlled laboratory every time. Users may move slightly, lighting may change, and cameras differ between devices.

Shen AI's technology is designed and evaluated with these real-world variables in mind. The combination of rPPG and rBCG, together with continuous signal-quality assessment, helps the system identify and work with the physiological information available in each measurement.

Clinical studies and technical evaluations provide additional evidence of how the technology performs under defined conditions.

Explore clinical evidence & validation →