Technology

We have developed novel computer-vision & AI-powered technology for data acquisition, analysis, and interpretation of vital signs.

Shen.AI analyzes facial skin texture and vital physiological signs in real-time. Our technology applies remote photoplethysmography (rPPG) – a contactless optical measurement technique of recording skin blood pulsations at different vascular depths. The user needs just a smartphone, tablet, or any other device equipped with a camera.

The rPPG signal represents beat-to-beat pulsatile fluctuations in the intensity of light reflected from the skin. While these fluctuations remain invisible to the human eye, they can be detected by a simple camera. The captured signals are then analyzed computationally to estimate various cardiovascular parameters.

An advanced image stabilization algorithm guarantees the best performance and reliable facial texture extraction. By rendering the video frame by frame, a parameterized exact 3D model of facial skin is generated.

Shen.AI captures and analyzes the following cardiovascular parameters: Heart Rate, Blood Pressure, Heart Rate Variability (HRV), Respiratory Rate. The future features also include parameters like: SpO2, Body Weight as well:

Mental markets: stress index, energy level, anxiety, basic emotions

Dermatological: skin type, hydration, early prediction of skin diseses

Ophthalmological: abnormalities of the iris, inflammations, macular degeneration.

Shen.AI API can be integrated in mobile apps and web pages as a white-label solution. Bring more value to your customers and grow your business by embedding easy and user-friendly vital signs analysis in your digital product.

How it works

Visual Data Acquisition

RGB video of facial skin is captured from a camera. Next, it is stabilized and wrapped onto a 3D mesh – a customized representation of face geometry. It is then transformed (mesh morphing, field of numerical simulation) to create a 3D model reflecting the face anatomy. The 3D mask made in this way is precisely tracked while the image is projected onto it every frame, resulting in a stable video of the face texture, UV-parameterized for further analysis.

Signal Processing

By employing data processing techniques, the captured images are converted. As a result, RGB color information is transposed into rPPG signal in order to extract a low-noise heart pulse signal for further analysis of every pixel of the skin image. Such an optimized model is a rich information source for the Machine Learning (ML) pipeline that analyzes each pulse wave’s timing, shape, amplitude, and other features.

AI analysis

Machine learning (ML) models use the extracted signals to precisely measure blood pressure and other vital signs – the system is able to assess the risk of cardiovascular diseases (CVD) or those related to skin or eye health.

The science

Our technology leverages research, science, and innovation to ensure the highest accuracy and reliability. Here is the list of scientific papers we have applied while developing Shen.AI:

Existing scientific papers:

	Estimating Blood Pressure from the Photoplethysmogram Signal and Demographic Features Using Machine Learning Techniques	Read more
	Blood Pressure Estimation from Photoplethysmogram Using a Spectro-Temporal Deep Neural Network	Read more
	Cardiovascular assessment by imaging photoplethysmography - a review	Read more
	Algorithmic principles of remote PPG	Read more
	A healthy heart is not a metronome: an integrative review of the heart's anatomy and heart rate variability	Read more