Technology

We have developed novel computer-vision & AI-powered technology for data acquisition, analysis, and interpretation of vital signs.

Shen.AI analyzes facial skin texture and vital physiological signs in real-time. Our technology applies remote photoplethysmography (rPPG) – a contactless optical measurement technique of recording skin blood pulsations at different vascular depths. The user needs just a smartphone, tablet, or any other device equipped with a camera.

The rPPG signal represents beat-to-beat pulsatile fluctuations in the intensity of light reflected from the skin. While these fluctuations remain invisible to the human eye, they can be detected by a simple camera. The captured signals are then analyzed computationally to estimate various cardiovascular parameters. 

An advanced image stabilization algorithm guarantees the best performance and reliable facial texture extraction. By rendering the video frame by frame, a parameterized exact 3D model of facial skin is generated.

Shen.AI captures and analyzes the following cardiovascular parameters: Heart Rate, Blood Pressure, Heart Rate Variability (HRV), Respiratory Rate. The future features also include parameters like: SpO2, Body Weight as well:

Mental markets: stress index, energy level, anxiety, basic emotions

Dermatological: skin type, hydration, early prediction of skin diseses 

Ophthalmological: abnormalities of the iris, inflammations, macular degeneration.

Shen.AI API can be integrated in mobile apps and web pages as a white-label solution. Bring more value to your customers and grow your business by embedding easy and user-friendly vital signs analysis in your digital product. 

Shen.AI API can be integrated in mobile apps and web pages as a white-label solution. Bring more value to your customers and grow your business by embedding easy and user-friendly vital signs analysis in your digital product

How it works

Visual Data Acquisition

RGB video of facial skin is captured from a camera. Next, it is stabilized and wrapped onto a 3D mesh – a customized representation of face geometry. It is then transformed (mesh morphing, field of numerical simulation) to create a 3D model reflecting the face anatomy. The 3D mask made in this way is precisely tracked while the image is projected onto it every frame, resulting in a stable video of the face texture, UV-parameterized for further analysis.

Signal Processing

By employing data processing techniques, the captured images are converted. As a result, RGB color information is transposed into rPPG signal in order to extract a low-noise heart pulse signal for further analysis of every pixel of the skin image. Such an optimized model is a rich information source for the Machine Learning (ML) pipeline that analyzes each pulse wave’s timing, shape, amplitude, and other features.

AI analysis

Machine learning (ML) models use the extracted signals to precisely measure blood pressure and other vital signs – the system is able to assess the risk of cardiovascular diseases (CVD) or those related to skin or eye health.

The science

Our technology leverages research, science, and innovation to ensure the highest accuracy and reliability. Here is the list of scientific papers we have applied while developing Shen.AI:

Existing scientific papers:

Estimating Blood Pressure from the Photoplethysmogram Signal and Demographic Features Using Machine Learning Techniques Read more
Blood Pressure Estimation from Photoplethysmogram Using a Spectro-Temporal Deep Neural Network Read more
Cardiovascular assessment by imaging photoplethysmography - a review Read more
Algorithmic principles of remote PPG Read more
A healthy heart is not a metronome: an integrative review of the heart's anatomy and heart rate variability Read more

Applications

Mobile health applications. Integration of health metrics tool available for anyone, anytime only with smartphone brings prevention and self-care to a whole new level. An access to such diagnostic solution increases user engagement, re and provides with a historical data. 

Existing and potential applications of the Shen.AI platform include:

Mobile health/well-being applications. Introduce new features to your app. Find out how vital signs measurement can expand the functionality of your solution, increasing its usability, improving user experience as well grow your sales. 

  • Individual health assessment 
  • Skin health, beauty, and wellness
  • Mental health coaches and therapies 
  • Telecare and remote consultations
  • Digital health tests
  • Building habits
  • Meditation, fitness, relaxation, breathing
  • Stress reduction
  • Medical advice 
  • Women’s health

Hardware. Whether it’s a device for individuals aiming to prevent diseases actively, patients managing their own health, a robot, or one of the smart gadgets, Shen.AI helps capture data that matters. Easily integrable and interoperable standards make it effortless to enhance the power of a wide range of emerging technologies. 

  • Smart mirrors
  • Fitness screens (standalone or integrated)
  • Concentration and performance monitors
  • Driver attention monitors
  • Companion robots
  • Telepresence solutions

Telemedicine and healthcare. Use the power of every smartphone for diagnostics purposes. No external devices are required. If integrated with telemedicine or healthcare applications, Shen.AI provides input data for initial remote medical triage or precise vital signs monitoring: blood pressure, heart rate, HRV, and many more. They can then be used for the early identification of different diseases – with cardiovascular disorders at the forefront – shortening the critical time between the diagnosis and treatment. 

  • Remote medical triage 
  • Contactless vital signs measurement
  • Remote care and monitoring for elderly
  • Regular health check-ups 
  • Long-term health trends tracking for prevention 
  • Monitoring and managing of chronic diseases
  • First diagnostics aid and health assessment in rural areas 

applications