Shen.AI is a novel computer vision and AI-powered platform designed for the acquisition, analysis, and interpretation of vital physiological signs. This technology utilizes remote photoplethysmography (rPPG), a non-contact optical method for detecting blood volume changes in the skin, which reflect the physiological condition of the subject. It analyzes facial skin texture and extracts vital signs in real-time from the variations in light absorption, captured within the RGB pixel values of camera frames.
Encapsulated as a Software Development Kit (SDK), Shen.AI’s technology is engineered for integration across a variety of devices and frameworks, enabling its incorporation into third-party mobile or web applications. This versatility makes the technology accessible for widespread use in different digital environments.
The technology is designed to operate locally on the user’s end-device, ensuring that all computations and the resulting data are immediately available on-device after the measurement. This offline functionality is critical for real-time application and enhances user privacy.
Shen.AI’s approach is underpinned by proprietary deep learning AI models, trained on an extensive dataset comprising over 5 million heartbeats from 50,000 recordings. This robust dataset, collected by Shen.AI, ensures high accuracy and reliability in blood pressure estimation using AI algorithms.
Furthermore, Shen.AI’s technology has undergone clinical validation studies for the certification as a medical
device, focusing on the precision and accuracy of heart rate, heart rate variability, and breathing rate measurements. The health risk calculations performed by Shen.AI are based on established models such as the Framingham Heart Study.
Developed by a internal team of Engineers and AI experts and 10 PhDs with expertise in scientific and medical fields, Shen.AI represents a significant advancement in remote health monitoring, providing insights into physical health, disease detection, and recovery monitoring.
Accuracy and stability of our technology is of the utmost importance to us. We conducted clinical validation studies to prepare for the certification of Shen.AI as a medical device. The study concerned precision and accuracy of heart rate, heart rate variability, and breathing rate measurements.
Check the Clinical Trial Report from the study about Shen.AI accuracy.
Measurement accuracy
MD | SD | MAE | RMSE | |||
Heart rate (HR) * | average (60 s) instantaneous (10 s) instantaneous (4 s) | 0.1 0.1 0.1 | 0.4 0.5 0.8 | 0.1 0.2 0.4 | 0.4 0.5 0.8 | bpm |
Heart rate variability (HRV) * | SDNN (60 s) InRMSSD (60 s) | 2.8 0.2 | 3.6 0.2 | 3.5 0.2 | 4.5 0.3 | ms – |
Breathing rate (BR) * | average (60 s) | 0.2 | 1.5 | 1.2 | 1.5 | bpm |
Systolic Blood Pressure (SBP) | average (60s) | – | 10.9 | 8.45 | – | mmHg |
Diastolic Blood Pressure (DBP) | average (60s) | – | 7.13 | 5.56 | – | mmHg |
Our technology leverages research, science, and innovation to ensure the highest accuracy and reliability. Here is the list of scientific papers that provide context to our technology.
The two factors that are important in rPPG measurement are lighting and stability. Our engineers created top of the line algorithms for stabilization and normalization to achieve the widest possible spectrum of conditions in which measurements can be taken. It is like a traditional device like BP-cuffs – there are specific instructions to provide a good measurement.
For optimal results with Shen.AI technology, follow these simple steps:
- Sit comfortably in well-lit surroundings (around 400 – 500 lx).
- Ensure your hardware meets Shen.AI’s requirements.
- Relax in a seated position for at least 5 minutes before starting your measurement to stabilize your circulatory and respiratory systems.
- During measurement, maintain normal breathing and keep your head steady. Avoid talking or making facial expressions.
- Position your face correctly in front of the camera to fit within the screen frame
- When ready, simply press the START button.
Don’t rush the 5-minute relaxation; it’s key to accuracy. For more in-depth guidance, download our comprehensive guide on using Shen.AI.
Blood pressure is never constant and can fluctuate significantly over short intervals. Shen.AI measures blood pressure continuously for 60 seconds, yielding an average value. Factors like movement, speech, and cuff-based measurement can impact readings. Traditional cuff-based methods capture a momentary snapshot, while Shen.AI provides an averaged 60-second value.
Discrepancies between the two methods are normal, with reference devices having an error standard deviation of 3-5 mmHg. Consult a healthcare professional for any concerns. For more information check our guide.
The Shen.AI system operates effectively with a smartphone that meets specific requirements. While it doesn’t necessarily demand the latest or best camera, there are certain specifications to consider:
Camera: The smartphone needs a camera that can be accessed through native platform APIs and delivers stable 30 FPS video at a minimum resolution of 640×480 pixels. While a higher quality camera can enhance accuracy, it’s especially beneficial for challenging lighting and stability situations.
Network: Internet connectivity is essential for license validation. Additionally, there’s an option for telemetry and crash reporting, which also requires internet access. Beyond these aspects, there are minimum network requirements based on the target platform.
For more detailed technical requirements visit developer portal.
It takes under 1 day to integrate our SDK into your project. We have a lot of technical information, guidelines and examples available on the developer portal.
First vital sign, which is Heart Rate, is available in only 5 seconds, the rest for physiological reasons to keep the accuracy are available at the end of measurements in 60 seconds.
Yes, it can be also 30, or 45-seconds, however the accuracy of such measurement will be lower.
Yes, front-end examples are available but they are not part of the core SDK. Front-end can be parametrized or built entirely by a customer.
The signal processing is based on the rPPG waveforms acquired optically from user face scan. Waveforms among other features are then used for AI based algorithms for estimation of systolic and diastolic Blood Pressure. We have 5 mln heartbeats (waveforms) in our database we are using for training of our AI models. We have collected this data on our own and they come from 42k recordings.
Health risk calculations are based on the Framingham Heart Study. For example, in the case of cardiovascular disease risk assessment, factors such as age, sex, blood pressure, cholesterol levels, and smoking history may be considered. The software may use statistical models and algorithms to analyze these factors and calculate a person’s overall risk of developing cardiovascular disease over a certain period, such as ten years.
It’s important to note that any health risk calculation is only an estimate and cannot predict with 100% accuracy whether or not a person will develop a specific condition. Additionally, the accuracy of risk calculations may depend on the quality and accuracy of the data inputs and the particular algorithms and statistical models used by the software.
It’s always a good idea to consult a healthcare professional for personalized health risk assessments and advice.
As all SDK computations happen locally on the user’s end-device, their results are immediately available on-device after the measurement has finished. You can then easily send them to your system using technologies that you use for all other data transport.
All metrics are computed offline, on-device, without any server connectivity. The measurement result may be indefinitely persisted locally as part of the SDK integration. Internet connection is only required for license verification – after the initial verification the SDK may be used for up to 3 days without internet connectiv