AI-Driven App Detects Early Signs of Disease Using Only a Smartphone Camera

Startup Shen.AI is excited to announce its development of vital-signs monitoring using only real-time video from a smartphone camera, making advanced health management available anytime and anywhere to anyone. Using artificial intelligence, the application can recognize subtle signs of illness with mathematical precision, leveraging ubiquitous technology to democratize healthcare.

“Mobile phones with a camera are the most widely available yet least exploited health-analysis tools,” says Remi Koscielny, cofounder and CEO of Shen.AI. “We want to squeeze as much capability as possible out of the smartphones for diagnostic purposes without the need for any additional hardware.”

While over five billion people worldwide use mobile phones, the World Bank and the World Health Organization report that at least half of the global population lacks access to essential health services.

“Sixty-three percent of people in the world have mobile phones,” Przemek Jaworski, cofounder and CTO of Shen.AI, says, “and 2.87 billion of them have a smartphone equipped with a camera. Imagine turning every smartphone into a health-assessment device. That’s our mission.”

Health monitoring from Shen.AI starts with a short video of the subject’s face. The app stabilizes the real-time images and maps them to a customized three-dimensional face-geometry mesh. Computer-vision techniques then extract data patterns from facial skin, eyes, and other features such as tiny movements of the head and muscles. In order to analyze heart activity and extract vital signs, the engine uses remote photoplethysmography (rPPG), a contactless technique of recording blood pulsations in the skin vasculature.

Based on advanced data engineering and the latest research results, Shen.AI’s technology can measure various cardiovascular parameters (blood pressure, heart rate, and heart-rate variability), provide visual biofeedback, assess skin condition, and make a preliminary diagnosis with recommended next steps. The technology, named Shen.AI, requires only a smartphone, tablet, or computer with a camera. No other devices are needed. The Shen.AI platform will be available as a software development kit (SDK) for app developers and hardware manufacturers.

Shen.AI aims to develop a family of mobile apps starting with Heart Monitor, which is already available for free on Apple’s App Store and Google Play. Heart Monitor has been designed as a support solution for monitoring cardiovascular health and detecting hypertension, which could significantly improve access to early diagnosis of hypertension, a leading preventable risk factor for cardiovascular diseases.