Multi-modal sensing

Shen AI uses video input from a camera to detect the face and extract physiological vital signs. This is done by analyzing subtle pulsation signals from the skin, captured through remote photoplethysmography (rPPG). However, factors like lighting, skin tone, and movement can affect signal quality—making consistent accuracy a challenge.

‍

To address this, Shen AI integrates multimodal sensing, combining rPPG with remote ballistocardiography (rBCG). This approach enhances accuracy by fusing signals: when rPPG is limited, rBCG helps maintain reliable measurements. Multimodal sensing also ensures that Shen AI works across a wide range of lighting conditions and skin tones—making device-free health monitoring more accurate, inclusive, and accessible.