We caught up with Patrick Laufer ahead of InCabin Brussels, and he gave us some insights into Vital Sign Estimations and what we can expect to hear from him this June.
1. Why could Vital Sign estimation of Vehicle occupants be the next big thing for OEMs?
In-cabin Vital Sign estimation technology offers numerous benefits for OEMs, including enhancing driver and passenger safety, creating a ‘third space’ with well-being applications, and providing a tool for preventive and tele medicine.
2. Safety first: How does Vital Sign estimation prevent crashes?
Firstly, Vital Sign estimation plays a crucial role in in-cabin impairment recognition, particularly in classifying whether a person is under the influence of drugs or alcohol. If the intelligent vehicle can prevent an impaired driver from starting the vehicle, the risk of fatal crashes can be significantly reduced.
Secondly, vital sign estimation can assist in evaluating a person’s driving fitness, particularly during the phase where individuals may struggle to stay awake and experience drowsiness. Changes in heart rate and respiration signals occur during this phase, making it necessary to capture both vital signs to detect driver drowsiness at an early stage. Advanced assistance systems such as lane-keeping assistants can be increasingly engages during this phase to reduce the risk of micro sleep-based crashes. If this phase persists for an extended period, a controlled stop of the intelligent vehicle can also be considered to ensure the safety of the driver and other road users.
3. Safety is a must, but well-being is a need. How can Vital Sign estimation contribute to the occupant's well-being?
For the sake of comfort, many OEMs see emotion recognition as a crucial factor for well-being functions. Currently, many algorithms in this regard exclusively rely on facial images, which can be negatively influenced by factors such as beards or individuals with a “poker face.” While some people may be able to conceal their emotions through facial expressions, vital signs can reveal their true emotional state. For example, someone who appears focused and someone who appears angry may have similar facial expressions, but differences in their heart rates can help distinguish their emotional states.
Moreover, since some people spend long periods of time inside a car without much movement, long-term vital sign monitoring becomes possible. This opens up the possibility for preventive medicine, as the intelligent vehicle can monitor changes in both respiration and heart rate patterns, and predict the current fitness state, as well as the risk of developing cardiovascular conditions such as heart arrhythmias, heart attacks, and asthma.
4. You mentioned the use-case of long-term health predictions. Can vehicle interiors replace doctors in the future?
When we consider the vehicle cabin as a ‘third space’, it opens up the possibility of telemedicine. The vehicle has the potential to function as a replacement for the traditional doctor’s waiting room, allowing the patient to wait in their car while their current and long-term vital signs are remotely transmitted to the doctor via camera. The doctor can then determine whether an in-person appointment is necessary or prescribe rest and mild medication directly. Although I do not anticipate a complete substitution of doctors in the near future, I do believe that precise vital sign estimation can minimize the necessity for many in-person visits.
5. Why use the camera for Vital Sign estimation?
The camera is already installed inside the vehicle interior for driver monitoring purposes. Therefore, the vital sign estimation function can be added without any additional material costs. Apart from that, the estimation by the camera is contact-free, so there is no wear and tear on the sensor, and the measurement is not influenced by dirt, sweaty skin, or improper placement of the electrodes on the skin, as is the case with ECG electrodes.
6. How does camera-based Vital Signs estimation work?
There are basically two different approaches that we pursue. Firstly, the respiration rate can be extracted from the displacement of the chest and upper stomach regions that occur during the inhaling and exhaling phases. With a so-called “optical flow” method, this movement can be mapped to a respiration signal that reveals the respiration rate of a person.
Secondly, heart rate and respiration rate can be determined from the Photoplethysmography (PPG) signal, which measures changes in blood volume in tissues by detecting subtle changes in color and brightness during each pulse. Arterial blood absorbs light more efficiently than surrounding tissue, and when the arteries expand, the absorption rate of light changes. This change is captured by a camera that focuses on the face region and maps it to a PPG signal.
7. Is Vital Sign estimation by camera accurate enough for the in-cabin scenario?
While a conventional RGB mono camera is prone to changing light conditions that undoubtedly occur during driving tasks, the NIR camera is hardly influenced by ambient light. Furthermore, the NIR camera can capture vital signs even during night drives. By fusing both modalities of NIR and RGB, a more accurate and robust measurement can be obtained.
8. What are the limitations for camera based Vital Sign estimation?
The limitations of the PPG measurement method include makeup, facial occlusion that covers both cheeks and forehead, and often appearing head movements, while loose clothes can decrease the performance of the optical flow.
9. What are the future steps regarding in-cabin Vital Sign estimation?
To overcome the mentioned limitations and improve the accuracy of vital sign estimation further, IAV is currently researching a camera-radar fusion method. It is hoped that the results of this research will be presented at one of the InCabin events next year.
Take a look at the full agenda for InCabin Brussels here: https://auto-sens.com/incabin/brussels/agenda/
