Going from Driver-Monitoring to Driver-and-Occupants-Monitoring Systems; How to Capture the Entire Cabin for the Next Generation of In-Cabin Monitoring Systems

In this talk, we will present how Immervision can design cameras for mirror replacement by performing complex and complete vision system simulation. To optimize the final shape and maximize the field of view of such a camera, the most crucial aspect is to be able to characterize and predict the impact of each component of the vision system on the overall performance of learning-based applications, as well as human vision. As they can differ, it is also mandatory to establish a balance between human and machine perception to maximize the vehicle’s safety and the system’s usability. By simulating the image obtained through the entire imaging pipeline (protective layer, optics, sensor and image signal processor) we are able to study the impact of each isolated component before investing in the prototyping phase. Moreover, each optical system optimization can be done in context by adding an extra layer of modelling, such as any surface of plastic or glass that would interfere between the scene and the resulting captured image. Thanks to this capability, it is also possible to broaden the possibilities for the camera positioning in the car as it could be outside, inside or made invisible by adding an additional protective layer. Keeping the example of virtual e-mirrors but associated with object detection, we will highlight how we can simulate and validate the system’s component parameters to ensure accurate human vision and machine vision perception performance.