With a unique focus on mental modalities, Andrey Filimonov, Director of Research at Harman International, gives us an insight into the current landscape and future challenges of developing Driver Monitoring Systems, ahead of his presentation at InCabin USA this May.
1. The current focus in Driver Monitoring Systems (DMS) primarily revolves around visual distraction. Can you share the motivation behind Harman’s emphasis on mental distraction as a crucial factor in driving safety? What sparked the interest in exploring the impact of mental states on driving performance?
Yes, DMS systems are mostly about monitoring if people physically look to road. With well-developed eyegaze tracking technologies visual distraction or “eyes on road” is a low hanging fruit. However, visual distraction is just of the factors leading to worse driving performance. So called “cognitive distraction” is equally dangerous. Cognitive distraction is when your eyes are on road, but your mental effort is somewhere else. If you drive and if you ever picked up a phone call while driving, you might find yourself in a situation when you reached your destination point, but you didn’t remember how you’d done. You attention was so much consumed by the phone call that you drove almost subconsciously. We proved that doing things like this may result in having 5x bigger chances to get into an incident or 6x bigger chance to miss a route. It’s the same as you’d miss it because you didn’t look to road. The inspiration part of it comes from a much bigger scope. Harman has a research department focusing on human mental state detection. “Mind on road” is just one application for our human mental state sensing technology. The department emerged because we were able to gather a very strong cross-discipline team of psychophysiologists, mathematicians, physicists, machine learning and data analysis engineers together and tackle the problem of sensing human state from multiple directions. None of these areas of expertise alone would allow to solve the problem.
2. In your presentation, you aim to introduce the audience to the value of detecting the driver’s mental state. How does understanding the mental state contribute to assessing a driver’s level of attention to the road, and what are the potential safety implications?
I briefly touched this in my previous comment. I can indefinitely talk about what human states are, how we detect, how we measure accuracy of detection, but my favourite (and at the same time the easiest to explain to a random person) way is through bringing driving performance into equation and showing how it degrades if people are not in optimal driving states. Two examples of such degradation under cognitive distraction are given above, but we have much more. E.g. people miss stimuli or false react three times more often. If I change the terms to more practical ones, they may not notice red traffic light while passing a crossroad, or vice versa, false start on red.
AUTOSENS USA AGENDA SESSION
Mental Modalities and Impact on Driving Performance
Andrey Filimonov
Senior Director of Research
Harman International
3. Could you provide a glimpse into how you plan to demonstrate the impact of mental distraction on driving behavior during your presentation? Are there specific driving performance behaviors and metrics that you will be highlighting to showcase this impact?
Yes, exactly. To add to the list above, we asses reaction time, or how often people unintentionally depart from their lane, or to a road shoulder or even to an oncoming lane. Being mentally distracted, they overspeed eight times more often than they usually do. Of course, we do in a simulated environment. I’m going to share some info on how we trigger this situations and what kind of data we gather to be able to design state detection algorithms and systems.
4. Given the unique focus on mental modalities, what challenges and opportunities do you foresee in the development and implementation of technologies that detect and respond to the driver’s mental state? How do these considerations differ from those related to visual distraction?
We have a couple of them:
Fuzzy definition of mental states. To illustrate the point, if I ask you to define what drowsiness is, one’s first reaction is “this is obvious”. We use the word so often in our everyday life, so we tend to believe we understand it to the level when it doesn’t even require definition. However, if I insist and ask you to define, you’ll most likely end up just replacing one word with another like “sleepiness” which is equally fuzzy and undefined. There is no strict and formal definition of states. The words we historically use for the sake of state definition are at least imprecise, but in many cases have nothing to do with physiological or psychological phenomena of our mind, brain and body.
A consequence of the previous point. If you can’t define something, there is no way to measure.
Even if you defined, what would be the unit of such measure. In what kilograms, volts or decibels you’d measure human state?
Even if the unit exists, with what sensor (like weigher, voltmeter or sound pressure meter) you’d measure it precisely and objectively?
Even if you defined what mental state is, the definition is so complex, that you barely can explain it to a foreigner.
Mental sensing technologies are emerging now, and the society is not quite prepared for that. For example, if you look to how regulatory bodies address the problem of safety, namely the DMS aspect of it, it’s mostly about visual distraction and much less about drowsiness. The reason is that people know how to detect visual distraction, it’s as simple as “looking to the right instead of looking straight – you’re visually distracted”, it’s easy, at least conceptually. If you look into how drowsiness is regulated, then there are much more vague words and much less strict requirements. Automotive industry doesn’t know how to detect drowsiness or even what drowsiness objectively is. Hence such vague requirements.
Building such systems and solutions is a huge technical and scientific challenge. Just to give you some sense. We needed to collect around 3PB of data from 5000 people (this is thousands of hours of recordings) to be able to design. 20% of the team have PhD degrees in their respective areas of expertise. Harman invested millions into data collection alone.
5. As InCabin USA 2024 approaches, what aspects of the event are you most looking forward to? Is there a particular session, conversation, or development in in-cabin technology that you are excited to engage with during the conference?
We’re mostly looking to educate people and let them know that Harman leads an effort to bring mental state sensing as a means for increasing safety. We’d like to share that we’ve done it to the fullest extent starting from appropriate scientific research, through industrialization and making these technologies work inside cars to showing what kind of negative implications suboptimal mental state may have.
