Estimating driver state from facial behaviour and other signals

Automated driving refers to the capability of a vehicle to handle a set of driving tasks normally performed by a human. This set of tasks ranges from individual manoeuvers such as emergency braking or obstacle avoidance in safety and driver assistance applications to handling all driving tasks that can be managed by a human driver in fully automated driving. Increasing levels of automation require the automated driving system to be aware of the cognitive and physiological state of the driver, for instance when handing over control to the driver is required, or when the driver wishes to take over control from the system. Further situations include medical emergencies, during which a driver may become incapacitated and unable to control the vehicle, requiring immediate action by the system.
In this PhD project we propose to build a driver state estimation method capable of inferring cognitive and physical driver states based on both driving behaviour as well as on the driver’s facial appearance, cardiac, electro-dermal, respiratory and electro-muscular activity.
The challenges of this PhD project will relate to extracting and combining the information from the sensory data provided by the vehicle using methods from Computer Vision and Signal Processing (face tracking, deep neural networks) and Bayesian Machine Learning (probabilistic inference).
The first stage of the project should focus on the development of a model capable of predicting driver states such as “vigilant”, “excited”, “stressed” or “incapacitated” before moving on to more subtle emotional states.
A significant amount of existing driving simulator data will be available from the outset of the project and there is the potential for testing the developed methods during further driving simulator sessions.