Multi-Sensor information fusion to assess the Quality-of-User Experience using Physiological Signals and Deep Learning

Wearable IoT devices can be seen in a variety of applications including healthcare, activity trackers, stress detectors, navigation systems and smart textiles. Traditional wearables such as smart watches, glasses and smart belts make the user uncomfortable and therefore provide inaccurate data. However, pupil response is another unintended physiological signal that may be used to classify emotions. The pupil dilates when people are more attentive, have a more cognitive load, or due to emotional or cognitive arousal. Formally speaking Pupillometry is the study of such changes in pupil diameter in relation to different neurological activities in the human brain. In medical terms, a pupil diameter can vary from 1.5mm to 9mm and reacts to stimulation in about 200ms. Under standard lighting conditions, the normal pupil size is around 3mm.

Several studies have been released using pupillometry to translate different brain activities into different scenarios. However, due to the subjective nature of the signals, it is impossible to reproduce the reported results and therefore a major impediment to any acceptance. To overcome this problem, identification of the correlation between pupil responses and some other physiological signal can be very helpful, such as EEG, EGC, Heart rate or Galvanic Skin Response (GSR). The EEG and ECG study is quite mature and hence human responses for an audio/video stimulus can be recorded using multiple sensors simultaneously. This will not only aid in the identification of human emotions but will also help to explore quality of experience (QoE) for practical applications. The project will require ethical approval from the university.  

Please quote FNS_NKSept 2022 when applying for this studentship.