Multi-modality Based Smart System for Managing Resilience and Mental Wellbeing

Reference Number

Please refer to SCEBE-20-026-YZSF

Project Details

Resilience is our ability to recover after exposure to significant adversity, e.g., the covid-19 pandemic. There are many factors that can influence our resilience including our life experiences and support networks. Resilience can be improved using resources such as self-care techniques, mindfulness. Resilience had been shown to have a significant negative correlation with both anxiety and depression. Promoting resilience can directly impact on productivity, performance, and can improve quality of life. Measurements of resilience can be psychologist-led or self-assessment using questionnaires, although with questionnaires there can be a significant bias.

Recently, voice screening and monitoring for stress and depression have been reported. Machine-learning technology has been utilised to analyse speech samples obtained in the clinic or remotely (e.g., using a mobile App) for automated assessment of psychiatric. The technology of emotion recognition has been used to evaluate a stress resilience program in combination such as breathing and yoga. Online quantitative eye tracking using webcam has been used in emotion recognition. Also, in the field of Human Computer Interaction (HCI), a new area called affective computing has emerged with the goal of developing techniques for modelling emotions.

This project aims to study the unique role of multi-modality based smart system in managing resilience and mental wellbeing. It’ll scope and define the problem through a Multi-Disciplinary Team, including psychologists and health specialists.  To enable reliable and real-time management of resilience and mental wellbeing, this project will make use of data collected and extracted from multi-modalities, such as facial expression detection, speech recognition, body movement, gesture and eye tracking. Deep learning will be applied for resilience management. Furthermore, evaluation will be conducted based on feedbacks from the Multi-Disciplinary Team, as well as end users. The smart system will be deployed as a mobile app or a plugin for online platform to provide real-time resilience assessment and support for individual to manage their resilience and mental wellbeing.