Robust audiovisual speech recognition in man and machine

This industrial collaboration project with Google Research London bridges between artificial intelligence, computational linguistics, computational and cognitive neuroscience.

Robust speech recognition despite speaker variability, distortions and adverse noisy environments is a fundamental challenge for any natural or artificial speech recognition system. Critically, speech recognition performance of both human and artificial systems benefit from integrating information from audition and vision, such as speakers’ articulatory facial movements. Artificial intelligence and computational linguistics have developed a range of machine learning algorithms (e.g. recurrent neural networks, HMM, DNN) that are trained on large audiovisual corpora. Yet, while these algorithms are relatively efficient, they do not yet obtain the robustness of human observers/listeners.

This interdisciplinary project will combine expertise from human neuroscience (University of Birmingham), artificial intelligence and computational linguistics (Google Research London) to determine the neural mechanisms underlying audiovisual speech recognition in the human brain and advance speech recognition algorithms in machines. To study audiovisual speech recognition in artificial and human systems, we will train artificial systems on audiovisual corpora and measure eye movements, behavioural (psychophysics) and electrophysiological responses (EEG/fMRI) in human participants listening and/or seeing audiovisual speech. Comparing human and machine learning performance on speech recognition tasks will provide insights into similarities and differences in the representations, computations and learning. The results will advance our understanding of the neural mechanisms underlying audiovisual speech recognition and inspire innovations in algorithms and training schemes for artificial speech recognition systems.

The Computational Cognitive Neuroimaging Group (Uta Noppeney) is seeking an enthusiastic PhD candidate with strong analytical and quantitative abilities. Applicants should have a background in computer science, computational linguistics, neuroscience, psychology, physics or related areas. Prior experience in statistical analysis and/or machine learning would be an advantage.

The Computational Cognitive Neuroimaging Lab is based at the Department of Psychology and the Computational Neuroscience and Cognitive Robotics Centre of the University of Birmingham, UK. The centre provides an excellent multidisciplinary, interactive and collaborative research environment combining expertise in cognitive neuroimaging, psychophysics and computational neuroscience. The psychology department was rated 5th in the UK research assessment exercise.

http://www.birmingham.ac.uk/schools/psychology/research/labs/comp-cog-neuro/index.aspx
http://www.birmingham.ac.uk/research/activity/cncr/index.aspx
Informal enquires about the project can be directed to Professor Noppeney ([Email Address Removed])

How to apply:
• Apply directly via the University of Birmingham Postgraduate application system, the Postgraduate portal. (Apply button)
• Please detail the advertised supervisor and project title under the ’Research Information’ section of the application form.