Expecting Thunderclaps then Lightning Strikes: How do we integrate sound and light in a complex world?

The experience of hearing the sound of thunder after seeing lightning is familiar to everyone. This perception of a single multisensory event as two separate events is caused by the difference in relative speed and arrival times of sound versus light. Yet, when we perceive stimuli in our immediate vicinity, the effect sometimes reverses. Why? Part of the explanation is that the ear turns sound into nerve impulses faster than the eye turns light into nerve impulses. However, most of the time our brain somehow compensates, and we perceive light and sound synchronously.  

Correctly integrating visual and auditory sensory information is essential if the world is to make sense. For example, seeing lips move provides important timing information which helps us to understand speech. But although the timing delays of our eyes and ears are fixed, the delays in the world vary not only with distance but with the type of movement (slow, fast, big, small) and the type of sound (sudden like a bang or slowly growing like a car approaching). Does our brain correct for this variability? If so, how? Perhaps on the basis of recent experience? The real-world implications of this work are wide-ranging. For example, hearing loss and hearing aids change and delay sound. How do people cope with these changes? 

This project will address these questions using a multidisciplinary approach; combining behavioural methods (psychophysics) to measure perception, brain imaging (electroencephalography; EEG) to uncover the underlying neural processing and Bayesian computational models to interpret these data. You will devise experiments which test how we integrate sight and sound as they vary in their properties and timing.  

We are looking for a motivated, numerate student to do high-impact integrated neuroscience, using computational models and neuroimaging. Based in NTU Psychology, this interdisciplinary project would suit graduates of a wide range of quantitative disciplines (psychology, maths, physics, engineering) who are interested in studying the ultimate computational device: the brain.