Virtual Synaesthesia – Using Immersive and Virtual Environments with haptics and olfaction to emulate synaesthetic conditions

Visual and auditory signals are relatively easy to capture, generate, transmit and represent electronically, and hence provide the backbone of current media technologies.

Nevertheless, across a broad range of applications, the representation of other sensory information such as touch and force (haptics) or smell (olfaction) will permit new areas of investigation and the development of new products and systems.

The research proposed is ambitious in that it combines state-of-the-art IT research with new advances in psychological perception to create a completely new environment where we can investigate a range of human conditions pertaining to the neurological phenomena know as synaesthesia – the confusion or coupling of two or more senses.

The intention will be to augment conventional Immersive / Virtual Environments (CAVE) with devices and systems that can emulate additional sensory channels (haptics and olfaction). The creation of a multi-sensory immersive environment opens up the ability to emulate and investigate a range of synaesthetic conditions, from the more common such as grapheme-color synaesthesia, whereby letters or numbers are perceived as inherently coloured, to more advanced (and rarer) conditions such as auditory-tactile synaesthesia (whereby certain sounds can induce sensations in parts of the body) to olfactory synaesthesia (whereby “phantom smells” are generated when other senses are stimulated).

As part of the 4-year EPSRC research project “Context-Aware Networks for Sending Multiple Senses” CASMS we will be establishing a new VR/IR CAVE in the Advanced Networks Research Group (ANRG) at Liverpool. This will be connected via a dedicated high-speed SuperJANET connection to the Virtual Engineering Centre (VEC) CAVES at Daresbury and London, and on to a complimentary CAVE at University College London (UCL). Other partners include the BBC and Virtalis Ltd, with Jaguar LandRover and Alder Hey hospital also interested.

A PhD studentship opportunity is available in the Department of Electrical Engineering and Electronics at Liverpool. The funding will support full fees and stipend for 3 years (£60K).

The PhD student will be co-supervised by colleagues in the ANRG (Alan Marshall) and in Psychology (Georg Meyer/Rebecca Lawson/Sophie Wuerger), and will work alongside other researchers and engineers in the CASMS project. The research will involve development of theoretical concepts as well as new VR/IR systems, equipment and experimentation involving multi-sensory environments and human factors.