Creating and refining immersive, adaptive audio tools to enable real-time interaction with music and sound in Virtual Production

The research will focus on gestural interaction systems and procedural audio generation, advancing the ability for creators to manipulate sound and music dynamically within virtual worlds. The gestural component will investigate the use of motion tracking, hand gestures, and embodied interactions as intuitive tools for shaping soundscapes and musical structures. These systems will allow users to "perform" sound and music interactively within virtual spaces, leveraging the precision and responsiveness of cutting-edge XR platforms. By incorporating adaptive machine learning algorithms, the tools will respond in real-time to user movements, dynamically modifying audio textures, spatial placement, and musical patterns based on user intent and environmental context. On the audio side, this research will build advanced sound generation pipelines that generate dynamic soundscapes and music tailored to the needs of virtual production. These systems will include the generation of context-aware ambient sound, real-time music composition, and seamless integration with visual effects pipelines (UE & opensource), which can be demonstrated for example within the Futures Studio. Such tools will offer filmmakers, game designers, and XR developers highly customisable and responsive audio solutions that adapt to the narrative and aesthetic demands of their projects.

Dr. Christos Michalakos (Lead Supervisor), Dr. Jung In Jung (Second Supervisor), plus Expert Mentors (TBC) from Royal Holloway, and/or University of Surrey.

This PhD is aligned with the CoSTAR research themes: Createch Futures & Standards

For more information on CoSTAR and the PhD opportunities please see: https://www.costarnetwork.co.uk/latest/come-research-with-us-at-costar-national-lab