Creative affordances of orchestrated devices for immersive and interactive audio and audio-visual experiences

Immersive experiences are a significant growth area for the creative industries, presenting both challenge and opportunity in terms of delivering augmented, virtual, mixed or extended reality content. Many of the existing technologies for delivering immersive experiences require specific hardware or are limited to particular reproduction scenarios. For example: headset-based virtual reality is rarely suitable for multiple participants and there is currently limited market penetration of the requisite hardware; next generation audio can be used to drive high channel count spatial audio reproduction systems, but few listeners have access to systems with lots of loudspeakers; and binaural audio can give 3D listening experiences but requires the listener to wear headphones, making it challenging to deliver shared immersive experiences.

To mitigate against these barriers, this project will explore the creative possibilities of orchestrating the media devices present in the user/listener’s environment (such as smartphones, tablets, virtual assistants), using object-based media (where content and content metadata are considered separately and in parallel) for the delivery of immersive and interactive audio and audio-visual storytelling experiences. The use of orchestrated devices will be investigated by reviewing the existing literature and creative outputs, as well as working with production companies and audiences to determine beneficial use-cases. The project will focus on charting the historic and current landscape of orchestrated media, and development and testing of new orchestrated audio and audio-visual storytelling experiences from a production and/or an end-user viewpoint. On the production side, aspects such as workflow, tools, and how to express, capture, and represent creative intent will be investigated. The experience of the end user will be considered in order to clearly ascertain, demonstrate, and improve upon the tangible benefit to audiences of this type of content reproduction.

The project is part of the on-going relationship between the University of York Department of Electronic Engineering AudioLab (https://audiolab.york.ac.uk/) and BBC R&D (https://www.bbc.co.uk/rd), through the XR Stories AHRC Creative Cluster R&D Partnership (https://xrstories.co.uk/). The successful candidate will be supervised by Professor Damian Murphy and study in the Department of Electronic Engineering AudioLab at the University of York, spending short periods of time with the XR Stories team in York, and with BBC R&D at MediaCityUK. Up to 12 months of this 3.5 year PhD will be spent on placement with BBC R&D as a single block of time, or in shorter more focused periods, 3-6 months at a time.

The project will consider both research and development, as well as practice-based approaches, and user studies with colleagues from the BBC. The successful applicant should have a strong interest in sound, music and immersive audio technology and good programming skills. This project is highly multi-disciplinary in its nature and we welcome applicants from a broad range of core research backgrounds and interests, extending from audio signal processing to user experience design, human-computer interaction, as well as relevant creative sound design practice.

Candidates must have (or expect to obtain) a minimum of a UK upper second-class honours degree (2.1) or equivalent in Computer Science, Electronic Engineering, Music Technology or a related subject. Prior research or industry experience would also be an advantage.

This PhD is due to start 1st October 2020

How to apply:
Applicants must apply via the University’s online application system at https://www.york.ac.uk/study/postgraduate-research/apply/. Please read the application guidance first so that you understand the various steps in the application process. To apply, please select the PhD in Electronic Engineering or Music Technology for October 2020 entry. Please specify in your PhD application that you would like to be considered for this studentship.