PhD Studentship Opportunity in Computational lighting in video

This project will explore how advances in Generative networks and deep-learning approaches to beautification and stylistic transfer, can be combined to artificially recreate the appearance of professional lighting. The system will use recordings from one or more viewpoints of a scene with an unknown lighting setup, and will generate a video of the same scene but which a viewer would judge to have been recorded under “three point lighting”. The developed technology is of interest as a potential extension to the capabilities of the Ed system currently being developed by BBC R&D.

The Ed system is an automated system for creating edited coverage of live events. Its inputs are high-resolution locked-off cameras that are deployed at venues with live audiences. The system then creates “virtual camera” views by cropping these raw camera feeds and cuts between them to produce output. In real-world deployments, the lighting configuration is often constrained to a suboptimal setup. The ability to synthesise more appealing data than what can practically be achieved, would improve the output of the Ed system.

Program of research:
The first 15 months will be based at the University of Surrey, and will focus on integrating deep style transfer with computational lighting and monocular scene understanding. The initial research will focus primarily on static environments captured at Surrey’s Audio-Visual lab. This period will also include extensive personal development and training opportunities covering technical and professional skills.

The following 18 months will be undertaken at the BBC North site in Manchester, and will explore the temporal aspects of the problem including temporal lighting consistency and correct shadowing for dynamic objects. This will use dynamic footage captured at the BBC site and consequently the majority of the data collection will also happen during this period, with radio theatre as a likely use-case. Difficulties in capturing multiple lighting conditions for a dynamic scene and may necessitate an exploration of unsupervised/semi-supervised learning or domain-transfer from simulation or between frequency spectrums.

The remainder of the PhD is expected to be based primarily at BBC North, but some flexibility may be possible, subject to the requirements of the project and the student. The research during this period will focus on preliminary feasibility studies for potential follow-on research, such as conditional generation and structure/lighting disentanglement to allow the simulated lighting setup to be varied dynamically by Ed or a human user.

Key Highlights of this Studentship:
This studentship should appeal particularly to scientists and engineers who are interested in working with the creative industries, or have a particular interest in cutting edge and next generation vision technologies. The project will be very interdisciplinary, with exposure to both academic and industrial research centres. Furthermore, this studentship could potentially benefit from the University of Surrey’s Centre for Doctoral Training in Audio-Visual Machine Perception. This will constitute a rigorous fully-funded programme of personal development, including professional research and software skills, interdisciplinary hackathon events, and a co-located peer-support network of approximately 100 other PhD students in related areas. Student’s receiving their doctorate through this scheme are expected to be exceptionally well-qualified for a subsequent career in research.

Entry requirements:
The ideal candidate for this studentship should have a strong academic background in software development from studying Computer Science, Electronic Engineering or a related subject. Furthermore, the candidate should have a keen interest in AI/machine learning. Prior experience (including formal study, project work or MOOCs) is advantageous but not required. Candidates would be expected to have (by October 2019) either a 2:1 or First class honours undergraduate degree or a Masters in one of the areas highlighted above.
Non-native speakers of English will normally be required to have IELTS 6.5 or above (or equivalent) with no sub-test of less than 6.

How to apply:
Applications must specify the point of contact as Dr. Hadfield, and should be made through the online portal at - https://www.surrey.ac.uk/postgraduate/vision-speech-and-signal-processing-phd. You must also attach a CV, certified copies of degree certificates and transcripts, a personal statement describing relevant experience (maximum 2 pages), 2 references, and proof of eligibility (eg passport or residence permit). Shortlisted applicants will be contacted directly to arrange a suitable time for an interview.

The project will commence on the 1st October 2019 and will finish on the 1st September 2023.