Deep neural network for photographing in the dark (GONGH2U19SF)
Dr H Gong
Prof G D Finlayson
No more applications being accepted
Self-Funded PhD Students Only
Have you ever struggled in taking a good photo in low light? Despite of the dramatic improvements in camera sensors, imaging in darkness is still challenging because in many conditions, there is simply too little light on the scene and the imaging sensors are sensitive to noise in these conditions. This may be compensated by increasing exposure time. However, a problem is that this also introduces motion blur and we need to reduce video capturing frame rate to minimise the number of blurs. Meanwhile, some image de-noising, de-blurring and enhancement methods (e.g. [ii-iii]) have been proposed, but their applicability is limited in dim light conditions and not suitable for the real-time video capture in the extremely low light condition.
In this PhD project, we plan to develop a pipeline for enhancing nearly black images/videos taken in low light. The solution will be based on end-to-end training of a fully-convolutional network [iv]. The network directly operates on raw sensor data and is expected to perform re-lighting, tone mapping, and noise removal. We will start by developing a supervised learning approach which does require some ground truth pairs of dark images and well-lit images for training. In the next stage, we will further investigate an unsupervised learning framework which doses require any corresponding ground truth image pairs. Instead, the training will be based on analysing different un-annotated sets of dark and well-lit images available from the Internet. This is a higher level of intelligence that self-learns without a “supervisor”. The possible unsupervised learning solution could be based on an image-to-image Generative Adversarial Network (GAN) [v]. We will also compare our solution with start-of-the-art methods (e.g. [i]) using both quantitative evaluation and user experiments.
The outcome of this project is expected to provide the powerful applications in smartphone camera development and night-time surveillance.
A start date prior to October 2019 is possible but should be discussed with Dr Gong in the first instance.
For more information on the supervisor for this project, please go here: https://people.uea.ac.uk/en/persons/h-gong
The type of programme: PhD
The start date of the project: October 2019
The mode of study: full-time
Entry requirement: Acceptable first degree in Computer Science, Mathematics, Physics (or related degree), the standard minimum entry requirement is 2:1
This PhD project is offered on a self-funding basis. It is open to applicants with funding or those applying to funding sources. Details of tuition fees can be found at http://www.uea.ac.uk/study/postgraduate/research-degrees/fees-and-funding.
A bench fee is also payable on top of the tuition fee to cover specialist equipment or laboratory costs required for the research. The amount charged annually will vary considerably depending on the nature of the project and applicants should contact the primary supervisor for further information about the fee associated with the project.
i) Chen, Chen, et al. "Learning to See in the Dark." CVPR 2018.
ii) Guo, Xiaojie, Yu Li, and Haibin Ling. "LIME: Low-light image enhancement via illumination map estimation." IEEE Transactions on Image Processing 26.2 (2017): 982-993.
iii) Hu, Zhe, et al. "Deblurring Low-light Images with Light Streaks." IEEE Transactions on Pattern Analysis and Machine Intelligence (2017).
iv) Chen, Qifeng, Jia Xu, and Vladlen Koltun. "Fast image processing with fully-convolutional networks." IEEE International Conference on Computer Vision. Vol. 9. 2017.
v) Liu, Ming-Yu, Thomas Breuel, and Jan Kautz. "Unsupervised image-to-image translation networks." Advances in Neural Information Processing Systems. 2017.