Don't miss our weekly PhD newsletter | Sign up now Don't miss our weekly PhD newsletter | Sign up now

  Enhanced Deep Learning via Optimisation in Computational Imaging

   School of Engineering & Physical Sciences

This project is no longer listed on and may not be available.

Click here to search for PhD studentship opportunities
  Dr Audrey Repetti, Prof Jean-Christophe Pesquet  Applications accepted all year round  Funded PhD Project (European/UK Students Only)

About the Project

Context. The project will be co-supervised at Heriot-Watt University (Edinburgh, UK), and CentraleSupélec (University Paris-Saclay, France). Although the lead supervision will be in Edinburgh, we expect the PhD student to travel to France during the project.

Background. Data science transforms data into interpretable information to enable accurate decision-making. Such methods rely on advanced mathematical tools. Optimisation is one of them, and it is broadly used to design robust, fast, and scalable algorithms to minimise given objective functions. Since early 2000's, proximal methods have become state-of-the-art to solve minimisation problems, in particular in the context of inverse problems [1]. Proximal methods also usually benefit from theoretical results (i.e., convergence to a minimum of an objective function, convergence rate). In many applications, the reliability of the output is an important question to enable accurate decision-making processes. For instance, to accurately identify tumors in a magnetic resonance (MR) scan.

During the last decade, proximal algorithms involving neural networks (NNs) have emerged. Two main classes of such hybrid methods can be distinguished. The first approach consists in unrolling an optimisation algorithm over a fixed number of iterations, to build the layers of a NN, leading to unfolded NNs. Unfolded NNs are particular instances of end-to-end NNs, that are directly used to solve inverse problems, processing corrupted data to produce a corrected output. End-to-end NNs have shown to produce outstanding results for multiple applications [2,3]. The second approach relies on replacing the denoising steps of an optimisation algorithm by NNs, leading to PnP algorithms.These methods also have shown excellent performance in many applications [7, 8], and multiple works have recently investigated convergence of PnP algorithms [4-6].

Project abstract. Research related to optimisation-based NNs for inverse imaging problems is relatively recent, and devoted methods are evolving fast. Some of the challenges of interest in this eld are related to (i) theoretical understanding, (ii) design of new methods (including optimisation algorithms, sampling methods, NN architectures, etc.), (iii) applications (e.g., medical, astronomical, photon imaging). Finally, NNs can also be investigated in the context of xed point methods, which provide an even wider framework than optimisation approaches. Possible research directions for this project include (but are not restricted to):

  • Theoretical guarantees of hybrid optimisation-NN methods. Although convergence of PnP algorithms has started to be better understood during the last few years, many questions remain unanswered (or partially answered). For instance, to ensure convergence of PnP algorithms, NNs must satisfy some technical conditions. How to build such NNs? Similarly, when unrolling a fixed number of iteration of an optimisation algorithm, all the theoretical guarantees are lost. What type of guarantees do unfolded NNs offer?
  • Building flexible NNs for inverse problems. In PnP methods, the involved NNs depend on the underlying statistical models (e.g., higher noise level on the measurements requires stronger denoisers). Hence different NNs must be trained depending on the inverse problem' statistical model, which is computationally prohibitive. How to build more flexible NNs that can be adapted to multiple statistical models?
  • Improve NN efficiency using optimisation. Optimisation methods benefit from numerous acceleration strategies (e.g., inertia, preconditioning, backtracking, etc.). Can (unfolded) NNs benefit from such acceleration techniques to obtain better estimates with less layers?
  • Application to computational imaging. Developed methods will be used in computational imaging, e.g., to improve reconstruction quality of emerging modalities in medical imaging. Possible applications include microendoscopy using a multicore fibre, aiming to view biological processes in-vivo [9], and diffuse optical tomography using a single-photon camera that aims to image objects through a strongly di usive medium [10].
Computer Science (8) Engineering (12) Mathematics (25)


[1] Combettes & Pesquet, in Fixed-Point Algorithms for Inverse Problems in Science and Engineering (2011) 185
[2] Jin, McCann, Froustey & Unser, IEEE TIP 26 (2017) 4509
[3] Liang, Cao, Sun, Zhang, Van Gool & Timofte, in Proc. ICCV (2021)
[4] Pesquet, Repetti, Terris & Wiaux, SIAM SIIMS 14 (2021) 1206
[5] Repetti, Terris, Wiaux & Pesquet, EUSIPCO proceedings (2020)
[6] Laumont, De Bortoli, Almansa, et al., arXiv:2103.04715 (2021)
[7] Zhang, Li, Zuo, et al., IEEE TPAMI (2021)
[8] Ahmad, Bouman, Buzzard, et al., IEEE SPM 37 (2020) 105
[9] D. Choudhury, D. K. McNicholl, A. Repetti, I. Gris-Sánchez, S. Li, D. B. Phillips, G. Whyte, T. A. Birks, Y. Wiaux, and R. R. Thomson, Nature Comm. 11 (2020), 1.
[10] A. Lyons, F. Tonolini, A. Boccolini, A. Repetti, R. Henderson, Y. Wiaux, and D. Faccio, Nature Photon. 13 (2019) 575.
Search Suggestions
Search suggestions

Based on your current searches we recommend the following search filters.

 About the Project