In recent years, there has been substantial interest in the use of nonconvex and overparameterized schemes for dealing with problems arising in machine learning and data processing. Despite a comprehensive literature and efficient tools for convex optimization [1, 2], solving reliably nonconvex problems and analysing their performance remains a major challenge to the community. This difficulty stems from the complicated landscapes of the nonconvex functions that can admit many undesirable local optima, e.g. in training deep neural networks or computational imaging problems, and potentially deteriorate the prediction performance in these applications. Many of the current methodologies are somewhat ad-hoc and come without rigorous performance guarantees. In this project you will bridge this gap by developing new algorithms and analysis techniques towards reliable and transparent solutions to the aforementioned problems. Algorithmic designs will simultaneously take into account the computational scalability to the high-dimensional problems and the ability to escape from local sub-optimal solutions.
This project will address three key themes in the UKRI Centre for Doctoral Training in Accountable, Responsible and Transparent AI (ART-AI), namely, transparency and intelligibility of AI, risk and decision making with AI, and engineering applications of AI. Possible avenues to explore are:
• Understanding Neural Networks’ generalization power: Training a neural network requires minimizing a highly non-convex loss function that measures the predictive performance of the network on the training data. Since modern applications often use overparameterized and hence highly expressive networks, we might expect networks to learn overfitted models, i.e. to find model parameters that perfectly fit to the (noisy) training data, but while being applied to test data, the network makes inaccurate predictions. Surprisingly, this issue does not happen in practice! Stochastic optimization methods “somehow” (implicitly) regularize the problem, and manage to avoid undesirable minima and find good solutions; a phenomenon that has led to the impressive generalization ability of the neural networks and their widespread success in various inference problems, while not yet being well understood. This project aims to develop new theories to uncover this mystery using recent theoretical results e.g. [3, 4] for training shallow networks and to extend them to more popular deep network models.
• Application in Medical Image Computing: In Quantitative MRI (QMRI) we are interested in computing multitudes of the fundamental properties of tissues (e.g. relaxation times, proton densities, perfusion, diffusion) from a time-series of magnetisation images. These properties are proven powerful in tracking tissues’ pathological changes for diagnosing e.g. cancer and neurodegenerative disorders. Computing quantitative parameters requires fitting measurements to a nonlinear physical model i.e. the Bloch equations. Further, to make the scans fast and clinically applicable, the time-series images are severely under-sampled. As a result, QMRI requires solving a multi-dimensional, highly ill-posed and nonconvex optimization problem [5, 6]. Current QMRI algorithms either suffer from uncontrolled convergence to “bad” local minima (hence unreliable solutions), or the curse of dimensionality resulted by finely gridded multi-parametric solution space. This project aims to develop new algorithmic and analysis tools to tackle these challenges and achieve (guaranteed) reliable and scalable MRI quantification.
The UKRI CDT in ART-AI is looking for its second cohort of at least 10 students to start in September 2020. Further details of the Centre can be found at: http://www.bath.ac.uk/centres-for-doctoral-training/ukri-centre-for-doctoral-training-in-accountable-responsible-and-transparent-ai/
Desirable qualities in candidates include intellectual curiosity, a strong background in maths and programming experience.
Applicants should hold, or expect to receive, a First or Upper Second Class Honours degree. A master’s level qualification would also be advantageous.
This project will be supervised by Dr Mohammad Golbabaee (Department of Computer Science) and Dr Clarice Poon (Department of Mathematics). Informal enquiries about the project should be directed to Dr Golbabaee: [email protected]
Enquiries about the application process should be sent to [email protected]
Formal applications should be made via the University of Bath’s online application form: https://samis.bath.ac.uk/urd/sits.urd/run/siw_ipp_lgn.login?process=siw_ipp_app&code1=RDUCM-FP02&code2=0002
Start date: 28 September 2020
 Boyd, Stephen, and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.
 Rockafellar, R. Tyrrell. Convex analysis. Vol. 28. Princeton university press, 1970.
 Bach, Francis. "Breaking the curse of dimensionality with convex neural networks." The Journal of Machine Learning Research 18.1 (2017): 629-681.
 Chizat, Lenaic, and Francis Bach. "On the global convergence of gradient descent for over-parameterized models using optimal transport." Advances in neural information processing systems. 2018.
 Ma, Dan, et al. "Magnetic resonance fingerprinting." Nature 495.7440 (2013): 187.
 Dong, Guozhi,et al. "Quantitative Magnetic Resonance Imaging: From Fingerprinting to Integrated Physics-Based Models." SIAM Journal on Imaging Sciences 12.2 (2019): 927-971.