University College London Featured PhD Programmes
University of Oxford Featured PhD Programmes
University of Kent Featured PhD Programmes
University of Southampton Featured PhD Programmes
University College London Featured PhD Programmes

Optimization for large scale machine learning


Project Description

In recent years, there has been substantial interest in the use of nonconvex and overparameterized schemes for dealing with problems arising in machine learning and data processing. Despite a comprehensive literature and efficient tools for convex optimization [1, 2], solving reliably nonconvex problems and analysing their performance remains a major challenge to the community. This difficulty stems from the complicated landscapes of the nonconvex functions that can admit many undesirable local optima, e.g. in training deep neural networks or computational imaging problems, and potentially deteriorate the prediction performance in these applications. Many of the current methodologies are somewhat ad-hoc and come without rigorous performance guarantees. In this project you will bridge this gap by developing new algorithms and analysis techniques towards reliable and transparent solutions to the aforementioned problems. Algorithmic designs will simultaneously take into account the computational scalability to the high-dimensional problems and the ability to escape from local sub-optimal solutions.

This project will address three key themes in the UKRI Centre for Doctoral Training in Accountable, Responsible and Transparent AI (ART-AI), namely, transparency and intelligibility of AI, risk and decision making with AI, and engineering applications of AI. Possible avenues to explore are:
• Understanding Neural Networks’ generalization power: Training a neural network requires minimizing a highly non-convex loss function that measures the predictive performance of the network on the training data. Since modern applications often use overparameterized and hence highly expressive networks, we might expect networks to learn overfitted models, i.e. to find model parameters that perfectly fit to the (noisy) training data, but while being applied to test data, the network makes inaccurate predictions. Surprisingly, this issue does not happen in practice! Stochastic optimization methods “somehow” (implicitly) regularize the problem, and manage to avoid undesirable minima and find good solutions; a phenomenon that has led to the impressive generalization ability of the neural networks and their widespread success in various inference problems, while not yet being well understood. This project aims to develop new theories to uncover this mystery using recent theoretical results e.g. [3, 4] for training shallow networks and to extend them to more popular deep network models.
• Application in Medical Image Computing: In Quantitative MRI (QMRI) we are interested in computing multitudes of the fundamental properties of tissues (e.g. relaxation times, proton densities, perfusion, diffusion) from a time-series of magnetisation images. These properties are proven powerful in tracking tissues’ pathological changes for diagnosing e.g. cancer and neurodegenerative disorders. Computing quantitative parameters requires fitting measurements to a nonlinear physical model i.e. the Bloch equations. Further, to make the scans fast and clinically applicable, the time-series images are severely under-sampled. As a result, QMRI requires solving a multi-dimensional, highly ill-posed and nonconvex optimization problem [5, 6]. Current QMRI algorithms either suffer from uncontrolled convergence to “bad” local minima (hence unreliable solutions), or the curse of dimensionality resulted by finely gridded multi-parametric solution space. This project aims to develop new algorithmic and analysis tools to tackle these challenges and achieve (guaranteed) reliable and scalable MRI quantification.

The UKRI CDT in ART-AI is looking for its second cohort of at least 10 students to start in September 2020. Further details of the Centre can be found at: http://www.bath.ac.uk/centres-for-doctoral-training/ukri-centre-for-doctoral-training-in-accountable-responsible-and-transparent-ai/.

Desirable qualities in candidates include intellectual curiosity, a strong background in maths and programming experience.

Applicants should hold, or expect to receive, a First or Upper Second Class Honours degree. A master’s level qualification would also be advantageous.

This project will be supervised by Dr Mohammad Golbabaee (Department of Computer Science) and Dr Clarice Poon (Department of Mathematics). Informal enquiries about the project should be directed to Dr Golbabaee: .

Enquiries about the application process should be sent to .

Formal applications should be made via the University of Bath’s online application form: https://samis.bath.ac.uk/urd/sits.urd/run/siw_ipp_lgn.login?process=siw_ipp_app&code1=RDUCM-FP02&code2=0002

Start date: 28 September 2020

Funding Notes

ART-AI CDT studentships are available on a competition basis for UK and EU students for up to 4 years. Funding will cover UK/EU tuition fees as well as providing maintenance at the UKRI doctoral stipend rate (£15,009 per annum in 2019/20, increased annually in line with the GDP deflator) and a training support fee of £1,000 per annum.

We also welcome all-year-round applications from self-funded candidates and candidates who can source their own funding.

References

[1] Boyd, Stephen, and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.
[2] Rockafellar, R. Tyrrell. Convex analysis. Vol. 28. Princeton university press, 1970.
[3] Bach, Francis. "Breaking the curse of dimensionality with convex neural networks." The Journal of Machine Learning Research 18.1 (2017): 629-681.
[4] Chizat, Lenaic, and Francis Bach. "On the global convergence of gradient descent for over-parameterized models using optimal transport." Advances in neural information processing systems. 2018.
[5] Ma, Dan, et al. "Magnetic resonance fingerprinting." Nature 495.7440 (2013): 187.
[6] Dong, Guozhi,et al. "Quantitative Magnetic Resonance Imaging: From Fingerprinting to Integrated Physics-Based Models." SIAM Journal on Imaging Sciences 12.2 (2019): 927-971.

How good is research at University of Bath in Computer Science and Informatics?

FTE Category A staff submitted: 24.00

Research output data provided by the Research Excellence Framework (REF)

Click here to see the results for all UK universities

Email Now

Insert previous message below for editing? 
You haven’t included a message. Providing a specific message means universities will take your enquiry more seriously and helps them provide the information you need.
Why not add a message here
* required field
Send a copy to me for my own records.

Your enquiry has been emailed successfully





FindAPhD. Copyright 2005-2019
All rights reserved.