Don't miss our weekly PhD newsletter | Sign up now Don't miss our weekly PhD newsletter | Sign up now

  Reinforcement learning based discovery of numerical algorithms

   Centre for Accountable, Responsible and Transparent AI

This project is no longer listed on and may not be available.

Click here to search for PhD studentship opportunities
  Dr Pranav Singh, Dr Matthew Grayson  No more applications being accepted  Self-Funded PhD Students Only

About the Project

Deep neural networks have recently emerged as promising contenders for efficiently solving partial differential equations (PDEs) and ordinary differential equations (ODEs). However, current neural network approaches lack transparency and consequently have proven difficult to analyse or understand. Moreover, they are often found to lack stability, and the solutions typically do not respect conservation laws and come with little to no guarantee of convergence. 

In contrast, an extremely wide range of specialised numerical methods have been devised over decades for different ODEs and PDEs. These numerical methods are transparent and understandable, and come with various convergence, stability, and conservation guarantees. However, the discovery of such algorithms is highly intensive in human effort and ingenuity, typically requiring specialised training in various subfields of mathematics and the domain of application where the specialised PDE or ODE arises. Moreover, due to the exorbitant effort of deriving specialised algorithms for narrow subclasses of problems and specialised computer architectures, these algorithms tend to leave a significant scope for improvement in practice. 

In recent years, there has been a huge amount of interest in combining the flexibility and universality of machine learning techniques with specialised numerical methods for ODEs and PDEs with the aim of deriving efficient and flexible numerical methods with provable guarantees. A large amount of this effort is directed at deriving neural architectures and training inspired by ODE/PDE theory and algorithms. Other notable directions include embedding neural networks within existing numerical algorithms.

However, while some of these approaches can obey conservation laws, they are still riddled with lack of stability and convergence guarantees. Moreover, due to their inherent reliance on neural architectures, they lack transparency and interpretability.

Recently, a fundamentally different approach called AlphaTensor for discovering numerical algorithms using deep reinforcement learning has been proposed in the context of matrix factorisation algorithms ( This approach formulates the search for a novel algorithm as a high dimensional game, with the moves being chosen from a set of valid mathematical operations and the payoff estimating the expected cost of the algorithm. This approach builds on the success of the AlphaGo algorithm and comes the closest to mimicking a human expert’s approach for algorithm discovery. Since only valid mathematical operations are ever considered, at a structural level this approach is highly promising for the design of novel transparent, provable and analysable numerical methods. 

This project will explore how to extend the AlphaTensor approach beyond matrix multiplication algorithms and into the domain of numerical methods for solving ODEs and PDEs. A sophisticated algorithm for such problems requires the use of specialised operations from the domains of numerical linear algebra, Lie algebras and Lie groups, numerical quadrature, among others. Since the set of elementary operations is significantly larger, the search space is exponentially larger, and novel techniques will need to be developed to effectively explore this space. 

The ODEs and PDEs considered in this project will come from the domain of computational quantum chemistry. These ODEs and PDEs remain some of the most computationally challenging scientific problems. However, the advances in this project are expected to have much broader implications across a wide range of domains where numerical methods are employed. 

This project is associated with the UKRI Centre for Doctoral Training (CDT) in Accountable, Responsible and Transparent AI (ART-AI). We value people from different life experiences with a passion for research. The CDT's mission is to graduate diverse specialists with perspectives who can go out in the world and make a difference.

Applicants should hold, or expect to receive, a first or upper-second class honours degree in a relevant subject. Applicants should have taken a mathematics unit or a quantitative methods course at university or have at least grade B in A level maths or international equivalent. Experience with coding (any language) is desirable.

Formal applications should be accompanied by a research proposal and made via the University of Bath’s online application form. Enquiries about the application process should be sent to [Email Address Removed].

Start date: 2 October 2023.

Chemistry (6) Computer Science (8) Mathematics (25)

How good is research at University of Bath in Mathematical Sciences?

Research output data provided by the Research Excellence Framework (REF)

Click here to see the results for all UK universities
Search Suggestions
Search suggestions

Based on your current searches we recommend the following search filters.

 About the Project