General probability distributions for structured machine learning

Supervised machine learning models, such as artificial neural networks, are unstructured. They can be trained from data, but there is no possibility to include any additional knowledge, e.g. that two features are independent conditioned on knowing a third. To give a specific scenario, you know that air pressure affects rain which causes your weather station to measure rainfall. But air pressure has no direct affect on the rain measurement, and the two are independent if you know if it is raining or not. This cannot be included in traditional supervised machine learning.

Structure can be modelled using probabilistic graphical models. These are however limited by the inference algorithms (primarily belief propagation, Gibbs sampling and mean field variational methods), which require the use of simple probability distributions that are a poor fit to reality.

This PhD, which is aligned with the UKRI Centre for Doctoral Training in Accountable, Responsible and Transparent Artificial Intelligence (ART-AI), is about exploring more general representations, specifically arbitrary density estimates, that can represent any distribution. Prior work has almost entirely been particle based [1,2,3], and has approximations and/or inefficient search that compromises performance. There are many possible improvements that can be made to the particle approaches; additionally new alternatives can be explored.

Graphical models are a kind of explainable AI, as the structure can be human understandable. Unfortunately their underperformance relative to other models limits their usage, particularly in industry. Generalising their representative capabilities, to match better known models, is one step towards wider usage. Given the 'right to an explanation' requirement of the GDPR this may become legally necessary. Additionally, a graphical model introduces a modular structure that can be debugged. As AI makes it way into safety critical scenarios, such as self driving cars, graphical models may prove necessary for quality assurance.

Candidates should normally have a good first degree or a Master’s degree in computer science, maths, or a related discipline. A strong mathematical background is essential; good programming skill and previous machine learning experience highly desirable.

Informal enquiries about the project should be directed to Dr Tom Fincham Haines.

Formal applications should be accompanied by a research proposal and made via the University of Bath’s online application form. Enquiries about the application process should be sent to art-ai-applications@bath.ac.uk.

Start date: 2 October 2023.