Don't miss our weekly PhD newsletter | Sign up now Don't miss our weekly PhD newsletter | Sign up now

  Understanding & Controlling Generative NLP Models


   Department of Mathematics

This project is no longer listed on FindAPhD.com and may not be available.

Click here to search FindAPhD.com for PhD studentship opportunities
  Prof Theodore Papamarkou, Dr A Freitas  Applications accepted all year round  Self-Funded PhD Students Only

About the Project

Motivation

Large Language Models such as ChatGPT have demonstrated the ability to generate high quality answers to complex questions and instructions, demonstrating the ability of these models to learn and encode complex linguistic, context and inference patterns. The properties and mechanics of these induced representations are, however, poorly understood and have limited generation control, sometimes producing hallucinations and inconsistent results. This project aims to improve the understanding of the properties of large language models, aiming towards their improved control. With the support of Bayesian methods for NLP, the project aims towards providing insights and a better foundational understanding of in-context learning, as well as tools for predictive uncertainty quantification in NLP.

Project Overview

 Bayesian methods have been used in numerous NLP problems (Cohen 2018), including word segmentation (Goldwater et al. 2009), syntax (Johnson et al. 2007), morphology (Snyder & Barzilay 2008), coreference resolution (Haghighi & Klein 2007), machine translation (Blunsom et al. 2009), and active learning (Margatina et al, 2021). More recently, in-context learning in NLP has been formulated under the Bayesian paradigm (Xie et al, 2022). More specifically, (Xie et al, 2022) cast in-context learning as Bayesian inference by letting pretrained language models implicitly infer concepts upon conducting Bayesian marginalization over concepts. The work of (Xie et al, 2022) opens up multiple avenues of research to understand in-context learning with language models. For instance, one possible research question is on how to elicit previously unseen concepts, to be utilised under the Bayesian paradigm towards effective in-context learning. A second research avenue is to improve the estimation of the posterior predictive distribution of outputs, and their association to prompt optimisation (and their associated uncertainty).

The Candidate

Required attributes:

·      BSc in computer science, mathematics or related areas (with a solid mathematical basis).

·      Mathematical/probabilistic/statistical background in machine learning.

·      Comfortable in Python programming (evidenced by existing projects).

·      Fluent English.

At least one of the following attributes:

·      MSc in AI, Data Science or related areas.

·      Scientific publications.

The Supervision Team and Research Environment

This project is a collaboration between the departments of Computer Science and Mathematics. The Department of Computer Science at the University of Manchester is the longest established department of Computer Science in the United Kingdom and one of the largest. The Department of Mathematics at the University of Manchester is one of the largest unified mathematics departments in the United Kingdom, with over 90 academic staff and an undergraduate intake of roughly 400 students per year and approximately 200 postgraduate students in total. The University of Manchester is a member of the Russell Group, the N8 Group, and the worldwide Universities Research Association. The University of Manchester has 25 Nobel laureates among its past and present students and staff, the fourth-highest number of any single university in the United Kingdom. Manchester saw the birth of computer science, with the creation of the world's first stored-program computer. We continue to work on pioneering research with widespread activity and strength in a range of key aspects of computer science from hardware through to user interaction.

Funding 

At Manchester we offer a range of scholarships, studentships and awards at university, faculty and department level, to support both UK and overseas postgraduate researchers. 

For more information, visit our funding page or search our funding database for specific scholarships, studentships and awards you may be eligible for. 

Before you apply 

Qualified applicants are encouraged to informally contact:

·      Andre Freitas ([Email Address Removed]) and

·      Theo Papamarkou ([Email Address Removed])

with CV and transcripts to discuss the application prior to applying.

Equality, diversity and inclusion 

Equality, diversity and inclusion is fundamental to the success of The University of Manchester and is at the heart of all of our activities. We know that diversity strengthens our research community, leading to enhanced research creativity, productivity and quality, and societal and economic impact. We actively encourage applicants from diverse career paths and backgrounds and from all sections of the community, regardless of age, disability, ethnicity, gender, gender expression, sexual orientation and transgender status. 

We also support applications from those returning from a career break or other roles. We consider offering flexible study arrangements (including part-time: 50%, 60% or 80%, depending on the project/funder).

Computer Science (8) Mathematics (25)

References

References

[Xie at al, 2022] An explanation of in-context learning as implicit Bayesian inference.
[Margatina et al, 2021] Bayesian active learning with pre-trained language models.
[Cohen, 2018] Bayesian analysis in NLP.
Search Suggestions
Search suggestions

Based on your current searches we recommend the following search filters.

 About the Project