Don't miss our weekly PhD newsletter | Sign up now Don't miss our weekly PhD newsletter | Sign up now

  Hierarchical reinforcement learning

   Centre for Accountable, Responsible and Transparent AI

This project is no longer listed on and may not be available.

Click here to search for PhD studentship opportunities
  Dr Özgür Şimşek  No more applications being accepted  Self-Funded PhD Students Only

About the Project

One of the most fundamental problems in artificial intelligence (AI) is how to enable autonomous development of action hierarchies, in other words, how to endow artificial agents with the ability to autonomously form useful high-level behaviours (for example, grasping) from existing behavioural units (for example, primitive sensory and motor actions available to a robot). This ability allows a developmental process during which an agent can learn to display behaviours of increasing complexity through continuously building on its existing set of skills to acquire new ones. For example, grasping may be followed by manipulating objects in different ways, which may be followed by using a key to unlock the door to an adjacent room, and so on, forming a continuously growing hierarchy of behaviours.

This project will develop algorithms that enable artificial agents to go through such developmental processes autonomously. As a result, artificial agents will develop the capability of performing tasks of increasing difficulty without guidance tailored for the specific domains in which they operate. This type of open-ended developmental process is fundamentally different from how artificial agents learn to perform complex tasks today and has the potential to dramatically increase their capabilities.

The project will use the elegant theoretical framework of reinforcement learning. The algorithmic approaches explored will depend on the background and interests of the student. Application areas may include, but are not limited to, engineering and robotics.

The project will be supervised by Dr Özgür Şimşek at the Department of Computer Science at the University of Bath. The University is located in the UNESCO World Heritage city of Bath, providing a vibrant research environment in one of the most beautiful areas in the UK.

Informal enquires are welcome and should be directed to Dr Özgür Şimşek ([Email Address Removed]).

Applicants should hold, or expect to receive, a First or Upper Second Class Honours degree in a relevant subject. A master’s level qualification would also be advantageous. Desirable qualities in candidates include intellectual curiosity, a strong background in mathematics and programming experience.

Formal applications should be accompanied by a research proposal and made via the University of Bath’s online application form. Further information about the application process can be found here.

Start date: Between 8 January and 30 September 2024.

Computer Science (8) Engineering (12) Mathematics (25)

Funding Notes

We welcome applications from candidates who can source their own funding. Tuition fees for the 2023/4 academic year are £4,700 (full-time) for Home students and £26,600 (full-time) for International students. For information about eligibility for Home fee status:

How good is research at University of Bath in Computer Science and Informatics?

Research output data provided by the Research Excellence Framework (REF)

Click here to see the results for all UK universities

Where will I study?

Search Suggestions
Search suggestions

Based on your current searches we recommend the following search filters.