PhD studentship in model-based Reinforcement Learning

The School of Energy, Geoscience, Infrastructure and Society (EGIS) at Heriot-Watt University (Scotland, UK) is looking for a PhD candidate to work on engineering optimisation and control using Reinforcement Learning (RL) techniques. The successful candidate will be part of the project ECO-AI (Enabling CO2 capture and storage using AI). ECO-AI team includes academics based at Heriot-Watt university (School of Engineering and Physical Sciences (EPS) and School of Energy, Geoscience, Infrastructure and Society (EGIS)) and at Imperial College London (Department of Earth Sciences and Department of Chemical Engineering). The project webpage https://ai4netzero.github.io/ecoai_project/.

Reinforcement learning has been demonstrated as a viable approach to solve various optimisation and control tasks (e.g., video games, robotics, and active flow control). Most of these successes relied on what is called model-free RL algorithms where an agent learns an optimal policy by interacting with an environment without the building an internal model of the environment dynamics. In model-free RL algorithms, an agent utilizes a guided trial-and-error approach by playing millions of games to explore the game environment and to obtain a superhuman policy. However, the transfer of these advances to real-world applications, wherein the cost of interacting with the environment is often high, can be prohibitively challenging. For instance, conducting millions of interactions with an experimental setup can be practically infeasible, and interacting with a detailed simulation environment can be slow and computationally demanding. Model-based reinforcement learning algorithm offers a more sample efficient approach for learning control policies. Model-based RL algorithms aim to learn an internal model of the environment dynamics and uses the learned model to plan ahead. Model-based RL has the potential to considerably decrease the necessary interactions between the agent and the environment, yet it often encounters challenges due to inaccuracies in the learned environment dynamics. In this project, we aim to advance model-based RL techniques by embedding model errors and uncertainties in model-based RL algorithms. The developed algorithm will be demonstrated on a wide range of engineering applications including active flow control, renewable energy generation and industrial/chemical process control.

 Essential skills:

Master's degree in computational mathematics, physics or in a relevant engineering discipline with strong computational skills

Excellent programming skills preferably in Python and/or C++

Ability to write reports, collate information and present it in a clear and engaging manner

Excellent communication skills

Desirable skills:

Bayesian statistics and machine learning (theory and applications)

Experience with machine learning libraries (pytorch, jax, etc)

Reinforcement learning techniques

Numerical optimization

Funding

The scholarship will cover tuition fees (Home and overseas) and provide an annual stipend (£18,622 for 2023-24) for the 36-month duration of the studentship, as well as research support and travel costs. Thereafter, students will be expected to pay a continuing affiliation fee (currently £130) to cover their continued registration whilst writing up their thesis. 

How to Apply

To apply you must complete our online application form.

Please select PhD Applied Geosciences as the programme and include the full project title, reference number and supervisor name on your application form. Ensure that all fields marked as ‘required’ are complete.

Once have entered your personal details, click submit. You will be asked to upload your supporting documents. You must complete the section marked project proposal; provide a supporting statement (1-2 A4 pages) documenting your reasons for applying to this particular project, outlining your suitability and how you would approach the project. You must also upload your CV, a copy of your degree certificate and relevant transcripts and an academic reference in the relevant section of the application form.

You must also provide proof of your ability in the English language (if English is not your mother tongue or if you have not already studied for a degree that was taught in English within the last 2 years). We require an IELTS certificate showing an overall score of at least 6.5 with no component scoring less than 6.0 or a TOEFL certificate with a minimum score of 90 points.

Please contact Prof. Ahmed H. Elsheikh; ([Email Address Removed]) for further informal or an informal discussion.

Please contact [Email Address Removed] for technical support with your application.

Timeline

The closing date for applications is 23rd of June 2023 and applicants must be available to start in September 2023.