University of Leeds Featured PhD Programmes
Catalysis Hub Featured PhD Programmes
Engineering and Physical Sciences Research Council Featured PhD Programmes
University of Kent Featured PhD Programmes
University of Bristol Featured PhD Programmes

Towards understanding recurrent neural networks by means of network attractors. Computer Science PhD

  • Full or part time
  • Application Deadline
    Monday, May 13, 2019
  • Competition Funded PhD Project (European/UK Students Only)
    Competition Funded PhD Project (European/UK Students Only)

Project Description

The University of Exeter EPSRC DTP (Engineering and Physical Sciences Research Council Doctoral Training Partnership) is offering up to 4 fully funded doctoral studentships for 2019/20 entry. Students will be given sector-leading training and development with outstanding facilities and resources. Studentships will be awarded to outstanding applicants, the distribution will be overseen by the University’s EPSRC Strategy Group in partnership with the Doctoral College.

Dr Lorenzo Livi, Department of Computer Science, College of Engineering, Mathematics and Physical Sciences
Professor Peter Ashwin, Department of Mathematics, College of Engineering, Mathematics and Physical Sciences

Project description:
Artificial recurrent neural networks (RNNs) are non-autonomous dynamical systems driven by (time-varying) inputs that perform computations exploiting short-term memory mechanisms.
Although shown to be effective in a number of applications, training RNNs is hard as a consequence of the "vanishing/exploding gradient problem’’. Moreover, their high-dimensional, non-linear structure complicates interpretability of internal dynamics, which are characterized by complex, input-dependent spatio-temporal patterns of activity. This poses constraints on the applicability of RNNs, which are usually treated as black-box thus preventing the extraction of knowledge from experimental data. Similar issues affect also other architectures, stressing the need to develop general methodologies for explaining the behaviour of machine learning methods when used for decision-making (e.g., credit and insurance risk assessment) and in scientifically-relevant applications (e.g., bio-markers discovery for genetic diseases).

This PhD proposal seeks to developing novel methodologies for providing a mechanistic description of the behaviour of RNNs by using attractor models suitable to describe nonautonomous, multi-stable systems that can operate and perform computations also in a transient
regime. In particular, we will focus on network attractors, a novel family of attractors that are composed by (unstable) fixed points and (heteroclinic/excitable) connections between them. Initially, we will consider ESNs trained with two forms of perturbation matrices applied to the randomly initialized recurrent connections: (i) output feedback acting as a closed-loop system and (ii) low-rank matrices designed to implement specific computations. Successively, we will take into account standard RNNs as well as more advanced architectures with LSTM and GRU cells.

We expect the theoretical developments described in this proposal to lead to optimized and explainable RNN models characterized by fewer parameters and hence suitable for implementation on resource-constrained hardware, such as low-power neuromorphic chips (or
other emerging computing technologies based on non-Von Neumann computing architecture). To this end, the project involves interactions and collaborations with IBM research UK.

Funding Notes

For successful eligible applicants the studentship comprises:

An index-linked stipend for up to 3.5 years full time (currently £14,777 per annum for 2018/19), pro-rata for part-time students.
Payment of University tuition fees (UK/EU)
Research Training Support Grant (RTSG) of £5,000 over 3.5 years, or pro-rata for part-time students

Related Subjects

Email Now

Insert previous message below for editing? 
You haven’t included a message. Providing a specific message means universities will take your enquiry more seriously and helps them provide the information you need.
Why not add a message here
* required field
Send a copy to me for my own records.

Your enquiry has been emailed successfully

FindAPhD. Copyright 2005-2019
All rights reserved.