Towards understanding recurrent neural networks by means of network attractors, Computer Science, Mathematics – PhD (Funded)

The University of Exeter’s College of Engineering, Mathematics and Physical Sciences is inviting applications for an EPSRC NPIF PhD studentship to commence in September 2018. This project is one of two projects that are in competition for funding from the EPSRC NPIF scheme. For eligible students the studentship will cover tuition fees plus an annual tax-free stipend of at least £14,777 for 4 years full-time, or pro rata for part-time study. The student would be based in the Innovation Centre or Harrison Building in the College of Engineering, Mathematics and Physical Sciences at the Streatham Campus in Exeter.

Artificial recurrent neural networks (RNNs) are non-autonomous dynamical systems driven by (time-varying) inputs that perform computations exploiting short-term memory mechanisms. Although shown to be effective in a number of applications, training RNNs is hard as a consequence of the ``vanishing/exploding gradient problem’’. Moreover, their high-dimensional, non-linear structure complicates interpretability of internal dynamics, which are characterized by complex, input-dependent spatio-temporal patterns of activity. This poses constraints on the applicability of RNNs, which are usually treated as black-box thus preventing the extraction of knowledge from experimental data. Similar issues affect also other architectures, stressing the need to develop general methodologies for explaining the behaviour of machine learning methods when used for decision-making (e.g., credit and insurance risk assessment) and in scientifically-relevant applications (e.g., bio-markers discovery for genetic diseases).

This PhD proposal seeks to developing novel methodologies for providing a mechanistic description of the behaviour of RNNs by using attractor models suitable to describe non-autonomous, multi-stable systems that can operate and perform computations also in a transient regime. In particular, we will focus on network attractors, a novel family of attractors that are composed by (unstable) fixed points and (heteroclinic/excitable) connections between them.

Initially, we will consider ESNs trained with two forms of perturbation matrices applied to the randomly initialized recurrent connections: (i) output feedback acting as a closed-loop system and (ii) low-rank matrices designed to implement specific computations. Successively, we will take into account standard RNNs as well as more advanced architectures with LSTM and GRU cells.

We expect the theoretical developments described in this proposal to lead to optimized and explainable RNN models characterized by fewer parameters and hence suitable for implementation on resource-constrained hardware, such as low-power neuromorphic chips (or other emerging computing technologies).

The studentship will cover a stipend at the minimum Research Council rate £14,777 per annum (for 2018/19), research costs and tuition fees. This project is advertised under the EPSRC NPIF training grant under which a limited number of studentships can be allocated with open eligibility. This means students from outside of the UK are eligible to apply for this studentship.


Location: Computer Science, Streatham Campus, Exeter

Entry requirements:
Applicants for this studentship must have obtained, or be about to obtain, a First or Upper Second Class UK Honours degree, or the equivalent qualifications gained outside the UK, in Computer Science, Applied Mathematics, Physics, Electronic or Mechanical Engineering. Applicants with some training in dynamical systems and neural networks will be prioritized.
If English is not your first language you will need to have achieved at least 6.0 in IELTS and no less than 6.0 in any section by the start of the project. Alternative tests may be acceptable (see http://www.exeter.ac.uk/postgraduate/apply/english/).


How to apply

In the application process you will be asked to upload several documents:

• CV
• Letter of application (outlining your academic interests, prior research experience and reasons for wishing to undertake the project).
• Transcript(s) giving full details of subjects studied and grades/marks obtained (this should be an interim transcript if you are still studying)
• Two references from referees familiar with your academic work. If your referees prefer, they can email the reference direct to [Email Address Removed] quoting the studentship reference number.
• If you are not a national of a majority English-speaking country you will need to submit evidence of your proficiency in English


The closing date for applications is midnight on 12 July 2018. Interviews will be held on the University of Exeter Streatham Campus or via Skypein late July 2018.
If you have any general enquiries about the application process please email [Email Address Removed] or phone +44 (0)1392 722730. Project-specific queries should be directed to the main supervisor.