Are you applying to universities? | SHARE YOUR EXPERIENCE Are you applying to universities? | SHARE YOUR EXPERIENCE

Multi-objective evolutionary framework for autocurricula development in multiagent systems

   Cardiff School of Computer Science & Informatics

This project is no longer listed on and may not be available.

Click here to search for PhD studentship opportunities
  Dr Walter Colombo, Dr Juan Hernandez Vega, Dr Christopher Wallbridge, Prof Roger Whitaker  No more applications being accepted  Competition Funded PhD Project (Students Worldwide)

About the Project

AI has made remarkable progress attempting to solve a range of real-life problems with artificial neural networks (ANN) and the technique of reinforcement learning (RL). With attention to the field of robotics, traditional RL techniques have primarily focused on given topologies of the underlying neural networks with rewards targeted to the specific problem considered (e.g. self-driving vehicles, robot’s manipulation). However, further enhancements could derive from addressing multiple goals and a selection of tasks. This underlying idea has been applied in goal conditioned reinforcement learning where real or simulated robotic agents can learn to select, represent, and solve their own self-generated problems [1].  

In this way the AI agent will aim to build a ‘repertoire of skills’ allowing to solve several problems, including those that have potentially never seen before, with a certain degree of flexibility and increasing complexity. While most systems implement a single robotic agent by extending this to multi-agent systems such collections can develop into autocurricula of tasks entirely generated by the interacting agents, potentially cooperating with each other. This will minimise the scientist’s biases in the selection and definition of external goals and environments [2].  

A further limitation of RL in such scenarios lies on the rigidity of the underlying neural network topologies and evolutionary approaches might offer a solution in this sense and reduce complexity by implementing a population of ANNs that can be maintained in parallel with different parameters and topologies. This approach no longer requires fixed reward functions but can combine instead a variety of objectives. These include the extrinsic goals set by the problem but also intrinsic ones, thus also implementing components of novelty, curiosity am creativity [3]. Evolutionary frameworks have been successfully implemented with ANNs in the form of neuroevolutionary models [4] and have been more recently applied to simulated environments outperforming traditional RL applications [5]. 

The proposed project aims to explore these type of multi-agent systems to either simulated robotic environments as a starting point, such as those proposed by openAI gym [6], eventually considering real ones.  We will also apply this to multiple objectives and environments, thus simulating the diverse forces of the `natural selection’ in real world evolution.  Finally, and differently from current implementations, these ANN agents can interact and evolve by applying techniques that mimic the human evolution of cultural artefacts such as forms of social learning and mechanisms for non-random variation and replacement [7]. 

Keywords: Artificial Intelligence / Human-Centred Computing / Robotics / Neural Networks, Evolutionary Algorithms / Human Innovation. 

Contact for further information: Dr Walter Colombo  [Email Address Removed] 

Academic criteria: A 2:1 Honours undergraduate degree or a master's degree, in computing or a related subject. Applicants with appropriate professional experience are also considered. Degree-level mathematics (or equivalent) is required for research in some project areas. 

Applicants for whom English is not their first language must demonstrate proficiency by obtaining an IELTS score of at least 6.5 overall, with a minimum of 6.0 in each skills component.  

How to apply:  

Please contact the supervisors of the project prior to submitting your application to discuss and develop an individual research proposal that builds on the information provided in this advert. Once you have developed the proposal with support from the supervisors, please submit your application following the instructions provided below 

Please submit your application before the application deadline 13th March 2023 via Computer Science and Informatics - Study - Cardiff University 

In order to be considered candidates must submit the following information:  

  • Supporting statement  
  • CV  
  • In the ‘Research Proposal’ section of the application enter the name of the project you are applying to and upload your Individual research proposal, as mentioned above in BOLD 
  • In the funding field of your application, insert “I am applying for 2023 PhD Scholarship in Computer Science and Informatics”, and specify the project title and supervisors of this project in the text box provided. 
  • Qualification certificates and Transcripts 
  • References x 2  
  • Proof of English language (if applicable) 

If you have any questions or need more information, please contact [Email Address Removed] 

Funding Notes

A School-Funded PhD Scholarship is available for entry 2023/24.
In the Funding field of your application, insert "I am applying for 2023 PhD Scholarship" and specify the project title and supervisor of this project in the fields provided.
This project is also open to Self-Funded students worldwide. If you are interested in applying for a Self-Funded PhD, please search FindAPhD for this specific project title, supervisor or School within its Scholarships category


[1] Colas, C., Karch, T., Sigaud, O. and Oudeyer, P.Y., 2022. Autotelic agents with intrinsically motivated goal-conditioned reinforcement learning: a short survey. Journal of Artificial Intelligence Research.
[2] Leibo, J.Z., Hughes, E., Lanctot, M. and Graepel, T., 2019. Autocurricula and the emergence of innovation from social interaction: A manifesto for multi-agent intelligence research. arXiv preprint arXiv:1903.00742.
[3] Boden, M.A., 2009. Computer models of creativity. AI Magazine.
[4] Stanley, K.O. and Miikkulainen, R., 2002. Evolving neural networks through augmenting topologies. Evolutionary computation.
[5] Wang, R., Lehman, J., Rawal, A., Zhi, J., Li, Y., Clune, J. and Stanley, K., 2020, November. Enhanced POET: Open-ended reinforcement learning through unbounded invention of learning challenges and their solutions. In International Conference on Machine Learning, PMLR.
[6] Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J. and Zaremba, W., 2016. Openai gym. arXiv preprint arXiv:1606.01540.
[7] Boyd, R. and Richerson, P.J., 1988. Culture and the evolutionary process. University of Chicago press.

How good is research at Cardiff University in Computer Science and Informatics?

Research output data provided by the Research Excellence Framework (REF)

Click here to see the results for all UK universities
PhD saved successfully
View saved PhDs