Using Structured Expert Judgements to Assess Project Evaluation

This PhD research will develop methods for using structured expert judgements as part of the ongoing quantitative assessments of the impact of a large scale research hub. The objective of using structured approaches is to minimize different kinds of bias that may otherwise affect judgemental data.

Eligibility
Candidates are required to have:
• An excellent undergraduate degree with Honours in an analytical subject, e.g. mathematics, statistics, management science, computing science, etc.,
• A Masters degree (or equivalent) will be strongly preferred
• Students may also have other relevant experience or skills which are relevant to this project
• Candidates who are not native English speakers will be required to provide evidence for their English skills (such as by IELTS or similar tests that are approved by UKVI, or a degree completed in an English speaking country).

Project Details
This is an exciting opportunity to join the One Ocean Hub, a new, £20M global hub for interdisciplinary research funded through the UKRI Global Challenges Research Fund. The Hub seeks to address intractable challenges faced by developing countries in relation to ocean management. Through this 5-year initiative, researchers from the UK will work with colleagues across the world to address the challenges of South Africa, Namibia, Ghana, Fiji and Solomon Islands in realising the economic, socio-cultural and environmental benefits from the ocean. The Hub will weave learning from the ocean, ranging from the traditional knowledge of the peoples who rely upon it to marine sciences, innovative legal approaches and artistic methods. Our aim is to bridge the disconnections in law, science and society across all levels from the local to the international. This PhD position will directly feed into supporting some of the challenges facing the Hub.

Due to the highly faceted and interconnected nature of the socio, political, economic, and natural systems the Hub aims to affect, identifying successful strategies, measuring their impact, and evaluating what can be successfully replicated elsewhere will be a challenge. Structured expert judgement will enable quantitative estimates and evaluation when data does not exist, either because of the complexity of the measurement task, a lack of resources to devote to measurement and evaluation, or because the interest is in future outcome.

Structured expert judgement methods rely on quality interaction between a facilitator and the expert, in either individual interviews or group workshops. The facilitator(s) trains experts in expressing uncertainty through probabilistic assessments, guides the experts through the questions, and elicits qualitative information on the experts’ rationales alongside their quantitative estimates. When elicitation happens in group workshops, experts are also able to share their knowledge and experience with the group, ideally improving the quality of the resulting assessments. Although tools exist for remote elicitation, most limit opportunities for interaction between the facilitator and experts. This PhD student will develop new methods for remote elicitation that adopt evidence-based, in-person elicitation protocols to a setting with geographically dispersed experts and facilitators. The student will use these new approaches to support an elicitation-based evaluation strategy augmenting other work taking place in the Hub. It is envisaged that a series of elicitations will take place during the course of the project, with through-time repeated elicitations allowing for assessment of expert development in terms of ability to assess uncertainty and improved understanding of the changing system being considered.

The objectives of this PhD
• To review the literature on elicitation processes for uncertain quantities and to evaluate those of relevance to the questions arising in one-ocean
• To map a range of measures of interest with the leaders of the hub and in conjunction with the other PhD projects
• To develop elicitation protocols and approaches which support structured engagement between expert groups distributed in different parts of the world
• To run up to 3 longitudinal Structured Expert Judgement processes for the One Ocean programme to demonstrate the methodology and to write outputs in collaboration with other members of the team.
• To explore the assessments of dependencies between quantities being assessed

Supervisor
Dr Abigail Colson - https://www.strath.ac.uk/staff/colsonabigailms/
Professor Tim Bedford - https://www.strath.ac.uk/staff/universityleaders/associateprincipalprofessortimbedford/

Contact Us
For more information, please contact Dr Abigail Colson on [Email Address Removed]