The validation of ecological citizen science data

Citizen science approaches to wildlife monitoring are growing in scope, scale and prevalence. Long-running examples such as the Pan-European Common Birds Monitoring Scheme, and the UK’s Butterfly Monitoring Scheme show the important impact that these approaches can have on both science and monitoring. As schemes are widened to invite greater participation from non-experts, there is increasing interest in how the data submitted to citizen science recording schemes can be validated or verified. Automated verification could substantially reduce the burden on both experts and contributors, especially for uncontroversial records that typically make up the vast majority of records submitted to citizen science platforms. Here, we intend to address the problem of verification, assessing the impacts on national data aggregators (such as the National Biodiversity Network) of different rules and approaches to ranking the validity of submitted records.

Our specific aims are: (i) to assess existing ecological citizen science schemes in the UK to determine existing approaches to validation and verification, and the need for refinements; (ii) to develop statistical approaches to assign probabilities of validity to individual records, using contextual information both external (location, timing, known distributions, known frequency of occurrence, cross-referencing with other data sources) and internal (contributor demographics, contributor track-record and experience, wildlife community co-associations) to the focal schemes; (iii) for a small number of case study schemes, to work with the scheme coordinators and with developers to implement algorithms for automatic verification; (iv) to determine the impact, at a national level, of different approaches to ranking record validity and acceptance.

The project will begin with a review of existing approaches and the scale of the problem of data verification for citizen science. A range of approaches will be reviewed, including those used for UK systems to monitor butterflies (UKBMS), plants (NPMS) and bees (BeeWatch), as well as global citizen science platforms (such as BirdTrack and eBird). These include systems that verify non-photographic data based on ‘rules’ to indicate that sightings fall within expected parameters. Focusing on a number of key examples, methods for determining the probability that a record is valid will be assessed. This will involve Bayesian approaches that can take advantage of a range of contextual information in addition to prior probabilities, and yield posterior probabilities that a record is accurate. For a small number of focal systems, the student will develop a range of models to associate records with probabilities based on the models developed. By varying the approaches to verification, the student will simulate the impacts on records in the National Biodiversity Network, both from the point of view of numbers of accepted/rejected records and also in terms of inferences regarding the occupancy of key wildlife species. This will identify whether – and to what extent – verification is likely to affect interpretation for management and policy. To ensure the impact of the work, the student will work with developers to implement verification approaches within the focal platforms.