1. Professor Gary Collins
2. Dr Michael Schlussel
3. Dr Paula Dhiman
4. Dr Jennifer de Beyer
Research Project Outline
Artificial intelligence (AI) and machine learning (ML) are increasingly seen as solutions to many healthcare problems (e.g., for risk prediction, imaging), and the pace of development is showing no signs abating.
Although the opportunities for implementing AI and ML to improve outcomes for patients are vast, there are also considerable concerns surrounding their hype. Underpinning the hype are study design and methodological issues and poor reporting. Studies comparing different AI/ML strategies are often fraught with methodological flaws, such as small sample sizes, a focus on classification when prediction is required, incomplete or flawed model performance, and researcher bias (1). These issues make fair comparisons between the different approaches difficult and misleading. Critical appraisal of study methodology and reporting is therefore key to ensuring the findings from these studies can be appropriately judged before they are used to change clinical practice (2).
Methodological and reporting standards to ensure reproducibility and regulation of AI/ML technologies are currently in their infancy. Contributing to this uncertainty are concerns that the ‘black box’ nature of AI/ML will hamper the evaluation of the technology by independent researchers (e.g., independent evaluation of a prediction model developed using ML). Compounding this further are concerns around commercial exploitation, intellectual property, and unavailability of software for implementation.
This project, funded by Cancer Research UK, will focus on the evaluation of AI and ML methods for clinical prediction in both diagnosis and prognosis. The project will include an evaluation of the oncology literature to identify key and important sources of bias. The methodology and reporting of various study designs, including simulation studies, will be examined. The prevalence of hype and ‘spin’ (such as selective reporting and weak methodology) will be investigated and quantified. A classification scheme to identify sources of hype/’spin’ will be developed. The project will combine these findings to provide recommendations for making fair comparisons between the various AI/ML approaches and more traditional statistical methods.
Details of the Research Group
The DPhil will be jointly supervised by Professor Gary Collins, Dr Michael Schlussel, Dr Paula Dhiman, and Dr Jennifer de Beyer, all based in the UK EQUATOR Centre within the Centre for Statistics in Medicine (CSM), NDORMS, University of Oxford.
The Centre for Statistics in Medicine (CSM) in Oxford is committed to providing collaborative statistical support for the design, analysis and reporting of clinical research, carries out a methodological research programme and runs training courses. The EQUATOR Network is an international initiative that seeks to improve the quality of the health research literature, primarily by promoting transparent and accurate reporting of health research studies. It acts as an ‘umbrella’ organisation providing resources and training relating to the reporting of health research and assisting in the development, dissemination and implementation of reporting guidelines.
Prof Gary Collins’ research interests are focused on methodological aspects surrounding the development and validation of multivariable prediction models and has published widely in this area. (www.ndorms.ox.ac.uk/team/gary-collins)
Dr Michael Schlussel is a senior medical statistician. He has extensive experience in the design, conduct and reporting of biomedical research. (www.ndorms.ox.ac.uk/team/michael-maia-schlussel)
Dr Paula Dhiman is a post-doctoral research in meta-research, and has research interests in prediction models and evaluating the reporting and conduct of biomedical research. (www.ndorms.ox.ac.uk/team/paula-dhiman)
Dr Jennifer de Beyer is an expert in reporting guidelines, leads the UK EQUATOR Publication School, and has published studies evaluating ‘spin’ in biomedical research. (www.ndorms.ox.ac.uk/team/jennifer-de-beyer)
Training will be provided in relevant related research methodology. Attendance at formal training courses will be encouraged, and will include the ‘Real World Epidemiology Oxford Summer School’, and the Prognosis Methods Course at Keele University.
Courses on key skills for the completion of a successful DPhil thesis will be available from the Oxford Learning Institute, Oxford University Computing Services, Medical Sciences Division Graduate Training Programme, and UK EQUATOR Centre. The supervisors will also encourage the student to pursue any additional on-the-job training opportunities that arise.
A core curriculum of lectures organized departmentally will be taken in the first term to provide a solid foundation in a broad range of subjects, including epidemiology, health economics, and data analysis.
The student will attend weekly seminars within the department and relevant seminars in the wider University.
The student will be expected to present data regularly to the department and the research group and to attend external conferences to present their research globally. The student will also be expected to disseminate their research findings in peer-reviewed academic journal articles, with support from the supervisory team and wider research group.
How to apply
The department accepts applications throughout the year but it is recommended that, in the first instance, you contact the relevant supervisors or the Graduate Studies Officer, Sam Burnell ([email protected]
), who will be able to advise you of the essential requirements.
Interested applicants should have or expect to obtain a first or upper second class BSc degree or equivalent, and will also need to provide evidence of English language competence. The application guide and form are found online and the DPhil will commence in October 2020.
For further information, please visit http://www.ox.ac.uk/admissions/graduate/applying-to-oxford
and/or contact Professor Gary Collins ([email protected]