Predicting direction and magnitude of bias in clinical trials

Although teams conducting randomised controlled trials (RCTs) aim to minimise bias, some desirable features can be challenging to ensure in practice: e.g. successful blinding of all relevant parties and minimal attrition. If one of these features is either infeasible or its implementation has failed, it is desirable to understand the likely effect of this on the estimated intervention effect. Such an understanding would considerably aid decision-making about healthcare policy - either based on the RCT directly or via its inclusion in a meta-analysis. ‘Meta-epidemiological’ studies offer potential to inform us about the likely magnitude and direction of bias associated with particular undesirable study characteristics. These are collections of meta-analyses in which the association of characteristics with estimated intervention effects has been studied across a large number of RCTs. However, it is uncertain how well the results of such studies are applicable to any individual trial. Ultimately, in predicting the direction and magnitude of likely biases, it would be desirable to produce bias-adjusted effect estimates. Two main methods have been proposed for this. In the first method (Turner method) the adjustments are based on expert opinion about likely biases in individual RCTs while the other (Welton method). uses empirical evidence from meta-epidemiological studies. The recently developed integrated method (MRC-funded COMBAT study) combines the most attractive features of these two approaches: bias-adjustment is based on a combination of meta-epidemiological evidence and on the user’s own anticipation of both the direction and likely magnitude of bias. However, users find it difficult to predict these two things and require more guidance.

Aims
The project aims to lead to a better understanding of the results of clinical trials in the context of potential flaws, through prediction of the likely direction and magnitude of any bias. Methods of 'bias-adjustment' will be further developed and investigated, aiming to produce results that are less biased and less likely to be spuriously precise.

Methods
The primary purpose of the project is to develop methods for predicting the magnitude and direction of bias in a clinical trial. A key area of work will be the development of guidance for users of the newly developed (HTMR funded) tool for assessing risk of bias in RCTs (RoB 2.0) [available at www.riskofbias.info], which includes an optional facility of assessing the likely direction of any bias. An extensive library of examples will be collated from among our existing meta-epidemiological studies to gather insight into how large apparent biases might have arisen. The work will start by considering individual domains of bias and interactions between domains. Further re-analyses of meta-epidemiological data are expected to lead to insight into the extent to which the biases are additive.

A second area of work is to further develop methods for bias adjustment. The student will focus on the development of user-friendly integrated methods, based on a combination of expert opinion and empirical evidence. The student will develop a standardised way of converting qualitative RoB judgements (e.g. “high risk of bias in the direction of overestimation of treatment benefit”) into a numerical interquartile range estimate of bias on the ratio of odds ratio scale (e.g. “0.71-0.99”) and use this IQR as the opinion component for the COMBAT method of bias adjustment. We plan for the student to write software (e.g. a Stata command) for bias adjustments based on an integrated approach.

Details of supervision and training
Supervisory team have extensive expertise in methods relevant to this project and the completing student will develop skills and expertise in this area (e.g. clinical trial methods, epidemiology, evidence synthesis, Bayesian statistics, meta-analysis and decision theory). In addition, Jonathan Sterne, Nicky Welton and Rebecca Turner (MRC BSU Hub in Cambridge) will have an advisory role. Our School also offers a comprehensive and highly regarded programme of over 30 short courses in research methods, freely available to our PhD students.