FREE Virtual Study Fair | 1 - 2 March | REGISTER NOW FREE Virtual Study Fair | 1 - 2 March | REGISTER NOW

A methodological framework for incorporating multiple outcomes into hybrid implementation and effectiveness trials

   Institute of Applied Health Research

This project is no longer listed on and may not be available.

Click here to search for PhD studentship opportunities
  Dr K Hemming  No more applications being accepted  Competition Funded PhD Project (Students Worldwide)

About the Project

Many policies are adopted into practice without a solid evidence. Rigorous randomised trials are therefore needed to inform decisions about the implementation of policy interventions. Hybrid implementation effectiveness trials offer a means to both evaluate implementation and effectiveness within a single trial design [Curran Med Care. 2012;50(3)]. Cluster randomised designs are a mainstay for evaluation and are likely to become increasingly used. Hybrid implementation effectiveness trials often have multiple outcomes at different points in the system – ranging from upstream outcomes (i.e., process outcomes) to downstream outcomes (i.e., clinical outcomes), sometimes with key mediating intervening outcomes (i.e., biomarkers). When designing these trials investigators are faced with the conundrum of how to specify key outcomes at each of these levels. Is it, for example, appropriate to have one implementation outcome, one mediating outcome and one clinical outcome? Or, should investigators settle on one primary clinical outcome to satisfy reporting guidelines and increase likelihood of acceptance in a high-impact journal? Related to this is the question of whether or not success of the policy will be declared only in situations where signals are identified on all key outcomes (i.e., both upstream and downstream outcomes, and perhaps also on key mediating outcomes)?

By way of example, there is consistent evidence that women with a short cervical length are at higher risk of preterm birth and previous studies have shown benefit of treating with progesterone if the cervix is short. Together, these two pieces of evidence suggest that both screening for short cervix and subsequent treatment with progesterone, should be implemented as policy. However, it is unclear if i) screening programmes would be adopted and adhered to and ii) if treating with progesterone would be beneficial when nested within a screening programme. One way to evaluate this question would be a hybrid implementation effectiveness trial. There are at least two questions of interest based on different outcomes. Firstly, does implementation of the screening programme lead to those women identified as high risk having more specialist follow-up; and does this then lead to a reduction in preterm births. This could conceivably be extended to other mediating variables.

There is a dearth of literature relevant literature on how to handle multiple co-primary outcomes: much of what has been developed focuses on individual randomised trials (Romano et al JASA 2005; 100(469), not easily adaptable to CRTs, especially in the setting of a small number of clusters. Furthermore, the methodology in the area of multiple outcomes focuses on maintaining type-1 errors and does always clearly provide a one to one correspondence on how to report associated confidence intervals (Guilbaud O. Biometrical Journal 2008; 50(5): 678–692). Moreover, the literature has primarily focused on the setting where benefit need only be seen for one of the outcomes, which will in general not be appropriate here.

What the studentship will encompass

Objective: To establish a methodological framework for incorporating multiple outcomes into hybrid implementation and effectiveness trials.

Plans for project:

1.  Review of the literature and case study

a.   Review of hybrid implementation effectiveness trials to understand the number / type of outcomes commonly specified.

b.   Review of the statistical methodology for estimating effectiveness (and 95% CIs) of binary outcomes in CRTs whilst maintaining appropriate coverage for multiple correlated outcomes.

c.   Case study to illustrate the methodology to implement the identified methods.

2.  Simulation study to identify optimal analysis methods for trials with multiple correlated binary outcomes in cluster trials, ensuring appropriate coverage of confidence intervals, to identify the most powerful methods.

3.   Sample size and analysis tools to allow non-expert users to apply the methodology in key statistical packages (e.g., Stata, R).

PPI: The project will work with stakeholders to explore acceptability of recommended approaches.

Outputs: Provide guidance on how hybrid-implementation effectiveness trials should specify key outcomes at various levels, to ensure both nominal type 1 errors, statistically efficient use of data and models, whilst acknowledging the pragmatic need to consider impact at various levels. 


You are applying for a PhD studentship from the MRC TMRP DTP. A list of potential projects and the application form is available online at:

Please complete the form fully. Incomplete forms will not be considered. CVs will not be accepted for this scheme.

Please apply giving details for your first choice project. You can provide details of up to two other TMRP DTP projects you may be interested in at section B of the application form.

Before making an application, applicants should contact the project supervisor to find out more about the project and to discuss their interests in the research before 09 January 2023.

The deadline for applications is 4pm (GMT) 16 January 2023. Late applications will not be considered.

Completed application forms must be returned to: [Email Address Removed]

Informal enquiries may be made to Professor Karla Hemming - [Email Address Removed]

Funding Notes

Studentships are funded by the Medical Research Council (MRC) for 3 years. Funding will cover tuition fees at the UK rate only, a Research Training and Support Grant (RTSG) and stipend. We aim to support the most outstanding applicants from outside the UK and are able to offer a limited number of bursaries that will enable full studentships to be awarded to international applicants. These full studentships will only be awarded to exceptional quality candidates, due to the competitive nature of this scheme.
PhD saved successfully
View saved PhDs