Using Research Excellence Framework (REF) Results for PhD Study

The Research Excellence Framework, or ‘REF’, is the most detailed and extensive assessment of university research in the UK. It evaluates research performance across 36 different subject areas at each UK university. The results of the REF are used to determine the proportion of public funding allocated to individual universities for research.

For prospective PhD research students, the REF 2014 can be a very useful resource… provided you know what to look for.

That’s why we’ve created a convenient and accessible breakdown of the REF 2014 results. You can use it to view REF 2014 results by university, or by subject area. Whichever you choose we’ve provided helpful information to help you make sense of the results and understand how different aspects relate to the issues that matter most to prospective PhD students.

If you’d like to just get started looking at REF 2014 scores for your chosen universities and subject areas, you can click here. If you’d like to know more about the REF you can read on for a list of PhD student Ref 2014 FAQs.

Have questions about the REF 2014 results? What it is, how it works and why it matters to you as a PhD student? No problem. We’ve put together a list of FAQs about the REF from the point of view of prospective postgraduate research students. Even better, we’ve also included some answers!

First thing’s first then, what is the REF for?

The REF is used to determine how much public money is allocated to each UK university in order to fund their research (the funding awarded based on REF performance actually accounts for the largest proportion of research support received by UK universities).

Does the REF do anything else?

In addition to determining funding allocations, the REF process ensures universities are accountable for the public investment that supports their research. It also provides a means of benchmarking university research performance for other users… like prospective research students!

Is the REF is a bit like a university rankings table then?

Not as such. The REF does ‘rank’ university research according to its ability to meet given quality levels, but these are standards specified by the REF, not a direct comparison between different universities and their departments. So the REF itself isn't a PhD ranking.

So how can postgraduates use the REF?

In some ways, the REF result is actually better than a simple university league table. It assesses the specific departments that conduct research and allows you to ‘zoom in’ on the criteria that’s most important to you as a prospective research student: research. What’s more, the REF provides a detailed breakdown of different aspects of university research, including the quality of academic publications, the positive effect of research in wider society and – most importantly for PhD students – the standard of the departmental units and structures in which research is actually produced, including systems for enabling and supporting successful PhD projects!

OK, who organises the REF?

The REF is administered by different bodies responsible for regulating higher education across the UK: Research England, the Scottish Funding Council, the Higher Education Funding Council for Wales (HEFCW) and the government's Department for Employment and Learning. These are the groups that oversee the distribution of public research funding across the UK.

Is REF funding the same thing as Research Council funding then?

No – the four UK higher education Funding Councils are different entities to the seven UK Research Councils and REF funding is different to Research Council funding. Research Councils fund specific projects (including postgraduate research projects) based on their individual merits. Funding Councils provide ‘block grants’ to support departments within institutions based on their REF performance. This combination of Funding Council and Research Council funding is known as the ‘dual support system’.

What research areas and academic subjects does the REF cover?

All of them! Whatever research topic you’re interested in studying for your PhD and whichever university department you’re looking to do your research in, you’ll be able to access an appropriate REF result. This is because the REF organises university research into 36 different broad subject areas, referred to as ‘units of assessment’.

What is a unit of assessment?

As above, a unit of assessment is really just a category of related subject areas. This allows universities to organise their research units and courses as they see fit, whilst still making sense in the REF system. In practice university researchers submit work to the REF using the most relevant unit of assessment.

How is the REF actually undertaken?

There are three main stages to the REF process: the submission of research, its assessment by expert panels and the publication of results.

What happens at the submission stage?

Universities select the best examples of their research and submit them under the units of assessment that are most appropriate to their work.

What happens at the assessment stage?

Assessment of research for the REF is undertaken by expert panels. There are two types of expert panel. Most are ‘sub-panels’ assigned to each of the 36 units of assessment. In addition, there are also four main panels, responsible for overseeing the broader implementation of specific REF assessment criteria.

OK, so what are the REF assessment criteria?

Assessment criteria are one of the most important features of the REF. Research submissions are evaluated according to three specific criteria: Output, Impact and Environment. These are then combined to provide an Overall result for each department’s REF score.

What is Output?

Output is the simplest of the three assessment criteria. It measures the quality of academic work produced by a university’s researchers. Up to four research Outputs can be nominated for each academic whose research a university submits to the REF.

What are some examples of Output?

Examples of Output include publications like journal articles and book-length studies, as well as other fruits of academic research such as important data sets, new technologies and intellectual property.

Why does Output matter to PhD students?

A university’s Output score for a specific subject area can tell you how successful its academics are at generating high-quality publications. This might offer an indicator of the potential for you to take part in cutting-edge research projects (and the publications they can generate). A high Output score may also mean that your PhD will be supervised by academics who are recognised leaders in their fields.

What is Impact?

Impact is a new assessment criteria introduced for the REF 2014. It assesses the positive effects of a university’s research beyond the academy. Impact is assessed using submitted case studies that demonstrate the past effects of a university’s research as well as strategies for ensuring present and future impact.

What are some examples of Impact?

The REF defines impact as consisting of ‘any effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment, or quality of life, beyond academia’. It’s worth noting that Impact applies to all academic disciplines and subject areas. For example, medical science research might generate important changes to public health policy, whilst research in an arts and humanities subject area might have an impact on educational outreach and or underpin important exhibitions in public libraries and galleries.

Why does Impact matter to PhD students?

A university’s past success in demonstrating the impact of its research can indicate the opportunity for PhD students to take part in high profile projects and activities outside the university and to do work that is of an immediate and appreciable benefit to society as a whole. As well as being immensely rewarding, this will look excellent on your CV, whatever career path you pursue with your PhD.

What is Environment?

Environment is arguably the most important REF assessment criteria from the point of view of prospective research students. It measures the quality of the departments, academic units and research groups in which a university’s research is produced – the ‘environment’ in which you will work as a PhD student.

How is Environment assessed?

The quality of a university’s research Environment is assessed based on a range of factors. Of particular importance is evidence demonstrating the ‘sustainability’ and ‘vitality’ of research environments. This can take account of the continuity of research funding as well as the structures in place for the effective support, supervision and training of PhD students.

Why does Environment matter to PhD students?

Of all the REF assessment criteria, Environment is the most directly relevant to prospective PhD students. A university department with a high Environment score will be effectively-organised and sustainably-funded. It is also likely to have a proven track record of supporting postgraduate research projects and good strategies in place to ensure a positive and successful experience for future PhD students… like you!

How are the different assessment criteria used to produce an overall REF result?

The three individual assessment criteria are individually weighted and combined to produce an overall REF result for each unit of assessment. Output is worth 65% of the overall score, Impact is worth 20% and Environment is worth 15%.

OK, back to those expert panels then - who is on them?

Members of the expert panels that assess REF submissions are recruited and appointed at an early stage of the REF process. There are a lot of them, but they fall into two general types: Practising Researchers and Research Users. Practising Researchers are usually other academics working in a field appropriate to their assigned unit of assessment. They provide a form of peer-review similar to that used for academic publications. Research Users are selected from the audience research in a particular subject area is deemed to be of value to. They may also be academics using research data, or they may be representatives of industry, business or policy groups whose work draws on university research.

Right, so the different research submissions are examined by expert panels for each unit of assessment and assessment criteria – what happens next?

Once the process of expert panel review is complete (a process that takes around a year) the panels publish their results! They do so in the form of things called a ‘Quality Profiles’.

What’s a ‘quality profile’?

A quality profile is the term the REF uses to describe its presentation of the combined result for REF submissions in each unit of assessment. Each item in a submission is ranked according to its quality. The quality profile for each set of submissions then gives the proportion of its research that meets each ranking level. The submissions for each unit of assessment are actually given four quality profiles: one for Output, one for Impact, one for Environment and one for an Overall result. The exact descriptions of standards are tailored to different assessment criteria, but all are ranked from 1* to 4*.

That sounds a little complicated...

Well, it all depends on how the information is presented. We’ve designed our guide to the REF 2014 results to make them as clear and accessible as possible. Results for each subject area at each university are presented using an incredibly sophisticated and advanced graphical system (OK, it’s a pie chart). Why not go take a look?

Thanks – I will!

You’re welcome. You’ll find lots of information in our guide to help you get the most out of the REF 2014 results, but, if you forget what a ‘unit of assessment’ is, or the difference between a ‘quality profile’ and an ‘assessment criteria’, feel free to come back here.