Vice-Provost's View: A game-changing REF result
8 January 2015
Being of a generally sunny disposition, I casually stated in a December 2013 Vice-Provost's View: "I've no doubt that our university's research strengths will be properly recognised in a year's time." (I was to be pleasantly surprised.)
We had just made UCL's submission to the Research Excellence Framework (REF) 2014. It was the culmination of what I estimate to have been some 50 to 75 person-years of effort by UCL researchers and their professional services colleagues - and, indeed, drawing on UCL research conducted (and professionally supported) over several decades.
In retrospect, my 2013 article was somewhat over-optimistically entitled 'Beyond the Research Excellence Framework'. The matter was not done and dusted: for one thing, our excellent UCL REF Team, led by Andrew Cooper, was required to respond to HEFCE's audit process (we survived the scrutiny); for another, many of our academic colleagues (myself included) served on assessment panels and sub-panels throughout most of 2014.
Although I am now tempted to say that the release of the REF2014 results on 18 December 2014 marked the end of the process, REF2014 will resonate for at least the next six years - both in terms of our reputation as a leading research-intensive university and in terms of the Quality Related (QR) funding it will direct to UCL. (QR, as informed by the 2008 Research Assessment Exercise [RAE], was worth nearly £80 million to UCL last year.)
To have not only rivalled but overtaken Oxford and Cambridge is an outstanding achievement for UCL
It is sometimes suggested by my colleagues that the degree of my enthusiasm for research funding is such that it is possible for others to mistakenly conclude that I regard research income as an end in itself. For the avoidance of doubt: QR is important because it enables us to invest in the best people and facilities, to work at the cutting edge of disciplines (and, indeed, to forge new disciplines), to provide an environment in which early career researchers can thrive, to tackle the toughest questions facing humanity, to develop insights into the most profound issues, and to help to address the world's problems.
Research income enables research, but it is the outputs and resulting impacts that count!
What REF demonstrated
UCL's results exceeded our hopes and expectations. We were on the lookout for a strong showing on what we call 'research power': the overall quality of our submission (the 'grade point average' [GPA]) multiplied by the full-time equivalent (FTE) number of researchers submitted.
We considered challenging Cambridge on 'research power' to be plausible. However, as indicated in the UCL press release, we overtook both Oxford and Cambridge, reaching the top position (as demonstrated in the graphic and table below).
Our 'research power' was rated top - not only in the overall results, but in each of the three assessed elements of REF2014: publications and other outputs; research environment; and research impact. This is a simply stunning achievement and reflects the passion and commitment of our researchers and those who support them.
There are, of course, a number of other ways to calculate 'research power', such as varying the weightings of research rated 4* ("world-leading") and 3* ("internationally excellent"), including:
- % of 4* research x FTE, in which UCL places second - see the top seven HEIs by this measure
- reflecting the current HEFCE formula for QR, (% of 4* research x FTE x 3) + (% of 3* research x FTE), in which UCL places second - see the top seven HEIs by this measure.
If HEFCE maintains the current or a similar QR funding formula, there should be considerable financial benefit to UCL, although we acknowledge that HEFCE may, in future, adjust the formula it uses. If our funding increases as expected, that will contribute significantly to helping us to achieve both our strategic and financial aims.
Our notably inclusive approach reflects the view of the UCL REF Strategy Committee (in accordance with Excellence and the UCL Community and the UCL Research Strategy) that it would expect most units within a research-intensive university like ours to have a submission rate of at least 90% (with some reasonable exceptions). In particular, UCL had a strong commitment to including early career researchers. Our inclusive approach has resulted in our high ranking in 'intensity-weighted GPA'.
Research Fortnight has calculated 'market share', that is the amount of QR funding that REF2014 results would attract under HEFCE's current formula (were all subject areas funded equally, which they are not). On this basis, UCL has 6.07% of market share, second only to Oxford's 6.24% - see the top seven HEIs by this measure.
'Impact' was, of course, a new element in this exercise. The results indicate that UCL has the greatest 'impact power' (impact GPA x FTE) - see the top seven HEIs by this measure.
Main Panel results
Analysis of our results within REF2014 Main Panels (broad areas of research) indicates the following.
This is a simply stunning achievement and reflects the passion and commitment of our researchers and those who support them
Panel A (medicine and biological sciences)
In Panel A, we produce 10.5% of all 4* research. Oxford, in second, produces about 6.5%. Although there are some complexities in comparing this measure with RAE 2008, by our best figures we believe Oxford had a similar share at that time, while UCL had roughly 7.3%, so the improvement in our sector-leading position is spectacular.
Panel B (physical sciences, maths and engineering)
In Panel B we produce roughly 4.9% of all 4* research, fifth behind Cambridge, Oxford, Imperial and Southampton, and a significant improvement on the 2008 RAE where we had roughly 3.9%.
Within Panel B, in the Physical and Mathematical Science Units of Assessment (UoAs), we achieved 4.5% market share of 4* research, up from roughly 4.2% in the 2008 RAE, and in Engineering Science UoAs 5.3%, a significant improvement on the 3.7% in the 2008 RAE.
Panel A and B combined
Our combined performance in Panels A and B is such that we have, by a clear margin, the largest volume of 4* research in medicine, mathematics, physical and life sciences, and engineering - 13% ahead of Oxford, more than 20% ahead of Cambridge and 39% ahead of Imperial.
Panel C (social sciences)
In Panel C we have both the greatest 'research power' and share of 4* research. We produce roughly 7.5% of all 4* research, ahead of Oxford (roughly 7.0%) and LSE (roughly 5.6%). This highlights that the merger of Institute of Education and UCL has made us the leading institution in UK social sciences.
Panel D (arts and humanities)
In Panel D we produce roughly 3.1% of all 4* research, fifth behind Oxford, Cambridge, Edinburgh and King's, consolidating our overall position from the 2008 RAE, which was noteworthy because across the sector the spread of 4* research considerably increased and many leading institutions saw their share significantly reduced. Our GPAs, particularly when adjusted for staff inclusion, were strong across this panel, confirming our distinctive combination of breadth and excellence within an activity significantly smaller than those at Oxbridge.
I will not, in this space, probe deeper into individual REF2014 UoA results - although my office is undertaking detailed analysis at this level - other than to congratulate those UCL UoAs ranked in the top three by a variety of measures.
Those who have studied UCL 2034, our university's 20-year strategy, will have noticed that one of our objectives for the Principal Theme 'Academic leadership grounded in intellectual excellence' is "to be a leading performer, within the top two, in UK research income and Research Excellence Framework performance". It is pleasing to have achieved that stretching goal 20 years ahead of schedule, but we will of course now need to direct our efforts to ensure that we sustain this performance for the next two decades and beyond.
Research income enables research, but it is the outputs and resulting impacts that count!
Some colleagues jokingly asked me the day after the results came out whether I had started work on the next exercise yet. Actually, we've all been working on it for many years already, since it is likely publications from January 2014 on will count! We've also spent the last couple of years putting in place the key enabling infrastructure that will support the submission, and engaging with early consultations from HEFCE and others on how it might work. Several key issues are already apparent.
Meeting HEFCE's expectations on Open Access - with deadlines for eligibility linked to article acceptance, not publication - will be critical. If you haven't already familiarised yourself with the new requirements and UCL's support for doing so, then read UCL's guidance on Open Access for the next REF. These requirements are not only different from those used by other funders, but also put a particular responsibility on the submitting author to ensure they are met. There's a real possibility that thousands of excellent research outputs across the country will be ineligible for the next REF because their authors were not aware of these requirements or failed to act on them - we need all our researchers to make sure that they don't fall into this trap.
A new element in REF2014, impact is unlikely to go away and may in future carry a weight greater than the 20% of the overall result it counted for this time. We are developing plans to support, record and publicise the impact of UCL research and have evolved our team of REF editorial consultants into the Research Impact Curation & Support (RICS) team.
In collaboration with departments and faculties, the RICS team has begun to identify a portfolio of potential case studies for the next exercise ('REF2020'). We believe that achieving the best results next time will require ongoing monitoring of and investment in impact activities. The information being compiled by the RICS team will have many other uses beyond REF, such as impressing potential donors, planning our impact support activities and publicising our impact globally. Please contact the RICS team to find out how they can help you to maximise, record and evaluate the impacts of your research.
Please do browse the RICS team's new portfolio of examples of impact; at the moment it consists of case studies submitted to REF2014 (disentangled from the Byzantine format required for submission), but it will grow over time to reflect the breadth of our research impact.
Although we don't expect metrics to replace peer review in the next REF, we do expect they will play a larger part, and that we will require ever more data on publications, personnel, students, impact, income and expenditure. We are working to ensure that the data processes across this range of different issues are capable of addressing future REF requirements that are yet to be defined; this is a challenge, particularly when all the relevant systems are likely to be updated between now and the submission.
Beyond these matters, we have of course the usual (and joyful) task of helping the UCL community to continue producing original, significant and rigorous research, and bringing that research to bear on the world.
To have not only rivalled but overtaken Oxford and Cambridge - "two public nuisances" and "storehouses and nurseries of political corruption", according to our spiritual founder, Jeremy Bentham - is an outstanding development for UCL.
We can celebrate major achievements in every faculty. My congratulations and thanks to everyone who played a part. The efforts you made, and the consequent results, will reshape how the world sees us. These results should encourage those, like you, who seek to help London's Global University to fulfil the vision of its founders.
Professor David Price, Vice-Provost (Research)