UCL bibliometrics policy and the wider context
UCL is currently developing a policy on the responsible use of bibliometrics. The policy
will take into account a number of existing developments and best practices: the needs of
UCL authors, researchers and colleagues and their use of current
bibliometric tools; the San Francisco Declaration on Research Assessment (DORA); and the Leiden Manifesto for research metrics.
UCL was one of the first universities to sign DORA, which challenges the use of the Journal Impact Factor as a surrogate for the quality of individual research outputs. Along more general lines, the Leiden Manifesto identifies 10 principles to guide research evaluation.
UCL will use all these insights in the
development of an institutional policy on the use and management of
bibliometric approaches to research outputs.
It is recognised that bibliometrics are generally focussed on citation data from journal articles and may therefore be less relevant in disciplines that are less reliant on journal publishing, such as the arts, humanities, social sciences, computing science and engineering.
The wider context
- Examples of bibliometric analyses which include UCL
We have used Incites, which uses Web of Science data and is produced by Clarivate Analytics, to produce academic institution benchmarking figures looking at bibliometrics rankings of top worldwide insitutions to evaluate UCL's position. UCL ranks 4th in the number of publications. Further information.
Gunashekar S, Parks S, Calero-Medina C, Visser M, van Honk J, Wooding S. Bilbliometric analysis of highly cited publications of biomedical and health research in England, 2004–2013. Cambridge, UK: RAND Corporation, 2015
Report shows that UCL has the largest number of highly cited publications (HCPs) of any organisation in England. More information.
- Metrics standards
Initiatives to establish standards for methodologies and the use of metrics include:
- Snowball Metrics: Snowball Metrics is an international initiative for research-intensive universities from around the globe to agree on methodologies for metrics to enable confident comparisons.
- The Leiden Manifesto for research metrics presents ten principles for best practice in metrics-based research assessment.
- Bibliometrics and the REF (Research Excellence Framework)
Research outputs submitted for the REF2014 were assessed by expert review sub-panels, who could use citation data to inform their peer review judgements. Citation data were requested by around 30% of the sub-panels, mainly in the life or physical sciences areas, primarily to inform borderline cases.
An independent review, set up to investigate the roles that quantitative indicators can play in the assessment and management of research, was reported in "The Metric Tide", published by HEFCE in July 2015. The report concluded that although it is not feasible to assess the quality or impact of research outputs using quantitative indicators alone, the approach used in REF2014 of using quantitative data to complement peer/expert review should be continued and enhanced in future assessments.
The Stern report, Research Excellence Framework (REF) review: Building on success and learning from experience, published 28 July 2016 by the Department for Business, Energy & Industrial Strategy, is an independent review which makes recommendations on the future operation of the Research Excellence Framework (REF). It recommends the following in relation to the use of bibliometric data: "Panels should continue to assess on the basis of peer review. However, metrics should be provided to support panel members in their assessment, and panels should be transparent about their use."
- DORA (San Francisco Declaration on Research Assessment)
The San Francisco Declaration on Research Assessment (DORA) was initiated in 2012 by the American Society for Cell Biology (ASCB) and a group of editors and publishers of scholarly journals. The declaration recognises the need to improve the ways in which the outputs of scientific research are evaluated, including putting less reliance on the Journal Impact Factor as a measure. Individuals and organisations concerned about the appropriate assessment of scientific research are encouraged to sign the declaration. UCL is one of the first UK universities to have signed DORA.