Below you will find some guidance on how to use and interpret bibliometrics.
- General principles for the responsible use of bibliometric indicators
Whatever your purpose for using metrics, it is always important to consider the inherent limitations in citation indicators - it is easy to produce misleading figures without intending to.
In the link below, we highlight some common factors known to create biases in bibliometrics. You may encounter these during your analysis or interpretation, and where appropriate you should take steps to account for them.
- Recommended sources for bibliometric data
UCL has access to a number of tools for providing bibliometric and citation data. Each have their own strengths and weaknesses.
In the link below, we outline recommended sources for lists of publicaitons, citation counts and complex metrics.
- Recommended metrics
While no bibliometric indicator is perfect, some are better and more robust than others. In the link below, we describe some recommended metrics in more detail, provide a table of metrics and discuss their benefits and limitations .
- Metrics to avoid – the journal impact factor
What’s wrong with the journal impact factor?
UCL’s bibliometric policy places very few specific restrictions on metric use; instead it defers to the learned community’s judgement under a general framework that guides flexible metric use among disciplines and purposes. However, one key aspect is eliminating the use of the journal impact factor (JIF) as an indicator for article quality.
The PDF linked below outlines the problems that make JIF a poor indicator.
- Metrics to avoid – the h-index
The H-index in another commonly used metric that is potentially very misleading, and we do not recommend using it in most circumstances. The link below outlines how the H-index is typically misued.
- Responsible use of new and alternative metrics
In an age where data is being created and shared at scale, new and alternative metrics are continuously being developed to inform many aspects of the research system, including the reception, usage, and value of all types of research output. This creates a challenge to responsible metrics use initiatives, because it’s impossible to predict future metrics creation and misuse for specific metrics.
In the PDF below, we state how we rely on the general framework of UCLs bibliometrics policy and accompanying guidance to assist UCL personnel make informed decisions when considering new or alternative metrics.
- General statistical notes
Many issues with metrics use are often the result of overlooking fundamental statistical practices for interpretting numbers and statistics.
The following guidance applies some basic statistical principles to bibliometrics to help with responsbile use and interpretation.
- Metrics in REF
In the 2021 REF exercise, citation metrics will be used by some panels to support assessment in some units of assessment. Metrics can also play a role in impact and environemnt states. Formal guidance on this is provided by Research England.
Some additional guidance to help UCL personnel interpret the guidance from Research England can be found in the link below.
- Appraisals and Appointments
The UCL Academic Careers Framework explicitly states that UCL rejects the use of certain quantitative indicators. The following PDF provides some guidance on how to best use bibliometrics in the appointments and appraisals of staff.
- Compare and assess departments
The link below sets out some general advice on how to use citation metrics for assessing the output of research groups or departments, and guidance on where to find those metrics.
- Comparing UCL to other institutions
The link below sets out some general advice on how to use citation metrics for assessing the output of institutions, including comparisons to UCL.
- Metrics over time
One of the most common uses for metrics is for overall reporting and monitoring purposes – how many papers has a unit published over time? Can we tell if the rate of citations has changed?
The guidance below describes considerations when conducting analyses over time.
- Assessing the Arts, Humanities, and some Social Sciences
A central tenet of UCL's bibliometrics policy is to recognize the diversity of research areas, and to not impose metrics in field and contexts where they would be inappropriate.
Many disciplines within the Arts, Humanities, and Social Sciences produce outputs that are not as readily quanitfiable or citable as they can be in other fields. This makes bibliometric applications likely to be more problematic.
This paper sets out some general advice on appropriate ways to use citation metrics for publications in the arts, humanities, and some social sciences.
- Comparing journals
It is sometimes useful to compare a group of journals against each other, for example to help select a desirable publication venue, or to confirm that a journal is seen as reputable. A wide range of metrics are available which aggregate citations on a per-journal basis. The follwoing link outlines ways to compare journals.
- Dealing with multi-authored publications
In many fields, the majority of research is now authored by more than one person. However, most bibliometric analyses tend to assume equal contribution from all authors, when authors may have contributed different amounts of intellectual and technical/experimental work. This can result in metrics misestimating the productivity of researchers in high-collaboration fields, especially when compared to researchers in lower-collaboration fields.
The following guidance outlines appraoches to assign credit or weight to multi-authored publications.