XClose

Sustainable Development Goals

Home
Menu

How we measured UCL’s SDG-related activity

Simon Knowles, UCL’s Head of Coordination (SDGs) describes how the UCL SDGs Initiative measured SDG-related activity at UCL for the first time.

UCL is helping to address the SDGs through all areas of its activity, from our research and teaching to the activities undertaken by our students in their spare time and the way we operate as an institution.

2020–21 is the first year that UCL has published a report on the extent of SDGs-related activity across the university. Like many other institutions around the world, we are still exploring how we can best measure the different types of activity that are supporting the Goals, but we wanted to report on what we could measure.

This year, the first year of the UCL SDGs Initiative, we focused on measuring our research and teaching activity that is supporting the SDGs. We are aware neither are perfect and we set out some of the caveats to our methodologies below. We will refine our approach in future years.

Going forwards, we will also seek to measure our student, operational and external engagement activity that is supporting the SDGs in the longer term.

SDGs-related teaching at UCL

We classified the module descriptions of the 6,113 taught modules in UCL’s online catalogue by SDG using OSDG, an open access tool jointly developed in partnership by the UN SDG AI lab and research and policy analysis centre PPMI.

Combining several existing sets of SDG categories and augmenting them with additional keywords, OSDG compiled a set of SDG-relevant terminology. The list of original sources is available on the OSDG website.

OSDG searched for keywords in the module descriptions and attributed an SDG to them if the descriptions contained two or more keywords for that SDG. The results are shown in Figure 1.

The methodology relied on module leads using SDG keywords, which many may not have done – they were unaware their description would be mapped – so the number identified is likely to be an underestimate.  It also relies on a particular set of key words: as we discovered when mapping research activity (see Figure 1, below) very different results are generated by different sets of keywords. Finally, longer module descriptions have higher chances of being identified as relevant simply because they tend to utilise a larger vocabulary.

Figure 1

an image of a graph showing teaching module by SDGs at UCL

SDGs-related research at UCL

Like many other universities, we mapped the extent of UCL research activity related to each SDG by analysing the number of UCL-affiliated research publications matched to two sets of SDG-related keywords.

One set of SDG keywords was developed and validated by Elsevier. These are used in its SciVal product and by Times Higher Education (THE) to generate research publication lists for its Impact Rankings.

A different set of SDG keywords has been compiled by the Sustainable Development Solutions Network (SDSN). 

Both sets have limitations. The Elsevier keywords are conservative, generating relatively small publications lists with high confidence of SDG relevance. It may miss publications related to an SDG that do not match the specific keywords. The SDSN keywords are less restrictive, and so are likely to result in more false positives (matches to SDGs for papers that are not directly related to that SDG). For example, the SDSN keywords for SDG1 include ‘Class’, which returns research papers on class-based economic systems (as would be relevant for SDG1). But it also returns papers where class is used to classify objects or processes across many areas of research including physics, biology and many other contexts unrelated to SDG1 or poverty. For the purposes of this report we used the Elsevier keywords, in line with THE.

To be included, a paper had to be present in Scopus. We then categorised them by UCL faculty using UCL’s internal publications database using DOIs. A paper was counted once per faculty (even if it had multiple authors within a faculty). Figure 2, below, shows the numbers of SDG-related publications across UCL by SDG.

Figure 2

an image of a graph showing publications at UCL by SDG

Top 10% most cited and international research collaborations

The percentage of UCL publications in the top 10% most cited for all research of similar papers was calculated by comparing citations, with ‘similar papers’ referring to similar Web of Science subject categories, years, and document types (e.g., articles, reviews, etc). International research collaborations were measured by the percentage of publications with at least one co-author from a country outside the UK.

In both cases, publications were sourced by searching Scopus with Elsevier’s SDG search strings; these publications were then imported into Clarivate’s InCites to generate the citation metrics. About 12% of the Scopus publications were not included in the InCites analysis because InCites covers different source material.

Policy citations

Policy citations were sourced from Overton, an index of policy documents, guidelines, think tank publications and working papers, which collect data from more than 1,000 sources worldwide.