Close
Simon Knowles, UCL’s Head of Coordination (SDGs), describes how the UCL SDGs Initiative is improving the accuracy and extent of measuring SDG-related activity at UCL.
2021–22 is the second academic year that UCL has reported on the extent of SDGs-related activity across the university.
Since last year, we have improved the accuracy of how we are measuring our SDGs-related teaching and research. We have also started to measure the extent of our students’ extra-curricular activity, as well as aspects of our external partnerships that are addressing the SDGs.
We are aware some of our measures could be improved: we set out some of the caveats to our methodologies below. We continue to refine how we measure SDGs-related activity by learning from others in the sector and beyond. We will also identify further metrics to illustrate how our community is helping to achieve the Goals.
SDGs-related teaching activity at UCL
For the second year, we classified the descriptions of the 6,797 taught modules in UCL’s online module catalogue by Goal using OSDG, a multilingual open-access tool jointly developed by the UN Development Programme SDG AI Lab and research and policy analysis centre PPMI.
Combining several existing sets of SDG categories and augmenting them with additional keywords, OSDG compiled a set of SDG-relevant terminology. The list of original sources is available on OSDG’s website. For last year’s report, OSDG searched for keywords in the module descriptions and attributed an SDG to them if the descriptions contained two or more keywords for that SDG.
OSDG has further improved its methodology since then, by integrating machine learning models that predict the preliminary SDG labels, for them to be verified through the ontology/keyword matching. The updated approach is now able to identify modules that may have been missed previously, so the results should be more accurate than last year’s report.
The methodology relies on module leads using SDG keywords, which many may not have done – they were unaware their description would be mapped – so the number identified is likely to be an underestimate.
Similarly, longer module descriptions have higher chances of being identified as relevant to an SDG simply because they tend to use a wider vocabulary. Figures 1 and 2 (page 9–10 of the report) show the numbers of SDG-related modules by UCL faculty and by SDG.
SDGs-related research activity at UCL
Research publications
This year, to broaden our search for SDGs-related research, we mapped UCL research activity related to each SDG by combining the publications found in Elsevier’s Scopus database (using keyword searches) and in Clarivate’s Web of Science database (as categorised by their InCites tool); last year we only used Scopus. Elsevier also improved the accuracy of their keyword searches this year.
To be included, a paper had to be present in the Scopus or Web of Science databases, with any duplicates removed from the total. We then attributed each paper to UCL faculties by matching to UCL’s internal publications database by DOIs.
A paper was counted once per faculty (even if it had multiple authors within a faculty), but could be counted in more than one faculty if it has co-authors in multiple faculties. It could be counted in more than one SDG if it is given multiple SDG classifications by Scopus and InCites. Figures 3 and 5 (pages 12 and 16 of the report) show the number of SDGs-related publications by UCL faculty and by SDG.
SDGs-related research impact case studies
For the first time, we also used OSDG to classify the impact case studies we submitted to the UK Research Assessment Framework (REF), see Figure 4 (page 15 of the report).
Top 10% most cited and international research collaborations
The lists of publications for each SDG (combined from the Scopus and Web of Science databases) were imported as custom datasets into InCites for citation and collaboration analyses.
The percentage of UCL publications in the top 10% most cited for all research of similar papers was calculated by comparing citations with ‘similar papers’, referring to similar Web of Science subject categories, years, and document types (e.g. articles and reviews).
International research collaborations were measured by the percentage of publications with at least one co-author from a country outside the UK.
Policy citations
Policy citations were sourced from Overton, an index of policy documents, guidelines, think tank publications and working papers, which collects data from more than 1,000 sources worldwide.
Student extra-curricular activity addressing the SDGs
For the first time, we mapped how many of our students were engaged in activity outside of their course which was contributing to the SDGs. We surveyed student societies asking them to state which of the SDGs their activity was supporting. For those societies that didn’t respond, the UCL SDGs Initiative made that judgement on their behalf, based on information provided by the societies.
The Students’ Union UCL Volunteering Service did the same for the student-led volunteering projects. We used the number of students who were members of those societies and projects that were judged to be supporting an SDG as an indicator of the extent of student involvement in each Goal.
The figures rely on the subjective judgement of staff and students at UCL. The numbers also reflect particular a moment in the year, while membership numbers fluctuate during the academic year, and students may be members of more than one society or project. Figures 6 and 7 (pages 18–20 of the report) shows the numbers of students involved in addressing the SDGs through their student societies and volunteering projects.
External partnerships activity addressing the SDGs
For the first time, we also mapped the extent to which two types of external partnerships were supporting the SDGs. With UCL Innovation & Enterprise we measured how many of the student startups supported by the UCL Entrepreneurship Hub were addressing each of the SDGs (see Figure 8, page 29 of the report). Each startup was asked to identify which (if any) of the SDGs they thought they were addressing.
Similarly, working with the UCL Office of the Vice-President (Advancement), we measured solicited philanthropic income received by UCL in support of specific SDGs (see Figure 9, page 30 of the report). OVPA assigned each gift to one ‘lead’ SDG to avoid duplication and double counting.
