Listening to students’ perspectives on generative AI

Dr Matteo Tiratelli Lecturer (Teaching) at UCL Social Research Institute led a project to explore students’ opinions on generative AI. The results are fascinating.

Students’ views were gathered through focus groups and surveys and student leads Mia Meade and Ruth Ogundamisi tested the AI tools. The project was funded by UCL Changemakers. 

---

Patchy knowledge

Of the students consulted, about 60% believed that the use of AI tools, and ChatGPT in particular, was already widespread amongst students (“everyone is using it”). However, of those who disagreed, many thought that no one was using it. If there are pockets of high-use and pockets of low-use among our students, then this may have serious equality implications.

Students are excited about the possibilities of these tools but expressed that they didn’t know much about them. While students felt like they knew how to use them to cheat, they lacked positive examples for how these tools could improve their learning experience.

---

Homogenisation and decoloniality

Respondents discussed the ethics of using these tools for their studies and their awareness of tools’ illusion of objectivity. 

Students raised concerns surrounding the training datasets used for most Large Language Models. For instance, ChatGPT relies on sources that are primarily in English, which has potential to perpetuate an “Anglocentric hegemony” and ignore “other cultural perspectives”. Similarly, respondents recognised that as an aggregate or popular average, there is a risk it will reinforce existing prejudices and patterns of thought.

---

Deskilling and dependency

In particular students were concerned that reliance on ChatGPT might inhibit the development of skills such as: writing fluent English prose, understanding complex texts, developing a coherent argument, essay planning and structure.

On the other hand, many students felt that the extent to which independent thinking and creativity would be impacted in the short term has been overstated.

---

AI as a research assistant

Broadly students felt that UCL’s guidance on the use of ChatGPT was already out of date. Most students did not believe that their peers would cite the tool, either out of fear that it would impede on their marks or because they saw it like Google or any other internet tool.

While some students also raised concerns that academic integrity could be undermined by the tool, others likened it to a research assistant. If the tool is used to organise thought rather than generate it, they saw nothing wrong with its use. 

Students did however acknowledge the difficulties in discerning “good” from “bad” uses of these tools and were concerned that the opaque nature of ChatGPT makes it difficult to evaluate its sources.

---

Students’ suggestions

1.  Students were much more concerned about the equality implications than about academic integrity and plagiarism (e.g. more “techy” students would have an unfair advantage etc).
2. Classes on “how to use AI” including: what tools are available; how to use them for learning, not just assessments; how to use them effectively; the tools’ strengths and limits.
3. Educators should integrate these tools into their teaching and model “good use-cases”.
4. This is a good opportunity to rethink assessments more generally. Students emphasised the difference between “assessment-for-results” and “assessment-as-learning-opportunity” and the need to keep both in mind. They also focussed on the marking rubrics on the student's voice, flair, argument, critical thinking, and originality, rather than just summarising theories. They felt that the former represented something “distinctive” that the student can offer beyond an AI tool.
5.  Alternative assessments including: open essay questions (e.g. pick three theories from this course and use them to reflect on a contemporary event); “portfolio” assessments (e.g. 10% of grade is for essay plan, 15% for presentation of initial research, 75% for final essay); multi-mode assessment (e.g. 20% individual presentation, 30% group project, 50% final essay).

---

Recommendations

Clarification and consistency on rules

Students were unclear about what the rules were and many had received different messages from different members of staff. At present UCL guidance is that ChatGPT can be used in assessments but it must be cited. 

However, it is worth noting that respondents believed that most students would simply ignore this because: 

• it was assumed that staff would react negatively to seeing that ChatGPT had been cited
• there would be no way of checking
• they didn’t think it made sense to cite something which is not producing data or opinions of its own.

Educate students (and staff) about what it can and can’t do

It is concerning that there was almost universal agreement from students that ChatGPT could be used as a “more specific Google” or a “personal tutor”; in short, as an information retrieval service.

However, ChatGPT is a language model and has no concept of the validity of the truth claims it makes. 

Update guidance regularly

These technologies are improving all the time. We therefore need to commit resources to ongoing research and updating.

Open dialogue between staff and students

We suggest organising a staff-student “town hall” in early October to discuss what we should do next academic year, and set expectations for first years. 

We also recommend creating a committee of staff and students who meet regularly to discuss these themes.