Using AI tools for the reading-into-writing process

Peter Puxon, Jessica Brook and Ayanna Prevatt-Goldstein from the UCL Academic Communication Centre worked with three students to reflect on writing an assignment with the assistance of AI tools.

Writing is still a primary way we assess learning at UCL. However, writing is also a way of developing understanding and expertise in a subject area: we write to learn, not just to demonstrate learning.

Use of generative AI for parts of the writing process has recently been deemed acceptable in some university policies, including at UCL, but how do students feel about using these tools? And how can students’ experiences help staff to talk to students about using AI tools appropriately for the writing process?  

Aims of the project 

As part of a Changemakers Co-Creators AI project in summer 2023, we recruited three UCL students from different discipline areas (BSc Bioscience, BSc Linguistics, MSc International Planning) to explore what was gained and what was lost in the reading-into-writing process when using AI tools as reading and writing assistants.

We wanted to investigate:

• how useful AI tools were
• the impact of AI tools on the learning process
• students’ feelings about AI tools throughout

These findings and recommendations can support educators in discussing the appropriate use of AI in student written assignments, in the context of their discipline and learning objectives, alongside the toolkit on using generative AI in teaching and learning. 

---

Our process

Students first described their usual reading-into-writing process (i.e. without use of AI tools) and reflected on what they find challenging and what they find enjoyable. We broke down the writing process into stages to give students a common vocabulary to describe their process.

The writing process in three stages

• Reading, pre-writing and planning: reading; generating ideas, understanding the ideas of others, collecting information (note-taking, freewriting, brainstorming); organising and focusing ideas (mind mapping, clustering, listing, outline)
• Drafting and review: writing versions of a text focusing mainly on the development, organisation and elaboration of ideas
• Editing and proofreading: focusing attention on the surface-level features of the text

This not a linear process, and as we write we might go back and forth between these different stages.

(These stages are taken from a diagram in Curry, M.J. & Hewings, A. (2003). Approaches to teaching writing. In: Coffin, C., Curry, M.J., Goodman, S., Hewings, A., Lillis, T.M. & Swann, J. (eds.) Teaching Academic Writing: A Toolkit for Higher Education. Routledge, pp. 19-44. See original diagram on p.34)

Students selected an assignment set by their programme, which they had not completed previously and for which the deadline had already passed. They reflected on the intended learning outcomes of the task, why they thought it had been set, and what they might hope to learn through completing the assignment. 

Students chose from a range of free and paid-for AI tools (ChatPDF, Humata, SciSpace, ChatGPT, Quillbot, Jenni.ai, Writesonic, GrammarlyGo, Claude.ai) to assist in writing the assignment. They had three weeks to complete their assignments, recording their thoughts and reflections on the process in a daily diary.

On completing the assignments, the students were interviewed to find out how their non-AI and AI-assisted writing processes differed, what they felt they had gained and lost in using these tools and how they now felt about using AI tools. Some common themes and key recommendations for staff and students emerged from these discussions.

---

Findings

Students' feelings about AI tools

Annoying, ambivalent, disappointed

When asked to sum up their overall feelings towards using AI tools for the reading-into-writing process, the students chose annoying, ambivalent, and disappointed.

Fundamentally, they thought that these tools negatively affected their university learning process: 

“AI won’t give you that depth that we require for a first. It won’t give you that knowledge that you require in your later degrees and professional life. The student might ask why am I at university? Do I really want to learn? (Student 1)

[AI tools] don't understand the purpose of the assignment… which is not to write a perfect essay, but to learn and think independently. (Student 2)

How do I use these tools without outsourcing the entire project to them? How do I use them effectively, while maintaining my own voice? Am I getting dumber whilst using them? (Student 3)

The students felt that reliance on AI tools led to superficial learning and missed learning opportunities. They reflected that tutors had set the various assignments in order to develop skills and deepen understanding, so bypassing these processes missed the point. 

Time spent and distrust in the output

The students also found the process of selecting and familiarising themselves with the various tools time consuming, as was developing the skill of appropriate prompt writing. Although students recognized that the tools could save time, particularly in the early stages of the reading-into-writing process, these benefits were in part negated by a lack of trust in the tools' output.

Texts often had to be read in full to check AI output for accuracy as the tools could not 'understand' the texts in relation to the task set with sufficient depth and nuance. The students also felt that these tools could not represent a writer's argument or view accurately enough to be relied upon. 

Prompting a new perspective 

One student had a more positive view of AI tools in terms of the broader applications for university learning, including outside of assessment. The student felt cautiously optimistic about the possible role of AI in tailoring learning for individual students, improving engagement and accessibility for neurodiverse groups by disrupting traditional modes of instruction.

“There's potential to use it elsewhere, in terms of how I learn or to suggest different ways in which I can approach this task. I would be more keen on using it for those things than something which is strictly academic. (Student 3)

---

AI for the reading-into-writing stages

Reading, pre-writing and planning

Students recognized that the tools could save time in the reading, pre-writing and planning stages. They found the tools useful for:

• summarizing papers
• questioning a text
• locating information within a text
• overcoming writer's block
• suggesting a possible outline as a starting point.

However, the negatives at this early stage outweighed the positives: 

“By cutting down the meaningful time that you directly engage with your readings, it's impossible to get to the level of understanding that you would get to without using these systems... with traditional reading you'll be led to unexpected places and that's one of the main joys of learning... (Student 1)

Reading the original articles in full can serve as a model for my own writing, but these AI tools discourage this practice… I lost the ability to understand the article as a whole. (Student 2)

I would want to engage with that literature and come up with my own conclusions or observations. (Student 3)

The students recognized that:

• Reading widely and deeply for an assignment was necessary for acquiring sufficient subject knowledge, as well as exposing them to useful models of texts and raising their awareness of genre.
• Using these tools became a substitute for their own reading process, shifting their role from that of active reader to editor and fact-checker.
• There is a paradoxical need for a base level of subject knowledge in order to use the tools (create relevant and appropriate prompts; evaluate the AI-generated output) to complete their assignments, knowledge which could only be gained through their own reading.
• Using these tools means they missed an opportunity to develop key skills of criticality and autonomy. 

---

Drafting and reviewing

Students identified some benefits at the drafting stage. AI-generated text provided a grammatically accurate and fluent text which could be used as a model:

“The AI tools also produced ‘perfect writing in pieces’ i.e., no grammar mistakes and were useful at summarising and paraphrasing. After using AI, I understood what the shortages of AI were, but I also understood which parts of the reading into writing process that I had shortages in myself. AI writing tools also helped me become aware of my grammar mistakes. (Student 2)

Prompting AI tools to express an argument in different ways could serve as inspiration for variation in writing style. AI tools also made helpful suggestions for summary and paraphrase. However, as generative AI becomes ever more sophisticated in the generation of suggested text and embedded in the apps we use every day, a lack of clarity about where the boundary lies between appropriate and inappropriate use will continue to raise authorship concerns for both students and educators. 

These potential benefits aside, students discovered various issues with AI-generated text:

• The output could be superficial, misleading or irrelevant with missing or inaccurate references.
• Text often followed a generic and non-committal pattern of argumentation, lacking nuance and ‘stance’ and the students thought the rhetorical style tended to be bland.
• AI tools fell short when producing coherent longer texts. As a result, the output required extensive and time-consuming post-editing for flow, signposting, and transition.

---

Editing and proofreading

As noted in the previous sections, the use of generative AI for students’ assignments resulted in extensive post-editing variously for content, argument, referencing, cohesion and coherence.

The use of AI tools created an altered, perhaps greater, demand on the role of editor in the reading-into-writing process in comparison to ‘traditional’ writing:

“Within paragraphs, their output does not always follow a clear and logical order, which will lead to even more work if you want to edit it into a coherent whole. (Student 1)

The answers Humata writes can be used as notes for my essay … However, [they] need to be edited, but how to decide which to keep and which to delete is challenging because I must read the whole reference myself. (Student 2)

All students found AI tools useful for ‘polishing’ writing (e.g. improving grammar and spelling) and did not note any major disadvantages for this stage of the writing process, suggesting that this use is less problematic. However, issues of authorship and student learning are also relevant at the editing and proofreading stage, as with human editors and proofreaders. 

---

Recommendations

In discussion with our students, we developed the recommendations below to facilitate discussions around appropriate use of AI tools in the reading-into-writing process.

Considerations of varying student access to these tools, as well as the ethical and data privacy issues relating to use of generative AI, may take high priority as part of an initial discussion.

Return to top