Duolingo English Test can predict international students’ first year academic performance

Students arriving with higher DET scores are more likely to achieve better grades in their first year at university than students with lower DET scores. Whilst the scores can accurately predict non-Chinese students’ academic performance, the relationship was weaker for the larger cohort of Chinese students. The researchers note that this could be down to breadth of disciplines represented in their sample of Chinese students at UCL, who represented nearly three-quarters of participants in the study in terms of nationality.

The DET is a fully automated test, which means that it is both machine-delivered and scored. It was widely adopted during the COVID-19 pandemic following test-centre closures and other restrictions. Developed by Duolingo (but unrelated to their well-known language learning app), DET can be taken at any time and place for university applicants to prove their English language proficiency. The test continues to be commonly used for admissions testing, as well as the more established English proficiency tests used for that purpose such as IELTS and TOEFL iBT.

The research paper, published today in Language Testing, is the first study that examines the relationship between DET scores and academic grades, in light of the steep rise in use of the test for university admissions purposes in the UK and at other universities around the world. Researchers also took into account students’ degree level, nationality and field of study in their analyses.

Lead author Dr Talia Isaacs (IOE, UCL’s Faculty of Education & Society) said: “The rapid entrée of DET into the admissions testing arena meant that large-scale datasets allowing researchers to link language test scores to university grades have only recently become available. There is an urgent need for evidence supporting the high-stakes use of the test scores for university entrance purposes.

“Given its large student body and high proportion of international students, UCL was the ideal setting in which to conduct a study to begin to fill the evidence gap and answer the crucial question of whether students who score well on the test go on to perform well at university. Our study suggests that indeed the test does predict academic success, although there are some subtleties in the findings.”

The study team analysed correlations between English test scores and first year grades for a group of UCL students at the height of the pandemic (2020-2021 academic year), including 1,881 who were admitted using the DET, compared to 2,651 first-year UCL students who had taken IELTS and 436 who took TOEFL iBT.

The researchers found that for all three language proficiency tests, higher scores predicted better academic performance. The team also found differences in academic scores based on the test taken, which varied depending on students’ proficiency level. For example, among graduate students at the most common proficiency level (C1), DET students’ grades were lower than IELTS students’ grades by 2% on average. Results varied for other proficiency levels and tests. Such differences could be attributable to factors unrelated to the test. For example, academically weaker students may have applied to university later in the application cycle, when DET was the only test option because of pandemic restrictions. It is also possible that admissions thresholds (i.e., minimum scores needed to get into different programmes) need to be adjusted.

Co-author Dr Ruolin Hu (IOE, UCL’s Faculty of Education & Society) said: “These results reflect the unique circumstances that arose during the early stages of the COVID-19 pandemic, including sudden changes to university teaching, assessment and policy. Further research is needed to see whether our results hold in light of some recent changes to the test and also in non-pandemic situations.

“This study sets an important precedent and baseline for comparison. DET has garnered a lot of attention, the admissions testing and test preparation industry is rapidly changing, and we look forward to seeing what comes next.”

The study was funded by Duolingo and involved researchers at UCL and the University of York.

Links

• Research paper in Language Testing
• Lay summary of the research paper
• Dr Talia Isaacs’s academic profile
• Dr Ruolin Hu’s research profile
• IOE, UCL’s Faculty of Education & Society

Image

• Source: iStock

Media contact

Kate Corry

Tel: +44 (0)20 3108 6995

Email: k.corry [at] ucl.ac.uk