Frameworks to evaluate vocabulary learning conditions

Summary of some of the main frameworks proposed to analyse vocabulary learning conditions. Can be used to evaluate the effectiveness of vocabulary learning apps and EdTech products.

While other frameworks have been suggested to evaluate educational apps in general, this page focuses on those that have been proposed to examine vocabulary learning conditions in particular.

Frameworks

The first two frameworks were proposed to examine vocabulary learning programmes and conditions, without a particular focus on technology-mediated learning, while the third framework presented was designed to specifically evaluate vocabulary learning apps:

The Four Strands
Framework of vocabulary learning conditions
Framework to evaluate vocabulary learning apps

References

The Four Strands

According to this framework, a well-balanced programme should devote a quarter of the time to each of the four strands.

The principle of the four strands (Nation, 2007) states that a well-balanced programme should provide opportunities for both incidental and intentional learning through a combination of:

Strand 1: Meaning-focused input

Learning through reading and listening.
Learning from comprehensible input.

Refers to learning through reading and listening (receptive learning). It is largely incidental and thus, it requires large quantities of input to ensure enough target vocabulary is encountered several times (Webb & Nation, 2017). Input should be comprehensible, at the right level for the learner’s or user’s proficiency.

Strand 2: Meaning-focused output

Learning through writing and speaking.

Involves learning through writing and speaking (productive learning), which is more challenging than receptive learning, as it requires more precise knowledge of the vocabulary being used (Webb & Nation, 2017).

Strand 3: Language-focused learning

Deliberate attention to language features.

Provides opportunities to learn words and phrases deliberately and intentionally.

Strand 4: Fluency development

Making the best use of what is already known.

Activities aiming at making existing knowledge fluently available. Thus, it involves practising and working with material that learners or users are already familiar with. Fluency should be developed in each of the four skills, promoting fluent access to vocabulary in reading, listening, speaking and writing.

Framework of vocabulary learning conditions

Webb and Nation (2017) identified two key factors that underpinned any vocabulary learning condition, i.e. repetition and quality of attention. They argue that the more encounters and the deeper quality of those encounters, the more likely learning is to occur.

According to this framework, the conditions that facilitate vocabulary learning include repetition, noticing, retrieval, varied encounters and varied use, and elaboration - see table 1 for a brief description of each condition:

Table 1. Vocabulary learning conditions (adapted from Webb & Nation, 2017)
Repetition	N of encounters
Quality of attention	Type of approach: incidental or deliberate.
Noticing	How/if attention is driven to the target words.
Retrieval	Recalling what was encountered before.
Varied encounters	Repeated encounters but in different forms or contexts.
Elaboration	Enrichment of knowledge of the word by encountering more aspects of its form, meaning and use.

Framework to evaluate vocabulary learning apps

Lin et al. (2022) recently proposed a six-component framework to evaluate vocabulary-learning apps.

The framework is based on relevant existing frameworks and sets of instructional-design principles. It includes what they concluded were the key components of both vocabulary instruction and mobile learning.

Table 2. Lin et al. (2022) proposed six-component framework.
Criterion	Definition
Learning tasks	What types of educational tasks are deployed? E.g. matching, sentence writing, word pairs, reading… Studies seem to suggest that meaningful, real-life tasks (e.g., involving problem solving) tend to be more beneficial than drill-and-practice activities.
Goal clarity	Whether an app presents clear goals and whether learners can set their own goals.
Feedback	Feedback in this framework is defined as "supporting information provided on the quality of users’ performance" (Callaghan & Reich, 2021). Opportunities to receive feedback promote learners’ engagement, motivation, and learning outcomes.
Strategy instruction	Strategy instruction refers to tips on how to access the meanings, sounds, and/or spellings of the target words (van Merriënboer, 1997). Common strategy instruction types in the analysis conducted by Lin et al. (2022) included: word lists/flashcards contextual clues semantic associations word-parts, and mnemonics.
Repetition	Whether the app provides various encounters with the target vocabulary.
Instructional control	Whether the app allows users to select their own content and pace. More control over the content and pace seems to support vocabulary development and learning in general.

The framework by Lin et al. (2022) is, to our knowledge, the only framework specifically designed to support the evaluation of vocabulary learning apps.

While it made an effort to include both vocabulary learning principles and instructional-design principles, some of the important conditions and features identified in previous vocabulary learning frameworks are either missing or not explicitly mentioned. Thus, for a comprehensive evaluation, this framework should be supplemented with the criteria identified in previous frameworks.

References

Callaghan, M. N., & Reich, S. M. (2018). Are educational preschool apps designed to teach? An analysis of the app market. Learning, Media and Technology, 43(3), 280–293.
Lin, C., Zhou, K., Yang, S., & Sun, Z. (2022) Developing an evaluation framework for vocabulary-learning apps. Interactive Learning Environments.
Nation, I.S.P. (2007). The four strands. Innovation in Language Learning and Teaching, 1(1), 1-12.
Webb, S. & Nation, P. (2017). How vocabulary is learned. Oxford University Press.

Contact us

Centre for Applied Linguistics
Department of Culture, Communication and Media
IOE, UCL's Faculty of Education and Society
University College London
20 Bedford Way
London WC1H 0AL