Which one to use?: The Washington Group Questions or The Model Disability Survey

Department of Epidemiology and Public Health

University College London

Introduction & Overview

The realization that people with disabilities make up 15% of the world’s population and are disproportionately poorer and more marginalized than all other groupsi has prompted urgent calls for more accurate data to better understand the needs of this population.ii Monitoring the UN Convention on the Rights of Persons with Disabilities (CRPD)iii and the Sustainable Development Goalsiv – with its call to ‘leave no one behind’ – has generated further interest in accurate disability statistics.
To improve global disability data, two methodologies have received considerable attention -

1. The Washington Group Questions

2. The Model Disability Survey

However, government officials, practitioners, civil society groups, advocates and researchers often are unsure which methodology to use.

I argue here that the confusion is unnecessary. The methodologies are for the most part, designed to serve different purposes.

This paper presents a quick introduction to this question. Briefly:

The Washington Group (WG) Questions are targeted questions on individual functioning intended to provide a quick and low-cost way to collect data, which allows disaggregation by disability status. Designed initially for National Statistical Offices to be added to population-based censuses and surveys, the WG Questions increasingly are also used by nongovernmental organizations, researchers and advocates in a range of data collection tools. The Short Set of WG questions (WG SS) has only 6 questions and can be answered in about 1.5 minutes; the enhanced Short Set contains 6 additional questions; the Extended Set (WG ES) has 35 Questions and can be answered in 10-12 minutes. The methodology is widely used.

The Model Disability Survey (MDS) has been developed as a stand-alone data collection instrument to provide in-depth information for disabled populations at regional or national levels. Designed to be implemented every 5-10 years, the full MDS has 294 Questions and takes 120-150 minutes (2 hours to 2.5 hours) to administer. A ‘Brief Version’ of the MDS has 38 questions and is intended to be integrated into existing data collection tools, taking about 12 minutes to administer. Functioning questions in the Brief Version do not overlap entirely with the Washington Group questions; it also includes questions about assistive devices and personal support. The Brief MDS is new and has yet to be widely implemented.

I. The Washington Group Questions

The Washington Group on Disability Statisticsv is a UN City Group established under the United Nations Statistical Commission. Guided by a Secretariat that represents 135 National Statistical Offices, the WG addresses the urgent need for cross-nationally comparable population-based measures of disability.

There are a range of WG tools, but best known is the Washington Group Short Set (WG-SS). These 6 questions, which focus on difficulties in seeing, hearing, mobility, communication, cognition and self-care, allow self-reporting of functioning at the individual level. Each question has one of four response categories: No, no difficulty; yes, some difficulty, Yes, a lot of difficulty, Cannot do at all. For the purposes of data disaggregation, people are identified as disabled if they respond ‘a lot of difficulty’ or are ‘unable to do’ to at least 1 of the 6 questions.

The 6 questions can be dropped into any number of censuses and surveys routinely collecting demographic data – (i.e. gender, income, education, rural vs urban status). These questions enable the rapid collection and disaggregation of population data by disability status to see if persons with disabilities are participating equally in all aspects of society.

They provide a quick picture of how people with disability fare in comparison with their non-disabled peers’ whose demographic information is being collected in the same censuses or surveys1.

The methodology has been internationally validated and cognitively tested in a number of low, middle and highincome countries.

Washington Group Questions

Short Set (WG-SS)

6 Questions - takes about 1.5 minutes to administer
Easy to include in range of censuses, surveys and other data collection instruments
Widely used by national governments as well as NGOs, Disabled Peoples Organizations (DPOs), civil society organizations
Analysis is straightforward. Disaggregation can be done by anyone with a basic knowledge of statistics using SPSS syntax available on the WG website.2

Extended Set (WG-ES)

35 Questions – takes about 10-12 minutes to administer
The full WG-ES can be used in their entirety – but options allow inclusion of some but not all questions, depending on need. (This includes the ‘enhanced Short Set’, that adds 6 questions out of the 35 WG-ES to the WG-SS providing two additional functional domains, upper body and psychosocial functioning).
Increasingly used in surveys that have a major focus on disability.
Analysis is straightforward. Disaggregation can be done by anyone with a basic knowledge of statistics using SPSS syntax available on the WG website.3

UNICEF/WG Child Functioning Module (CMF)

UNICEF has collaborated with the WG to develop the CMF building on the WG methodology. It is now included in UNICEF’s Multiple Indicator Cluster Surveys (MICs)vi to better identify children with disabilities.

Additional Uses

Further WG linked methodologies are being developed in collaboration with the International Labor Organization (ILO) – a survey module on disability in Labor Force Surveys; and with UNICEF – a survey module on barriers and facilitators to Inclusive Education.vii

Washington Group - SUMMARY

Pluses

Low-cost, quick and easy to administer
Easy to analyze
Through disaggregation, provides enough detail on disability for many development programs, government outreach efforts and civil society initiatives.
Useable for monitoring and evaluation related to the UN Convention on the Rights of Persons with Disabilities;viii and Sustainable Development Goals

Drawbacks

The WG-SS does not directly address mental health functioning. However, the enhanced WGSS, adds 6 questions from the WG-ES, including 2 on anxiety and 2 on depression, significantly improving the quality of data on psychosocial functioning.
The WG-SS and WG-ES Questions are not considered accurate for individuals below the age of 5 years. Also, they miss many children with developmental disabilities. However, the CFM (now included in the MICS) specifically covers younger children (2 to 4 years of age) as well as those 5 to 17 years, and better identifies children with developmental disabilities.
Identifies individuals with more significant disabilities but may miss some with less severe disabilities.
Provides disaggregated data in 6 domains for analysis but may miss more detailed information on which to develop in-depth services to enhance social participation and inclusion.ix

II. The Model Disability Survey (MDS)

The MDS has been developed and promoted by the Disability Unit of the WHO World Health Organization with collaboration from the World Bank. It is designed as a stand-alone data collection instrument ‘providing comprehensive and systematic documentation on all aspects of functioning within a population.’x

The MDS (and the Brief MDS) are general population household surveys, so questions are intended for a representative sample of the whole population, allowing comparison between disabled and non-disabled members of the community.

The full MDS has been internationally validated and cognitively tested in 6 countries (including low, middle and high-income countries); and piloted in 3 more. The Brief MDS is new and has not yet been independently validated nor cognitively tested, but questions selected are based on analytical results from full MDS.

Model Disability Survey (MDS)

A comprehensive, in-depth survey, providing a wealth of detailed data about disabled populations at regional or national level for in-depth analysis, discussion and planning.
The individual questionnaire (answered by a randomly selected person in the household), has 294 questions.4 The accompanying household survey has an additional 21 questions.
The full survey takes 120-150 minutes (2 hours to 2.5 hours).
Analysis focuses on building a ‘disability scale’ ranging from 0-100, that can then be partitioned into different levels of disability, using ‘fit-for-purpose’ cut offs, based on scores of people with medical conditions that are highly associated with disability. Compared to the WG, the MDS relies on more complex and less intuitive calculations to create a more detailed continuum of functioning.
Data collection and analysis are specific to this survey – the survey questions are stand-alone and are not intended to be added to other data collection efforts.
Analysis is intended to be undertaken by people with some technical expertise in statistics, but is assumed to be within the technical capacity of National Statistics Offices. Where needed backup assistance available via WHO MDS team.
Intended to be administered once every 5-10 years.

Brief Model Disability Survey (MDS)

A new Brief version of the MDS is being implemented this year (2018).xi
It has 38 questions, and is intended to be added to other surveys and censuses.
It asks a range of questions related to functioning – similar but not identical to the Washington Group questions, as well as to a series of questions on environmental factors, assistive devises and personal assistance.
Analysis uses the same methodology as the full MDS. It creates a disability scale from 0-100, allowing identification of local and national barriers faced by people with disabilities. This allows comparisons between both between disabled and nondisabled members of the community as well as nuanced differences between people with different types of disability and levels of severity.
Analysis is intended to be undertaken by people with some technical expertise in statistics, but is assumed to be within the technical capacity of National Statistics Offices

Model Disability Survey – Summary

Pluses

Full MDS provides in-depth understanding of how people with disabilities fare at community and country level.
Analysis for both full and Brief MDS is based on building a ‘disability scale’ from 0100 and then establishing cut-off points and identifying different levels of severity, with interpretation benefiting from understanding of local/national laws. Thus, the full MDS offers governments in particular, is an in-depth tool for short and long-term planning.
Useable for monitoring and evaluation related to the UN Convention on the Rights of Persons with Disabilities, but full MDS does not provide annual information.

Drawbacks

Full Model Disability Survey

The MDS is comprehensive, but long and expensive to administer.
Analysis is complex, although assumed to be within the technical capacity of National Statistics Officesxii although backup assistance is available via WHO MDS. Availability of support for researchers, advocates, etc., will have to be negotiated with WHO MDS but they are willing to help. Thus, it provides important, nuanced results but requires a commitment to competent levels of analysis and interpretation.
Because of time and expense, as well as the fact that such in-depth information is not needed annually, recommended for use every 5-10 years. Thus, the in-depth information it offers is most valuable if survey is repeated over time to allow comparison.
Level of detail about disabled populations generated by the MDS is not needed in all development programs, government outreach efforts or civil society initiatives.
SDGs & CRPD: CRPD-relevant data collected but data collection is not done annually. Not recommended for SDGs. For disaggregating the SDGs, the SDG indicators must be collected the same way for people with and without disabilities. So, for example, if a country is using its annual Labor Force Survey for the employment SDG, users cannot compare an employment indicator from the Labor Force Survey with one from the MDS. This is because the instruments will have different samples and different employment questions. For that reason – as well as the fact that it is implemented every 5-10 years, it is not recommended.

Brief Model Disability Survey

Not been independently validated nor cognitively tested, however the selection of the questions are based on analytical results from data from full MDS implementations. 
38 questions are a large number of questions to another data collection instrument.
Brief questions are not the same as the WG-SS or WG-ES, so users are unable to make comparison between findings from the Brief MDS and any tools that use the more widely used WG-SS and WG-ES Questions.
As with full MDS, analysis is more complex than WG, being based on a disability ‘score’ from 0-100. Backup assistance is available via WHO MDS team in Geneva.
It is new and has yet to be routinely used (2018).
SDGs and CRPD – can be used for the CRPD. It is intended to be usable with SDGs, but data collection and analysis has yet to be implemented in field conditions so the jury is still out.xiii

Who is Using Which Methodologies?

Washington Group: The WG, particularly the WG-SS are widely used:xiv

60 countries currently have used WG-SS or WG-ES questions in national censuses with this number expected to increase significantly in the 2020 round of censuses
Most UN agencies now collect disability data using either the WG-SS or WG-ES or MICs
Many bilateral organizations – including DFID/ DFAT are strongly supporting inclusion of WG Questions in many/ most funded development efforts
The UNICEF/WG Child Functioning Module is now included in UNICEF’s Multiple Indicator Cluster Survey (MICS) and is expected to rollout in 70 countries by 2020.
The World Bank in the Bank-sponsored Living Standards Measurement Studies (LSMS) that will cover 70 countries has recommended the WG-SS for use.
An optional disability module containing the WG Questions has been added to DHS – used widely – especially by US AID
Used widely and strongly supported by a number of NGO and DPOsxv
Growing number of advocates and researchers now use WG questions–Short, enhanced and Extended Sets
SDG & CRPD disaggregation – widely used

Model Disability Survey: Current use of the MDS and Brief MDS is more limited than the WG instruments:

MDS has been used in 7 countries: Chile; Sri Lanka; Philippines; Qatar; Dubai, UAE; Pakistan and Cameroon – with survey sizes ranging from 500-11,000; another 7 countries are scheduled to use it in the near future.
The Brief Survey is new. Recently tested, future use will provide more insights into who is using this methodology and how useful they find it.
SDG & CRPD disaggregation - The MDS is not appropriate for use with SDGs; use of Brief version unclear at this point. MDS can be used for CRPD but not annually, Brief MDG use for CRPD as yet unclear.

Common Ground

Both methodologies based on the UN International Classification of Functioning, Disability and Health.xvi
Neither methodology intended for clinical diagnosis or determination of disability status at the individual level. 
UN Statistics has an important role to play in this debate, but it is currently reviewing and assessing the different methodologies

Can you use both methodologies?

Governments and others could include WG-SS, enhanced WG-SS, or WG-ES; or the CFM (in the MICS or in other surveys), on a national census, an education, labor force or a household expenditure survey. If there is interest, time and funding, an MDS could also be undertaken to provide in-depth information every 5 to 10 years, as recommended by WHO.

Governments and others who chose to use both methodologies are encouraged to consider what the collected from both sets of information will be used for to ensure optimum benefit is derived from such an undertaking.
In many other situations – NGOs, DPOs, researchers, etc., use of both methodologies might generate far more data than is needed to explore most demographic and international development – (health, education, employment, social protection) – concerns.
Level of detail about disabled populations generated by the MDS is extensive but may not be needed in all development programs, government outreach efforts or civil society initiatives.

For more Information

1. Washington Group on Disability Statistics Website: www.washingtongroup-disability.com

2. Model Disability Surveyhttp://www.who.int/disabilities/data/mds/en/

References

i WHO/ World Bank. 2011 World Report on Disability

ii Leonard Cheshire and DFID. The Disability Data Portal. https://www.disabilitydataportal.com/

iii United Nations 2006. Convention on the Rights of Persons with Disabilities. https://www.un.org/development/desa/disabilities/convention-on-the-rights-of-persons-with-disabilities.html

iv United Nations. 2015. Sustainable Development Goals. www.un.org/sustainabledevelopment/sustainable-development-goals/

v UN Groups are named after the first city in which a meeting of the group is held. In this case, the first meeting was in Washington, DC in 2002. The Washington Group has no affiliation with the US Government.

vi UNICEF. 2017. Multi Indicator Cluster Surveys. http://mics.unicef.org/

vii For more in-depth discussion, see Washington Group Website: www.washingtongroup-disabilty.com

viii Madans J, Loeb M, Altman B. 2011. Measuring disability and monitoring the UN Convention on the Rights of Persons with Disabilities: the work of the Washington Group on Disability Statistics. BMC Public Health. 11(Suppl 4): S4. May 31

ix Sabariego C, Oberhauser C, Posarac A, Bickenbach J, Kostanjsek N. Chatterji S, Officer A, Coenen M, Chhan L, Cieza A. 2015. Measuring Disability: Comparing the Impact of Two Data Collection Approaches on Disability Rates. International Journal of Environmental Research and Public Health. 12:10329-10351.

x World Health Organization. The Model Disability Survey. Disability and Rehabilitation section: World Health Organisation website: http://www.who.int/disabilities/data/mds/en/

xi World Health Organization. Model Disability Survey. http://www.who.int/disabilities/data/brief-model-disabilitysurvey5.pdf?ua=1

xii Sabariego C, Oberhauser C, Posarac A, Bickenbach J, Kostanjsek N. Chatterji S, Officer A, Coenen M, Chan L, Cieza A. 2016. Response to Madans et al. Comments on Sabariego et al, Measuring Disability: Comparing the Impact of Two Data Collection Approaches. International Journal of Environmental Research and Public Health. 13:666.

xiii Sabariego C. 2017. The role of the World Bank and WHO Model Disability Survey in public health. Public Health Forum. https://www.degruyter.com/view/j/pubhef.2017.25.issue-4/pubhef-2017-0039/pubhef-2017-0039.xml

xiv Groce, N. E., & Mont, D. (2017). Counting disability: emerging consensus on the Washington Group questionnaire. The Lancet Global Health, 5 (7), 649-650

xv Groce, N. E., & Mont, D. (2017). Ibid.

xvi World Health Organization. UN International Classification of Functioning, Disability and Health http://www.who.int/classifications/icf/en/