Close
Examining the impact of funded area-level interventions across England (April 2019 – March 2022)
Research Team
Dr Jess Deighton, Professor Jane Barlow, Professor Steve Morris, Dr Tanya Lereya, Dr Emily Stapley
Theme
Early interventions for children across the life course and their families
What we found out
- HeadStart, an area-level mental health intervention, aimed to improve the wellbeing of secondary school students did not impact school absenteeism or attainment.
- However, it prevented about 800 students from experiencing a school exclusion in 2016/17, its first year of implementation.
- This represents a 10% relative reduction in the exclusion rates in areas that received programme funding. This impact is on the boundary of statistical significance.
- The study used a method called the Synthetic Control Method to evaluate the impact of HeadStart. The analysis provides encouraging evidence that it is an appropriate methodology to use to evaluate the impact of other complex area level interventions.
Why is this important
There has been a dramatic rise in mental health disorders among adolescents seen in the past decade across the world, there is an urgent need for robust evidence on what works to combat this trend. This work provides the first robust evaluation of the impacts on school outcomes of 6-year funding programme (HeadStart) for area-level mental health interventions for adolescents.
Policy implications
Findings show that areas level interventions of this kind can impact on wider educational outcomes such as exclusion rates, but there are challenges associated to sustaining impacts over time despite continued funding. They also indicate that synthetic control method is a promising methodology to investigate the impact of complex area level programmes.
What we did
To investigate the effectiveness of HeadStart on school level outcomes, we used the synthetic control method developed by Abadie, Diamond, and Hainmueller (2010) and analysed routinely collected data from the National Pupil Database (Department for Education).
The outcome variables included:
a. Absenteeism: Authorised and unauthorised absence – measured in terms of proportion of all sessions missed for authorised/unauthorised reasons
b. Exclusion: Proportion of students ever excluded (fixed period exclusion or permanent exclusion) and proportion of all sessions missed because of a fixed period exclusion.
c. Attainment: Proportion of students obtaining 5 GCSEs A*-C.
The idea behind employing the synthetic control method is that there may not be a single local authority that can be a good comparator to the HeadStart local authorities. In contrast, a weighted average of all the LAs that did not get the funding may create a good control group. The goal is to find the weights to assign each LAs that did not get the funding so that the weighted average of their outcomes in the pre-intervention period mimics the average of outcomes in the HeadStart LAs as closely as possible.
We sought to achieve this balance in terms of the outcome variable measured in each year of the pre-treatment period, as well as LA pupil composition across the pre-intervention years in terms of gender, ethnicity, eligibility for free school meals, SEN, referrals into children’s social care and area deprivation.
If it is possible to weigh the non-HeadStart local authorities in a way that achieves a good comparison to the HeadStart LAs pre-intervention, the outcome of the synthetic control group can be used as `counterfactual’ for the HeadStart LAs once the intervention takes place. In other words, any difference that we observe between the outcomes of the HeadStart local authorities, and the outcomes of the synthetic control group can be attributed to the impact of HeadStart.
