UCL Mathematical & Physical Sciences


Parent-led tool opens up NHS children's heart surgery data to families

21 June 2016

Researchers are calling for the end to an era of confusion and alarm about children's heart surgery statistics by launching an innovative communication tool that will help people make sense of published survival data about children's heart surgery in the UK and Ireland.

Parent-led tool opens up NHS children's heart surgery data to families

The website was developed by Dr Christina Pagel from UCL Mathematics and Professor Sir David Spiegelhalter from the University of Cambridge, in collaboration with the charity Sense about Science and experimental psychologist Dr Tim Rakow from King's College London. It explains a risk adjustment method known as PRAiS (Partial Risk Adjustment in Surgery).

Understanding Children's Heart Surgery Outcomes, which launches today, shows decision makers and parents that hospitals should not be ranked by their survival rates because hospitals treat different patients - high performing hospitals can have lower survival rates simply because they are taking on the most complex cases. An individual hospital's actual survival rate should only be compared to its own predicted range, which is determined by the complexity of the procedures it undertakes, among other factors. The website also sets out why if a hospital's survival rate is below its predicted range, it need not indicate alarm, but rather serves as a trigger for further investigation.1

"This website draws a line under an era of poor risk communication of hospital surgery statistics," said Tracey Brown, director of Sense about Science. "In 2013 over-interpretation of faulty data resulted in temporary hospital closure of Leeds General Infirmary's paediatric heart unit. Parents and children were faced with all the additional stress, risks and costs of travelling further for operations, and for others the horrendous unnecessary guilt as they wondered if their child's outcome would have been better at another unit. There could not be a stronger case for professionals and decision makers using the risk adjustment model and communicating it well."

Making data public can only drive accountability and improvement if it is also easy to understand. Photograph: Christina Pagel and David Spiegelhalter

"Because different hospitals treat different children and some children can have more complex medical problems than others, it is not valid to directly compare survival rates between hospitals," said Pagel, Reader in Operational Research at UCL, who helped develop the formula the NHS uses to evaluate hospital survival rates. "We involved families from the beginning of the project and throughout to help us researchers communicate these complicated concepts clearly. I definitely learned that incorporating their feedback was absolutely crucial to building something useful. An accountable NHS is one where we can all understand how it is doing- and for this you need to listen to patients and families."

Each hospital that performs children's heart surgery in the UK and Ireland has had its overall survival rates published by the National Congenital Heart Disease Audit (NCHDA)2 since 2013. The researchers used PRAiS to calculate a predicted range of survival for each specific hospital, taking into account the complexity of each individual child's medical condition and surgery. No hospital will have exactly the same predicted range of survival as another hospital, because each hospital treated different children.

However, the report the NCHDA publishes is lengthy, hard to find and hard to understand without expert knowledge. Sense about Science ran user-testing workshops to involve the public, patients' families and medical charities in co-designing the website with Pagel and Spiegelhalter, a first in this area.
"There is an understandable urge to put hospitals in a league table when comparing survival rates," said Spiegelhalter, Winton Professor for the Public Understanding of Risk at Cambridge. "But rather than try and directly compare hospitals with each other, we need to compare a hospital's survival rate with what we would predict it to be, taking into account how severe their cases are. This is a tricky idea but, with the help of many families, I think we have made it clear."
Spiegelhalter and Pagel are calling for other researchers, companies and government to make health statistics accessible to patients and families by making them understandable. Transparency without accessibility is not enough; improved understanding by decision makers, health care professionals, patients and families can prevent misuse, confusion and unfounded anxiety.

1) There are four possible reasons why a hospital might be outside its predicted range:
•    inaccurate or incomplete data
•    the formula used to calculate the predicted range doesn't work well for that specific hospital
•    chance (1 time in 20). (However, if you look at all 13 hospitals that carry out children's heart surgery at once, at least one hospital will fall outside its range just by chance about 9 times in 20.)
•    because the chances of survival at that hospital are different to what is predicted.

2) NCHDA is managed by the National Institute for Cardiovascular Outcomes Research (NICOR)