Using PRAiS to monitor outcomes after paediatric heart surgery
*** UPDATE JUNE 2016: Version 3.0 of the PRAiS software is now available for download from UCLB E-Lucid ***
As described in a paper published in Heart in April 2013, CORU have developed an Excel file “Monitoring Outcomes with PRAiS” that is designed to allow UK paediatric cardiac centres to
routinely monitor their short term surgical outcomes. This uses data that UK & Ireland centres collect for the National Congenital Heart Disease Audit (NCHDA), managed by the National Institute for Cardiovascular Outcomes Research (NICOR). To purchase this software, please visit the University College London Business E-lucid website.
Outcomes are adjusted for risk using a model (Partial Risk Adjustment in Surgery: PRAiS)
that estimates the risk of death within 30 days of a surgical procedure
based on NCHDA specific procedure, age, weight and a patient's recorded diagnoses and
additional health problems. The original risk model was published in Crowe
et al. JTCVS 2013 (online July 2012) and was developed on a random 70% subset of ten years’ of UK national audit data and used the 2010 NCHDA specific procedure algorithm. It was validated in the remaining 30% of the
national data set which had not been used for model development. The full final report on the risk model development has been published by NIHR in Health Services and Delivery Research and is open access.
Version 1.0 of the PRAiS software used an updated version of the PRAiS risk model which was recalibrated on the full 2007-2010 NCHDA data set using the 2012 NCHDAspecific procedure algorithm. In response to more complete data entry and an apparent improvement in UK outcomes since the 2007-2010 period, we recalibrated the PRAiS risk model again on cleaned 2009-2012 UK data, with the agreement of the NICOR congenital steering committee, using the 2013 version of the Specific Procedure algorithm.
We comprehensively updated the risk model PRAiS in 2016, revisiting all the risk factors as part of a year-long NIHR funded research project, which included extensive input from an expert panel of clinicians and data experts. This update will be written up for academic publication in the summer of 2016.
The VLAD charts do not incorporate statistical thresholds, but in
response to requests from UK units we have added functionality to estimate
prediction intervals for observed survival. A summary of our thoughts on the role of our software in quality assurance and improvement initiatives is given in the Heart paper and in our document called "Use of PRAiS for risk adjustment". We have also published an open-access editorial in Heart discussing the benefits and risks of monitoring risk-adjusted mortality in paediatric cardiac surgery.
Before implementing the risk model, we exclude certain records: duplicated records, records for patients over 16, records which do not pertain to a cardiac procedure or have no procedure codes, records which are for patients who only had catheter procedures or other procedures which are neither "bypass" or "non-bypass" or "non-HLHS hybrid". 30-day episodes of care are then generated as follows: Each patient’s first episode starts with their first surgical procedure and the patient’s vital status at 30 days is assigned as a primary outcome. Any further procedures within this 30 day episode constitute a secondary outcome. A surgical procedure more than 30 days after the first procedure constitutes the beginning of a new episode.
Finally, to implement the risk model as given in the specification, the relevant risk factors need to be generated from the raw data. This is done by using mappings from the European Paediatric Cardiac Code Short List (EPCC short codes) for diagnosis groupings and additional health factors. You can download all the relevant mappings for version 3.0.
The Excel file “Monitoring Outcomes with PRAiS v3.0” performs all of this processing for the user on raw data in the format needed for UK national audit, including basic error checking and generation of the NCHDA "specific procedure" category. The user can then use the software to generate estimates of risk for all 30 day episodes of care and produce a Variable Life Adjusted Display (VLAD) chart covering the period of the data. VLAD charts allow units to examine their own programme-level outcomes and quickly identify trends in outcomes (positive or negative) that might warrant further investigation. An illustrative VLAD output of the software is shown below. Note that this does not correspond to actual data.