Supping with the devil

Every academic who takes on politically sensitive research is haunted by the ghost of David Kelly. By all accounts a humble, honest and capable man, Kelly talked to a journalist about a scientific issue which had become central to a key political debate. In what was later described as a 'feeding frenzy' by press and politicians, Kelly was publicly named as a source; his academic reputation and integrity were called into question; and he was thrown into a state of desperation and self doubt that cost him his life. Alongside weapons of mass destruction in the premier league of 'sensitive research' sit a number of ongoing research themes at UCL, including human embryos, climate change, third-world debt, and large-scale government IT programmes. My own work on the last of these explores a key component of the National Programme for IT in the NHS (estimated to be costing the taxpayer £12 billion over the next few years). Should a summary of the medical record of every NHS patient be uploaded onto a central spine, potentially accessible to any member of NHS staff (and, some claim, to hackers)? If so, is the current effort to achieve this goal competently managed and sufficiently secure?

'Evidence', however rigorously and impartially generated, is not neutral, nor can we stipulate how people and interest groups will use it. As academics, we have two choices: stay out of the kitchen, or deal with the heat.

A month ago, my team's report on networked electronic records in the NHS hit the public domain with barely a ripple. The tabloids ignored it, and 'The Guardian' described it as "one of the most thought-provoking analyses yet of the Blair government's infatuation with reforming public services with the help of big IT". More significantly, all the key political stakeholders (one of whom had told me verbally a month before publication: "if we don't like it, we might rubbish it") endorsed the report and immediately set up various working groups, on which my team were invited to be represented, to address its recommendations. The senior civil servant whose work I had criticised took me out to dinner (note: I paid my share of the bill).

What went right? I think there are five important lessons:

First, we took governance seriously and set up an external advisory group with teeth. I appointed a lay chairperson ('lay' in the sense that she was neither an academic nor an IT expert, though she did have extensive experience as a non-executive director and chair of other public bodies) and invited all stakeholders including various patient groups to send representatives. There were some single-issue stakeholders whose presence I found irksome, but potential adversaries were all located 'inside the tent, pissing out'. I also lined up four academic heavyweights from other universities, whose collective endorsement of our findings would make them hard for politicians and civil servants to "rubbish".

At the first meeting of the group, I tabled some draft terms of reference, which were amended and agreed to read as follows: "The External Advisory Group will be the main vehicle of governance for the project. It will monitor the project against the work plan set out in the contract and supplementary documents, and suggest changes to methods, timings and costs as required. This group will be a critical audience for our ideas and emerging findings, and a source of information and advice. The Advisory Group will meet 3 times over the 12 month project period, and will ensure accountability by reviewing formal progress reports of the evaluation, including a brief report on financial expenditure."

Over the subsequent few months, I found myself repeatedly under pressure to move the goalposts of the research and question my own competence and leadership. Various stakeholders, including one or two in the corridors of power, contacted me to say that our remit should be either broader or narrower. Some said our methods were weak; others that our analysis was biased; and others that we were shirking on our ethical obligations to 'blow the whistle'. At an early stage, a medical tabloid reported that we had sold out to the Department of Health and were not the 'objective' assessors promised by a Ministerial Task Force.

All these pressures and criticisms were fed into the regular meetings of our external advisory group. Decisions were made collectively and carefully minuted. Responsibility for them was thus corporate, not individual - and the various representatives had either signed up to them or been formally outvoted.

The second thing we did right was to table a risk assessment as a recurring agenda item at the advisory group. For every risk, we were required to present the group with a plan to address it. Initial risks identified included "key stakeholders may not understand the nature and purpose of the evaluation", "key stakeholders may fail to engage with the evaluation as a formative process" and "service users may be threatened or confused by the evaluation". For all of these, the plan of action included a good deal of senior-level pressing of the flesh and listening to points of view. The advisory group proved an excellent critical friend here. I was not allowed to get away with excuses like "I'm too busy to do this", "I hate putting on pin-striped suits" or "I need to meet a deadline for writing an academic paper".

Thirdly, we instigated (in advance) a formal consultation process on our draft report. With the input of the external advisory group, we identified potential audiences and set up a timetable for sending out drafts, receiving comments, and responding to them. This generated a lot of work, but we delivered on our promise to make the consultation more than a cosmetic exercise. The non-UCL academics on the advisory group were called in (and paid) to adjudicate when powerful stakeholders said of our findings, "you do not have evidence for this statement" or "this is not my experience". This process involved me preparing a hefty file for independent scrutiny containing examples of raw data, interim analyses, minutes of internal meetings, and examples of how we had drawn higher-order insights by interpreting and synthesising (allegedly) 'soft' data.

Fourthly, when preparing the final version of the report, we distinguished between requests to change or remove passages that were incidental to the academic message (eg which stakeholders found offensive and/or which were potential press bait) and altering substantive empirical findings or academic analysis. The advisory group's requirement for a point-by-point response to every criticism focused us squarely on this distinction. Whilst the final version of the report was a good deal more muted than the first draft (indeed, one source complained that it was completely devoid of soundbites), I was able to rebut a direct question from a journalist: "Is it true that adverse findings [about the Department of Health] were edited out between the draft and the final report?"

Finally, we worked in advance with the UCL Media Relations to convert an academically 'worthy' 7-page executive summary into something the press and public could engage with, in a way that did not involve 'sexing it up'. The press office also helped us anticipate and coordinate the media response, and worked closely with us during the embargo period to provide a final common pathway for press enquiries and operationalise the dissemination process (eg circulating the link to our website). UCL Media Relations understood two things of which I was only dimly aware at the time: press interest can be exhausting, and academics are generally undertrained to deal with persistent journalists. I greatly valued their support.

What went wrong? Oh, plenty.

I badly misjudged the amount of time needed for the political shenanigans and consultation exercise, so the last few weeks felt like panic and involved a run of 16-hour days for everyone on the team. I probably asked my junior staff to do things that were not on their job descriptions - and whilst they rose to the occasion admirably, many of the tasks could and should have been anticipated and appropriate training given. I made an off the record comment that was published two days later. And I gave too many people my mobile phone number.

Why do I do politically sensitive research? For the same reason as anyone does any kind of research - because I want to find out the answers. And because as Jeremy Bentham pointed out two centuries ago, an academic's most urgent duty is to keep the government of the day on its toes.

Trisha Greenhalgh, Professor of Primary Health Care, UCL

• Links:
• Summary Care Record Independent Evaluation
  
• UCL Press Release
  
• UCL Media Relations