Opinion: Why is the government buying Covid tests without evidence they work?
5 August 2020
Dr Dami Collier co-writes an op-ed questioning the decision of the UK Government to purchase the new 90 minute rapid diagnostic tests when there is no present evidence to show they work.
Many of us working in NHS hospitals welcomed the news earlier this week that the government had purchased 90-minute Covid-19 tests. Rapid swab tests, called LamPORE, and 5,000 machines, supplied by DnaNudge, will soon be available in adult care settings and laboratories. If they’re effective, they could allow for rapid, on-the-spot testing. But there’s no publicly available data about the accuracy of these tests or how they perform, raising concerns about why the government has endorsed – and purchased - them.
The lab-free testing machine from DnaNudge is potentially exciting. The government has placed a £161m order for these machines, which are currently being evaluated on saliva. If they work, people could be tested for Covid-19 without having to take the nose or throat swab that many find unpleasant. This has particular advantages in children, though it has a low throughput – each machine can complete only a maximum of 15 tests a day. The developers of the LamPORE swab test say they can complete up to 15,000 tests a day, although no real-world data is publicly available.
During the pandemic, the urgency that has produced valuable data has also led to the fast dissemination of research and equipment that isn’t subject to normal systems of evaluation, such as peer review. As clinical researchers, we went looking for information about these two new tests, and could find no data on either, despite the DnaNudge website stating excellent performance in clinical studies. The LamPORE website says that analytical performance testing is “ongoing”.
LamPORE and DnaNudge are undoubtedly exciting technologies. But the fact the government announcement didn’t even include a “pre-print” publication (the term given to research that is published prior to undergoing peer review) about either test is concerning. According to Imperial College London, DnaNudge was evaluated from April in eight London hospitals.
Nobody wants to hinder the development of new, potentially lifesaving technologies. But in order to fully understand how the tests perform in a lab, pre-clinical testing should be rigorous.
For a test to be properly validated, a number of conditions need to be satisfied. This takes time. First, a new diagnostic test designed to look for virus genetic material is assessed in the laboratory, using artificially made genetic segments that look like virus. This artificial sample has a set number of copies of genetic material, and it is then diluted down a number of times. Each diluted sample is tested on the device multiple times to see whether it can be detected reproducibly and reliably.
This experiment provides what’s known as the detection limit of the test, expressed as the number of “virus” genetic copies in each millilitre of water that can be picked up by the test. The test needs to be run with only water repeated times to check that there are no “false positive” results.
Next, you take a nose or throat swab from a Covid-19 negative individual and “spike” the sample with some of the artificial genetic material from coronavirus, in order to check that the swab or the secretions that are collected do not interfere with the test. Again, multiple tests are done with varying amounts of genetic material in the “spiked” sample.
The next step is to access a bank of swabs from patients known to be positive and negative in what’s known as a “gold standard” lab test, such as the tests developed by Public Health England that can quantify the amount of virus present in a person with Covid-19 symptoms. Large numbers of these positive samples, perhaps 50 or more, are tested with the new device. It’s just as important to test a large number of samples from non-Covid individuals to check that no “false positives” arise.
The final step is to test the new device in real time, against a gold-standard test. This step is the most challenging as it sometimes requires ethical approval, and must be done in a clinical environment in real time. When we tested the rapid portable device, Samba II, in Cambridge, we tested it against the standard lab test in 170 patients.
This required significant regulatory oversight and planning. We showed that Samba II produced results in two hours and was highly accurate compared to the standard lab test, and within three weeks Addenbrooke’s hospital in Cambridgeshire adopted it. And within two weeks of the end of the study, the results of the lab validation and clinical evaluation studies were put online as a pre-print publication for scrutiny.
The number of new Covid-19 cases across the UK is now relatively low, so obtaining appreciable numbers of positive Covid-19 patients is more difficult for validating new tests. An alternative option would be to validate tests in countries experiencing high infection rates, though this would be time-consuming and complex.
Public funds have been used to purchase these tests, so transparency about their effectiveness is really important. Recent experience where the government purchased millions of sub-optimal antibody tests before they had been robustly evaluated shows the danger of purchasing equipment on the basis of manufacturers’ claims alone.
We may be entering an era of trade-offs, where the accuracy of tests is balanced against the number of tests we can achieve. Many see more testing, done in a decentralised fashion, closer to communities and targeting specific at-risks groups, as the best route out of the Covid-19 crisis. In hospitals and care homes, where a single infection can lead to severe consequences in vulnerable people, we need highly accurate tests. In the community, where largely well people need to be tested repeatedly, a compromise between the accuracy, scale and cost of tests can perhaps be reached.
This article was first published in the Guardian on 5 August.
- Profile: Dr Dami Collier
- Dr Dami Collier