Today I wrote to Professor Matt Westmore, Chief Executive of the NHS Health Research Authority, which protects and promotes the interests of patients and the public in health and social care research.
27 July 2021
Dear Professor Westmore
(cc Carol Monaghan, MP Science and Technology Committee)
I am writing in a personal capacity, not as a representative of my employer.
I am following up on a letter to Norman Lamb MP then chair of the Science and Technology Committee from then Chair of the HRA Professor Sir Jonathan Montgomery. In the letter, he concludes the PACE Trial was sound science and the PACE investigators exceeded expectations for transparency.
There were several responses to the letter, including one from John Peters (attached). The HRA declined to respond. This is disappointing given the detailed and reasonable challenges Peters and others have made to statements made in Montgomery’s letter.
Cochrane and the robustness of PACE.
Montgomery states “…if there was a general and consistent view that [PACE] was of poor scientific quality then it would give us cause for concern. This is not the case in relation to PACE. A review of the reception of the study shows indications that the science is sound…results of the trial were published in high impact journals with peer review processes that would generally be regarded as robust …The robustness of the PACE trial has been considered in a Cochrane review that classified it as high quality.”
I am not sure where the assertion that Cochrane classified PACE as high-quality science came from. Cochrane reviews don’t explicitly describe the quality of individual included trials, but they do give them a risk of bias rating. The authors of Exercise therapy for chronic fatigue syndrome rated PACE as at low risk of bias for selective reporting. There are fourteen lengthy critical comments published alongside review which challenge this judgement, alongside many other significant problems with the way the review was conducted. Ultimately, a formal complaint to Cochrane in 2018 led to an internal review where the reviewer agreed with the complainant that PACE should be rated as at high risk of bias for selective reporting.
The amended version of the review, published in October 2019, still rates PACE at low risk of bias for selective reporting, contrary to the recommendations of Cochrane’s own internal review. However, it now has an editorial note on it linking to a news article acknowledging a whole new approach is needed. Cochrane’s Editor-in-Chief says in the news article that the “… amended review is still based on a research question and a set of methods from 2002 and reflects evidence from studies that applied definitions of ME/CFS from the 1990s”.
Most recently the general and consistent view that PACE was poor science has been backed up by NICE. NICE published its draft updated guideline on the diagnosis and management of myalgic encephalomyelitis (or encephalopathy)/chronic fatigue syndrome (ME/CFS) in November 2020. In the evidence review of non-pharmacological treatments the reviewers judged the trial evidence for CBT and Exercise (including that from the PACE trial) as low or very low quality. They also reported they could not use the Cochrane review of Exercise because it did not include all critical outcomes specified in the NICE protocol and included study populations where not all participants had ME/CFS.
NICE has therefore updated its 2007 recommendations on the use of psycho-behavioural treatments for ME/CFS, concluding that Graded Exercise Therapy (GET) should no longer be offered to patients.
In a statement made in August 2018 , Professor Fiona Watt, Chair of the MRC, the major funders of PACE, rejected criticism of the trial. She held Cochrane to be the gold-standard of evidence, using this as part of the defence of PACE as sound science.
Transparency and data sharing
Fiona Watt also said in her statement that the MRC was supporting the PACE triallists to anonymise the PACE study data and make it available via the CSDR web portal with six months. Three years later this has not happened.
Regarding the failure of PACE investigators to share data, Montgomery states that “… looking forward, there is a need for further clarification of transparency requirements, particularly as a result of the second tribunal decision, in which a number of
arguments raised by QMUL were rejected. The HRA will take this forward as part of its work on transparency. It is entirely proper that trials are debated in the scientific community. One of the objectives of transparency regulation is to ensure that this is possible”
Eminent and experienced investigators such as the PACE investigators should not need support from the funder or pressure from a regulator to make data from a publicly funded trial available for independent scrutiny. Claiming inability to anonymise data and spending £250,000 on lawyers to argue against releasing it on spurious grounds does not meet even the lowest standards of competence or transparency.
I wrote to Professor Fiona Watt last week to update her on the status of Cochrane evidence in this area. I also asked for an explanation for the data from PACE still not being available, despite the MRC supporting the investigators to do so by early 2019. Please could you and the MRC work together to deliver on your shared commitment to transparency and ethical standards in research to ensure the full and anonymised data from PACE is made available as soon as possible.
With best wishes
Caroline Struthers
Caroline Struthers
Senior EQUATOR Research Fellow
UK EQUATOR Centre
Centre for Statistics in Medicine
NDORMS – University of Oxford
Botnar Research Centre
Old Road, OX3 7LD
Oxford, United Kingdom