On 27 January 2022, Gill Leng replied to my email of 31 December 2021. I have posted her reply below. Her understanding of the key issues I raise is probably greater than her reply would suggest. This selective understanding avoided addressing the issues I am most concerned about.
Firstly, that NICE has entered into a collaboration agreement with Cochrane
…to facilitate the sharing of literature searches, unpublished extracted data and analyses and assessments (“Unpublished Data”) to inform the development of NICE guidelines and Cochrane Reviews…
Collaboration Agreement between NICE and Cochrane, 24 August 2021
Cochrane has recently refused withdraw its review of exercise for ME/CFS where the authors used GRADE to give a rating of moderate quality evidence for one outcome, and concluded that exercise might be beneficial. NICE reviewers concluded, also using GRADE, that the evidence for the benefit of exercise was either low or very-low quality, and so the Guideline Development Committee reached consensus that it should not be recommended as a treatment. Cochrane and GRADE leaders have publicly criticized NICE for bias, a “disastrous” misapplication of GRADE, and defended the Cochrane review on social media and in influential journals such as The Lancet. Should NICE accept GRADE assessments shared with them by Cochrane when they could fail to downgrade the quality of the evidence appropriately, and result in a treatment being recommended by NICE on the basis of very-low quality evidence of benefit?
The second point which was not addressed by Professor Leng was the shocking revelation that covert messages were sent directly to her and her colleagues to attempt to tamper with an evidence review already signed of by herself and the Executive Board. Perhaps this sort of attempt at corruption is so common, she felt it wasn’t worth mentioning.
Thursday 27 January 2022
Dear Caroline
Thank you for your email regarding the Cochrane/NICE collaboration. I understand that you raise 2 key issues for NICE which I will address in turn.
Firstly, your concern about the ongoing Cochrane review into exercise therapy. It would be inappropriate for me to comment on decisions taken by Cochrane about the continuation of their review. From a broader perspective on NICE’s collaboration with Cochrane, we are developing a Guideline Support Document (GSD). This will be an operational guide that broadly aims to support the use of Cochrane reviews and topic expertise within Cochrane Review Groups in the development of NICE guidelines. We are currently compiling a working draft of the GSD for inclusion within the 2022 update to the guideline manual, where it will be subject to public consultation later this year.
We recognise the inherent value of reducing duplication of reviewing activity. However, we also accept that there will be occasions where undertaking separate reviews will be beneficial/appropriate. This is supported by the following extract from the working draft of the GSD:
“Discuss and seek agreement from the NICE quality assurance team for the rationale for the exclusion of a Cochrane review, at the earliest opportunity, if the guideline committee is of the view that it is not appropriate for consideration in the guideline and a new review is required. On occasion, it might be appropriate or beneficial for Cochrane and NICE to produce separate systematic reviews on a topic. For example, where differing perspectives (NICE’s UK focus versus Cochrane’s global perspective) may impact on how the question is framed, or where there are particular controversies or uncertainties in a research area and replication of a review could be helpful.“
Secondly, in terms of the quality assurance of Cochrane reviews. As above, we recognise that there will be occasions where replication of a review to ensure NICE’s needs are met will be appropriate (for example, where differences in PICOs, methodological approach etc between NICE and Cochrane might be entirely reasonable). As you say, our guideline development and quality assurance processes are designed to mitigate risk. Importantly, the GSD also provides mechanisms for linking guideline developers, the NICE QA team, Cochrane authors and the Cochrane editorial (QA) team for surfacing and understanding points of divergence which will be key in informing ongoing development of systematic reviewing and guideline development methods and processes.
Yours sincerely,
Professor Gillian Leng CBE
Chief executive
National Institute for Health and Care Excellence
2nd Floor, 2 Redman Place
London
E20 1JQ