Below is a slightly abridged version of a proposal for collaboration between EQUATOR and Cochrane made in September last year. It is an idea which is not dependant on Cochrane’s involvement, but it might be a good way for Cochrane to make progress on its stated goal to advocate for evidence, and improve the primary research on which its reviews are based.
Turning the (risk of bias) tables green
Advocating for evidence is one of the four key goals of Cochrane’s strategy to 2020. One of the steps identified for achieving this goal is “to promote reliable high-quality primary research that is prioritised to answer real-world health questions and improves the evidence base on which [Cochrane’s] work is based.”
Only 14% of the studies included in Cochrane reviews are reliable high-quality primary research, indicated by scoring all green “low risk of bias” ratings in the risk of bias table. A staggering 86% of included studies have problems with design (high risk of bias) or reporting (unclear risk of bias). Doubts about the reliability of many included studies means reviews are often of little practical use to patients and clinicians making decisions, can’t be used to change policy, and can’t contribute evidence for clinical guidelines.
In an attempt to have their reviews contribute in other ways, some review teams go into detail in the “Implications for research” section about how future primary research should be designed and reported. Many simply state that more and better research is needed.
To help expand the impact of its reviews, Cochrane is considering an advocacy campaign targeting funders, making clear the areas where more and better research is needed to influence funding agendas. This advocacy campaign has the potential to influence funders beyond simply what should be funded, but also how the funded research should be designed and reported to ensure it can contribute to future reviews and reduce uncertainty.
The EQUATOR Network brings a reputation in the research integrity world, extensive knowledge on complete reporting of health research using reporting guidelines, and training expertise in planning, design and analysis, and reporting. An EQUATOR-Cochrane partnership has the potential to maximise the impact and reach of Cochrane’s advocacy campaign.
The campaign could alert funders to known gaps in the literature and to the scale of the design and reporting problems in the research they already fund to fill those gaps. It will give funders actionable information that they can use to ensure that the research they fund in the future is needed and usable. Cochrane, as the evidence synthesis expert, will provide information from reviews on what good design should look like to answer the review question. EQUATOR, as the reporting expert, will provide information on how to ensure complete and transparent reporting.
A logical extension of the proposed advocacy campaign is for Cochrane to create a clear well-structured primary research “prescription” alongside every systematic review. These pre-protocols would formally report the design and methods decisions required to ensure the research question can be answered and bias is minimised. Primary researchers would be able to use these documents to guide their study design and full protocol development. As a result, every review with uncertain findings due to few or poorly conducted studies, and every empty review on a high-priority question would still have constructive usable output.
This step would formalise a process that already occurs under the best circumstances: Ian Roberts responded to the uncertainty revealed in his 2000 Cochrane review of the use of steroids in traumatic brain injury with the CRASH trial, which showed that steroids increased mortality in this scenario.
EQUATOR has recently developed an online tool called GoodReports which supports clear and complete reporting by making reporting checklists easier to find and easier to use. The tool won the Cochrane-REWARD prize in 2018. We are now developing it to allow users to generate article templates. An EQUATOR-Cochrane template tool for use by Cochrane reviewers could generate standardised primary research design templates that could be shared with funders. In turn, by sharing these with their grantees, funders will be assured that the research they have funded is well-designed and of clear relevance to patients and carers. Funders could also use them to monitor and follow up adherence with research teams.
Cochrane’s proposed advocacy campaign also offers scope for patient and public involvement in developing all of the proposed materials. Involvement of Cochrane consumers in this campaign could ensure that the research questions proposed to funders have been prioritised by patients and carers, measure patient-important outcomes, include adequate measurement and reporting of harms, and have the optimal design and sample size to identify if there is clinically significant benefit. Existing resources for enabling consumer involvement could be extended to include ways review teams could involve them in designing future high-quality studies.
In summary, an EQUATOR-Cochrane partnership could create an advocacy campaign and tools designed to promote high-quality primary research with low risk of bias that can be integrated into future Cochrane reviews.
1 comment