Publishing high quality systematic reviews

In 2016 I was appointed by Environment International (IF 7.94) as the world's first specialist systematic review editor in the field of environmental health. My role is to ensure that the systematic reviews and evidence maps which we publish provide high-quality summaries of what is already known in answer to a research question. We set a challenging standard for authors to meet; the most important part of my work is communicating the value of these standards to authors and implementing submission workflows which help researchers meet them.

Transparent, Educational Triage

I created the CREST-Triage tool to facilitate consistent, transparent editorial decision-making for systematic review submissions. It helps editors ensure that a SR submission meets their minimum expectations, improves the efficiency of the peer-review process, and provides structured feedback to authors as to the editor's expectations about SR submissions.

Deep Author Engagement via Special Issues

Special Issues are valuable because they provide a focused collection of publications on a common theme. When a single group puts together a multi-paper issue, it also provides an opportunity for the sort of sustained engagement which can yield a step-change in research capacity for a research team.

I have edited five Special Issues for Environment International.

  • WHO Global Air Quality Guidelines (in preparation)

  • US EPA systematic reviews of potential human health outcomes associated with exposure to phthalates (link) . First set of EPA IRIS program SRs.

  • GRADE Guidelines for Rating Quality of Evidence in Environmental Health Research (link). Mechanism for publication of application of GRADE Framework to environmental health topics.

  • WHO/ILO work-related burden of disease and injury: systematic reviews (link). First-ever use of SR methods by WHO/ILO for estimating burden of disease from an environmental challenge; first-ever use of SR protocols by WHO/ILO.

  • Systematic Review Methods for Advancing Chemical Risk Assessment (link): First ever Special Issue on topic of applying SR methods to chemical risk assessment.

Editorial Involvement in SR Planning

Most systematic reviews are rejected for reasons which could easily have been fixed, if only editors and peer-reviewers had been involved in the planning stage of a project and not just been the recipients of a final manuscript.

Environment International was therefore the first environmental health journal to adopt the Registered Reports model of research publication for systematic reviews, encouraging authors to submit a draft protocol, which we then review. Once the protocol is accepted, we publish it as a full manuscript, and in principle accept the final SR so long as it adheres to the protocol - whatever its findings.

Enforcement of Reporting Standards

If journals only recommend that authors use a reporting standard (such as PRISMA or ROSES), rather than enforce their use, then the standard seems to have minimal impact on the quality of the SRs which the journal is publishing. This is why Environment International is not only one of the three environmental health journals so far to endorse PRISMA; as far as we are aware, we are the only one which requires authors to submit a PRISMA or ROSES report at point of submission.

Enhanced Peer-Review

We have introduced a policy of assigning four peer-reviewers per systematic review submission (two topic experts and two methods experts, unless one reviewer covers both) to try and maximise the likelihood that all important issues with a submitted manuscript will be noticed and corrected.

I am also working on a critical appraisal tool, CREST-SR, to facilitate comprehensive, exacting and transparent peer-review of SR manuscripts.