Research

Standards, methods, and guidance for evidence synthesis in environmental health

My main area of research is adaptation of systematic review methods from clinical to environmental health and toxicological contexts. I work on best practice standards for conduct of systematic reviews and the development of critical appraisal tools for assessing the quality of primary and secondary research - in particular, how such tools can be applied in manuscript production workflows to raise publishing standards.

I also working on methods for computable research, that would allow data about study methods and findings to be automatically added to research databases. This could enable a step-change in research accessibility, taking data out from behind journal paywalls, allowing automated systematic review, and enabling full access to the wealth of scientific data humans generate but now exceeds our capacity to review.

Research Themes

Evidence Mapping and Computable Research

Evidence maps are a novel evidence synthesis product which allow systematic identification of gaps and gluts in decision-critical evidence. However, they only really work if we can move our vast store of scientific information from unstructured archives of PDFs into queryable databases. This requires us to teach machines to "read" scientific documents, develop semantic authoring tools for writing scientific studies, and create comprehensive research metadata records in suitable underlying knowledge organisation systems. If we do this, we could move away from manual systematic reviews into more automated methods of synthesising evidence.

Best Practice in Systematic Review

I work on developing and promoting best practice standards for conduct of systematic reviews and evidence maps in the environmental health sciences. This mainly involves adapting existing standards in use in biomedical research to the toxicology and environmental health context. The idea is that, if we can provide environmental health researchers with an authoritative recipe-book for conducting systematic reviews, it should help them reduce the risk of conducting unsuccessful SR projects.

Critical Appraisal Guidance and Instruments

While there are a lot of critical appraisal tools for helping people distinguish more credible studies from less, it is not always clear which tool should be used and when, nor if any given tool is actually fit-for-purpose. I am working on or contributing to several tools for critically appraising systematic reviews and primary studies, concentrating on ensuring they target an appropriate quality construct for the context in which they are to be used, that are reasonably intuitive, and they reliably distinguish higher-quality from lower-quality research.

Peer-reviewed journal articles

Hoffmann et al. (2022) Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks. ALTEX. https://doi.org/10.14573/altex.2202141

Frampton et al. (2022) Principles and framework for assessing the risk of bias for studies included in comparative quantitative environmental systematic reviews. Environmental Evidence. https://doi.org/10.1186/s13750-022-00264-0

Wilkins et al. (2022) Assessing author willingness to enter study information into structured data templates as part of the manuscript submission process: A pilot study. Heliyon. https://doi.org/10.1016/j.heliyon.2022.e09095

Whaley et al. (2022) Biological plausibility in environmental health systematic reviews: a GRADE concept paper. J. Clin. Epi. https://doi.org/10.1016/j.jclinepi.2022.02.011

Hoffmann et al. (2022) How evidence-based methodologies can help identify and reduce uncertainty in chemical risk assessment. ALTEX https://doi.org/10.14573/altex.2201131

Verbeek et al. (2021) An approach to quantifying the potential importance of residual confounding in systematic reviews of observational studies: A GRADE concept paper Environment International https://doi.org/10.1016/j.envint.2021.106868

Whaley et al. (2021) 'Improving the quality of toxicology and environmental health systematic reviews: What journal editors can do' ALTEX https://doi.org/10.14573/altex.2106111

Sharma et al. (2021) 'Towards guidelines for time-trend reviews examining temporal variability in human biomonitoring data of pollutants' Environment International https://doi.org/10.1016/j.envint.2021.106437

de Vries et al. (2021) 'Applying evidence-based methods to the development and use of adverse outcome pathways.' ALTEX https://doi.org/10.14573/altex.2101211

Whaley P. et al. (2020) Knowledge organization systems for systematic chemical assessments.' Environmental Health Perspectives https://doi.org/10.1289/EHP6994

Whaley P. et al. (2020) 'Recommendations for conduct of systematic reviews in toxicology and environmental health research (COSTER)' Environment International https://doi.org/10.1016/j.envint.2020.105926

Robinson, C. et al. (2020) ‘Achieving a high level of protection from pesticides in Europe: Problems with the current risk assessment procedure and solutions’ European Journal of Risk Regulation https://doi.org/10.1017/err.2020.18

Wolffe, T. et al. (2020) ‘A survey of systematic evidence mapping practice and the case for knowledge graphs in environmental health & toxicology’ Toxicological Sciences doi: https://doi.org/10.1093/toxsci/kfaa025

Wittwehr, C. et al. (2020) ‘Artificial Intelligence for Chemical Risk Assessment’, Computational Toxicology doi: https://doi.org/10.1016/j.comtox.2019.100114

Wolffe, T. A. M., Whaley, P., et al. (2019). 'Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management.' Environment International. doi: https://doi.org/10.1016/j.envint.2019.05.065

Hartung, T. et al. (2019) ‘Toward Good In Vitro Reporting Standards.’, ALTEX, doi: https://doi.org/10.14573/altex.1812191

Morgan, R. L. et al. (2019) ‘A risk of bias instrument for non-randomized studies of exposures: A users’ guide to its application in the context of GRADE’, Environment International. doi: https://doi.org/10.1016/j.envint.2018.11.004

Morgan, R. L. et al. (2018) ‘Identifying the PECO: A framework for formulating good questions to explore the association of environmental and other exposures with health outcomes’, Environment International. doi: https://doi.org/10.1016/j.envint.2018.07.015

Haddaway, N. R. et al. (2018) ‘ROSES Reporting standards for Systematic Evidence Syntheses: Pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps’, Environmental Evidence. doi: https://doi.org/10.1186/s13750-018-0121-7

Whaley, P. & Wikoff, D. (2018) 'Qualitative methods for integrating evidence within and across evidence streams for hazard identification' EFSA Supporting Publications. doi: https://doi.org/10.2903/sp.efsa.2018.EN-1396

Hoffmann, S. et al. (2017) ‘A primer on systematic reviews in toxicology’, Archives of Toxicology. doi: https://doi.org/10.1007/s00204-017-1980-3

Vandenberg, L. N. et al. (2016) ‘A proposed framework for the systematic review and integrated assessment (SYRINA) of endocrine disrupting chemicals’, Environmental Health. doi: https://doi.org/10.1186/s12940-016-0156-6

Whaley, P. et al. (2016) ‘Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations’, Environment International. doi: https://doi.org/10.1016/j.envint.2015.11.002

Reports

US National Academies of Sciences, Engineering and Medicine (2022) Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments. https://doi.org/10.17226/26289

WHO Chemical Risk Assessment Network (2021) Framework for the use of systematic review in chemical risk assessment. https://apps.who.int/iris/handle/10665/347876

Robinson, C. et al. (2018) Ensuring a higher level of protection from pesticides in Europe: The problems with current pesticide risk assessment procedures in the EU - and proposed solutions. PAN Europe. doi: https://doi.org/10.5281/zenodo.2543743

Whaley, P. and Halsall C. (2014) Identifying and Monitoring Pollution Risks from On-Shore Hydraulic Fracturing Operations. Lancaster University and Inteb Ltd.

Whaley, P. (2014) EFSA's Draft 2014 Scientific Opinion on the risks to public health related to the presence of bisphenol-A (BPA) in foodstuffs: a critical appraisal. Policy from Science Project

Whaley, P. (2013) Systematic Review and the Future of Evidence in Chemicals Policy. Policy from Science Project

Editorials

Pega et al. (2021) Systematic reviews and meta-analyses for the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury. Environment International https://doi.org/10.1016/j.envint.2021.106605

Radke, E. G. et al. (2020). Application of US EPA IRIS systematic review methods to the health effects of phthalates: Lessons learned and path forward. Environment International. doi: https://doi.org/10.1016/j.envint.2020.105820

Whaley, P. and Halsall, C. (2016) ‘Assuring high-quality evidence reviews for chemical risk assessment: Five lessons from guest editing the first environmental health journal special issue dedicated to systematic review’, Environment International. doi: https://doi.org/10.1016/j.envint.2016.04.016

Whaley, P. et al. (2016) ‘Raising the standard of systematic reviews published in Environment International’, Environment International. doi: https://doi.org/10.1016/j.envint.2016.08.007

Correspondence

Verbeek et al. (2022) Potential importance of residual confounding in systematic reviews of observational studies: Answer to Mathur and VanderWeele. Environment International. https://doi.org/10.1016/j.envint.2021.107010

Haddaway, N. R. et al. (2018) ‘Response to “Every ROSE has its thorns”’, Environmental Evidence. doi: https://doi.org/10.1186/s13750-018-0133-3

Page, J. et al. (2012) ‘Priorities for cancer prevention’, The Lancet Oncology, 13(6). doi: https://doi.org/10.1016/S1470-2045(12)70140-7

Book Chapters

Whaley, P. (2014) ‘Childhood obesity and the environment’. in D Haslam, A Sharma & C le Roux (eds), Controversies in obesity. Springer, pp. 97-102.

Protocols

Menon et al. (2020) A Survey of the Basic Scientific Quality of Systematic Reviews in Environmental Health. https://osf.io/mnpvs

de Vries, R.B.M. & Whaley, P.A. (2018) In Vitro Critical Appraisal Tool (IV-CAT): Tool Development Protocol. Evidence-Based Toxicology Collaboration doi: https://doi.org/10.5281/zenodo.1493498

Conferences, Workshops, Presentations

2022

Study quality assessment methods in the draft TSCA SR protocol (13 April 2022) US EPA Scientific Advisory Committee on Chemicals

Speaker

IV-CAT: a tool for facilitating peer review of in vitro studies (31 March 2022) Society of Toxicology

Speaker

Four Tips for Success with Evidence Reviews: An Editor’s Strategy (27 March 2022) Society of Toxicology

Speaker

Systematic review: why, what, and how? (22 March 2022) US FDA Center for Food Safety and Applied Nutrition

Speaker

Ontologies for toxicology and environmental health (16 March 2022) EBTC Scientific Symposium

Organiser, Speaker, Moderator

Recommendations for the conduct of systematic reviews in toxicology and environmental health research (3 March 2022) Lancaster University

Speaker

How to conduct high-quality systematic reviews (11 Feb 2022) University of New England, Australia

Speaker

2021

Environment International’s editorial processes for handling the WHO EMF Systematic Reviews (17 December 2021) Navigation Guide Working Group Webinar

Speaker

From in here to out there: the fascinating challenge of standardizing toxicity assays (14 December 2021) EBTC Scientific Symposium

Moderator

EBTC 10-Year Anniversary Symposium (11 November 2021) EBTC Scientific Symposium

Organiser

Submitting and peer-reviewing research papers: an editor’s perspective (22 October 2021) Sense About Science VOYS Workshop

Panellist

Recommendations for the conduct of systematic reviews in toxicology and environmental health research (30 September 2021) BioEM

Speaker

Research Standards for Systematic Reviews, and How to Use Them (27 September 2021) EUROTOX

Speaker

Systematic reviews and chemical risk assessment: current challenges and the need for AI in overcoming them (27 September 2021) EUROTOX

Speaker

Maximizing the value of peer review in toxicology (4 August 2021) Toxicology Forum Summer Meeting

Speaker

Overcoming data availability obstacles in the way of evidence-based toxicology (25 June 2021) EBTC Scientific Symposium

Organiser, Moderator

Overcoming the “streetlight effect” in evidence synthesis (4 May 2021) ONTOX Kick-Off Meeting

Speaker

Application of US EPA IRIS systematic review methods to health effects of phthalate exposures (21 Feb 2021) EBTC Scientific Symposium

Organiser, Moderator

2020

The results of a survey of current practices in systematic evidence maps. (6 August 2020) EBTC Scientific Advisory Council

Speaker

The concept of "biological plausibility" in environmental health systematic reviews (17 June 2020) GRADE Working Group Annual Meeting - Large Group Session

Speaker

How certain are we? Progress in adapting systematic review methods to environmental health assessments (15 June 2020) Evidence-Based Toxicology Collaboration Satellite Workshop to the GRADE Working Group Annual Meeting

Organiser

Certainty in Adverse Outcome Pathways (17 June 2020) GRADE Working Group Annual Meeting - Small Group Session

Organiser

Research Quality and Peer Review Workshop. (20 June 2020) Sense About Science "Voice of Young Science" Network.

Panellist

Knowledge from Information: Bioinformatics as the Science of the Anthropocene. (14 February) AAAS American Association for the Advancement of Science Annual Meeting 2020. Seattle, USA

Symposium. Convener, Speaker

BBC/UKRI: The Future of Artificial Intelligence. (14 February) AAAS American Association for the Advancement of Science Annual Meeting 2020. Seattle, USA

Panellist

Applying AI to solve problems more complex than people can understand: the epistemological and ethical implications of "machine knowing" COGITO at Glasgow University, UK

Speaker. Cancelled due to COVID-19

Conduct and reporting standards for systematic reviews in toxicology and risk assessment. EUROTOX 2020

Speaker. Cancelled due to COVID-19

Systematic reviews and chemical risk assessment: current challenges, and the need for AI in overcoming them EUROTOX 2020

Speaker. Cancelled due to COVID-19

2019

Procedures for reporting and appraising systematic reviews (12 Nov) US EPA Systematic Review Community of Practice Webinar

Speaker

Overview of SR editorial and peer-review processes for the Editorial Board at Environment International (20 Nov) Environment International Editorial Board Webinar

Speaker

A Journal Editors’ Guide to Successful Systematic Reviews: Common Challenges Faced by Submitting Authors, and How to Address Them. (25 August 2019) ISEE 2019, Utrecht, NL

Workshop. Organiser and speaker

International Editor’s Workshop on raising the standard of systematic reviews published in environmental health and toxicology journals. (29-31 May 2019) Research Triangle Park, NC, USA

Workshop. Organiser

Practical challenges in assessing the external validity of individual studies, and the implications for integrating multiple streams of evidence in systematic reviews. (2 June 2019) US National Academies of Sciences Workshop on Evidence Integration in Chemical Assessments. Washington DC, USA

Speaker

Machine Learning and Knowledge Systems for the Anthropocene. (11 January 2019) Lancaster Environment Centre Winter Conference. Lancaster, UK.

Speaker

SR submission procedures at Environment International (11 April) Second Meeting of Lead Reviewers of Systematic Reviews for the WHO/ILO Joint Methodology for Estimating the Work-Related Burden of Disease and Injury. EU OSHA, Bilbao, Spain

Speaker, Trainer

EBTC GRADE Working Group Pre-Meeting: Systematic Methods for Identification, Appraisal and Integration of Mechanistic Data in Systematic Reviews. (11 June) McMaster University, Canada.

Rapporteur

EBTC/EFSA International Workshop on Systematic Review Methods and Adverse Outcome Pathways. (2-3 October) Parma, Italy

Group Lead, Facilitator

Lancaster University (training seminar) Authorship Seminar for Early Career Researchers: An Editor’s Dos and Don’ts. Elsevier Early Career Researcher Seminar at Lancaster University. Lancaster, UK

Speaker

Continuing Education Session PM11: Conducting Systematic Review in Toxicology—Why, When, How? SOT 2019 (Baltimore, MD, USA) Contributing training presentation: “Reporting and Reviewing Systematic Reviews.” General toxicology audience.

Speaker

2018

WHO/ILO Systematic Reviews reinforce the value of formal pre-publication of SR protocols. (4 December 2018) Navigation Guide Working Group Quarterly Webinar.

Speaker

Keynote: Systematic Review, Machine Learning and the Liberation of Knowledge from Information in Environmental Health Research. (15 October 2018) Research Triangle Environmental Health Collaborative Summit on Artificial Intelligence in Environmental Health. Research Triangle Park, NC, USA.

Speaker

Systematic Reviews, Evidence Maps and a Need for Interactive Data Visualization in Environmental Health Research. (25 August 2018) iLab Calgary Invited Speaker Seminar. University of Calgary, Canada.

Speaker

From gatekeeper to midwife: how journals can improve the quality of published systematic reviews. (19 April 2018) Collaboration for Environmental Evidence 2nd International Conference. Paris, France.

Speaker

Areas in which AI could contribute to the conduct of systematic reviews. (3 April 2018) AI4CRA Meeting. EU Joint Researcher Centre, Ispra, Italy.

Speaker

OECD Level 1 Workshop on Systematic Review of Existing Data of Endocrine Disruptors. (25 October 2018) OECD, Paris, France.

UK Expert Representative

IV International Congress on Safety and Health at Work (Bilbao, Spain). Training on systematic approaches to Assessment of Scientific Evidence session.

Speaker

Elsevier Researcher Academy (webinar). Education webinar Systematic Reviews 101. Introduction to SRs, editorial handling thereof, and advice on successful SR project planning. Varied early-career audience. [Link]

Speaker

EU Joint Research Centre (Ispra, Italy). 2-day training on how to conduct and appraise SRs, with a focus on value for research commissioners

Organiser, Speaker

2017

Gatekeeper and midwife: the two roles of the academic editor. (17 December 2017) Lancaster University “New Café”, Lancaster, UK

Speaker

Ensuring only high-quality systematic reviews get published: New obligations and strategies for environmental health journals (23 November 2017) World Health Organization Global Burden of Disease Estimation Planning Meeting, Zurich, Switzerland.

Speaker

Invited expert stakeholder: Comments on EFSA’s draft BPA hazard assessment protocol. (12 September 2017) European Food Safety Authority Meeting on Draft BPA Hazard Assessment Protocol. Brussels, Belgium.

Speaker

Ensuring only high-quality systematic reviews get published: New obligations and strategies for environmental health journals (19 August 2017) US National Toxicology Program 4th International Symposium on Systematic Review and Meta-Analysis of Laboratory Animal Studies. Research Triangle Park, NC, USA.

Speaker

The WHO Chemical Risk Assessment Network Systematic Review Framework Document. (21 June 2017) Second Meeting of the WHO Chemical Risk Assessment Network. Parma, Italy.

Speaker

Appraisal of study quality in environmental health systematic reviews should be based on risk of bias and eschew scores and scales (Zurich, Switzerland). World Health Organization Global Burden of Disease Estimation Planning Meeting.

Speaker

Introduction to systematic review of toxicological data (Parma, Italy). WHO Chemical Risk Assessment Network.

Speaker

Systematic review methods for environmental health research. (Debrecen, Hungary) DIMOPEX Training School. Debrecen, Hungary.

Speaker

Developing Environmental Standards. (London) Sustainability Network for Standardisation. London, UK.

Speaker

2016

International Expert Workshop: Defining best-practice standards for planning and conducting environmental health and toxicology systematic reviews. (1-2 December 2016) Royal Society of Chemistry, London, UK

Workshop. Organiser

Panel Session on “Systematic reviews and preventing the misuse of Bradford Hill criteria” (16 October 2016) Austin Bradford Hill Anniversary Symposium. Royal Society of Medicine, London, UK.

Session Chair

Themed Session “How to Ensure Only High-Quality Systematic Reviews Get Published: Strategies from Medical and Chemical Risk Research”. (16 August 2016) Collaboration for Environmental Evidence 1st International Conference. Stockholm, Sweden.

Convener, Speaker

Improving the quality of systematic evidence synthesis in toxicology. (16 August 2016) Collaboration for Environmental Evidence 1st International Conference. Stockholm, Sweden.

Speaker

Systematic review as an approach to advancing the transparency and scientific robustness of chemical risk assessment. European Parliament ECR Group

Organiser

2015

Innovation vs the Precautionary Principle and avoiding market entrenchment of problem products. (15 June 2016) Cereals 2016. Cambridge, UK

Panellist

Contemporary reporting practices in environmental health systematic reviews (15 December 2015) Navigation Guide Working Group Webinar

Speaker

Systematic review methods for chemical risk assessment (22 June 2015) Precaution and Innovation Workshop, Brunel University, London

Speaker

Discussion Leader with Sir Dick Taverne QC. “Better science for better EU regulation.” (26 February 2015) Centre for European Policy Studies Ideas Lab. Brussels, Belgium.

Panellist

2014

International Expert Workshop: Systematic review methods for chemical risk assessment: challenges and opportunities. (14 November 2014) Royal Society of Chemistry, London, UK

Workshop. Organiser

European Food Safety Authority Scientific Opinions on Bisphenol-A: a critical appraisal (3 December 2014) Navigation Guide Working Group Webinar

Speaker

EFSA’s Opinions on risks to health posed by BPA: methodological considerations (11 Sept 2014) European Network of Scientists for Social and Environmental Responsibility Annual Meeting. Berlin, Germany

Speaker

Literature review in toxicological research and chemical risk assessment: the state of the science (13 May 2014) Poster Spotlight presentation during session “Persistent, Bioaccumulative and Toxic (PBT) substances – identification, assessment and regulatory decision making with a special focus on socio-economic aspects”

Speaker

Policy from Science Project Literature Review Appraisal Toolkit (12 February 2014) Highwire Coffee Club, Lancaster University, UK

Speaker

2013

Systematic review and the future of evidence in chemicals policy (15 October 2013) Report and campaign launch event at European Parliament with Corinne LePage MEP, Brussels, Belgium.

European Parliament hosted event. Organiser, Speaker

Should I drink out of this? (5 December 2013) SR+ Seminar, Lancaster University

Speaker

We need to talk about literature reviews (29 November 2013) Postgraduate Seminar, Lancaster University

Speaker

The Policy from Science Project (19 June 2013) EEB Chemicals Working Group Meeting, Brussels, Belgium

Speaker

How to evaluate study credibility: the example set by evidence-based medicine (11 June 2013) Royal Society of Chemistry Environmental Health and Safety Committee AGM. RSC, London, UK

Speaker

How to evaluate study credibility: the example set by evidence-based medicine (12 May 2013) SETAC Europe, Glasgow, Scotland

Speaker