Crochet robot, by Erinn Simon - Flickr - CC BY-ND

Evidence synthesis in environmental health

My main area of research is adaptation of systematic review methods from clinical to environmental health and toxicological contexts. I work on best practice standards for conduct of systematic reviews and the development of critical appraisal tools for assessing the quality of primary and secondary research - in particular, how such tools can be applied in manuscript production workflows to raise publishing standards.

I recently began moving into the field of artificial intelligence and machine learning for automated evidence synthesis, focusing on improved automated data extraction methods and developing knowledge-based systems for systematic evidence mapping. These are technologies which would address current challenges around the scalability of SR methods, enabling use of the specific type of big data analytics and machine learning methods for the complex analytical challenges which are a defining characteristic of SRs in the environmental health field.

Research Themes

Evidence Maps and Automating Systematic Review

Evidence maps are a novel evidence synthesis product which allow systematic identification of gaps and gluts in decision-critical evidence. However, they only really work if we can move our vast store of scientific information from unstructured archives of PDFs into queryable databases. This requires us to teach machines to "read" scientific documents, develop semantic authoring tools for writing scientific studies, and create comprehensive research metadata records in suitable underlying knowledge organisation systems. If we do this, we could move away from manual systematic reviews into more automated methods of synthesising evidence.

Best Practices in Systematic Review

I work on developing and promoting best practice standards for conduct of systematic reviews and evidence maps in the environmental health sciences. This mainly involves adapting existing standards in use in biomedical research to the toxicology and environmental health context. The idea is that, if we can provide environmental health researchers with an authoritative recipe-book for conducting systematic reviews, it should help them reduce the risk of conducting unsuccessful SR projects.

Critical Appraisal Tools

While there are a lot of critical appraisal tools for helping people distinguish more credible studies from less, it is not always clear which tool should be used and when, nor if any given tool is actually fit-for-purpose. I am working on several tools for critically appraising systematic reviews and primary studies, concentrating on ensuring they target an appropriate quality construct for the context in which they are to be used, that are reasonably intuitive, and they reliably distinguish higher-quality from lower-quality research.

Peer-reviewed journal articles

Whaley, P. et al. (2021) 'Improving the quality of toxicology and environmental health systematic reviews: What journal editors can do' ALTEX

Sharma et al. (2021) 'Towards guidelines for time-trend reviews examining temporal variability in human biomonitoring data of pollutants' Environment International

de Vries et al. (2021) 'Applying evidence-based methods to the development and use of adverse outcome pathways.' ALTEX

Whaley P. et al. (2020) Knowledge organization systems for systematic chemical assessments.' Environmental Health Perspectives

Whaley P. et al. (2020) 'Recommendations for conduct of systematic reviews in toxicology and environmental health research (COSTER)' Environment International

Robinson, C. et al. (2020) ‘Achieving a high level of protection from pesticides in Europe: Problems with the current risk assessment procedure and solutions’ European Journal of Risk Regulation

Wolffe, T. et al. (2020) ‘A survey of systematic evidence mapping practice and the case for knowledge graphs in environmental health & toxicology’ Toxicological Sciences doi:

Wittwehr, C. et al. (2020) ‘Artificial Intelligence for Chemical Risk Assessment’, Computational Toxicology doi:

Wolffe, T. A. M., Whaley, P., et al. (2019). 'Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management.' Environment International. doi:

Hartung, T. et al. (2019) ‘Toward Good In Vitro Reporting Standards.’, ALTEX, doi:

Morgan, R. L. et al. (2019) ‘A risk of bias instrument for non-randomized studies of exposures: A users’ guide to its application in the context of GRADE’, Environment International. doi:

Morgan, R. L. et al. (2018) ‘Identifying the PECO: A framework for formulating good questions to explore the association of environmental and other exposures with health outcomes’, Environment International. doi:

Haddaway, N. R. et al. (2018) ‘ROSES Reporting standards for Systematic Evidence Syntheses: Pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps’, Environmental Evidence. doi:

Whaley, P. & Wikoff, D. (2018) 'Qualitative methods for integrating evidence within and across evidence streams for hazard identification' EFSA Supporting Publications. doi:

Hoffmann, S. et al. (2017) ‘A primer on systematic reviews in toxicology’, Archives of Toxicology. doi:

Vandenberg, L. N. et al. (2016) ‘A proposed framework for the systematic review and integrated assessment (SYRINA) of endocrine disrupting chemicals’, Environmental Health. doi:

Whaley, P. et al. (2016) ‘Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations’, Environment International. doi:


Robinson, C. et al. (2018) Ensuring a higher level of protection from pesticides in Europe: The problems with current pesticide risk assessment procedures in the EU - and proposed solutions. PAN Europe. doi:

Whaley, P. and Halsall C. (2014) Identifying and Monitoring Pollution Risks from On-Shore Hydraulic Fracturing Operations. Lancaster University and Inteb Ltd.

Whaley, P. (2014) EFSA's Draft 2014 Scientific Opinion on the risks to public health related to the presence of bisphenol-A (BPA) in foodstuffs: a critical appraisal. Policy from Science Project

Whaley, P. (2013) Systematic Review and the Future of Evidence in Chemicals Policy. Policy from Science Project


Pega et al. (2021) Systematic reviews and meta-analyses for the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury. Environment International

Radke, E. G. et al. (2020). Application of US EPA IRIS systematic review methods to the health effects of phthalates: Lessons learned and path forward. Environment International. doi:

Whaley, P. and Halsall, C. (2016) ‘Assuring high-quality evidence reviews for chemical risk assessment: Five lessons from guest editing the first environmental health journal special issue dedicated to systematic review’, Environment International. doi:

Whaley, P. et al. (2016) ‘Raising the standard of systematic reviews published in Environment International’, Environment International. doi:


Haddaway, N. R. et al. (2018) ‘Response to “Every ROSE has its thorns”’, Environmental Evidence. doi:

Page, J. et al. (2012) ‘Priorities for cancer prevention’, The Lancet Oncology, 13(6). doi:

Book Chapters

Whaley, P. (2014) ‘Childhood obesity and the environment’. in D Haslam, A Sharma & C le Roux (eds), Controversies in obesity. Springer, pp. 97-102.


de Vries, R.B.M. & Whaley, P.A. (2018) In Vitro Critical Appraisal Tool (IV-CAT): Tool Development Protocol. Evidence-Based Toxicology Collaboration doi:

Conferences and Workshops


The results of a survey of current practices in systematic evidence maps. (6 August 2020) EBTC Scientific Advisory Council


The concept of "biological plausibility" in environmental health systematic reviews (17 June 2020) GRADE Working Group Annual Meeting - Large Group Session


How certain are we? Progress in adapting systematic review methods to environmental health assessments (15 June 2020) Evidence-Based Toxicology Collaboration Satellite Workshop to the GRADE Working Group Annual Meeting


Certainty in Adverse Outcome Pathways (17 June 2020) GRADE Working Group Annual Meeting - Small Group Session


Research Quality and Peer Review Workshop. (20 June 2020) Sense About Science "Voice of Young Science" Network.


Knowledge from Information: Bioinformatics as the Science of the Anthropocene. (14 February) AAAS American Association for the Advancement of Science Annual Meeting 2020. Seattle, USA

Symposium. Convener, Speaker

BBC/UKRI: The Future of Artificial Intelligence. (14 February) AAAS American Association for the Advancement of Science Annual Meeting 2020. Seattle, USA


Applying AI to solve problems more complex than people can understand: the epistemological and ethical implications of "machine knowing" COGITO at Glasgow University, UK

Speaker. Cancelled due to COVID-19

Conduct and reporting standards for systematic reviews in toxicology and risk assessment. EUROTOX 2020

Speaker. Cancelled due to COVID-19

Systematic reviews and chemical risk assessment: current challenges, and the need for AI in overcoming them EUROTOX 2020

Speaker. Cancelled due to COVID-19


Procedures for reporting and appraising systematic reviews (12 Nov) US EPA Systematic Review Community of Practice Webinar


Overview of SR editorial and peer-review processes for the Editorial Board at Environment International (20 Nov) Environment International Editorial Board Webinar


A Journal Editors’ Guide to Successful Systematic Reviews: Common Challenges Faced by Submitting Authors, and How to Address Them. (25 August 2019) ISEE 2019, Utrecht, NL

Workshop. Organiser and speaker

International Editor’s Workshop on raising the standard of systematic reviews published in environmental health and toxicology journals. (29-31 May 2019) Research Triangle Park, NC, USA

Workshop. Organiser

Practical challenges in assessing the external validity of individual studies, and the implications for integrating multiple streams of evidence in systematic reviews. (2 June 2019) US National Academies of Sciences Workshop on Evidence Integration in Chemical Assessments. Washington DC, USA


Machine Learning and Knowledge Systems for the Anthropocene. (11 January 2019) Lancaster Environment Centre Winter Conference. Lancaster, UK.


SR submission procedures at Environment International (11 April) Second Meeting of Lead Reviewers of Systematic Reviews for the WHO/ILO Joint Methodology for Estimating the Work-Related Burden of Disease and Injury. EU OSHA, Bilbao, Spain

Speaker, Trainer

EBTC GRADE Working Group Pre-Meeting: Systematic Methods for Identification, Appraisal and Integration of Mechanistic Data in Systematic Reviews. (11 June) McMaster University, Canada.


EBTC/EFSA International Workshop on Systematic Review Methods and Adverse Outcome Pathways. (2-3 October) Parma, Italy

Group Lead, Facilitator

Lancaster University (training seminar) Authorship Seminar for Early Career Researchers: An Editor’s Dos and Don’ts. Elsevier Early Career Researcher Seminar at Lancaster University. Lancaster, UK


Continuing Education Session PM11: Conducting Systematic Review in Toxicology—Why, When, How? SOT 2019 (Baltimore, MD, USA) Contributing training presentation: “Reporting and Reviewing Systematic Reviews.” General toxicology audience.



WHO/ILO Systematic Reviews reinforce the value of formal pre-publication of SR protocols. (4 December 2018) Navigation Guide Working Group Quarterly Webinar.


Keynote: Systematic Review, Machine Learning and the Liberation of Knowledge from Information in Environmental Health Research. (15 October 2018) Research Triangle Environmental Health Collaborative Summit on Artificial Intelligence in Environmental Health. Research Triangle Park, NC, USA.


Systematic Reviews, Evidence Maps and a Need for Interactive Data Visualization in Environmental Health Research. (25 August 2018) iLab Calgary Invited Speaker Seminar. University of Calgary, Canada.


From gatekeeper to midwife: how journals can improve the quality of published systematic reviews. (19 April 2018) Collaboration for Environmental Evidence 2nd International Conference. Paris, France.


Areas in which AI could contribute to the conduct of systematic reviews. (3 April 2018) AI4CRA Meeting. EU Joint Researcher Centre, Ispra, Italy.


OECD Level 1 Workshop on Systematic Review of Existing Data of Endocrine Disruptors. (25 October 2018) OECD, Paris, France.

UK Expert Representative

IV International Congress on Safety and Health at Work (Bilbao, Spain). Training on systematic approaches to Assessment of Scientific Evidence session.


Elsevier Researcher Academy (webinar). Education webinar Systematic Reviews 101. Introduction to SRs, editorial handling thereof, and advice on successful SR project planning. Varied early-career audience. [Link]


EU Joint Research Centre (Ispra, Italy). 2-day training on how to conduct and appraise SRs, with a focus on value for research commissioners

Organiser, Speaker


Gatekeeper and midwife: the two roles of the academic editor. (17 December 2017) Lancaster University “New Café”, Lancaster, UK


Ensuring only high-quality systematic reviews get published: New obligations and strategies for environmental health journals (23 November 2017) World Health Organization Global Burden of Disease Estimation Planning Meeting, Zurich, Switzerland.


Invited expert stakeholder: Comments on EFSA’s draft BPA hazard assessment protocol. (12 September 2017) European Food Safety Authority Meeting on Draft BPA Hazard Assessment Protocol. Brussels, Belgium.


Ensuring only high-quality systematic reviews get published: New obligations and strategies for environmental health journals (19 August 2017) US National Toxicology Program 4th International Symposium on Systematic Review and Meta-Analysis of Laboratory Animal Studies. Research Triangle Park, NC, USA.


The WHO Chemical Risk Assessment Network Systematic Review Framework Document. (21 June 2017) Second Meeting of the WHO Chemical Risk Assessment Network. Parma, Italy.


Appraisal of study quality in environmental health systematic reviews should be based on risk of bias and eschew scores and scales (Zurich, Switzerland). World Health Organization Global Burden of Disease Estimation Planning Meeting.


Introduction to systematic review of toxicological data (Parma, Italy). WHO Chemical Risk Assessment Network.


Systematic review methods for environmental health research. (Debrecen, Hungary) DIMOPEX Training School. Debrecen, Hungary.


Developing Environmental Standards. (London) Sustainability Network for Standardisation. London, UK.



International Expert Workshop: Defining best-practice standards for planning and conducting environmental health and toxicology systematic reviews. (1-2 December 2016) Royal Society of Chemistry, London, UK

Workshop. Organiser

Panel Session on “Systematic reviews and preventing the misuse of Bradford Hill criteria” (16 October 2016) Austin Bradford Hill Anniversary Symposium. Royal Society of Medicine, London, UK.

Session Chair

Themed Session “How to Ensure Only High-Quality Systematic Reviews Get Published: Strategies from Medical and Chemical Risk Research”. (16 August 2016) Collaboration for Environmental Evidence 1st International Conference. Stockholm, Sweden.

Convener, Speaker

Improving the quality of systematic evidence synthesis in toxicology. (16 August 2016) Collaboration for Environmental Evidence 1st International Conference. Stockholm, Sweden.


Systematic review as an approach to advancing the transparency and scientific robustness of chemical risk assessment. European Parliament ECR Group



Innovation vs the Precautionary Principle and avoiding market entrenchment of problem products. (15 June 2016) Cereals 2016. Cambridge, UK


Contemporary reporting practices in environmental health systematic reviews (15 December 2015) Navigation Guide Working Group Webinar


Systematic review methods for chemical risk assessment (22 June 2015) Precaution and Innovation Workshop, Brunel University, London


Discussion Leader with Sir Dick Taverne QC. “Better science for better EU regulation.” (26 February 2015) Centre for European Policy Studies Ideas Lab. Brussels, Belgium.



International Expert Workshop: Systematic review methods for chemical risk assessment: challenges and opportunities. (14 November 2014) Royal Society of Chemistry, London, UK

Workshop. Organiser

European Food Safety Authority Scientific Opinions on Bisphenol-A: a critical appraisal (3 December 2014) Navigation Guide Working Group Webinar


EFSA’s Opinions on risks to health posed by BPA: methodological considerations (11 Sept 2014) European Network of Scientists for Social and Environmental Responsibility Annual Meeting. Berlin, Germany


Literature review in toxicological research and chemical risk assessment: the state of the science (13 May 2014) Poster Spotlight presentation during session “Persistent, Bioaccumulative and Toxic (PBT) substances – identification, assessment and regulatory decision making with a special focus on socio-economic aspects”


Policy from Science Project Literature Review Appraisal Toolkit (12 February 2014) Highwire Coffee Club, Lancaster University, UK



Systematic review and the future of evidence in chemicals policy (15 October 2013) Report and campaign launch event at European Parliament with Corinne LePage MEP, Brussels, Belgium.

European Parliament hosted event. Organiser, Speaker

Should I drink out of this? (5 December 2013) SR+ Seminar, Lancaster University


We need to talk about literature reviews (29 November 2013) Postgraduate Seminar, Lancaster University


The Policy from Science Project (19 June 2013) EEB Chemicals Working Group Meeting, Brussels, Belgium


How to evaluate study credibility: the example set by evidence-based medicine (11 June 2013) Royal Society of Chemistry Environmental Health and Safety Committee AGM. RSC, London, UK


How to evaluate study credibility: the example set by evidence-based medicine (12 May 2013) SETAC Europe, Glasgow, Scotland