Research
Standards, methods, and guidance for evidence synthesis in environmental health
My main area of research is adaptation of systematic review methods from clinical to environmental health and toxicological contexts. I work on best practice standards for conduct of systematic reviews and the development of critical appraisal tools for assessing the quality of primary and secondary research - in particular, how such tools can be applied in manuscript production workflows to raise publishing standards.
I also working on methods for computable research, that would allow data about study methods and findings to be automatically added to research databases. This could enable a step-change in research accessibility, taking data out from behind journal paywalls, allowing automated systematic review, and enabling full access to the wealth of scientific data humans generate but now exceeds our capacity to review.
Research Themes
Evidence Mapping and Computable Research
Evidence maps are a novel evidence synthesis product which allow systematic identification of gaps and gluts in decision-critical evidence. However, they only really work if we can move our vast store of scientific information from unstructured archives of PDFs into queryable databases. This requires us to teach machines to "read" scientific documents, develop semantic authoring tools for writing scientific studies, and create comprehensive research metadata records in suitable underlying knowledge organisation systems. If we do this, we could move away from manual systematic reviews into more automated methods of synthesising evidence.
Best Practice in Systematic Review
I work on developing and promoting best practice standards for conduct of systematic reviews and evidence maps in the environmental health sciences. This mainly involves adapting existing standards in use in biomedical research to the toxicology and environmental health context. The idea is that, if we can provide environmental health researchers with an authoritative recipe-book for conducting systematic reviews, it should help them reduce the risk of conducting unsuccessful SR projects.
Critical Appraisal Guidance and Instruments
While there are a lot of critical appraisal tools for helping people distinguish more credible studies from less, it is not always clear which tool should be used and when, nor if any given tool is actually fit-for-purpose. I am working on or contributing to several tools for critically appraising systematic reviews and primary studies, concentrating on ensuring they target an appropriate quality construct for the context in which they are to be used, that are reasonably intuitive, and they reliably distinguish higher-quality from lower-quality research.
Peer-reviewed journal articles
Hoffmann et al. (2022) Application of evidence-based methods to construct mechanism-driven chemical assessment frameworks. ALTEX. https://doi.org/10.14573/altex.2202141
Frampton et al. (2022) Principles and framework for assessing the risk of bias for studies included in comparative quantitative environmental systematic reviews. Environmental Evidence. https://doi.org/10.1186/s13750-022-00264-0
Wilkins et al. (2022) Assessing author willingness to enter study information into structured data templates as part of the manuscript submission process: A pilot study. Heliyon. https://doi.org/10.1016/j.heliyon.2022.e09095
Whaley et al. (2022) Biological plausibility in environmental health systematic reviews: a GRADE concept paper. J. Clin. Epi. https://doi.org/10.1016/j.jclinepi.2022.02.011
Hoffmann et al. (2022) How evidence-based methodologies can help identify and reduce uncertainty in chemical risk assessment. ALTEX https://doi.org/10.14573/altex.2201131
Verbeek et al. (2021) An approach to quantifying the potential importance of residual confounding in systematic reviews of observational studies: A GRADE concept paper Environment International https://doi.org/10.1016/j.envint.2021.106868
Whaley et al. (2021) 'Improving the quality of toxicology and environmental health systematic reviews: What journal editors can do' ALTEX https://doi.org/10.14573/altex.2106111
Sharma et al. (2021) 'Towards guidelines for time-trend reviews examining temporal variability in human biomonitoring data of pollutants' Environment International https://doi.org/10.1016/j.envint.2021.106437
de Vries et al. (2021) 'Applying evidence-based methods to the development and use of adverse outcome pathways.' ALTEX https://doi.org/10.14573/altex.2101211
Whaley P. et al. (2020) Knowledge organization systems for systematic chemical assessments.' Environmental Health Perspectives https://doi.org/10.1289/EHP6994
Whaley P. et al. (2020) 'Recommendations for conduct of systematic reviews in toxicology and environmental health research (COSTER)' Environment International https://doi.org/10.1016/j.envint.2020.105926
Robinson, C. et al. (2020) ‘Achieving a high level of protection from pesticides in Europe: Problems with the current risk assessment procedure and solutions’ European Journal of Risk Regulation https://doi.org/10.1017/err.2020.18
Wolffe, T. et al. (2020) ‘A survey of systematic evidence mapping practice and the case for knowledge graphs in environmental health & toxicology’ Toxicological Sciences doi: https://doi.org/10.1093/toxsci/kfaa025
Wittwehr, C. et al. (2020) ‘Artificial Intelligence for Chemical Risk Assessment’, Computational Toxicology doi: https://doi.org/10.1016/j.comtox.2019.100114
Wolffe, T. A. M., Whaley, P., et al. (2019). 'Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management.' Environment International. doi: https://doi.org/10.1016/j.envint.2019.05.065
Hartung, T. et al. (2019) ‘Toward Good In Vitro Reporting Standards.’, ALTEX, doi: https://doi.org/10.14573/altex.1812191
Morgan, R. L. et al. (2019) ‘A risk of bias instrument for non-randomized studies of exposures: A users’ guide to its application in the context of GRADE’, Environment International. doi: https://doi.org/10.1016/j.envint.2018.11.004
Morgan, R. L. et al. (2018) ‘Identifying the PECO: A framework for formulating good questions to explore the association of environmental and other exposures with health outcomes’, Environment International. doi: https://doi.org/10.1016/j.envint.2018.07.015
Haddaway, N. R. et al. (2018) ‘ROSES Reporting standards for Systematic Evidence Syntheses: Pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps’, Environmental Evidence. doi: https://doi.org/10.1186/s13750-018-0121-7
Whaley, P. & Wikoff, D. (2018) 'Qualitative methods for integrating evidence within and across evidence streams for hazard identification' EFSA Supporting Publications. doi: https://doi.org/10.2903/sp.efsa.2018.EN-1396
Hoffmann, S. et al. (2017) ‘A primer on systematic reviews in toxicology’, Archives of Toxicology. doi: https://doi.org/10.1007/s00204-017-1980-3
Vandenberg, L. N. et al. (2016) ‘A proposed framework for the systematic review and integrated assessment (SYRINA) of endocrine disrupting chemicals’, Environmental Health. doi: https://doi.org/10.1186/s12940-016-0156-6
Whaley, P. et al. (2016) ‘Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations’, Environment International. doi: https://doi.org/10.1016/j.envint.2015.11.002
Reports
US National Academies of Sciences, Engineering and Medicine (2022) Review of U.S. EPA's ORD Staff Handbook for Developing IRIS Assessments. https://doi.org/10.17226/26289
WHO Chemical Risk Assessment Network (2021) Framework for the use of systematic review in chemical risk assessment. https://apps.who.int/iris/handle/10665/347876
Robinson, C. et al. (2018) Ensuring a higher level of protection from pesticides in Europe: The problems with current pesticide risk assessment procedures in the EU - and proposed solutions. PAN Europe. doi: https://doi.org/10.5281/zenodo.2543743
Whaley, P. and Halsall C. (2014) Identifying and Monitoring Pollution Risks from On-Shore Hydraulic Fracturing Operations. Lancaster University and Inteb Ltd.
Whaley, P. (2014) EFSA's Draft 2014 Scientific Opinion on the risks to public health related to the presence of bisphenol-A (BPA) in foodstuffs: a critical appraisal. Policy from Science Project
Whaley, P. (2013) Systematic Review and the Future of Evidence in Chemicals Policy. Policy from Science Project
Editorials
Pega et al. (2021) Systematic reviews and meta-analyses for the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury. Environment International https://doi.org/10.1016/j.envint.2021.106605
Radke, E. G. et al. (2020). Application of US EPA IRIS systematic review methods to the health effects of phthalates: Lessons learned and path forward. Environment International. doi: https://doi.org/10.1016/j.envint.2020.105820
Whaley, P. and Halsall, C. (2016) ‘Assuring high-quality evidence reviews for chemical risk assessment: Five lessons from guest editing the first environmental health journal special issue dedicated to systematic review’, Environment International. doi: https://doi.org/10.1016/j.envint.2016.04.016
Whaley, P. et al. (2016) ‘Raising the standard of systematic reviews published in Environment International’, Environment International. doi: https://doi.org/10.1016/j.envint.2016.08.007
Correspondence
Verbeek et al. (2022) Potential importance of residual confounding in systematic reviews of observational studies: Answer to Mathur and VanderWeele. Environment International. https://doi.org/10.1016/j.envint.2021.107010
Haddaway, N. R. et al. (2018) ‘Response to “Every ROSE has its thorns”’, Environmental Evidence. doi: https://doi.org/10.1186/s13750-018-0133-3
Page, J. et al. (2012) ‘Priorities for cancer prevention’, The Lancet Oncology, 13(6). doi: https://doi.org/10.1016/S1470-2045(12)70140-7
Book Chapters
Whaley, P. (2014) ‘Childhood obesity and the environment’. in D Haslam, A Sharma & C le Roux (eds), Controversies in obesity. Springer, pp. 97-102.
Protocols
Menon et al. (2020) A Survey of the Basic Scientific Quality of Systematic Reviews in Environmental Health. https://osf.io/mnpvs
de Vries, R.B.M. & Whaley, P.A. (2018) In Vitro Critical Appraisal Tool (IV-CAT): Tool Development Protocol. Evidence-Based Toxicology Collaboration doi: https://doi.org/10.5281/zenodo.1493498
Conferences, Workshops, Presentations
2022
Study quality assessment methods in the draft TSCA SR protocol (13 April 2022) US EPA Scientific Advisory Committee on Chemicals
SpeakerIV-CAT: a tool for facilitating peer review of in vitro studies (31 March 2022) Society of Toxicology
SpeakerFour Tips for Success with Evidence Reviews: An Editor’s Strategy (27 March 2022) Society of Toxicology
SpeakerSystematic review: why, what, and how? (22 March 2022) US FDA Center for Food Safety and Applied Nutrition
SpeakerOntologies for toxicology and environmental health (16 March 2022) EBTC Scientific Symposium
Organiser, Speaker, Moderator
Recommendations for the conduct of systematic reviews in toxicology and environmental health research (3 March 2022) Lancaster University
SpeakerHow to conduct high-quality systematic reviews (11 Feb 2022) University of New England, Australia
Speaker2021
Environment International’s editorial processes for handling the WHO EMF Systematic Reviews (17 December 2021) Navigation Guide Working Group Webinar
SpeakerFrom in here to out there: the fascinating challenge of standardizing toxicity assays (14 December 2021) EBTC Scientific Symposium
ModeratorEBTC 10-Year Anniversary Symposium (11 November 2021) EBTC Scientific Symposium
OrganiserSubmitting and peer-reviewing research papers: an editor’s perspective (22 October 2021) Sense About Science VOYS Workshop
PanellistRecommendations for the conduct of systematic reviews in toxicology and environmental health research (30 September 2021) BioEM
SpeakerResearch Standards for Systematic Reviews, and How to Use Them (27 September 2021) EUROTOX
SpeakerSystematic reviews and chemical risk assessment: current challenges and the need for AI in overcoming them (27 September 2021) EUROTOX
SpeakerMaximizing the value of peer review in toxicology (4 August 2021) Toxicology Forum Summer Meeting
SpeakerOvercoming data availability obstacles in the way of evidence-based toxicology (25 June 2021) EBTC Scientific Symposium
Organiser, ModeratorOvercoming the “streetlight effect” in evidence synthesis (4 May 2021) ONTOX Kick-Off Meeting
SpeakerApplication of US EPA IRIS systematic review methods to health effects of phthalate exposures (21 Feb 2021) EBTC Scientific Symposium
Organiser, Moderator2020
The results of a survey of current practices in systematic evidence maps. (6 August 2020) EBTC Scientific Advisory Council
SpeakerThe concept of "biological plausibility" in environmental health systematic reviews (17 June 2020) GRADE Working Group Annual Meeting - Large Group Session
SpeakerHow certain are we? Progress in adapting systematic review methods to environmental health assessments (15 June 2020) Evidence-Based Toxicology Collaboration Satellite Workshop to the GRADE Working Group Annual Meeting
OrganiserCertainty in Adverse Outcome Pathways (17 June 2020) GRADE Working Group Annual Meeting - Small Group Session
OrganiserResearch Quality and Peer Review Workshop. (20 June 2020) Sense About Science "Voice of Young Science" Network.
PanellistKnowledge from Information: Bioinformatics as the Science of the Anthropocene. (14 February) AAAS American Association for the Advancement of Science Annual Meeting 2020. Seattle, USA
Symposium. Convener, SpeakerBBC/UKRI: The Future of Artificial Intelligence. (14 February) AAAS American Association for the Advancement of Science Annual Meeting 2020. Seattle, USA
PanellistApplying AI to solve problems more complex than people can understand: the epistemological and ethical implications of "machine knowing" COGITO at Glasgow University, UK
Speaker. Cancelled due to COVID-19Conduct and reporting standards for systematic reviews in toxicology and risk assessment. EUROTOX 2020
Speaker. Cancelled due to COVID-19Systematic reviews and chemical risk assessment: current challenges, and the need for AI in overcoming them EUROTOX 2020
Speaker. Cancelled due to COVID-192019
Procedures for reporting and appraising systematic reviews (12 Nov) US EPA Systematic Review Community of Practice Webinar
SpeakerOverview of SR editorial and peer-review processes for the Editorial Board at Environment International (20 Nov) Environment International Editorial Board Webinar
SpeakerA Journal Editors’ Guide to Successful Systematic Reviews: Common Challenges Faced by Submitting Authors, and How to Address Them. (25 August 2019) ISEE 2019, Utrecht, NL
Workshop. Organiser and speakerInternational Editor’s Workshop on raising the standard of systematic reviews published in environmental health and toxicology journals. (29-31 May 2019) Research Triangle Park, NC, USA
Workshop. OrganiserPractical challenges in assessing the external validity of individual studies, and the implications for integrating multiple streams of evidence in systematic reviews. (2 June 2019) US National Academies of Sciences Workshop on Evidence Integration in Chemical Assessments. Washington DC, USA
SpeakerMachine Learning and Knowledge Systems for the Anthropocene. (11 January 2019) Lancaster Environment Centre Winter Conference. Lancaster, UK.
SpeakerSR submission procedures at Environment International (11 April) Second Meeting of Lead Reviewers of Systematic Reviews for the WHO/ILO Joint Methodology for Estimating the Work-Related Burden of Disease and Injury. EU OSHA, Bilbao, Spain
Speaker, TrainerEBTC GRADE Working Group Pre-Meeting: Systematic Methods for Identification, Appraisal and Integration of Mechanistic Data in Systematic Reviews. (11 June) McMaster University, Canada.
RapporteurEBTC/EFSA International Workshop on Systematic Review Methods and Adverse Outcome Pathways. (2-3 October) Parma, Italy
Group Lead, FacilitatorLancaster University (training seminar) Authorship Seminar for Early Career Researchers: An Editor’s Dos and Don’ts. Elsevier Early Career Researcher Seminar at Lancaster University. Lancaster, UK
SpeakerContinuing Education Session PM11: Conducting Systematic Review in Toxicology—Why, When, How? SOT 2019 (Baltimore, MD, USA) Contributing training presentation: “Reporting and Reviewing Systematic Reviews.” General toxicology audience.
Speaker2018
WHO/ILO Systematic Reviews reinforce the value of formal pre-publication of SR protocols. (4 December 2018) Navigation Guide Working Group Quarterly Webinar.
SpeakerKeynote: Systematic Review, Machine Learning and the Liberation of Knowledge from Information in Environmental Health Research. (15 October 2018) Research Triangle Environmental Health Collaborative Summit on Artificial Intelligence in Environmental Health. Research Triangle Park, NC, USA.
SpeakerSystematic Reviews, Evidence Maps and a Need for Interactive Data Visualization in Environmental Health Research. (25 August 2018) iLab Calgary Invited Speaker Seminar. University of Calgary, Canada.
SpeakerFrom gatekeeper to midwife: how journals can improve the quality of published systematic reviews. (19 April 2018) Collaboration for Environmental Evidence 2nd International Conference. Paris, France.
SpeakerAreas in which AI could contribute to the conduct of systematic reviews. (3 April 2018) AI4CRA Meeting. EU Joint Researcher Centre, Ispra, Italy.
SpeakerOECD Level 1 Workshop on Systematic Review of Existing Data of Endocrine Disruptors. (25 October 2018) OECD, Paris, France.
UK Expert RepresentativeIV International Congress on Safety and Health at Work (Bilbao, Spain). Training on systematic approaches to Assessment of Scientific Evidence session.
SpeakerElsevier Researcher Academy (webinar). Education webinar Systematic Reviews 101. Introduction to SRs, editorial handling thereof, and advice on successful SR project planning. Varied early-career audience. [Link]
SpeakerEU Joint Research Centre (Ispra, Italy). 2-day training on how to conduct and appraise SRs, with a focus on value for research commissioners
Organiser, Speaker2017
Gatekeeper and midwife: the two roles of the academic editor. (17 December 2017) Lancaster University “New Café”, Lancaster, UK
SpeakerEnsuring only high-quality systematic reviews get published: New obligations and strategies for environmental health journals (23 November 2017) World Health Organization Global Burden of Disease Estimation Planning Meeting, Zurich, Switzerland.
SpeakerInvited expert stakeholder: Comments on EFSA’s draft BPA hazard assessment protocol. (12 September 2017) European Food Safety Authority Meeting on Draft BPA Hazard Assessment Protocol. Brussels, Belgium.
SpeakerEnsuring only high-quality systematic reviews get published: New obligations and strategies for environmental health journals (19 August 2017) US National Toxicology Program 4th International Symposium on Systematic Review and Meta-Analysis of Laboratory Animal Studies. Research Triangle Park, NC, USA.
SpeakerThe WHO Chemical Risk Assessment Network Systematic Review Framework Document. (21 June 2017) Second Meeting of the WHO Chemical Risk Assessment Network. Parma, Italy.
SpeakerAppraisal of study quality in environmental health systematic reviews should be based on risk of bias and eschew scores and scales (Zurich, Switzerland). World Health Organization Global Burden of Disease Estimation Planning Meeting.
SpeakerIntroduction to systematic review of toxicological data (Parma, Italy). WHO Chemical Risk Assessment Network.
SpeakerSystematic review methods for environmental health research. (Debrecen, Hungary) DIMOPEX Training School. Debrecen, Hungary.
SpeakerDeveloping Environmental Standards. (London) Sustainability Network for Standardisation. London, UK.
Speaker2016
International Expert Workshop: Defining best-practice standards for planning and conducting environmental health and toxicology systematic reviews. (1-2 December 2016) Royal Society of Chemistry, London, UK
Workshop. OrganiserPanel Session on “Systematic reviews and preventing the misuse of Bradford Hill criteria” (16 October 2016) Austin Bradford Hill Anniversary Symposium. Royal Society of Medicine, London, UK.
Session ChairThemed Session “How to Ensure Only High-Quality Systematic Reviews Get Published: Strategies from Medical and Chemical Risk Research”. (16 August 2016) Collaboration for Environmental Evidence 1st International Conference. Stockholm, Sweden.
Convener, SpeakerImproving the quality of systematic evidence synthesis in toxicology. (16 August 2016) Collaboration for Environmental Evidence 1st International Conference. Stockholm, Sweden.
SpeakerSystematic review as an approach to advancing the transparency and scientific robustness of chemical risk assessment. European Parliament ECR Group
Organiser2015
Innovation vs the Precautionary Principle and avoiding market entrenchment of problem products. (15 June 2016) Cereals 2016. Cambridge, UK
PanellistContemporary reporting practices in environmental health systematic reviews (15 December 2015) Navigation Guide Working Group Webinar
SpeakerSystematic review methods for chemical risk assessment (22 June 2015) Precaution and Innovation Workshop, Brunel University, London
SpeakerDiscussion Leader with Sir Dick Taverne QC. “Better science for better EU regulation.” (26 February 2015) Centre for European Policy Studies Ideas Lab. Brussels, Belgium.
Panellist2014
International Expert Workshop: Systematic review methods for chemical risk assessment: challenges and opportunities. (14 November 2014) Royal Society of Chemistry, London, UK
Workshop. OrganiserEuropean Food Safety Authority Scientific Opinions on Bisphenol-A: a critical appraisal (3 December 2014) Navigation Guide Working Group Webinar
SpeakerEFSA’s Opinions on risks to health posed by BPA: methodological considerations (11 Sept 2014) European Network of Scientists for Social and Environmental Responsibility Annual Meeting. Berlin, Germany
SpeakerLiterature review in toxicological research and chemical risk assessment: the state of the science (13 May 2014) Poster Spotlight presentation during session “Persistent, Bioaccumulative and Toxic (PBT) substances – identification, assessment and regulatory decision making with a special focus on socio-economic aspects”
SpeakerPolicy from Science Project Literature Review Appraisal Toolkit (12 February 2014) Highwire Coffee Club, Lancaster University, UK
Speaker2013
Systematic review and the future of evidence in chemicals policy (15 October 2013) Report and campaign launch event at European Parliament with Corinne LePage MEP, Brussels, Belgium.
European Parliament hosted event. Organiser, SpeakerShould I drink out of this? (5 December 2013) SR+ Seminar, Lancaster University
SpeakerWe need to talk about literature reviews (29 November 2013) Postgraduate Seminar, Lancaster University
SpeakerThe Policy from Science Project (19 June 2013) EEB Chemicals Working Group Meeting, Brussels, Belgium
SpeakerHow to evaluate study credibility: the example set by evidence-based medicine (11 June 2013) Royal Society of Chemistry Environmental Health and Safety Committee AGM. RSC, London, UK
SpeakerHow to evaluate study credibility: the example set by evidence-based medicine (12 May 2013) SETAC Europe, Glasgow, Scotland
Speaker