Research
Systematic reviews and artificial intelligence
My main area of research is adaptation of systematic review methods from clinical to environmental health and toxicological contexts. I work on best practice standards for conduct of systematic reviews and the development of critical appraisal tools for assessing the quality of primary and secondary research - in particular, how such tools can be applied in manuscript production workflows to raise publishing standards.
I recently began moving into the field of artificial intelligence and machine learning for automated evidence synthesis, focusing on improved automated data extraction methods and developing knowledge-based systems for systematic evidence mapping. These are technologies which would address current challenges around the scalability of SR methods, enabling use of the specific type of big data analytics and machine learning methods for the complex analytical challenges which are a defining characteristic of SRs in the environmental health field.
Research Themes
Evidence Maps and Automating Systematic Review
Evidence maps are a novel evidence synthesis product which allow systematic identification of gaps and gluts in decision-critical evidence. However, they only really work if we can move our vast store of scientific information from unstructured archives of PDFs into queryable databases. This requires us to teach machines to "read" scientific documents, develop semantic authoring tools for writing scientific studies, and create comprehensive research metadata records in suitable underlying knowledge organisation systems. If we do this, we could move away from manual systematic reviews into more automated methods of synthesising evidence.
Best Practices in Systematic Review
I work on developing and promoting best practice standards for conduct of systematic reviews and evidence maps in the environmental health sciences. This mainly involves adapting existing standards in use in biomedical research to the toxicology and environmental health context. The idea is that, if we can provide environmental health researchers with an authoritative recipe-book for conducting systematic reviews, it should help them reduce the risk of conducting unsuccessful SR projects.
Critical Appraisal Tools
While there are a lot of critical appraisal tools for helping people distinguish more credible studies from less, it is not always clear which tool should be used and when, nor if any given tool is actually fit-for-purpose. I am working on several tools for critically appraising systematic reviews and primary studies, concentrating on ensuring they target an appropriate quality construct for the context in which they are to be used, that are reasonably intuitive, and they reliably distinguish higher-quality from lower-quality research.
Publications
Journal Articles
Whaley, P. et al. (2021) 'Improving the quality of toxicology and environmental health systematic reviews: What journal editors can do' ALTEX https://doi.org/10.14573/altex.2106111
Sharma et al. (2021) 'Towards guidelines for time-trend reviews examining temporal variability in human biomonitoring data of pollutants' Environment International https://doi.org/10.1016/j.envint.2021.106437
de Vries et al. (2021) 'Applying evidence-based methods to the development and use of adverse outcome pathways.' ALTEX https://doi.org/10.14573/altex.2101211
Whaley P. et al. (2020) Knowledge organization systems for systematic chemical assessments.' Environmental Health Perspectives https://doi.org/10.1289/EHP6994
Whaley P. et al. (2020) 'Recommendations for conduct of systematic reviews in toxicology and environmental health research (COSTER)' Environment International https://doi.org/10.1016/j.envint.2020.105926
Robinson, C. et al. (2020) ‘Achieving a high level of protection from pesticides in Europe: Problems with the current risk assessment procedure and solutions’ European Journal of Risk Regulation https://doi.org/10.1017/err.2020.18
Wolffe, T. et al. (2020) ‘A survey of systematic evidence mapping practice and the case for knowledge graphs in environmental health & toxicology’ Toxicological Sciences doi: https://doi.org/10.1093/toxsci/kfaa025
Wittwehr, C. et al. (2020) ‘Artificial Intelligence for Chemical Risk Assessment’, Computational Toxicology doi: https://doi.org/10.1016/j.comtox.2019.100114
Wolffe, T. A. M., Whaley, P., et al. (2019). 'Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management.' Environment International. doi: https://doi.org/10.1016/j.envint.2019.05.065
Hartung, T. et al. (2019) ‘Toward Good In Vitro Reporting Standards.’, ALTEX, doi: https://doi.org/10.14573/altex.1812191
Morgan, R. L. et al. (2019) ‘A risk of bias instrument for non-randomized studies of exposures: A users’ guide to its application in the context of GRADE’, Environment International. doi: https://doi.org/10.1016/j.envint.2018.11.004
Morgan, R. L. et al. (2018) ‘Identifying the PECO: A framework for formulating good questions to explore the association of environmental and other exposures with health outcomes’, Environment International. doi: https://doi.org/10.1016/j.envint.2018.07.015
Haddaway, N. R. et al. (2018) ‘ROSES Reporting standards for Systematic Evidence Syntheses: Pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps’, Environmental Evidence. doi: https://doi.org/10.1186/s13750-018-0121-7
Whaley, P. & Wikoff, D. (2018) 'Qualitative methods for integrating evidence within and across evidence streams for hazard identification' EFSA Supporting Publications. doi: https://doi.org/10.2903/sp.efsa.2018.EN-1396
Hoffmann, S. et al. (2017) ‘A primer on systematic reviews in toxicology’, Archives of Toxicology. doi: https://doi.org/10.1007/s00204-017-1980-3
Vandenberg, L. N. et al. (2016) ‘A proposed framework for the systematic review and integrated assessment (SYRINA) of endocrine disrupting chemicals’, Environmental Health. doi: https://doi.org/10.1186/s12940-016-0156-6
Whaley, P. et al. (2016) ‘Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations’, Environment International. doi: https://doi.org/10.1016/j.envint.2015.11.002
Editorials
Pega et al. (2021) Systematic reviews and meta-analyses for the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury. Environment International https://doi.org/10.1016/j.envint.2021.106605
Radke, E. G. et al. (2020). Application of US EPA IRIS systematic review methods to the health effects of phthalates: Lessons learned and path forward. Environment International. doi: https://doi.org/10.1016/j.envint.2020.105820
Whaley, P. and Halsall, C. (2016) ‘Assuring high-quality evidence reviews for chemical risk assessment: Five lessons from guest editing the first environmental health journal special issue dedicated to systematic review’, Environment International. doi: https://doi.org/10.1016/j.envint.2016.04.016
Whaley, P. et al. (2016) ‘Raising the standard of systematic reviews published in Environment International’, Environment International. doi: https://doi.org/10.1016/j.envint.2016.08.007
Comment / Debate / Correspondence
Haddaway, N. R. et al. (2018) ‘Response to “Every ROSE has its thorns”’, Environmental Evidence. doi: https://doi.org/10.1186/s13750-018-0133-3
Page, J. et al. (2012) ‘Priorities for cancer prevention’, The Lancet Oncology, 13(6). doi: https://doi.org/10.1016/S1470-2045(12)70140-7
Book Chapters
Whaley, P. (2014) ‘Childhood obesity and the environment’. in D Haslam, A Sharma & C le Roux (eds), Controversies in obesity. Springer, pp. 97-102.
Other Contributions
Robinson, C. et al. (2018) Ensuring a higher level of protection from pesticides in Europe: The problems with current pesticide risk assessment procedures in the EU - and proposed solutions. PAN Europe. doi: https://doi.org/10.5281/zenodo.2543743
de Vries, R.B.M. & Whaley, P.A. (2018) In Vitro Critical Appraisal Tool (IV-CAT): Tool Development Protocol. Evidence-Based Toxicology Collaboration doi: https://doi.org/10.5281/zenodo.1493498
Whaley, P. and Halsall C. (2014) Identifying and Monitoring Pollution Risks from On-Shore Hydraulic Fracturing Operations. Lancaster University and Inteb Ltd.
Whaley, P. (2014) EFSA's Draft 2014 Scientific Opinion on the risks to public health related to the presence of bisphenol-A (BPA) in foodstuffs: a critical appraisal. Policy from Science Project
Whaley, P. (2013) Systematic Review and the Future of Evidence in Chemicals Policy. Policy from Science Project
Conferences and Workshops
2020
The results of a survey of current practices in systematic evidence maps. (6 August 2020) EBTC Scientific Advisory Council
SpeakerThe concept of "biological plausibility" in environmental health systematic reviews (17 June 2020) GRADE Working Group Annual Meeting - Large Group Session
SpeakerHow certain are we? Progress in adapting systematic review methods to environmental health assessments (15 June 2020) Evidence-Based Toxicology Collaboration Satellite Workshop to the GRADE Working Group Annual Meeting
OrganiserCertainty in Adverse Outcome Pathways (17 June 2020) GRADE Working Group Annual Meeting - Small Group Session
OrganiserResearch Quality and Peer Review Workshop. (20 June 2020) Sense About Science "Voice of Young Science" Network.
PanellistKnowledge from Information: Bioinformatics as the Science of the Anthropocene. (14 February) AAAS American Association for the Advancement of Science Annual Meeting 2020. Seattle, USA
Symposium. Convener, SpeakerBBC/UKRI: The Future of Artificial Intelligence. (14 February) AAAS American Association for the Advancement of Science Annual Meeting 2020. Seattle, USA
PanellistApplying AI to solve problems more complex than people can understand: the epistemological and ethical implications of "machine knowing" COGITO at Glasgow University, UK
Speaker. Cancelled due to COVID-19Conduct and reporting standards for systematic reviews in toxicology and risk assessment. EUROTOX 2020
Speaker. Cancelled due to COVID-19Systematic reviews and chemical risk assessment: current challenges, and the need for AI in overcoming them EUROTOX 2020
Speaker. Cancelled due to COVID-192019
Procedures for reporting and appraising systematic reviews (12 Nov) US EPA Systematic Review Community of Practice Webinar
SpeakerOverview of SR editorial and peer-review processes for the Editorial Board at Environment International (20 Nov) Environment International Editorial Board Webinar
SpeakerA Journal Editors’ Guide to Successful Systematic Reviews: Common Challenges Faced by Submitting Authors, and How to Address Them. (25 August 2019) ISEE 2019, Utrecht, NL
Workshop. Organiser and speakerInternational Editor’s Workshop on raising the standard of systematic reviews published in environmental health and toxicology journals. (29-31 May 2019) Research Triangle Park, NC, USA
Workshop. OrganiserPractical challenges in assessing the external validity of individual studies, and the implications for integrating multiple streams of evidence in systematic reviews. (2 June 2019) US National Academies of Sciences Workshop on Evidence Integration in Chemical Assessments. Washington DC, USA
SpeakerMachine Learning and Knowledge Systems for the Anthropocene. (11 January 2019) Lancaster Environment Centre Winter Conference. Lancaster, UK.
SpeakerSR submission procedures at Environment International (11 April) Second Meeting of Lead Reviewers of Systematic Reviews for the WHO/ILO Joint Methodology for Estimating the Work-Related Burden of Disease and Injury. EU OSHA, Bilbao, Spain
Speaker, TrainerEBTC GRADE Working Group Pre-Meeting: Systematic Methods for Identification, Appraisal and Integration of Mechanistic Data in Systematic Reviews. (11 June) McMaster University, Canada.
RapporteurEBTC/EFSA International Workshop on Systematic Review Methods and Adverse Outcome Pathways. (2-3 October) Parma, Italy
Group Lead, FacilitatorLancaster University (training seminar) Authorship Seminar for Early Career Researchers: An Editor’s Dos and Don’ts. Elsevier Early Career Researcher Seminar at Lancaster University. Lancaster, UK
SpeakerContinuing Education Session PM11: Conducting Systematic Review in Toxicology—Why, When, How? SOT 2019 (Baltimore, MD, USA) Contributing training presentation: “Reporting and Reviewing Systematic Reviews.” General toxicology audience.
Speaker2018
WHO/ILO Systematic Reviews reinforce the value of formal pre-publication of SR protocols. (4 December 2018) Navigation Guide Working Group Quarterly Webinar.
SpeakerKeynote: Systematic Review, Machine Learning and the Liberation of Knowledge from Information in Environmental Health Research. (15 October 2018) Research Triangle Environmental Health Collaborative Summit on Artificial Intelligence in Environmental Health. Research Triangle Park, NC, USA.
SpeakerSystematic Reviews, Evidence Maps and a Need for Interactive Data Visualization in Environmental Health Research. (25 August 2018) iLab Calgary Invited Speaker Seminar. University of Calgary, Canada.
SpeakerFrom gatekeeper to midwife: how journals can improve the quality of published systematic reviews. (19 April 2018) Collaboration for Environmental Evidence 2nd International Conference. Paris, France.
SpeakerAreas in which AI could contribute to the conduct of systematic reviews. (3 April 2018) AI4CRA Meeting. EU Joint Researcher Centre, Ispra, Italy.
SpeakerOECD Level 1 Workshop on Systematic Review of Existing Data of Endocrine Disruptors. (25 October 2018) OECD, Paris, France.
UK Expert RepresentativeIV International Congress on Safety and Health at Work (Bilbao, Spain). Training on systematic approaches to Assessment of Scientific Evidence session.
SpeakerElsevier Researcher Academy (webinar). Education webinar Systematic Reviews 101. Introduction to SRs, editorial handling thereof, and advice on successful SR project planning. Varied early-career audience. [Link]
SpeakerEU Joint Research Centre (Ispra, Italy). 2-day training on how to conduct and appraise SRs, with a focus on value for research commissioners
Organiser, Speaker2017
Gatekeeper and midwife: the two roles of the academic editor. (17 December 2017) Lancaster University “New Café”, Lancaster, UK
SpeakerEnsuring only high-quality systematic reviews get published: New obligations and strategies for environmental health journals (23 November 2017) World Health Organization Global Burden of Disease Estimation Planning Meeting, Zurich, Switzerland.
SpeakerInvited expert stakeholder: Comments on EFSA’s draft BPA hazard assessment protocol. (12 September 2017) European Food Safety Authority Meeting on Draft BPA Hazard Assessment Protocol. Brussels, Belgium.
SpeakerEnsuring only high-quality systematic reviews get published: New obligations and strategies for environmental health journals (19 August 2017) US National Toxicology Program 4th International Symposium on Systematic Review and Meta-Analysis of Laboratory Animal Studies. Research Triangle Park, NC, USA.
SpeakerThe WHO Chemical Risk Assessment Network Systematic Review Framework Document. (21 June 2017) Second Meeting of the WHO Chemical Risk Assessment Network. Parma, Italy.
SpeakerAppraisal of study quality in environmental health systematic reviews should be based on risk of bias and eschew scores and scales (Zurich, Switzerland). World Health Organization Global Burden of Disease Estimation Planning Meeting.
SpeakerIntroduction to systematic review of toxicological data (Parma, Italy). WHO Chemical Risk Assessment Network.
SpeakerSystematic review methods for environmental health research. (Debrecen, Hungary) DIMOPEX Training School. Debrecen, Hungary.
SpeakerDeveloping Environmental Standards. (London) Sustainability Network for Standardisation. London, UK.
Speaker2016
International Expert Workshop: Defining best-practice standards for planning and conducting environmental health and toxicology systematic reviews. (1-2 December 2016) Royal Society of Chemistry, London, UK
Workshop. OrganiserPanel Session on “Systematic reviews and preventing the misuse of Bradford Hill criteria” (16 October 2016) Austin Bradford Hill Anniversary Symposium. Royal Society of Medicine, London, UK.
Session ChairThemed Session “How to Ensure Only High-Quality Systematic Reviews Get Published: Strategies from Medical and Chemical Risk Research”. (16 August 2016) Collaboration for Environmental Evidence 1st International Conference. Stockholm, Sweden.
Convener, SpeakerImproving the quality of systematic evidence synthesis in toxicology. (16 August 2016) Collaboration for Environmental Evidence 1st International Conference. Stockholm, Sweden.
SpeakerSystematic review as an approach to advancing the transparency and scientific robustness of chemical risk assessment. European Parliament ECR Group
Organiser2015
Innovation vs the Precautionary Principle and avoiding market entrenchment of problem products. (15 June 2016) Cereals 2016. Cambridge, UK
PanellistContemporary reporting practices in environmental health systematic reviews (15 December 2015) Navigation Guide Working Group Webinar
SpeakerSystematic review methods for chemical risk assessment (22 June 2015) Precaution and Innovation Workshop, Brunel University, London
SpeakerDiscussion Leader with Sir Dick Taverne QC. “Better science for better EU regulation.” (26 February 2015) Centre for European Policy Studies Ideas Lab. Brussels, Belgium.
Panellist2014
International Expert Workshop: Systematic review methods for chemical risk assessment: challenges and opportunities. (14 November 2014) Royal Society of Chemistry, London, UK
Workshop. OrganiserEuropean Food Safety Authority Scientific Opinions on Bisphenol-A: a critical appraisal (3 December 2014) Navigation Guide Working Group Webinar
SpeakerEFSA’s Opinions on risks to health posed by BPA: methodological considerations (11 Sept 2014) European Network of Scientists for Social and Environmental Responsibility Annual Meeting. Berlin, Germany
SpeakerLiterature review in toxicological research and chemical risk assessment: the state of the science (13 May 2014) Poster Spotlight presentation during session “Persistent, Bioaccumulative and Toxic (PBT) substances – identification, assessment and regulatory decision making with a special focus on socio-economic aspects”
SpeakerPolicy from Science Project Literature Review Appraisal Toolkit (12 February 2014) Highwire Coffee Club, Lancaster University, UK
Speaker2013
Systematic review and the future of evidence in chemicals policy (15 October 2013) Report and campaign launch event at European Parliament with Corinne LePage MEP, Brussels, Belgium.
European Parliament hosted event. Organiser, SpeakerShould I drink out of this? (5 December 2013) SR+ Seminar, Lancaster University
SpeakerWe need to talk about literature reviews (29 November 2013) Postgraduate Seminar, Lancaster University
SpeakerThe Policy from Science Project (19 June 2013) EEB Chemicals Working Group Meeting, Brussels, Belgium
SpeakerHow to evaluate study credibility: the example set by evidence-based medicine (11 June 2013) Royal Society of Chemistry Environmental Health and Safety Committee AGM. RSC, London, UK
SpeakerHow to evaluate study credibility: the example set by evidence-based medicine (12 May 2013) SETAC Europe, Glasgow, Scotland
Speaker