Enhancing systematic reviews: An in-depth analysis on the impact of active learning parameter combinations for biomedical abstract screening

Abstract

Systematic Review (SR) are foundational to influencing policies and decision-making in healthcare and beyond. SRs thoroughly synthesise primary research on a specific topic while maintaining reproducibility and transparency. However, the rigorous nature of SRs introduces two main challenges: significant time involved and the continuously growing literature, resulting in potential data omission, making most SRs become outmoded even before they are published. As a solution, AI techniques have been leveraged to simplify the SR process, especially the abstract screening phase. Active learning (AL) has emerged as a preferred method among these AI techniques, allowing interactive learning through human input. Several AL software have been proposed for abstract screening. Despite its prowess, how the various parameters involved in AL influence the software’s efficacy is still unclear. This research seeks to demystify this by exploring how different AL strategies, such as initial training set, query strategies etc. impact SR automation. Experimental evaluations were conducted on five complex medical SR datasets, and the GLM model was used to interpret the findings statistically. Some AL variables, such as the feature extractor, initial training size, and classifiers, showed notable observations and practical conclusions were drawn within the context of SR and beyond where AL is deployed. • This study explores optimal Active Learning (AL) combinations for systematic reviews (SRs). • Smaller initial training samples improve performance metrics in datasets. • TF-IDF consistently outperformed Doc2Vec and S-BERT. • Certainty and Uncertainty strategies gave comparative results and effectively interacted with the TF-IDF. • The impact of AL variables in SR automation varies according to the specific dataset.

Topics & Concepts