When performing a Systematic Literature Review (SLR), the Abstract Screening Process (ASP) can be a very consuming and laborious task, especially when researchers retrieve a significant number of citations after running queries in scientific databases. By casting the ASP as a supervised machine learning (classification) problem, we propose a machine learning-based framework that generalises from a small subset of manually classified abstracts that greatly reduces the researchers' workload. Crucially, our approach explains the judgements that must be made and proposes different approaches from which researchers can choose, detailing the advantages and drawbacks of each choice. Our approach seeks a careful balance between ease of use and defence against overfitting on the one hand and lack of complexity on the other. A key innovation is the use of the acceptance sampling method from industrial engineering to evaluate the risk of non-detected positive instances rejected by our tool.

Our methods are implemented in a tool we call AI-driven Comprehensive Abstract Selection (ACAS). Implemented in Python, the ACAS tool is easy to use and freely available to researchers via GitHub. It can either be used out-of-the box or calibrated according to the researchers' needs. Numerical results on a real-life case study (a SLR on Operations Research methods in healthcare) shows a reduction of workload by 86.32% (corresponding to 522.46 hours of work), while offering a similar error risk as a human expert reviewer.