A Quantum Annealing Instance Selection Approach for Efficient and Effective Transformer Fine-Tuning
Andrea Pasin, Washington Cunha, Marcos André Gonçalves, Nicola Ferro
Abstract
Deep Learning approaches have become pervasive in recent years due to their ability to solve complex tasks. However, these models need huge datasets for proper training and good generalization. This translates into high training and fine-tuning time, even several days for the most complex models and large datasets. In this work, we present a novel quantum Instance Selection (IS) approach that allows to significantly reduce the size of the training datasets (by up to 28%) while maintaining the model's effectiveness, thus promoting (training) speedups and scalability. Our solution is innovative in the sense that it exploits a different computing paradigm - Quantum Annealing (QA) - a specific Quantum Computing paradigm that can be used to tackle optimization problems. To the best of our knowledge, there have been no prior attempts to tackle the IS problem using QA. Furthermore, we propose a new Quadratic Unconstrained Binary Optimization formulation specific for the IS problem, which is a contribution in itself. Through an extensive set of experiments with several Text Classification benchmarks, we empirically demonstrate our quantum solution's feasibility and competitiveness with the current state-of-the-art IS solutions.