The fine art of fine-tuning: A structured review of advanced LLM fine-tuning techniques
Samar Pratap, Alston Richard Aranha, Divyanshu Kumar, Gautam Malhotra, Anantharaman Palacode Narayana Iyer, S. S. Shylaja
Abstract
Transformer-based models have consistently demonstrated superior accuracy compared to various traditional models across a range of downstream tasks. However, due to their large nature, training or fine-tuning them for specific tasks has heavy computational and memory demands. This causes the creation of specialized transformer-based models to be almost impossible in the generally present constrained scenarios. To tackle this issue and to make these large models more accessible, a plethora of techniques have been developed. In this study, we will be reviewing the types of techniques developed, their impacts and benefits concerning performance and resource usage along with the latest developments in the domain. We have broadly categorized these techniques into six key areas: Changes in Training Method, Changes in Adapter, Quantization, Parameter Selection, Mixture of Experts, and Application based methods. We collated the results of various techniques on common benchmarks and also evaluated their performance on different datasets and base models. • Categorization of fine-tuning by structure and use cases to analyze LLM trends. • Comparative analysis of methods across benchmarks to weigh trade-offs better. • Overview of trends in fine-tuning that enhance scalability and efficiency of LLMs.