Efficient Model Compression and Knowledge Distillation on LLama 2: Achieving High Performance with Reduced Computational Cost
Qionglin Huangpu, Huixiang Gao
Abstract
This study investigates the application of model compression and knowledge distillation techniques to enhance the computational efficiency of LLama 2, a Large Language Model (LLM) with 7 billion parameters. Through a comprehensive methodology incorporating pruning, quantization, and parameter sharing, alongside a rigorous knowledge distillation process, we aim to reduce the model's size and computational demands without substantially affecting its performance. Our results demonstrate significant reductions in model size and inference times, while maintaining competitive performance metrics. Furthermore, the distilled model not only captures the essence of the original LLama 2 but also shows improved efficiency, making it suitable for deployment in resource-constrained environments. These findings underline the potential of compression and distillation techniques in making LLMs more accessible and sustainable. Future research directions include optimizing these methods further, exploring their applicability across a broader range of tasks and languages, and developing automated optimization tools to facilitate the widespread adoption of efficient LLMs.