QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
Hossein Rajabzadeh, Mojtaba Valipour, Tianshu Zhu, Marzieh S. Tahaei, Hyock Ju Kwon, Ali Ghodsi, Boxing Chen, Mehdi Rezagholizadeh
Abstract
Hossein Rajabzadeh, Mojtaba Valipour, Tianshu Zhu, Marzieh S. Tahaei, Hyock Ju Kwon, Ali Ghodsi, Boxing Chen, Mehdi Rezagholizadeh. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track. 2024.
Topics & Concepts
Computer scienceAdaptation (eye)Rank (graph theory)Language modelArtificial intelligenceMathematicsPhysicsOpticsCombinatoricsSpeech Recognition and SynthesisTopic Modeling