LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models
Yifan Yang, Jiajun Zhou, Ngai Wong, Zheng Zhang
Abstract
Yifan Yang, Jiajun Zhou, Ngai Wong, Zheng Zhang. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.
Topics & Concepts
Adaptation (eye)Tensor (intrinsic definition)Computer scienceRank (graph theory)Language modelArtificial intelligenceMathematicsPhysicsCombinatoricsOpticsPure mathematicsComputational Physics and Python ApplicationsTensor decomposition and applicationsTopic Modeling