Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning
Zhaorui Yang, Tianyu Pang, Haozhe Feng, Han Wang, Wei Chen, Minfeng Zhu, Qian Liu
Abstract
Zhaorui Yang, Tianyu Pang, Haozhe Feng, Han Wang, Wei Chen, Minfeng Zhu, Qian Liu. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.
Topics & Concepts
DistillationComputer scienceDistribution (mathematics)Language modelChemistryArtificial intelligenceMathematicsChromatographyMathematical analysisNatural Language Processing TechniquesTopic Modeling