Why logit distillation works: A novel knowledge distillation technique by deriving target augmentation and logits distortion
Md. Imtiaz Hossain, Sharmen Akhter, Nosin Ibna Mahbub, Choong Seon Hong, Eui‐Nam Huh
Topics & Concepts
DistillationLogitDistortion (music)Computer scienceLogistic regressionMachine learningChemistryChromatographyTelecommunicationsAmplifierBandwidth (computing)Data Stream Mining TechniquesMachine Learning and ELMStochastic Gradient Optimization Techniques