Efficient One Pass Self-distillation with Zipf’s Label Smoothing
Jiajun Liang, Linze Li, Zhaodong Bing, Borui Zhao, Yao Tang, Lin Bo, Haoqiang Fan
Topics & Concepts
Computer scienceZipf's lawSmoothingSoftmax functionOverhead (engineering)DistillationSortingExploitArtificial intelligenceMachine learningAlgorithmArtificial neural networkStatisticsMathematicsComputer securityChemistryOperating systemOrganic chemistryComputer visionAdversarial Robustness in Machine LearningAdvanced Neural Network ApplicationsMachine Learning and Data Classification