Joint structured pruning and dense knowledge distillation for efficient transformer model compression
Baiyun Cui, Yingming Li, Zhongfei Zhang
Topics & Concepts
Computer scienceDistillationTransformerLeverage (statistics)InferencePruningSpeedupArtificial intelligenceMachine learningJoint (building)Compression (physics)Data miningParallel computingOrganic chemistryBiologyChemistryComposite materialQuantum mechanicsAgronomyPhysicsMaterials scienceEngineeringVoltageArchitectural engineeringTopic ModelingNatural Language Processing TechniquesNeural Networks and Applications