QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
Dan Alistarh, Saleh Ashkboos, Pashmina Cameron, Maximilian L. Croci, James Hensman, Torsten Hoefler, Martin Jaggi, Bo Li, Amirkeivan Mohtashami
Topics & Concepts
Artificial intelligenceComputer scienceInferenceMathematicsFeature (linguistics)Pattern recognition (psychology)Noise (video)Computer visionAlgorithmConstraint (computer-aided design)Parallel Computing and Optimization TechniquesSoftware-Defined Networks and 5GCryptographic Implementations and Security