Pure tensor program rewriting via access patterns (representation pearl)
Gus Henry Smith, Andrew Liu, Steven Lyubomirsky, Scott Davidson, Joseph McMahan, Michael Taylor, Luís Ceze, Zachary Tatlock
Abstract
Tensor kernels in machine learning (ML) often correspond to pure mathematical expressions, making term rewriting an attractive strategy for optimization and mapping to specialized hardware accelerators. However, existing ML intermediate representations (IRs) tend to either be pure but high-level, making low-level rewrites to hardware targets inexpressible, or low-level but impure, hampering the use of term rewriting altogether.
Topics & Concepts
RewritingComputer scienceRepresentation (politics)Term (time)Programming languageTensor (intrinsic definition)PearlTheoretical computer scienceConfluenceArtificial intelligenceMathematicsPure mathematicsPolitical scienceTheologyPoliticsQuantum mechanicsPhilosophyPhysicsLawParallel Computing and Optimization TechniquesComputational Physics and Python ApplicationsDistributed and Parallel Computing Systems