Lossy Scientific Data Compression With SPERR
Shaomeng Li, Peter Lindström, John Clyne
Abstract
As the need for data reduction in high-performance computing (HPC) continues to grow, we introduce a new and highly effective tool to help achieve this goal—SPERR. SPERR is a versatile lossy compressor for structured scientific data; it is built on top of an advanced wavelet compression algorithm, SPECK, and provides additional capabilities valued in HPC environments. These capabilities include parallel execution for large volumes and a compression mode that satisfies a maximum point-wise error tolerance. Evaluation shows that in most settings SPERR achieves the best rate-distortion trade-off among current popular lossy scientific data compressors.
Topics & Concepts
Lossy compressionComputer scienceData compressionGas compressorCompression (physics)Reduction (mathematics)Distortion (music)Computer engineeringAlgorithmArtificial intelligenceEngineeringBandwidth (computing)TelecommunicationsMaterials scienceComposite materialGeometryMathematicsAmplifierMechanical engineeringAdvanced Data Compression TechniquesImage and Signal Denoising MethodsAlgorithms and Data Compression