Decomposed bounded floats for fast compression and queries
Chunwei Liu, Hao Jiang, John Paparrizos, Aaron J. Elmore
Abstract
Modern data-intensive applications often generate large amounts of low precision float data with a limited range of values. Despite the prevalence of such data, there is a lack of an effective solution to ingest, store, and analyze bounded, low-precision, numeric data. To address this gap, we propose Buff, a new compression technique that uses a decomposed columnar storage and encoding methods to provide effective compression, fast ingestion, and high-speed in-situ adaptive query operators with SIMD support.
Topics & Concepts
Computer scienceSIMDBounded functionCompression (physics)Data compressionFloat (project management)Range (aeronautics)Encoding (memory)Computational scienceParallel computingAlgorithmData miningMathematicsArtificial intelligenceMaterials scienceMathematical analysisComposite materialEngineeringMarine engineeringAlgorithms and Data CompressionAdvanced Data Storage TechnologiesParallel Computing and Optimization Techniques