The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Authors: Tao Lu (DapuStor Corporation); Yu Zhong, Zibin Sun, Xiang Chen, You Zhou, and Fei Wu (Huazhong University of Science & Technology); and Ying Yang, Yunxin Huang, and Yafei Yang (DapuStor Corporation)
Abstract: SZ is a lossy floating-point data compressor that excels in compression ratio and throughput for high-performance computing (HPC), time series databases, and deep learning applications. However, SZ performs poorly for small chunks and has slow decompression. We pinpoint the Huffman tree in the quantization factor encoder as the bottleneck of SZ. In this paper, we propose ADT-FSE, a new quantization factor encoder for SZ. Based on the Gaussian distribution of quantization factors, we design an adaptive data transcoding (ADT) scheme to map quantization factors to codes for better compressibility, and then use finite state entropy (FSE) to compress the codes. Experiments show that ADT-FSE improves the quantization factor compression ratio, compression and decompression throughput by up to 5x, 2x and 8x, respectively, over the original SZ Huffman encoder. On average, SZ_ADT is over 2x faster than ZFP in decompression.
Back to Technical Papers Archive Listing