SC23 Proceedings

The International Conference for High Performance Computing, Networking, Storage, and Analysis

Research Posters Archive

Integrating TEZIP into LibPressio: A Case Study of Integrating a Dynamic Application into a Static C Environment

Authors: Isita Talukdar (University of California, Berkeley; RIKEN Center for Computational Science (R-CCS)); Amarjit Singh (RIKEN Center for Computational Science (R-CCS)); Robert Underwood (Argonne National Laboratory (ANL)); Kento Sato (RIKEN Center for Computational Science (R-CCS)); and Weikuan Yu (Florida State University)

Abstract: LCLS-II at SLAC, SNS at Oak Ridge Laboratory, and other instruments use software written in C and C++, producing huge volumes of time evolving data at high rate. Data compression can decrease the volume of data we need to move and store. TEZIP is a neural network (NN) based compressor designed for high-quality compression of time-evolving data. However, TEZIP is written in Python and is not easily usable from or ported to C++. In this work, we develop new components in LibPressio that allow us to integrate with TEZIP and other external compressors efficiently and evaluate them with a systematic approach. We find that TEZIP’s compression ratio (Error Bound 1e-06) for Hurricane Isabel is 128, which is 2.4 times greater than the leading SZ3’s, 52.8. Our basic integration of TEZIP into Libpressio sets a precedent for the integration of non C/C++ compressors into LibPressio.

Best Poster Finalist (BP): no

Poster: PDF
Poster summary: PDF

Back to Poster Archive Listing