The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop: Workshop on Memory Technologies, Systems, and Applications
Authors: Deepak B. Hegde and Preeti Malakar (Indian Institute of Technology (IIT), Kanpur)
Abstract: The relatively slower data transfer speeds resulting in I/O bottlenecks in scientific simulations is one of the critical challenges in exascale computing. Simulations generate large data and analysis applications consume this data to provide time-critical insights. The limited size and high power consumption of Dynamic Random Access Memory (DRAM) capacity leaves slow storage devices as the primary option for large-scale data transfers. Non-volatile memory (NVM) devices such as Intel Optane bridges the gap between storage and volatile memory by providing DRAM-comparable performance and persistence. We present PQueue, a data transfer library for in situ analysis of simulation output using persistent memory. PQueue leverages NVM and provides an API that resembles high-level parallel I/O libraries such as PnetCDF to enable seamless transition for application developers. We achieved a maximum of 7X improvement in write times and a maximum of 10X improvement in read times as compared to PnetCDF.
Back to Workshop on Memory Technologies, Systems, and Applications Archive Listing