Research Posters Archive

Why Wait!? Hades: An Active, Content-Aware System for Precalculating Derived Quantities

Authors: Jaime Cernuda, Luke Logan, Anthony Kougkas, and Xian-He Sum (Illinois Institute of Technology)

Abstract: Modern scientific applications produce vast amounts of data, typically stored in monolithic files on parallel file systems (PFS). Analyzing these large files often results in inefficiency due to I/O stalls. To mitigate these stalls, certain data can be pre-computed during the production phase and queried during analysis. However, this solution demands added storage capacity and an astute use of storage hierarchies. In this context, we introduce Hades, an I/O engine seamlessly integrated with the Adios2 framework. Hades offers hierarchical buffering, which enables smart data placement and prefetching across the spectrum of I/O devices. Additionally, it is adept at computing basic derived quantities required by I/O applications, such as the global and local min/max values. A notable feature of Hades is its memory-first metadata management strategy, which is designed for querying derived data, significantly enhancing system performance.

Best Poster Finalist (BP): no

Poster: PDF
Poster summary: PDF


Back to Poster Archive Listing