Workshops Archive

Advancing Automated I/O Analysis with Multi-Perspective Views

Workshop: PDSW23: 8th International Parallel Data Systems Workshop

Authors: Izzet Yildirim (Illinois Institute of Technology), Hariharan Devarajan (Lawrence Livermore National Laboratory), Anthony Kougkas and Xian-He Sun (Illinois Institute of Technology), and Kathryn Mohror (Lawrence Livermore National Laboratory)

Abstract: Real-world HPC workloads, including simulations and machine learning, place significant strain on storage infrastructure due to their data-dependency, exacerbated by the diverse storage options in modern HPC environments, leading to I/O bottlenecks. To mitigate these bottlenecks, past analysis methods relied on manual evaluations and tools like Darshan for I/O trace collection, often necessitating expert involvement and substantial time commitments. In response to the imperative for automation due to the time-intensive nature of manual analysis and the pressing need to effectively mitigate I/O bottlenecks, analysis tools were developed. According to our findings, these tools, while providing automation, can still benefit from transitioning from heuristics-based approaches to a more data-driven decision-making. To address this, we propose a data-driven approach that leverages multi-perspective views.

Back to PDSW23: 8th International Parallel Data Systems Workshop Archive Listing

Back to Full Workshop Archive Listing