The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop: The 18th Workshop on Workflows in Support of Large-Scale Science (WORKS23) - Part 1 of 2
Authors: Donatello Elia and Sonia Scardigno (Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici); Jorge Ejarque (Barcelona Supercomputing Center); Alessandro D’Anca, Gabriele Accarino, Enrico Scoccimarro, Davide Donno, and Daniele Peano (Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici); and Francesco Immorlano and Giovanni Aloisio (Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici; Department of Innovation Engineering, University of Salento)
Abstract: Current scientific workflow systems do not typically integrate simulation-centric and data-centric aspects due to their very different software/infrastructure requirements. A transparent integration of such components into a single end-to-end workflow would lead to a more efficient and automated way for generating insights from large simulation data. This work presents a complex case study related to extreme events analysis of future climate data that integrates in the same workflow numerical simulations, Big Data analytics and Machine Learning models. The case study is being implemented in the context of the eFlows4HPC project using the project's software stack for deployment and orchestration of the workflow. The solution implemented in the project has shown to simplify the development and execution of end-to-end climate workflows with heterogeneous software requirements. Moreover, such an approach can, in the long term, increase the reuse of workflows by scientists and their portability over different HPC infrastructures.