Workshops Archive

Operationalizing HPC Tasks for Space Weather Forecasting Using Celery and Django: Making Automated, HPC-Powered Scientific Results Accessible in Near-Real Time.

Workshop: WHPC@SC23: 16th International Women in HPC Workshop

Authors: Jenny Wong (University of Birmingham, UK)

Abstract: Forecasting space weather conditions in the Earth’s ionosphere is critical to protecting key infrastructure, such as satellite-based positioning and navigation systems, high-frequency radio communications, and the electric power grid. Variations in space weather are caused by coronal mass ejections from the Sun’s surface, energizing electrons in the ionosphere to produce disturbances in communications and electrical systems, as well as spectacular aurorae.

We present a system for operationalizing HPC tasks for data assimilation in space weather forecasting using Celery and Django. Celery is used to execute and distribute tasks asynchronously, while Django is a popular web framework. Our system integrates these tools to automate running space weather simulations on an HPC cluster for data assimilation and presenting outputs on a website in near real-time. Our system applies to a wide range of HPC tasks in research software, and we believe this is useful for researchers to operationalize similar workflows.

Back to WHPC@SC23: 16th International Women in HPC Workshop Archive Listing

Back to Full Workshop Archive Listing