The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Authors: Yiyuan Li, Xiting Ju, Yi Xiao, Qilong Jia, and Yongxiao Zhou (Tsinghua University, China); Simeng Qian (National Supercomputing Center in Wuxi); Rongfen Lin (National Research Center of Parallel Computer Engineering and Technology, China); Bin Yang (Tsinghua University, China); Shupeng Shi (National Supercomputing Center in Wuxi); Xin Liu, Jie Gao, Zhen Wang, Sha Liu, Jian Tan, and Xuan Wang (National Research Center of Parallel Computer Engineering and Technology); Zhengding Hu (University of Science and Technology of China); Limin Yan (Beijing Sankuai Online Technology Co, Ltd; National Supercomputing Center in Wuxi); and Wei Xue (Tsinghua University, China; Department of Computer Technology and Application, Qinghai University)
Abstract: Atmospheric data assimilation is essential for numerical weather prediction. Ensemble-based data assimilation connects multiple instances of atmospheric model through Kalman-filter-based algorithm, which is regarded as a challenging computing task today. In this work, we present our efforts to build a fast, low-cost, and scalable atmospheric data assimilation prototype for the new-generation Sunway supercomputer, including (1) A UNet-neural-network-based surrogate model for atmospheric dynamic simulation to generate all the background ensemble with both satisfactory accuracy and reasonable robustness; (2) Batched LETKF with an efficient eigenvalue decomposition implementation and a data staging strategy to cover the observation IO time ; (3) A framework able to flexibly deploy the components, thus available to reach the maximum resource efficiency. Experimental evaluations show that our AI-integrated ensemble data assimilation prototype can finish hour-cycle assimilation in minutes, keep linear scalability and save an order of magnitude of computing resources compared with the traditional scientific method.
Back to Technical Papers Archive Listing