The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop: High Performance Python for Science at Scale
Authors: Julius Donnert (Hewlett Packard Enterprise (HPE)), Lindsey Gordon (University of Minnesota), Christopher Nolting (Gustavus Adolphus College), and Peter Mendygral (Hewlett Packard Enterprise (HPE))
Abstract: We present a prototype solution for a polyglot computational fluid dynamics code using the Python Multiprocessing API and Dragon. The code uses an actor-based dataflow architecture with a directed graph to explicitly express program execution including parallelization and asynchronous communication. Computation-heavy parts are covered with individual Fortran executables, the shared state description is written in C with Fortran and Cython wrappers. Our code demonstrates data flow programming in Python for a classical tightly coupled HPC problem, combining cloud-native programming paradigms with HPC communication techniques like RDMA through the Dragon runtime. We demonstrate how a scalable software architecture for classical HPC, AI/ML and HPC workflow applications could look like in the future.
Back to High Performance Python for Science at Scale Archive Listing