The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop: First International Workshop on HPC Testing and Evaluation of Systems, Tools, and Software (HPCTESTS 2023)
Authors: Doug Jacobsen and Bob Bird (Google LLC)
Abstract: Reproducibility and replicability are extremely important components of scientific computing, and any computational research. The ability to replicate a set of experiments aids many other computational use cases, such as systems acceptance where a compute center requires the ability to produce execute the same experiment as a hardware vendor. Several test harnesses and frameworks exist, and attempt to increase the replicability of these experiments.
We introduce Ramble. a new Python-based experimentation framework. Ramble provides a domain specific language for abstracting how experiments can be creating from applications, and a flexible templating engine for creating experiments. Ramble can be used for automating system tests, scientific parameter studies, performance focused benchmarking, and many other software experiments. We will introduce Ramble, describe its architecture, and give some concrete use cases where it can be applied to HPC application experimentation.