Birds of a Feather

Current and Future HPC Storage Environments

Authors: Bill Anderson (National Center for Atmospheric Research (NCAR)), Irfan Elahi (National Center for Atmospheric Research (NCAR)), Richard Lawrence (UK Met Office), Gabe Turner (3M), Jesse Caldwell (University of Colorado)

Abstract: Storage is an important part of HPC environments, especially with the explosion of data that comes with increasing computational power. But there are a number of evolving options and tradeoffs for storage (POSIX/S3, SSD/HDD/tape, on-premises/public cloud, management policies, etc.). The goal of this BoF is to facilitate a discussion about storage environments, and to share and hear plans and ideas from the audience and from the BoF leaders. Ultimately, we hope to help each other and the community better understand the options and best practices in the storage landscape.

Long Description: Storage is an important part of HPC environments and choosing the right storage solutions can improve productivity and reduce costs. However, there are many options and tradeoffs in the solution space. There are different types of storage hardware, different user interfaces to storage (e.g., POSIX vs S3), as well as different policies for management, such as tiering policies.

The goal of this BOF is to bring HPC storage folks together to exchange ideas and best practices. We propose to facilitate a discussion on different storage environments to hear what various organizations are doing and to discuss trade-offs between different environments and solutions. The session facilitators are storage experts from different types of organizations, which will bring diverse and helpful perspectives, including an expert from a university (Jesse Caldwell, University of Colorado), an expert from a government weather forecast center (Richard Lawrence, UK Met Office), an expert from industry (Gabe Turner, 3M), and experts from a government research center (Irfan Elahi and Bill Anderson, NCAR).

These experts will briefly share their current and future environments (~1 slide). Then, the BOF leaders will facilitate a discussion on various potential storage solutions. Ultimately, we want this to be an event with a very high amount of audience participation that results in an exchange of ideas that the attendees can use to improve their own storage environments.

Since this is a broad subject, we propose to focus on the following discussion topics and will move on to others as time permits.

- Best practices for managing tiers of storage - Pros and cons of cloud storage - Ideas for reducing storage costs - S3 use cases



Back to Birds of a Feather Archive Listing