The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop: WHPC@SC23: 16th International Women in HPC Workshop
Authors: Chelsea Maria John and Andreas Herten (Forschungzentrum Juelich)
Abstract: In the era of Natural Language Processing with Large Language Models, the OpenGPT-X project brings forth a platform for researching, producing, and using language models. The project is a German initiative with ten collaborative partners, focusing their efforts to contribute a multilingual Open Source language model. Models trained within the project will also be used for pilot cases by industry partners and commercialized through Gaia-X federation. The memory and compute for training large language models efficiently demands high performance computing systems like JUWELS Booster. This paper outlines the advancements and challenges in the project from the perspective of Jülich Supercomputing Centre and showcases the results of exploration of novel hardware architecture conducted within the scope of the project.
Back to WHPC@SC23: 16th International Women in HPC Workshop Archive Listing