The International Conference for High Performance Computing, Networking, Storage, and Analysis
The International Conference for High Performance Computing, Networking, Storage, and Analysis
Workshop: RSDHA: Redefining Scalability for Diversely Heterogeneous Architectures
Authors: Zheming Jin (Oak Ridge National Laboratory (ORNL)) and Jeffrey Vetter (IEEE Computer Society)
Abstract: A deep neural network library (DNNL) is an optimized library of low-level computational primitives for deep neural networks. In this study, we choose the softmax function, a primitive commonly used in new computing models for DNNs, as a case study on evaluating the unique programming models adopted by the vendors’ DNNLs (cuDNN, MIOpen, and oneDNN) and the performance and portability of DNNLs on NVIDIA and AMD GPUs. We find that cuDNN selects different compute kernels to execute based on the problem size for the primitive, which may have a significant performance impact. oneDNN successfully enables functional portability of the primitive across vendors’ platforms, but performance portability will need to be improved. In addition, the performance of a primitive in the DNNLs may be suboptimal compared to a custom implementation.
Back to RSDHA: Redefining Scalability for Diversely Heterogeneous Architectures Archive Listing