Optimizing the performance of the spectral/hp element method with collective linear algebra operations
Comput. Meth. Appl. Mech. Eng., vol. 310, pp. 628–645 (2016)
@article{moxey-2016b,
title = {Optimizing the performance of the spectral/hp element method with collective linear algebra operations},
author = {Moxey, D. and Cantwell, C. D. and Kirby, R. M. and Sherwin, S. J.},
journal = cmame,
volume = {310},
pages = {628--645},
year = {2016},
abstract = {As high-performance computing hardware evolves, increasing core counts mean that memory bandwidth is becoming the deciding factor in attaining peak CPU performance. Methods that make efficient use of memory and caches are therefore essential for modern hardware. High-order finite element methods, such as those implemented in the spectral/hp framework \nekpp, are particularly well-suited to this environment. Unlike low-order methods that typically utilize sparse storage, matrices representing high-order operators have greater density and richer structure. In this paper, we show how these qualities can be exploited to increase runtime performance by amalgamating the action of key operators on multiple elements into a single, memory-efficient block. We investigate different strategies for achieving optimal performance across a range of polynomial orders and element types. As these strategies all depend on external factors such as BLAS implementation and the geometry of interest, we present a technique for automatically selecting the most efficient strategy at runtime.},
url = {http://www.sciencedirect.com/science/article/pii/S0045782516306739},
doi = {10.1016/j.cma.2016.07.001}
}
As core counts rise it is memory bandwidth, not arithmetic, that decides how close a code gets to peak performance. Matrices representing high- order operators are denser and more structured than the sparse ones low- order methods use, and this paper exploits that by amalgamating the action of an operator on many elements into a single memory-efficient block. Since the best strategy depends on the BLAS implementation and the geometry, it is chosen automatically at runtime.
Abstract
As high-performance computing hardware evolves, increasing core counts mean that memory bandwidth is becoming the deciding factor in attaining peak CPU performance. Methods that make efficient use of memory and caches are therefore essential for modern hardware. High-order finite element methods, such as those implemented in the spectral/hp framework \nekpp, are particularly well-suited to this environment. Unlike low-order methods that typically utilize sparse storage, matrices representing high-order operators have greater density and richer structure. In this paper, we show how these qualities can be exploited to increase runtime performance by amalgamating the action of key operators on multiple elements into a single, memory-efficient block. We investigate different strategies for achieving optimal performance across a range of polynomial orders and element types. As these strategies all depend on external factors such as BLAS implementation and the geometry of interest, we present a technique for automatically selecting the most efficient strategy at runtime.