An adaptable parallel algorithm for the direct numerical simulation of incompressible turbulent flows using a Fourier spectral/hp element method and MPI virtual topologies
Comput. Phys. Commun., vol. 206, pp. 17–25 (2016)
@article{bolis-2016,
title = {{An adaptable parallel algorithm for the direct numerical simulation of incompressible turbulent flows using a Fourier spectral/hp element method and MPI virtual topologies}},
author = {Bolis, A. and Cantwell, C. D. and Moxey, D. and Serson, D. and Sherwin, S. J.},
journal = cpc,
volume = {206},
pages = {17--25},
year = {2016},
abstract = {A hybrid parallelisation technique for distributed memory systems is investigated for a coupled Fourier-\spectralhp element discretisation of domains characterised by geometric homogeneity in one or more directions. The performance of the approach is mathematically modelled in terms of operation count and communication costs for identifying the most efficient parameter choices. The model is calibrated to target a specific hardware platform after which it is shown to accurately predict the performance in the hybrid regime. The method is applied to modelling turbulent flow using the incompressible Navier-Stokes equations in an axisymmetric pipe and square channel. The hybrid method extends the practical limitations of the discretisation, allowing greater parallelism and reduced wall times. Performance is shown to continue to scale when both parallelisation strategies are used.},
doi = {10.1016/j.cpc.2016.04.011},
url = {http://www.sciencedirect.com/science/article/pii/S001046551630100X}
}
Where a domain is geometrically uniform in one or more directions, a Fourier expansion can be combined with a spectral/hp element discretisation, and the two can be parallelised in different ways. This paper models the operation count and communication cost of the hybrid scheme to identify the most efficient split, calibrates the model to a particular machine, and applies it to turbulent flow in an axisymmetric pipe and a square channel.
Abstract
A hybrid parallelisation technique for distributed memory systems is investigated for a coupled Fourier-\spectralhp element discretisation of domains characterised by geometric homogeneity in one or more directions. The performance of the approach is mathematically modelled in terms of operation count and communication costs for identifying the most efficient parameter choices. The model is calibrated to target a specific hardware platform after which it is shown to accurately predict the performance in the hybrid regime. The method is applied to modelling turbulent flow using the incompressible Navier-Stokes equations in an axisymmetric pipe and square channel. The hybrid method extends the practical limitations of the discretisation, allowing greater parallelism and reduced wall times. Performance is shown to continue to scale when both parallelisation strategies are used.