libHPC: Software sustainability and reuse through metadata preservation
First Workshop on Maintainable Software Practices in e-Science, 8th IEEE International Conference on eScience (2012)
@inproceedings{cohen-2012,
title = {libHPC: Software sustainability and reuse through metadata preservation},
booktitle = {First Workshop on Maintainable Software Practices in e-Science, 8th IEEE International Conference on eScience},
author = {Cohen, J. and Darlington, J. and Fuchs, B. and Moxey, D. and Cantwell, C. D. and Burovskiy, P. and Sherwin, S. J. and Hong, N. P. Chue},
year = {2012},
url = {https://davidmoxey.uk/assets/pubs/2012-escience.pdf},
abstract = {Software development, particularly of complex scientific applications, requires a detailed understanding of the problem(s) to be solved and an ability to translate this understanding into the generic constructs of a programming language. We believe that such knowledge -- information about a code's ``building blocks'', especially the low-level functions and procedures in which domain-specific tasks are implemented -- can be very effectively leveraged to optimise code execution across platforms and operating systems. However, all too often such knowledge gets lost during the development process, which can bury the scientist's understanding in the code in a manner that makes it difficult to recover or extract later on. In this paper, we describe our work in the EPSRC-funded libHPC project to build a framework that captures and utilises this information to achieve optimised performance in dynamic, heterogeneous networked execution environments. The aim of the framework is to allow scientists to work in high-level scripting environments based on component libraries to provide descriptions of applications which can then be mapped to optimal execution configurations based on available resources. A key element in our approach is the use of ``co-ordination forms'' -- or functional paradigms -- for creating optimised execution plans from components. Our main exemplar application is an advanced finite element framework, Nektar++, and we detail ongoing work to undertake profiling and performance analysis to extract software metadata and derive optimal execution configurations, to target resources based on their hardware metadata.}
}
Much of what a scientist understands about a problem ends up buried in code in a way that is hard to recover later. This paper describes the libHPC project's framework for capturing that understanding as metadata about a code's building blocks, and using it to map an application onto an efficient execution plan for whatever resources happen to be available.
Abstract
Software development, particularly of complex scientific applications, requires a detailed understanding of the problem(s) to be solved and an ability to translate this understanding into the generic constructs of a programming language. We believe that such knowledge – information about a code’s “building blocks”, especially the low-level functions and procedures in which domain-specific tasks are implemented – can be very effectively leveraged to optimise code execution across platforms and operating systems. However, all too often such knowledge gets lost during the development process, which can bury the scientist’s understanding in the code in a manner that makes it difficult to recover or extract later on. In this paper, we describe our work in the EPSRC-funded libHPC project to build a framework that captures and utilises this information to achieve optimised performance in dynamic, heterogeneous networked execution environments. The aim of the framework is to allow scientists to work in high-level scripting environments based on component libraries to provide descriptions of applications which can then be mapped to optimal execution configurations based on available resources. A key element in our approach is the use of “co-ordination forms” – or functional paradigms – for creating optimised execution plans from components. Our main exemplar application is an advanced finite element framework, Nektar++, and we detail ongoing work to undertake profiling and performance analysis to extract software metadata and derive optimal execution configurations, to target resources based on their hardware metadata.