BSC researchers Rosa M. Badia and Jorge Ejarque will give a lecture titled “Simplifying the life-cycle management of HPC, data analytics and AI workflows” on 5 July from 2:00-5:00PM during the ACM Summer school 2023, that is held in Barcelona.
Abstract
With Exaflop systems already here, High-Performance Computing (HPC) involves everytime larger and complex supercomputers. At the same time, the user community is aware of the underlying performance and eager to leverage it by providing more complex application workflows to leverage them. What is more, current application trends aim to use data analytics and artificial intelligence combined with HPC modeling and simulation. However, the programming models and tools are different in these fields, and there is a need for methodologies that enable the development of workflows that combine HPC software, data analytics, and artificial intelligence. PyCOMPSs is a parallel task-based programming in Python. Based on simple annotations, sequential Python programs can be executed in parallel in HPC-clusters and other distributed infrastructures. PyCOMPSs has been extended to support tasks that invoke HPC applications and can be combined with Artificial Intelligence and Data analytics frameworks. These extensions have been done in the eFlows4HPC project, which aims at providing a workflow software stack that fulfills the previous mentioned need. The project is also developing the HPC Workflows as a Service (HPCWaaS) methodology that aims at providing tools to simplify the development, deployment, execution, and reuse of workflows. In particular, the project has been working on the Container Image Creation service, a component which automates the creation of the container images tailored to a specific HPC platform. The lecture will be composed of two parts. The first part will consist of a presentation about the eFlows4HPC project with an especial emphasis on the PyCOMPSs programming model and the Container Image Creation service. The second part will include a hands-on where the students will be able to practice on the programming with PyCOMPSs and on the creation of HPC-ready container images with the Container Image Creation service.
