EFlow4HPC experts will participate on Sunday, 12 November from 2:42pm – 3pm MST (room 501-502) at the session titled “End-to-End Workflows for Climate Science: Integrating HPC Simulations, Big Data Processing, and Machine Learning” during the SC23 conference that is held in Denver.
Current scientific workflow systems do not typically integrate simulation-centric and data-centric aspects due to their very different software/infrastructure requirements. A transparent integration of such components into a single end-to-end workflow would lead to a more efficient and automated way for generating insights from large simulation data. This work presents a complex case study related to extreme events analysis of future climate data that integrates in the same workflow numerical simulations, Big Data analytics and Machine Learning models. The case study is being implemented in the context of the eFlows4HPC project using the project’s software stack for deployment and orchestration of the workflow. The solution implemented in the project has shown to simplify the development and execution of end-to-end climate workflows with heterogeneous software requirements. Moreover, such an approach can, in the long term, increase the reuse of workflows by scientists and their portability over different HPC infrastructures.
In addition, our BSC researcher Jorge Ejarque will also present at the BoF session titled “Workflows Community: Modern Workflows for Continuum and Cross-Facility Computing” held on Tuesday, 14 November from 12:15pm – 1:15pm MST.
The final presentation has been uploaded here: https://eflows4hpc.eu/publication/automatic-steering-in-large-task-based-workflows/