- With an overall funding of €7,6M, the eFlows4HPC project enhances complex HPC workflows thanks to big data analytics and AI techniques
- HPC Workflows as a Service (HPCWaaS) and eFlows4HPC software stack notably boost scientific results for natural disasters, climate change and manufacturing processes
- A broad spectrum of users from various scientific domains will benefit from this platform with better tools for the development, deployment and execution of complex workflows improving overall productivity
After three years of dedicated research and collaborative efforts, the eFlows4HPC project has successfully delivered a comprehensive workflow platform and an additional set of services facilitating the seamless integration of High-Performance Computing (HPC) simulation and modeling with big data analytics and machine learning techniques. Significant advancements in defining complex workflows resulted in enhanced workflow efficiency, and increased overall development and deployment productivity. A variety of scientific and industrial domains, including geosciences, climate science, manufacturing, and other initiatives within the European HPC and scientific applications ecosystem, will benefit from this innovative technology, reflecting the wide-reaching and transformative impact of the project.
The two major project outcomes, High-Performance Computing Workflows as a Service (HPCWaaS) and the project software stack, are crucial in supporting the development, deployment and execution of complex workflows. These advancements play a key role in tackling challenges related to natural disasters, climate change and the optimization of manufacturing processes. HPCWaaS helps users and developers to manage complex scientific workflows by simplifying their lifecycle steps and widening the access to HPC for newcomers. Additionally, the project has released as open source the eFlows4HPC software stack, which is versatile and applicable across numerous scientific and industrial applications requiring HPC, AI and big data techniques.
Rosa M. Badia, eFlows4HPC coordinator, thinks that “the achieved results mark a significant milestone, demonstrating the versatility of the eFlows4HPC software stack in leveraging HPC resources. This translates to a notable decrease in time-to-solution for application developers that need to combine traditional HPC with AI or big data in a single workflow.” She thinks that “we are shaping the future of HPC workflows and empowering researchers to tackle emerging challenges that require complex workflows.”
eFlows4HPC impact: revolutionising HPC Workflows
eFlows4HPC consortium has been highly engaged with multiple initiatives in order to ensure broader adoption of the eFlows4HPC methodologies by other scientific and industrial communities. Joint collaboration activities have been organized with other European initiatives and HPC Centers of Excellence (CoEs), such as ESiWACE3 (climate and weather simulations) and PerMedCoE (personalized medicine) communities. The project has also supported ChEESE and ChEESE-2P by developing complex workflows for earthquake impact simulation. eFlows4HPC methodologies will also be used for the development of workflows in DT-GEO, that aims to build a digital twin for geophysical extremes, and the CAELESTIS project. Finally, the SKA Regional Centres, the regional support network for the international initiative providing square-kilometre array telescopes, showed interest in evaluating the HPCWaaS methodology.
Additional resources:
Key Exploitable results : https://eflows4hpc.eu/key-exploitable-results/
Infographics: https://eflows4hpc.eu/wp-content/uploads/2024/04/Insights.pdf and https://eflows4hpc.eu/wp-content/uploads/2024/04/Data.pdf
HPCWaaS step-by-step guide: https://www.youtube.com/watch?v=2nCUjHoA3s0
Success stories: https://eflows4hpc.eu/success-stories/
Software repository: https://eflows4hpc.eu/software/
Press release in PDF
The eFlows4HPC project has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 955558. The JU receives support from the European Union’s Horizon 2020 research and innovation programme and Spain, Germany, France, Italy, Poland, Switzerland, Norway. It also received funding from MCIN/AEI/10.13039/501100011033 and the European Union NextGenerationEU/PRTR (PCI2021-121957).