... | ... | @@ -32,29 +32,36 @@ |
|
|
- ...
|
|
|
|
|
|
**Meeting notes**
|
|
|
<details><summary>Meeting 16.01. </summary>
|
|
|
|
|
|
- next ESM User forum 23./24.02. online, preliminary agenda: https://gitlab.jsc.fz-juelich.de/hedgedoc/fTfV74qMTgmEWyvhvJLfMg, final agenda and invitation following soon
|
|
|
- feel free to add your JL topics and interests in the tables at the bottom of the activity websites ([Workflow scalability](https://jugit.fz-juelich.de/jl-exaesm/project/-/wikis/Exascale-Workflow-Scalability) and [Code scalability](https://jugit.fz-juelich.de/jl-exaesm/project/-/wikis/Exascale-Code-Scalability))
|
|
|
- please advertise the JL (and the informal coffee meetings) with your collegues within HFG working on HPC topics and ESM
|
|
|
<details><summary>Meeting 17.07.2023 </summary>
|
|
|
|
|
|
- Daniel attended a workshop by [ChEESE](https://cheese-coe.eu/) on "High Performance Computing for solid earth geosciences"
|
|
|
- even though this community is not a traditional user of HPC they are usage is increasing and they face similar issues like our community
|
|
|
- similar approaches are for example coding sprints
|
|
|
- recommendation: lobby our interests with European HPC
|
|
|
- once workshop proceedings are available Daniel will share them
|
|
|
- coding sprints within [natESM](https://www.nat-esm.de/services/support-through-sprints) and [ESiWACE3](https://www.esiwace.eu/services/software-support)
|
|
|
- Lars and Astrid reported from their experience in natESM sprint submission and running a sprint
|
|
|
- no experiences with ESiWACE3 sprints so far but Lars heard that chances are good to get a sprint accepted
|
|
|
</details>
|
|
|
<details><summary>Meeting 27.02. </summary>
|
|
|
|
|
|
- overview of activity _Exascale Workflow Scalability_ (see attached slides) with main topics "Storage Concepts", "Data Services and Exchange" and "Postprocesing and Analysis"
|
|
|
- [20230227_Exascale_Workflow_Scalability.pdf](uploads/1e07bf5f78f40f62ca18815cb6b84fdb/20230227_Exascale_Workflow_Scalability.pdf)
|
|
|
- discussion around presentation
|
|
|
- overlap of topics presented and work by Tobias (CI/CD, compression, automated workflows)
|
|
|
- data bridges: as part of WarmWorld data connection to ECMWF (Olaf)
|
|
|
- Waves2Weather: SFB project with LMU, Uni Mainz and KIT als main lead, review of 3rd phase ongoing (KIT contributing with automated ICON runs)
|
|
|
- possible collaborations with existing projects (in addition to final slide of presentation): ExaOcean (BMBF, SCALEXA), DETECT (DFG), Warm World Module 3 zu CI/CD
|
|
|
- (online) visualization another relevant topic of postprocessing and analysis, e.g. related to ICON at FZJ and DKRZ or bottom-up by Uğur
|
|
|
- I/O Sea project and usage of data nodes for faster analysis (potential future application, Daniel)
|
|
|
- decided that CI/CD relevant topic for both activities
|
|
|
- update activity websites with point-of-contact for subtopics
|
|
|
<details><summary>Meeting 08.05.2023 </summary>
|
|
|
|
|
|
- presentation on _automated workflows_ by Tobias Meisel (UFZ)
|
|
|
- automated workflows including, pre- and postprocessing and execution of simulations with OpenGeoSys based on different workflow components embedded within another (i.e. one workflow calls the next one)
|
|
|
- developed partly during ExaESM Pilotlab
|
|
|
- based on _snake make_ (python tool, appropriate since code in C++ and python): file-system based workflow, one considered alternative is _parsl_
|
|
|
- workflow composed of components (in and output files) and rules (e.g. regridding and temporal statistics)
|
|
|
- can run one experiment but also range of experiments (e.g. based on wildcards or global parameters) in parallel supported with workflow
|
|
|
- if repeat experiment _lazy rules_ available: only changed parts are re-evaluated
|
|
|
- different types of workflows developed: e.g. for checking correctness, for evaluating performance/efficiency or determining reproducibility, for analysing impact of specific parameters
|
|
|
- a full workflow is specific to a project but components and rules can be re-used for one model
|
|
|
- repository with OpenGeoSys workflows not open to public but planned to wrap out generic components and make them available
|
|
|
- please contact Tobias directly if you have further questions
|
|
|
- Lars: consider registering your software with the Helmholtz Research Software Directory (HSD): [https://helmholtz.software/](https://helmholtz.software/)
|
|
|
|
|
|
</details>
|
|
|
|
|
|
<details><summary>Meeting 20.03. </summary>
|
|
|
<details><summary>Meeting 20.03.2023 </summary>
|
|
|
|
|
|
- overview of activity _Exascale Code Scalability_ by Daniel CV
|
|
|
- aspects of Exascale readiness: scalability; resource and energy efficiency; portability and future-proofing
|
... | ... | @@ -71,19 +78,26 @@ |
|
|
- next presentation (8 May): Tobias Meisel (UFZ) on automated workflows
|
|
|
</details>
|
|
|
|
|
|
<details><summary>Meeting 08.05. </summary>
|
|
|
<details><summary>Meeting 27.02. </summary>
|
|
|
|
|
|
- presentation on _automated workflows_ by Tobias Meisel (UFZ)
|
|
|
- automated workflows including, pre- and postprocessing and execution of simulations with OpenGeoSys based on different workflow components embedded within another (i.e. one workflow calls the next one)
|
|
|
- developed partly during ExaESM Pilotlab
|
|
|
- based on _snake make_ (python tool, appropriate since code in C++ and python): file-system based workflow, one considered alternative is _parsl_
|
|
|
- workflow composed of components (in and output files) and rules (e.g. regridding and temporal statistics)
|
|
|
- can run one experiment but also range of experiments (e.g. based on wildcards or global parameters) in parallel supported with workflow
|
|
|
- if repeat experiment _lazy rules_ available: only changed parts are re-evaluated
|
|
|
- different types of workflows developed: e.g. for checking correctness, for evaluating performance/efficiency or determining reproducibility, for analysing impact of specific parameters
|
|
|
- a full workflow is specific to a project but components and rules can be re-used for one model
|
|
|
- repository with OpenGeoSys workflows not open to public but planned to wrap out generic components and make them available
|
|
|
- please contact Tobias directly if you have further questions
|
|
|
- Lars: consider registering your software with the Helmholtz Research Software Directory (HSD): [https://helmholtz.software/](https://helmholtz.software/)
|
|
|
- overview of activity _Exascale Workflow Scalability_ (see attached slides) with main topics "Storage Concepts", "Data Services and Exchange" and "Postprocesing and Analysis"
|
|
|
- [20230227_Exascale_Workflow_Scalability.pdf](uploads/1e07bf5f78f40f62ca18815cb6b84fdb/20230227_Exascale_Workflow_Scalability.pdf)
|
|
|
- discussion around presentation
|
|
|
- overlap of topics presented and work by Tobias (CI/CD, compression, automated workflows)
|
|
|
- data bridges: as part of WarmWorld data connection to ECMWF (Olaf)
|
|
|
- Waves2Weather: SFB project with LMU, Uni Mainz and KIT als main lead, review of 3rd phase ongoing (KIT contributing with automated ICON runs)
|
|
|
- possible collaborations with existing projects (in addition to final slide of presentation): ExaOcean (BMBF, SCALEXA), DETECT (DFG), Warm World Module 3 zu CI/CD
|
|
|
- (online) visualization another relevant topic of postprocessing and analysis, e.g. related to ICON at FZJ and DKRZ or bottom-up by Uğur
|
|
|
- I/O Sea project and usage of data nodes for faster analysis (potential future application, Daniel)
|
|
|
- decided that CI/CD relevant topic for both activities
|
|
|
- update activity websites with point-of-contact for subtopics
|
|
|
</details>
|
|
|
|
|
|
|
|
|
<details><summary>Meeting 16.01.2023 </summary>
|
|
|
|
|
|
- next ESM User forum 23./24.02. online, preliminary agenda: https://gitlab.jsc.fz-juelich.de/hedgedoc/fTfV74qMTgmEWyvhvJLfMg, final agenda and invitation following soon
|
|
|
- feel free to add your JL topics and interests in the tables at the bottom of the activity websites ([Workflow scalability](https://jugit.fz-juelich.de/jl-exaesm/project/-/wikis/Exascale-Workflow-Scalability) and [Code scalability](https://jugit.fz-juelich.de/jl-exaesm/project/-/wikis/Exascale-Code-Scalability))
|
|
|
- please advertise the JL (and the informal coffee meetings) with your collegues within HFG working on HPC topics and ESM
|
|
|
|
|
|
</details> |
|
|
\ No newline at end of file |