Requirements for tools for heterogeneous resources integration
Research conducted by Goethe University Frankfurt.
Within Research Area I, the following requirements have been identified for tools that can be used in heterogeneous resources integration:
- Minimization of dependencies. Reliance on third-party software should be reduced.
- Simplified installation. Installation requirements should be decreased for easier setup.
- Enhanced portability. Application portability can be increased by simplifying installation and environment configuration.
- Singularity principle. This principle allows creating and running containers that package software in a portable and reproducible way.
- Large computational resources. Such resources are required to process large volumes of High Energy Physics data.
- Reliable workload manager. The tool should be powerful and reliable enough to manage large computing clusters effectively.
SLURM as workload manager
During the research, the performance of several representative reconstruction algorithms when processed on CPUs and GPU has been compared. In the result, SLURM has been identified as a suitable workload manager because it is:
- Free and open-source. Its usage does not require extra costs.
- Management functionality. The tool is able to manage access to computing resources, resolve conflicts, and collect statistics efficiently.
Current state
At the moment, the following tasks have been performed/are being performed using SLURM:
- Test Tasks. Simple test tasks have been run successfully.
- Tasks with customized scripts. Tasks that use customized scripts have been performed successfully.
- Container-based computing with Singularity is being organized.
Future work
During future work on the heterogeneous resources integration, scripts and containers for automated execution of reconstruction tasks for the CBM experiment are going to be prepared.