Caching systems

Research conducted by the University of Wuppertal.
 

Originally, the goal of our work in Research Area II was developing and setting up data lakes. However, after thorough research, we determined that existing tools for monitoring local storage systems are already quite effective. So we focused on optimizing caching systems, which play a crucial role in enhancing data access and performance.

Our plan for working with caching systems includes several steps:

  • dCache instance setup across multiple institutions, including DESY, the University of Hamburg, and the University of Wuppertal.
  • XCache deployment at the University of Wuppertal.
  • Performance studies conducted under real-world conditions, particularly at the ATLAS Tier-2 facility in Wuppertal. This includes focusing on workflows like end-user data analysis.
  • COBalD/TARDIS integration study.
  • RUCIO cache-awareness is enhanced by using both dCache and XCache, minimizing unnecessary data transfers and improving job placement efficiency.

Currently, XCache is deployed on Kubernetes through Service Layer at the Edge (SLATE), a system that simplifies service deployment and configuration. Our next steps involve testing XCache in its official capacity and exploring ways to simplify its setup, making it more accessible for opportunistic use.

Future work

As part of our ongoing collaboration with partners at DESY, CERN, and the University of Hamburg, we’re focusing on the following tasks for the FIDIUM extension:

  • Evaluating caching solutions by studying the current status of dCache and XCache to assess their effectiveness as caching solutions.
  • Simplifying setup and operations by developing easier methods for setting up and managing caching systems, making them more user-friendly and efficient.
  • Integration with RUCIO includes integrating caching systems into the RUCIO data management system, streamlining data handling, and improving performance.
Cookie Consent mit Real Cookie Banner