Optimization of Large Scale HEP Data Analysis in LHCb

• Published in: Journal of physics. Conference series 2011-12, Vol.331 (7), p.072060-8
• Main Authors: Remenska, Daniela, Aaij, Roel, Raven, Gerhard, Merk, Marcel, Templon, Jeff, Bril, Reinder J
• Format: Article
• Language: English
• Subjects: Caching; Central processing units; Computation; Data analysis; Data processing; Efficiency; Farms; Input output analysis; Optimization; Physicists; Physics; Remote searching; Reproduction
• ISSN: 1742-6596; 1742-6588; 1742-6596
• DOI: 10.1088/1742-6596/331/7/072060

Observation has lead to a conclusion that the physics analysis jobs run by LHCb physicists on a local computing farm (i.e. non-grid) require more efficient access to the data which resides on the Grid. Our experiments have shown that the I/O bound nature of the analysis jobs in combination with the latency due to the remote access protocols (e.g. rfio, dcap) cause a low CPU efficiency of these jobs. In addition to causing a low CPU efficiency, the remote access protocols give rise to high overhead (in terms of amount of data transferred). This paper gives an overview of the concept of pre-fetching and caching of input files in the proximity of the processing resources, which is exploited to cope with the I/O bound analysis jobs. The files are copied from Grid storage elements (using GridFTP), while concurrently performing computations, inspired from a similar idea used in the ATLAS experiment. The results illustrate that this file staging approach is relatively insensitive to the original location of the data, and a significant improvement can be achieved in terms of the CPU efficiency of an analysis job. Dealing with scalability of such a solution on the Grid environment is discussed briefly.