Integrated monitoring of the ATLAS online computing farm

The online farm of the ATLAS experiment at the LHC, consisting of nearly 4100 PCs with various characteristics, provides configuration and control of the detector and performs the collection, processing, selection and conveyance of event data from the front-end electronics to mass storage. The statu...

Full description

Saved in:
Bibliographic Details
Published in:Journal of physics. Conference series 2017-10, Vol.898 (9), p.92008
Main Authors: Ballestrero, S, Brasolin, F, Fazio, D, Gament, C, Lee, C J, Scannicchio, D A, Twomey, M S
Format: Article
Language:English
Subjects:
Configurations
Ganglia
Load distribution (forces)
On-line systems
Physics
Citations: Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The online farm of the ATLAS experiment at the LHC, consisting of nearly 4100 PCs with various characteristics, provides configuration and control of the detector and performs the collection, processing, selection and conveyance of event data from the front-end electronics to mass storage. The status and health of every host must be constantly monitored to ensure the correct and reliable operation of the whole online system. This is the first line of defense, which should not only promptly provide alerts in case of failure but, whenever possible, warn of impending issues. The monitoring system should be able to check up to 100000 health parameters and provide alerts on a selected subset. In this paper we present the implementation and validation of our new monitoring and alerting system based on Icinga 2 and Ganglia. We describe how the load distribution and high availability features of Icinga 2 allowed us to have a centralised but scalable system, with a configuration model that allows full flexibility while still guaranteeing complete farm coverage. Finally, we cover the integration of Icinga 2 with Ganglia and other data sources, such as SNMP for system information and IPMI for hardware health.
ISSN:1742-6588
1742-6596
DOI:10.1088/1742-6596/898/9/092008