An Efficient Strategy for Online Performance Monitoring of Datacenters via Adaptive Sampling

• Published in: IEEE transactions on cloud computing 2019-01, Vol.7 (1), p.155-169
• Main Authors: Huang, Tingshan, Kandasamy, Nagarajan, Sethu, Harish, Stamm, Matthew C.
• Format: Article
• Language: English
• Subjects: Adaptive sampling; Anomalies; anomaly detection; Capacity planning; Cloud computing; compressive sampling; Data centers; Monitoring; Online monitoring; Provisioning; Resource allocation; Sensors; Servers; Videos
• ISSN: 2168-7161; 2168-7161; 2372-0018
• DOI: 10.1109/TCC.2016.2603473

Performance monitoring of datacenters provides vital information for dynamic resource provisioning, anomaly detection, and capacity planning decisions. Online monitoring, however, incurs a variety of costs: the very act of monitoring a system interferes with its performance, consuming network bandwidth and disk space. With the goal of reducing these costs, this paper develops and validates a strategy based on adaptive-rate compressive sampling. It exploits the fact that the signals of interest often can be sparsified under an appropriate representation basis and that the sampling rate can be tuned as a function of sparsity. We use the Trade6 application as our experimental platform and measure the signals of interest-in our case, signals pertaining to memory and disk I/O activity-using adaptive sampling. We then evaluate whether the reconstructed signals can be used for trend detection to track the gradual deterioration of system performance associated with software aging. Our experiments show that the signals recovered by our methods can be used to detect, with high confidence, the existence of trends within the original signal. We also evaluate the reconstructed signals for threshold-violation detection wherein the magnitude of the signal exceeds a preset value. Our experiments show that performance bottlenecks and anomalies that manifest themselves in portions of the signal where its magnitude exceeds a threshold value can also be detected using the reconstructed signals. Most importantly, detection of these anomalies is achieved using a substantially reduced sample size-a reduction of more than 70 percent when compared to the standard fixed-rate sampling method.