BIPeC: A Combined Change-Point Analyzer to Identify Performance Regressions in Large-scale Database Systems

Performance testing in large-scale database systems like SAP HANA is a crucial yet labor-intensive task, involving extensive manual analysis of thousands of measurements, such as CPU time and elapsed time. Manual maintenance of these metrics is time-consuming and susceptible to human error, making e...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-08
Main Authors: Lyu, Zhan, Bach, Thomas, Li, Yong, Nguyen Minh Le, Hoemke, Lars
Format: Article
Language:English
Subjects:
Algorithms
Bayesian analysis
Data base management systems
Data management
Error analysis
Error detection
Human error
Human performance
Statistical inference
Systems analysis
Time measurement
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Performance testing in large-scale database systems like SAP HANA is a crucial yet labor-intensive task, involving extensive manual analysis of thousands of measurements, such as CPU time and elapsed time. Manual maintenance of these metrics is time-consuming and susceptible to human error, making early detection of performance regressions challenging. We address these issues by proposing an automated approach to detect performance regressions in such measurements. Our approach integrates Bayesian inference with the Pruned Exact Linear Time (PELT) algorithm, enhancing the detection of change points and performance regressions with high precision and efficiency compared to previous approaches. Our method minimizes false negatives and ensures SAP HANA's system's reliability and performance quality. The proposed solution can accelerate testing and contribute to more sustainable performance management practices in large-scale data management environments.
ISSN:2331-8422