Using Fuzzy Matching of Queries to optimize Database workloads

Directed Acyclic Graphs (DAGs) are commonly used in Databases and Big Data computational engines like Apache Spark for representing the execution plan of queries. We refer to such graphs as Query Directed Acyclic Graphs (QDAGs). This paper uses similarity hashing to arrive at a fingerprint such that...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2022-07
Main Authors: Singh, Sweta, Kulkarni, Vaibhav, Briggs, Mario, Mahajan, Deepak, Farchi, Eitan
Format: Article
Language:English
Subjects:
Fingerprints
Graph theory
Graphical representations
Graphs
Queries
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Directed Acyclic Graphs (DAGs) are commonly used in Databases and Big Data computational engines like Apache Spark for representing the execution plan of queries. We refer to such graphs as Query Directed Acyclic Graphs (QDAGs). This paper uses similarity hashing to arrive at a fingerprint such that the fingerprint embodies the compute requirements of the query for QDAGs. The fingerprint, thus obtained, can be used to predict the runtime behaviour of a query based on queries executed in the past having similar QDAGs. We discuss two approaches to arrive at a fingerprint, their pros and cons and how aspects of both approaches can be combined to improve the predictions. Using a hybrid approach, we demonstrate that we are able to predict runtime behaviour of a QDAG with more than 80% accuracy.
ISSN:2331-8422