HCube: Routing and similarity search in Data Centers

• Published in: Journal of network and computer applications 2016-01, Vol.59, p.386-398
• Main Authors: Villaça, R.S., Pasquini, R., de Paula, L.B., Magalhães, M.F.
• Format: Article
• Language: English
• Subjects: Big Data; Data centre; Flat routing; Hamming similarity; Similarity search
• ISSN: 1084-8045; 1095-8592
• DOI: 10.1016/j.jnca.2014.08.012

The current Big Data scenario is mainly characterized by the huge amount of data available on the Internet. Some deployed mechanisms for handling such raw data rely on Data Centres (DCs) based on massive storage, memory and processing capacity, in which solutions like BigTable, MapReduce and Dynamo process information in order to provide its retrieval. The HCube presents a DC alternative for data storage/retrieval based on the similarity search, in which similar content is concentrated on servers physically close within the HCube, simplifying the recovery of similar data. A similarity search is performed using a primitive get(k,sim), in which k represents the reference content and sim a similarity threshold. The HCube network is organized in a three dimensional structure, in which the Gray Space Filling Curve (SFC) in conjunction with the Random Hyperplane Hashing (RHH) function and the XOR-based flat routing mechanism offer an efficient and powerful mechanism for the similarity search. In this context, this work presents the HCube networking solution, detailing the benefits of using the Gray SFC and the XOR-based flat routing mechanism for the similarity search.