Early Integration Testing for Entity Reconciliation in the Context of Heterogeneous Data Sources

• Published in: IEEE transactions on reliability 2018-06, Vol.67 (2), p.538-556
• Main Authors: Blanco, Raquel, Enriquez, Jose G., Dominguez-Mayo, Francisco J., Escalona, M. J., Tuya, Javier
• Format: Article
• Language: English
• Subjects: Big Data; Business; Data models; Early testing; entity reconciliation; Erbium; heterogeneous data sources; Information and communication technology; model-driven engineering; Software; software testing; specification-based testing; Testing
• ISSN: 0018-9529; 1558-1721
• DOI: 10.1109/TR.2018.2809866

Entity reconciliation (ER) aims to combine data from different sources for a unified vision. The management of large volumes of data has given rise to significant challenges to the ER problem due to facts such as data becoming more unstructured, unclean, and incomplete or the existence of many datasets that store information about the same topic. Testing the applications that implement the ER problem is crucial to ensure both the correctness of the reconciliation process and the quality of the reconciled data. This paper presents an approach based on model-driven engineering that allows the creation of test models for the early integration testing of ER applications, contributing in three main aspects: the description of the elements of the proposed framework, the definition of the testing model, and the validation of the proposal through two real-world case studies. This validation verifies that the early integration testing of the ER application is capable of detecting a series of deficiencies, which a priori are not known and that will help to improve the final result that the ER application offers.