Loading…

Dynamic constraints for record matching

This paper investigates constraints for matching records from unreliable data sources. (a) We introduce a class of matching dependencies ( md s) for specifying the semantics of unreliable data. As opposed to static constraints for schema design, md s are developed for record matching, and are define...

Full description

Saved in:
Bibliographic Details
Published in:The VLDB journal 2011-08, Vol.20 (4), p.495-520
Main Authors: Fan, Wenfei, Gao, Hong, Jia, Xibei, Li, Jianzhong, Ma, Shuai
Format: Article
Language:English
Subjects:
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper investigates constraints for matching records from unreliable data sources. (a) We introduce a class of matching dependencies ( md s) for specifying the semantics of unreliable data. As opposed to static constraints for schema design, md s are developed for record matching, and are defined in terms of similarity predicates and a dynamic semantics . (b) We identify a special case of md s, referred to as relative candidate keys ( rck s), to determine what attributes to compare and how to compare them when matching records across possibly different relations. (c) We propose a mechanism for inferring md s, a departure from traditional implication analysis, such that when we cannot match records by comparing attributes that contain errors, we may still find matches by using other, more reliable attributes. Moreover, we develop a sound and complete system for inferring md s. (d) We provide a quadratic-time algorithm for inferring md s and an effective algorithm for deducing a set of high-quality rck s from md s. (e) We experimentally verify that the algorithms help matching tools efficiently identify keys at compile time for matching, blocking or windowing and in addition, that the md -based techniques effectively improve the quality and efficiency of various record matching methods.
ISSN:1066-8888
0949-877X
DOI:10.1007/s00778-010-0206-6