Mining local and tail dependence structures based on pointwise mutual information

• Published in: Data mining and knowledge discovery 2012-01, Vol.24 (1), p.78-102
• Main Author: Takada, Teruko
• Format: Article
• Language: English
• Subjects: Artificial Intelligence; Causality; Chemistry and Earth Sciences; Computer Science; Data mining; Data Mining and Knowledge Discovery; Datasets; Dynamical systems; Estimators; Information Storage and Retrieval; Mining; Nonlinearity; Physics; Risk; Statistical significance; Statistics for Engineering; Stock exchanges
• ISSN: 1384-5810; 1573-756X
• DOI: 10.1007/s10618-011-0220-3

The behavior of events that occur infrequently but have a large impact tends to differ from that of the central tendency, and identifying the tail dependence structure among key factors is critical for controlling risks. However, due to technical difficulties, conventional analyses of dependence have focused on the global average dependence. This article proposes a novel approach for analyzing the entire structure of nonlinear dependence between two data sets on the basis of accurate pointwise mutual information estimation. The emphasis is on fat-tailed distributions that tend to appear in events involving sudden large changes. The proposed pointwise mutual information estimator is sufficiently robust and efficient for exploring tail dependence, and its good performance was confirmed in an experimental study. The significance of the identified dependence structure was assessed using the proposed bootstrap procedure. New facts were discovered from its application to daily returns and volume on the New York stock Exchange (NYSE) Composite Index.