Are data-mining techniques useful for selecting ecological indicators in biodiverse regions? Bridges between market basket analysis and indicator value analysis from a case study in the neotropics

• Published in: Ecological indicators 2020-02, Vol.109, p.105833, Article 105833
• Main Authors: Leote, Pedro, Cajaiba, Reinaldo Lucas, Cabral, João Alexandre, Brescovit, Antônio Domingos, Santos, Mário
• Format: Article
• Language: English
• Subjects: Data mining; Indicator value analysis; Market basket analysis; Method comparison; Soil surface dwelling arthropods
• ISSN: 1470-160X; 1872-7034
• DOI: 10.1016/j.ecolind.2019.105833

[Display omitted] •Indicator Value Analysis (IndVal) is the standard for ecological indicators selection.•IndVal shortcomings are mostly associated with large datasets processing.•Market Basket Analysis (MBA) indicators selection was compared with Indval outputs.•MBA has a more efficient algorithm, IndVal is better at summarisation.•MBA was proposed to complement IndVal, especially for large datasets. Ecological monitoring research relies heavily on signals to detect ecosystem changes, making the selection of indicators a crucial methodological requirement. Over the years, individual species and species assemblages have been widely used, thereby, giving rise to reference methods that support the detection of ecological indicators. One such method, the Indicator Value Analysis (IndVal), has been adapted to identify not only species but also combinations of species, assuming collective responses to environmental factors. However, the IndVal method requires a pre-selection of species before performing the analysis, especially in the case of large datasets (e.g. high species richness), when it becomes ineffective. Species pre-selection might introduce subjectivity and a bias into the database, which can cause possible impacts on the final set of indicators. To address these issues, the authors propose the use of Market Basket Analysis (MBA) – a data mining method – which is mathematically similar to IndVal but designed to handle large amounts of data. Both methods were applied to select indicators from gradually larger datasets of Soil Surface Dwelling Arthropods from the Brazilian Amazon, using threshold-dependent indices to assess concordance between results. In general, the results obtained by applying both methods were found to be similar, with an average Jaccard's distance of 0.432 (±0.346) and an average True Skill Statistic of 0.991 (±0.012). As expected, MBA was able to select ecological indicators without species pre-selection as well as from datasets where IndVal had been unsuccessful. In such cases, and by means of objective association rules, the authors demonstrate that MBA could be used to pre-select ecological indicators, which can then be further processed and summarized with the IndVal method. In this study, the authors briefly outline the potential of MBA to complement IndVal and discuss advantages and disadvantages of using MBA for ecological indicators (pre-) selection.