Curating global datasets of structural linguistic features for independence

• Published in: Scientific data 2025-01, Vol.12 (1), p.106-23, Article 106
• Main Authors: Graff, Anna, Chousou-Polydouri, Natalia, Inman, David, Skirgård, Hedvig, Lischka, Marc, Zakharko, Taras, Barbieri, Chiara, Bickel, Balthasar
• Format: Article
• Language: English
• Subjects: 631/181/1403/2473; 631/181/19; 706/689/19; Data Curation; Data Descriptor; Databases, Factual; Datasets; Humanities and Social Sciences; Humans; Language; Linguistics; multidisciplinary; Science; Science (multidisciplinary); Statistical analysis; Statistics
• ISSN: 2052-4463; 2052-4463
• DOI: 10.1038/s41597-024-04319-4

The increasing availability of cross-linguistic databases dedicated to documenting morphosyntactic, lexical and phonological features has proliferated the use of such data for studies on language evolution and human history. However, most of these databases were not designed to ensure independence of features, such that it is not valid to jointly use all their features in large-scale statistical analyses assuming independence of inputs. Here, we curate published data from five large linguistic databases to generate two global-scale cross-linguistic datasets: GBI (from the Grambank dataset), and TLI (using inputs from the World Atlas of Language Structures, AUTOTYP, PHOIBLE and Lexibank). The datasets minimize logical dependencies of features and forms of strong statistical dependencies that go beyond phylogenetic and geographical signal. They are also made available in densified form, reducing the proportion of missing data. We document our curation principles and workflows to ensure reusability of this framework with other inputs or thresholds of independence. Our curation steps on both datasets reveal robust and comparable global patterns of structural linguistic diversity.