Hierarchical Structures Induce Long-Range Dynamical Correlations in Written Texts

• Published in: Proceedings of the National Academy of Sciences - PNAS 2006-05, Vol.103 (21), p.7956-7961
• Main Authors: Alvarez-Lacalle, E., Dorow, B., Eckmann, J.-P., Moses, E.
• Format: Article
• Language: English
• Subjects: Autocorrelation; Connectivity; Correlations; Language; Mathematical vectors; Mathematics; Matrices; Memory; Models, Statistical; Nouns; Physical Sciences; Power laws; Reading comprehension; Semantics; Systems Analysis; Trajectories; Vector spaces; Vocabulary; Words
• ISSN: 0027-8424; 1091-6490
• DOI: 10.1073/pnas.0510673103

Thoughts and ideas are multidimensional and often concurrent, yet they can be expressed surprisingly well sequentially by the translation into language. This reduction of dimensions occurs naturally but requires memory and necessitates the existence of correlations, e.g., in written text. However, correlations in word appearance decay quickly, while previous observations of long-range correlations using random walk approaches yield little insight on memory or on semantic context. Instead, we study combinations of words that a reader is exposed to within a "window of attention," spanning about 100 words. We define a vector space of such word combinations by looking at words that co-occur within the window of attention, and analyze its structure. Singular value decomposition of the co-occurrence matrix identifies a basis whose vectors correspond to specific topics, or "concepts" that are relevant to the text. As the reader follows a text, the "vector of attention" traces out a trajectory of directions in this "concept space." We find that memory of the direction is retained over long times, forming power-law correlations. The appearance of power laws hints at the existence of an underlying hierarchical network. Indeed, imposing a hierarchy similar to that defined by volumes, chapters, paragraphs, etc. succeeds in creating correlations in a surrogate random text that are identical to those of the original text. We conclude that hierarchical structures in text serve to create long-range correlations, and use the reader's memory in reenacting some of the multidimensionality of the thoughts being expressed.