Aggregation consistency and frequency of Chinese words and characters

• Published in: Journal of documentation 2006-09, Vol.62 (5), p.606-633
• Main Author: Arsenault, Clément
• Format: Article
• Language: English
• Subjects: Ambiguity; Bibliographic data bases; Bibliographic Databases; Bibliographic records; Bibliographic systems; Bibliometrics; Bibliometrics. Scientometrics. Evaluation; Character recognition equipment; Character user interfaces; China; Chinese; Chinese language materials; Chinese languages; Ethnic groups; Exact sciences and technology; Frequency distributions; Information and communication sciences; Information science. Documentation; Libraries; Library and information science. General aspects; Library catalogs; Personality; Phonology; Romanization; Sciences and techniques of general use; Standard Spoken Usage; Syllables; Terms; Word processing; Writing; Zipf's Law
• ISSN: 0022-0418; 1758-7379
• DOI: 10.1108/00220410610688750

Purpose - Aims to measure syllable aggregation consistency of Romanized Chinese data in the title fields of bibliographic records. Also aims to verify if the term frequency distributions satisfy conventional bibliometric laws.Design methodology approach - Uses Cooper's interindexer formula to evaluate aggregation consistency within and between two sets of Chinese bibliographic data. Compares the term frequency distributions of polysyllabic words and monosyllabic characters (for vernacular and Romanized data) with the Lotka and the generalised Zipf theoretical distributions. The fits are tested with the Kolmogorov-Smirnov test.Findings - Finds high internal aggregation consistency within each data set but some aggregation discrepancy between sets. Shows that word (polysyllabic) distributions satisfy Lotka's law but that character (monosyllabic) distributions do not abide by the law.Research limitations implications - The findings are limited to only two sets of bibliographic data (for aggregation consistency analysis) and to one set of data for the frequency distribution analysis. Only two bibliometric distributions are tested. Internal consistency within each database remains fairly high. Therefore the main argument against syllable aggregation does not appear to hold true. The analysis revealed that Chinese words and characters behave differently in terms of frequency distribution but that there is no noticeable difference between vernacular and Romanized data. The distribution of Romanized characters exhibits the worst case in terms of fit to either Lotka's or Zipf's laws, which indicates that Romanized data in aggregated form appear to be a preferable option.Originality value - Provides empirical data on consistency and distribution of Romanized Chinese titles in bibliographic records.