Predicting the impact of scientific concepts using full-text features

• Published in: Journal of the Association for Information Science and Technology 2016-11, Vol.67 (11), p.2684-2696
• Main Authors: McKeown, Kathy, Daume III, Hal, Chaturvedi, Snigdha, Paparrizos, John, Thadani, Kapil, Barrio, Pablo, Biran, Or, Bothe, Suvarna, Collins, Michael, Fleischmann, Kenneth R., Gravano, Luis, Jha, Rahul, King, Ben, McInerney, Kevin, Moon, Taesun, Neelakantan, Arvind, O'Seaghdha, Diarmuid, Radev, Dragomir, Templeton, Clay, Teufel, Simone
• Format: Article
• Language: English
• Subjects: Information retrieval; machine learning; Metadata; natural language processing; scientometrics; Sentences; Tasks; Texts
• ISSN: 2330-1635; 2330-1643
• DOI: 10.1002/asi.23612

New scientific concepts, interpreted broadly, are continuously introduced in the literature, but relatively few concepts have a long‐term impact on society. The identification of such concepts is a challenging prediction task that would help multiple parties—including researchers and the general public—focus their attention within the vast scientific literature. In this paper we present a system that predicts the future impact of a scientific concept, represented as a technical term, based on the information available from recently published research articles. We analyze the usefulness of rich features derived from the full text of the articles through a variety of approaches, including rhetorical sentence analysis, information extraction, and time‐series analysis. The results from two large‐scale experiments with 3.8 million full‐text articles and 48 million metadata records support the conclusion that full‐text features are significantly more useful for prediction than metadata‐only features and that the most accurate predictions result from combining the metadata and full‐text features. Surprisingly, these results hold even when the metadata features are available for a much larger number of documents than are available for the full‐text features.