Modeling time-dependent and -independent indicators to facilitate identification of breakthrough research papers

• Published in: Scientometrics 2016-05, Vol.107 (2), p.807-817
• Main Authors: Wolcott, Holly N., Fouch, Matthew J., Hsu, Elizabeth R., DiJoseph, Leo G., Bernaciak, Catherine A., Corrigan, James G., Williams, Duane E.
• Format: Article
• Language: English
• Subjects: Computer Science; Information Storage and Retrieval; Library Science
• ISSN: 0138-9130; 1588-2861
• DOI: 10.1007/s11192-016-1861-1

Research funding organizations invest substantial resources to monitor mission-relevant research findings to identify and support promising new lines of inquiry. To that end, we have been pursuing the development of tools to identify research publications that have a strong likelihood of driving new avenues of research. This paper describes our work towards incorporating multiple time-dependent and -independent features of publications into a model to identify candidate breakthrough papers as early as possible following publication. We used multiple random forest models to assess the ability of indicators to reliably distinguish a gold standard set of breakthrough publications as identified by subject matter experts from among a comparison group of similar Thomson Reuters Web of Science™ publications. These indicators were then tested for their predictive value in random forest models. Model parameter optimization and variable selection were used to construct a final model based on indicators that can be measured within 6 months post-publication; the final model had an estimated true positive rate of 0.77 and false positive rate of 0.01.