Automated Phrase Mining from Massive Text Corpora

As one of the fundamental tasks in text analysis, phrase mining aims at extracting quality phrases from a text corpus and has various downstream applications including information extraction/retrieval, taxonomy construction, and topic modeling. Most existing methods rely on complex, trained linguist...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on knowledge and data engineering 2018-10, Vol.30 (10), p.1825-1837
Main Authors: Shang, Jingbo, Liu, Jialu, Jiang, Meng, Ren, Xiang, Voss, Clare R., Han, Jiawei
Format: Article
Language:English
Subjects:
Analyzers
Automatic phrase mining
Automation
distant training
Domains
Electronic publishing
Encyclopedias
Information retrieval
Internet
Knowledge base
Knowledge based systems
Labeling
Mining
multiple languages
part-of-speech tag
phrase mining
Pragmatics
State of the art
Taxonomy
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:As one of the fundamental tasks in text analysis, phrase mining aims at extracting quality phrases from a text corpus and has various downstream applications including information extraction/retrieval, taxonomy construction, and topic modeling. Most existing methods rely on complex, trained linguistic analyzers, and thus likely have unsatisfactory performance on text corpora of new domains and genres without extra but expensive adaption. None of the state-of-the-art models, even data-driven models, is fully automated because they require human experts for designing rules or labeling phrases. In this paper, we propose a novel framework for automated phrase mining, AutoPhrase, which supports any language as long as a general knowledge base (e.g., Wikipedia) in that language is available, while benefiting from, but not requiring, a POS tagger. Compared to the state-of-the-art methods, AutoPhrase has shown significant improvements in both effectiveness and efficiency on five real-world datasets across different domains and languages. Besides, AutoPhrase can be extended to model single-word quality phrases.
ISSN:1041-4347
1558-2191
DOI:10.1109/TKDE.2018.2812203