Business environmental analysis for textual data using data mining and sentence-level classification

• Published in: Industrial management + data systems 2019-02, Vol.119 (1), p.69-88
• Main Authors: Kim, Yoon-Sung, Rim, Hae-Chang, Lee, Do-Gil
• Format: Article
• Language: English
• Subjects: Accuracy; Algorithms; Artificial intelligence; Automation; Big Data; Business; Classification; Clustering; Competition; Data mining; Decision making; Decision trees; Economic analysis; Keywords; Machine learning; Methodology; Methods; Performance evaluation; Semantics; Sentences; SWOT analysis; Training; Unstructured data
• ISSN: 0263-5577; 1758-5783
• DOI: 10.1108/IMDS-07-2017-0317

Purpose The purpose of this paper is to propose a methodology to analyze a large amount of unstructured textual data into categories of business environmental analysis frameworks. Design/methodology/approach This paper uses machine learning to classify a vast amount of unstructured textual data by category of business environmental analysis framework. Generally, it is difficult to produce high quality and massive training data for machine-learning-based system in terms of cost. Semi-supervised learning techniques are used to improve the classification performance. Additionally, the lack of feature problem that traditional classification systems have suffered is resolved by applying semantic features by utilizing word embedding, a new technique in text mining. Findings The proposed methodology can be used for various business environmental analyses and the system is fully automated in both the training and classifying phases. Semi-supervised learning can solve the problems with insufficient training data. The proposed semantic features can be helpful for improving traditional classification systems. Research limitations/implications This paper focuses on classifying sentences that contain the information of business environmental analysis in large amount of documents. However, the proposed methodology has a limitation on the advanced analyses which can directly help managers establish strategies, since it does not summarize the environmental variables that are implied in the classified sentences. Using the advanced summarization and recommendation techniques could extract the environmental variables among the sentences, and they can assist managers to establish effective strategies. Originality/value The feature selection technique developed in this paper has not been used in traditional systems for business and industry, so that the whole process can be fully automated. It also demonstrates practicality so that it can be applied to various business environmental analysis frameworks. In addition, the system is more economical than traditional systems because of semi-supervised learning, and can resolve the lack of feature problem that traditional systems suffer. This work is valuable for analyzing environmental factors and establishing strategies for companies.