Stock market one-day ahead movement prediction using disparate data sources

• Published in: Expert systems with applications 2017-08, Vol.79, p.153-163
• Main Authors: Weng, Bin, Ahmed, Mohamed A., Megahed, Fadel M.
• Format: Article
• Language: English
• Subjects: Artificial intelligence; Data sources; Decision making; Decision trees; Expert systems; Feature selection; Financial expert system; Google news; Inference; Knowledge; Knowledge bases (artificial intelligence); Machine learning; Markets; Neural networks; Performance enhancement; Predictions; R programming; Stock exchanges; Support vector machines; Time series; Traffic information; Wikipedia
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2017.02.041

•A financial expert system for predicting the daily stock movements.•“Knowledge base” captures both traditional and online data sources.•The “inference engine” uses three artificial intelligence techniques.•Prediction accuracy of 85% is higher than the reported results in the literature.•The system is hosted online and freely available for investors and researchers. There are several commercial financial expert systems that can be used for trading on the stock exchange. However, their predictions are somewhat limited since they primarily rely on time-series analysis of the market. With the rise of the Internet, new forms of collective intelligence (e.g. Google and Wikipedia) have emerged, representing a new generation of “crowd-sourced” knowledge bases. They collate information on publicly traded companies, while capturing web traffic statistics that reflect the public’s collective interest. Google and Wikipedia have become important “knowledge bases” for investors. In this research, we hypothesize that combining disparate online data sources with traditional time-series and technical indicators for a stock can provide a more effective and intelligent daily trading expert system. Three machine learning models, decision trees, neural networks and support vector machines, serve as the basis for our “inference engine”. To evaluate the performance of our expert system, we present a case study based on the AAPL (Apple NASDAQ) stock. Our expert system had an 85% accuracy in predicting the next-day AAPL stock movement, which outperforms the reported rates in the literature. Our results suggest that: (a) the knowledge base of financial expert systems can benefit from data captured from nontraditional “experts” like Google and Wikipedia; (b) diversifying the knowledge base by combining data from disparate sources can help improve the performance of financial expert systems; and (c) the use of simple machine learning models for inference and rule generation is appropriate with our rich knowledge database. Finally, an intelligent decision making tool is provided to assist investors in making trading decisions on any stock, commodity or index.