A text-based decision support system for financial sequence prediction

• Published in: Decision Support Systems 2011-12, Vol.52 (1), p.189-198
• Main Authors: Chan, Samuel W.K., Franklin, James
• Format: Article
• Language: English
• Subjects: Accuracy; Applied sciences; Artificial intelligence; Classifier based machine learning; Computer science; control theory; systems; Data mining; Data processing. List processing. Character string processing; Decision support systems; Electronics; Engines; Exact sciences and technology; Feature extraction; Inference; Information systems. Data bases; Intelligent systems; Knowledge management; Mathematical models; Memory organisation. Data processing; Operational research and scientific management; Operational research. Management science; Portfolio theory; Predictions; Software; Speech and sound recognition and synthesis. Linguistics; Studies; Texts; Textual analysis
• ISSN: 0167-9236; 1873-5797
• DOI: 10.1016/j.dss.2011.07.003

Although most quantitative financial data are analyzed using traditional statistical, artificial intelligence or data mining techniques, the abundance of online electronic financial news articles has opened up new possibilities for intelligent systems that can extract and organize relevant knowledge automatically in a usable format. Most information extraction systems require a hand-built dictionary of templates and thus need continual modification to accommodate new patterns that are observed in the text. In this research, we propose a novel text-based decision support system (DSS) that (i) extracts event sequences from shallow text patterns, and (ii) predicts the likelihood of the occurrence of events using a classifier-based inference engine. The prediction relies on two major, but complementary, feature sets: adjacent events and a set of information-theoretic functions. In contrast to other approaches, the proposed text-based DSS gives explanatory hypotheses about its predictions from a coalition of intimations learned from the inference engine, while preserving robustness and without indulging in formalism. We investigate more than 2000 financial reports with 28,000 sentences. Experiments show that the prediction accuracy of our model outperforms similar statistical models by 7% for the seen data while significantly improving the prediction accuracy for the unseen data. Further comparisons substantiate the experimental findings. ► To explain a classifier-based inference engine for financial sequence prediction. ► To demonstrate how to extract event sequences using shallow text parsing. ► To provide a head-to-head comparison and evaluation with the hidden Markov model.