An intelligent financial portfolio trading strategy using deep Q-learning

• Published in: Expert systems with applications 2020-11, Vol.158, p.113573, Article 113573
• Main Authors: Park, Hyungjun, Sim, Min Kyu, Choi, Dong Gu
• Format: Article
• Language: English
• Subjects: Combinatorial analysis; Deep neural network; Deep Q-learning; Intelligent agents; Machine learning; Markov decision process; Markov processes; Portfolio trading; Reinforcement learning; Strategy
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2020.113573

•An approach for financial portfolio trading using deep Q-learning is proposed.•The approach can derive a multi-asset portfolio trading strategy.•The approach adopts a discrete combinatorial action space.•To overcome the technical challenges, the approach has three novel features.•Numerical tests show the superiority of our approach. Portfolio traders strive to identify dynamic portfolio allocation schemes that can allocate their total budgets efficiently through the investment horizon. This study proposes a novel portfolio trading strategy in which an intelligent agent is trained to identify an optimal trading action using deep Q-learning. We formulate a Markov decision process model for the portfolio trading process that adopts a discrete combinatorial action space and determines the trading direction at a prespecified trading size for each asset, thus ensuring practical applicability. Our novel portfolio trading strategy takes advantage of three features to outperform other strategies in real-world trading. First, a mapping function is devised to handle and transform any action that is initially proposed but found to be infeasible into a similar and valuable feasible action. Second, by overcoming the dimensionality problem, this study establishes agent and Q-network models to derive a multi-asset trading strategy in the predefined action space. Last, this study introduces a technique that can derive a well-fitted multi-asset trading strategy by designing an agent to simulate all feasible actions in each state. To validate our approach, we conduct backtesting for two representative portfolios and demonstrate superior results over the benchmark strategies.