Financial portfolio optimization with online deep reinforcement learning and restricted stacked autoencoder—DeepBreath

• Published in: Expert systems with applications 2020-10, Vol.156, p.113456, Article 113456
• Main Authors: Soleymani, Farzan, Paquet, Eric
• Format: Article
• Language: English
• Subjects: Artificial neural networks; Blockchain; Cryptography; Deep learning; Deep reinforcement learning; Distance learning; Investment strategy; Machine learning; Online leaning; Optimization; Portfolio management; Restricted stacked autoencoder; Return on investment; Risk; Settlement risk; Test sets
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2020.113456

•Deep reinforcement learning framework called DeepBreath for portfolio management.•Extracting high-level features using restricted stacked autoencoder.•A convolutional neural network is employed to enforce the policy.•The reinforcement learning framework is trained both offline and online.•The settlement problem of stock market transactions is solved with a blockchain. The process of continuously reallocating funds into financial assets, aiming to increase the expected return of investment and minimizing the risk, is known as portfolio management. In this paper, a portfolio management framework is developed based on a deep reinforcement learning framework called DeepBreath. The DeepBreath methodology combines a restricted stacked autoencoder and a convolutional neural network (CNN) into an integrated framework. The restricted stacked autoencoder is employed in order to conduct dimensionality reduction and features selection, thus ensuring that only the most informative abstract features are retained. The CNN is used to learn and enforce the investment policy which consists of reallocating the various assets in order to increase the expected return on investment. The framework consists of both offline and online learning strategies: the former is required to train the CNN while the latter handles concept drifts i.e. a change in the data distribution resulting from unforeseen circumstances. These are based on passive concept drift detection and online stochastic batching. Settlement risk may occur as a result of a delay in between the acquisition of an asset and its payment failing to deliver the terms of a contract. In order to tackle this challenging issue, a blockchain is employed. Finally, the performance of the DeepBreath framework is tested with four test sets over three distinct investment periods. The results show that the return of investment achieved by our approach outperforms current expert investment strategies while minimizing the market risk.