The emerging threat of artificial intelligence on competition in liberalized electricity markets: A deep Q-network approach

• Published in: Applied energy 2022-11, Vol.325, p.119813, Article 119813
• Main Authors: Esmaeili Aliabadi, Danial, Chan, Katrina
• Format: Article
• Language: English
• Subjects: Collusion; Day-ahead electricity market; Deep Q-network; Nash equilibrium
• ISSN: 0306-2619; 1872-9118
• DOI: 10.1016/j.apenergy.2022.119813

According to Sustainable Development Goals (SDGs), societies should have access to affordable, reliable, and sustainable energy. Liberalized electricity markets have been established to provide affordable electricity for end-users through advertising competition. Although these new markets are designed to serve competition, there are recorded incidents where participants abused their market power and disrupted the competition through collusion. Unfortunately, modern autonomous pricing algorithms may further assist myopic players to discover collusive strategies with a minimum amount of sensitive information. Therefore, in this study, we investigate the impact of emerging learning algorithms on the bidding strategies of Power Generating Companies (GenCos) and compare their performance against game-theoretic expectations. A novel deep Q-network (DQN) model is developed, by which GenCos determine the bidding strategies to maximize average long-term payoffs in a day-ahead market. The presented DQN model assumes that GenCos have no information regarding the rivals’ true generation costs and profits. To the best of the authors’ knowledge, this is the first study that thoroughly investigates players’ behavior utilizing a modern DQN model and compares its results with equilibria of the non-cooperative single-stage and infinitely-repeated games in the context of electricity markets. The outcomes articulate that GenCos equipped with advanced learning models may be able to collude unintentionally while trying to ameliorate long-term profits. Moreover, GenCos that employ the presented DQN model could discover and sustain more profitable (e.g., collusive) strategies vis-à-vis a conventional Q-learning method. Collusive strategies can lead to exorbitant electric bills for end-users, which is one of the influential factors in energy poverty. Thus, policymakers and market designers should be vigilant regarding the combined effect of information disclosure and autonomous pricing, as new models exploit information more effectively. [Display omitted] •Thoroughly investigating the effect of GenCos’ learning on tacit collusion.•Exploring the effect of algorithmic pricing in the day-ahead electricity market.•Comparing simulation results with game-theoretic expectations in a collusive setting.•Designing a novel deep Q-network algorithm with the LIFO scheme for dynamic markets.•Discovering the critical information set to find and sustain collusion.