Multi-Objective Reinforcement Learning Based Healthcare Expansion Planning Considering Pandemic Events

• Published in: IEEE journal of biomedical and health informatics 2023-06, Vol.27 (6), p.2760-2770
• Main Authors: Shuvo, Salman Sadiq, Symum, Hasan, Ahmed, Md Rubel, Yilmaz, Yasin, Zayas-Castro, JosA L.
• Format: Article
• Language: English
• Subjects: Algorithms; Costs; Deep RL; denial of service; Denial of service attacks; Economic impact; Forecasting; Health care; healthcare capacity expansion; Hospitals; Markov decision process; multi-objective optimization; Pandemics; Planning; Reinforcement; Sociology; Statistics
• ISSN: 2168-2194; 2168-2208
• DOI: 10.1109/JBHI.2022.3187950

Hospital capacity expansion planning is critical for a healthcare authority, especially in regions with a growing diverse population. Policymaking to this end often requires satisfying two conflicting objectives, minimizing capacity expansion cost and minimizing the number of denial of service (DoS) for patients seeking hospital admission. The uncertainty in hospital demand, especially considering a pandemic event, makes expansion planning even more challenging. This work presents a multi-objective reinforcement learning (MORL) based solution for healthcare expansion planning to optimize expansion cost and DoS simultaneously for pandemic and non-pandemic scenarios. Importantly, our model provides a simple and intuitive way to set the balance between these two objectives by only determining their priority percentages, making it suitable across policymakers with different capabilities, preferences, and needs. Specifically, we propose a multi-objective adaptation of the popular Advantage Actor-Critic (A2C) algorithm to avoid forced conversion of DoS discomfort cost to a monetary cost. Our case study for the state of Florida illustrates the success of our MORL based approach compared to the existing benchmark policies, including a state-of-the-art deep RL policy that converts DoS to economic cost to optimize a single objective.