Scheduling single-satellite observation and transmission tasks by using hybrid Actor-Critic reinforcement learning

• Published in: Advances in space research 2023-05, Vol.71 (9), p.3883-3896
• Main Authors: Wen, Zhijiang, Li, Lu, Song, Jiakai, Zhang, Shengyu, Hu, Haiying
• Format: Article
• Language: English
• Subjects: Deep-reinforcement learning; Earth observation satellite; Integrated scheduling problem
• ISSN: 0273-1177; 1879-1948
• DOI: 10.1016/j.asr.2022.10.024

•Integrated scheduling data transmission and observation tasks to achieve better schedule results.•In order to make more flexible scheduling, a time-continuous model for the data transmission process is established to achieve accurate time decisions.•A hybrid Actor-Critic reinforcement learning method is designed for solving single-satellite observation and transmission task scheduling problems, which performs well under intensive observation scenarios.•Different training methods are applied to meet various scheduling requirements and random scenarios. Earth observation satellites(EOS) generate a large amount of observation data in intensive observation scenarios, while the ability of data storage on the EOS is limited. It makes integrating satellite observation and data transmission tasks for EOS imperative. This paper establishes a time-continuous model for the single EOS integrated scheduling problem, which considers data transmission and observation simultaneously. A hybrid Actor-Critic reinforcement learning method is adopted to solve the EOS integrated scheduling problem for a more efficient solution in intensive observation scenarios. Furthermore, the algorithm can flexibly determine the start and end time of the data transmission task. Experimental results show that the hybrid Actor-Critic reinforcement learning method deals with a large scale of problems with high efficiency and good results.