Title: Cooperative Guidance Strategy for Active Defense Spacecraft with Imperfect Information via Deep Reinforcement Learning Show Chinese title

Title: 基于深度强化学习的非完美信息下主动防御航天器协同引导策略

Abstract: In this paper, an adaptive cooperative guidance strategy for the active protection of a target spacecraft trying to evade an interceptor was developed. The target spacecraft performs evasive maneuvers, launching an active defense vehicle to divert the interceptor. Instead of classical strategies, which are based on optimal control or differential game theory, the problem was solved by using the deep reinforcement learning method, and imperfect information was assumed for the interceptor maneuverability. To address the sparse reward problem, a universal reward design method and an increasingly difficult training approach were presented utilizing the shaping technique. Guidance law, reward function, and training approach were demonstrated through the learning process and Monte Carlo simulations. The application of the non-sparse reward function and increasingly difficult training approach accelerated the model convergence, alleviating the overfitting problem. Considering a standard optimal guidance law as a benchmark, the effectiveness, and the advantages, that guarantee the target spacecraft's escape and win rates in a multi-agent game, of the proposed guidance strategy were validated by the simulation results. The trained agent's adaptiveness to the interceptor maneuverability was superior to the optimal guidance law. Moreover, compared to the standard optimal guidance law, the proposed guidance strategy performed better with less prior knowledge. Show Chinese abstract

Abstract: 本文提出了一种针对目标航天器主动防护的自适应协同制导策略，该航天器试图规避拦截器。目标航天器执行规避机动，并发射主动防御飞行器以偏转拦截器。与基于最优控制或微分博弈理论的经典策略不同，该问题通过深度强化学习方法解决，并假设了拦截器机动性中的不完美信息。为了解决稀疏奖励问题，提出了利用塑造技术的通用奖励设计方法和逐步困难的训练方法。通过学习过程和蒙特卡罗仿真验证了制导律、奖励函数以及训练方法。非稀疏奖励函数的应用和逐步困难的训练方法加速了模型收敛，缓解了过拟合问题。以标准最优制导律作为基准，仿真结果验证了所提出的制导策略的有效性和优势，保证了目标航天器在多智能体博弈中的逃脱和胜率。此外，与标准最优制导律相比，所提出的制导策略具有更少的先验知识但仍表现得更好。