标题： 基于深度强化学习的无线传感器网络中无人机轨迹规划以最小化能量消耗 显示英文标题

标题： UAV Trajectory Planning in Wireless Sensor Networks for Energy Consumption Minimization by Deep Reinforcement Learning

摘要： 无人机（UAV）已成为大规模无线传感器网络（WSN）数据收集的有前景的候选解决方案。本文研究了一种基于无人机辅助的WSN，其中簇头（CHs）从其成员节点接收数据，然后无人机按照预定轨迹收集这些簇头的数据。我们的目标是最小化整个数据收集周期内UAV-WSN系统的总能耗。为此，我们将能耗最小化问题表述为一个受限的组合优化问题，通过联合选择簇内的CHs以及规划无人机访问选定CHs的顺序来实现。该能耗最小化问题被证明是NP难的，因此难以获得最优解。为了应对这一挑战，我们提出了一种新颖的深度强化学习（DRL）技术，指针网络-A*（Ptr-A*），它可以高效地从经验中学习无人机轨迹策略以最小化能耗。无人机的起点和一组预定义簇的WSN被输入到Ptr-A*中，Ptr-A*输出一组CHs及其访问顺序，即无人机的轨迹。通过使用无监督的方式利用演员-评论家算法，Ptr-A*的参数在小规模簇问题实例上进行训练，以加快训练速度。在推理阶段，还提出了三种搜索策略以提高解决方案的质量。仿真结果表明，基于20簇和40簇的训练模型具有良好的泛化能力，可以解决不同簇数量的WSN中的无人机轨迹规划问题，而无需重新训练模型。此外，结果还显示，我们提出的DRL算法优于两种基线技术。 显示英文摘要

摘要： Unmanned aerial vehicles (UAVs) have emerged as a promising candidate solution for data collection of large-scale wireless sensor networks (WSNs). In this paper, we investigate a UAV-aided WSN, where cluster heads (CHs) receive data from their member nodes, and a UAV is dispatched to collect data from CHs along the planned trajectory. We aim to minimize the total energy consumption of the UAV-WSN system in a complete round of data collection. Toward this end, we formulate the energy consumption minimization problem as a constrained combinatorial optimization problem by jointly selecting CHs from nodes within clusters and planning the UAV's visiting order to the selected CHs. The formulated energy consumption minimization problem is NP-hard, and hence, hard to solve optimally. In order to tackle this challenge, we propose a novel deep reinforcement learning (DRL) technique, pointer network-A* (Ptr-A*), which can efficiently learn from experiences the UAV trajectory policy for minimizing the energy consumption. The UAV's start point and the WSN with a set of pre-determined clusters are fed into the Ptr-A*, and the Ptr-A* outputs a group of CHs and the visiting order to these CHs, i.e., the UAV's trajectory. The parameters of the Ptr-A* are trained on small-scale clusters problem instances for faster training by using the actor-critic algorithm in an unsupervised manner. At inference, three search strategies are also proposed to improve the quality of solutions. Simulation results show that the trained models based on 20-clusters and 40-clusters have a good generalization ability to solve the UAV's trajectory planning problem in WSNs with different numbers of clusters, without the need to retrain the models. Furthermore, the results show that our proposed DRL algorithm outperforms two baseline techniques.