Unmanned-Surface-Vehicle-Aided Maritime Data Collection Using Deep Reinforcement Learning
Na Su, Jun-Bo Wang, Cheng Zeng, Hua Zhang, Min Lin, Geoffrey Ye Li
Abstract
Employing unmanned surface vehicles (USVs) as marine data collectors is promising for large-scale environment sensing in remote ocean monitoring network. In this article, we consider a USV-aided marine data collection network, where a USV collects data from multiple monitoring terminals while avoiding collisions with monitoring terminals and obstacles. Aiming at minimizing energy consumption and data loss, we formulate a trajectory optimization problem with practical constraints, including collision avoidance, steering angle, and velocity limitation. The problem is intractable due to the stochastic arrived data and the random emergence and movement of dynamic obstacles. To efficiently solve it, we transform it as a constrained Markov decision process (MDP) problem and address it using a target-oriented double deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula> -learning network (D2QN)-based collision avoidance and trajectory planning algorithm. In the proposed algorithm, the USV acts as an agent to explore and learn its trajectory planning policy by utilizing the causal knowledge. Numerical results demonstrate that the performance of the proposed algorithm is superior in terms of successful probability, energy consumption, and data loss.