A Q-Learning-Based Hierarchical Routing Protocol With Unequal Clustering for Underwater Acoustic Sensor Networks
Yufan Yuan, Meiyan Liu, Xiaoxiao Zhuo, Yan Wei, Xingbin Tu, Fengzhong Qu
Abstract
Underwater acoustic sensor networks (UASNs) have emerged as a viable networking approach due to their numerous aquatic applications in recent years. As a vital component of UASNs, routing protocols are essential for ensuring reliable data transmissions and extending the longevity of UASNs. Recently, several clustering-based routing protocols have been proposed to reduce energy consumption and overcome the resource constraints of deployed sensor nodes. However, they rarely consider the hot-spots’ problem and the sink node isolation problem in the multihop underwater sensor networks. In this article, we propose a Q-learning-based hierarchical routing protocol with unequal clustering (QHUC) for determining an effective data forwarding path to extend the lifespan of UASNs. First, a hierarchical network structure is constructed for initialization. Then, a combination of unequal clustering and the Q-learning algorithm is applied to the hierarchical structure to disperse the remaining energy more evenly throughout the network. With the use of the Q-learning algorithm, a global optimal CH and next-hop can be determined better than a greedy one. In addition, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula> value that guarantees the optimal routing decisions can be computed without incurring any additional costs by combining the Q-learning algorithm with clustering. The simulation results show that the QHUC can achieve efficient routing and prolong the network lifetime significantly.