Reinforcement Learning-Based MAC Protocol for Underwater Multimedia Sensor Networks
Firoj Gazi, Nurzaman Ahmed, Sudip Misra, Wei Wei
Abstract
High propagation delay, high error probability, floating node mobility, and low data rates are the key challenges for Underwater Wireless Multimedia Sensor Networks (UMWSNs). In this article, we propose RL-MAC, a Reinforcement Learning (RL)–based Medium Access Control (MAC) protocol for multimedia sensing in an Underwater Acoustic Network (UAN) environment. The proposed scheme uses Transmission Opportunity (TXOP) for relay nodes in a multi-hop network for improved efficiency concerning the mobility of the relays and sensor nodes. The access point (AP) and relay nodes calculate traffic demands from the initial contention of the sensor nodes. Our solution uses Q-learning to enhance the contention mechanism at the initial phase of multimedia transmission. Based on the traffic demands, RL-MAC allocates TXOP duration for the uplink multimedia reception. Further, the Structural Similarity Index Measure (SSIM) and compression techniques are used for calculating the image quality at the receiver end and reducing the image at the destination, respectively. We implement a prototype of the proposed scheme over an off-the-shelf, low-cost hardware setup. Moreover, extensive simulation over NS-3 shows a significant packet delivery ratio and throughput compared with the existing state-of-the-art.