Imperfect CSI-Based Resource Management in Cognitive IoT Networks: A Deep Recurrent Reinforcement Learning Framework
Amandeep Kaur, Krishan Kumar, Arun Prakash, Rajeev Tripathi
Abstract
The proliferation of the Internet of Things (IoT) technology for wide range of wireless applications increase raw data, leads to spectrum scarcity, and also burdens available spectrum resources. Cognitive Radio (CR) is one of the key enablers to build future IoT systems, which has the ability to exploit dynamic networking characteristics to optimize network performance. One of the critical challenges is to obtain perfect Channel State Information (CSI) to efficiently allocate the transmission power and frequency resources among IoT nodes. In this paper, we focus on resource management optimization problem which is transformed into Markov Decision Process (MDP) and solved with proposed Deep Recurrent Reinforcement Learning (DRRL) based scheme with the consideration of energy-efficiency to deal with partial observability from limited information. Furthermore, to heighten the performance of proposed scheme, cooperative framework is introduced with Gated Recurrent Unit (GRU) layer to improve the resource allocation policy by accessing the historical information. Finally, extensive simulations corroborate the superiority of proposed scheme in terms of faster convergence and higher reward. The results also show that CSI imperfections is non-negligible, thus highlights the importance of robust and intelligent design that maintains users’ QoS.