UAV Assisted Cooperative Caching on Network Edge Using Multi-Agent Actor-Critic Reinforcement Learning
Sadman Araf, Adittya Soukarjya Saha, Sadia Hamid Kazi, Nguyen H. Tran, Md. Golam Rabiul Alam
Abstract
In recent times, caching at edge nodes is a well-known technique to overcome the limitation of strict latency, which simultaneously improves users' Quality of Experience (QoE). However, choosing an appropriate caching policy and content placement poses another significant issue that has been acknowledged in this research. Conventional caching policies that are experimented with at the edge do not consider the dynamic and stochastic characteristics of edge caching. As a result, we have proposed a cooperative deep reinforcement learning algorithm that deals with the dynamic nature of content demand. It also ensures efficient use of storage through the cooperation between nodes. In addition, previous works on cooperative caching have assumed the users to be static and didn't consider the mobile nature of users. Therefore, we have proposed UAVs as aerial Base Stations (UAV-BS) to assist in peak hours where a ground base station is insufficient to support the surge in user requests. In this novel research, we have demonstrated the cooperation between aerial and Ground Base Stations (GBS) and aimed at maximizing the global cache hit ratio. Simulations have shown that our proposed Cooperative Multi-Agent Actor-Critic algorithm outperforms conventional and reinforcement learning based caching methods and achieves a state-of-the-art global cache hit ratio when there is a surge in user requests. Thus, it opens the door for further research on cooperative caching in joint air and ground architecture.