Collaborative Caching Strategy for RL-Based Content Downloading Algorithm in Clustered Vehicular Networks
Xiaodan Bi, Lian Zhao
Abstract
With the explosive growth of content request services in the vehicle network, there is an urgent need to speed up the response process of content requests and reduce the backhaul burden on base stations (BSs). However, most traditional content caching strategies only consider the content popularity or cluster-based caching strategies individually, and the access paths are fixed. This article proposes a collaborative caching strategy for reinforcement learning (RL)-based content downloading. Specifically, the vehicles are first clustered by the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> -means algorithm, and the content transmission distance is reduced by caching the contents with high popularity in the cluster head (CH). Then, according to the historical content request information, the long short-term memory is used to predict the popularity of content. The contents with high popularity will be collaboratively cached in the BS and CHs. Finally, the content downloading problem can be described as a Markov decision process, using a deep RL algorithm, deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> network (DQN), to solve the target problem which is to minimize the weighted cost, including the downloading delay and failure cost. With the DQN algorithm, the CH can make the access decision for the content request. The proposed collaborative caching strategy for the RL-based content downloading algorithm can greatly reduce the response process and the burden at the BS. The simulation results show that the proposed RL-based method achieved outstanding performance to improve the access hit ratio and reduce the content downloading delay.