Enabling High-Throughput Routing for LEO Satellite Broadband Networks: A Flow-Centric Deep Reinforcement Learning Approach
Huashuo Liu, Junyu Lai, Junhong Zhu, Lianqiang Gan, Zheng Chang
Abstract
Routing optimization within a low Earth orbit (LEO) satellite broadband network (LSBN) has seen advancements through deep reinforcement learning (DRL) approaches in academia. Nonetheless, a crucial aspect often overlooked in these approaches pertains to the inference time of deep neural network (DNN) models during the routing of packets. Our investigation reveals that this oversight can significantly impair routing throughput in LSBN. In response, this paper innovatively proposes a decentralized flow-centric DRL approach, shifting the focus from routing individual packets to entire traffic flows. To align with the large-scale feature of LSBN, we embrace a fully-distributed architecture for flow-centric routing, which is modeled as a partially observable Markov decision process. In this construct, each satellite operates as an independent agent, locally classifying flows following a tailor-designed definition, and is responsible for forwarding a flow to an adjacent satellite based on its internal policy. Notably, the DNN inference is conducted only once on each agent to determine the route for the initial packet of a specific flow; subsequent packets are directed along the same route. Recognizing the potential impact of dynamic LSBN topologies on routing performance, we also introduce an adaptive flow routing update scheme. This scheme is completely free from LSBN environment modelling and aims to bolster the efficacy of the flow-centric approach. Comparative experiments showcase the superiority of the proposed approach over baseline algorithms across various metrics. Consequently, the flow-centric DRL approach can enable high-throughput traffic transmission for LSBN.