Traffic Engineering in Partially Deployed Segment Routing Over IPv6 Network With Deep Reinforcement Learning
Ying Tian, Zhiliang Wang, Xia Yin, Xingang Shi, Yingya Guo, Haijun Geng, Jiahai Yang
Abstract
Segment Routing (SR) is a source routing paradigm which is widely used in Traffic Engineering (TE). By using SR, a node steers a packet through an ordered list of instructions called segments. By some extensions of interior gateway protocol, SR can be applied to IP/MPLS or IPv6 network without signal protocol. SR over IPv6 (SRv6) is attracting wide attention because of its interoperation ability with IPv6. However, upgrading the existing IPv6 network directly to a full SRv6 one can be difficult, because large-scale equipment replacement or software upgrade may cause economic and technical problems. TE in partially deployed SR network is becoming a hot research topic. In this paper, we propose the TE algorithm Weight Adjustment-SRTE (WA-SRTE) in partially deployed SRv6 network, in which SRv6 capable nodes are dispersedly deployed. Our objective is to minimize the network's maximum link utilization. WA-SRTE converts the TE problem into a Deep Reinforcement Learning problem and optimizes the OSPF weight, SRv6 node deployment and traffic paths simultaneously. Besides, traffic variation is also considered and we use a representative Traffic Matrix (TM) to epitomize the traffic characteristics over a period of time. Experiments demonstrate that with 20% to 40% of the SRv6 nodes deployed, we can achieve TE performance as good as in a full SR network for the experiment topologies. The results with WA remarkably outperform the results without it. Our algorithm also gets near-optimal results with changing traffic.