CoLive: Edge-Assisted Clustered Learning Framework for Viewport Prediction in 360$^{\circ }$ Live Streaming
Mu Wang, Xingyan Chen, Xu Yang, P. Shuai, Yu Zhao, Mingwei Xu, Changqiao Xu
Abstract
The exceptionally high bandwidth requirement for delivering high-quality live 360 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> video poses a significant challenge to current network capacity. Mitigating such bandwidth starvation necessitates accurate field-of-view (FoV) prediction to focus limited resources on the viewer's area of interest. However, FoV prediction for live 360 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> streaming can be complex due to the time-sensitive nature of live content and the limited knowledge available for model training. Our paper introduces a novel framework, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CoLive</i> , for predicting the FoV in 360 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> live streaming. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CoLive</i> accelerates FoV prediction by offloading model training from viewers to the edge and migrating saliency feature detection to the server side. Observations on user clustering of viewing behaviors further motivate us to propose a novel dynamic clustered learning algorithm. The algorithm dynamically groups users according to their model update gradients and enables them to train a shared model that better suits their viewing preferences. We conduct extensive experiments on the public 360 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> video datasets and demonstrate that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CoLive</i> outperforms state-of-the-art solutions in terms of prediction performance and bandwidth savings.