Joint Quantum Reinforcement Learning and Stabilized Control for Spatio-Temporal Coordination in Metaverse
Soohyun Park, Jaehyun Chung, Chanyoung Park, Soyi Jung, Minseok Choi, Sungrae Cho, Joongheon Kim
Abstract
In order to build realistic metaverse systems, enabling high synchronization between physical-space and virtual meta-space is essentially required. For this purpose, this paper proposes a novel system-wide coordination algorithm for high synchronization under characteristics ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , highly realistic meta-space construction under the constraints of physical-space). The proposed algorithm consists of the following three stages. The first stage is quantum multi-agent reinforcement learning (QMARL)-based scheduling for low-delay temporal-synchronization using differentiated age-of-information (AoI) during data gathering in physical-space by observers for meta-space construction. This is beneficial for scalability according to action dimension reduction in reinforcement learning computation. The second stage is for creating virtual contents under delay constraints in meta-space based on the gathered data. When rendering regions that have received more user attention, avatar-popularity is considered for spatio-synchronization. Thus, a stabilized control mechanism is designed for time-average reality quality maximization for each region. The last stage is for caching based on avatar-popularity and AoI which can be helpful in constructing low-delay realistic meta-space. Furthermore, the concept of AoI is divided into two separate sub-concepts of physical AoI and virtual AoI such that the AoI in virtual meta-space can be thoroughly implemented.