Self-Attention Mechanism Enhanced User Interests Modeling for Personalized Recommendation Services in Cyber-Physical-Social Systems
Renjie Zhou, Hao Qian, Jilin Zhang, Chuxiang Liu, Jian Wan, Yongjian Ren, Naixue Xiong, Nailiang Zhao, Sanyuan Zhang
Abstract
Cyber-Physical-Social Systems (CPSS) provide great value to our lives, but they also cause data overload problems. Data-driven personalized recommendation service is one of the most efficient means to solve such problems, which is currently receiving wide attention from research and industrial communities. The most important task of personalized recommender systems is to predict the click-through rate of given items, which is especially true for personalized advertisement recommendation systems. Recently, a number of deep click-through models have been proposed, which obtain low-dimensional dense embedding vectors of features, and then concatenate together and input into multi-layer perceptron to learn the nonlinear relationship between the features. However, the existing models don't dig deep enough into the user preferences and habits in users’ behavior history. In this paper, we propose a new model: Self-attention based Deep Neural Network (DeepSA), which addresses this issue by constructing Ad-related graph and training graph embedding vectors to enhance the representation of the advertisement for capturing user interests, and learns the internal correlation between user behaviors via the self-attention mechanism, which better explores interests and preferences hidden in users’ historical behaviors. Experiments on two public datasets and an industrial dataset demonstrate the proposed method outperforms the state-of-the-art models.